Scraping-Engine design

Each scrape in Scraping-Engine follows the same set of steps.

  1. Fetch the configured resource into a Page using a Fetcher
  2. Parse the Page using a Parser and return a Result
  3. Store the Result into a configured location
  4. Notify on success
  • If an error occurs, Notify this instead