This is an exported version of the JIRA issue tracker. Please use the Google Code site to open new tickets or report updates to these existing tickets. Feel free to contact the mailing list with any questions.

Scraping

• SCB-26 - Upgrade to HttpClient 3.0.1 from HttpClient 2.0.2 (Open)
• SCB-25 - support for fetching non - HTML/Text files content-type: application/vnd.ms-word (Open)
• SCB-24 - Ability to specify the Fetcher to use (Closed)
• SCB-23 - Create a tutorial entry on how to scrape from RSS (Open)
• SCB-22 - Ability to scrape pop:// urls (Open)
• SCB-21 - Ability to scrape ftp:// urls (Closed)
• SCB-20 - Sub-fetching should pass referrer of last page (Open)
• SCB-19 - Set a referer for initial scrape (Open)
• SCB-18 - Add userAgent configuration (Closed)
• SCB-17 - Support Cookies (Closed)
• SCB-16 - Add support for POST requests (Closed)
• SCB-15 - Create a default csv-parser (Open)
• SCB-14 - Create .bat files for the example (Closed)
• SCB-12 - Move the example UrlScraper into the main distro so everyone may use it (Closed)
• SCB-9 - When running the scraping engine, allow the scraper to be chosen (Open)
• SCB-8 - Examples zip download (Closed)
• SCB-7 - Add a FileStore concept so that scraped data may be saved (Closed)
• SCB-6 - Have linkhunter know about mime types (Open)
• SCB-5 - Add the PrinterStore, a store that prints out (Closed)
• SCB-4 - Field validation (Open)
• SCB-3 - Needs to respect robots.txt before being unleashed fully (Closed)
• SCB-2 - Infinite loops on africainsight.org (Closed)