previous - contents - next

Configuring custom plugins

You can plug your own code into many parts of Scraping-Engine/OSCube; and the openness of simple-jndi allows you to easily get your configuration there too. Given the following configuration: 

org.osjava.scrapers=SlashRss
SlashRss.uri=http://slashdot.org/index.rss
SlashRss.parser=com.example.ExampleParser
the ExampleParser can easily access the configuration while in its parse(String uri, Config config, Session session) method by invoking config.getString("uri"). If ExampleParser wishes to have its own configuration, above and beyond Scraping-Engine's default concepts; it can do so with remarkable ease, simply by adding that config to the configuration file, under the same prefix. 
org.osjava.scrapers=SlashRss
SlashRss.uri=http://slashdot.org/index.rss
SlashRss.parser=com.example.ExampleParser
SlashRss.answer=42
and then accessing it during its code via config.getString("answer"). As the Javadoc for gj-config shows, it would probably be better to use config.getInt("answer") to get 42 as an int instead of a String.

Sessions

In case you are wondering, the Session is nothing more than a sub-class of Config, allowing scheduled scrapings to store Object values between scrapes.