Projects/Lanvalia

Lanvalia is a small Web Crawler for research purposes (data mining). It is programmed in Ruby using the excellent sqlite and Hpricot libraries.

Blocking

Lanvalia obeys the robots.txt exlusion standard.

Example robots.txt entry

User-Agent: Lanvalia~<br /~> Disallow: /

Problems

If the bot is misbehaving, please contact me (don't forget to include access log or at least time, ip and url).