RubyGems.org

spider

0.4.4

A Web spidering library: handles robots.txt, scraping, finding more links, and doing it all over again.

installgem install spider
Authors

John Nagro

266,476 total downloads 210,929 for this version
Owners

905e6bb5d089e0aa77448218f037fcd301b9172e9f53e0955a1af7458752aa9b

Gemfile
gem 'spider', '~> 0.4.4'
Versions
  1. 0.4.4 May 21, 2009 (30.5 KB)
  2. 0.4.3 October 9, 2008 (20.5 KB)
  3. 0.4.2 July 6, 2008 (20.5 KB)
  4. 0.4.1 November 10, 2007 (19.5 KB)
  5. 0.4.0 November 2, 2007 (19 KB)
Show all versions (9 total)