RubyGems.org

spider

0.2.0

A Web spidering library: handles robots.txt, scraping, finding more links, and doing it all over again.

installgem install spider -v 0.2.0
Authors

Mike Burns

267,742 total downloads 1,215 for this version
Owners

905e6bb5d089e0aa77448218f037fcd301b9172e9f53e0955a1af7458752aa9b

Gemfile
gem 'spider', '~> 0.2.0'
Versions
  1. 0.4.4 May 21, 2009 (30.5 KB)
  2. 0.4.3 October 9, 2008 (20.5 KB)
  3. 0.4.2 July 6, 2008 (20.5 KB)
  4. 0.4.1 November 10, 2007 (19.5 KB)
  5. 0.4.0 November 2, 2007 (19 KB)
Show all versions (9 total)