RubyGems.org

cobweb

0.0.27

Web Crawler that uses resque background job engine to allow you to cluster your crawl.

installgem install cobweb -v 0.0.27
Authors

Stewart McKee

70,169 total downloads 1,012 for this version
Owners

8cdb978dffae6ab207dbdeb60d01af2f

Gemfile
gem 'cobweb', '~> 0.0.27'
Versions
  1. 1.0.19 November 26, 2013 (9.28 MB)
  2. 1.0.18 October 17, 2013 (9.28 MB)
  3. 1.0.17 August 23, 2013 (9.28 MB)
  4. 1.0.16 August 22, 2013 (9.28 MB)
  5. 1.0.15 August 19, 2013 (9.28 MB)
Show all versions (81 total)