Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Required Ruby Version

None

Authors

dchuk

Versions

  1. 0.4.1

    1. source

      1. 0.4.1 source January 17, 2014 (5 KB)
  2. 0.4.0

    1. source

      1. 0.4.0 source January 16, 2014 (4.5 KB)
  3. 0.3.2

    1. source

      1. 0.3.2 source January 16, 2014 (5 KB)
  4. 0.3.1

    1. source

      1. 0.3.1 source January 16, 2014 (5 KB)
  5. 0.3.0

    1. source

      1. 0.3.0 source January 16, 2014 (4.5 KB)
Show all versions (12 total)

SHA 256 checksum

← Previous version