arachnid 0.4.0

Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

0.4.1 - January 17, 2014 (5 KB)
0.4.0 - January 16, 2014 (4.5 KB)
0.3.2 - January 16, 2014 (5 KB)
0.3.1 - January 16, 2014 (5 KB)
0.3.0 - January 16, 2014 (4.5 KB)

Show all versions (12 total)

Runtime Dependencies (4):

bloomfilter-rb = 2.1.1

domainatrix >= 0

nokogiri = 1.5.0

typhoeus = 0.3.2

Show all transitive dependencies

Owners:

Authors:

dchuk

SHA 256 checksum:

ebf9412a4fd4d2f3238c888bfb990d162d6682c7aa596aac0047a597678b3259

arachnid 0.4.0

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

Runtime Dependencies (4):

Owners:

Authors:

SHA 256 checksum:

Total downloads 48,650

For this version 3,322

Licenses:
N/A

Required Ruby Version: None

Links:

arachnid 0.4.0

Gemfile: = Copy to clipboard Copied!

install: =

Versions:

Runtime Dependencies (4):

Owners:

Authors:

SHA 256 checksum:

Total downloads 48,650

For this version 3,322

Licenses: N/A

Required Ruby Version: None

Links:

Gemfile:
= Copy to clipboard Copied!

install:
=

Licenses:
N/A