Web Crawler that uses resque background job engine to allow you to cluster your crawl.
Stewart McKee
N/A