Reverse dependencies for spidr 0.6.0
validate-website is a web crawler for checking the markup validity with XML Schema / DT...
A library to search for emails in search engines
Ronin Web is a Ruby library for Ronin that provides support for web scraping and spider...
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a ...
Ronin Scanners is a Ruby library for Ronin that provides Ruby interfaces to various thi...
Monet is a web UI change comparer.
A Ruby library for generating and working with word-lists. Wordlist allows one to effic...
A Gem to produce competitive intelligence data
Hacker-gems installs most needed gems for hackers.
Ping all the links in your app
Crawler that auto generates SitePrism Page Objects
Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files on...
Command Line Interface (CLI) for the spidr gem.
Crawl a site and check various health indicators, such as: HTTP 4XX, 5XX status, valid ...