robot_rules 0.9.1

A tool to determine if the robots.txt would prevent a given user agent from making a request to a given URI.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

0.9.3 - July 21, 2010 (7 KB)
0.9.2 - July 21, 2010 (7 KB)
0.9.1 - July 09, 2010 (7 KB)

Owners:

Authors:

James Edward Gray II, Jeremy Friesen, Edgar Gonzalez

SHA 256 checksum:

71002de75ccf85885d6f5c0560b5fb07bf7acfa772528709b0fe9b56baffced1