robot_rules 0.9.3

A tool to determine if the robots.txt would prevent a given user agent from making a request to a given URI.

Gemfile:
= Copier Copié!

installation:
=

Versions:

0.9.3 - July 21, 2010 (7 ko)
0.9.2 - July 21, 2010 (7 ko)
0.9.1 - July 09, 2010 (7 ko)

Dépendances de Runtime (1):

addressable >= 2.1.2

Show all transitive dependencies

Propriétaires:

Auteurs:

James Edward Gray II, Jeremy Friesen, Edgar Gonzalez

Total de contrôle SHA 256:

e9ef85a1ba4939df3bc5501bb62370b89771cb01f40031f96270095900a5abe2