jeremyf-robot_rules 0.9.1
A tool to determine if the robots.txt would prevent a given user agent from making a request to a given URI.
Gemfile:
=
instalar:
=
Versiones:
- 0.9.1 August 11, 2014 (7 KB)
Autores:
-
James Edward Gray II, Jeremy Friesen
SHA 256 checksum:
=