robotstxt 0.5.3

Robotstxt Parser allows you to the check the accessibility of URLs and get other data. Full support for the robots.txt RFC, wildcards and Sitemap: rules.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

0.5.4 - February 13, 2010 (7.5 KB)
0.5.3 - February 13, 2010 (7.5 KB)
0.5.2 - December 19, 2009 (7.5 KB)
0.5.1 - December 06, 2009 (5 KB)
0.5.0 - December 06, 2009 (5 KB)

Development Dependencies (2):

echoe ~> 3.1

rake ~> 0.8

Show all transitive dependencies

Owners:

Authors:

Simone Rinzivillo

SHA 256 checksum:

35cceda41379cc4adb0e665d429e471b2fed2ecaab6d8cb0356ed601c896b759

Total downloads 30,053

For this version 3,347

Licenses:
N/A

Required Ruby Version: None

Required Rubygems Version: >= 1.2

Links:

Homepage Source Code Documentation Download Review changes Badge RSS Report abuse Reverse dependencies