Crawlr is a modern Ruby web scraping framework built for respectful and efficient data extraction. Features async HTTP requests, robots.txt compliance, cookie management, proxy rotation, flexible CSS/XPath selectors, and comprehensive error handling. Designed for both simple scraping tasks and complex, large-scale data extraction projects.

Required Ruby Version

>= 3.1.0

Authors

Aristotelis Rapai

Versions

  1. 0.2.3

    1. source

      1. 0.2.3 source October 03, 2025 (31.5 KB)
  2. 0.2.2

    1. source

      1. 0.2.2 source October 01, 2025 (31.5 KB)
  3. 0.2.1

    1. source

      1. 0.2.1 source September 30, 2025 (31 KB)
  4. 0.2.0

    1. source

      1. 0.2.0 source September 29, 2025 (31 KB)
  5. 0.1.0

    1. source

      1. 0.1.0 source September 29, 2025 (30.5 KB)

Pushed by

aristorap

SHA 256 checksum

← Previous version