Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-da gem provides a PostgreSQL-based content meta-data store and work priority queue.
David Kellum
gem "iudex-da", "~> 1.3.3"