wgit 0.0.8

Wgit is a WWW indexer/scraper which crawls URL's, retrieves and serialises their page contents for later use. You can use Wgit to copy entire website if required. Wgit also provides a means to search indexed documents stored in a database. Therefore, this library provides the main components of a WWW search engine. The Wgit API is easily extendable allowing you to pull out the parts of a webpage that are important to you, the external links or keywords for example. As Wgit is an API, it's very useful in many different application types.