A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.

Features

  • link checking
  • load-testing
  • website downloading

Project Activity

See All Activity >