iWebCrawler is a web spider(crawler) application, iWebCrawler crawls Internet and retrieves plenty of Internet information with amazing speed. iWebCrawler offers downloading and saving network resources like web pages, audio/video/image binaries as well as documentations and applications to your local physical disk.
iWebCrawler has following characteristics
•High-speed performance of downloading Internet resources
•Wide scope of target network resources
•Configurable target resource types and properties.
•Configurable crawler behaviors (you can either let the crawler crawls a specific domain, or all domains belong to same company, or the whole Internet from being specified entry pages).
•Configurable domain filter.
•Configurable performance settings.
•Multi-threads-enabled resource downloading
•Asynchronous DNS querying
•Depth-prioritized crawling mechanism
•Use proxy for resource retrieving.
•Network bandwidth control.