网页邮箱爬虫工具 Advanced Email Extractor
Designing a web crawler. It can only download the target web page and outlink-pages in the same domain.
设计了相应的网页爬虫,只下载待分类目标网页及其域内后向链接网页。
参考来源 - 基于贝叶斯算法和后向链接的中文网页组合分类研究·2,447,543篇论文数据，部分数据来源于NoteExpress
然后，在搜索引擎关键技术的基础上，基于一个轻量级的架构设计了搜索引擎的三个主要模块：网页爬虫、索引器与搜索器。
Then, on basic of search engine's core technologies, based on a lightweight architecture, its three main modules were designed: crawler, indexer and searcher.
例如，一个网站可能不排名，如果你的服务器停止服务的网页爬虫，或如果您已经改变了的网址，有很大一部分你网站的页面。
For example, a site may not rank well if your server stops serving pages to Googlebot, or if you've changed the URLs for a large portion of your site's pages.
当爬虫程序爬过不同的Web站点时，它将建立一个数据库，该数据库中包括它所爬过的站点和网页、每一页所包含的链接、每一页的分析结果等数据。
As the program crawled the various Web sites, it would build a database of the sites and pages crawled, the links each page contained, the results of analysis on each pages, and so on.
应用推荐