Each search engine has its own automated program called a "web spider" or "web crawler" that crawls the web.
每个搜索引擎都有自己爬行网页的自动化程序，叫做“网络蜘蛛（web spider）”或“网络爬虫（web crawler）”。
Using it, you can setup a multi-threaded web crawler in few minutes.
提供了简单易用的接口，可以在几分钟内创建一个多线程网络爬虫。
The main goals of focused web crawler are to get more web pages which are correlative with a certain topic and prepare data for users querying.
聚焦网络爬虫并不追求大的覆盖，而将目标定为抓取与某一特定主题内容相关的网页，为面向主题的用户查询准备数据资源。
First, the search depth is not ideal. Search engines obtain information on Internet via web crawler, so they cannot get the Shared information stored in users' personal computer.
第一是搜索深度不够，当前搜索引擎通过网络蜘蛛获取互联网上的资源，无法检索用户个人电脑上的共享资源。
The CollectUrls Web crawler program takes advantage of a fixed-size thread pool.
CollectUrlsWeb crawler程序利用一个固定大小的线程池。
Including full text search and Web crawler.
包括全文搜索和Web爬虫。
This information helps the Web crawler determine what the set of pages is and when to crawl them.
这一信息能帮助web爬虫程序决定要爬行哪些页面以及爬行的时间。
To expand the Web crawler, consider collecting image references or searching for specific text strings.
要扩展该Web crawler，可以考虑收集图像引用或搜索特定的文本字符串。
The high-level task for this article is a Web crawler: given a base URL for a Web site, you'll collect elements from the site that you can use for some purpose.
本文的高级任务是创建一个Web crawler：给定一个网站的基URL，从该网站收集可以用作某种用途的元素。
Web crawler is a system which can automatically get web pages from Internet. It helps searching engine download web pages, so it is an important part of searching engine.
网络爬虫是一个可以从因特网上自动提取网页的系统，它为搜索引擎从万维网上下载网页，是搜索引擎的重要组成。
Web-Crawler is a important part of search engine, it is responsible for the network information gathering.
网络爬虫是搜索引擎的重要组成部分，它在搜索引擎中负责网络信息的采集。
Web-Crawler is a important part of search engine, it is responsible for the network information gathering.
网络爬虫是搜索引擎的重要组成部分，它在搜索引擎中负责网络信息的采集。
应用推荐