One of the most important parts of search engine is WebCrawler which can get the original information from network for the search engine.

搜索引擎一个重要部分是网络爬虫程序，依靠网络爬虫，搜索引擎可以获取用来检索的原材料信息。

youdao

In this article we use the heuristic search to get the specific information. This can reduce the links largely and make the links visited by WebCrawler point to useful information.

在本文中，提出使用人工智能中的启发式搜索来获取特定的信息，这样可以极大地减少遍历的链接数量，使被访问到的链接尽量地指向有用的信息。

youdao

It uses information sources such as AltaVista, Excite, Yahoo, HotBot, Infoseek, Lycos, Magellan, Open Text, WebCrawler, and DejaNews.

CNN: Downloads to improve your Web experience

To spot watermarked pictures being used illicitly, a program known as a webcrawler can trawl the Internet looking for them.

ECONOMIST: In the picture