Usually on a site, there are pages that are not useful for a search engine to index, such as files in the scripts directory, administrative pages, error pages, and so on.

站点上通常都含有那些搜索引擎没必要建立索引的页面，例如脚本目录中的文件、管理页面和错误页面等等。

youdao

This site is a link directory for Google's custom search engine.

该站点是一个Google定制搜索引擎链接目录。

youdao

This directory contains the search index files produced by the Lucene search engine as shown in Figure 10.

此目录包含Lucene搜索引擎产生的搜索索引文件，如图10中所示。

youdao

The Walt Disney Company has bought 43% of Infoseek, the Internet search engine and directory service.

BBC: News | The Company File | Disneyseek