Maximum database size for crawler 2GB 2GB
爬虫的最大数据库大小2GB 2GB
Define the crawler subdirectories, as illustrated in Figure 8.
定义爬虫子目录，如图8所示。
The basic design of this crawler is to load the first link to check onto a queue.
这个爬虫的基本设计是加载第一个链接并将其放入一个队列。
The CollectUrls Web crawler program takes advantage of a fixed-size thread pool.
CollectUrlsWebcrawler程序利用一个固定大小的线程池。
The crawler works with multiple threads to fetch URLs from the crawler queue.
爬虫软件使用多个线程从队列爬虫获取url。
The following steps show you how to create the collection and the crawler, and to get it started
以下步骤演示如何创建集合和爬虫，准备开始
This information helps the Web crawler determine what the set of pages is and when to crawl them.
这一信息能帮助web爬虫程序决定要爬行哪些页面以及爬行的时间。
Select UNIX file system as the crawler type, as shown in Figure 6, and then click on the Next button.
选择UNIXfilesystem作为爬虫类型，如图6 所示，然后单击Next 按钮。
To expand the Web crawler, consider collecting image references or searching for specific text strings.
要扩展该Webcrawler，可以考虑收集图像引用或搜索特定的文本字符串。
Instead of converting your entire site to static URLs, pick the pages you want to index by a crawler.
代替将整个站点转换为静态url，您可以挑选一些希望爬行器建立索引的页面。
What you need to do, in short, is to generate a list of page references (URLs) for a crawler to fetch.
简而言之，我们需要做的就是生成一个页面引用列表（URL），爬虫程序通过这个列表获取信息。
Provides an entry point for the search engine crawler to easily follow the links within your Web pages.
为搜索引擎爬网程序提供入口点，以使爬网程序轻松地跟踪您的Web页面内的链接。
It's one of the most important signals your page offers to a crawler, so why not include a local signal?
这是你的页面向搜索引擎爬虫提供的最为重要的信息之一，所以为什么不提供一些本地化的信息呢？
Each search engine has its own automated program called a "web spider" or "web crawler" that crawls the web.
每个搜索引擎都有自己爬行网页的自动化程序，叫做“网络蜘蛛（web spider）”或“网络爬虫（web crawler）”。
The Sitemaps 0.90 protocol offers the option of a Sitemaps index file to be provided to the crawler as well.
Sitemaps 0.90提供了一个选项，可以把Sitemapsindex文件提供给爬虫程序。
First, let's look at how crawler-based search engines work (both Google and Yahoo fall in this category).
首先，让我们看看基于爬虫(crawler - based)的搜索引擎是如何工作的(Google和Yahoo都是这种类型)。
A sample scenario is based on a scheduled crawler for a website with regularly updated announcement pages.
一个样例场景就是包含定期更新的通知页面的网站的调度爬行器。
It USES a scheduler to initiate periodic events, such as crawler executions and full-text index maintenance.
它使用一个调度程序来发起周期性的事件，比如爬网程序执行和全文索引维护。
Next, navigate to the crawler details page and click 'Start full recrawl', as shown at the bottom of Figure 3.
接下来，导航到爬行器的细节页面并单击“Startfull recrawl”，如图3底部所示。
Use the OmniFind database crawler wizard to configure a crawler to access the VSAM content through the nickname.
使用OmniFind数据库crawler向导来配置一个crawler，从而通过昵称访问VSAM内容。
We can see that this file-based discovery complements UDDI, and may be used in a crawler-like fashion by clients.
我们可以看到，这种基于文件的发现对uddi是一个补充，而且可以被客户以类似crawler的方式使用。
Define the crawler name (UNIX file system crawler 1, for example), as shown in Figure 7, and then click on the Next button.
定义爬虫名称(例如，UNIXfilesystemcrawler1)，如图7所示，然后单击Next按钮。
E-mail harvesting can be one of the easiest crawling activities, as you'll see in the final crawler example in this article.
E - mail收集可能是最容易的一种爬行行为，在本文中最后一个爬虫例子中我们会看到这一点。
The mighty Crawler, which ferried Shuttles to the launchpad, will be reduced to hauling more terrestrial freight around the Space Center.
把航天飞机运到发射台的巨大的“爬行者”，将重新做航天中心附近更多的陆上牵引。
Word on the street is that director Marc Webb is looking to create a new love interest for his planned reboot of the wall-crawler franchise.
有传言说，导演马克·韦伯会为重新启动的攀墙者系列寻找一个新的恋爱目标。
Project Barcelona, a new project in the works from Microsoft, will give enterprises Web crawler-like tools for searching and storing information.
近日，微软准备推出自己最新的计划，他们称之为“Project Barcelona”。这个项目将用于企业级搜索和信息存储所用的网络爬虫工具。
The high-level task for this article is a Web crawler: given a base URL for a Web site, you'll collect elements from the site that you can use for some purpose.
本文的高级任务是创建一个Webcrawler：给定一个网站的基URL，从该网站收集可以用作某种用途的元素。
As with any search project, our faceted navigation demo requires that we define content sources and configure a crawler or other device to ingest and index them.
对于任何搜索项目，分面导航演示都要求定义内容源，配置收集这些内容源并为它们建立索引的信息收集器或其他设备。
As with any search project, our faceted navigation demo requires that we define content sources and configure a crawler or other device to ingest and index them.
对于任何搜索项目，分面导航演示都要求定义内容源，配置收集这些内容源并为它们建立索引的信息收集器或其他设备。
应用推荐