It's reasonably priced at $2 per million pages crawled and 3 cents per CPU hour.
而它的价格很合理，每爬行100万个页面只要2美元，每片CPU每小时只收3分钱。
As the program crawled the various Web sites, it would build a database of the sites and pages crawled, the links each page contained, the results of analysis on each pages, and so on.
当爬虫程序爬过不同的Web站点时，它将建立一个数据库，该数据库中包括它所爬过的站点和网页、每一页所包含的链接、每一页的分析结果等数据。
Once you set up the sitemap to contain all relevant pages to be crawled by the respective robots, the information is ready to export to the file system as a Sitemaps 0.90 compliant XML file.
建立了站点地图，并使之包含所有将被各机器爬行的相关页面之后，便可以开始把信息作为Sitemaps 0.90兼容XML文件导出到文件系统。
Basically, the portlet pages are crawled on the server, and then downloaded and cached locally on the device for viewing when the device is offline.
基本上说来，portlet页面先在服务器上被搜寻，然后在设备上被本地下载和高速缓存以在设备脱机状态下查看。
In my case, I wanted to only search official documents, and that's exactly what Google has done - returning results that are crawled from those pages only. No garbage, spam sites or erroneous data.
在我的例子中，我只要搜索官方文件，而这些正是Google已经做的：返回的结果中从包含设置的网站中抓取的，没有垃圾网站和错误的数据。
For the mass number of web pages needed to be crawled, how the information crawling system gets more web pages with good quality effectively, is related to the system performance.
信息采集系统需要采集的页面数量非常浩大，如何有效地采集到高质量的网页关乎系统性能。
Improved index coverage is not instantaneous as it takes some time for the pages to be crawled and indexed.
改善指数的涵盖范围是不是瞬时，因为它需要一些时间，为的网页抓取，索引。
Improved index coverage is not instantaneous as it takes some time for the pages to be crawled and indexed.
改善指数的涵盖范围是不是瞬时，因为它需要一些时间，为的网页抓取，索引。
应用推荐