Our method solved the problems of low level automatic and domain constrain in the deep web crawling.
我们的方法解决了现有深度网络信息爬取中自动化程度低，适用领域窄的问题。
However the current two ways of web crawling used by focus crawler are low efficiency.
然而目前的主题爬虫所采用的两种基本抓取网页的方式效率比较低下。
But all the major and legitimate Web crawling engines obey the requests in robots.txt.
不过，所有主要的合法Web爬虫引擎都会遵从robots . txt内的要求。
But all the major and legitimate Web crawling engines obey the requests in robots.txt.
不过，所有主要的合法Web爬虫引擎都会遵从robots . txt内的要求。
应用推荐