Avoid anything that makes it hard for the spider to crawl your site.
避免任何会阻碍爬行器在站点中漫游的东西。
youdao
Click on Site Configuration and select Crawl Access.
点击网站配置-抓取工具权限。
应用推荐
模块上移
模块下移
不移动