1
我正在使用WordPress。其中一个文件functions.php
包含阻止Google抓取的function do_robots() {...
。我已经取代了此功能有以下:如何在WordPress上创建Google Bot友好的robot.txt?
function do_robots() {
header('Content-Type: text/plain; charset=utf-8');
do_action('do_robotstxt');
if ('0' == get_option('blog_public')) {
echo "User-agent: *";
echo "\nDisallow: /wp-admin";
echo "\nDisallow: /wp-includes";
echo "\nDisallow: /wp-content";
echo "\nDisallow: /stylesheets";
echo "\nDisallow: /_db_backups";
echo "\nDisallow: /cgi";
echo "\nDisallow: /store";
echo "\nDisallow: /wp-includes\n";
} else {
echo "User-agent: *";
echo "\nDisallow: /wp-admin";
echo "\nDisallow: /wp-includes";
echo "\nDisallow: /wp-content";
echo "\nDisallow: /stylesheets";
echo "\nDisallow: /_db_backups";
echo "\nDisallow: /cgi";
echo "\nDisallow: /store";
echo "\nDisallow: /wp-includes\n";
}
}
- 我不太肯定
Allow
。难道只要我不Disallow
,默认是Allow
? - 为什么Google Bot仍然被上述
function
阻挡?