3

Does anyone know how I can have a unique robots.txt file for every domain served by my wp-multisite-installation? I did a search on plugins but couldn't find anything suitable.

3

Straight from the source, (line 1845 wp-includes/functions.php, 3.3.1):

function do_robots() {
    header( 'Content-Type: text/plain; charset=utf-8' );

    do_action( 'do_robotstxt' );

    $output = "User-agent: *\n";
    $public = get_option( 'blog_public' );
    if ( '0' == $public ) {
        $output .= "Disallow: /\n";
    } else {
        $site_url = parse_url( site_url() );
        $path = ( !empty( $site_url['path'] ) ) ? $site_url['path'] : '';
        $output .= "Disallow: $path/wp-admin/\n";
        $output .= "Disallow: $path/wp-includes/\n";
    }

    echo apply_filters('robots_txt', $output, $public);
}

So to customise it:

function my_custom_robots( $robots )
{
    if ( my_condition() )
        $robots .= "\nDisallow: /something/else/";
    return $robots;
}
add_filter( 'robots_txt', 'my_custom_robots' );
  • Thanks for this. A lot of blog posts just talk about the filters but yours is the first that demonstrates by actually using it. – racl101 Nov 22 '12 at 16:42
0

I was able to get this working to create a sitemap using the following:

function my_custom_robots( $robots ){
    $GLOBALS['current_blog']->blog_id; 
    if ( $blog_id !=(1) )
    $robots .= "Sitemap: /sitemap.xml";
    return $robots;
  }
add_filter( 'robots_txt', 'my_custom_robots' );

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.