Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

Does anyone know how I can have a unique robots.txt file for every domain served by my wp-multisite-installation? I did a search on plugins but couldn't find anything suitable.

share|improve this question

1 Answer 1

Straight from the source, (line 1845 wp-includes/functions.php, 3.3.1):

function do_robots() {
    header( 'Content-Type: text/plain; charset=utf-8' );

    do_action( 'do_robotstxt' );

    $output = "User-agent: *\n";
    $public = get_option( 'blog_public' );
    if ( '0' == $public ) {
        $output .= "Disallow: /\n";
    } else {
        $site_url = parse_url( site_url() );
        $path = ( !empty( $site_url['path'] ) ) ? $site_url['path'] : '';
        $output .= "Disallow: $path/wp-admin/\n";
        $output .= "Disallow: $path/wp-includes/\n";
    }

    echo apply_filters('robots_txt', $output, $public);
}

So to customise it:

function my_custom_robots( $robots )
{
    if ( my_condition() )
        $robots .= "\nDisallow: /something/else/";
    return $robots;
}
add_filter( 'robots_txt', 'my_custom_robots' );
share|improve this answer
    
Thanks for this. A lot of blog posts just talk about the filters but yours is the first that demonstrates by actually using it. –  racl101 Nov 22 '12 at 16:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.