Why does WP not like my <a> container?

Question

Lately I've been working on some shortcodes where all the elements the shortcode produces are wrapped inside an tag. Unfortunately this is not rendered like wanted, because of the filter wpautop on the_content or whatever you're using to display shortcodes and/or regular content.

The setup

Imagine that we created the following shortcode:

<?php

function example_shortcode($atts, $content = null) {
    extract(shortcode_atts(
        array(
           'before' => '',
           'after' => '',
           'link' => '',
        ), $atts)
    );

    $before = (!empty($atts['before'])) ? '<div class="before">'.$atts['before'].'</div>' : '';
    $after = (!empty($atts['after'])) ? '<div class="after">'.$atts['after'].'</div>' : '';
    $link = (!empty($atts['link'])) ? ' href="'.$atts['link'].'"' : '';

    return '<a'.$link.'>'.$before.do_shortcode($content).$after.'</a>';
}
add_shortcode('example_shortcode', 'example_shortcode');

Also we created a page with the following content.

some text before the shortcode

[example_shortcode before="Welcome" after="Goodbye" link="#"]

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

[/example_shortcode]

some text after the shortcode

The tests

With wpautop filter added with the default priority the output looks like this:

<p>some text before the shortcode</p>
<a href="#"><div class="before">Welcome</div></p>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>
<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum</p>
<p><div class="after">Goodbye</div></a>
<p>some text after the shortcode</p>

Or with remove_filter('the_content', 'wpautop'); added it returns correct HTML as well:

some text before the shortcode

<a href="#"><div class="before">Welcome</div>

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

<div class="after">Goodbye</div></a>

some text after the shortcode

But when the wpautop filter is added with a higher priority for example...

<?php
remove_filter('the_content', 'wpautop');
add_filter('the_content', 'wpautop', PHP_INT_MAX);

Then the output returns all messed up, like this:

<p>some text before the shortcode</p>
<p><a href="#">
</a></p><div class="before"><a href="#">Welcome</a></div><a href="#">
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>
<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum</p>
<div class="after">Goodbye</div>
</a><p><a href="#"></a></p>
<p>some text after the shortcode</p>

The problem and my goal

Why is the <a> tag added multiple times when it only should open and close once?

This is the HTML what I want to render and is possible to achieve that when changing the <a> tag inside the example_shortcode function to e.g. a <div> tag. Why does the output of these tags differ?

<p>some text before the shortcode</p>
<a href="#">
<div class="before">Welcome</div>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>
<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum</p>
<div class="after">Goodbye</div>
</a>
<p>some text after the shortcode</p>

Answer 1

I think it's safe to say that wpautop() is a basket case without hurting anyone's feelings, but I wouldn't remove & add it at a different priority as that just makes things worse, as demonstrated (although what you posted is the browser trying to make sense of broken html, rather than the actual output, which does only have the <a> tag once).

To fix up the mangled html produced at normal priority I've found applying this code to the shortcode content fixes up most problems:

function cleanup_sc_content( $sc_content ) {
    $sc_content = force_balance_tags( $sc_content );
    $sc_content = preg_replace( '/<p>\s*+(<br\s*\/*>)?\s*<\/p>/i', '', $sc_content );
    return $sc_content;
}

which is a repurposing (with trivial modifications) of the original remove_empty_p() function posted by @Michelle, which in turn is due to @mtinsley (github ninnypants, original gist ).

To use it, I wouldn't personally add it as a filter but apply it directly to the shortcode content, eg:

return '<a' . $link . '>' . $before  
   . cleanup_sc_content( do_shortcode( shortcode_unautop( $content ) ) ) 
   . $after . '</a>';

(note the shortcode_unautop() hack as well is needed, as although wpautop() runs on the content of shortcodes, shortcode_unautop() doesn't - yes, really). However given the constant whack-a-mole "fixes" of wpautop() and the many outstanding and confusing bugs (spanning years) that may or may not apply, this (and any) solution will be unstable.