This happens because you are missing the neccessary nonce in the URL, which is being checked in wp-login.php
case 'logout' :
check_admin_referer('log-out');
...
Use wp_logout_url in order to retreive the URL including the nonce. If you want to redirect to a custom URL, simply pass it as an argument.
<a href="<?php echo wp_logout_url('/redirect/url/goes/here') ?>">Log out</a>
You could also use wp_loginout which generates the link for you including translation:
echo wp_loginout('/redirect/url/goes/here')