8

Not sure why this thread was closed, but this is the same issue inflicting many people.

All my WP config settings are in order:

//define('DB_CHARSET', 'utf8');
//define('DB_CHARSET', 'utf8_unicode_ci');
//define('DB_COLLATE', '');

I even tried enabling them one by one. None worked.

When I save a post, weird characters appear in place of apostrophes and spaces. This happens whether I type content manually or

I've tried a few plugins.

  1. UTF-8 Sanitize
  2. Convert WP to UTF-8

..etc.

None of them work. The problem persists.

I've also changed the database's character set and collation in MYSQL. Screenshot:

MySQL tables/columns are all utf-8

This is a screenshot of me entering some content by pasting it:

Text is fine when entering it in the WP UI

But immediately upon saving, the text comes back with garbled characters having replaced it:

Annoying garbled characters

What else?

I went through the rigmarole of dumping the entire MySQL DB, then removing all older non-utf8 characters via TR command.

tr -cd '\11\12\15\40-\176' < file-with-binary-chars > clean-file

Where the file-with-binary-chars was the MySQL dump. Then I restored the table.

My MySQL config is all utf8:

[client]
default-character-set=utf8

[mysqld]
character-set-client=utf8
collation-server=utf8_unicode_ci
character-set-server=utf8

My browser is Chrome. The encoding is UTF-8 (in the VIEW menu).

What else can I do? Do I need to make all the plugin files utf-8 as well?

FYI, this blog is one of the Wordpress blogs. There are other newer Wordpress installations on the same server using the same installation of MySQL 5.6.17, but they don't have such an issue. My guess is that this being an older blog may have some variations in the text entered long ago, but frankly after having done all of the above, I really don't know what else I can do.

Thanks for any inputs or pointers!

1

6 Answers 6

9

This is typically caused when you are copying/pasting MS Word information into the WordPress content editor. WordPress uses something called "Smart Quotes", via a function named wptexturize().

Ideal Solution

The ideal solution would be to go back through your content, and replace all single/double quotes using the keyboard.

However, if you're working with massive copy/pastes, this may not be feasible.

Disable wptexturize() Filter

Another option is to disable the wptexturize() filter from running; which can be accomplished by placing the following code in your child theme functions.php file:

remove_filter('the_content', 'wptexturize');

You may also wish to remove the filter from comments and/or excerpts:

remove_filter('comment_text', 'wptexturize');
remove_filter('the_excerpt', 'wptexturize');

Or for titles:

remove_filter ('single_post_title', 'wptexturize');
remove_filter ('the_title', 'wptexturize');
remove_filter ('wp_title', 'wptexturize');

Clean Database

For existing content which has already saved the "weird" characters into the database; you may need to clean the database by running the following queries from PHPMyAdmin (be sure to take a database backup first):

UPDATE wp_posts SET post_content = REPLACE(post_content, '“', '“');
UPDATE wp_posts SET post_content = REPLACE(post_content, 'â€', '”');
UPDATE wp_posts SET post_content = REPLACE(post_content, '’', '’');
UPDATE wp_posts SET post_content = REPLACE(post_content, '‘', '‘');
UPDATE wp_posts SET post_content = REPLACE(post_content, '—', '–');
UPDATE wp_posts SET post_content = REPLACE(post_content, '–', '—');
UPDATE wp_posts SET post_content = REPLACE(post_content, '•', '-');
UPDATE wp_posts SET post_content = REPLACE(post_content, '…', '…');

Plugins

Well... it's WordPress. You can always use a plugin to help manage the wptexturize() filter. Take a look through This List, and see if one is right for you.

1
  • Thank you for this very informative post and you're right the quotes do cause these woes. But it's not limited to quotes. Sometimes just the space before and after a period has these strange characters too. Secondly, clearly something somewhere is wrong, because this does not happen on any other blog on the exact same server, using the same base installation of MySQL and everything else. The config files are quite similar too.
    – PKHunter
    Aug 4, 2014 at 6:00
7

I was facing the same exact problem.

I tried everything just like you and at the end I tried one last thing.

Changed the DB_CHARSET in wp-config file to latin1.

And things worked :/

define('DB_CHARSET', 'latin1');
define('DB_COLLATE', 'utf8mb4_general_ci');
2
  • 1
    This worked great for me. One note, I think it's advisable to use utf8mb4_unicode_ci based on the information in this post At the very least, the options are laid out well here for anyone to make their own call on it. Oct 25, 2018 at 18:03
  • @ZacharyDow exactly... And it does depends on their personal setup.
    – vs_lala
    Nov 3, 2018 at 13:19
4

In another case, if you are using PHP Dom (loadHTML) somewhere, there is a need to load HTML as UTF-8. I have fixed it by:

Replacing

@$dom->loadHTML($html);

to

@$dom->loadHTML('<?xml encoding="UTF-8">' . $html);
3

The following fixed it for me, so posting here to share. Even though I ran the SQL updates suggested, I was still getting the funny symbols before the "read more" links.

I'm totally shocked. After hours of trying things and testing, I finally found the "Settings" then "Readings" option below: "Encoding for pages and feeds". After changing from UTF-7 to UTF-8, everything looks good again.

enter image description here

And even stranger, after changing it to UTF-8, the option disappears from the page. According to WordPress site, this option was removed since release 3.5.

1
  • 1
    Thanks @Neal I've fixed my problem as well with this solution! Aug 13, 2018 at 20:20
1

The issue can cause by the plugins and direct FTP uploads.

Here's the detail:

http://iiiji.com/wordpress-appearing-weird-characterscode/

1
  • 1
    Please add crux of the solution if not everything so that even if the link dies this answer is viable
    – bravokeyl
    Sep 10, 2016 at 5:15
1

In my case the strange characters weren't written to the database. Check if the raw SQL is showing the incorrect characters as well. If you're seeing regular apostrophes in the database, comment out the following two lines in the wp-config.php:

// define('DB_CHARSET', 'utf8');
// define('DB_COLLATE', '');

That fixed it for me right away.

"This character encoding problem can happen after a database upgrade" - Link

Not the answer you're looking for? Browse other questions tagged or ask your own question.