Hot answers tagged encoding
11
< and > are encoded as +ADw- and +AD4- in UTF-7. Now imagine the following:
Someone sends +ADw-script+AD4-alert(+ACI-Hello+ACI-)+ADw-/script+AD4- as comment text. It will pass all validation unescaped.
The database expects and treats all incoming data as UTF-8. Since all UTF-7 streams are valid UTF-8 too, this will never result in a SQL error, and ...
5
See the discussion for Taxonomy Short Description for a better way to shorten a string. I’m not aware of a WP function that is getting truncation right.
Here is my code based on the linked discussion:
/**
* Shortens an UTF-8 encoded string without breaking words.
*
* @param string $string string to shorten
* @param int $max_chars maximal ...
5
Interesting question. This could be a useful part of a specific language file. It cannot be done in CSS, because CSS is (mostly) character agnostic. But using a filter and PHP it is possible and on topic:
add_filter( 'the_content', 't5_chinese_spacing' );
function t5_chinese_spacing( $content )
{
return preg_replace(
...
3
It happens because when you save a post, WordPress calls sanitize_title function to sanitize your title. This function applies sanitize_title filter.
One of core hooks for sanitize_title filter is sanitize_title_with_dashes function, which checks title on utf8 format by calling seems_utf8 function and if the title has utf8 format, the function call ...
3
My guess is that you pasted something from a Windows environment and that's the CR (carriage return, I think) character. I'd go into the editor and just remove those line breaks. That should fix the problem. Also, you might look into standardizing the content type meta tag. You've got
<meta charset="UTF-8" />
I could be wrong about this, but in ...
3
So after about a year (on and off!) I had managed to hopefully get a fix on encoding issue.
Why it breaks
What my experience boiled down to, is that encoding issue like this are mostly caused by miscommunication when moving data around.
in best case this is read mismatch, when correct data is wrongly interpreted
in worst case that is write mismatch, when ...
2
The problem is an invalid UTF-8 character in the excerpt <description>. The text is cut off in the middle of a multi-byte character, rendering your complete feed invalid. You get a slightly hint when you look at it in Opera.
Do you filter 'bloginfo_rss' or 'get_bloginfo_rss'? Disable all plugins and look if the feed is still broken.
2
Omar,
A good article about charsets can be found at http://codex.wordpress.org/Converting_Database_Character_Sets.
Unless you have very specific requirements to the languages you are using, you will want to use charset utf8 with collation utf8_general_ci.
The default WP settings are OK for this. I.e. charset definition in wp-config.php will be UTF-8. No ...
2
To output JSON always the function json_encode( $string ). The function is not available on all hosts. Don't worry, WordPress offers a fallback in wp-includes/compat.php. That's a wrapper for class Services_JSON::encodeUnsafe() (see wp-includes/class-json.php).
If you take a look at the source you'll see: It's not a trivial job to encode a string. :)
There ...
2
The reason commenting out (see q ) didn't work is that the data was wrong in the database, so no encoding change could fix it.
I restored the database by copy pasting the sql script into PHPMyAdmin, and the text box interpreted the text as having the wrong encoding. By using import in phpmyadmin and selecting the file, that data came in correctly.
2
Your browser doesn’t send the copied text UTF-8 encoded to the server. On the first character that isn’t UTF-8 compatible, in your example the apostrophe – ’ –, the stream to the data base is broken and not recovered. That’s a browser issue.
If I had to guess – well, I have because you didn’t mention it – I would say Internet Explorer.
There are two ways to ...
2
There are $wpdb->charset and $wpdb->collate. I am not sure if or when one of these values might be empty, so it is better to prepare for empty values …
From my DB class:
/**
* Get table charset and collation.
*
* @since 2012.10.22
* @return string
*/
protected static function get_wp_charset_collate() {
global $wpdb;
$charset_collate = ...
1
The file wp-admin/includes/upgrade.php includes wp-admin/includes/schema.php. This at the top declare as global (see source):
// Declare these as global in case schema.php is included from a function.
global $wpdb, $wp_queries, $charset_collate;
...
$charset_collate = '';
if ( ! empty( $wpdb->charset ) )
$charset_collate = "DEFAULT CHARACTER SET ...
1
The problem ended up being (I think) Powershell's wrangling of the output from mysqldump. In powershell, I had been using:
mysqldump -u**** -p**** -h**** wordpress --default-character-set=utf8 |
out-file out.sql -Encoding UTF8
I was even good about explicitly outputting UTF8 for both mysqldump and out-file! However, it seems (and this is really hard ...
1
It looks to me like you're not using an actual apostrophe, but one of the curly apostrophes, such as those that Microsoft Word might be configured to auto-correct to. There is a chance that your mysqldump was actually exporting it incorrectly. In my test database, both PowerShell and Command Prompt were dumping as ΓÇÖ, whereas Cygwin was dumping as ’. If ...
1
According to this page, you can use the tiny_mce_before_init filter, make sure the entity encoding is set to named, and then add whichever special characters you want to the entities array.
1
EDIT:
Disable magic_quotes_gpc in your server.
try adding in .HTACCESS file (if you on shared hosting):
php_flag magic_quotes_gpc off
If you'll get 500 server error after you added it - delete it and put this:
ini_set ('magic_quotes_gpc', 0); in theme's functions.php file. And with function bellow check is it on.
You can create php file with this content ...
1
No, it is not entirely possible.
WordPress is using Percent encoding to sanitise the slug. You could undo this, and it would deliver nonlatin characters, but the browser would immediately swap these out for percent encoded characters when you tried to visit the page.
Wether your database will store these characters, and the table encoding is irrelevant, as ...
1
It seems that the "convert_chars" filter is causing the trouble, try removing it by:
remove_filter( 'the_content', 'convert_chars' );
Also try removing the richedit_pre hook (which formats the text for the rich text editor, also uses convert_chars function) by:
remove_filter( 'the_editor_content', 'wp_richedit_pre' );
Hope that helps!
1
After doing a little searching on this problem, It's my understanding that the data is actually encoded in utf-8, but being handled like latin. You just need to trick it into reading it correctly with a little juggling.
Try this:
Export the db in its current state, copy the dump file for backup.
Create a new database usng utf-8.
Change the character set ...
1
The content you get is not utf-8 encoded or the encoding is damaged by the importer. Usually SimplePie – the feed reader WordPress uses – is quite good at handling such cases. I wonder what an importer you use …
Anyway, I’ve written a function to enforce utf-8 output. You should implement this very early in the import process. The article is in German – let ...
1
After spending the entire day working on this, I finally found a guide that worked perfectly:
http://www.blogsuccessjournal.com/blog-tips-and-advice/wordpress-tips-advice/seeing-weird-characters-on-blog-how-to-fix-wordpress-character-encoding-latin1-to-utf8/
Before that, I tried following @Rarst's information, tried exporting the database and manually ...
1
Maybe you should consider option C).
Convert all accented characters to normal UTF-8 characters.
So EXPRESSÃO.jpg -> EXPRESSAO.jpg
I think this would help you a lot, not only when it come sto coding and file systems, but also storing names / references in databases.
Update
This is a function I use for removing accents. I found this solution somewhere on ...
1
Consider this function as a rough placeholder. It has more flaws than you might imagine … :)
There are many plugins to improve the conversion for different languages and needs. You may take a look at my plugin Germanix to see how this could be done.
1
Yetti,
Pls look at my answer for Word press replacing unicode characters with "?"s. Amongst others you could take look at the mentioned article http://codex.wordpress.org/Converting_Database_Character_Sets.
You should really stick to UTF8 unless there's a very good reason not to do so.
If code and db are all in sync using utf-8, you shouldnt have any ...
1
Try this (not tested, I hope I am not short circuiting it into endless loop):
add_filter('sanitize_user', 'non_strict_login', 10, 3);
function non_strict_login( $username, $raw_username, $strict ) {
if( !$strict )
return $username;
return sanitize_user(stripslashes($raw_username), false);
}
1
Let's give this a guess out of the blue: Check the encoding of your plugin file. It should be ASCII, also named US-ASCII or ascii-7bit. This ensures that your plugin is most compatible to wordpress installations out there.
This information is probably useful for you to find out about the actual encoding of your plugins files (you need to encode all as ascii ...
1
You must go about the language functions-interface. See my example for german users: http://wordpress-buch.bueltge.de/wp-content/uploads/2010/05/de_DE.phps
We must also build differences on the permalinks for aour charachters in our language
/* define it global */
$umlaut_chars['in'] = array( chr(196), chr(228), chr(214), chr(246), chr(220), ...
Only top voted, non community-wiki answers of a minimum length are eligible
