Arabic permalink not found

Question

I am working on Arabic wordpress website and I have a problem when using arabic title for the post. the single post page always redirect to 404 page but with english permalinks it work perfectly

when I tried to use the_permalink(); it returns:

http://a3lamy.com/%d8%aa%d8%ac%d8%b1%d8%a8%d8%a9/

while in wordpress admin dashboard when edit post the permalink looks like:

http://a3lamy.com/تجربة/

the datebase server collation is utf8_general_ci

when I open the wp-posts table I found the post-name is :

%d8%aa%d8%ac%d8%b1%d8%a8%d8%a9

and when I change it manually it become تجربة in the database but still the permalink redirects to 404 not found page!

My web.config

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="WordPress Rule" stopProcessing="true">
<match url=".*" />
<conditions>
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
</conditions>
<action type="Rewrite" url="index.php" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>

Any help will be appreciated.

Answer 1

This issues is caused due to IIS does not recognize the multi language urls,

You need to add the following code at the end of the wp-config.php file:

if ( isset($_SERVER['UNENCODED_URL']) ) {
$_SERVER['REQUEST_URI'] = $_SERVER['UNENCODED_URL'];}

Answer 2

That's because what you want isn't possible, the RFC says URLs must be a subset of ANSI characters, which covers a subset of latin a-Z characters, numbers and symbols. There is no such thing as a Unicode URL, and it has nothing to do with your database encoding, WordPress is doing its job and this is expected behaviour.

http://www.blooberry.com/indexdot/html/topics/urlencoding.htm

"...Only alphanumerics [0-9a-zA-Z], the special characters "$-_.+!*'()," [not including the quotes - ed], and reserved characters used for their reserved purposes may be used unencoded within a URL."

However, not every country speaks US english, so we have RFC3986: http://www.faqs.org/rfcs/rfc3986.html

Wikipedia says:

While URIs are limited to a subset of the ASCII character set, IRIs may contain characters from the Universal Character Set (Unicode/ISO 10646), including Chinese or Japanese kanji, Korean, Cyrillic characters, and so forth.

Syntax

IRI extend upon URIs by using the Universal Character Set whereas URIs were limited to the ASCII with far fewer characters. IRIs may be represented by a sequence of octets but by definition is defined as a sequence of characters because IRIs can be spoken or written by hand.2 This is how URLs work with foreign non-ANSI characters. Because URLs only support a subset of ANSI, non-latin characters must be encoded.

It's not great, but the original HTTP spec can't handle non-english characters, and this is the hack they used to get around that. The same thing will happen with Kanji, emoji, and other non-english letters

An Experiment

So if I create a test page named تجربة :

Then visit the page:

Everything looks correct, but if I copy paste the URL, I get this:

https://tomjn.com/%D8%AA%D8%AC%D8%B1%D8%A8%D8%A9/

Which is the https://tomjn.com/تجربة/ encoded as a URL

%D8%AA%D8%AC%D8%B1%D8%A8%D8%A9 is a set of arabic characters, where each % encoded octet represents the character code in the universal character set

This is expected behaviour, and how it's supposed to work and is implemented in all browsers and HTTP speaking applications that support internationalised URLs and domains

The reason the_permalink does this is because it runs the URL through esc_url and urlencode, but if you removed this and output it as is on the page, it wouldn't change things as the browser would then do it automatically at the users end. If it didn't, then you'd end up with a mangled HTTP request that didn't work correctly.

So Where Do the 404s Come From?

If you go into the database and change the slug manually to تجربة then WordPress will never be able to find it. The browser will change it to %d8%aa%d8%ac%d8%b1%d8%a8%d8%a9, and WP will then search for that in the database. It won't find it though as it's been changed to تجربة