That's because what you want isn't possible, the RFC says URLs must be a subset of ANSI characters, which covers a subset of latin a-Z characters, numbers and symbols. There is no such thing as a Unicode URL, and it has nothing to do with your database encoding, WordPress is doing its job and this is expected behaviour.
"...Only alphanumerics [0-9a-zA-Z], the special characters "$-_.+!*'()," [not including the quotes - ed], and reserved characters used for their reserved purposes may be used unencoded within a URL."
However, not every country speaks US english, so we have RFC3986:
While URIs are limited to a subset of the ASCII character set, IRIs may contain characters from the Universal Character Set (Unicode/ISO 10646), including Chinese or Japanese kanji, Korean, Cyrillic characters, and so forth.
IRI extend upon URIs by using the Universal Character Set whereas URIs were limited to the ASCII with far fewer characters. IRIs may be represented by a sequence of octets but by definition is defined as a sequence of characters because IRIs can be spoken or written by hand.2
This is how URLs work with foreign non-ANSI characters. Because URLs only support a subset of ANSI, non-latin characters must be encoded.
It's not great, but the original HTTP spec can't handle non-english characters, and this is the hack they used to get around that. The same thing will happen with Kanji, emoji, and other non-english letters
So if I create a test page named تجربة :
Then visit the page:
Everything looks correct, but if I copy paste the URL, I get this:
Which is the https://tomjn.com/تجربة/ encoded as a URL
%D8%AA%D8%AC%D8%B1%D8%A8%D8%A9 is a set of arabic characters, where each % encoded octet represents the character code in the universal character set
This is expected behaviour, and how it's supposed to work and is implemented in all browsers and HTTP speaking applications that support internationalised URLs and domains
the_permalink does this is because it runs the URL through
urlencode, but if you removed this and output it as is on the page, it wouldn't change things as the browser would then do it automatically at the users end. If it didn't, then you'd end up with a mangled HTTP request that didn't work correctly.
So Where Do the 404s Come From?
If you go into the database and change the slug manually to
تجربة then WordPress will never be able to find it. The browser will change it to
%d8%aa%d8%ac%d8%b1%d8%a8%d8%a9, and WP will then search for that in the database. It won't find it though as it's been changed to