HTTP 403 Error when passing parameters to URL

Question

Our website uses WordPress. When we visit URL https://www.datanumen.com/outlook-repair/?utm_source=adword via browser such as Chrome and FireFox, we can see everything is fine. However, Google Ads reports "Destination not working" error for the URL and the detailed info shows "HTTP error 403".

After consulting Google Experts, they use https://httpstatus.io/ and find https://www.datanumen.com/outlook-repair/?utm_source=adword will return 403 code.

However, if removing the parameters from the URL, then the URL https://www.datanumen.com/outlook-repair/ will return 200 code.

So it seems the parameters in the URL make it return 403 code for bot-like access.

We find a similar post WordPress 403 issue when passing parameters in the URL, but we do not Cerber Security Plugin installed. We use WordFence Premium. We try to disable the WordFence, but the problem still exist.

We try another URL https://www.datanumen.com/outlook-repair-order/?utm_source=adword and it returns 200. The only difference between it and https://www.datanumen.com/outlook-repair/?utm_source=adword is that the latter is a custom post and we using Yoast SEO to remove the category from the permalink. So it seems the problem comes from that. But we don't know the accurate reason. And we have to remove the category from the permalink based on our company's requirement.

So, how to solve this problem?

Update:

I check the raw access log and find the following related lines:

173.212.203.156 - - [22/Jun/2020:17:32:55 -0700] "HEAD /outlook-repair/?utm_source=adword HTTP/1.1" 403 - "https://www.datanumen.com/outlook-repair/?utm_source=adword" "WordPress/5.3.4; https://newproxylists.com"

173.212.203.156 - - [22/Jun/2020:17:32:56 -0700] "GET /outlook-repair/?utm_source=adword HTTP/1.1" 403 18394 "https://www.datanumen.com/outlook-repair/?utm_source=adword" "WordPress/5.3.4; https://newproxylists.com"

Update 2

Some more tests:

1. Disable WordFense. Still 403.
2. Disable WOrdFense and Clear Cache. Still 403.
3. Disable WordFense, Yoast SEO, WPFastest Cache. Still 403.

Update 3

It seems https://webmasters.stackexchange.com/questions/128675/can-i-test-if-my-sites-urls-have-403-errors-for-google-ads-bot mentions the same problem as mine.

Update 4

I use UpdraftPlus Migrator to make a clone of the website https://www.datanumen.com/ to http://www.sybase-recovery.com/ , then tests on boths sites, the former will return 403 but the latter will return 200.

Update 5

I perform a more thorough tests on both the old site and the new site in Update 4, and finally find one URL in the new site also returns 403. All other URLs in the new site will continue return 200. stackexchange does not me to put more URLs in this post any more. Anyway, I can start with that specic URL in the new site to see what is the problem.

Answer 1

I can guarantee that somewhere on your technology stack (edge CDN, Firewall, Web Server, WordPress Plugin) that you have something configured that throws a 403 error whenever you try to access a URL on your site without the User-Agent Header set. Hence why it works when you access in the browser (as this will contain the User-Agent Header) and why it is failing when Google Ads is crawling (likely because the User-Agent Header they use for that is not in your whitelist).

Have a test using something like Postman to confirm.

These types of 'security features' are nonsense, it takes very little knowledge to spoof the User-Agent Header and hence cannot be trusted. Whatever is doing this, bin it off as it's clearly causing more problems than it's helping.... Then get something that provides actual security in place.

In fact... just had a quick play (but leaving the above in for reference as it doesn't sound like it is your issue, but it is handy to know for debugging...). With you using WordFence, plugins like that often send 403 whenever they detect an 'odd looking query string' which is often one of the settings. When you say the problem still existed after you turned it off, do you also use a Caching plugin too? If so, turn off ALL caching EVERYWHERE you use it (edge CDN, Firewall, Web Server, WordPress Plugin), then I'd expect the problem to go away. What is odd is that it is only happening on that one page though, suggests it's either stuck in the cache, or something more odd is going on.

Next steps to debugging, run through these common steps which help you to narrow down the problem, https://www.contradodigital.com/2016/08/12/how-to-debug-wordpress-problems/

Good hunting!