1

I am using wp_remote_get() to get access to certain pages on the net. However, when I make a call using wp_remote_get(), the response I recieve is a webpage that contains the following lines of code:

<body>
    <noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript>
</body>

So, the page is not returning in full because of the way it is programmed. Is there a way to change the call so that the non-noscript page returns instead?

Improve this question

1 Answer 1

Reset to default
2

This result is expected since wp_get_remote() doesn't include a Javascript engine.

It should give you the same result as the View Source option in your browser and fetch the plain HTML document.

It looks like the page your are trying to fetch, is loading a Javascript file, in the header tag, to render the content:

<html>
    <head>
        <script src="/js/load-content.js"></script>
    </head>
    <body>
        <noscript>Your browser doesn't support Javascript!</noscript>
    </body>
</html>

So you should get both the script and the noscript part.

What you are looking for is called headless browsing.

If you want to run the Javascript, you might want to take a look at solutions like: PhantomJS

PhantomJS is a headless WebKit scriptable with a JavaScript API.

[...] One major use case of PhantomJS is headless testing of web applications. It is suitable for general command-line based testing, within a precommit hook, and as part of a continuous integration system.

There exists many test frameworks for PhantomJS, like Jasmine, Mocha and QUnit.

Alternatively: Try to load the Javascript file with wp_remote_get() and see if you can extract the content that way ;-)

Improve this answer
2
  • Couple that with CapserJS and you have some real power at your hands. That aside, wouldn't setting the user-agent property of the args parameter on wp_remote_get be enough to have the page serve up its JS?
    – Adam
    Commented May 3, 2014 at 3:22
  • @userabuser great suggestion regarding CasperJS. I think wp_get_remote() will just serve the plain HTML, like the "View Source" in the browser. So you will get both the script and noscript tags, regardless of the user-agent string in the header.
    – birgire
    Commented May 3, 2014 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.