wp_remote_get(), downloading and saving files

Question

From what I can see wp_remote_get() saves remote file contents to memory.

The files that I need to download are compressed in either ZIP or GZIP and within which will either be a CVS or XMl file

What I need to do first is download the remote file to the harddrive as ZIP either GZIP and then unzip them

Is it possible to use wp_remote_get() to download the whole file and save it to a directory?

A non-Wordpress solution I used before was cURL:

public function grab_file($url, $new_file) {

    //get file
    $ch = curl_init();
    $fp = fopen(DIR_PATH."zip/$new_file", "w");

    $options = array(CURLOPT_URL => $url, CURLOPT_HEADER => 0, CURLOPT_FAILONERROR =>
        1, CURLOPT_AUTOREFERER => 1, CURLOPT_BINARYTRANSFER => 1, CURLOPT_RETURNTRANSFER =>
        1, CURLOPT_FOLLOWLOCATION => 1, CURLOPT_TIMEOUT => 5, CURLOPT_FILE => $fp);

    curl_setopt_array($ch, $options);
    $file = curl_exec($ch);
    curl_close($ch);
    fclose($fp);

    if (!$file) {
        //$error = "cURL error number:" . curl_errno($ch);
        //$error .= "cURL error:" . curl_error($ch);
        return false;

    } else {

        return true;

    }
}

Answer 1

Well here's the solution that I came up with:

// Use wp_remote_get to fetch the data
$response = wp_remote_get($url);

// Save the body part to a variable
$zip = $data['body'];

// In the header info is the name of the XML or CVS file. I used preg_match to find it
preg_match("/.datafeed_([0-9]*)\../", $response['headers']['content-disposition'], $match);

// Create the name of the file and the declare the directory and path
$file = DIR_PATH."zip/"."datafeed_".$match[1].".zip";

// Now use the standard PHP file functions
$fp = fopen($file, "w");
fwrite($fp, $zip);
fclose($fp);

// to unzip the file use the Wordpress unzip_file() function
// You MUST declare WP_Filesystem() first
WP_Filesystem();
if (unzip_file($file, DIR_PATH."feeds")) {
// Now that the zip file has been used, destroy it
unlink($file);
return true;
} else {
return false;
}

Dealing with a GZIP file was a bit different:

// It is necessary to use the raw URL and find the extension of the encluse XML or CVS file first
private function save_ungzip($data, $url, $ext) {

// As with ZIP, save the body of wp_remote_get() to memory
$gzip = $data['body'];

// As with ZIP, look for the name of the file in the header
preg_match("/.datafeed_([0-9]*)\../", $data['headers']['content-disposition'], $match);
$file = DIR_PATH."feeds/"."datafeed_".$match[1].".$ext";

// now you need to use both the PHP gzip functions and the PHP file functions
$remote = gzopen($url, "rb");
$home = fopen($file, "w");

while ($string = gzread($remote, 4096)) {
    fwrite($home, $string, strlen($string));
}

gzclose($remote);
fclose($home);

}

So in conclusion. When you use wp_remote_get() to retrieve a remote file use the inbuilt PHP file functions to then save it where you want to. If you have a solution that utilizes Wordpress functions then please post it up.

Answer 2

Althoug this question is from a few years back, since it pops up in Google as one of the first results, I thought I'd give a better suggestion to write wp_remote_get()'s response to a file.

The beginning is good, but the OP uses fopen() and fwrite() to write the file, while that's not necessary:

// Use wp_remote_get to fetch the data
$data = wp_remote_get($url);

// Save the body part to a variable
$zip = $data['body'];

// Then the unzipping part...

// Create the name of the file and the declare the directory and path
$file = '/path/to/file.zip';

// Write the file using put_contents instead of fopen(), etc.
global $wp_filesystem;

$wp_filesystem->put_contents($file, $zip);

In some cases $wp_filesystem isn't accessible. In which case you can create a function to get it:

function filesystem()
{
    global $wp_filesystem;

    if ( is_null( $wp_filesystem ) ) {
        require_once ABSPATH . '/wp-admin/includes/file.php';
        WP_Filesystem();
    }

    return $wp_filesystem;
}