1

I made a plugin that basically reads a CSV file and imports data to relevant tables.

However, the action seems to create an error:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 65015808 bytes) in /var/www/proj/wp-includes/functions.php on line

which led me to this code in functions.php:

function wp_ob_end_flush_all() {
    $levels = ob_get_level();
    for ($i=0; $i<$levels; $i++)
        ob_end_flush();
}

I did a Google and came across two popular solutions, both of which didn't seem to work.

Solution 1: disabling zlib - this is already disabled.
Solution 2: remove_action('shutdown', 'wp_ob_end_flush_all', 1);

Solution 2 still errors but with no message, which, isn't exactly ideal.

This is the script that's causing the error:

<?php
    ini_set('display_startup_errors', 1);
    ini_set('display_errors', 1);
    error_reporting(-1);

    # load core wp fnc
    require_once $_SERVER['DOCUMENT_ROOT']. '/wp-load.php';
    # load db functions
    require_once $_SERVER['DOCUMENT_ROOT']. '/wp-admin/includes/upgrade.php';
    # load admin fnc
    require_once $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/admin/inc/admin-functions.php';

    global $wpdb;

    $admin_functions = new vendor_module_admin_functions();

    # get csv
    $file = $_FILES['csv'];

    $name = $file['name'];
    $dir = $file['tmp_name'];

    # rm spaces, replace with _
    $name = str_replace(' ', '_', $name);

    $file_location = $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/uploads/import/'. $name;

    # if successfully moved, carry on, else return
    $moved = ($file['error'] == 0 ? true : false);

    $error = false;

    if (!$moved) {
        echo 'Error! CSV file may be incorrectly formatted or there was an issue in reading the file. Please try again.';
    } else {
        move_uploaded_file($dir, $file_location);

        $id = $_POST['val'];
        $type = $_POST['type'];

        $table = ($type == 1 ? 'vendor_module_type_one' : 'vendor_module_type_two');

        $handle = fopen($file_location, 'r');

        $parts = array();
        $components = array();

        $i = 0;

        while (($data = fgetcsv($handle, 1000, ',')) !== false)
        {
            if (is_array($data)) {
                if (empty($data[0])) {
                    echo 'Error! Reference can\'t be empty. Please ensure all rows are using a ref no.';
                    $error = true;

                    continue;
                }

                # get data
                $get_for_sql = 'SELECT `id` FROM `'. $wpdb->prefix. $table .'` WHERE `ref` = %s';
                $get_for_res = $wpdb->get_results($wpdb->prepare($get_for_sql, array($data[0])));

                if (count($get_for_res) <= 0) {
                    echo 'Error! Reference has no match. Please ensure the CSV is using the correct ref no.';
                    $error = true;

                    exit();
                }

                $get_for_id = $get_for_res[0]->id;

                # create data arrays
                $parts[$i]['name'] = $data[1];
                $parts[$i]['ref'] = $data[2];
                $parts[$i]['for'] = $get_for_id;

                $components[$i]['part_ref'] = $data[2];
                $components[$i]['component_ref'] = $data[3];
                $components[$i]['sku'] = $data[4];
                $components[$i]['desc'] = utf8_decode($data[5]);
                $components[$i]['req'] = $data[6];
                $components[$i]['price'] = $data[7];

                unset($get_for_id);
                unset($get_for_res);
                unset($get_for_sql);

                $i++;
            }
        }

        fclose($handle);
        unlink($file_location);

        # get unique parts only
        $parts = array_unique($parts, SORT_REGULAR);

        # check to see if part already exists, if so delete
        $search_field = ($type == 1 ? 'id_field_one' : 'id_field_two');

        $check_sql = 'SELECT `id` FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
        $delete_parts_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
        $delete_components_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_components` WHERE `part_id` = %d';

        $check_res = $wpdb->get_results($wpdb->prepare($check_sql, array($id)));

        if ($check_res) {
            $wpdb->query($wpdb->prepare($delete_parts_sql, array($id)));
        }

        $part_ids = $admin_functions->insert_parts($parts, $type);

        unset($parts);
        unset($delete_parts_sql);
        unset($search_field);
        unset($check_sql);
        unset($check_res);
        unset($i);

        # should never be empty, but just as a precaution ...
        if (!empty($part_ids)) {
            foreach ($components as $key => $component)
            {
                $components[$key]['part_id'] = $part_ids[$component['part_ref']];
            }

            # rm components from assoc part id
            foreach ($part_ids as $id)
            {
                $wpdb->query($wpdb->prepare($delete_components_sql, array($id)));
            }

            # insert components
            $admin_functions->insert_components($components);
        } else {
            echo 'Error!';
        }

        echo (!$error ? 'Success! File Successfully Imported.' : 'There be something wrong with the import. Please try again.');
    }

it's triggered through a button press and uses AJAX to handle it etc.

I'm not sure why a memory leak is occurring or why WordPress doesn't offer more useful error messages. I don't call that function.. so I'm guessing it's something WordPress is doing in the background when things are run.

My info:

PHP 7.2.10
Apache 2.4
Linux Mint 19

Also happens on my server:

PHP 7.1.25
Apache 2.4
CentOS 7.6.1810

WordPress running version: 4.9.8

  • 1
    Why are you using a custom php page and not defining your own AJAX hook or (even better) custom REST endpoint? If the csv is too large, can't you split this operation into multiple operations? – kero Dec 11 '18 at 16:10
  • @kero what do you mean? :) All I've done is in my script.js file added an ajax function to a .click which points to this page ... never really done WP development - more of a Symfony/Magento dev - used to this way, so just assumed it would be ok haha – treyBake Dec 11 '18 at 16:11
  • 2
    You require some WP files. This is not the WordPress way, the documentation describes how to do AJAX or using the more modern REST API. I haven't looked too deep in your code but the thing is: you break WP flow by including some files, so it is possible that default WP behaviour is broken. Instead, if you do it the WP way, you can be sure, that things like shutdown handlers work as intended – kero Dec 11 '18 at 16:16
  • 1
    @treyBake Create a custom REST endpoint - it is really easy. How is this a backwards way? You wouldn't go create a custom file and require 'vendor/symfony/process/InputStream.php';, but create a class, make use auf autoloading, etc. -> you would use symfony architecture. And here, you should use WP architecture – kero Dec 11 '18 at 16:24
  • 1
    A WP CLI command would have been a much better way of doing this, avoiding both the memory and the PHP time execution limits. Also, WP Admin AJAX is an ancient API, use a REST API endpoint as kero suggests. Also consider that your CSV may be too large to fit in the available memory, there won't be a config option you can change to just fix it unless you start cleaning up variables after you're finished with them, or you modify php.ini to give the script more memory – Tom J Nowell Dec 11 '18 at 16:33
1

Depending on what exactly you're trying to achieve, I agree with Tom's comment, that a WP-CLI command might be better. The advantage is that the command runs from php directly on the server (usually has no max execution time, loads different php.ini, etc.) and you don't need to involve the webserver.


If that is not possible, the next best way is probably to create a custom REST endpoint. WordPress has a class WP_REST_Controller, usually I write classes that extend this and work from there. For simplicity the following example is not using inheritence, but I try to keep to the same lingo.

1. Register new route

New/custom routes are registered via register_rest_route() like so

$version = 1;
$namespace = sprintf('acme/v%u', $version);
$base = '/import';

\register_rest_route(
    $namespace,
    $base,
    [
        [
            'methods' => \WP_REST_Server::CREATABLE,
                         // equals ['POST','PUT','PATCH']

            'callback' => [$this, 'import_csv'],
            'permission_callback' => [$this, 'get_import_permissions_check'],
            'args' => [],
            // used for OPTIONS calls, left out for simplicity's sake
        ],
    ]
);

This will create a new route that you can call via

http://www.example.com/wp-json/acme/v1/import/
   default REST start-^       ^       ^
       namespace with version-|       |-base

2. Define permissions check

Maybe you need authentication? Use nonces?

public function get_import_permissions_check($request)
{
    //TODO: implement
    return true;
}

3. Create your actual endpoint callback

The method previously defined gets passed a WP_REST_Request object, use that to access request body, etc. To stay consistent, it is usually best to return a WP_REST_Response instead of custom printing of JSON or similar.

public function import_csv($request)
{
    $data = [];
    // do stuff
    return new \WP_REST_Response($data, 200);
}

If you do all of this in OOP style, you'll get the following class

class Import_CSV
{

    /**
     * register routes for this controller
     */
    public function register_routes()
    {
        $version = 1;
        $namespace = sprintf('acme/v%u', $version);
        $base = '/import';

        \register_rest_route(
            $namespace,
            $base,
            [
                [
                    'methods' => \WP_REST_Server::CREATABLE,
                    'callback' => [$this, 'import_csv'],
                    'permission_callback' => [$this, 'get_import_permissions_check'],
                    'args' => [],
                ],
            ]
        );
    }

    /**
     * endpoint for POST|PUT|PATCH /acme/v1/import
     */
    public function import_csv($request)
    {
        $data = [];
        // do stuff
        return new \WP_REST_Response($data, 200);
    }

    /**
     * check if user is permitted to access the import route
     */
    public function get_import_permissions_check($request)
    {
        //TODO: implement
        return true;
    }

}

But .. still 404? Yes, simply defining the class sadly doesn't work (no autoloading by default :( ), so we need to run register_routes() like so (in your plugin file)

require_once 'Import_CSV.php';
add_action('rest_api_init', function(){
    $import_csv = new \Import_CSV;
    $import_csv->register_routes();
});
  • this is clearer than the docs - ty :) one last question - how does my JS call it? :) E.g. I essentially have a form that allows user to import + select a product for that import (so it passes $_FILES and an $id) - where does the JS fit in for this? – treyBake Dec 11 '18 at 17:28
  • If you hardcode namespace & base in the REST controller, just call that from JS. If not, you could use wp_localize_script() to inject the REST url into a global namespace and use that. And then you work inside import_csv() as you do in your custom script right now – kero Dec 11 '18 at 17:31
  • I fixed everything to point to a REST Endpoint and it still memory leaks :/ – treyBake Dec 12 '18 at 10:34
  • 1
    I solved it and marked your solution as accepted. I don't think my AJAX implementation was incorrect, or at least, the cause of the error. Now I understand it, I definitely prefer this methodology - hence my acceptance of your answer. However, I installed xdebug and debugged my function - turns out my fputcsv was erroring because $handle was a boolean and not opening the import file correctly. This was due to the uploads/import folder not existing on the server. Though, not sure why WordPress errors with a memory leak, to me it should return the actual error.. so WP still sucks ;) – treyBake Dec 12 '18 at 16:50
  • 1
    hahaha that's fair enough ;) I think WP can be fun, but done my way. I refuse to use HTML within the function declarations.. I will always opt to build html and echo the string haha xD but again thank you! wouldn't have done it without you :) – treyBake Dec 12 '18 at 16:57
0

In your functions file do something like this:

add_action('wp_ajax_import_parts_abc', 'import_parts_abc');
function import_parts_abc() {
    include_once('/admin/inc/scripts/import-parts.php');
    exit();

}

Then in your js file something like this:

jQuery(document).ready(function() {
     jQuery('.my-button').click(function() {
jQuery.ajax({  
            type: 'GET',  
            url: ajaxurl,  
            data: { 
                action : 'import_parts_abc'
            },  
            success: function(textStatus){
                alert('processing complete')
            },
            error: function(MLHttpRequest, textStatus, errorThrown){  
                console.log(errorThrown);  
            }  
      }); 
});

});
  • This fixes the security issue, but doesn't address the problem the OP asked about. Also a REST API endpoint + fetch() would have been better – Tom J Nowell Dec 11 '18 at 16:39
  • It should solve his error problem which is what the OP asked about? I would agree the REST API is slightly faster (15-25%), but that doesn't answer the OP's question regarding AJAX. – Tim Hallman Dec 11 '18 at 16:42
  • The users problem is they're running out of memory, AJAX was just us in the comments realising the OP was calling a PHP file directly and loading in WP – Tom J Nowell Dec 11 '18 at 20:51
  • The memory leak was a result of improperly implementing AJAX. – Tim Hallman Dec 11 '18 at 22:50
  • 1
    @TimHallman I solved it! Using kero's answer and xdebug, I found the issue - $handle was false as uploads/import wasn't a valid folder path. Not sure why WordPress error the way it does with a memory leak instead of a fputcsv expects param 2 to be an object error... maybe some WP issues behind-the-scenes with handling errors like that :s – treyBake Dec 12 '18 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.