external Integration with wordpress timeout error

Question

I have developed a python script that sends information to a wordpress site.

Inside the wordpress site I have a plugin that receives the data and processes it. It takes somewhere between 30 minutes and a couple of hours to process all the data, which includes updating the catalog of a woocommerce store.

The plugin stops from time to time, without any error, completing without problems some of the times. The process has completed while taking an hour, and at other times it just stops after a few minutes.

I have talked to the host, nothing can be found on the error log of the apache, and I have a log file generated by the plugin that saves all processing that it is making, along with notices, warnings and errors from PHP, and it just stops writing to the logs with no apparent reason.

I have set_time_limit and ignore_user_abort in php, but it seems that the python script sometimes loses connection with the webserver, and after that sometimes the php script keeps running, sometimes it stops.

I have thought that this could be some sort of connection broken due to no output being sent to the browser. I have done several tests with ob_flush and flush just to see if I can get content to my browser chunks at a time to avoid connection from being lost.

My #1 question is how can I turn of wordpress and / or woocommerce buffers, output compression and whatever, so I can get output sent chunks at the time to the browser? I now have a while with an echo and a sleep inside it, with ob_flush and flush, but the content is only printed after the while.

My #2 question is asking for opinions as to what might be causing this, as another person might have faced the same problem.

Thank you for any help.

Answer 1

Just in case anyone else stumbles upon the same problem, here is the most robust disabling I could think of, and it seems to work as expected:

        ini_set('max_execution_time', 0);
        set_time_limit(0);

        ignore_user_abort(true);

        ini_set('output_buffering', 'off');
        ini_set('zlib.output_compression', false); 
        ini_set('implicit_flush', true);
        ob_implicit_flush(true);

        while (ob_get_level()) {
            ob_end_flush();
        }
        flush();