Allowed memory size of 33554432 bytes exhausted (tried to allocate 9967617 bytes) in ...\wp-includes\functions.php

Question

When a script times out, and there is nothing you can do about it ( cause it's running on a bloddy shared hosted resource where you have 0 control over the script time-outs ), can you run register shutdown or something to that effect?

You may ask what on earth does this question have to with wordpress...

Well, it kind of does.

in my migration routine, I process a 10,000 record table and do some heavy duty parsing. I have to do select * cause I need everything there. The page eventually times out. Next time I run the same page I get a Allowed memory size of 33554432 bytes exhausted (tried to allocate 9967617 bytes) in ...\wp-includes\functions.php on line 3335

No plug ins involved, no settings changed. I can generate this error whenever I want. So don't say check your last actions & plug ins and all that pls. This is on a vanilla version, the one you get after the 5 min install.

Well, my first instinct was to run the mysql_free_result over that RS!

but the question is at what point? the perfect point for this would be just before the time out occurs. Thus the need for shutdown function... but the question is how do I make it in there?

Answer 1

I faced with the same issue when I was writing my backup plugin. The bug appear when you fetch all data from db by calling $wpdb->get_results( ... ). And as you can suppose it fetch all data into memory allocated for php. The better approach is to use mysql_* functions to fetch row one by one and store data into temporary file. This approach will reduce memory usage tremendously.

I would recommend you to do it something like this:

$result = @mysql_query( sprintf( 'SELECT * FROM `%s`.`my_big_table`', $wpdb->dbname ), $wpdb->dbh );
if ( $result ) {
    while ( ( $row = @mysql_fetch_array( $result, MYSQL_NUM ) ) ) {
        // save each row to temp file
        fwrite( $handle, prepare_row_to_save_into_file( $row ) );
    }
    @mysql_free_result( $result );
}