Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

I am attempting to update a large array of feeds into WordPress, using fetch_feeds. As can be expected, when going over 50 feeds the server is starting to time out. I would still like to update the feeds in bulk but without crashing the server.

Can you suggest a method through which I can break things down? Maybe using AJAX? Is that the best technique? Any examples I can look at?

share|improve this question
2  
What have you tried so far? –  kaiser Mar 19 '13 at 8:22

3 Answers 3

You can try using wp cron functions, programing a feed import every 10 minutes or so, until you have nothing left in queue.

Example code:

<?php
register_activation_hook( __FILE__, 'wp1903_schedule_cron' );

function wp1903_schedule_cron() {
    $counter = 0;
    $feeds = array(
        'http://example.com/feed1',
        'http://example.com/feed2',
        'http://example.com/feed3',
        'http://example.com/feed4'
    );

    foreach ($feeds as $feed) {
        $counter++;
        // schedule every 10 mins
        $time = time() + (600 * $counter);
        wp_schedule_single_event( $time, 'wp1903_process_feed', $feed );
    }
}

add_action('wp1903_process_feed', 'wp1903_import_feed');

function wp1903_import_feed($feed_url = '') {
    // do the import
}

It is essential only to run the scheduler on a activation hook, or else at every page load the crons will be rescheduled.

share|improve this answer
    
Sounds good but I need it to run on demand, rather than just on activation. –  drtanz Apr 18 '13 at 12:04

You've tried to extend the script's maximum execution time with set_time_limit()? This will not cause the server to crash (because it has a memory limit too):

set_time_limit( 0 ); // No time limit is imposed

I'm not sure what you mean with "breaking down" the script, you might want to have a second thought on that.

What you could do if there are to many rows in the feed's XML you also could make your own import script and take care of its handle your own way by processing small part by small part.

$xml_handler = new ProductsParser();
$xml_parser  = xml_parser_create();

$source = '--URL--';

xml_set_object( $xml_parser, $xml_handler );
xml_parser_set_option( $xml_parser, XML_OPTION_CASE_FOLDING, false );
xml_set_element_handler( $xml_parser, 'startElement', 'endElement' );
xml_set_character_data_handler( $xml_parser, 'cdata' );

$fp = fopen ( $source, 'r' );
while ( $chunk = fread( $fp, 4096 ) ) {
  xml_parse( $xml_parser, $chunk, feof( $fp ) );
  flush();
}

fclose( $fp );

class ProductsParser {
  public $product; # Holds the record values
  public $elem_key; # Holds the current element key while reading
  public $record; # Holds the record tag

  function __construct( $args ) {
    $this->product  = false;
    $this->elem_key = false;
    $this->record   = '--RECORD-NAME--';
  }

  function startElement( $parser, $tag, $attributes ) {
    if ( $this->record == $tag && ! is_array( $this->product ) ) {
      $this->product = array();
    } elseif( is_array( $this->product ) ) {
      $this->elem_key = $tag;
    }
  }

  function endElement( $parser, $tag ) {
    if ( $this->record == $tag && is_array( $this->product ) ) {
      // Process the product row

      $this->product = false;
    } elseif ( is_array( $this->product ) && $this->elem_key != false ) {
      $this->elem_key = false;
    }
  }

  function cdata( $parser, $cdata ) {
    if ( is_array( $this->product ) && $this->elem_key != false ) {
      $this->product[$this->elem_key] = $cdata;
    }
  }
}

More information about the XML parser can be found in the PHP manual.

share|improve this answer

When it comes down to high execution time scripts I usually split it up as simple as possible using ajax so here is a concept class that should get you started with creating an admin page and ajaxed import loop:

<?php
class Ajax_Import{

    public function __construct(){
        if(is_admin()){
            add_action('admin_menu', array($this, 'add_plugin_page'));
        }
        add_action('wp_ajax_do_feed_import',array($this,'do_import'));
    }

    public function add_plugin_page(){
        // This page will be under "Settings"
        add_options_page('ajaxed Import', 'ajaxed Import', 'manage_options', 'test-setting-admin', array($this, 'create_admin_page'));
    }

    public function create_admin_page(){
        wp_enqueue_script( 'jquery');
        ?>
        <div class="wrap">
            <?php screen_icon(); ?>
            <h2><?php _e('AJAXed Import'); ?></h2>          
            <table class="form-table">
                <tbody>
                    <th scope="row">
                        <label for="my-text-field"><?php _e('Feeds To Import'); ?></label>
                    </th>
                    <td>
                        <textarea id="feeds"></textarea>
                        <br>
                        <span class="description"><?php _e('Enter Each feed in a new line.'); ?></span>
                    </td>
                </tbody>
            </table>
            <p class="submit"><input type="submit" id="do_import" value="Save Changes" class="button-primary" name="Submit"></p>
            <div id="import_status" style="display:none"></div>
        </div>
        <?php
        $this->import_loop_js();
    }

    public function do_import(){
        //do your import here ex:
        $rss = fetch_feed(esc_url( $_POST['feed_url']));
        if (!is_wp_error( $rss ) ) {
            $count = $rss->get_item_quantity(); 
            //do actuall import loop or whatever
            echo json_encode(array('message' => '<div id="message" class="success"><p>imported ' . $count . ' items from '.esc_url( $_POST['feed_url']).'</p></div>'));
        }else{
            echo json_encode(array('errors' => '<div id="message" class="error"><p>' . $rss->get_error_message() . '</p></div>'));
        }
        die();
    }

    public function import_loop_js(){
        ?>
        <script type="text/javascript">
        jQuery(document).ready(function($){
            var ajax_import = (function(){
                var instantiated;
                var feeds;
                function init (){
                    return {
                        start_import: function(){

                            instantiated.feeds = $("#feeds").val().split('\n');
                            instantiated.update_status('', true);
                            instantiated.next_feed();
                        },
                        make_ajax_call:function(feed){
                            data = {
                                action: "do_feed_import",
                                feed_url: feed
                            };
                            $.post( 
                                ajaxurl,
                                data, 
                                function(res){
                                    if (res.error){//add error message
                                        instantiated.update_status(res.error);
                                    }else{//add success message
                                        instantiated.update_status(res.message);
                                    }
                                    //call next feed
                                    instantiated.next_feed();
                                }, 
                                'json'
                            );
                        },
                        next_feed:function(){
                            if (instantiated.feeds.length > 0) {//if we have feeds in the list starts ajax calls
                                instantiated.make_ajax_call(instantiated.feeds.shift());
                            } else{
                                instantiated.update_status("all Done!!!");
                                $("#do_import").removeAttr("disabled");
                            }
                        },
                        update_status:function(m,clear){
                            clear = clear || false;
                            if (clear)
                                $("#import_status").html('').show();
                            else
                                $("#import_status").append(m);
                        }
                    }
                }

                return {
                    getInstance :function(){
                        if (!instantiated){
                            instantiated = init();
                        }
                        return instantiated; 
                    }
                }
            })();
            $("#do_import").on('click',function(){
                $(this).attr('disabled','disabled');
                ajax_import.getInstance().start_import();
            });
        });
        </script>
        <?php

    }
}

$Ajax_Import = new Ajax_Import();
share|improve this answer
    
Thanks, could you suggest a plugin which implements something like this so I can see it working? –  drtanz Apr 18 '13 at 12:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.