1

I got a "theorical" question here. I made a Wordpress plugin that scrapes content from remote website. It as a Scraper_Class with generic "methods" to scrape remote content. eg.

Scraper_Class {
    var $title;
    var $url;
    function fetch_content(){
        $this->title = $this->get_remote_title();
    }
    function get_remote_title(){
        //...
        return $title;
    }
}

But I also want to have "presets" that would make evolve that Scraper_Class depending on the input URL (automatically set selectors, etc.)

I have a doubt about how doing this.

extend Scraper_Class ?

My first idea was to create several ..._Preset_Class that would extend the main Scraper_Class.

eg.

Website_Preset_Class extends Scraper_Class {
    static function can_load_url($input_url){
        //...
        return true;
    }
    function get_remote_title(){
        //...
        return $title;
    }
}

When creating the scraper; it would loop through all the Preset class available to use the one that matches the input URL.

function get_scraper{
    $scraper = new Scraper_Class();
    foreach($presets as $preset){
        if ( $preset::can_load_url($url) ){
            $scraper = new preset();
            break; //we've got our preset
        }
    }
}

use action/filter hooks ?

OR

I also had the idea of using "temporary" filters and actions: they should only run once; so they would be removed once at the end of the fetch_content() function.

Scraper_Class {
    var $url;
    var $title;
    var $filters;

    function fetch_content(){
        $this->title = $this->get_remote_title();
        //...
        $this->remove_temporary_filters(); //<-- HERE
    }

    function get_remote_title(){
        //...
        return apply_filters('get_remote_title',$title);
    }
    function add_temporary_filter($tag,$function_to_add,$priority = 10,$accepted_args = 1){
        $this->filters = array($tag,function_to_add,priority);
        add_filter($tag,function_to_add,priority);
    }
    function remove_temporary_filters(){
        foreach((array)$this->filters  as $filter){
            remove_filter($filter[0],$filter[1],$filter[2]);
        }

    }
}

Website_Preset_Class { //not extending Scraper_Class
    function __construct($scraper){
        if ( self::can_load_url($scraper->url) ){
            $scraper->add_temporary_filter('get_remote_title',array($this,'get_preset_title');
        }
    }
    static function can_load_url($input_url){
        //...
        return true;
    }
    function get_preset_title(){
        //...
        return $title;
    }
}

I'm quite of struggling with this.

I have the feeling that both ways of doing this are quite a mess...

Second way (filters) is maybe great (I could stack several presets) but is maybe quite "hacky" while first way (extending the class) is perhaps more clean but also more limited.

Do you understand my issue ? What would you do ?

Thanks !

  • use a single class. add a table for your plugin with preset data that will be created when the plugin is installed. in the constructor, check the database and load preset data if a match is found. – inarilo Nov 24 '17 at 11:50
  • Not a bad idea, but A also needs specific method for some presets (how to fetch a title for examples). It's not only a question of storing selectors/datas, so this can't work... – gordie Nov 25 '17 at 11:55
  • once the preset is selected, it can be stored in a member variable, and that can determine how to return the title, if there is a fixed number of ways. or a property of the preset can specify the method to use. but if you want it to be customisable down the line by others, then you'll need subclasses. – inarilo Dec 2 '17 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.