Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

I have a client who wants to be able to upload Posts using information stored in a CSV file. I've already got this portion working.

Just recently, I changed the CSV File Uploader from pure PHP to an AJAX Uploader in an effort to eliminate server timeouts as well as provide a nice interface to keep track of each new post's progress.

The Problem:

Each custom post contains about 130 Custom Fields total. Each one of my CSV Upload Tests contained roughly 20 new Posts. Even with async left on, it still takes a bit for each new Post to be processed.

All of the Custom Fields are broken up into 5 separate Meta Boxes. Originally, I had it set up to store each Meta Box as a JSON Encoded string, and save that as part of the post's metadata. This brought the total number of Custom Fields from 130 to 5, and write time was increased dramatically.

The drawback to this, however, is that each individual Custom Field is no longer searchable by Wordpress' standards.

Proposed Solutions:

I can do one of a few things:

  1. I can leave it as-is and take the slower write times while retaining both searchable and queryable Metadata, but perhaps run into issues with CSV files that contain a few hundred or - God forbid - a few thousand Posts.
  2. I can store each Meta Box as a single Custom Field and modify Wordpress' search functionality to find Metadata via custom SQL, but lose the ability to retrieve Posts via meta_query.
  3. I can overhaul the existing structure and write all Custom Post Metadata to a separate table. This means losing the built-in search functionality, as well as the ability to use meta_query.

The Question:

What would be the best approach to handling this? Theoretically, there are going to be hundreds of thousands of these Custom Posts. Giving users the ability to search for these posts both by title, taxonomy, and metadata is required.

Personally, I try to avoid writing any new tables to a Wordpress Installation at all costs. But would this be considered a situation that demands it? Or would I lose too much functionality by using my own less-sophisticated queries to fix a problem that might not even necessarily BE a problem?

Most of all, is there any other way that I might be missing that can solve this issue?

UPDATE:
In an effort to clarify what the data entails without violating my NDA, each Custom Post is a Guitar.

Each Meta Box signifies a Section of a Guitar (Neck, Body, etc.), with each Custom Field being a fixed piece of information attributed to that particular section (Wood, Manufacturer, etc.).

class Guitar
{
    public $ID;
    public $Year;
    public $Make;
    public $Model;

    public $Section1;
    public $Section2;
    public $Section3;
    public $Section4;
    public $Section5;
    public $Section6;
    public $Section7;
    public $General;

    private $_data;
    private static $_dataFlags = array('Section1', 'Section2', 'Section3', 'Section4', 'Section5', 'Section6', 'Section7', 'General');
    public function __construct($args)
    {
        $this->ID = $this->_query_existing();
        foreach(self::$_dataFlags as $sec)
            $this->$sec = new $sec($args[$sec]);
    }
}
abstract class Guitar_Section
{
    public function __construct($arr=array())
    {
            if(!empty($arr))
            $this->_set_props($arr); //Set Properties
    }
    public function update_meta($id, $useprops=false)
    {
        foreach(get_object_vars($this) as $k=>$v)
        {
            $this->$k = !$useprops ? $_POST['guitar_'.$this->_get_section_name().'_'.$k] : $v;
            update_post_meta($id, '_guitar_'.$this->_get_section_name().'_'.$k, $this->$k);
        }
    }
}
class Section1 extends Guitar_Section
{
    public $Wood;
    public $Attachment;
    public $etc;

    protected function _get_section_name()
    {
        return 'Section1';
    }
}

This displays how each Class works in tandem with each other. I have it so that each Guitar Section is encapsulated by the Guitar Class, with each section extending the abstract class. Each property is a Custom Field, and is set utilizing the Methods within the Abstract Class, while the Guitar Class essentially queries Wordpress and sets any available ID, Year, Make, and Model.

Most of this requires a LOT of string manipulation due to the fact that CSV files are used to set the class and its properties.

share|improve this question
3  
I cringed at this: "Each custom post contains about 130 Custom Fields total." The first thing I would do would be to look at your data schema, to see if there are any opportunities to make your data more efficient. Can you edit your question to include a description of your schema? –  Chip Bennett Jan 14 '13 at 20:20
1  
Also, as-written, your question really falls under "not constructive", because you don't reference your specific data, and you're asking an open-ended question that solicits discussion and opinion, rather than an explicit solution. –  Chip Bennett Jan 14 '13 at 20:22
1  
@ChipBennett I agree. I cringed about the amount of data as well. Problem is, I can't go too much into detail about actual Data (dude is pretty serious about his NDAs). So while I do absolutely understand the Non-Constructive aspect of this question, the most I can do is give an idea of what the Custom Fields themselves actually entail. Would this be acceptable? Or too vague to salvage? –  maiorano84 Jan 14 '13 at 20:27
1  
Explaining generic data structure shouldn't violate anything. We don't need the actual data. Pretend you are building X-Wing Fighters or something. If you can do that, maybe its salvageable. I can already tell that I am going to suggest you make your own tables though. :) –  s_ha_dum Jan 14 '13 at 20:46
    
@s_ha_dum Ok, I think I have enough for a general idea on how all this is structured. Let me know if the update is enough for everybody to work with. –  maiorano84 Jan 14 '13 at 21:05

1 Answer 1

I hope this 2 cents isn't too much of a tangent but couldn't / shouldn't some of these be taxonomies? I'm not sure if that helps A LOT but it would seem to me what some of the attributes have a fairly fixed set of value, and that being able to click on one of those would be something desirable.

Again, pardon me for not attacking the question directly but perhaps there's some value in these 2 cents I just shared :)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.