Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

I have a custom CMS, that we're looking to port over into Wordpress. (I can see you asking yourself, why not just stick with the custom CMS? Well, it's outdated, clunky, and getting to be too much of a hassle to maintain; there are also many features of WP that we would really like to take advantage of).

So here's the issue -- We have a main type of content (let's call them Tours). Each Tour has about 30 pieces of meta data associated with it (all currently different columns on the Tours table). We currently have about 600,000 Tours.

Each Tour has several different types of images associated to it, from Interior Images, to Exterior, to Floor Plans, etc. Each Image has about 10 pieces of meta data associated with it (again, all currently columns on the Images table). We currently have about 7 million Images in our system.

You can likely see our problem -- if we put all Tours and Images into the default wp_posts and wp_postmeta table, we're going to end up with millions of rows in wp_posts, and far more than that in the wp_postmeta table. I'm not sure how much mySQL can handle as far as numbers of rows, but it seems likely that we will reach a slowdown point, especially when querying the wp_postmeta table.

My questions:

1) One possibility we've discussed is only holding "active" Tours in the wp_posts and archiving older Tours to a separate table. Is anyone aware of any plugins that do anything like this?

2) Another possibility is putting the Tours into the wp_posts table, and keeping the Images separate from that. One thing we would really like to use, though, is the Wordpress image uploader. Is there anyway to override where the image data is saved?

3) Any other suggestions of how we might best implement this much data into Wordpress?

share|improve this question
1  
Be careful of the exponential number of files that will be created by importing images into WP. Plugins/themes register new image sizes and each image size gets its own resize/crop file in the file system. 7 Million images X 4 default WP sizes is quite mind boggling to say the least. –  Brian Fegter Sep 16 '12 at 4:36
add comment

2 Answers 2

Give a look to http://podsframework.org/. It's a custom post type enhancer that allows you even to create a stand alone table for every kind of object you need.

share|improve this answer
add comment

I would suggest inserting your tours as posts (or custom posts) and using a custom reference table with the id matching the post id of the Tour post.

You could use Wordpress's built in meta box functions to be able to edit each Tour in the edit posts page.

You could update your reference table by hooking into the 'save_post' action.

There will be a considerable look-up speed improvement if you choose to create this reference table as MYISAM and use fixed column widths along with indexing searchable columns.

With that much data, you could use this option to not only tap in to much of the great Wordpress built-in functionality, but ensure speedy look-ups on the extra data.

share|improve this answer
    
So to make sure I understand what you're saying, you're suggesting putting the Tours in the wp_posts table, and then storing all the data that would normally go into wp_postmeta into its own separate table? That's definitely doable, and would make data retrievable more manageable (and more searchable). What are your thoughts on the Images? They're the bulk of the data, actually, and putting each of them in as an attachment to the Tour is really going to bulk up the wp_posts table. –  user19856 Sep 5 '12 at 23:34
    
I would suggest storing them on the server in numbered file folders referring to the post id of the 'Tour'. Then you would only need to store their file name in the reference table. That way you can still keep the look-ups fast. Hope that helps (Its the way I handle large sets of data myself with Wordpress) –  Douglas.Sesar Sep 6 '12 at 0:42
    
Also, if you need some guidance on some of the functions involved, I could help with the specifics. –  Douglas.Sesar Sep 6 '12 at 0:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.