Good methodology for upgrading a large/complex sites?

Question

Imagine you have a large wp site that is on 3.1 and uses like all features wordpress has (lots of CPTs, is a network-install, has lots of pages/posts with different post-templates, has child themes for the blogs in the network) and also has +30 plugins installed - which are of course all outdated.

You now need to upgrade to the current version. The upgrade task itself is done within 5 minutes, but how do you know everything is working afterwards?

Basically you haven't developed the whole stuff by yourself and are just in charge of performing the upgrade.

Clearly it will take you weeks to visit every post of the site and to try every little nifty backend feature to make sure it still works.

So what I'm asking is: Can anyone describe his methodology of performing this kind of tasks? Are there any tools that can aid in this process?

I'm looking for general advises or something like a step by step workflow like:

• let tool x index the whole site,
• perform the upgrade,
• run wp core unit tests,
• let tool x index the site again and diff the whole html output for unwanted changes,
• ....

Answer 1

I don't know of a great before and after comparitor that would get you the results you need. In fact I would be worried that any tool like that won't catch any failures in functionality.

Honestly I would suggest incrementally building up a testing suite that goes through some of your standard processes and verifies the results and then just run those scripts every time you do an update. Such as:

1. Create new page
   • add text
   • add image
   • add link
   • publish
   • verify text change
   • verify image
   • verify link

Do the same thing for new post, then do all of the above plus verifying that you can edit text and links on previously existing posts and pages.

Selenium is a decent tool. You can also build out the automated tests in Python or whatever language you love best.