0

I'm trying to run WordPress automated tests as described in this article. Predictably, I've been getting a lot of skipped tests marked S, because, according to the article, "this usually means that the test is linked to a bug report, and that bug report remains unfixed." I'd like to link tests to bug reports, too, but I'm having difficulty finding information about how to do that. How can I link a test to a bug report? Can I do that with GitHub issues, too?

Edit: Ok, I found this in the same article: "Many tests are marked with a @ticket annotation, which indicates they were the result of that WordPress Trac ticket. The test is skipped if the ticket is still open – in these cases, we treat the test as a “known bug” that will likely fail."

And I found tests/phpunit/includes/testcase.php -> knownWPBug( $ticket_id ), but I'd love to know if there's a library for GitHub tickets like the class TracTickets so that I can poll my GitHub repo to see if an issue is open or not.

  • I think those tests have been manually marked for skipping by someone who was in said bug report, there's no automatic linkage involved – Tom J Nowell Feb 24 '15 at 18:07
1

I don't know of a library, but you could probably build one using TracTickets as a guide and using GitHub's API.

I'm actually answering to say don't do this. WordPress actually no longer uses this method of skipping tests. See ticket #30284. I quote from there:

Our previous convention was to write unit tests to demonstrate bugs, annotate those tests with a @ticket flag, and commit them to the repo. Then, when running the suite, WP_UnitTestCase::knownWPBug() would get a list of open tickets from Trac, and skip tests against open tickets, allowing the build to pass.

But:

The new convention is to commit tests only when fixing the bug.

I recommend you consider the reasons that WordPress has abandoned this method (which are outlined in that ticket), and I think you'll probably conclude that you don't want to do this. Instead, the tests should committed along with the fix. You don't want failing tests, what you want is fixed bugs.

But this leaves dozens of failed tests in the repo. These tests should be pulled out, converted to patches, and uploaded to their relevant Trac ticket with a note indicating what's just happened. Once all the failed tests are out of the repo, we can remove the knownWPBug behavior from the test runner.

This has been done, all but removing knownWPBug(). It was probably decided to leave it in for backward compatibility.

The reason you are still seeing a lot of skipped tests is that some tests require multisite to be on (or off), and other require PHP to be compiled with certain extensions disabled. There are other reasons that tests are skipped too.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.