Your WordPress tests are leaking state

WordPress was never designed to be run this way.

A normal WordPress request has a very convenient cleanup mechanism: PHP exits.

WordPress boots. Plugins register post types, taxonomies, metadata, rewrite rules, and everything else they need. The request is handled. Then the process ends and all of that in-memory state disappears with it.

A test suite breaks that assumption.

We boot WordPress once and then run hundreds or thousands of tests against the same process. Every test is supposed to behave as if it starts from a known state, but much of WordPress was designed with the assumption that the process itself would provide that isolation.

The database is the obvious state, and WordPress's test suite handles it well.

The runtime is harder.

That's how you end up with one of my least favorite classes of test failure:

The test passes by itself, but fails when I run the whole suite.

There is a worse version, too:

The test passes in the suite, but fails by itself.

The first one wastes your time. The second one should worry you. This leaves us with tests conflicting with each other and pleanty of setUp()/tearDown() cleanup madness.

It means your test is green because another test happened to prepare part of its environment for it. That isn't a well written test (but it probably isn't your fault).

The database is clean. WordPress isn't.

Consider a fairly unremarkable test:

 1public function test_something_with_books(): void {
 2	register_post_type(
 3		'book',
 4		[
 5			'public'   => true,
 6			'supports' => [ 'title', 'editor' ],
 7		]
 8	);
 9
10	$this->assertTrue( post_type_exists( 'book' ) );
11
12	// Test something interesting...
13}

register_post_type() doesn't write a row to the database. It changes WordPress's in-memory state.

Specifically, book now exists in the global post type registry. (Using globals for everything was a great idea, right?!)

So a later test can do this:

1public function test_book_is_not_registered(): void {
2	$this->assertFalse( post_type_exists( 'book' ) );
3}

Run this test alone and it passes.

Run it after the first test and it fails.

Nothing flaky happened. Both tests behaved perfectly consistently.

They just weren't isolated. And post types are only one example.

WordPress keeps a surprising amount of its runtime configuration in process-wide state: post types, taxonomies, post statuses, registered meta, rewrite state, public query variables, sitemap state, post type features, roles, capabilities, and more.

That architecture makes sense for WordPress's normal lifecycle. Registration generally happens once during bootstrap and the process disappears shortly afterward.

It becomes much more interesting when the process sticks around for another 2,000 tests.

WordPress core knows about this problem

There is an important detail in the WordPress test framework that I missed for a long time.

WordPress already has methods for resetting post types, taxonomies, and post statuses between tests.

But look at when it uses them:

1if ( defined( 'WP_RUN_CORE_TESTS' ) && WP_RUN_CORE_TESTS ) {
2	$this->reset_post_types();
3	$this->reset_taxonomies();
4	$this->reset_post_statuses();
5
6	// ...
7}

Those resets happen when testing WordPress core. For non-core tests, WordPress deliberately skips them.

And the reason is completely reasonable: plugins register post types and taxonomies during init.

Imagine booting your plugin, letting it register its book post type, and then having the testing framework blindly reset the post type registry before your first assertion.

Your test would be isolated.

It would also be testing an environment that no longer resembles your application.

This is the difficult part of the problem.

Cleaning global state isn't enough. You need to know which state belongs to the application and which state belongs to the previous test. WordPress core can't determine that for your plugin. But your testing layer can.

Cleanup works until you forget something

The first instinct is usually to make every test clean up after itself.

1public function tearDown(): void {
2	unregister_post_type( 'book' );
3
4	parent::tearDown();
5}

For simple cases, that works. We've done this one before a few (or more) times.

I've come to dislike this model for framework-level isolation, though, because it asks the test that mutated the environment to also know every piece of state that mutation affected.

And WordPress's state is rarely that simple.

We found a good example of this recently in Mantle.

Mantle's testing framework was already preserving $wp_post_types. A test could unregister a post type, and we'd restore the post type registry before the next test.

That sounded correct.

It wasn't.

WordPress also stores post type features such as title, editor, and thumbnail in a separate global: $_wp_post_type_features.

So after restoring the post type, you could get this wonderfully inconsistent result:

1$this->assertTrue( post_type_exists( 'book' ) );
2
3$this->assertFalse( post_type_supports( 'book', 'title' ) );

The post type existed again. Part of the state describing it didn't. We had restored what we knew to restore and leaked what we didn't.

That's why I don't think teardown is the right abstraction for this problem.

Restore a known state instead

Mantle's Testkit takes the opposite approach.

Instead of asking every test:

What did you change, and how do we undo it?

we ask:

What did WordPress look like before you changed it?

Then we restore that state.

Today, the set of globals Mantle snapshots includes:

1protected const GLOBALS_TO_BACKUP = [
2	'_wp_post_type_features',
3	'wp_meta_keys',
4	'wp_post_statuses',
5	'wp_post_types',
6	'wp_rewrite',
7	'wp_sitemaps',
8	'wp_taxonomies',
9];

Public query variables are preserved separately as well.

There's another detail here that matters: we don't simply snapshot WordPress before the entire test suite and reset to that state every time.

A test class may legitimately establish state in setUpBeforeClass().

So the lifecycle is roughly:

Capture WordPress's original state.
Allow the test class to perform its setup.
Capture that as the baseline for the class.
Restore that baseline before each test.
Restore WordPress's original state when the class is finished.

Objects in those globals have to be deep-copied, too. Otherwise your "snapshot" can contain references to the same objects the test is about to mutate.

The goal isn't an empty WordPress installation before every test.

It's a known WordPress installation before every test.

That distinction is what makes the technique useful for plugins and applications.

You can find these leaks without Testkit

You don't need Mantle to discover whether your test suite has this problem.

Randomize your test order:

1vendor/bin/phpunit --order-by=random

Then run it again.

And again.

If different orders produce different results, start looking for shared state.

When you find a suspicious test, run it independently:

1vendor/bin/phpunit tests/Feature/BooksTest.php

Then run its entire test class. Then the suite.

You're looking for a test whose result depends on what executed before it.

Test-order randomization isn't only a way to find flaky tests. It's a way to test one of the assumptions underneath your entire suite:

Does this test actually establish everything it needs?

That's a useful question in any test suite.

It's particularly useful in WordPress.

We're still finding state

I would love to say that after maintaining Testkit for years we've now found every WordPress global that can leak between tests.

I don't believe that.

$_wp_post_type_features was added to our preservation logic because we found a real case our previous implementation didn't handle.

We still have an open issue around roles and capabilities. That's another particularly awkward piece of WordPress runtime state: tests need to be able to modify roles, while application-defined roles need to survive between tests.

I'm sure it won't be the last one.

And I think that's the more useful way to think about this problem.

WordPress wasn't designed around the idea that every operation had to be reversible so another test could immediately reuse the same PHP process. It was designed around requests, where process termination gives you that reset for free.

Testing removes that reset. Once you recognize that, a lot of mysterious WordPress test behavior stops being mysterious. A green test doesn't only need a clean database. It needs a known runtime.

And in WordPress, those are very different things.