Tests that pass, and why that isn't the same as trust

Green is not the same as safe

A passing test suite is a comforting thing. It's also the single most over-trusted signal in software. All "green" tells you is that the code did what the tests asked. It says nothing about whether the tests asked for anything that matters.

I've reviewed suites with a thousand passing tests that would not have caught a single one of the bugs that actually shipped. They weren't fraudulent. They were just testing the wrong layer, in the wrong way, against themselves.

The tests that don't earn their keep

A few patterns come up again and again.

Tests that assert the mock. The test sets up a mock to return 5, calls the method, and asserts the result is 5. The database, the mapping, the real query — none of it ran. You've tested that your mock framework works.

_repo.Setup(r => r.GetTotal(id)).ReturnsAsync(5);
var result = await _service.GetTotal(id);
Assert.Equal(5, result);   // this can never fail for a real reason

Tests with no meaningful assertion. They call the method, and as long as it doesn't throw, they pass. A whole class of logic errors sails straight through because nothing actually checks the value.

Tests that restate the implementation. If the production code says x = a + b and the test also computes a + b and compares, they'll always agree — including when a + b was the wrong formula all along.

Tests coupled to structure, not behaviour. They assert that a private method was called, or that three internal steps happened in order. Refactor the internals — behaviour identical — and the suite goes red. So people stop refactoring. Now the tests are actively protecting the mess.

The question that sorts them

For any test, I ask one thing: what real defect would this have caught?

If I can't name a plausible bug the test would have turned red for, the test isn't a safety net — it's a green light with nothing behind it. A good test has a clear failure it exists to catch. You should be able to say, in one sentence, "this fails if X breaks," and X should be something a human might actually get wrong.

A quick, brutal check: mutate the production code and see if anything notices. Flip a < to <=. Return the wrong branch. Delete a line. If the suite stays green, those tests were never watching that code.

// change this…
if (balance < amount) throw new InsufficientFunds();
// …to this, and run the suite:
if (balance <= amount) throw new InsufficientFunds();

If nothing fails, you've just learned that your overdraft boundary is untested — no matter how many green ticks sit next to it.

What I actually assess

• Do the tests exercise real behaviour, or restate mocks and implementation?
• Would a plausible bug make them fail? I try a few mutations and watch.
• Are they coupled to what the code does or how it's built? The second kind punishes the refactoring you most need.
• Is the important logic covered at all — the money, the boundaries, the auth decisions — or only the easy, uncontroversial paths?

Coverage percentage measures lines executed. It does not measure whether a failure would be caught. Those are different questions, and the gap between them is exactly where the bugs you trusted the suite to stop are living. Finding that gap — the code that's "tested" but not actually watched — is one of the most valuable things a second reader can do for you.