Test coverage measures which lines of code your test suite actually executes ā a genuinely useful diagnostic for finding untested code, and a genuinely dangerous metric when treated as a target to maximize rather than a signal to interpret.
1What Coverage Actually Measures: Execution, Not Correctness
coverage.py (typically invoked via the pytest-cov plugin, as pytest --cov=myapp) instruments your code while tests run and tracks, precisely, which specific lines were actually executed during that test run ā a genuinely objective, mechanically-measured fact, not a subjective assessment or estimate. The resulting report's Missing column lists exactly the line numbers that were *never* executed by any test in the suite at all.
This is directly actionable in the way it's most valuable: myapp/payments.py, lines 15-26 showing as missing, tells you *precisely* that refund_payment's validation logic has zero test coverage whatsoever ā no test in the entire suite has ever exercised that code path, meaning there is currently no test evidence at all about whether it behaves correctly, whether it's even reachable, or whether it contains an obvious bug that's simply never been triggered by anything in the test suite.
This diagnostic value ā pinpointing exactly which code paths have literally never been run by any test ā is coverage's genuine, real strength, and it's the reason coverage measurement is a standard, valuable part of a mature testing practice. The critical thing to hold onto precisely, and the source of coverage's most common misuse, is what it does *not* measure, covered next.
$ pytest --cov=myapp --cov-report=term-missing
Name Stmts Miss Cover Missing
-----------------------------------------------
myapp/orders.py 45 3 93% 67-69
myapp/payments.py 30 12 60% 15-26
-----------------------------------------------
TOTAL 75 15 80%These specific lines never ran in ANY test ā a precise, actionable gap
2100% Coverage Does Not Mean Correctly Tested
test_calculate_discount() calling calculate_discount(100, 10) genuinely does execute every line of calculate_discount, satisfying coverage completely for that function ā and it verifies almost nothing, because the function's *return value* is never checked against an expected result with an assert. If calculate_discount's implementation had a bug ā using + instead of -, or a wrong percentage calculation entirely ā this test would still pass, with 100% coverage, because coverage only asks 'did this line run', never 'did this line's result match what was actually expected.'
This is the single most important caveat about coverage as a metric: a high coverage percentage is a necessary but drastically insufficient condition for a well-tested codebase. It tells you code was *exercised*; it tells you nothing about whether the tests exercising it made any meaningful assertions about correct behavior at all. A test suite can trivially achieve 100% coverage by calling every function once with no assertions whatsoever, and that suite would catch essentially zero real bugs despite its perfect coverage number.
The professional discipline this motivates: treat coverage reports specifically as a tool for finding *completely untested* code (the genuinely valuable signal covered in the previous section), never as a target number to chase for its own sake, and never as a proxy for 'this code is well-tested' without actually reading the tests themselves to confirm they contain meaningful, specific assertions about expected behavior ā not just calls that happen to execute the code.
# myapp/payments.py, lines 15-26 (never executed by any test)
def refund_payment(payment_id: str, amount: float) -> bool:
if amount <= 0:
raise ValueError("Refund amount must be positive") # line 15 -- never tested!
# ... 11 more lines never exercised by the test suite at allA completely broken function could pass this 'fully covered' test
3Using Coverage as a Signal, Not a Score to Maximize
The healthy way to use coverage, given its genuine strength (finding never-executed code) and its genuine limitation (saying nothing about assertion quality), is as a *diagnostic tool run periodically* ā after writing a meaningful chunk of new code and its tests, run coverage, and specifically investigate any surprising gaps: is this missing code genuinely untested by oversight, or is it defensive code for a condition that's difficult or impossible to trigger in a test (in which case, low coverage there might be entirely acceptable and expected)?
Chasing a specific coverage percentage as an organizational target (a CI gate requiring 90%+ coverage, for instance) has a well-documented failure mode: it creates pressure to write tests that execute code without meaningfully verifying it, purely to satisfy the number ā exactly the test_calculate_discount() anti-pattern, scaled across an entire codebase, actively making the test suite *look* more trustworthy than it actually is. A lower coverage number with every existing test containing genuine, specific assertions is a materially healthier codebase than a higher coverage number achieved partly through assertion-free tests written to satisfy a gate.
The professional judgment this section builds toward: use coverage to find gaps worth investigating, not as a scoreboard to maximize; and when reviewing any test (your own or a colleague's), verify it contains real assertions checking specific, expected behavior ā coverage numbers alone, however high, are never sufficient evidence that a codebase is genuinely well-tested.
def calculate_discount(price: float, percent: float) -> float:
return price * (1 - percent / 100) # this line EXECUTES...
def test_calculate_discount():
calculate_discount(100, 10) # ...but the RESULT is never asserted!
# 100% coverage on this function -- and it could return the WRONG value and this test would still passNever treat the percentage itself as the goal
4Step-by-Step Breakdown
100% test coverage doesn't mean your code is correctly tested ā it means every line RAN during testing, which is a much weaker guarantee than most people assume.
pytest-cov measures which lines of your code actually EXECUTED during the test run -- a direct, objective measurement, not a guess.
'Missing' lines are executable code your test suite NEVER ran -- a direct, actionable signal for exactly where to add tests.
Checkpoint: What does the "Missing" column (lines 15-26) in the coverage report actually tell you?
- āThese specific lines were never executed by any test in the test suite at all
- āThese lines were executed, but the assertions checking them failed
100% coverage does NOT mean correctly tested -- a line can RUN without its result ever being checked by an assertion.
Checkpoint: Why does test_calculate_discount() achieve 100% coverage on that function while providing almost no real verification?
- āThe line executes (satisfying coverage), but its RESULT is never checked with an assert ā so a wrong return value would go completely undetected
- āThis is a bug in coverage.py that will eventually be fixed
That completes the core pytest toolkit ā fixtures, mocking, parameterization, and coverage. The Native Python Test Runner lesson closes this section with the standard library's own, dependency-free alternative.
Compute Real Test Coverage. Finish coverage_percent(): coverage is the fraction of lines actually executed.
Level Up š
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Best Practices
Use coverage reports to find genuinely untested code paths, investigated case by case
This is coverage's real strength ā precisely identifying lines with zero test execution, worth a deliberate decision about whether to add a test or whether the gap is acceptable (e.g. unreachable defensive code).
Never treat a coverage percentage as a target to maximize, or as proof of correct testing on its own
Coverage measures execution, not assertion quality ā a high percentage achieved through assertion-free tests provides a false sense of confidence and can be actively worse than a lower, honestly-earned number.
Frequent Bugs
Treating a required coverage percentage as a CI gate without also reviewing whether the tests achieving that coverage contain genuine, meaningful assertions, allowing assertion-free 'coverage-only' tests to satisfy the gate.
Use coverage reports as a diagnostic for finding untested code, but always review actual test content for real assertions ā never rely on a coverage percentage alone as evidence of correct, meaningful testing.
Real-World Examples
Investigating a Coverage Gap to Find a Genuinely Untested Edge Case
A coverage report reveals that a payment refund function's negative-amount validation branch has never been executed by any test, prompting the team to investigate and add a missing test case.
# Coverage report flags this branch as never executed:
def refund_payment(payment_id: str, amount: float) -> bool:
if amount <= 0:
raise ValueError("Refund amount must be positive")
return payment_gateway.refund(payment_id, amount)
# The missing test, added after investigating the gap:
def test_refund_payment_rejects_non_positive_amount():
with pytest.raises(ValueError, match="must be positive"):
refund_payment("pay_123", -10.0)