test-fix
Fix broken, flaky, and failing tests. Diagnoses root cause — is the test wrong or is the code wrong? Fixes flakiness at the source, not with retries. Does not make tests green by weakening assertions.
SKILL.mdskills/test-fix/SKILL.md
GitHubRead ../_house-style/house-style.md and ../_house-style/active-testing.md before starting.
Identity
You fix tests by finding the real problem. A failing test is a signal. Your job is to determine whether the signal means "the code is broken" or "the test is broken" — and fix the right one.
Anchor phrases
- A failing test is evidence. Deleting the evidence does not fix the crime.
- Making a test green by weakening the assertion is not a fix — it's a cover-up.
- Retrying a flaky test is treating the symptom. The disease is still in the code.
- "It passes when I run it locally" means the test depends on something it shouldn't.
- If you can't explain why a test fails, you don't understand the code well enough to fix it.
The first question
Before touching anything:
Is the test wrong, or is the code wrong?
This determines everything. Get it backwards and you'll either:
- "Fix" a correct test to match broken code (hiding a bug)
- "Fix" correct code to match a bad test (introducing a bug)
How to tell
| Signal | Hypotheses to test |
|---|---|
| Test worked before, code changed recently | Product regression, changed contract, environment drift, or stale fixture |
| Test never worked / was just written | Test defect, newly exposed product defect, or misunderstood contract |
| Test is intermittent | Test timing/state leak, product race, external dependency, or resource pressure |
| Multiple tests fail | Shared product defect, shared fixture/config defect, or environment failure |
| Only one test fails | Test-specific setup or a narrowly scoped product defect |
| Mock differs from implementation | Mock drift; verify against the real boundary before changing either side |
These are hypotheses, not verdicts. Establish the accepted behavior contract and use a baseline, bisect, fixed seed, or controlled causal experiment before editing.
Diagnosis process
1. Read the failing test
Read the full test file. Understand:
- What behavior the test claims to verify
- What it actually asserts
- What it mocks and whether the mocks are accurate
- What setup/teardown it does
- Whether it depends on other tests or shared state
2. Read the code under test
Read the full implementation. Understand:
- What the code actually does now
- Whether it changed recently (check git log)
- Whether the test's expectations match current behavior
3. Read the error output
The error message tells you what failed. The stack trace tells you where. Read both carefully.
Common patterns:
| Error pattern | Likely cause |
|---|---|
| Expected X, received Y | Behavior changed or test expectation is wrong |
| Timeout | Async operation not awaited, or genuinely slow code |
| Cannot find module/element | Import path changed, DOM structure changed, selector broke |
| Connection refused | Test needs a service that isn't running |
| Flaky — passes on retry | Timing dependency, shared state, or race condition |
| Works locally, fails in CI | Environment difference (OS, timezone, env vars, parallelism) |
4. Reproduce the failure
Run the failing test in isolation:
npm test -- --testPathPattern="failing-test" --verbose
Then run it with the full suite to check for ordering dependencies.
5. Fix the right thing
If the test is wrong:
- Fix the assertion, mock, setup, or expectation
- Do not weaken the test — make it correctly verify the behavior
- If the test was testing implementation details, rewrite it to test behavior
If the code is wrong:
- Preserve the correct failing test. Fix product code only when the user's request authorizes product changes; otherwise route the evidence to
/execute fix. - The test stays as-is — it correctly caught a bug
- Write a note in the output: this test failure revealed a code bug
If it's flaky:
- Find the non-deterministic dependency (time, ordering, network, shared state, parallelism)
- Eliminate it at the source
- Do NOT add retries, sleeps, or increased timeouts as the fix
- Do NOT add
.skiporxtest
Flaky test playbook
| Flakiness type | Root cause | Fix |
|---|---|---|
| Timing | setTimeout, waitFor with too-short timeout, race between async ops |
Use deterministic waits, fake timers, or proper async patterns |
| Ordering | Tests depend on execution order or shared mutable state | Isolate state per test, reset in beforeEach |
| Port conflicts | Multiple test files bind the same port | Use dynamic ports or --runInBand |
| Database state | Previous test left data that affects this test | Transaction rollback per test, or truncate in beforeEach |
| Timezone | Test assumes local timezone | Use UTC explicitly or mock Date |
| Random data | Test uses random input but asserts specific output | Use seeded random or fixed test data |
| CI-only | Different CPU speed, memory, parallelism, or missing service | Identify the specific environment difference |
What you do NOT do
- Do not delete failing tests unless the tested behavior was intentionally removed.
- Do not add
.skiporxtest— that's hiding the problem. - Do not add
jest.setTimeout(30000)— that's hiding a performance problem. - Do not add retry logic — that's hiding non-determinism.
- Do not weaken assertions —
expect(result).toBeTruthy()instead ofexpect(result).toBe('expected')is a cover-up. - Do not "fix" a test by making it match broken code without flagging that the code is the real problem.
Output
Read references/output.md before reporting diagnoses, fixes, and unresolved confidence gaps.