A fix without a failing test is a claim about a fix.

1. Reproduce firstEstablish the failing behaviour in an isolated environment with harmless fixture state, and record the exact command and output.
2. Write the test against the vulnerable buildAssert the security property, not the symptom. Run it and confirm it fails, and confirm it fails for the reason you expect rather than for a setup error.
3. Repair the canonical invariantFix the boundary that owns the rule, not the call site that exposed it. A repair applied at one caller leaves the siblings intact.
4. Rerun without changing the testThe same command, the same assertion. Editing the assertion between the failing run and the passing run discards the evidence.
5. Assert the legitimate path in the same testOtherwise a blanket denial passes as a fix, and the regression suite will happily protect a broken feature.
6. Record the command and the assertionThe report contract requires both, plus a status. A prose claim that the fix was tested is not a regression record.

The paired evaluation fixtures are built to enforce this habit. Every fixture ships a vulnerable variant and a clean variant of the same file, and the clean variant still has to do its job. Several pairs are deliberately built so the alarming-looking file is the safe one, which means an assertion keyed to surface patterns fails the pair rather than passing it.

One fixture pair makes the point precisely. The vulnerable variant of a URL fetcher has an address allowlist; it validates the destination once and then follows redirects without re-validating, so a permitted host can redirect the fetch to a private address. The clean sibling disables automatic redirects and revalidates every hop in a bounded loop. A test asserting only that an allowlist exists passes both.

The published case study followed the same rule. Ten regression assertions cover the fix, and they are not all denials.

A test that calls the service function directly will pass while the route, the worker, or the database policy remains defective. Choose the layer by asking where the rule is canonically enforced, then test one layer below where you would prefer to test.

Object authorization

At the data-access boundary, with two owned identities, not at the route handler.

Tenant isolation

Against the database with the runtime role the application actually connects as.

Single-use transitions

With two controlled concurrent callers, counting committed effects.

Response headers and framing

Against the built artifact as served, not against the configuration object.

Tool authority

Against the dispatcher call log, with every tool stubbed to record rather than act.

The report contract records the result as a structured object rather than as prose, and each Gold Check Pack names the regression fixture its class is expected to leave behind. The status vocabulary is deliberately small.

"regression": {
  "status": "PASS",
  "command": "pnpm vitest run tests/security.test.ts",
  "assertion": "Every response carries frame-ancestors 'none' and X-Frame-Options: DENY,
                and a relative CTA href still renders."
}

• NOT_RUN, PASS, FAIL, and NOT_PRACTICAL are the four permitted statuses. There is no state that means "probably fine".
• command and assertion are both required, so the claim is reproducible by someone who was not there.
• Scanner output alone cannot close a verified finding. A clean rescan is not a regression proof.

• A stale build cacheIn the published run the first retest appeared to fail: the headers were still missing and the hostile URL was still in the document. The cause was a stale prerender cache plus a server still bound to the old port. Only a clean rebuild proved the fix.
• A green typecheck read as proofTypes compiling says nothing about what the built application serves. The rule is written into the workflow for exactly this reason.
• An assertion edited between runsIf the test that failed is not the test that passed, there is no before-and-after. Change the code, never the assertion.
• A test that never failedWritten after the repair, it has no demonstrated relationship to the defect. It is a smoke test wearing a security label.
• The symptom instead of the invariantAsserting that one payload is blocked protects against that payload. Asserting the invariant protects against the class.

When the retest is right, the evidence is unambiguous. In the recorded run the browser refused the frame itself rather than the test asserting an absence.

Framing 'http://localhost:3009/' violates the following Content Security Policy
directive: "frame-ancestors 'none'". The request has been blocked.

A regression record only matters if something reads it. Generate the canonical report in one step and run the gate in a separate, fail-closed step. Both commands are standard-library Python from the open repository, available once you have installed the skill.

python scripts/validate_contract.py report report.json
python scripts/security_gate.py report.json --policy policies/default.json
# exit 0  PASS or PASS_WITH_KNOWN_RISK
# exit 1  BLOCKED
# exit 2  INCOMPLETE or malformed input

• Treat exit codes 1 and 2 as non-green. A run that could not produce evidence is not a run that found nothing.
• INCOMPLETE exists so that missing evidence has somewhere to go other than PASS.
• PASS_WITH_KNOWN_RISK requires an owner, a reason, an approval time, and an expiry. A model may summarize an accepted risk; it may not approve one.
• Coverage counts travel with the report, including how many integrity-critical hypotheses ended UNKNOWN.