How to verify AI agent work: do not trust the report

The agent says it is done. The agent also wrote the summary saying it is done. Those two facts are why verification exists as a separate step, and why the verification has to be done by something other than the agent that did the work.

The report is a claim, not evidence

A confident summary is the cheapest thing an agent produces. Treat "done", "fixed", and "SUCCESS" as claims that need proof, not as facts. Everything changes when you ask one question: show me the output that proves it.

Re-run the checks yourself

The acceptance criteria are the contract. Run them. If a check says "the export opens in Excel with one row per order", open the file and count. If it says "the reset email arrives in under a minute", send one and watch the clock. Checks that are provable by command or URL are the ones you can re-run in seconds, which is why they matter more than opinion checks.

Separate Auto from human checks

Auto checks (commands, URLs, files, tests) can be re-run mechanically. Human checks need a named person to confirm one specific thing. Keep the two separate in the contract and in your review. An agent that passed every Auto check and skipped the human one is not done, no matter what the summary says.

Demand an evidence chain

For anything non-trivial, ask for the trail: which tool ran, what it returned, which artifact it produced, and who reviewed it. A run with a real evidence chain can be audited after the fact. A run with only a summary is a story.

Use a different reviewer than the worker

The strongest check is an independent one: a separate reviewer who did not do the work grading the acceptance criteria against the evidence. That is the entire point of the MECHA run: workers produce, a reviewer verifies, and you get the exit code plus the evidence chain back. Honest failure beats fake SUCCESS, and the exit code tells you which one you got.

When things go wrong

• Unverifiable criteria: the contract is the problem. Fix the checks, not the agent.
• Vacuous pass: the check was satisfiable without doing the job. Rewrite it to require the artifact.
• Drift: the agent touched things outside its boundaries. Tighten the non-goals and rerun.

Write contracts that can be verified in the first place with the work order guide, start from a template, and run the $1 audit before a run so the checks are sharp when it matters.