Acceptance criteria for AI agents: write checks that cannot be faked

When an agent run goes wrong, it rarely goes wrong at the tool call. It goes wrong at the acceptance criteria. The agent did what the words said, and the words did not say what you meant.

The test: could a stranger prove it?

Every acceptance criterion should be checkable by someone who has never met you, never read your mind, and is mildly hostile. If a stranger cannot prove the check passes or fails, neither can a verifier, which means the agent can claim success and be right.

Vague check

Fix the login flow.

What does fixed mean? Faster? Fewer steps? Different error text? Nobody can grade this, including the agent. It will do something plausible and report SUCCESS.

Checkable check

A user with a valid account can sign in from the login page and reach the dashboard. Wrong password shows a clear error and does not lock the account.

A stranger can run this. They know what passing looks like, and they know what failing looks like. That is the entire bar.

Three failure modes to kill

1. Activity instead of outcome

"Implement the export feature" describes work, not a finished state. Outcome phrasing: "Export produces a CSV that opens in Excel with one row per order."

2. Vacuous passes

A check the agent can pass without doing the job is worse than no check. "Handle errors gracefully" passes by doing nothing when there are no errors. Instead: "When the payment API returns 500, the user sees a retry message and no charge is recorded."

3. Unverifiable claims

"Improve performance" cannot be graded. "Homepage loads in under 1.5 seconds on a mid-range phone over 4G" can. Same for "robust" (grade by a concrete failure scenario), "clean" (grade by the lint command), and "user-friendly" (grade by the three-step path).

Non-goals are half the contract

Boundaries are acceptance criteria in disguise. An agent with no non-goals will refactor, upgrade, and "improve" its way past your deadline. Write the list of what it must not touch: no dependency upgrades, no redesign, no production deploy, no emails to real customers.

A template that works

Goal: A finished state, one sentence.
Checks:
  - Auto: provable by command, URL, file, or test
  - Human: a named person confirms one specific thing
Boundaries: what the agent must not touch
Budget: how many attempts before it must stop

Grab a ready-made work order template, or chat with Grok to turn plain English into a contract. Then run the $1 audit to see how a hostile reviewer grades it.