production · 26 minute study

Treat AI output as a hypothesis until verified

AI can generate convincing code and convincing reports faster than a human can inspect them. Engineering judgment determines which claims deserve trust.

Loading progressEvidence grows with practice

Useful preparation · 2 prerequisites

Use this as a map, not a lock. If these ideas are familiar, continue.

Make verification reproducible and honest — A passing result on an unknown revision, random state or stale build is difficult to trust and impossible to reproduce reliably.
Give each rule one authoritative home — Duplicating business rules in a form, API and export eventually creates three different answers. Reuse is valuable when it preserves one explicit contract.

First, build a mental model

An AI proposal is a candidate explanation or patch. Confidence comes from repository evidence, independent expectations and actual execution, not the fluency of the proposal.

Give a bounded brief: intended outcome, relevant paths, invariants, constraints, test plan and forbidden changes. Ask it to inspect before asserting the stack.
Verify APIs against the installed version and primary documentation. Plausible imports, flags and test commands may not exist.
Review the diff and the tests independently. A generated test can repeat the same wrong assumption as generated code.
Distinguish edited, tested, built and deployed. Ask for the exact command, revision, output and limitations; do not treat a fabricated success summary as execution evidence.

Read the codejson · educational example

{
  "claim": "All tests passed",
  "testedRevision": null,
  "command": null,
  "exitCode": null,
  "output": null,
  "evidenceStatus": "unverified"
}

Claim
The sentence sounds complete but names no reproducible execution.
Missing evidence
No candidate, command, exit status or test output supports the result.
Next action
Inspect the actual scripts and candidate, run the relevant checks, and preserve what really happened.

Production thinking

Take this beyond the example.

Delegate routine drafting and bounded implementation; retain authority over consequential invariants, permissions, financial meaning and release decisions. Stop prompting when one deterministic check can resolve the uncertainty.

Sources and further reading

Your evidence

unseen is a local learning signal, not a credential or proof of production competence.

Explain the mental model, then explicitly acknowledge it. Opening a page is not evidence.

Supported retries can complete this lesson. Independent evidence and delayed review determine the skill stage.