| Term | Definition |
|---|---|
| Digital twin | A deterministic, stateful emulation of one real SaaS API (GitHub, Slack, Stripe, Gmail, Linear). Same request/response shapes and error semantics as the live service, no production side effects. See How Pome works. |
| Task | The exam: a markdown file with three required sections — ## Config, ## Prompt, ## Success Criteria — and an optional ## Seed State. See Tasks. |
| Seed state | JSON the digital twin loads before the agent starts — the initial world (repos, channels, charges). |
| Examinee | The clone of the agent under test. It works the task against the twins and never sees the grading criteria. |
| Coach | The agent driving the test — your own coding agent, working over the Pome MCP. It installs, launches runs, finalizes, and reads reports. The coach can see criteria; the examinee cannot. |
| Grader | The deterministic engine behind a [code] criterion — SQL over the digital twin’s final state and its tape. It is the only thing that scores, and every verdict it returns ships with the evidence pointer behind it. |
| Narrator | The LLM. It writes each run’s summary and handoff, and an advisory reading of every [model] criterion. It holds no score authority: nothing it writes moves a number. |
| Judge | The LLM scoring a [model] criterion rather than reading it. Beta, off by default, chosen per agent (evaluator_id, which defaults to narrator). A [code] criterion has a grader and never a judge. |
| Run | One execution of a task by one agent: boot the twin with the seed, record every call, score the result. Artifacts land in runs/<task>/<run-id>/ and on the dashboard. |
| Sandbox | What you start: one id, one TTL, one price, and the 1–3 digital twins a run talks to. Defined in full on Sandboxes. It expires; finalize the run while the sandbox is still live, or it cannot be scored. The wire spells it /v1/sessions, session_id, and the ses_ id prefix. |
| Finalize | The call (finalize_run) that closes a run and triggers scoring. |
| Criteria | A task’s acceptance checks, and what owns the verdict. [code] criteria are deterministic checks on twin state or events, and they are what the score is out of; [model] criteria are plain-English judgment calls the narrator reads and answers in prose. |
| Score | 0–100 satisfaction across the [code] criteria that were evaluated — [model] readings are advisory and never enter the denominator. A run passes when the score meets the task’s passThreshold (default 100). A task with no [code] criteria reports narrative only — 0 scored criteria instead of a percentage. |
| Trace | The recorded stream of everything the examinee did (events.jsonl): twin HTTP calls, LLM calls, run lifecycle events. |
| Surface | A place you drive the loop from: the CLI, the coach skills, the MCP, or the dashboard. Same loop and the same run row, wherever you start it. |
pome.json | The manifest that connects a repo to Pome: the agent’s slug, its start command, and the twins it uses. Written by pome init, read by every run. See Your own agent. |
Terminology
Terminology
Every term the Pome docs use — digital twin, task, examinee, coach, grader, narrator, run, score — defined in one place.
Every coined term in these docs, defined once. Linked pages go deeper.