Skip to main content
A run holds one thing still and moves another. The task is the fixed part: what world to boot, what to ask for, and what counts as success. Your agent is the part that moves. Between the two sits a sandbox, where the work actually happens, and after it a score that reads what the twin recorded. The loop runs the same way from the coach skills in your coding agent, from the CLI, or from the dashboard.

​
The loop

The run loop. A task supplies the seed state, the instruction and the success criteria, and stays unchanged between runs. Your agent, where the prompt, tools and model live, acts inside a sandbox where the digital twin boots from the seed and records every call. A score then runs code over the tape and the final state. An arrow returns from the score to your agent: change the agent, then run the same task again. The run loop. A task supplies the seed state, the instruction and the success criteria, and stays unchanged between runs. Your agent, where the prompt, tools and model live, acts inside a sandbox where the digital twin boots from the seed and records every call. A score then runs code over the tape and the final state. An arrow returns from the score to your agent: change the agent, then run the same task again. No step in that chain depends on the agent telling the truth about itself. The twin does the writing, and the score reads what the twin wrote. The arrow back matters as much as the ones going forward, and it lands on the agent rather than on the task. What you change after a bad run is your side: a system prompt, a tool added or taken away, a different model, the code behind a tool. The task stays exactly what it was, and that is the only reason the second score means anything next to the first. You can of course change the task as well, to make it harder or to fix a bad criterion, but then you are running a different exam and the old number stops being a baseline. If you would rather watch it than read about it, the graded capstone runs the whole chain once.

​
What a run needs

A task is one markdown file carrying three things: the world to boot, the instruction to hand the agent, and what counts as success. The agent under test receives the instruction and never the criteria, because it is sitting the exam rather than marking it. Write a task documents every section that file can carry.

​
Why the twin resets

A digital twin is a stateful service that reproduces the response shapes and error semantics of a real API without ever calling it. The property that makes it useful for testing is not the emulation, it is the reset. Every run starts from the task’s seed state, and the twin returns to that state between runs. Two runs that score differently differ because the agent behaved differently, not because the world moved underneath them. That is what makes a number from yesterday comparable to a number from today. Which endpoints a twin implements, and how deeply, differs per API. How we pick what a twin covers is the method, and status.pome.sh publishes how each surface measured against the real API today.

​
Why the score is code, not a judge

Every criterion a task declares is one of two kinds, and the difference decides what the score means. [code] criteria are checked by the grader: queries over the twin’s final state and over the tape, the record of every call the twin served and every mutation that landed. These are what the score is out of. Each verdict comes back with the evidence pointer it was read from, so a failure hands you the row it read rather than a claim about it. [model] criteria are the judgment calls no query answers, like whether the agent explained itself or noticed that a label was contextually wrong. The narrator reads those and answers them in prose. Its reading never enters the denominator, so a task with three [code] criteria and two [model] ones scores out of three. Keeping the LLM outside the denominator is deliberate. A score you can reproduce is a score you can regress against, and one that moves between two identical runs cannot gate a merge.

​
What the run leaves behind

The tape outlives the run. It is what the grader read, what the narrator read, and what you can export into an eval stack you already have. The twin’s state before and after the agent sits beside it, so a criterion can assert on the world the agent left behind rather than on the account it gave of itself. Hosted runs also land on the dashboard, with the full trace, the score broken down per criterion, and the narrator’s handoff when something failed. pome run documents the artifact directory and every file in it; Use your existing eval stack grades a Pome run in Braintrust or LangSmith instead.

​
Next

The graded capstone

The exam you just read about, end to end: a scored run, a failure, a one-sentence fix.

Your own agent

Point Pome at your own repo and get a scored run.

Dashboard

Where runs, sandboxes, and the narrator’s handoff live.