Skip to main content
The first-run loop over the Pome MCP, for a human following along or an agent executing the calls directly.

​
The loop

​
1. Read the playbook

Returns the coach playbook — read it before doing anything else.

​
2. Confirm identity and quota

Confirms identity, team, plan tier, and quota headroom.

​
3. Register the agent under test

Use intake_clone_scope for the rich path — it captures the agent’s clone scope so run_task can assemble the examinee launch spec:
Or register_agent for a lightweight registration:
Pass twins — the services the task will exercise. Naming none enables only github, and run_task refuses a task whose twins are not enabled. Keep agent.slug from either response — you’ll pass it as agent_slug to run_task, the durable handle; agent_id is the opaque alternative.
Both MCP verbs are for the no-local-repo (Managed-Agents / hosted) case. If you have a local repo with a pome.json, register from the CLI instead — pome register agent "<name>" [--twins …] — so the server-canonical slug lands in pome.json. See Your own agent.

​
4. Choose or author a task

Pick an existing task:
Or author one. Read the twin’s declared checks first, and build every [code] criterion from that list — a hand-written criterion that binds nothing is never graded:
Then sanity-check the seed before you spend a run on it:

​
5. Run the task

Or run_trials for N trials of the same task sharing a group:
agent_version is the label this run declares itself to be — the manifest’s agent.version. It keeps one version’s trials from averaging with another’s, so pass it on every run and change it whenever you change the examinee’s prompt. This provisions a sandbox with the task’s twins and returns examinee_launch — the clone spec. Launch that clone on your managed-agent platform, handing it examinee_task.prompt and every key of examinee_launch.env. Before you launch, show the user eval_cost.headline from the response. Nothing is charged until finalize_run, so this is the last point at which stopping costs them nothing.
Set mcp_permission_policy { type: "always_allow" } on every examinee MCP toolset from examinee_launch.mcp_servers[], or the headless clone deadlocks on its first MCP call.
Export examinee_launch.env whole, not a hand-picked subset. Besides the per-twin REST bases and the session bearer it carries the session-scoped OTLP exporter wiring; drop it and the report shows twin HTTP calls with no record of the examinee’s own model, per-turn tokens or latency.

​
6. Finalize

Pass session_id alone. The capture bearer is re-issued server-side and the task was stamped at run_task, so you need neither — do not carry the sandbox JWT around just to finalize. Grades synchronously: pulls the recorded twin tape, scores it, tears down the sandbox. The score comes back in run, no polling. Call it before the sandbox expires.

​
7. Get the report

Returns the scored markdown report plus a human weblink to app.pome.sh.

​
8. Fix, then re-run as a new version

A failing report names what the agent did wrong. The builder edits the examinee’s prompt — never the task — and you re-run:
Three things change together, and each one is load-bearing:
  • agent_version — the edited prompt is a different thing under test. Left at v1, its trials fold in with the failing ones and the dashboard reports the fix as a reliability spread (“1 of 2 clean”) rather than a fix.
  • a fresh group_id — the re-run is its own run-set. Reusing the baseline’s merges the two into one aggregate.
  • baseline_group_id — pairs the two run-sets, which is what puts Compare to failing baseline on the green run-set’s dashboard page.
Coach vs examinee: the agent making these tool calls is the coach — it sees criteria and picks tasks. The examinee is the separately launched clone that attempts the task; it holds no Pome tools and never sees the criteria.

MCP tools

Full input/output reference for every tool.

Digital twins reference

What run_task provisions against.

CLI reference

The CLI equivalent of this loop.