Skip to main content
The graded capstone runs a bundled example so you can watch a scored run without any code of your own. This page is your repo. It goes all the way: register the agent, prove the wiring, and finish on a scored report of your agent, with the same run → read the failure → fix → re-run loop the example taught you. Three steps, and the third one produces the score.

Before you start

  • The graded capstone, done once. Two commands and one paste-prompt on a bundled example. Every step the coach takes there repeats on your own agent here, so you will recognize each one; skip it and the first unfamiliar thing on this page is also the first thing you have to debug.
  • Its setup, also done once: the coach skills installed (npx skills add pome-sh/digital-twins --skill '*') and the Pome MCP added at https://mcp.pome.sh/mcp. Connect to the MCP carries the wiring for each client.
  • The Pome CLI, which writes and reads the manifest:
    pome login opens app.pome.sh, signs you in, and stores a pme_… key in your macOS Keychain (or ~/.pome/credentials.json). CI can set POME_API_KEY instead.
  • Your agent’s repo, open in your coding agent. It should talk to a service Pome has a twin for (GitHub, Slack, Stripe, Gmail, Linear), and you should know how it starts (e.g. npm start).

1. Register your agent

Open your coding agent in your agent’s repo and paste:
The coach runs the CLI on your behalf and shows you each command first. When it’s done, your repo has a pome.json you never typed by hand and a registered agent on the dashboard:
  • pome.jsonagent.slug, the real start command, and the twins this agent may exercise. This is the manifest every run reads.
  • A registered agent on app.pome.sh, scoped to your team.
The registered agt_… id is cached in gitignored .pome/link.json, never in the committed manifest. pome.json carries only the portable agent.slug, so forking a repo can’t collide two projects onto one cloud agent.

2. Preflight the wiring

Four checks, in order, stopping at the first red with one named cause and one fix: the manifest validates, the twin boots and serves, your agent’s requests go to the twin rather than a production host still hardcoded in the source, and the deny-by-default egress floor is active. This is a gate, not a suggestion. pome run runs the same checks itself and refuses to start when one fails — there is no --force, because a run whose agent quietly talked to the real API produces a trace that looks complete and grades nothing. pome doctor has the check table; Troubleshooting has what to do with each red.

3. Your first scored run

Two routes to the same thing: a finalized, scored run of your agent. The first is one command. The second is the one you will keep.

The short one: pome run

With no path, pome run copies tasks/first-run-demo.md into your project on first use and runs your agent against it with runs: 5 pinned: five isolated trials of one task, each in a fresh sandbox, so what comes back is a pass rate rather than one lucky attempt. Five trials are five agent evals off your plan’s monthly allowance — each is charged when it is graded at finalize, and Billing semantics is the exact rule; on the free plan a spent allowance refuses the next finalize with 402 rather than billing anything. The file is yours — edit the prompt and criteria freely (runs are scored against the criteria it declares), or delete it and the next bare pome run re-copies the original. It prints the per-trial verdict table and a link to the run on the dashboard. pome run documents the flags, the environment it injects into your agent, and the exit codes.
The demo task is a GitHub triage exam — twins: [github], two [code] criteria over issue #1 in acme/api, and a passThreshold of 100. Two things follow. Your agent needs github among its enabled twins, or the run stops on Requested twins are not enabled on agent … — heal it with pome register agent "<name>" --twins github,… --force (the list is unioned with the manifest’s, never swapped for it; --force is what lets an already-linked repo re-resolve), or enable it on the project’s Services tab. And a low score here is honest rather than broken: an agent that does not triage GitHub issues is being asked the wrong question, which is what the other route below fixes.

The thorough one: a task written for your agent

The demo task is a real exam, but it is not your exam. The coach skills go from your agent’s actual code to a task written against what it could get wrong. Paste this in the same repo:
The twin tape lives in the running sandbox. finalize_run must be called while that sandbox is still live — the installed pome-run-task skill enforces this, and the prompt repeats it because a sandbox that expires first cannot be scored.
Evaluate an agent over MCP is that same loop written out call by call, for when you want to drive it yourself rather than by prompt.

Reading the score

The report is 0–100 across the [code] criteria that were evaluated, and a run passes when it meets the task’s passThreshold. Each verdict carries the evidence pointer it came from — the call in the twin’s tape, or the row of final state, that decided it. Start with the failed criterion and open its evidence before you change anything: the first honest failure is usually a rule the agent was never told, not a bug in its code. [model] criteria come back as prose from the narrator. They are advisory and never enter the denominator, so a run can read badly and still score 100, or the reverse. A task that declares only [model] criteria has an empty denominator and reports narrative only — 0 scored criteria instead of a percentage: if you want a number, the task needs at least one [code] row, and it has to be one the seed leaves at risk.

The fix, declared as a new version

Change the agent, then declare that you changed it. Bump agent.version in pome.json (or pass --agent-version to pome run, or agent_version to run_task) so the fix lands in its own run-set. Without it both runs land in one set reporting 50% clean under “same agent, same prompt”, which reads as unreliability rather than a repair. Two prompts are two things under test. That loop — run, read the failure, fix, re-run — is the product.

Next

Write a task

The task file’s shape, [code] vs [model], and the closed set of checks each twin declares.

Coach skills

The six coach skills, and why a repo-backed agent registers via the CLI.

Use your existing eval stack

Score the same run in Braintrust, LangSmith, or your own harness — every Pome criterion as its own column.

Dashboard

Where your registered agent, its runs, and its run handoffs live: app.pome.sh.