Before you start
- The graded capstone, done once. Two commands and one paste-prompt on a bundled example. Every step the coach takes there repeats on your own agent here, so you will recognize each one; skip it and the first unfamiliar thing on this page is also the first thing you have to debug.
-
Its setup, also done once: the coach skills installed
(
npx skills add pome-sh/digital-twins --skill '*') and the Pome MCP added athttps://mcp.pome.sh/mcp. Connect to the MCP carries the wiring for each client. -
The Pome CLI, which writes and reads the manifest:
pome loginopensapp.pome.sh, signs you in, and stores apme_…key in your macOS Keychain (or~/.pome/credentials.json). CI can setPOME_API_KEYinstead. -
Your agent’s repo, open in your coding agent. It should talk to a service
Pome has a twin for (GitHub, Slack, Stripe, Gmail, Linear), and you should
know how it starts (e.g.
npm start).
1. Register your agent
Open your coding agent in your agent’s repo and paste:pome.json you never typed by hand and a registered
agent on the dashboard:
pome.json—agent.slug, the real startcommand, and thetwinsthis agent may exercise. This is the manifest every run reads.- A registered agent on
app.pome.sh, scoped to your team.
The registered
agt_… id is cached in gitignored .pome/link.json, never in
the committed manifest. pome.json carries only the portable agent.slug, so
forking a repo can’t collide two projects onto one cloud agent.2. Preflight the wiring
pome run runs the same
checks itself and refuses to start when one fails — there is no --force,
because a run whose agent quietly talked to the real API produces a trace that
looks complete and grades nothing. pome doctor has the
check table; Troubleshooting has what to do with each
red.
3. Your first scored run
Two routes to the same thing: a finalized, scored run of your agent. The first is one command. The second is the one you will keep.The short one: pome run
pome run copies tasks/first-run-demo.md into your project on
first use and runs your agent against it with runs: 5 pinned: five
isolated trials of one task, each in a fresh sandbox, so what comes back is a
pass rate rather than one lucky attempt. Five trials are five agent evals
off your plan’s monthly allowance — each is charged when it is graded at
finalize, and Billing semantics is the exact rule; on the
free plan a spent allowance refuses the next finalize with 402 rather than
billing anything. The file is yours — edit the prompt and criteria freely
(runs are scored against the criteria it declares), or delete it and the next
bare pome run re-copies the original.
It prints the per-trial verdict table and a link to the run on the dashboard.
pome run documents the flags, the environment it injects into
your agent, and the exit codes.
The demo task is a GitHub triage exam —
twins: [github], two [code]
criteria over issue #1 in acme/api, and a passThreshold of 100. Two things
follow. Your agent needs github among its enabled twins, or the run stops on
Requested twins are not enabled on agent … — heal it with
pome register agent "<name>" --twins github,… --force (the list is unioned
with the manifest’s, never swapped for it; --force is what lets an
already-linked repo re-resolve), or enable it on the project’s Services tab. And
a low score here is honest rather than broken: an agent that does not triage
GitHub issues is being asked the wrong question, which is what the other route
below fixes.The thorough one: a task written for your agent
The demo task is a real exam, but it is not your exam. The coach skills go from your agent’s actual code to a task written against what it could get wrong. Paste this in the same repo:Reading the score
The report is 0–100 across the[code] criteria that were evaluated, and a run
passes when it meets the task’s passThreshold. Each verdict carries the
evidence pointer it came from — the call in the twin’s tape, or the row of final
state, that decided it. Start with the failed criterion and open its evidence
before you change anything: the first honest failure is usually a rule the agent
was never told, not a bug in its code.
[model] criteria come back as prose from the narrator. They are advisory and
never enter the denominator, so a run can read badly and still score 100, or the
reverse. A task that declares only [model] criteria has an empty
denominator and reports narrative only — 0 scored criteria instead of a
percentage: if you want a number, the task needs at least one [code] row, and
it has to be one the seed leaves at risk.
The fix, declared as a new version
Change the agent, then declare that you changed it. Bumpagent.version in
pome.json (or pass --agent-version to pome run, or agent_version to
run_task) so the fix lands in its own run-set. Without it both runs land in
one set reporting 50% clean under “same agent, same prompt”, which reads as
unreliability rather than a repair. Two prompts are two things under test.
That loop — run, read the failure, fix, re-run — is the product.
Next
Write a task
The task file’s shape,
[code] vs [model], and the closed set of checks
each twin declares.Coach skills
The six coach skills, and why a repo-backed agent registers via the CLI.
Use your existing eval stack
Score the same run in Braintrust, LangSmith, or your own harness — every
Pome criterion as its own column.
Dashboard
Where your registered agent, its runs, and its run handoffs live: app.pome.sh.