Skip to main content

​
Orientation

​
get_started

Call this first. Returns the coach playbook. Inputs: none. Returns: the coach playbook (markdown).

​
get_platform_context

Confirms identity, team, plan tier, and quota usage before doing anything else. Inputs: none. Returns: { team, me, usage, note } — resolved identity, team, plan tier, quota usage. Quota trips are HTTP 402, never 429.

​
list_twins

Returns the hosted digital-twin catalog. Inputs: none. Returns: the hosted digital-twin catalog. Today the hosted MCP catalog surfaces the GitHub and Slack digital twins; more twins are reachable via the CLI (pome twin start).

​
Authoring

​
list_scenarios

Returns the team’s scenario catalog with grading criteria visible to the coach. Inputs: none. Returns: the team’s scenario catalog with prompt, setup, and grading criteria. An empty list is not an auth error.

​
validate_task

Grammar-only validation. Creates nothing. Inputs: task_source (scenario markdown, max 256 KiB). Returns: { valid, title, twins, timeout_seconds, criteria_count, criteria_kinds: { code, model }, criteria[] } or { valid: false, issue }.

​
save_scenario

Validates then upserts a scenario by name. Inputs: scenario_source (required), name (optional; defaults to the # heading). Returns: { scenario_id, name, source_hash, ... }.

​
evaluate_criteria

Dry-runs the deterministic [code] criteria against the seed’s initial state without running the examinee — it boots the twin in-process. [model] criteria are returned as informational. Inputs: exactly one of scenario_id or scenario_source. Returns: { twins, has_seed_state, deterministic, model }.

​
verify_seed

Seed sanity check: flags any deterministic criterion that already passes on the seed’s initial state — a “broken seed.” Inputs: exactly one of scenario_id or scenario_source. Returns: { no_criteria_prepass, has_seed_state, already_passing, deterministic, model, notes, verdict }.

​
register_agent

Lightweight registration of the agent under test. Idempotent on slug. Inputs: name (required), slug (optional), twins (optional allowlist), transport (rest | mcp). Returns: { agent } — use agent.id as run_scenario’s agent_id.

​
intake_clone_scope

Registers the agent and captures its clone scope so run_scenario can assemble the examinee. Mints a durable twin façade token, returned once. Inputs: slug (required), plus optional display_name, mcp_servers[] ({ name, twin }), env_packages[], memory_policy, initial_events[], transport. Returns: { agent }.

​
list_agents

Lists registered agents. Inputs: none. Returns: registered agents (id, slug, display name, framework, judge model, twins, clone scope).

​
Run lifecycle

​
run_scenario

Provisions one isolated twin sandbox per twin, captures pristine initial state, and returns the assembled examinee_launch spec. Does not launch the clone — the coach does. Retrying with the same idempotency_key replays the same session.
Set mcp_permission_policy { type: "always_allow" } on every examinee MCP toolset, or the headless clone deadlocks on its first MCP call.
Inputs: one of scenario_id or scenario_source, optional agent_id, group_id, idempotency_key. Returns: examinee_launch (transport, instructions, mcp_servers[] with per-session twin URLs https://twins.pome.sh/<twin>/s/<sid>/mcp, mcp_permission_policy { type: "always_allow" }, rest_urls, env with POME_<TWIN>_REST_URL / POME_AUTH_TOKEN, network allowlist), plus session_id, expires_at, agent_token, examinee_task, and how_to_finalize.

​
run_trials

Batch of N trials of one scenario sharing a group. Not idempotent. Inputs: one of scenario_id or scenario_source, optional agent_id, n (1-10), group_id. Returns: { group_id, requested, provisioned, trials[], errors[], note }. A 402 quota trip stops further provisioning but returns what was provisioned.

​
list_sessions

Recent twin sessions with state and expiry. Inputs: optional limit (1-200, default 50). Returns: recent twin sessions with state and expiry.

​
get_session

Side-effect-free session probe. Use when twin URLs start 404ing. Inputs: session_id. Returns: { state, expires_at, twin_activity_detected, idle_seconds, activity, verdict }.
Never finalize on idle_seconds alone.

​
stop_session

Stops a session early and frees the concurrency slot without evaluating. Unrecoverable. Inputs: session_id. Returns: confirmation of the stop.

​
finalize_run

Grades synchronously: pulls the recorded twin tape, scores it, tears down the sandbox. Must be called before the session expires. One evaluation per run. Inputs: session_id, agent_token (required), optional scenario_id or scenario_source (inferred from the session if omitted), agent_model, agent_sdk. Returns: { run, note } — the score is in run directly, no polling.

​
check_evaluation

Polls a legacy async evaluation job. Not needed for the default synchronous finalize_run. Inputs: session_id. Returns: job state — queued | running | completed | failed.

​
get_report

Scored evaluation report as markdown, plus a weblink for humans. Inputs: run_id. Returns: the scored evaluation report as markdown (criteria results, judge narrative, handoff), plus the app.pome.sh/runs/<id> weblink.

​
list_runs

Lists evaluation runs newest-first. Inputs: optional limit, group_id, task_name. Returns: evaluation runs newest-first, plus a dashboard link.