pome init scaffolds one, and pome tasks <twin> --copy gives you runnable examples to start from.
Two kinds of criterion
[code] is graded by running a check against the twin’s own state, or against the calls the twin recorded. Deterministic: the same run scores the same way every time, with no model in the loop.
[model] is graded by the LLM judge reading the run’s trace. Plain English, for the judgment calls — tone, reasoning, whether an explanation actually holds up.
Pick the check; let Pome write the sentence
GitHub declares a closed set of checks — 12 in total today. You choose one and fill its parameters; Pome renders the English into the file. The sentence and the check cannot disagree, because the sentence is what the check produced. Read the set from a terminal:list_checks returns the same set the grader binds against, and save_task takes structured picks instead of prose:
When you pass structured
criteria, Pome writes the ## Success Criteria section for
you — so your task source must not already contain one.What a check reads
Every check declares its substrate: the material it is allowed to look at. That is not a detail — it decides which questions the check can answer at all.final — The state the twin exports when the run ends. Most checks read this. It answers “is the world how it should be?” and nothing about how it got there.
seed+final — A delta: it compares the world the task seeded against the world the run left behind. Without the seed the question is unanswerable, so the engine reports skipped / seed_missing rather than guessing — guessing would hand a negative criterion a free pass on exactly the run it exists to catch.
tape — The ordered record of the calls the twin answered, scoped to that twin. Some assertions the final state cannot see at all: a call that was rejected, or one that changed nothing, leaves no trace in the state it did not modify.
GitHub
github declares 12 checks. A [code] criterion on this twin must be one of them.
sha256:ae84a69c55fe1ea5560be9c4638e3596081979e890336e32335d80bbbbba4694. list_checks returns it alongside the set, so a consumer can tell whether it is holding the same vocabulary the grader binds against.
github.issue-exists
Asserts an issue with this number is present in the repository’s final state. It says NOTHING about the issue’s content, state, labels or assignee — pair it with those checks when they matter. Its natural use is a task whose examinee must CREATE the issue; asserting the existence of a seeded issue is trivially true and grades nothing.
github.issue-state
Compares the issue row’s state column against the named state. A missing issue FAILS; an issue whose export carries no state at all is SKIPPED rather than judged, because absent is not the same as open. The open form is a prohibition — it asks the examinee NOT to close the issue — which is why polarity is read from the state word.
github.issue-has-label
Asserts the label is among those APPLIED to the issue — it does not assert the issue carries only that one. An agent that applies the right label alongside three wrong ones passes this check; github.issue-exactly-one-label is the assertion that catches that. The comparison is case-insensitive, because GitHub creates label names case-insensitively while preserving the caller’s display casing.
github.issue-exactly-one-label
Asserts the issue carries EXACTLY ONE applied label and that it is this one. Strictly stronger than github.issue-has-label: it fails an agent that piles a correct label on top of an incorrect one, which is the defect a triage task usually exists to catch. It counts every applied label, not only ones a human would call a classification.
github.issue-assignee
Asserts this login is among the issue’s assignees. GitHub issues can carry several, so this does not assert sole ownership. It compares LOGINS exactly and case-sensitively, not display names — alice matches the collaborator alice, and Alice Smith matches nothing.
github.issue-comment-contains
Scans the bodies of every comment on the issue for this text as a SUBSTRING, case-sensitively. It does not assert who commented, how many did, or where in the body the text sits. Because the text is hunted inside free prose rather than compared to a field, a redaction rule that destroys it makes this check unable to fire — the engine skips it as subject_redacted rather than passing it vacuously.
github.no-new-labels
Compares the repository’s label DEFINITIONS in the seed against the final state. Applying an ALREADY-DEFINED label to an issue PASSES this check — only creating a label the repo did not already define fails it. addIssueLabels rejects an undefined label, so an examinee cannot apply a new one without creating it first, which is what makes this tight. Needs the seed: it is a delta, not a state assertion.
github.pr-state
Reads the pull request’s merged flag for merged/not merged, and its state column for open/closed. These are DIFFERENT fields and a PR can be closed without being merged, so the two pairs do not imply each other. Whichever field the sentence turns on must be present: an export missing it is SKIPPED, because defaulting it to false would let is not merged pass against a world we cannot see.
github.pr-review-exists
Asserts at least one submitted review on the pull request carries this state. It does not assert who reviewed, how recently, or that no other review disagrees — an APPROVED review alongside a CHANGES_REQUESTED one satisfies both. A pull request whose export carries no reviews section at all is SKIPPED, because absent is not the same as none.
github.file-exists
Asserts a file with this exact path exists in the repository on ANY branch — the twin exports files per branch and this check does not distinguish them, so a file committed only to a side branch satisfies it. The path is compared exactly and case-sensitively; it asserts nothing about the file’s contents.
github.commit-status
Asserts at least one commit status reported under this context carries this state. It does not say WHICH commit: the twin exports every status row for the repo and this check scans them all, so a green status on an old commit satisfies it. Nor does it assert the status is the latest one for that context.
github.no-unsupported-endpoint
Scans the recorded call tape for any request the twin answered with fidelity “unsupported” — a route it does not implement, answered 501. It asserts nothing about whether the run SUCCEEDED, and nothing about calls that were merely rejected: a 404 or a 422 from a route the twin does implement is a semantic answer and passes this check. The tape is scoped to this twin by the engine before the check sees it, so an unsupported call to a DIFFERENT twin in a multi-twin session cannot fail it.
Twins without a published vocabulary
stripe, slack, gmail, linear do not declare a closed set yet. Their [code] criteria still grade — the engine carries per-twin phrase rules for them — but those rules are not published as a vocabulary you can author from, so list_checks returns an empty set and save_task can tell you a sentence does not bind without being able to show you the alternatives. Until they land, start from a criterion in a working task (pome tasks <twin> --copy) rather than composing a new sentence.