見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。
Create a verification skill
A coding agent that changes an app should be able to prove the change the way a user would see it. This skill writes that proof recipe for one project as a skill, .agents/skills/verify-<app>/, built on e2e. The config launches the app, e2e mcp drives it live, tests pin what must hold, e2e explore bug bashes it, and .e2e/ keeps the evidence. Write the output for the next agent, not for a human. It will be read cold, mid-task, by an agent that has never seen the app.
The e2e skill is the reference for everything e2e does. Read it first and link to it rather than restate it. It is .agents/skills/e2e/SKILL.md in the project once npx e2e init or npx skills add tester-army/e2e ran, and npx e2e guide <topic> prints it from the installed package. Topics named below (setup, writing-tests, mcp, explore, bug-bash, running, debugging) are its reference files.
1. Interview the repo, not the user
Answer these from the codebase and ask the user only what you cannot observe.
| Question | Where the answer is | What it decides |
|---|---|---|
| Surface | What a user touches. A web UI, an iOS or Android app, an API | One skill per primary surface; name the others for a later run. Web goes through @e2e-dev/web, iOS and Android through @e2e-dev/mobile. An API with no UI still gets a web() target with app.url and app.command so the runner starts it, and its tests take only app and use fetch (topic writing-tests). A native desktop app has no e2e engine yet, say so and stop |
| Run | The documented dev command, its ports, env vars, seed data, auth | The target's app.command (topic setup). A dev server that compiles a route on first visit reads as broken to an agent, so prefer a production build command when the repo has one |
| e2e state | e2e.config.ts, the tests glob, e2e in package.json, .agents/skills/e2e/, the MCP server in .mcp.json or .cursor/mcp.json, and any Playwright or Cypress suite | Nothing there means npx e2e init first (topic setup), which also installs the skill and registers the server. An existing Playwright or Cypress suite is the best source of locators, seeds, and sign-in steps; read it. A config with no model runs the deterministic steps and the MCP tools, and nothing else |
| Observe | The trace page, screenshots, video, the report, and what the app leaves behind in files, rows, sent messages, logs | The side-effect stores the Evidence section names. Find them now |
| Isolate | Can two instances run side by side? | Port 0 in app.url gives each run its own app; command.reuseExisting: true attaches to one already running on the port. A shared instance that cannot be duplicated (one database, one simulator) becomes a rule in the generated skill. Refusing to double-drive it beats corrupting the user's session |
| Name | The package name or the repo name | <app> in verify-<app>, lowercase, no scope |
If the checkout does not build or start as-is, fix that first or report it precisely before generating. A skill written against a broken base teaches wrong steps. An irrelevant missing asset (a static dir the API never serves, a sample config) may be created by the generated skill, marked as verification scaffolding and removed in Cleanup.
An existing verify-<app>/ is read before anything is written. Keep its SKILL.md and feature files, add the features the map lacks, and change an existing file only where the app contradicts it.
2. Generate the skill
Write .agents/skills/verify-<app>/SKILL.md with YAML frontmatter. name: verify-<app> and a description naming the app, the surface, and when to reach for it; without frontmatter the skill never registers. When the project has .claude/skills/, link it the way e2e init does, a relative symlink .claude/skills/verify-<app> to ../../.agents/skills/verify-<app>. Every section says what the interview found, no placeholders.
| Section | Holds |
|---|---|
| Read first | One line that sends the reader to the e2e skill and names the config file and tests glob this skill drives |
| Launch | The exact command that starts the app for verification and how readiness shows. With a target app.command, launch is npx e2e run <smoke file> or an open_session over MCP; the runner starts the app, waits for the URL, and stops it. Without one, a target with app.url only, the dev command the user starts by hand, and the line that proves the port answers; or keep the command and set reuseExisting: true. Say which ports and env the command needs and what seeds the data |
| Doctor | One read-only check that answers "is this instance worth driving?". npx e2e list loads the config and prints every test-target pair without starting anything, so a config error shows here first. Then the smoke test, a model-free test that opens the app and pins one landmark; exit 0 means the app starts, the engine runs, and the landmark renders. No smoke test in the project: write tests/smoke.e2e.ts now. A signed-in app's auth doctor is its session setup test. An agent runs Doctor first whenever anything looks off |
| Drive | Two lanes and when to use each. Live over MCP: open_session, observe, the verbs, and locate before any locator is written (topic mcp); list the project tools and credentials the session's catalog offers. Scripted: a test under the tests glob, screen where the name on screen is known, agent.act where the flow varies (topic writing-tests). Prefer stable handles, roles and accessible names, labels, test ids, routes. Name the credentials and sessions a signed-in drive starts from |
| Evidence | What a proof keeps and where. Every run writes .e2e/report.json; a failed test gets .e2e/results/<test>/trace.md, --trace on gives one to passing tests too, --video records the attempt (topic running), app.screenshot('<name>') saves a named frame (topic writing-tests). The next run clears .e2e/results/, Doctor included, so a proof that must outlive it runs with --output .e2e/proof/<name> or its attempt directory is copied out before anything else runs. State the proof standards (below) |
| Bug bash | How this app is bashed (topic bug-bash): the bug-bash config to write (what the local app cannot do, the postures, the isolation choice from the interview), the accounts to seed, .e2e/bugbash/ added to .gitignore, and the rule that the charters come from the feature map (section 3). The verification bar: a finding is a bug only once tests/bugbash/<slug>.e2e.ts fails with ASSERTION_FAILED on the assertion that encodes it |
| Cleanup | What to tear down. The runner stops the app it started and close_session stops the one a session started; a server the user started stays theirs. Never kill by process name; stop what you started. Cleanup removes instances and scratch state (seeded accounts, scaffolding, charters whose findings were all rejected), never the evidence; name where the evidence is |
| Helpers | Any script the skill ships is executable and its invocation is in the skill body. A helper the reader has to reverse-engineer is not a helper. Most projects need none; the e2e CLI is the helper |
Proof standards, for the Evidence section: exercise the real user path, not internal setters or test-only endpoints; capture the action and the resulting state, not only the final screen; verify side effects (files written, rows inserted, messages sent) in the stores the interview found; mock only where a production boundary already isolates the external system. When the safe path is a dry run or a test mode, verify what it skips by watching files, network, and refs rather than trusting its name.
3. Seed the feature map
Create .agents/skills/verify-<app>/features/README.md plus one file per user-facing feature you can identify, the top three to five first, from routes, navigation, commands, or docs. Follow references/example/verify-playground/, the skill this generator produced and ran for the e2e playground app. The index is a table of feature, route, test file, and status (tested or untested). Each feature file has these H2s, from the user's point of view.
| H2 | Holds | |||
|---|---|---|---|---|
Sub-features | What the feature is made of, one line each | |||
How to get to it (user POV) | Route, navigation, the account it needs | |||
Driving it with e2e | The test file that pins it, the locators that matter (checked with locate), the MCP steps for a live walk | |||
Proof | The observable end state that means it works, on screen and in the side-effect store | |||
Charters | One or two e2e explore goals for this feature, each one area, one posture, one sentence (topic bug-bash, step 2). Written as the fan-out line `slug\ | target\ | agent\ | charter, so the bug bash pastes them; a charter that starts signed in names its --session` and runs alone |
Gotchas | What reads as broken but is not; what the local app cannot do |
The map is the project's maintained verification source, and the bug bash reads its charters. A proof that drives one convenient entry point is incomplete when the map lists others, and a bug bash that skips a mapped feature reports the gap. A feature without a test gets one in step 4 or is marked untested in the index.
4. Prove the generated skill before handing it over
Run its own instructions end to end once, from a shell in the project.
- Doctor: the
npx e2e listcall the skill names, then the smoke test;
both exit 0.
- Drive one mapped feature through its test file. One is enough; the map
exists so later runs cover the rest. A missing test: write it now from the feature file, with locators checked over MCP.
- Bug bash one charter from the map: `npx e2e explore '<charter>'
--config <bug-bash config> --agent <persona> --max-steps 3 --output .e2e/bugbash/<slug>, plus --session <name> when the charter names one. Read its Findings` and triage; do not verify every finding now. This proves the model, the config, and the output directory, not the app. Without a model, skip this step and say in the handover that the bug bash is unproven.
- Cleanup, then confirm the evidence still exists at the named location.
A cleanup that eats the proof fails this step.
Fix what fails, and run Cleanup after every failed iteration too, so broken attempts do not strand ports and processes. A generated skill that was never executed is a draft, not a deliverable. Leave the project as the user had it. The generated skill, its feature map, the gitignore line, and any test you added are the change; .e2e/ output and the bug-bash config stay untracked unless the user asks otherwise.
5. Hand over
Tell the user what landed. The skill path, the features mapped and which have tests, the charters, what the proof run showed, and the evidence paths. Running the map keeps it honest. When a route or label changes and a test fails, the feature file changes in the same commit. Rerun this skill when the app grows a feature the map lacks; section 1 says how an existing map is treated.
Rules
- Generate from what you observed. A section that could apply to any app
is not finished.
- One sentence from the e2e skill beats a paragraph of your own. Link the
topic.
- Never put a secret in the generated skill or the map. Name the
credential (topic setup).
- Do not commit on the user's behalf. List what you created.
