shiplight

Shiplight

The single entry point for Shiplight QA work. This skill takes a subcommand and routes it to the right workflow. Everything Shiplight-branded comes through here; the description above is deliberately gated so this skill fires only when the user names "shiplight" or types /shiplight — never on a generic "write a test".

Routing contract

1. Identify the subcommand. Match the argument against the dispatch table —

do not rely on the first token alone, because synonyms are often multi-word (yaml test, set up tests for my app) and intents may lead with a non-token word (create yaml test, show failing tests). In order: (a) an exact canonical token as the leading word; (b) the longest canonical-token or synonym phrase the argument contains; (c) overall intent against the synonym/intent column. Pick the single best-matching subcommand.

1. Pass-through context. Forward whatever the selector didn't consume to the

subcommand as context. Natural phrasing is expected — users won't type the exact hyphenated token; match the intent and treat the rest as the target. Examples:

• /shiplight cover checkout flow → cover + context checkout flow
• /shiplight create a yaml test for login → create-yaml-tests (matched on

"yaml test") + context for login

• /shiplight create agent verification for the signup flow →

create-agent-verification (matched on "agent verification") + context for the signup flow

1. Dispatch. Read the matching references/<subcommand>.md (or

references/<subcommand>/index.md for nested subcommands) and follow it, carrying the context forward.

1. Clarify, don't guess. If the selector is empty (/shiplight alone) or

ambiguous (see Ambiguity notes), show the menu and ask one clarifying question — the user wants to act but didn't say how. This differs from help, which is informational: help lists/explains subcommands and never executes (see references/help.md).

1. Confirm destructive actions. Never auto-run init against a non-empty

project — confirm first.

Shared layer

• On every subcommand invocation (skip for help and support), identify the test project

root, then run references/_shared/update-check.md once (daily skill refresh + shiplightai CLI version gate). It can halt the run: a CLI behind the latest published version stops the subcommand before it starts, and offers an upgrade.

• Each subcommand names the references/_shared/ modules it needs (auth, mcp,

evidence-and-report, project-layout, ground-truth, knowledge, secrets, vocabularies, test-spec-template). Read those before acting — they are the single source of truth, not restated per subcommand.

Subcommands (menu)

Show this grouped menu when invoked bare or when clarifying.

Setup

• init — scaffold a Shiplight test project + write specs/context.md
• setup-test-auth — configure login for the application under test and save reusable browser/Playwright state
• update — refresh installed Shiplight skills + the shiplightai CLI

Author

• create-yaml-tests — implement deterministic YAML E2E tests from a spec
• create-agent-verification — create a reusable agent-run verification script
• cover — decide test format + effort, plan, drive the producers, report

Maintain

• fix — reproduce and repair failing or drifted tests

Check

• verify — verify UI changes in the browser during local development

Review

• review — app-quality review (security, privacy, design, performance, …)

Ship

• ci — wire CI workflows + failure-triage pipeline
• cloud — read Shiplight Cloud test results (runs, failing/flaky tests, artifacts) and analytics (health summary, pass-rate/run trends, slowest/flaky rankings, failure attribution)

Help

• help — list subcommands, or help <subcommand> for details (does not execute)
• support — get human help: file a support ticket with session diagnostics, check replies

End the menu with one footer line: Stuck? /shiplight support reaches a human — the ticket drafts itself from this session.

Dispatch table

Ambiguity notes

• "test" / "write a test" → could be create-yaml-tests (deterministic, UI

focused) or create-agent-verification (spans UI and backend state). Default to create-yaml-tests unless the user signals a core journey, backend/ cross-layer proof, release smoke, or live-env judgment — but if unclear, ask.

• "smoke test" → ambiguous alone. A quick UI pass over key screens is

create-yaml-tests; a pre-release check that a core journey works and left the right backend state is create-agent-verification. Ask which, unless the user names a backend expectation or a release gate.

• "verify" / "verification" → the verb (check a change now) is verify;

creating a reusable verification script is create-agent-verification. The create- framing is the tell. Ask if the user's phrasing doesn't disambiguate.

• "triage" → in Shiplight this means fix (repair failing tests). Do not

confuse with review's internal triage/plan step.

• "failing tests" / "flaky tests" → reading them from CI ("in the cloud",

"from the last run", plural reporting) is cloud (hosted results); repairing a broken test ("my test is failing", "fix this") is fix. Ask if the phrasing doesn't say which.

• "report a bug" / "X is broken" → depends on what is broken. The user's

app misbehaving is ground truth to report (_shared/ground-truth.md), not a subcommand; Shiplight itself misbehaving (skill, CLI, cloud API) is support. Ask if unclear which one the user means.

• "login" / "auth" → ask what is being authenticated when the target is

absent. Authentication for the application under test is setup-test-auth; Shiplight platform authentication and SHIPLIGHT_API_TOKEN setup is the CLI command npx shiplight setup-api-token, not a /shiplight subcommand.

After a subcommand completes or aborts (next-step suggestion)

After a subcommand's final report — including the report of an aborted run — optionally append one next-step suggestion. Rules:

• Evidence-only. A suggestion must be triggered by something already

observed during the run — the diff analyzed, the failure diagnosed, the project state read. Never run extra analysis (a new git diff, file scan, or browser session) just to decide a suggestion.

• Silence is the default. No trigger from the table → no suggestion line at

all. Failure states mostly suggest nothing: the user's next step is fixing the product, not another Shiplight command. Suggestions fire on success — "you proved it works; now make that durable / continuous / visible." The one failure that does fire is the Shiplight-side failure row (support), and it fires wherever the run stops — a mid-run abort report counts.

• Gate on the nature of the change. When the run started from a code change

that is backend-only or barely touches UI, suppress the browser-flavored suggestions (verify, create-yaml-tests, review design); cover (picks unit/contract/integration via its capability map) and create-agent-verification (cross-layer live checks) are the useful pointers there.

• One line, statement not question. Format:

Next: /shiplight <cmd> — <reason from this run>. Never auto-run the suggested command; never ask a blocking yes/no. The user decides.

• At most one suggestion — two only when genuinely forked (e.g. cover vs

create-yaml-tests by scope).

• Skip entirely in CI / non-interactive mode, and after update / help /

support.