cli-agent-spec/cli-agent-spec

cli-agent-onboard

Onboard a CLI tool for agent evaluation — reads agent docs, detects OS, runtime, toolchain, and binary, discovers non-interactive flags and config.

소스 보기
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

CLI Onboard — Environment Profile

Runtime requirements

  • Requires access to the CLI being onboarded
  • Works on macOS, Linux, and Windows

Build and save a reusable environment profile for a CLI tool.

Inputs

  • CLI — the CLI tool to profile: a command name (e.g. gh), a binary path, or enough context to locate it in the current working directory
  • `--force` (optional) — skip the confirmation prompt in Step 0 and overwrite any existing profile without asking. Used by cli-agent-audit when --refresh is set.

Local Memory Artifact

This skill produces one local artifact:

FileContent
evaluations/<cli-name>/environment.mdOS, runtime, binary, version, non-interactive flags, config env vars

See Step 0 for handling an existing profile.


Step 0 — Check for existing profile

Before doing anything else: check whether evaluations/<cli-name>/environment.md already exists.

  • If it exists and `--force` is set: proceed directly to Step 1 and overwrite on Step 6 — no prompt.
  • If it exists and `--force` is not set: display its contents, then ask the user: "A profile already exists for <cli-name>. Refresh it? (yes/no)". If no: skip Steps 1–6 — the existing profile is ready to use as-is by the calling skill. If yes: proceed to Step 1 and overwrite on Step 6.
  • If it does not exist: proceed directly to Step 1.

Step 1 — Read agent-facing docs

Check for these files in the current working directory and read any that exist:

  1. AGENTS.md — canonical invocation, env vars, non-interactive flags, input conventions
  2. CODING_AGENTS.md — runtime constraints, package manager, language version
  3. README.md — fallback for binary name and install instructions

Step 2 — Detect runtime and toolchain

Identify the project type from manifest files in CWD:

ManifestRuntimePreferred runner
pyproject.toml / setup.pyPythonCheck for uv, poetry, pip; read [project.scripts] for entry point name
package.jsonNodeCheck for pnpm, yarn, npm; read bin field
Cargo.tomlRustLook for compiled binary in target/release/
go.modGogo run . or compiled binary
None of the aboveUnknownTry the CLI name directly on PATH

Step 3 — Locate the binary

Resolve the actual invocable command in this order:

  1. Entry point from manifest ([project.scripts] / bin)
  2. Scripts in .venv/bin/, node_modules/.bin/, target/release/
  3. PATH lookup
  4. Preferred runner prefix (e.g. uv run <entry>, npx <entry>)

Verify with <resolved-command> --version or <resolved-command> --help.

If no manifest is found AND the CLI name is not on PATH: ask the user for the binary location or install command. Do not proceed with an incomplete runtime — the profile requires a verified binary path.


Step 4 — Detect OS constraints

OSConstraintImplication
macOSNo GNU timeout / gtimeoutUse subprocess.run(..., timeout=N) in Python, or perl -e 'alarm(N); exec(...)'
macOSpython not on PATH (Xcode stub)Use python3 or venv-qualified path
LinuxGNU coreutils presenttimeout N <cmd> works
WindowsDifferent shell semanticsUse PowerShell Start-Process with -Wait and timeout

Step 5 — Discover non-interactive flags and config

Run <resolved-command> --help and scan for:

  • Flags that suppress prompts: --yes, --no, --non-interactive, --force, --defaults, --answers
  • Flags that control output format: --format, --json, --output
  • Flags that set input data: --json, --input, --data
  • Environment variables used for config (look in AGENTS.md first, then --help output)

Step 6 — Save the profile

Create evaluations/<cli-name>/ if it does not exist. Save as evaluations/<cli-name>/environment.md:

markdown
# <cli-name> — Environment Profile

**Generated:** <ISO date>

## OS
- Platform: <darwin | linux | win32>
- Version: <uname -r or equivalent>

## Runtime
- Language: <Python | Node | Go | Rust | ...>
- Version: <x.y.z>
- Toolchain: <uv | poetry | npm | pnpm | cargo | ...>

## Binary
- Entry point: <exact command to invoke, e.g. `uv run bean`>
- Version: <output of --version>
- Resolved path: <absolute path if known>

## Non-Interactive Flags
- <flag>: <what it does>

## Output Format Flags
- <flag>: <what it does>

## Config
- <env var>: <purpose>

## Timeout Method
- <`subprocess.run(timeout=N)` | `timeout N` | `perl -e 'alarm(N); exec(...)'`>

## Source
- <which docs were read: AGENTS.md, CODING_AGENTS.md, README.md, pyproject.toml, ...>

Rules

  • Create evaluations/<cli-name>/ if it does not exist before saving the profile
  • Do not overwrite an existing evaluations/<cli-name>/environment.md without user confirmation, unless --force is passed
  • Record only values actually discovered — no placeholders or guesses
  • If a doc file (AGENTS.md, etc.) contradicts what --help shows, note the discrepancy in Source
같은 저장소의 Skills

더 많은 Skills

모든 Skills
cli-agent-spec
커뮤니티

cli-agent-audit

Autonomous end-to-end CLI audit pipeline. Downloads and installs the CLI non-interactively, onboards it, scores proactive readiness, evaluates all Critical failure modes, and generates the full report bundle (dev, agent-dev, runtime, issues, index, LinkedIn post). Single command, zero human steps required in the happy path.

설치 수
1
GitHub Stars
9
업데이트
8월 7일
cli-agent-spec
커뮤니티

cli-agent-diagnose

Classify a failed agent CLI tool call against the CLI Agent Spec §N failure taxonomy. Given a failed command with stdout, stderr, and exit code, identifies the matching failure mode, returns an actionable workaround, and produces a memory string and skill patch to prevent recurrence. Use when a CLI invocation fails and you need to understand why and how to work around it.

설치 수
1
GitHub Stars
9
업데이트
8월 7일
cli-agent-spec
커뮤니티

cli-agent-evaluate

Evaluate a CLI tool against a single CLI Agent Spec failure mode. Runs the failure mode's check, scores 0–3, and provides an applicable agent workaround if the score is below 3. Use this for targeted single-failure-mode evaluation. For multi-failure-mode evaluation use cli-agent-evaluate-batch.

설치 수
1
GitHub Stars
9
업데이트
8월 7일
cli-agent-spec
커뮤니티

cli-agent-evaluate-batch

Evaluate a CLI tool against multiple CLI Agent Spec failure modes in one run. Accepts a severity filter, part number, or an explicit list of §N identifiers. Skips already-evaluated failure modes (resumable). Emits a scorecard table when done.

설치 수
1
GitHub Stars
9
업데이트
8월 7일