affaan-m/ecc

skill-comply

Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines.

ソースを見る

使い方

  1. インストールコマンドをコピーし、自分の環境へ導入する前にソースを確認します。
  2. 必要なツール、認証情報、対応環境を原文で確認します。
  3. 入力と成果物を指定し、出力を資料と照合します。
npx skills add https://github.com/affaan-m/ecc --skill skill-comply

開始前の確認

  • 導入にはモデル契約、API枠、ファイルへのアクセス許可は含まれません。
  • スター数や導入数は活動の指標であり、独立した品質・安全性試験ではありません。
リポジトリの原文

見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。

skill-comply: Automated Compliance Measurement

Measures whether coding agents actually follow skills, rules, or agent definitions by:

  1. Auto-generating expected behavioral sequences (specs) from any .md file
  2. Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
  3. Running claude -p and capturing tool call traces via stream-json
  4. Classifying tool calls against spec steps using LLM (not regex)
  5. Checking temporal ordering deterministically
  6. Generating self-contained reports with spec, prompts, and timelines

Supported Targets

  • Skills (skills/*/SKILL.md): Workflow skills like search-first, TDD guides
  • Rules (rules/common/*.md): Mandatory rules like testing.md, security.md, git-workflow.md
  • Agent definitions (agents/*.md): Whether an agent gets invoked when expected (internal workflow verification not yet supported)

When to Activate

  • User runs /skill-comply <path>
  • User asks "is this rule actually being followed?"
  • After adding new rules/skills, to verify agent compliance
  • Periodically as part of quality maintenance

Usage

bash
# Full run
uv run python -m scripts.run ~/.claude/rules/common/testing.md

# Dry run (no cost, spec + scenarios only)
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md

# Custom models
uv run python -m scripts.run --gen-model haiku --model sonnet <path>

# Long multi-step workflows: raise the per-scenario turn cap and timeout
uv run python -m scripts.run --max-turns 120 --timeout 2400 <path>

# Copy a directory (plans, config, sources) into every scenario sandbox
uv run python -m scripts.run --fixtures ./fixtures <path>
FlagDefaultMeaning
--max-turns N30Turn cap per scenario
--timeout S900Seconds per scenario; on timeout the partial trace is still graded and a warning is logged
--fixtures DIRnoneCopied into each sandbox (links/reparse points rejected, .git skipped case-insensitively at any depth)

Sandbox

Each scenario uses a private per-user directory under the OS temporary directory. This is fixture isolation, not an ECC sandbox tier or a network/credential isolation boundary. It holds the files the scenario lists under files: plus any --fixtures, committed as a baseline. It has a local bare origin with origin/HEAD set, so push and merge steps work offline. A stub gh is first on PATH: it explicitly simulates pr create, pr view, pr list and repo view and fails on other subcommands. Reports label this boundary and any timeout; they cannot establish that a real PR, merge or hosted check succeeded. Auxiliary .bin and .origin.git directories are replaced only when they carry matching tool-owned metadata.

Key Concept: Prompt Independence

Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.

Report Contents

Reports are self-contained and include:

  1. Expected behavioral sequence (auto-generated spec)
  2. Scenario prompts (what was asked at each strictness level)
  3. Compliance scores per scenario
  4. Tool call timelines with LLM classification labels

Advanced (optional)

For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.

同じリポジトリから

関連する Skills

すべての Skills
affaan-m
コミュニティ

rust-testing

Rust testing patterns including unit tests, integration tests, async testing, property-based testing, mocking, and coverage. Follows TDD methodology. Use when writing Rust tests — unit, integration, async, property-based, or coverage.

24時間の導入数
489
GitHub Stars
30.8万
更新日
11月20日
affaan-m
コミュニティ

search-first

Research-before-coding workflow: search npm/PyPI, MCP servers, skills, and GitHub for existing tools before writing custom code, then adopt, extend, or build. Launches the researcher agent for non-trivial needs. Use when starting a feature, adding a dependency or integration, or about to write a utility that may already exist.

1時間の導入数
5
GitHub Stars
30.8万
更新日
11月20日
affaan-m
コミュニティ

browser-qa

Run automated post-deploy UI verification with a browser automation MCP (claude-in-chrome, Playwright, or Puppeteer): console-error and Core Web Vitals smoke checks, form and auth-flow interaction tests, screenshot visual regression across three breakpoints, and axe-core accessibility audits ending in a SHIP / DO-NOT-SHIP verdict. Use when testing a deployed feature on staging or preview, before shipping frontend changes, reviewing a frontend PR, or checking responsive layout and accessibility.

1時間の導入数
2
GitHub Stars
30.8万
更新日
11月20日
affaan-m
コミュニティ

docker-patterns

Docker and Docker Compose patterns for local development, hardened CLI installer harnesses, container security, networking, volumes, and multi-service orchestration. Use when creating or reviewing Dockerfiles and Compose services, testing installers across Linux distributions, or planning accurate native macOS and Windows validation.

1時間の導入数
2
GitHub Stars
30.8万
更新日
11月20日