tacoda/tack

tack-eval

Prove the repo's committed context actually makes agents effective — run canary tasks that a fresh agent must complete using only the text files (AGENTS.md, docs, ADRs), measure pass/fail, and feed every failure back into the context in the same PR.

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/tacoda/tack --skill tack-eval

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

tack-eval: prove the context works

"Agent-ready" should be measured, not asserted. A canary task is a small, realistic, verifiable piece of work an agent must complete using only the committed text — no oral context. If agents pass the canaries, the text represents the truth well enough to work from; if they fail, the failure points at exactly which file is stale or missing.

Procedure

  1. Pick canaries from templates/canary-tasks.md — small, realistic, and

mechanically checkable (commands to run, tests that must pass, files that must exist with expected properties).

  1. Run each canary with a fresh agent that sees only the repo (plus its

normal harness). Record pass/fail + where it got stuck.

  1. Every failure is a context gap: fix the SSOT text file in the same PR

(same-commit rule), not the agent's memory.

  1. Re-run the failed canaries until green.
  2. Track the score (passed/total) in the characterization report

(docs/agent-ready/characterization-report.md) — it should trend up. Optionally run a canary subset in CI on a schedule or on changes to AGENTS.md/docs.

Details: references/eval-practice.md. Canary template: templates/canary-tasks.md.

Exit criteria

  • [ ] A canary set exists and is run on demand (and ideally in CI)
  • [ ] Current score recorded in the characterization report
  • [ ] Every known failure has a same-PR context fix or an owned backlog item