tacoda/tack

tack-eval

Prove the repo's committed context actually makes agents effective — run canary tasks that a fresh agent must complete using only the text files (AGENTS.md, docs, ADRs), measure pass/fail, and feed every failure back into the context in the same PR.

查看源码

使用方法

  1. 复制安装命令,在自己的智能体环境安装前先检查源码包。
  2. 从原始文档核对所需工具、凭据和支持的宿主。
  3. 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/tacoda/tack --skill tack-eval

开始前需了解

  • 安装不包含模型订阅、API 额度或访问文件的权限。
  • 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
仓库原始内容

按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。

tack-eval: prove the context works

"Agent-ready" should be measured, not asserted. A canary task is a small, realistic, verifiable piece of work an agent must complete using only the committed text — no oral context. If agents pass the canaries, the text represents the truth well enough to work from; if they fail, the failure points at exactly which file is stale or missing.

Procedure

  1. Pick canaries from templates/canary-tasks.md — small, realistic, and

mechanically checkable (commands to run, tests that must pass, files that must exist with expected properties).

  1. Run each canary with a fresh agent that sees only the repo (plus its

normal harness). Record pass/fail + where it got stuck.

  1. Every failure is a context gap: fix the SSOT text file in the same PR

(same-commit rule), not the agent's memory.

  1. Re-run the failed canaries until green.
  2. Track the score (passed/total) in the characterization report

(docs/agent-ready/characterization-report.md) — it should trend up. Optionally run a canary subset in CI on a schedule or on changes to AGENTS.md/docs.

Details: references/eval-practice.md. Canary template: templates/canary-tasks.md.

Exit criteria

  • [ ] A canary set exists and is run on demand (and ideally in CI)
  • [ ] Current score recorded in the characterization report
  • [ ] Every known failure has a same-PR context fix or an owned backlog item