Mode d’emploi
- Copiez la commande et examinez le paquet avant de l’installer dans votre environnement agent.
- Consultez le document original pour les outils, identifiants et environnements requis.
- Décrivez les entrées et le livrable souhaité ; comparez la sortie aux sources.
npx skills add https://github.com/tacoda/tack --skill tack-evalAvant de commencer
- L’installation n’inclut ni abonnement au modèle, ni quota API, ni accès aux fichiers.
- Étoiles et installations reflètent l’activité, pas un test indépendant de qualité ou de sécurité.
Rendu depuis le dépôt source en conservant titres, exemples, code, tableaux, liens et images.
tack-eval: prove the context works
"Agent-ready" should be measured, not asserted. A canary task is a small, realistic, verifiable piece of work an agent must complete using only the committed text — no oral context. If agents pass the canaries, the text represents the truth well enough to work from; if they fail, the failure points at exactly which file is stale or missing.
Procedure
- Pick canaries from
templates/canary-tasks.md— small, realistic, and
mechanically checkable (commands to run, tests that must pass, files that must exist with expected properties).
- Run each canary with a fresh agent that sees only the repo (plus its
normal harness). Record pass/fail + where it got stuck.
- Every failure is a context gap: fix the SSOT text file in the same PR
(same-commit rule), not the agent's memory.
- Re-run the failed canaries until green.
- Track the score (passed/total) in the characterization report
(docs/agent-ready/characterization-report.md) — it should trend up. Optionally run a canary subset in CI on a schedule or on changes to AGENTS.md/docs.
Details: references/eval-practice.md. Canary template: templates/canary-tasks.md.
Exit criteria
- [ ] A canary set exists and is run on demand (and ideally in CI)
- [ ] Current score recorded in the characterization report
- [ ] Every known failure has a same-PR context fix or an owned backlog item