使い方
- インストールコマンドをコピーし、自分の環境へ導入する前にソースを確認します。
- 必要なツール、認証情報、対応環境を原文で確認します。
- 入力と成果物を指定し、出力を資料と照合します。
npx skills add https://github.com/ai-evals-course/evals-skills --skill evals-start開始前の確認
- 導入にはモデル契約、API枠、ファイルへのアクセス許可は含まれません。
- スター数や導入数は活動の指標であり、独立した品質・安全性試験ではありません。
リポジトリの原文
見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。
Evals Start
This plugin splits eval work into targeted skills. Your job here is small: find the row below that matches the user's situation, tell the user which skill you are loading and why, then load that skill and follow its workflow from start to finish instead of improvising your own version of it.
| Situation | Skill to load |
|---|---|
| Has traces, wants to find failure modes, no established taxonomy yet | error-discovery |
| Has an existing eval pipeline and wants to know if it can be trusted | eval-audit |
| Has a known failure mode that code can check (e.g., format, schema, regex, execution) | write-code-eval |
| Has a known failure mode and wants an LLM judge for it | write-judge-prompt |
| Has an LLM judge or evaluator and wants to check its quality | validate-evaluator |
| Has no traces to review yet | generate-synthetic-data, then error-discovery |
| Wants a custom annotation interface for some other labeling task | build-review-interface |
| Wants to evaluate a RAG pipeline | evaluate-rag |
Most requests that mention error analysis with traces in hand mean error-discovery. New users with an existing pipeline usually need eval-audit first. This file holds only routing. When in doubt about which row fits, ask the user instead of guessing. The workflow lives in the targeted skill.
