hamelsmu · Agent Skills
3 Skills with substantive, source-verified documentation. Open a card to read the original instructions and review package and installation details.
evaluate-rag
Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies.
- 1h installs
- 1
- GitHub stars
- 1.7K
- Updated
- Aug 16
validate-evaluator
Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).
- 1h installs
- 1
- GitHub stars
- 1.7K
- Updated
- Aug 16
write-judge-prompt
Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode requires interpretation (tone, faithfulness, relevance, completeness). Do NOT use when the failure mode can be checked with code (regex, schema validation, execution tests). Do NOT use when you need to validate or calibrate the judge — use validate-evaluator instead.
- 1h installs
- 1
- GitHub stars
- 1.7K
- Updated
- Aug 16
