사용 방법
- 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
- 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
- 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/owl-listener/ai-design-skills --skill output-quality-rubrics시작 전 확인
- 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
- 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.
Output Quality Rubrics
Without a rubric, quality evaluation is subjective and inconsistent. A rubric defines what "good" means in concrete, measurable terms — so different evaluators reach the same conclusions.
Core Quality Dimensions
- Accuracy: Is the information correct? Are claims verifiable? Are there hallucinations?
- Relevance: Does the output address what the user actually asked? Is everything included necessary?
- Completeness: Does the output cover everything needed? Are there gaps?
- Helpfulness: Can the user actually use this output to accomplish their goal?
- Clarity: Is the output easy to understand? Is it well-structured?
- Tone appropriateness: Does the output match the expected tone for the context?
- Safety: Is the output free from harmful, biased, or inappropriate content?
Building a Rubric
For each dimension, define a scale: Example — Accuracy (1-5):
- 5: All claims are verifiable and correct. No hallucinations.
- 4: Minor inaccuracies that don't affect usefulness. No hallucinations.
- 3: Some inaccuracies that could mislead if not caught. No dangerous hallucinations.
- 2: Significant inaccuracies. User would need to verify most claims.
- 1: Major hallucinations or factually wrong information presented confidently.
Weighting Dimensions
Not all dimensions matter equally for every use case:
- A medical AI weights accuracy and safety highest
- A creative writing AI weights helpfulness and tone highest
- A coding AI weights accuracy and completeness highest
- A customer service AI weights tone and helpfulness highest
Define weights when creating the rubric. Make the priorities explicit.
Rubric Calibration
A rubric is only useful if evaluators use it consistently:
- Anchor examples: Provide sample outputs at each score level
- Calibration sessions: Have multiple evaluators score the same outputs and discuss disagreements
- Inter-rater reliability: Measure agreement between evaluators and refine the rubric until agreement is high
- Edge case guidance: Document how to score ambiguous cases
Design Artefacts
- Scoring rubric with dimension definitions and scales
- Anchor examples at each score level
- Dimension weighting specifications per use case
- Calibration session protocols
- Scoring templates and checklists