owl-listener/ai-design-skills

output-quality-rubrics

Defining what "good" looks like for AI outputs — accuracy, relevance, helpfulness.

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/owl-listener/ai-design-skills --skill output-quality-rubrics

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

Output Quality Rubrics

Without a rubric, quality evaluation is subjective and inconsistent. A rubric defines what "good" means in concrete, measurable terms — so different evaluators reach the same conclusions.

Core Quality Dimensions

  • Accuracy: Is the information correct? Are claims verifiable? Are there hallucinations?
  • Relevance: Does the output address what the user actually asked? Is everything included necessary?
  • Completeness: Does the output cover everything needed? Are there gaps?
  • Helpfulness: Can the user actually use this output to accomplish their goal?
  • Clarity: Is the output easy to understand? Is it well-structured?
  • Tone appropriateness: Does the output match the expected tone for the context?
  • Safety: Is the output free from harmful, biased, or inappropriate content?

Building a Rubric

For each dimension, define a scale: Example — Accuracy (1-5):

  • 5: All claims are verifiable and correct. No hallucinations.
  • 4: Minor inaccuracies that don't affect usefulness. No hallucinations.
  • 3: Some inaccuracies that could mislead if not caught. No dangerous hallucinations.
  • 2: Significant inaccuracies. User would need to verify most claims.
  • 1: Major hallucinations or factually wrong information presented confidently.

Weighting Dimensions

Not all dimensions matter equally for every use case:

  • A medical AI weights accuracy and safety highest
  • A creative writing AI weights helpfulness and tone highest
  • A coding AI weights accuracy and completeness highest
  • A customer service AI weights tone and helpfulness highest

Define weights when creating the rubric. Make the priorities explicit.

Rubric Calibration

A rubric is only useful if evaluators use it consistently:

  • Anchor examples: Provide sample outputs at each score level
  • Calibration sessions: Have multiple evaluators score the same outputs and discuss disagreements
  • Inter-rater reliability: Measure agreement between evaluators and refine the rubric until agreement is high
  • Edge case guidance: Document how to score ambiguous cases

Design Artefacts

  • Scoring rubric with dimension definitions and scales
  • Anchor examples at each score level
  • Dimension weighting specifications per use case
  • Calibration session protocols
  • Scoring templates and checklists
같은 저장소의 Skills

더 많은 Skills

모든 Skills