사용 방법
- 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
- 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
- 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/lilmgenius/paperthin --skill mandela시작 전 확인
- 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
- 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.
Audit whether a validation receives independent outside ground-truth.
Goal
The Mandela Effect names a population confidently remembering something that never independently happened. A leaky validation has the same shape: designer, model, and scorer confirm each other. mandela uses eight leakage patterns to check whether a validation is independent.
Workflow
- Identify the validation: eval, metric, experiment, holdout, or "how we'll know". Name its components: model, scorer, designer, and dataset.
- Ask the core question: does external ground-truth enter independently?
- Test all 8 patterns below. Some apply only to particular components, such as a human subject or a scorer. Report only patterns that fire, each by name.
- Recall, not reason: a memorized answer recited instead of one actually derived; the system already knows the result it is supposedly computing.
- Wrong null hypothesis: an ablation that removes a surface label but not the underlying signal the system actually exploits, so the "control" still leaks.
- Shared hallucination: two components verifying each other; circularity reported as a number.
- Tautology: a scorer grading buckets it drew itself.
- Verifier = designer: a private, unreproducible recipe in a holdout's clothes.
- Shared-pool bias: train and holdout drawn from one labeler pool, so one bias enters both sides.
- Frame injection: a question that hands the subject the hypothesis.
- Demand characteristics: measured subjects who know they're being measured.
- Give the independent-ground-truth fix for each hit.
Rules
- Blinding the output value can still leak the collection recipe.
- Read-only: name the leak and the independence fix; don't rewrite the experiment.
- For a high-stakes validation, you may add one independent auditor with fresh context (
N=1). Give it only the validation design, without this session's reasoning, to rerun the eight patterns blind. The default remains same-session and read-only.
Verification
Turn mandela on this audit:
- Run patterns #3 through #5 on yourself: are you a scorer grading buckets you drew (Tautology, #4)? Is the verifier the designer (#5)? Is your verdict a shared hallucination with the design's own claims (#3)?
- Could a reader who didn't run the audit reach your verdict from the cited evidence alone, using independent ground-truth?
- The report names the root, not a laundry list.