gnibu/bento

bento-hillclimb

Tune anything measurable — retrieval quality, latency, eval scores, a prompt — by disciplined one-variable-at-a-time experiments with a kept/reverted log.

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/gnibu/bento --skill bento-hillclimb

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

Hillclimb

The failure mode: change three things at once, the number moves, you don't know which tweak did it — or whether it was noise. Next week it drops and you can't retrace. One variable at a time, logged, keep-or-revert.

Repo layer first. Read the repo's AGENTS.md/CLAUDE.md and, if it exists, the overlay .agents/bento/bento-hillclimb.md at the repo root; follow it under the overlay rules.

Steps (paste as todos)

  1. Freeze the harness first. Fix the dataset, the metric, and how it's measured

before touching the thing you're tuning. If the ruler moves, no result means anything.

  1. Record the baseline. Run the frozen harness once. Write the number down.
  2. Change ONE variable. One knob per experiment. Never stack untested changes.
  3. Measure against baseline. Same harness, same input set.
  4. Keep or revert — and log it either way. Keep only a real gain (significant, not

noise; re-run if the sample is small). Log the attempt, the delta, and the decision. A reverted attempt is still knowledge — write it down so you don't retry it.

  1. New baseline on a keeper, then back to step 3.
  2. Don't stop on a lucky first hit. Run enough iterations that one good roll can't

fool you. State the stop condition up front (target metric, or N no-improvement rounds).

Guardrails

  • Regression check on every keeper: a gain that breaks something else is not a gain.
  • If two knobs seem coupled, that's a finding — still change one, note the interaction.
  • The log is the deliverable as much as the final number. No log = you learned nothing

transferable.

관련 작업