observe-it

Observe It

Make important behavior visible in production without drowning in noise or leaking sensitive data.

Match signals to the quality regime (handbook concepts/11-quality-regimes.md):

Operating contract

Shared by every DecisionNerd/dev-skills skill. Canonical copy: handbook/concepts/14-operating-contract.md.

• Define done first. Before using tools, write the finish line in one or two lines: acceptance criteria (existing issue AC, BDD scenarios, tests, or contract; otherwise propose them and where they should live) and the evidence that will prove them. Re-check it before reporting done. Never report done on unverified work.
• Requested scope sets the finish line. A question ("is it ready?", "why is it broken?", "what next?") ends with the answer and a Next: line naming the exact next invoke. An outcome request ("fix", "finish", "land", "#42") continues through the chain (diagnose → fix → test → check-readiness → merge-it) until the outcome or a real blocker. Do not end a turn with "Do you want me to…?" for in-scope, in-repo work.
• Stop only for real blockers. Stop and ask only when you cannot continue without the user, or before: deleting data or unmerged work, force-push or history rewrite, changing anything outside this repository (GitHub objects, deployments, live data, production or paid resources, external services), or leaving the requested scope, unless the user's request already named that exact action. Keep the harness's permission prompts for risky commands. Otherwise keep going and put status notes in the same message as the next action.
• Ask well, once. For a genuine question use the harness's structured question tool when it has one (Claude Code: AskUserQuestion; Codex: request_user_input when the current mode supports it; T3 Code: the question tool) with concrete options; otherwise plain text with numbered options. Treat the answer as settled; do not re-open earlier verdicts, plans, or answers unless asked.
• Fan out when work is parallel. Use subagents for independent reads (repo survey, evidence gathering, per-option research, per-area audits) and independent verification (a reviewer that did not write the change). Writes stay single-owner per path set and sequential. Brief every child with goal, done-when, constraints, must-not, and return shape; verify each result before consolidating. Use Claude Code's Workflow tool only when orchestrating many subagents truly needs it; it is expensive. T3 Code: its delegated subagents — with optional worktree isolation for independent verification — cover this without the Workflow tool.
• Pick the model tier per child; defer to routing config. If the harness or user config already routes subagents, follow it and do not pass a model. Otherwise pick a tier per child: mechanical search or inventory → fast/cheap; implementation and evidence gathering → mid; planning, review, adversarial verification → top. Tier names differ per harness (Claude Code: haiku / sonnet / opus or fable; T3 Code: inherit by default or pick from its live provider/model catalog). Set the choice with the harness's spawn model param or agent frontmatter model: (per-harness details: skills/agents/references/sub.md). Where the harness cannot choose, children inherit or the harness picks; say which in the status note.
• Keep a checklist on long runs. For more than about five steps or work that crosses skills, keep TASKS.md at the repo root and tick items as they finish. Do not commit it unless the repo already tracks one.
• Close every run with three headings. Blocked on me (the one genuine question or blocker, else "none"); Changed (files, commits, GitHub objects, deploys, else "nothing"); Found (evidence, verdict, and Next: <exact invoke>).

When to use

• "Add logging/metrics/tracing", "observe it", "how do we detect this failing?", "add Langfuse/evals online"
• After diagnose-bug / troubleshoot-app / agents analyze when the root cause was invisible
• Before merge-it Stage/Ship phases when release risk needs signals

When NOT to use

• Fixing a known bug without needing new signals → fix-it / diagnose-bug
• Proving behavior in CI → test-it (obs complements tests; does not replace them)
• Designing the agent itself → agents design|optimize

Workflow

Done when: each named question is answerable from an emitted signal verified locally or in staging.

1. Name the regime and the questions ops/product must answer

• e.g. success rate, latency, auth denials, queue lag, LCP regression, eval score drift, funnel drop-off.

1. Discover existing stack

• Read docs, .env* names, SDK imports, dashboards, alert rules, Langfuse/OTel exporters.
• Reuse current logger/tracer/metrics/analytics/eval plane — don't add a second system.

1. Instrument the smallest useful surface

• Logs: structured fields, correlation/request/job/trace IDs, error cause chains.
• Metrics: counters/histograms with low-cardinality labels.
• Traces: spans on external calls and critical path; for C, model + tool + retrieval spans.
• Scores (C): user feedback, heuristic, or judge scores attached to traces/sessions.
• Analytics: product events only when product already uses that bus.
• Errors: report unexpected failures to the existing error tracker.

1. Privacy & volume

• No secrets, tokens, raw PII, or full prompts/payloads in default logs.
• Avoid high-cardinality labels (user id as metric label).
• Sample or rate-limit chatty paths.
• Ask before changing production alert thresholds or creating paid vendor resources (legitimate gate: use the question tool with options).

1. Verify

• Local or staging: emit a signal and confirm it shows where expected (or document why it can't).
• Note dashboard/query stubs if the repo keeps them.

1. Wrap up

• Next: document-it for runbooks, test-it for instrumentation/eval guards, merge-it (Ship phase) when ready.

Guardrails

• Observability is not a substitute for tests (test-it) or fixes (fix-it).
• Don't "log everything"; prefer decision points and failure boundaries.

Output

**Questions we can now answer**
- ...

**Stack**
- ...

**Instrumentation**
- ...

**Privacy / cardinality notes**
- ...

**How to verify**
- ...

## Blocked on me

none

## Changed

- ...

## Found

- ...
- Next: document-it (runbook) / test-it (guards) / merge-it

Grounding

This skill's TTPs are grounded in current engineering baselines (DORA, GitHub Docs, Fowler/Beck, Google SRE & SWE book, OpenTelemetry, OWASP LLM / NIST AI RMF, Diátaxis — see handbook sources.md).

Signals follow quality regimes (handbook/concepts/11-quality-regimes.md): SRE golden signals (+ pipeline SLIs) for compute; Web Vitals/RUM for products; LLM traces + scores (Langfuse / OTel) for generative. Watch cardinality and PII.

Handbook card: handbook/practices/observe-it.md.