posthog/ai-plugin

auditing-experiments-flags

Audit PostHog experiments and feature flags for configuration issues, staleness, and best-practice violations.

Voir la source

Mode d’emploi

  1. Copiez la commande et examinez le paquet avant de l’installer dans votre environnement agent.
  2. Consultez le document original pour les outils, identifiants et environnements requis.
  3. Décrivez les entrées et le livrable souhaité ; comparez la sortie aux sources.
npx skills add https://github.com/posthog/ai-plugin --skill auditing-experiments-flags

Avant de commencer

  • L’installation n’inclut ni abonnement au modèle, ni quota API, ni accès aux fichiers.
  • Étoiles et installations reflètent l’activité, pas un test indépendant de qualité ou de sécurité.
Document Skill original

Rendu depuis le dépôt source en conservant titres, exemples, code, tableaux, liens et images.

Auditing experiments and feature flags

This skill teaches you how to run configuration audits on experiments and feature flags. All checks use the experiment and feature flag read tools (experiment-get, experiment-list, feature-flag-get-definition, feature-flag-get-all) — no SQL queries are needed for Phase 1 checks.

Usage modes

Quick check (single entity)

When the user asks about a specific experiment or flag:

  1. Fetch the entity via experiment-get (experiment ID) or feature-flag-get-definition (numeric flag ID).
  2. Apply the relevant checks from experiment checks or flag checks.
  3. Report findings inline as markdown, grouped by severity (CRITICAL first, then WARNING, then INFO).
  4. Include entity links as [Experiment: name](/experiments/id) or [Flag: key](/feature_flags/id).

Scoped audit (one domain)

When the user asks to audit all experiments or all flags:

  1. Bulk-fetch via experiment-list or feature-flag-get-all.
  2. Run all checks for that domain against each entity.
  3. Group findings by severity, then by entity.
  4. Report as inline markdown.

Full audit (comprehensive)

When the user asks for a comprehensive audit of both experiments and flags:

  1. Fetch all experiments via experiment-list and all flags via feature-flag-get-all.
  2. Run all experiment checks and all flag checks.
  3. Apply recurring patterns to identify patterns across multiple findings.
  4. If there are more than 5 entities with findings, output as a notebook artifact via notebooks-create for easier navigation. Otherwise report inline.

Output format

For each finding, include:

  • Severity badge: 🔴 CRITICAL, 🟡 WARNING, or 🔵 INFO
  • Check name: Which check produced this finding
  • Entity link: Markdown link to the entity
  • What's wrong: One-sentence description
  • Action: What to do about it (see remediation actions)

Example:

🟡 WARNING — Flag integration · Experiment: checkout-redesign The linked feature flag is inactive (paused). Traffic is not being split. Action: Re-enable the flag or end the experiment.

Handling unavailable data

Some checks require activity logs (feature-flags-activity-retrieve for flags), which may not be available in every session. If activity log data is unavailable:

  • Skip checkActivityHistory (experiment check) entirely.
  • Skip the "toggle instability" and "never activated" sub-checks in flag lifecycle checks.
  • In your report, note which checks were skipped and why:
Skipped: Activity history checks (activity logs not available via current tools)

Partial failures

If a fetch call fails for some entities:

  • Continue with the entities you could fetch.
  • Report which entities could not be assessed and why.
  • Do not silently omit entities from the audit.

Reference files

Tâches connexes

du même dépôt

Autres Skills

Tous les Skills
posthog
Officiel

exploring-llm-costs

Investigate LLM spend in PostHog — total cost over time, cost by model, provider, user, trace, or custom dimension, token and cache-hit economics, and cost regressions. Use when the user asks "how much are we spending on LLMs?", "which model / user / feature is most expensive?", "why did cost spike?", wants to build a cost dashboard or alert, or pastes a trace URL and asks about its cost.

Installations / 1h
2
GitHub Stars
98
Mis à jour
6 nov.
posthog
Officiel

exploring-llm-traces

Debug and inspect LLM/AI agent traces using PostHog's MCP tools. Use when the user pastes a trace or session URL (e.g. /ai-observability/traces/ or /ai-observability/sessions/ ), asks to debug a trace, figure out what went wrong, check if an agent used a tool correctly, verify context/files were surfaced, inspect subagent behavior, investigate LLM decisions, or analyze token usage and costs. Also use when raw SQL/HogQL against events.properties.$aiinput / $aioutputchoices returns empty — message

Installations / 1h
2
GitHub Stars
98
Mis à jour
6 nov.
posthog
Officiel

instrument-integration

Add PostHog SDK integration to your application. Use when setting up PostHog for the first time or reviewing PRs that need PostHog initialization. Covers SDK installation, provider setup, and basic configuration for any framework.

Installations / 1h
2
GitHub Stars
98
Mis à jour
6 nov.
posthog
Officiel

investigating-error-issue

Investigates a single PostHog error tracking issue end-to-end. Use when the user provides an issue ID or pastes an issue URL (/error-tracking/ ) and wants to understand the error — who it affects, what triggers it, when it started, whether it correlates with a release, browser, OS, or feature flag, and what the next step should be. Pulls aggregated metrics, sample exception events, segment breakdowns, linked replays, and synthesizes a hypothesis-grade summary in one pass.

Installations / 1h
2
GitHub Stars
98
Mis à jour
6 nov.