sanity-io/agent-toolkit

content-experimentation-best-practices

Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis pitfalls.

View source

How to use

  1. Copy the installation command and review the source package before installing it in your own agent environment.
  2. Check the source document for required tools, credentials and supported hosts.
  3. Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/sanity-io/agent-toolkit --skill content-experimentation-best-practices

Before you start

  • Installation does not include a model subscription, API allowance or permission to access your files.
  • Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

Content Experimentation Best Practices

Principles and patterns for running effective content experiments to improve conversion rates, engagement, and user experience.

When to Apply

Reference these guidelines when:

  • Setting up A/B or multivariate testing infrastructure
  • Designing experiments for content changes
  • Analyzing and interpreting test results
  • Building CMS integrations for experimentation
  • Deciding what to test and how

Core Concepts

A/B Testing

Comparing two variants (A vs B) to determine which performs better.

Multivariate Testing

Testing multiple variables simultaneously to find optimal combinations.

Statistical Significance

The confidence level that results aren't due to random chance.

Experimentation Culture

Making decisions based on data rather than opinions (HiPPO avoidance).

References

Start with the reference that matches the current problem, such as design, statistics, CMS integration, or pitfalls. See references/ for detailed guidance:

  • references/experiment-design.md — Hypothesis framework, metrics, sample size, and what to test
  • references/statistical-foundations.md — p-values, confidence intervals, power analysis, Bayesian methods
  • references/cms-integration.md — CMS-managed variants, field-level variants, external platforms
  • references/common-pitfalls.md — 17 common mistakes across statistics, design, execution, and interpretation

Related by task

from this repository

More skills

All skills
sanity-io
Official

sanity-best-practices

Sanity development best practices for schema design, GROQ queries, TypeGen, Visual Editing, images, Portable Text, Studio structure, localization, migrations, Sanity Functions, webhooks, Blueprints, and framework integrations such as Next.js, Nuxt, Astro, Remix, SvelteKit, Angular, Hydrogen, the App SDK, and Studios or apps running in the Sanity Dashboard (defineApplication). Use this skill whenever working with Sanity schemas, defineType or defineField, GROQ or defineQuery, content modeling, Pr

1h installs
3
GitHub stars
201
Updated
Nov 19
sanity-io
Official

sanity-migration

Plans, implements, and reviews migrations from other CMSes and content systems into Sanity. Use when migrating or replatforming to Sanity from AEM, Adobe Experience Manager, Contentful, Strapi, Webflow, WordPress, Payload, Drupal, Markdown/MDX/frontmatter files, WXR/XML exports, CMS APIs, database dumps, static HTML, or when designing extraction, transformation, Portable Text conversion, asset migration, redirects, validation, and cutover workflows.

1h installs
3
GitHub stars
201
Updated
Nov 19
sanity-io
Official

content-modeling-best-practices

Structured content modeling guidance for schema design, content architecture, content reuse, references versus embedded objects, separation of concerns, and taxonomies across Sanity and other headless CMSes. Use this skill when designing or refactoring content types, deciding field shapes, debating reusable versus nested content, planning omnichannel content models, or reviewing whether a schema is too page-shaped or presentation-driven.

1h installs
2
GitHub stars
197
Updated
Nov 10
sanity-io
Official

sanity-studio-upgrade

Produces a tailored Sanity Studio upgrade plan by inspecting the repository's installed versions, config, and source, then reporting only the breaking changes that actually apply. Covers Studio v3 and later; v2 projects are identified and redirected, not planned. Use this skill whenever someone wants to upgrade, migrate, or modernize a Sanity Studio across one or more major versions from v3 onward, asks what will break if they bump the sanity package, asks why their Studio broke after an upgrade

1h installs
2
GitHub stars
196
Updated
Oct 27