Experimentation

Also called: online experimentation

Last updated:

Experimentation is the discipline of validating changes with controlled experiments instead of opinion. You form a hypothesis, expose variations to randomly assigned users, measure a predefined metric, and keep the variation that wins by a statistically significant margin. A/B testing is its most common form; multivariate and bandit tests extend the same idea.

Where feature flags fit

Experimentation needs three things: a way to assign users to variations, a way to keep that assignment stable, and a way to measure outcomes. Feature flags supply the first two — a flag serves each user a variation, and a deterministic hash keeps them in their group for the duration — so the same flag that controls a release can also hold a clean experimental split. The measurement itself comes from the metrics you already track against each variation.

Experimentation vs feature management

They overlap but optimise for different goals. Feature flags exist to control release — decoupling deploy from launch, ramping safely, killing bad code. Experimentation exists to control learning — deciding which version to keep. The same flag can serve both: ship a feature gradually and measure its effect on the metric you care about.

Want the full picture? Read the concept guide: Rollout strategies →

Related terms

A/B testing

A/B testing serves two variations to randomly assigned groups of users and measures which one performs better on a chosen metric — turning a product decision into a controlled experiment instead of a guess.

Percentage rollout

A percentage rollout serves a feature to a defined share of users — chosen by a deterministic hash — so the same users stay in the cohort as you ramp the percentage up.

Feature flag

A feature flag is a conditional in your code whose value is controlled from a remote dashboard, letting you turn functionality on or off without deploying new code.

Browse the full feature flag glossary →

Try it in your own app

Free Solo plan covers 10 flags and 2 environments. No credit card, no demo call — sign up and ship.

Get started free See pricing