A/B Testing
Comparing two versions of a design by showing each to a slice of live traffic and measuring which drives the target metric better.
An A/B test splits live users between a control (A) and one or more variants (B), then measures which produces more of the behavior you care about — signups, purchases, upgrades. Because the comparison happens on real traffic at the same time, it isolates the effect of the change from seasonality and noise.
A/B testing is the engine of conversion rate optimization and growth design. Its main traps are calling results before reaching statistical significance and testing changes too small to matter — discipline about sample size and hypothesis quality is what separates signal from superstition.
In practice
In a B2B SaaS product, an A/B test might pit the current three-step signup against a variant that defers email verification until after the first project is created. Half of new visitors see each version; after two weeks, the variant shows an 11% lift in completed onboarding with no rise in spam accounts — so it ships to everyone. The discipline is in what didn't happen: nobody argued from taste, and the change was judged on the metric it was designed to move.
Frequently Asked Questions
Related terms