A/B Testing
A controlled experiment that randomly splits users between two or more versions of a page, email or feature to measure which performs better.
A/B testing replaces opinion with measured user behavior, so teams can roll out page, pricing or messaging changes that raise conversion or revenue and reject changes that look good but perform worse.
An ecommerce team splits checkout traffic 50/50 between the current page and a version with fewer form fields, runs the test for two full weeks, and ships the variant only after it beats control at 95% significance on completed orders while refund rate stays flat.
Tests need enough traffic to detect a realistic effect, so low-volume B2B pages often cannot reach significance. Stopping early on interim results, changing several elements at once or tracking a single metric can produce winners that do not hold.
Users are randomly assigned to a control (A) and one or more variants (B), and the assignment stays stable per user through a cookie or user ID. Before launch, the team fixes a hypothesis, a primary metric such as Conversion Rate (CVR), guardrail metrics and a sample size based on baseline rate, minimum detectable effect and significance level. Results are evaluated with frequentist or Bayesian statistics, and an A/A test can check the setup.
Performance Advertising, Revenue Operations
Optimizely — A/B testing — https://www.optimizely.com/optimization-glossary/ab-testing/; Nielsen Norman Group — A/B Testing 101 — https://www.nngroup.com/articles/ab-testing/