Alternative Hypothesis

Alternative Hypothesis is the statement in hypothesis testing that proposes there is a real, measurable difference between the control and treatment variations in an A/B test.

Meaning & Context

Denoted as H₁ or Hₐ, the alternative hypothesis is what you're trying to find evidence for in your experiment. It directly opposes the null hypothesis and represents the claim that your variation causes a change in the metric you're measuring. Alternative hypotheses can be one-tailed (directional, predicting improvement or decline) or two-tailed (non-directional, simply predicting a difference). Most A/B tests use one-tailed alternatives because you're specifically testing whether a variation performs better.

Why It Matters

Clearly defining your alternative hypothesis before running a test ensures you're measuring the right metrics and sets the foundation for proper statistical analysis. It helps determine your required sample size, informs whether you should use a one-tailed or two-tailed test, and guides the interpretation of results. A well-formulated alternative hypothesis includes the specific metric, direction of change, and ideally the minimum detectable effect you care about.

Example

For a button color test, your alternative hypothesis might state: 'Changing the CTA button from blue to red will increase the click-through rate by at least 10% compared to the control.' This stands in contrast to your null hypothesis that there's no difference between the two colors.