CUPED

Last updated 28 October 2026

Quick answer

CUPED (Controlled-experiment Using Pre-Experiment Data) is a variance-reduction method for A/B tests. It adjusts the experiment metric using a covariate measured before assignment, so the same lift can be detected with less noise and usually fewer users.

Key takeaways

CUPED adjusts each user's experiment metric by how unusual their pre-experiment metric was.
If the covariate correlates with the outcome at ρ, variance falls by about ρ², and the sample you need falls by about the same fraction.
The covariate has to be measured before assignment. Using something the variant changed will bias the result.

Definition

What CUPED means in A/B testing

Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker described CUPED at Microsoft in 2013. Let Y be the metric during the experiment, such as revenue per user, and let X be the same kind of metric in a window before the experiment. Estimate θ as the covariance of Y and X divided by the variance of X, which is the slope of a simple regression of Y on X. The adjusted value is: Y_cuped = Y − θ (X − E[X]) Users who were already high on X are pulled down. Users who were already low are pulled up. The arm means are then compared on Y_cuped. If the correlation between X and Y is ρ, the variance of the adjusted metric is about (1 − ρ²) times the variance of Y. Sample size for a fixed minimum detectable effect scales with that variance, so a correlation of 0.5 (ρ² = 0.25) cuts the required sample by about a quarter. A correlation near 0 does almost nothing. A covariate the variant itself changed will bias θ and should not be used.

Why CUPED matters

Revenue and other heavy-tailed metrics are noisy. A handful of large buyers can land in one arm and drown a real 5% lift, or create a fake one. CUPED uses the fact that those buyers were often large before the test too. Removing that pre-existing difference leaves more of the difference the variant could have caused. It is not a substitute for randomization, a pre-registered metric, or a check that the split itself is intact.

Example of CUPED

A store's primary metric is revenue per user over 14 test days. The covariate is revenue per user over the 14 days before assignment. Suppose θ comes out at 0.8. A shopper whose pre-period revenue is $20 above the pre-period average has 0.8 × $20 = $16 subtracted from their test-period revenue. A shopper $20 below the average has $16 added. Control's raw mean is $30 and the variant's raw mean is $33. After the adjustment, both means move, but the gap is estimated with a smaller standard error. If ρ is 0.5, plan the next test at about 75% of the sample size the unadjusted metric would have needed, and confirm the covariate is balanced across arms before you trust the tighter interval.

How to use CUPED

Pick X before the test, from data that already exists at the moment of assignment. Compute θ on the pooled sample, apply the same θ to every user, and compare arm means of Y cuped . Publish both the raw lift and the adjusted lift. If X is missing for new users, say how you treated them. Dropping every new user changes the population. A common choice is to adjust only returning users and to report new users separately.

Common mistake

A common mistake is using an in-test behavior as X, such as "pages viewed during the test." The variant changes that number, so the adjustment removes part of the treatment effect and can hide a real winner. Another is applying CUPED and then peeking early because the interval got tighter. The sample plan still has to be the plan you set.

How Mida works with CUPED

Mida's experiment views are not documented as computing θ or Y_cuped. What the site does document is the path into a warehouse. The Segment integration says that once exposure and conversion events reach Segment, a Snowflake, BigQuery, or Redshift destination can store them for long-term lift analysis and CUPED. You join those exposures to a pre-period metric you already have, and you run the formula outside Mida. The sample size calculator still sizes the unadjusted test. Use it as the baseline, then scale by (1 − ρ²) only when you have a correlation from a previous experiment, not from a guess.

Related A/B testing terms

FAQ

Does CUPED change which variant won?

It should not create a lift that was not there. It shrinks the noise around each arm's mean. A real difference becomes easier to see, which is why the same minimum detectable effect can need fewer users. If the adjusted result flips the winner, check that the covariate was pre-assignment and that the two arms had a similar covariate distribution.

What is a legal covariate?

Something recorded before the visitor was bucketed, and correlated with the metric you will judge. Pre-period revenue, pre-period purchase rate, or a previous session's conversion are typical. The variant's own clicks, or any event that happens after exposure, are not covariates. Those are outcomes.

Where do I run CUPED if the testing tool only shows raw means?

In the warehouse, on an export of exposure and the pre-period metric. Mida's Segment integration says exposure and conversion events can land in Snowflake, BigQuery, or Redshift for that kind of long-term lift work. The experiment dashboard is not documented as applying the CUPED formula for you.