Credible Interval

A credible interval is a range of values within which a parameter (such as conversion rate or effect size) lies with a specified probability in Bayesian analysis, representing the uncertainty around an estimate after observing data.

Meaning & Context

Unlike frequentist confidence intervals, credible intervals can be directly interpreted as probability statements about the parameter of interest. A 95% credible interval means there's a 95% probability that the true value falls within that range, given the data and prior beliefs. Credible intervals are derived from the posterior distribution and naturally incorporate all sources of uncertainty in the analysis.

Why It Matters

Credible intervals provide an intuitive way to communicate uncertainty and effect sizes to stakeholders, avoiding the common misinterpretations associated with confidence intervals. They enable better risk assessment by clearly showing the range of plausible outcomes. When credible intervals for the difference between variations exclude zero, this provides strong evidence that one variation outperforms the other.

Example

Your Bayesian A/B test analysis shows the new landing page has a conversion rate with a 95% credible interval of 4.2% to 5.8%, meaning you can be 95% certain the true conversion rate lies within this range, compared to the control's credible interval of 3.1% to 4.3%.