Confidence Interval (CI)

A confidence interval (CI) is a range of values, calculated from data, that is likely to contain the true value of a metric — expressed alongside a confidence level, usually 95%. Instead of reporting a single point estimate (a conversion rate of 5%, a lift of 8%), a confidence interval reports a range (say, 3% to 7%, or a lift of 2% to 14%) that acknowledges the uncertainty inherent in measuring from a sample rather than the whole population. It is how good analysis expresses not just what it estimates, but how sure it is.

A confidence interval comes from the reality that estimates from samples are uncertain: measure a conversion rate from 1,000 visitors and you get an estimate, not the exact truth, and the CI quantifies the plausible range around it. The width of the interval reflects how much data and how much variability there is — more data and less variability give a narrower, more precise interval, while small samples give wide intervals that admit a large range of possibilities. The confidence level (95% is conventional) describes the procedure's reliability: a 95% confidence interval is constructed so that, across many repeated samples, about 95% of such intervals would contain the true value — which is a subtler statement than the common misreading that 'there is a 95% chance the true value is in this interval.' In marketing, confidence intervals are essential for honest interpretation of A/B TESTS and estimates: a test showing a +8% lift with a CI of −1% to +17% has not actually demonstrated a reliable win (the interval includes zero and negative values), whereas +8% with a CI of +5% to +11% has. Reporting a point estimate without its interval hides the uncertainty and invites overconfident decisions. Confidence intervals connect directly to STATISTICAL SIGNIFICANCE (if a difference's CI excludes zero at the chosen level, it is significant) and to sample size (bigger samples narrow the interval).

Confidence intervals matter most wherever decisions rest on estimates from data — A/B test results, conversion rates, survey findings, lift measurements, forecasts — which is most of marketing analytics. The discipline is to report and read estimates with their intervals rather than as false-precision point numbers, to recognize that a wide interval means 'we don't really know yet' (often a call for more data), and to avoid acting on differences whose intervals overlap or include zero. Used well, confidence intervals keep decisions honest about uncertainty; ignored, they let teams treat noisy estimates as solid facts and act with unwarranted confidence on results that the data does not actually support.

The confidence interval was introduced by statistician Jerzy Neyman in the 1930s as a way to express the uncertainty of an estimate as a range with a stated coverage probability. It is foundational to statistical inference and, in marketing, to the honest interpretation of A/B tests, surveys, and any metric estimated from a sample.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where confidence interval (ci) is a core concern: