Difference-in-Differences

Difference-in-differences (DiD) is the quasi-experimental method for measuring an intervention's effect when you cannot randomize: compare the change over time in treated units (markets where the campaign ran) against the change in untreated ones (markets where it didn't), and the difference of those differences is the effect. The genius is in what cancels — seasonality, macro trends, anything moving both groups alike subtracts itself out, leaving only the gap that opened beyond the shared path.

The setup needs treated and control units observed before and after: launch the campaign in some regions, hold others out, measure both periods in both groups. Treated markets up 14% while controls rose 9% means the intervention owns five points, not fourteen — the naive before-after read overstates by exactly the trend the controls reveal (the CAUSAL-INFERENCE discipline operationalized). Everything rests on one assumption with a name worth memorizing: parallel trends — absent the intervention, treated and control groups would have moved alike. It is checkable in the rear-view (did the groups track each other before the launch?) and breakable in practice (treated markets chosen because they were already accelerating poisons the comparison — selection's revenge). The marketing habitats: geo-split campaign launches (DiD is the analysis engine of geo-experiments and CONVERSION-LIFT's regional cousin), pricing and policy changes rolled to some markets first, retail-distribution gains, and any natural experiment where something changed for some units and not others. The craft extensions handle real-world mess: matched controls chosen for pre-period similarity, synthetic controls built as weighted blends when no single market parallels the treated one, and event-study plots showing the gap period by period — flat before, opening after is the picture that convinces.

DiD matters whenever a clean A/B is impossible but a control group exists in space or time — regional launches, market-by-market rollouts, platform changes that hit some accounts first. It matters most as the antidote to before-after storytelling, which credits campaigns with whatever the season was doing anyway. The discipline is parallel-trends verification in the pre-period, controls chosen before results are known, and the event-study plot shown with every claim — the method is one subtraction; the credibility is all in the setup.

Difference-in-differences traces to John Snow's 1850s cholera work — comparing changes between London water districts — and became econometrics' workhorse quasi-experiment (Card and Krueger's 1994 minimum-wage study its modern landmark) before marketing adopted it as the analysis engine of geo-experiments.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where difference-in-differences is a core concern: