Incrementality testing: the only attribution method that produces causal answers.

What incrementality testing actually is

Incrementality testing is the practice of running a controlled experiment to measure a marketing channel's true causal contribution to conversions. The team holds a portion of users or geographies as a control group, runs the channel only for the rest, and measures the difference in outcomes. The gap is the channel's incremental lift. Incrementality is the only attribution method that produces causal answers. Every other method — first-touch, last-touch, data-driven, MMM — is correlational at best.

The work matters because correlation hides reality. A channel that gets the last click before conversion looks valuable to last-click attribution even when the customer would have bought anyway. A channel that gets early-funnel touches looks valuable to first-touch even when the touches did nothing. Only an experiment with a control group separates real lift from coincidence. Every other model is guessing about causation from observed paths.

The cost is time and complexity. A geo-holdout test takes 30+ days to run and requires holding spend in markets that look like the test markets. Conversion-lift studies on platforms are easier but vulnerable to platform-controlled measurement. Both are expensive in lost revenue during the test period, but the answer they produce is the only honest one.

Claim: Across roughly 50 audits per year at Real Growth Matters, incrementality tests typically reveal that 20-40 percent of paid-channel spend is non-incremental — the conversions would have happened anyway through organic or other channels. The number varies widely by channel but the pattern is universal. Source: Real Growth Matters Inc., internal audit data, 2024-2026. Context: The non-incremental share is biggest on branded search (people typing the brand name would have found the site anyway) and on retargeting (people already in the funnel were going to convert). The fix is rarely to cut the spend entirely; it is to rebalance bids and budgets so the marginal dollar moves to a channel with higher incrementality.

The three main incrementality testing methods

Three methods cover almost every practical incrementality test. Geo holdouts are the most rigorous and most common. Conversion lift studies are the platform version, easier to run but vulnerable to platform-controlled measurement. Switchback tests are useful for marketplace and high-frequency products. Each method has its own statistical assumptions and its own typical use case.

The three main incrementality testing methods, what each does, and when to use each

Method	How it works	Best fit
Geo holdouts	Turn channel off in matched geos, measure difference	National advertisers; large enough geo footprint; 30+ days
Conversion lift studies	Platform randomly assigns users to test/control cells	Single-platform tests; faster to run than geo
Switchback tests	Toggle channel on/off over time across same users	Marketplaces, high-frequency products; shorter cycles

How to run a geo holdout test

Geo holdout tests are the rigorous standard. The team picks 10-15 designated market areas (DMAs) that match the rest of the country in size, demographics, and product-mix. The team turns off the test channel in those DMAs for 30 days. The team then compares conversions per capita in held-out vs. running DMAs. The difference is the channel's incremental contribution. Tools like Meta Geo Lift Studies, Google Geo Experiments, and bespoke MMM-based geo tests all use this method.

1. Pick test and control geos.Use synthetic-control or matched-pair methodology to find 10-15 DMAs that mirror the rest of the country. Meta and Google both ship tools that do this automatically. Bespoke tests use Difference-in-Differences (DiD) or synthetic control to build the comparison group.
2. Hold the test for at least 30 days.Most channels have lag between exposure and conversion. A 30-day window is the minimum for retail and DTC; B2B may need 60-90 days. Shorter tests miss conversions that would have lagged into the post-test period.
3. Measure conversions per capita.Normalize for population differences between test and control. The lift is the per-capita conversion gap multiplied by the population in the test geos. Statistical significance is computed with a t-test or Bayesian equivalent.
4. Compute incremental ROAS.Lift in conversions multiplied by average order value gives incremental revenue. Divide by spend in the test geos to get incremental ROAS. This is the channel's true ROAS, not the attributed ROAS.
5. Re-run quarterly on the biggest channels.Incrementality changes over time as markets shift and the channel matures. A channel with 80 percent incrementality this year may drop to 40 percent next year as competition rises. Quarterly re-tests catch the drift.

Conversion lift studies on platforms

Conversion lift studies are the platform version of incrementality testing. Meta, Google, TikTok, and LinkedIn all offer built-in studies that randomly assign users to test and control cells, then report the lift back to the advertiser. Conversion lift studies are easier to set up than geo holdouts but produce platform-controlled results — the same company runs the experiment and reports the outcome. Use with caution and triangulate against geo holdouts when possible.

The mechanics are similar to geo holdouts but at the user level. The platform randomly assigns eligible users (those in the targeted audience) to either see the ad (treatment) or not see it (ghost bid placeholder, control). After the campaign ends, the platform compares conversion rates between the two groups and reports the lift. The math is honest. The data is platform-controlled.

The trap is selection bias. The platform decides who is in the eligible audience. If the targeting algorithm preferentially shows ads to users likely to convert anyway, the conversion lift study will overstate true incremental lift. Cross-validating with geo holdouts or with third-party MMM is the standard defense.

When to use incrementality vs. attribution models

Use attribution models (DDA, last-click, MMM) for day-to-day optimization. Use incrementality tests for quarterly truth-checking. The attribution model tells you how to spend tomorrow. The incrementality test tells you whether the attribution model is right. Most teams use only attribution and never validate; the validation is what separates teams that scale profitably from teams that scale into margin collapse.

The cadence we recommend in audit: attribution-model dashboards reviewed weekly, incrementality tests on the largest channels run quarterly, marketing-mix modeling re-run annually. Each layer corrects the layer above. Attribution is daily noise; incrementality is the quarterly truth-check; MMM is the annual structural picture.

Three common incrementality testing mistakes

Three failure modes appear in almost every audit of an incrementality program. Each is fixable but expensive if it has been running wrong for months.

Failure 1: test window too short

The team runs a 7- or 14-day test and tries to draw conclusions. Most channels have conversion lag longer than that. A 30-day minimum is the practical standard for retail and DTC; 60-90 days for B2B. Tests run too short systematically understate true incremental lift because the lagged conversions never get counted.

Failure 2: trusting platform-reported lift only

The team runs conversion lift studies on each platform and trusts the platform's reported lift. Selection bias and platform-controlled measurement inflate the numbers. The fix is to cross-validate with geo holdouts or third-party MMM at least once a year.

Failure 3: not re-testing

The team runs an incrementality test once, gets a number, and uses that number for two years. Markets shift, competition rises, and the incrementality changes underneath. Quarterly re-tests on the biggest channels are the minimum cadence to keep the data honest.

Quick answers

What is incrementality testing in plain English?

Running an experiment to figure out whether a marketing channel is actually driving conversions or whether the conversions would have happened anyway. Turn the channel off in some markets, leave it on in others, compare the difference.

Why is incrementality the gold standard?

Because it is the only method that produces causal answers. Every other attribution method (last-click, DDA, MMM) is correlational. Only an experiment with a control group tells you what caused what.

What is a geo holdout test?

Turn the channel off in 10-15 matched geos for 30 days. Measure the conversion gap between held-out and running geos. The gap is the channel's incremental contribution. Meta Geo Lift and Google Geo Experiments are common implementations.

What is a conversion lift study?

The platform version. Meta, Google, TikTok, and LinkedIn randomly assign users to see ads or not see them, then report the lift. Easier to run than geo holdouts but vulnerable to platform-controlled measurement.

How long does an incrementality test take?

30 days minimum for retail and DTC, 60-90 days for B2B. Shorter windows systematically understate true lift because lagged conversions never get counted.

How often should I re-test?

Quarterly on the biggest two or three channels. Incrementality changes over time as markets shift and the channel matures. A channel with 80 percent incrementality this year may drop to 40 percent next year.

Frequently asked

What is incrementality testing?

Running a controlled experiment to measure a marketing channel's true causal contribution to conversions. The team holds a control group (typically a set of geographic markets) where the channel is paused, then compares to the running markets. The gap is the channel's incremental lift.

Why does incrementality matter?

Because attribution models are correlational, not causal. A channel that gets the last click looks valuable to last-click attribution even if the customer would have bought anyway. Only an experiment with a control group separates real lift from coincidence.

What is a geo holdout?

The most common incrementality testing method. Pick 10-15 designated market areas (DMAs) that match the rest of the country, turn the channel off in those DMAs for 30+ days, measure the per-capita conversion gap. Tools like Meta Geo Lift and Google Geo Experiments automate the setup.

What is a conversion lift study?

The platform-native version of incrementality testing. Meta, Google, TikTok, and LinkedIn each offer built-in studies that randomly assign users to see ads or not see them. The platform reports the lift. Easier to set up than geo holdouts; vulnerable to platform-controlled measurement.

How is incrementality different from attribution?

Attribution assigns credit for conversions that did happen. Incrementality measures whether the channel caused them. Attribution: backward-looking and correlational. Incrementality: forward-looking and causal.

How much does an incrementality test cost?

Direct cost is the held-out spend in the control geos (typically 10-30 percent of channel spend for 30 days). Indirect cost is the opportunity cost of foregone conversions in the held-out markets. Total cost varies widely; budget at least $25,000-$100,000 for a meaningful test on a mid-market channel.

What is a typical incrementality finding?

Across audits, 20-40 percent of paid-channel spend is typically non-incremental. The non-incremental share is biggest on branded search and retargeting. The fix is rarely to cut the spend entirely; it is to rebalance bids and budgets toward channels with higher incrementality.

Should incrementality replace attribution models?

No. Use both. Attribution models give daily decision support; incrementality tests give quarterly truth-checks. The cadence is attribution weekly, incrementality quarterly, MMM annually. Each layer validates the layer above.