Measure Channel Incrementality with Holdouts and Geo Tests

Many organisations, including analytically mature ones, confuse sophisticated attribution with incrementality. GA4's data-driven attribution improves on last click, but it is an observational model: it describes which touchpoints preceded a conversion. What marketers and executives need to know is which channels caused conversions that would not otherwise have happened.

The gap shows in two questions attribution cannot answer. If a user saw an ad and then converted, would they have converted without it? And if the ad were switched off, would conversions fall, or would the same users simply arrive through another path? Brand search and retargeting are the usual examples: they sit close to conversion and collect credit, yet much of that demand would arrive anyway. These are causal questions, and they need a different analytical design, not a more detailed path model.

The strongest evidence comes from controlled experiments: user-level holdout groups deliberately excluded from a campaign, or geo experiments in which spend is paused or increased in some regions and held constant in matched ones. Where randomisation is not possible, quasi-experimental methods such as difference-in-differences or synthetic control estimate the counterfactual from comparable regions or periods. In both cases the pre-period should be checked to confirm that treatment and control moved in parallel before the intervention.

The GA4 BigQuery export supplies the outcome data, including the geo.country, geo.region and geo.city fields needed to assign regions to test cells. dbt models turn raw events into analysis-ready tables: daily conversions and revenue per region or test cell, flagged as treatment or control, with a defined pre-period. Joining these cohort-level tables to media spend expresses lift as incremental conversions per unit of spend rather than as a share of credit.

Causal estimates are only as reliable as the outcome measurement. If tracking coverage differs between regions or changes during the test, for example through different consent rates or a tagging release, the measurement artefact will be read as lift. Consistent server-side conversion data, such as that collected by Robin Signals, makes control groups more trustworthy in a privacy-constrained setting.

Done well, lift testing separates channels that generate business from channels that mainly reach users who were going to convert anyway. That is the number budget allocation needs.