A plain-language guide to incrementality testing: what it costs, when it is worth it, and how it shows which marketing spend actually creates revenue.
Every advertising platform will tell you it is working. Meta reports conversions, Google reports conversions, and when you add up everything each platform claims, the total is somehow larger than your actual revenue. If you are the person who signs the media budget, you have probably noticed this and wondered which numbers to believe.
Incrementality testing is the discipline that answers the only question a finance leader actually cares about: how much of this revenue would have arrived anyway? This is a plain-language guide to how it works, what it costs, and when it is worth doing.
The problem: platforms grade their own homework
Platform reporting is built on attribution, which is a claiming system, not a measurement system. If someone saw your ad and later bought, the platform claims the sale. It cannot tell you whether the ad caused the purchase or merely preceded it.
The gap matters most where advertising overlaps with demand that already exists. Brand search campaigns intercepting people who typed your name. Retargeting ads shown to visitors who were already coming back. Both look spectacular in platform reports. Both are, in part, buying revenue you already owned.
What incrementality testing actually is
The logic is the same as a clinical trial. You split a market into two comparable groups, run the advertising in one and not the other, and measure the difference in actual revenue. Whatever the exposed group generates beyond the control group is incremental: revenue the advertising genuinely created.
In practice the common designs are simple. A geo holdout pauses or reduces spend in a set of matched regions and compares outcomes. A spend-level test raises budget in some markets and holds it in others to find where returns flatten. An audience holdout withholds ads from a slice of an eligible audience. None of this needs exotic tooling. It needs clean revenue data by region or segment, discipline about not touching the test mid-flight, and enough volume for the difference to be readable.
What it tells you that attribution cannot
Run properly, incrementality testing reallocates money with confidence. It is common to discover that a channel reporting a strong return is substantially non-incremental, while a channel that looks mediocre in last-click reporting is quietly doing heavy lifting earlier in the journey. Without testing, budget flows to whichever channel claims loudest. With it, budget flows to what causes revenue.
It also settles internal arguments. When channel owners each defend their own numbers, a holdout test is the neutral referee. The result is not an opinion; it is a measured difference in revenue between two groups of real customers.
What it costs, in money and patience
The honest costs are threefold. There is a small amount of media efficiency sacrificed during the test, because you are deliberately not spending in the control group. There is analyst time to design the split and read the result. And there is patience: most tests need four to eight weeks to produce an answer you can trust, longer for considered purchases with slow sales cycles.
The threshold question is scale. Below roughly £20,000 a month in media spend, the cost of testing usually outweighs the value of the answer, and you are better served by tight tracking and measurement hygiene and common sense about brand search. Above that, the maths shifts quickly: a single reallocation decision informed by a good test typically pays for the testing programme many times over.
How this fits a measurement stack
Incrementality does not replace attribution; it calibrates it. Attribution remains useful for day-to-day optimisation inside a channel. Incrementality tells you how much weight each channel's claims deserve, and periodic retesting keeps those weights honest as markets shift. We cover the broader discipline of connecting spend to money in our guide to commercially useful marketing analytics, and our attribution intelligence work builds exactly this layered view for paid media.
The commercial effect is real. When we rebuilt budget modelling for Hubject, understanding what genuinely drove qualified demand contributed to 610 percent growth in qualified leads, because money stopped flowing to activity that merely claimed credit.
Questions to ask before you commission a test
Three questions separate useful tests from theatre. Is the revenue data clean enough to read by region or segment? Is the organisation willing to leave the test alone for its full duration, even when a channel owner gets nervous? And is there a decision waiting on the answer, a real reallocation that will happen once the result lands? If any answer is no, fix that first.
If the answers are yes, incrementality testing is one of the highest-return exercises in marketing. Our performance testing team designs and runs these programmes end to end, from test design through to the reallocation decision. Talk to us if you want to know what your platform numbers are actually worth.
