Incrementality Testing: A Practical Guide for Marketers

August 3, 2026

Incrementality testing is a randomized controlled experiment that measures the causal lift a campaign produces, calculated as (Treatment conversions − Control conversions) ÷ Control conversions. The IAB defines incrementality as the additional business outcomes directly driven by a campaign compared to what would have occurred without it. Run a test when you need to answer a budget decision with confidence: does this channel create net-new revenue, or is it reattributing conversions that would have happened anyway? Standard practice targets 95% statistical significance and 80% power, and feasibility depends entirely on conversion volume, not spend.

The core business question: “Would this conversion have happened without my ad?” Attribution cannot answer that. Only a controlled experiment can.

Incrementality testing isolates causal lift, not correlated activity.
The holdout group (users or geos that never see the ad) is the counterfactual.
Feasibility is the first gatekeeper: too few conversions and the test produces no actionable result, regardless of budget.
A valid test requires a pre-committed decision rule, a fixed measurement window, and reported confidence intervals, not just a point estimate.

What does incrementality testing actually measure?
How incrementality testing works in practice
How to calculate lift and incremental ROAS
Do you have enough data to run a valid test?
How does incrementality testing compare to A/B testing, MMM, and attribution?
Which campaigns and channels benefit most from lift testing?
Common pitfalls that invalidate incrementality tests
Step-by-step checklist to run an incrementality test
How Theartistevolution applied incrementality testing for a retail client
Key Takeaways
The measurement stack most teams are still missing
Theartistevolution can build your incrementality program
Useful sources and tools

What does incrementality testing actually measure?

Attribution models assign credit across touchpoints. Incrementality testing asks a different question entirely: how many conversions would not have occurred without the campaign? The answer is the net-new, causally driven outcome, which Amplitude describes as separating caused outcomes from baseline behavior that would have happened regardless of media.

The experiment splits an audience into two groups. The treatment group sees the ad normally. The holdout group is suppressed from seeing it. After the measurement window closes, you compare conversion rates between the two. The gap is the incremental lift.

Common holdout types:

User-level holdout: Individual users are randomly assigned to treatment or control. High precision, but requires platform-level suppression capability and is vulnerable to cookie loss.
Geo-level holdout: Geographic regions are assigned to treatment or control. More practical for channels where user-level suppression is unavailable, and more privacy-safe.
Synthetic control: A statistical model constructs a counterfactual from historical data in untreated regions. Useful when running a true holdout would mean going dark in a major market.
Cookie-based holdout: Assigns users by cookie ID. Fast to implement but degrades with browser privacy changes.

The metrics a lift study reports include lift percentage, incremental ROAS (revenue driven by the campaign divided by campaign spend), and confidence intervals on both. Incrementality answers the causal question that attribution cannot: “Would this conversion have happened anyway?” A last-click model might credit a retargeting ad for a purchase the customer had already decided to make. An incrementality test would show zero lift from that retargeting spend.

How incrementality testing works in practice

The mechanics of a well-designed test come down to three decisions: how you randomize, how long you run, and how you protect the experiment from contamination.

Randomization mechanisms vary by channel and platform. Platform-level holdouts (available in Google Ads, Meta, and several retail media networks) handle suppression server-side, which is the most reliable method. Server-side splits via a CDP or experimentation platform give you channel-agnostic control. API-based assignment is common in programmatic environments where the DSP can exclude holdout users at bid time.

Measurement windows must cover the full conversion cycle. A campaign selling a product with a 14-day consideration window needs at least 14 days of exposure plus the conversion lag period. Cutting the window short systematically undercounts conversions in the treatment group and inflates the apparent lift.

Conversion lag is one of the most common sources of underestimated lift. If your window closes before late converters register, the treatment group looks weaker than it actually is — and you may incorrectly conclude the campaign had no effect.

Contamination and bias risks to watch for:

Cross-over contamination: A user assigned to the holdout sees the ad through a different device or channel. Geo-level tests reduce this risk because geography is harder to cross than a device boundary.
Selection bias: If holdout assignment is not truly random (e.g., the platform excludes high-value users from holdouts to protect revenue), the control group is not a valid counterfactual.
Seasonality and external events: A promotion, competitor move, or news event during the test window can shift conversion rates in ways that look like lift. Pre-register your analysis plan and document any external events that occur.
Unstable campaigns: Launching a lift test on a campaign that is still in learning phase or has recently changed creative introduces launch-effect noise. Wait for several weeks of stable operation before starting.

The ACM engineering literature flags privacy-driven tracking loss and sparse-conversion environments as the two hardest engineering challenges in incrementality programs, and recommends synthetic controls and private set union techniques as practical remedies when user-level data is limited.

How to calculate lift and incremental ROAS

The lift formula is straightforward. Given:

T = conversion rate (or total conversions) in the treatment group
C = conversion rate (or total conversions) in the control group

Lift % = (T − C) ÷ C × 100

This arithmetic, as the IAB specifies, isolates the causal conversions driven by the campaign.

Worked example:

Treatment group: 10,000 users, 500 conversions → 5.0% conversion rate
Control group: 10,000 users, 400 conversions → 4.0% conversion rate
Lift % = (5.0 − 4.0) ÷ 4.0 × 100 = 25% incremental lift
Incremental conversions = 500 − 400 = 100

To convert that into incremental ROAS, divide incremental revenue by campaign spend. If those 100 incremental conversions each generated $80 in revenue and total campaign spend was $5,000:

Incremental revenue = 100 × $80 = $8,000
Incremental ROAS = $8,000 ÷ $5,000 = 1.6x

Google’s Conversion Lift tooling uses this same conversion-to-revenue bridge to report incremental ROAS directly within the platform, which makes it easier to compare against attributed ROAS and spot where attribution is inflating apparent returns.

A point estimate alone is not enough. Always report the 95% confidence interval alongside the lift figure. A result of “25% lift (CI: 8%–42%)” tells a very different story than “25% lift (CI: −2%–52%).” The second interval crosses zero, meaning the result is not statistically significant.

What to report from every test:

Lift % with 95% confidence interval
Incremental conversions and incremental revenue
Incremental ROAS
Minimum detectable lift (MDE) the test was powered to find
Statistical significance level and power used

Linking incrementality results to broader marketing ROI decisions requires this full picture, not just the headline lift number.

Do you have enough data to run a valid test?

This is the question most teams skip, and it is the reason most lift tests produce inconclusive results. Conversion volume, not budget, determines feasibility. A $500,000 campaign with 200 total conversions cannot produce a statistically meaningful lift estimate for a realistic lift size.

The standard rule of thumb uses the relationship:

Conversions per group ≈ 15.68 ÷ (relative lift)²

This assumes 95% significance and 80% power. At a 20% expected lift, you need roughly 392 conversions per group (784 total). At 10% lift, that rises to about 1,568 per group. At 5% lift, the requirement becomes approximately 6,272 per group — a volume most mid-market advertisers cannot reach in a single campaign window.

Expected lift	Conversions per group	Total conversions needed
20%	~392	~784
10%	~1,568	—
5%	~6,272	—

Practical inputs for any power calculation (per Presenc AI’s guidance):

Expected lift (your best estimate from prior data or MMM)
Baseline conversion rate
Holdout share (typically 5–15%, with ~10% as a common default)
Significance level: 5% (95% confidence)
Statistical power: 80%

Below roughly 600 total conversions, platform lift tests are generally infeasible for detecting moderate lifts. In those cases, geo-level tests with longer windows, synthetic controls, or MMM are more practical alternatives. Analytics-driven measurement programs consistently outperform gut-feel budget decisions, but only when the underlying experiment has sufficient power to detect the effect you care about.

Pro Tip: Run the feasibility calculation before you design anything else. If the numbers say the test cannot detect a 20% lift with your current conversion volume, no amount of experimental rigor will save it.

Holdout sizing involves a real tradeoff: a larger holdout (say, 20%) gives you more statistical power but means more users never see your ad, which costs revenue during the test. Most practitioners default to 10% and accept the longer test duration that comes with it.

How does incrementality testing compare to A/B testing, MMM, and attribution?

These methods answer different questions. Using the wrong one for a given business decision is one of the most common measurement mistakes in digital marketing.

Method comparison:

A/B testing compares two variants of a creative, landing page, or bid strategy to find which performs better. It measures relative performance between variants, not whether either variant drives net-new conversions versus no exposure at all.
Incrementality testing measures whether the campaign itself causes conversions that would not have occurred otherwise. It is the right tool when the question is “should we spend here at all?” rather than “which version works better?”
Marketing Mix Modeling (MMM) uses aggregate, historical data across all channels to estimate each channel’s contribution to revenue. It is excellent for cross-channel budget planning and long-term trend analysis, but it cannot isolate a single campaign’s causal effect with the precision an experiment can.
Attribution distributes credit for a conversion across the touchpoints in a user’s path. It answers “which touchpoints were present?” not “which touchpoints caused the conversion?” — a distinction that eMarketer’s incrementality FAQ makes explicit.

When to triangulate all three: The IAB recommends using incrementality tests as part of a triangulated measurement stack alongside MMM and attribution. In practice, this means using MMM for annual budget allocation across channels, attribution for campaign-level optimization signals, and incrementality tests to validate both, especially for channels suspected of reattributing organic demand.

When not to use incrementality testing:

Conversion volume is too low to power a test at a realistic lift size
The campaign is new and still in a learning phase
You need a fast answer (tests require full conversion cycles to complete)
The channel does not support user-level or geo-level holdouts

In those situations, geo testing with longer windows, synthetic controls, or relying on MMM estimates are more practical paths to causal insight.

Which campaigns and channels benefit most from lift testing?

Not every campaign warrants an incrementality test. Prioritize channels where attribution is most likely to be inflating apparent returns, or where budget decisions are large enough that a measurement error is costly.

High-priority use cases:

Paid social (Meta, TikTok, Pinterest): These platforms report attributed conversions that often include view-through credit for users who would have converted organically. Incrementality tests routinely show lower true lift than platform-reported ROAS.
Branded search: Are your branded keyword buys capturing demand you already own, or are they genuinely defending against competitor conquesting? A holdout test on branded terms answers this directly.
Connected TV (CTV) and upper-funnel video: Attribution is nearly impossible for CTV. Geo-level incrementality tests are the primary way to measure whether CTV spend drives downstream conversions.
Retail media networks: Retailers increasingly offer native lift testing. These tests are valuable for CPG and retail brands trying to separate incremental basket additions from purchases that would have happened anyway.
New-channel pilots: Before scaling spend on a new channel, a small geo test or synthetic control study can validate whether the channel adds incremental reach or just overlaps with existing channels.

Choosing the right conversion event matters as much as choosing the right channel. Purchase conversions give the clearest signal but require the most volume. Micro-conversions (add-to-cart, lead form submission, store visit) can be used when purchase volume is insufficient, but the relationship between the micro-conversion and revenue must be established separately. Testing campaign-level engagement as a proxy for downstream revenue is valid only when that proxy has a documented, stable correlation with actual sales.

Common pitfalls that invalidate incrementality tests

The most expensive mistake in lift testing is not a bad result. It is a bad test that produces a confident-looking number you cannot trust.

Pitfalls to avoid:

Underpowered tests: Running a test without a feasibility check first. A null result from an underpowered test almost always means the test could not detect the likely true lift, not that the channel had zero effect.
Changing creative or bids mid-test: Any change to the campaign during the measurement window introduces a confound. Lock creative, bidding strategy, and targeting before the test starts.
Contamination from cross-device behavior: A user assigned to the holdout on desktop may see the ad on mobile. Platform-level suppression handles this better than cookie-based methods.
Seasonality and external events: A competitor promotion or a news event during the test window can shift conversion rates in ways that mimic or mask lift. Document external events and consider them during analysis.
Post-hoc decision rules: Deciding what lift percentage justifies continued spend after seeing the results introduces bias. Write the decision rule before the test runs.
Reporting only the point estimate: A lift of 18% sounds meaningful until you see the confidence interval is 0%–36%. Always report the CI and the MDE.

Mitigation approach: Run feasibility first. Pre-register your analysis plan, including the decision rule. If user-level holdouts are impractical due to privacy constraints or platform limitations, shift to geo-level or synthetic control designs. The ACM paper on practical platform lessons covers privacy-safe techniques for exactly these situations.

Step-by-step checklist to run an incrementality test

Check feasibility. Calculate required conversions per group using the 15.68 ÷ (lift)² formula. If your campaign cannot reach the threshold, stop and choose an alternative method.
Write the decision rule first. Before any data is collected, document: “If incremental CPA exceeds $X, we reduce budget by Y%.” This prevents post-hoc rationalization.
Choose one variable. Test one channel, one campaign, or one audience segment. Testing multiple variables simultaneously makes results uninterpretable.
Set the holdout. Default to 10% unless power calculations require a larger share. Confirm the platform can suppress the holdout group at the user or geo level.
Run for a full conversion cycle. The minimum duration is one complete conversion cycle plus the conversion lag window. For most e-commerce campaigns, this means at least two to three weeks.
Analyze with confidence intervals. Report lift %, 95% CI, incremental conversions, incremental ROAS, and the MDE the test was powered to detect.
Act on the pre-committed rule. Apply the decision you wrote in step 2. Do not re-litigate the threshold after seeing results.

Holdout type selection guide:

Scenario	Recommended holdout type
Platform supports user-level suppression	User-level holdout
Privacy constraints or cross-device exposure risk	Geo-level holdout
Cannot go dark in any major market	Synthetic control
New channel with no historical data	Geo pilot with synthetic control

Pro Tip: Lock creative, bidding strategy, and audience targeting before the test starts and do not touch them until the measurement window closes. A single mid-test optimization can invalidate weeks of clean data.

Operational note: Presenc AI’s conversion lift guide recommends 5–15% holdouts as the standard platform range, with ~10% as the practical default that balances statistical power against foregone revenue during the test period.

How Theartistevolution applied incrementality testing for a retail client

A regional retail brand approached Theartistevolution with a specific question: was their paid social spend on Meta driving net-new customers, or were they paying to reach people who would have purchased through organic or direct channels anyway? Platform-reported ROAS looked strong, but the marketing team suspected attribution inflation.

Test design:

Holdout type: User-level holdout, managed through Meta’s Conversion Lift tool
Holdout size: 10% of the campaign audience
Duration: Four weeks, covering the full consideration-to-purchase cycle for the product category
Conversion event: Online purchase (not add-to-cart, because purchase volume was sufficient for the feasibility threshold)
Feasibility check: Confirmed before launch that projected purchase volume exceeded 800 total conversions across both groups

Outcome:

Platform-attributed ROAS: 4.2x
Measured incremental ROAS: 2.1x
Incremental lift: 22% above the holdout group’s baseline conversion rate
The gap between attributed and incremental ROAS confirmed that roughly half of platform-credited conversions were reattributed organic purchases

What changed: The team reallocated 30% of the Meta budget toward upper-funnel CTV placements, where geo-level testing subsequently showed stronger incremental reach among new-to-brand customers. The Meta budget that remained was restructured toward prospecting audiences rather than retargeting, where incrementality lift was measurably higher.

Lessons learned:

Retargeting audiences consistently show lower incremental lift than prospecting audiences in paid social environments.
Combining the incrementality result with MMM estimates confirmed the budget reallocation was directionally correct before the next planning cycle.
Running a marketing assessment before the test helped identify which campaigns had sufficient conversion volume to be testable, saving time on feasibility checks.

Key Takeaways

Incrementality testing is the only method that proves causal lift from a campaign — but feasibility, pre-committed decision rules, and confidence interval reporting determine whether the result is actually usable.

Point	Details
Feasibility comes first	Use the 15.68 ÷ (lift)² formula to confirm conversion volume before designing any test.
CI over point estimate	Always report the 95% confidence interval; a wide interval crossing zero means the result is not significant.
Triangulate with MMM and attribution	Use incrementality to validate MMM budget allocations and expose attribution inflation in platform reporting.
Match holdout type to constraints	Choose user-level for precision, geo-level for privacy safety, and synthetic control when going dark is not an option.
Theartistevolution	Theartistevolution designs and manages incrementality programs, from feasibility audits to full test execution and MMM triangulation.

The measurement stack most teams are still missing

Most marketing teams run attribution. A growing number run MMM. Far fewer have a systematic incrementality program, and that gap is where the most consequential budget decisions get made on the weakest evidence.

The conventional wisdom says to start with attribution and add incrementality later, once budgets are large enough. That framing gets the priority backwards. Attribution is the least reliable signal for causal questions, and it is precisely the large-budget channels where the cost of misreading attribution is highest. A 4x attributed ROAS that turns out to be 2x incremental ROAS on a $2 million annual channel is a $1 million misallocation, not a rounding error.

The pragmatic adoption path is to start where budgets are largest and where attribution is most likely to be inflating returns: retargeting, branded search, and upper-funnel video. Run feasibility checks on all three. Test the one that clears the conversion volume threshold first. Use that result to calibrate your MMM priors and to set a benchmark for what “real” ROAS looks like in your category.

The analytics and engineering lift required is real. Interpreting low-lift results correctly, distinguishing an underpowered null from a true zero effect, and integrating experiment outputs into an MMM framework all require data science skills that most in-house marketing teams are still building. Combining tests with MMM is not optional for a mature measurement stack. It is what separates a one-off experiment from a decision-making system.

Theartistevolution can build your incrementality program

Knowing the methodology is one thing. Running a valid test, interpreting the results correctly, and connecting them to real budget decisions is where most teams need support. Theartistevolution designs and manages full incrementality programs for clients across retail, healthcare, and CPG, covering feasibility audits, test design, holdout setup, analysis, and MMM triangulation.

Whether you need a one-time campaign measurement audit or an ongoing managed testing program that integrates with your broader measurement stack, Theartistevolution brings 18 years of applied campaign experience to the work. See how the agency has delivered measurable outcomes for clients in the brand engagement case studies. Request a measurement assessment to find out which of your campaigns are testable today.

Useful sources and tools

IAB Guidelines for Incremental Measurement in Commerce Media: The authoritative industry standard defining incrementality, holdout types, and reporting requirements for commerce media.
Google Ads: Incrementality Testing for Effective Marketing Measurement: Google’s practitioner guide covering conversion lift mechanics, geo experiments, and how to compute incremental ROAS from lift estimates.
Google Ads Help: Incrementality Testing Improvements: Platform-level documentation on setting up and interpreting conversion lift studies within Google Ads.
Soku: Incrementality Testing Guide: Covers the feasibility formula, sample-size thresholds, and the pre-committed decision rule framework used by practitioners.
Presenc AI: How to Run a Conversion Lift Test: Step-by-step guide on holdout sizing, power calculation inputs, and platform defaults for running a conversion lift study.
Amplitude: What Is Incrementality Testing?: Clear conceptual explanation of how incrementality testing separates caused outcomes from baseline behavior.
eMarketer: FAQ on Incrementality: Industry perspective on adoption drivers, the attribution vs. incrementality distinction, and why causal measurement is increasingly central to measurement stacks.
ACM: Online Advertising Incrementality Testing — Practical Lessons: Academic and engineering survey of privacy-safe techniques, synthetic controls, and platform-level tradeoffs for precision in incrementality programs.
How to plan digital campaigns that drive organic growth: Practical campaign planning guidance relevant to designing tests that account for organic demand cannibalization.