Incrementality vs attribution: what each one tells you, and when to use which
Attribution says who touched the sale. Incrementality says whether the sale would have happened anyway. They answer different questions.
Key takeaways
- Attribution assigns credit for a sale to the touchpoints near it.
- Incrementality measures whether the marketing caused the sale, by comparing with a group that didn't get it.
- Attribution overstates retention marketing, because loyal customers buy anyway.
- A holdout test is the simplest way to measure incrementality.
Incrementality and attribution both try to answer “did our marketing work?”, but they answer different versions of the question. Attribution assigns credit for a sale to the marketing that touched it. Incrementality asks whether the sale would have happened without that marketing at all. For online stores, especially in email and retention marketing, the gap between the two is often large. This guide explains both, why they disagree, and how to run a simple incrementality test.
What is marketing attribution?
Attribution looks at the touchpoints before a purchase, such as an email click, an ad view or a search, and assigns credit to them. Models differ in how they split the credit: last click gives it all to the final touch, multi-touch spreads it across several.
Attribution is fast and always on. Its weakness is that it can't tell whether a touchpoint changed the outcome. It only knows it was there.
What is incrementality?
Incrementality measures the difference marketing makes. You split an audience at random, send the campaign to one group and hold the other back. Because the groups were chosen at random, the holdout shows what would have happened anyway.
Incremental orders = orders from the sent group − orders the sent group would have placed anyway
Why attribution overstates retention marketing
Retention campaigns go to people who already buy from you. Many of them would order this week regardless. Attribution counts their orders as the campaign's, incrementality doesn't. The email holdout test guide shows how to measure this on a single Klaviyo flow.
Here's an illustrative win-back email sent to 8,100 customers, with 900 held out:
| Customers | Orders | Rate | |
|---|---|---|---|
| Sent the email | 8,100 | 478 | 5.9% |
| Held out | 900 | 36 | 4.0% |
| Expected anyway (4.0% of 8,100) | 324 | ||
| Caused by the email | 154 |
At an $86.80 average order, the email platform would attribute about $41,490 to the campaign. The holdout shows it caused about $13,370. Both numbers are real; only one tells you what the email was worth.
A smaller number you can defend beats a bigger one nobody believes.
When to use attribution vs incrementality
| Question | Use |
|---|---|
| Which ad creative is performing today? | Attribution |
| Did this campaign change behavior? | Incrementality |
| Should we keep paying for this channel? | Incrementality, or marketing mix modeling |
| How do we split next quarter's budget? | Marketing mix modeling, checked with incrementality tests |
How to run your first holdout test
- Pick one campaign you send repeatedly, such as a win-back flow.
- Hold out a random 10% of the audience. Random matters; don't hand-pick.
- Check the sample size before you send. The holdout sample size calculator page shows how big the audience needs to be.
- Fix the window, such as 30 days, and don't stop early because the numbers look good.
- Compare order rates between the groups and multiply the difference by the sent group size.
In LiftSignals
LiftSignals keeps a random holdout on every audience it syncs, excludes it at the destination, and reports caused orders and revenue with a confidence interval. Results that aren't significant yet are shown as not proven.
Incrementality vs attribution questions
Is incrementality better than attribution?
They're for different jobs. Attribution is fast and useful for comparing ads day to day. Incrementality is slower but tells you whether the spend changed anything, which is what budget decisions need.
Why do platforms report more revenue than my store made?
Each platform credits itself for the orders it touched. When one order was touched by email, Meta and Google, all three can claim it.
What is incremental revenue?
The revenue that happened because of a campaign and wouldn't have happened without it. It's measured as the difference between a group that got the campaign and a comparable group that didn't.
What is a geo-lift test?
An incrementality test where whole regions are held out instead of individual customers. It's used for channels like TV or broad ads that can't be withheld from specific people.
Put it to work: Holdout sample size calculator → · Incrementality testing → · Marketing mix modeling →
Keep reading
All incrementality & measurement guides →Start here · Email & Klaviyo
Email holdout test: how to prove what a Klaviyo flow actually earns
Your email platform says the flow made $9,000. A coin flip and a little patience will tell you whether it made $2,600.
11 min read · Updated Nov 18, 2026
Start here · Predictive analytics
Predictive analytics for ecommerce: what it predicts, what it needs, and where it misleads
A good model tells you who's about to buy. A good marketer asks the next question: would they have bought anyway?
12 min read · Updated Nov 18, 2026
Start here · Discounts & pricing
Discount addiction: how online stores train customers to wait for a sale, and how to stop
The coupon that pays for itself on the dashboard and quietly costs you on the P&L. Here's how to tell the difference, and how to stop.
12 min read · Updated Nov 17, 2026
Stop calculating by hand.
LiftSignals scores every customer, every night, from your own orders. 30 days free.