LiftSignals is live. Every plan starts with a 30-day free trial — no credit card required. Start free →
Incrementality & measurement

Incrementality vs attribution: what each one tells you, and when to use which

Attribution says who touched the sale. Incrementality says whether the sale would have happened anyway. They answer different questions.

By LiftSignals editorial team · Published · 4 min read

A brass caliper measuring a small orange clay block beside a gray one

Key takeaways

  • Attribution assigns credit for a sale to the touchpoints near it.
  • Incrementality measures whether the marketing caused the sale, by comparing with a group that didn't get it.
  • Attribution overstates retention marketing, because loyal customers buy anyway.
  • A holdout test is the simplest way to measure incrementality.

Incrementality and attribution both try to answer “did our marketing work?”, but they answer different versions of the question. Attribution assigns credit for a sale to the marketing that touched it. Incrementality asks whether the sale would have happened without that marketing at all. For online stores, especially in email and retention marketing, the gap between the two is often large. This guide explains both, why they disagree, and how to run a simple incrementality test.

What is marketing attribution?

Attribution looks at the touchpoints before a purchase, such as an email click, an ad view or a search, and assigns credit to them. Models differ in how they split the credit: last click gives it all to the final touch, multi-touch spreads it across several.

Attribution is fast and always on. Its weakness is that it can't tell whether a touchpoint changed the outcome. It only knows it was there.

What is incrementality?

Incrementality measures the difference marketing makes. You split an audience at random, send the campaign to one group and hold the other back. Because the groups were chosen at random, the holdout shows what would have happened anyway.

Incremental orders = orders from the sent group − orders the sent group would have placed anyway

Why attribution overstates retention marketing

Retention campaigns go to people who already buy from you. Many of them would order this week regardless. Attribution counts their orders as the campaign's, incrementality doesn't. The email holdout test guide shows how to measure this on a single Klaviyo flow.

Here's an illustrative win-back email sent to 8,100 customers, with 900 held out:

Customers Orders Rate
Sent the email 8,100 478 5.9%
Held out 900 36 4.0%
Expected anyway (4.0% of 8,100) 324
Caused by the email 154

At an $86.80 average order, the email platform would attribute about $41,490 to the campaign. The holdout shows it caused about $13,370. Both numbers are real; only one tells you what the email was worth.

A smaller number you can defend beats a bigger one nobody believes.

When to use attribution vs incrementality

Question Use
Which ad creative is performing today? Attribution
Did this campaign change behavior? Incrementality
Should we keep paying for this channel? Incrementality, or marketing mix modeling
How do we split next quarter's budget? Marketing mix modeling, checked with incrementality tests

How to run your first holdout test

  1. Pick one campaign you send repeatedly, such as a win-back flow.
  2. Hold out a random 10% of the audience. Random matters; don't hand-pick.
  3. Check the sample size before you send. The holdout sample size calculator page shows how big the audience needs to be.
  4. Fix the window, such as 30 days, and don't stop early because the numbers look good.
  5. Compare order rates between the groups and multiply the difference by the sent group size.

In LiftSignals

LiftSignals keeps a random holdout on every audience it syncs, excludes it at the destination, and reports caused orders and revenue with a confidence interval. Results that aren't significant yet are shown as not proven.

Incrementality vs attribution questions

Is incrementality better than attribution?

They're for different jobs. Attribution is fast and useful for comparing ads day to day. Incrementality is slower but tells you whether the spend changed anything, which is what budget decisions need.

Why do platforms report more revenue than my store made?

Each platform credits itself for the orders it touched. When one order was touched by email, Meta and Google, all three can claim it.

What is incremental revenue?

The revenue that happened because of a campaign and wouldn't have happened without it. It's measured as the difference between a group that got the campaign and a comparable group that didn't.

What is a geo-lift test?

An incrementality test where whole regions are held out instead of individual customers. It's used for channels like TV or broad ads that can't be withheld from specific people.

Put it to work: Holdout sample size calculator → · Incrementality testing → · Marketing mix modeling →