Incrementality: What It Is, What to Test, and How to Measure It

A loyal customer reorders the same vape cartridge from your dispensary every two weeks. It's restock day, and she's already decided to buy. On her way to your online menu she taps a retargeting ad for the exact product she was coming for. The ad platform records a conversion and claims the sale. But that customer was always going to reorder — the ad changed nothing. It harvested a purchase that was already going to happen.

Now multiply that across your entire returning-customer base. A large share of the revenue your dashboard credits to advertising is exactly this: demand that existed with or without the ad. That gap — between the credit advertising claims and the outcomes it actually caused — is the single most expensive blind spot in cannabis marketing. Incrementality is how you close it. This guide explains incrementality from every angle: what it is, what you can test, when it's worth the investment, and how to measure it. For the broader measurement picture this fits into, see Marketing Attribution for Cannabis.

What Is Incrementality?

Incrementality tells you whether a change in marketing caused a change in outcome. Did increasing your Meta spend actually drive more orders, or would those orders have happened anyway? Incrementality answers that by borrowing the scientific method — the same logic a clinical trial uses to prove a drug works.

In a drug trial, one group gets the treatment and another gets a placebo, and scientists compare outcomes. In marketing, you split your audience: one group sees a campaign (the treatment group), one group doesn't (the control, or holdout, group). If the treatment group converts meaningfully more than the control group, the difference is incremental — conversions that exist because of your advertising and would not have happened otherwise.

Everything else is baseline: demand that was going to convert regardless of whether an ad ran. Standard attribution can't tell the two apart. Incrementality can — and that distinction is what makes it the most trustworthy read on marketing performance, especially as privacy rules erode user-level tracking.

What Can You Test With Incrementality?

A marketer's job is really two tasks: making decisions, and figuring out whether those decisions worked. Incrementality handles the second so you get better at the first. The range of things you can test is wide.

Channel-mix decisions

• Which new channel is actually worth investing in?
• What's the most incremental mix of channels for your brand?
• Are you spending the right amount on each channel — or past the point of diminishing returns?
• How should you reallocate budget during peak season (420, Green Wednesday, holidays)?

Campaign-level questions

• Should you include or exclude branded search terms in your campaigns?
• Is upper-funnel awareness creative or lower-funnel promotional creative more incremental?
• What's the right ratio of top-of-funnel to bottom-of-funnel media?
• Broad targeting or intent-based targeting?

Omnichannel questions

• Are customers converting in-store, through delivery, or on your ecommerce menu — and is advertising shifting them between channels?
• What's the true ROI of an in-store or merchandising program?
• Is your display spend lifting walk-in traffic you're not crediting? (See The Halo Effect in Marketing for measuring this.)

The Incremental Lift Formula

Incremental lift has a simple formula. Compare the conversion rate of the treatment group (saw the ads) to the control group (didn't):

Incremental lift = (Treatment conversion rate − Control conversion rate) ÷ Control conversion rate

If your treatment group converts at 5% and your control group converts at 2%, the lift is (0.05 − 0.02) ÷ 0.02 = 1.5. In plain terms: customers were 150% more likely to convert because they saw the ad. That's a campaign doing real work. If the two groups convert at nearly the same rate, the campaign isn't incremental — it's getting credit for demand that already existed.

When Should You Invest in Incrementality?

Incrementality testing takes time and budget, so it isn't for everyone yet. Ask yourself these five questions — the more you answer yes, the more you need it.

1. Are you spending on more than two channels?

With one or two channels, the cause-and-effect picture is simple. Add programmatic, CTV, DOOH, search, social, and email, and it becomes nearly impossible to know which is actually driving sales. More channels, more need for incrementality.

2. Are you spending meaningfully on media each month?

The more you spend, the more an inefficiency costs you. A test that reveals you can cut 25% of a channel's budget with no drop in conversions pays for itself many times over.

3. Is your business omnichannel?

If you sell in-store, through delivery, and on an ecommerce menu, platform reporting can't cleanly track lift across all of them. Incrementality can measure media impact across every sales channel with one methodology.

4. Have you hit a growth plateau?

A few flat quarters and you're not sure where the problem is. Incrementality quickly surfaces which channels are genuinely working and which only looked good in platform metrics, so you can cut the dead weight and reallocate.

5. Are MMM and MTA not cutting it?

Media mix modeling and multi-touch attribution both rely on correlation, not causation. MTA is breaking down as privacy rules tighten, and traditional MMM takes too long to deliver an actionable read. Incrementality is causal — you can even use it to calibrate those models. We cover the distinction in Media Mix Modeling vs. Attribution.

How to Measure Incrementality

The foundation is a holdout test. Run a campaign where most of your audience sees it (the treatment group) and a portion doesn't (the holdout, or control). Then compare conversion rates between the two using the lift formula above.

The quality of the test depends entirely on how comparable the two groups are. If your treatment and control groups behave differently to begin with, the comparison is meaningless. The best tests use synthetic control or careful matching to build groups that look alike on the dimensions that matter — prior purchase behavior, geography, frequency — so the only meaningful difference between them is the advertising.

This is the methodology built into DataJel: exposed and control cohorts matched at the customer level, compared with Difference-in-Differences and Double Stratification so the lift number holds up.

How to Run an Incrementality Experiment

Step 1: Design the experiment

Start with the biggest question you can't currently answer — the one that keeps you up at night. Say it's: "Are we spending enough on CTV?" Then control for variables by building test groups that are characteristically indistinguishable. You wouldn't compare a mature, high-volume California market against a brand-new launch market and call the difference "lift" — the markets were never alike to begin with. Test and control groups need similar baseline sales behavior, customer profiles, and competitive density, so the only meaningful difference between them is the advertising you're testing.

Define your testing variable (CTV spend), set your groups — for example, business-as-usual spend, double spend, and a no-CTV control — and pick your KPI (say, new ecommerce orders). Then choose a runtime. Longer tests give more confidence and suit long consideration cycles, but cost more because part of your audience is under-marketed for the duration. Balance the two deliberately.

Step 2: Launch the test

Run the campaign and collect KPI data in real time. You want visibility into lift amount, lift percent, and lift likelihood (the probability the true lift is greater than zero) so you can read significance, not just direction.

Step 3: Analyze and act

When the results land, you'll have the metrics to move. A few common next steps:

• Optimize spend levels — if lift flattens when you double spend, you've found diminishing returns. Don't scale past them.
• Plan the next test — every answer raises new questions. Incrementality compounds.
• Calibrate platform attribution — use the incrementality factor (below) to correct over-credited channels.
• Reset CPA/ROAS thresholds — translate true cost-per-incremental into the platform target you should optimize against.

The Metrics That Matter

Incrementality Factor (IF): of the conversions the platform claims, how many were actually incremental. If a platform reports 1,000 conversions and your IF is 0.6, only 600 were real lift — the platform over-credited by 40%.

Lift Amount: the incremental units of your KPI observed in the treated audience during the test.

Lift Percent: lift amount divided by total volume of that KPI.

CPI / iROAS: cost per incremental acquisition (or incremental return on ad spend) — the real cost of driving one more incremental conversion. This is the number to optimize against, not platform-reported CPA.

Worked example: your goal is a $150 cost per incremental acquisition. Your IF on a channel is 0.60. Multiply: $150 × 0.60 = $90. So as long as the platform reports a CPA of $90 or below, you're hitting your true $150 incremental goal.

Incrementality vs. Attribution vs. MMM

These three get conflated constantly. Attribution assigns credit for conversions to touchpoints — useful, but correlation-based. Media mix modeling estimates channel contribution at the brand level over long time horizons — strategic, but slow and also correlational. Incrementality is the only one of the three that measures causation: it proves what your advertising actually caused by comparing against a control.

They work best together: incrementality reads can calibrate both MTA and MMM, anchoring correlational models to causal truth. For the attribution side, see Marketing Attribution for Cannabis; for the modeling distinction, Media Mix Modeling vs. Attribution. For the channel-spillover question that incrementality methodology also answers, see The Halo Effect in Marketing.

Incrementality and View-Through Attribution

Incrementality is also the only credible validation of view-through attribution. View-through credits ads that were seen but not clicked — useful, but easy to inflate with overlong windows. Running an exposed-vs-control test against view-through credit tells you whether the credit reflects real lift or harvested baseline. If you report view-through without validating it, you're trusting the platform's word.

Why Incrementality Matters More in Cannabis

Cannabis makes incrementality more valuable, not less. Three reasons. First, the channels that carry the load — display, CTV, DOOH — are exactly the ones standard attribution undercounts, because they're seen rather than clicked. Without incrementality, you'll systematically defund the channels doing the most work.

Second, brand loyalty is low and shoppers switch on price, so a large share of "attributed" revenue is returning customers who'd have come back regardless. Incrementality separates the customers your advertising actually moved from the ones who were always going to reorder.

Third, multi-state operators have a natural laboratory for geo-experiments: run a campaign in some markets, hold out comparable ones, and measure the difference. A market-level incrementality test is often the cleanest read an MSO can get — and it sidesteps the user-level tracking restrictions that hamstring other measurement in cannabis.

Key Takeaways

• Incrementality measures the revenue your advertising caused, not the revenue it got credit for.
• It works by comparing a treated group to a matched control (holdout) group and measuring the lift.
• Lift = (treatment rate − control rate) ÷ control rate. The incrementality factor tells you how much platform credit is real.
• Invest when you run multiple channels, spend meaningfully, sell omnichannel, hit a plateau, or outgrow MMM/MTA.
• In cannabis it matters more: restricted channels, low loyalty, and multi-state geo-experiments all raise the value of a causal read.

Put Incrementality to Work

Incrementality is the measurement engine inside DataJel, MediaJel's analytics platform for cannabis operators. It builds exposed and control cohorts from your own POS data and reports lift, incremental ROAS, and revenue defended — the numbers that defend a budget. To see how incrementality reframes your return on ad spend, read How to Measure Cannabis ROAS. For the full attribution picture this fits into, see Marketing Attribution for Cannabis.