Skip to content
Edition NigeriaGlobal
Country

You are in the Nigeria edition. Type your country to switch.

Set up your profile

Marketing analytics and measurement · Lesson 4 of 9

Experiments and incrementality

The only way to know what a campaign caused is to compare people who saw it with similar people who did not. This lesson explains test and control groups, geographic tests and holdouts, then works an uplift calculation from conversion rates to incremental sales and cost per incremental sale.

Advanced7 min read5 steps

This lesson is complete in writing. A video for it is on its way.

What you will learn

  • By the end you can explain what incrementality means and why it is the only measure of what marketing caused.
  • You can describe a test group, a control group, a holdout and a geographic test.
  • You can calculate uplift, incremental sales and cost per incremental sale from two conversion rates.
  • You can say why a platform's cost per sale and the cost per incremental sale differ so much.

Key terms in this lesson

Incrementality
The extra result that a piece of marketing caused, found by comparing what happened with it against what would have happened without it.
Control group
A set of people or places that is deliberately not shown a piece of marketing so that its results can be compared fairly with a matched group that was.
Percentage point
The unit for the gap between two percentages. A rate that moves from 4% to 5% has risen 1 percentage point, which is a 25% rise in the rate itself.
Sample size
The number of observations, such as trades, in a test. Small samples make it hard to tell a real effect from luck.

Step 1 of 5

What incrementality means

Opening this step…

Step 2 of 5

Test, control, holdout and geo

This step opens when you finish the one above.

Step 3 of 5

A worked uplift calculation

This step opens when you finish the one above.

Step 4 of 5

Reported cost per sale against incremental

This step opens when you finish the one above.

Step 5 of 5

Running a test that can be trusted

This step opens when you finish the one above.

Key takeaways

The key takeaways open when you finish every step.

Lesson quiz

5 questions. You need 85% to pass.

The quiz opens when you finish every step.

Read first

Stockertim Learn is education. It explains how investing works and never recommends buying or selling anything.

Up next

Media strategy & planningMarketing mix modelling in plain…Read nowMarketing mix modelling in plain wordsMedia strategy & planning · Advanced · 5 minMedia strategy & planningBrand and performance metrics to…Read nowBrand and performance metrics togetherMedia strategy & planning · Advanced · 5 minMedia strategy & planningUnit economics: acquisition cost…Read nowUnit economics: acquisition cost, lifetime value and return on spendMedia strategy & planning · Advanced · 5 minMedia strategy & planningDashboards, reporting and tellin…Read nowDashboards, reporting and telling the storyMedia strategy & planning · Advanced · 5 minMedia strategy & planningAttribution models and their lim…Read nowAttribution models and their limitsMedia strategy & planning · Advanced · 5 minMedia strategy & planningThe measurement plan: questions …Read nowThe measurement plan: questions before dataMedia strategy & planning · Advanced · 5 minAI at workTesting and optimisationRead nowTesting and optimisationAI at work · Intermediate · 15 minMedia strategy & planningMetrics across the funnelRead nowMetrics across the funnelMedia strategy & planning · Advanced · 5 minMedia strategy & planningA first digital plan: channels, …Read nowA first digital plan: channels, budget and resultsMedia strategy & planning · Beginner · 5 minMedia strategy & planningMedia costs: CPM, CPP, CPC and CPARead nowMedia costs: CPM, CPP, CPC and CPAMedia strategy & planning · Intermediate · 5 minMedia strategy & planningMobile, apps and messaging platf…Read nowMobile, apps and messaging platformsMedia strategy & planning · Beginner · 5 minMedia strategy & planningSearch marketing: SEO and paid s…Read nowSearch marketing: SEO and paid searchMedia strategy & planning · Beginner · 5 min
English