Stockertim Dictionary · Marketing
Control group
Also called holdout group, holdout, control cell
In short
A set of people or places deliberately not shown a piece of marketing, so that their results can be compared fairly with a matched group that was.
- Beginner
- Concept
- Checked
In detail
A control group supplies the comparison that makes an experiment readable. The idea comes from medical trials: to learn whether a treatment works, researchers compare people who received it with similar people who did not. In marketing the treatment is a campaign, a message or an offer, and the control group shows what would have happened without it.
The two groups have to be alike before the test begins. For individuals, random assignment does this: a list of customers is split by chance, so differences in age, loyalty or habits balance out. For television, radio or posters, where people cannot be held back one at a time, whole regions act as units and are paired on past sales and population, with one of each pair receiving the campaign.
A holdout is a common form. A small share of customers, often a few per cent, is kept out of all marketing for a period, giving a running measure of what the whole programme adds. Because the held-out people represent a deliberate cost, the share is kept just large enough to give a usable answer.
Control groups fail when they are contaminated, for instance when another campaign reaches them, when they differ from the exposed group from the start, or when they are too small to show a realistic difference. Analysts check that the groups moved together before the test and fix the period and metric in advance.
Worked example
Brightway Bank holds back 5% of its 200,000 customers from a savings email. The held-back group opens accounts at 1.2% and the emailed group at 1.5%.
- Holdout size: 200,000 × 5% = 10,000
- Emailed customers: 200,000 − 10,000 = 190,000
- Accounts expected without the email: 190,000 × 1.2% = 2,280
- Accounts opened: 190,000 × 1.5% = 2,850
- Accounts added by the email: 2,850 − 2,280 = 570
The holdout shows that the email added about 570 accounts; the rest would have been opened anyway.
In practice
Email, app and loyalty teams keep standing holdouts; media teams run matched-region tests; product teams use control groups when testing pages and features against one another.
Watch out
- A group chosen for convenience, such as customers who opted out, is not a fair control.
- Letting other campaigns reach the control group shrinks the measured uplift.
Learn it properly
- Experiments and incrementality5 min lesson Free course: Marketing analytics and measurement
- Measuring AI-driven marketing15 min lesson Free course: AI for marketing
- Distressed debt and restructuring15 min lesson Free course: Credit analysis
- Sovereign credit analysis15 min lesson Free course: Credit analysis
Official sources
- What is experimental design? · National Institute of Standards and Technology
- Choosing an experimental design · National Institute of Standards and Technology
- The Magenta Book: guidance on evaluation · HM Treasury
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.