Skip to content

Stockertim Dictionary · Marketing

Control group

Also called holdout group, holdout, control cell

In short

A set of people or places deliberately not shown a piece of marketing, so that their results can be compared fairly with a matched group that was.

  • Beginner
  • Concept
  • Checked

In detail

A control group supplies the comparison that makes an experiment readable. The idea comes from medical trials: to learn whether a treatment works, researchers compare people who received it with similar people who did not. In marketing the treatment is a campaign, a message or an offer, and the control group shows what would have happened without it.

The two groups have to be alike before the test begins. For individuals, random assignment does this: a list of customers is split by chance, so differences in age, loyalty or habits balance out. For television, radio or posters, where people cannot be held back one at a time, whole regions act as units and are paired on past sales and population, with one of each pair receiving the campaign.

A holdout is a common form. A small share of customers, often a few per cent, is kept out of all marketing for a period, giving a running measure of what the whole programme adds. Because the held-out people represent a deliberate cost, the share is kept just large enough to give a usable answer.

Control groups fail when they are contaminated, for instance when another campaign reaches them, when they differ from the exposed group from the start, or when they are too small to show a realistic difference. Analysts check that the groups moved together before the test and fix the period and metric in advance.

Worked example

Brightway Bank holds back 5% of its 200,000 customers from a savings email. The held-back group opens accounts at 1.2% and the emailed group at 1.5%.

  1. Holdout size: 200,000 × 5% = 10,000
  2. Emailed customers: 200,000 − 10,000 = 190,000
  3. Accounts expected without the email: 190,000 × 1.2% = 2,280
  4. Accounts opened: 190,000 × 1.5% = 2,850
  5. Accounts added by the email: 2,850 − 2,280 = 570

The holdout shows that the email added about 570 accounts; the rest would have been opened anyway.

In practice

Email, app and loyalty teams keep standing holdouts; media teams run matched-region tests; product teams use control groups when testing pages and features against one another.

Watch out

  • A group chosen for convenience, such as customers who opted out, is not a fair control.
  • Letting other campaigns reach the control group shrinks the measured uplift.

Learn it properly

Official sources

Facts last checked on 3 November 2026.

Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.

English