Skip to content
Edition NigeriaGlobal
Country: Mexico

You are in the Mexico edition. Type your country to switch.

The Americas

Set up your profile

You are learning in the Mexico edition, with examples in pesos and from Mexico.

Statistics and fundamental analytics · Lesson 4 of 8

Significance and data-mining

Significance tests ask how surprising a result would be if there were nothing real behind it. Noisy market returns mean short track records prove little, and testing many ideas always produces some lucky winners, so every variation tried counts as a test.

Intermediate10 min read5 steps

This lesson is complete in writing. A video for it is on its way.

What you will learn

  • By the end you can explain a null hypothesis, a p-value and statistical significance in plain words.
  • You can calculate a standard error and a t-statistic for an average return and judge whether it stands out from noise.
  • You can explain why testing many ideas produces impressive-looking results by chance.
  • You can describe simple defences against data mining: fewer tests, stricter thresholds and fresh data.

Key terms in this lesson

Null hypothesis
The plain explanation a statistical test tries to rule out, such as 'the strategy's true average excess return is zero'.
P-value
The probability of seeing a result at least as strong as the one observed if the null hypothesis were true. It is not the probability that the idea is true.
Standard error
The uncertainty of an average estimated from a sample: the standard deviation divided by the square root of the number of observations.
Statistical significance
A result is called statistically significant when its p-value is below a threshold set in advance, often 5%. It does not mean the effect is large or important.
Data mining
In investing, searching many rules, filters or variables and keeping those that worked in the past. Reporting only the winners makes chance results look like real effects.
Multiple testing
The problem that testing many ideas at a fixed threshold makes some pass by luck. The more tests, the higher the threshold each should meet.
Backtest
A test of how a strategy would have performed on past data.
Overfitting
Tuning a model or strategy so closely to past data that it captures noise and fails on new data.

Step 1 of 5

Signal or noise?

Opening this step…

Step 2 of 5

Standard error and the t-statistic

This step opens when you finish the one above.

Step 3 of 5

What significance does and does not mean

This step opens when you finish the one above.

Step 4 of 5

Data mining: when you test too many ideas

This step opens when you finish the one above.

Step 5 of 5

Defences against fooling yourself

This step opens when you finish the one above.

Key takeaways

The key takeaways open when you finish every step.

Lesson quiz

6 questions. You need 85% to pass.

The quiz opens when you finish every step.

Read first

Stockertim Learn is education. It explains how investing works and never recommends buying or selling anything.

Up next

Financial data & analyticsRatio analysis at scaleRead nowRatio analysis at scaleFinancial data & analytics · Intermediate · 15 minFinancial data & analyticsScreening shares with dataRead nowScreening shares with dataFinancial data & analytics · Intermediate · 15 minFinancial data & analyticsPeer comparisonRead nowPeer comparisonFinancial data & analytics · Intermediate · 15 minFinancial data & analyticsBuilding a simple valuation mode…Read nowBuilding a simple valuation model from dataFinancial data & analytics · Intermediate · 15 minFinancial crimeCyber-enabled and crypto-related…Read nowCyber-enabled and crypto-related financial crimeFinancial crime · Intermediate · 15 minAI at workWriting and editing with AIRead nowWriting and editing with AIAI at work · Beginner · 15 minMedia strategy & planningGRPs, TRPs and impressionsRead nowGRPs, TRPs and impressionsMedia strategy & planning · Intermediate · 5 minMarket operationsMargin and risk desks in operati…Read nowMargin and risk desks in operationsMarket operations · Intermediate · 15 minRisk, governance & ethicsEthical decision-making: a step-…Read nowEthical decision-making: a step-by-step frameworkRisk, governance & ethics · Beginner · 15 minMexican marketsBolsa Mexicana de Valores (BMV)Read nowWhat is Bolsa Mexicana de Valores (BMV)?Mexican markets · Beginner · 15 minFinancial journalismAttribution and quotes: who says…Read nowAttribution and quotes: who says so, and whereFinancial journalism · Beginner · 15 minRisk, governance & ethicsethics matters in businessRead nowWhy ethics matters in businessRisk, governance & ethics · Beginner · 15 min
English