Country: Mexico
You are in the Mexico edition. Type your country to switch.
Nigeria
The Americas
Loading every country…
No country matches. Check the spelling, or choose "Any country" for the Global edition.
You are learning in the Mexico edition, with examples in pesos and from Mexico.
Your learning profile
Statistics and fundamental analytics · Lesson 4 of 8
Significance and data-mining
Significance tests ask how surprising a result would be if there were nothing real behind it. Noisy market returns mean short track records prove little, and testing many ideas always produces some lucky winners, so every variation tried counts as a test.
This lesson is complete in writing. A video for it is on its way.
What you will learn
- By the end you can explain a null hypothesis, a p-value and statistical significance in plain words.
- You can calculate a standard error and a t-statistic for an average return and judge whether it stands out from noise.
- You can explain why testing many ideas produces impressive-looking results by chance.
- You can describe simple defences against data mining: fewer tests, stricter thresholds and fresh data.
Key terms in this lesson
- Null hypothesis
- The plain explanation a statistical test tries to rule out, such as 'the strategy's true average excess return is zero'.
- P-value
- The probability of seeing a result at least as strong as the one observed if the null hypothesis were true. It is not the probability that the idea is true.
- Standard error
- The uncertainty of an average estimated from a sample: the standard deviation divided by the square root of the number of observations.
- Statistical significance
- A result is called statistically significant when its p-value is below a threshold set in advance, often 5%. It does not mean the effect is large or important.
- Data mining
- In investing, searching many rules, filters or variables and keeping those that worked in the past. Reporting only the winners makes chance results look like real effects.
- Multiple testing
- The problem that testing many ideas at a fixed threshold makes some pass by luck. The more tests, the higher the threshold each should meet.
- Backtest
- A test of how a strategy would have performed on past data.
- Overfitting
- Tuning a model or strategy so closely to past data that it captures noise and fails on new data.
- Step 1: Signal or noise?, not started
- Step 2: Standard error and the t-statistic, not started
- Step 3: What significance does and does not mean, not started
- Step 4: Data mining: when you test too many ideas, not started
- Step 5: Defences against fooling yourself, not started
About 9 min of reading
Step 1 of 5
Signal or noise?
Opening this step…
Step 2 of 5
Standard error and the t-statistic
This step opens when you finish the one above.
Step 3 of 5
What significance does and does not mean
This step opens when you finish the one above.
Step 4 of 5
Data mining: when you test too many ideas
This step opens when you finish the one above.
Step 5 of 5
Defences against fooling yourself
This step opens when you finish the one above.
Key takeaways
The key takeaways open when you finish every step.
Lesson quiz
6 questions. You need 85% to pass.
The quiz opens when you finish every step.
Read first
Stockertim Learn is education. It explains how investing works and never recommends buying or selling anything.
Popular in Mexico: Financial markets law and regulation: foundationsProtect your money and accounts