Even a genuinely random sample won't perfectly match the whole population every time — there's natural variation between different samples, just by chance. A key rule of thumb: larger samples tend to give results closer to the true population value, with less variation between repeated samples, while smaller samples can swing more wildly from one to the next. This is why a poll of 10 people is far less reliable than a poll of 1,000, even if both are collected randomly — it's not about bias, just the mathematics of how much random variation shrinks as sample size grows.
Example
Flipping a coin 10 times might land 7 heads, 3 tails (70% heads) purely by chance — quite far from the true 50%. Flipping it 1,000 times is far more likely to land close to 500 heads (50%), because the larger sample smooths out random swings that are more noticeable in a small sample.
Key terms
Sample variation:
The natural difference in results between different random samples of the same population.
Sample size:
The number of individuals or data points included in a sample.
Questions
1. Sample variation refers to:
Natural differences in results between different random samples
A sample that is always identical every time
A mistake in data collection
A type of census
2. Larger samples tend to give results:
Closer to the true population value, with less variation
Always exactly identical to a small sample
Further from the true value
Completely unrelated to the population
3. A poll of 1,000 people is generally more reliable than a poll of:
10 people
Another 1,000 people
The entire population
No one at all
4. Flipping a coin only 10 times is more likely to show a result far from 50% heads because:
Small samples have more random variation
Coins behave differently in small samples
This never actually happens
Small samples are always exactly accurate
5. A random sample is:
Selected by chance, giving everyone a fair opportunity to be included
Always chosen based on convenience
Never a fair representation
Only ever the entire population
6. Why might two random samples of the same size, taken from the same population, give slightly different results?
Random chance naturally creates some variation between different samples
Random samples always give identical results every time
This would only happen due to a mistake
Sample size has no effect on variation
7. Sample size refers to:
The number of individuals or data points in a sample
The physical size of the survey paper
The population size only
A type of census
8. Flipping a coin 10 times gives 7 heads (70%). Flipping it 1,000 times gives 510 heads (51%). Why is the second result closer to the true 50%?
Larger sample sizes tend to smooth out random variation
This is purely a coincidence with no underlying reason
Coins behave differently after many flips
50% is only true for samples over 500
9. A survey of 20 people finds 60% prefer chocolate ice cream. Why should this result be treated cautiously as an estimate of the whole population?
A small sample size can vary considerably from the true population value just by chance
A sample of 20 always exactly matches the population
Ice cream preferences cannot vary by chance
This sample size is always sufficient for full confidence
10. Why might a pollster increase their sample size before an important election prediction?
A larger sample reduces random variation, giving a more reliable estimate
Sample size has no effect on the reliability of a prediction
Larger samples always introduce more bias
Increasing sample size guarantees a perfectly accurate prediction
11. Comparing samples of size 10 and size 500 from the same population, which would you expect to vary more between repeats?
The sample of size 10
The sample of size 500
Both would vary identically
Neither would ever vary
12. Why does "random" not mean a sample will always perfectly reflect the population?
Random selection still involves chance, which naturally creates some variation from the true population value
Random samples are actually guaranteed to be perfectly accurate
Randomness eliminates all possible variation
This statement is not actually true
13. A school survey asks 5 students and 50 students the same question. Which result would you trust more as representative of the whole school?
The result from 50 students
The result from 5 students
Both are equally trustworthy
Neither can be trusted at all
14. Why might repeating a survey with a new random sample of the same size give a slightly different result each time?
Each sample includes different individuals by chance, introducing natural variation
Repeating a survey should always give an identical result
Random sampling eliminates the possibility of any variation
This would indicate an error in the survey process
15. Why is it mathematically true that increasing sample size reduces (but never completely eliminates) variation between samples?
A larger sample averages out more individual random fluctuations, though some chance variation always remains possible
Sample size has absolutely no relationship to variation
A large enough sample size eliminates all variation completely
Variation actually increases as sample size grows
16. A news article reports "a survey of 15 people found 80% support a policy." Why should a careful reader be skeptical of generalising this to the whole population?
Such a small sample size could easily show 80% by chance, even if true population support is quite different
A sample of 15 people always accurately represents millions of people
Skepticism is never warranted for any survey result
Small samples are always more reliable than large ones
17. Why might election polls from different companies, using different random samples, report slightly different results even when using proper random sampling methods?
Natural sample variation means each poll's random sample won't be identical, even if all methods are valid
Different results always indicate that one company made an error
Proper random sampling should always produce identical results
Polling companies never use random sampling methods
18. Why might doubling a sample size not simply double the reliability of a result?
The relationship between sample size and reduced variation follows a mathematical pattern, not a simple linear one
Doubling sample size always exactly doubles reliability in a simple, linear way
Sample size has no mathematical relationship to reliability at all
Reliability decreases as sample size increases
19. A quality control team samples 50 items from a factory batch of 10,000 to check for defects. Why is understanding sample variation important when interpreting their result?
The defect rate found in the sample is an estimate, and the true batch rate could reasonably differ somewhat by chance
The sample result will always exactly match the entire batch
Sample variation is irrelevant to quality control decisions
A sample of 50 is always too small to be useful for any purpose
20. A clinical drug trial with only 8 participants shows a positive result. Why would researchers be cautious about generalising this to the wider population?
Such a small sample could show a positive result by chance, even if the drug has little real effect on most people
A sample of 8 people is always sufficient to confirm a drug works for everyone
Small clinical trials never involve any element of chance
Sample size has no bearing on how confidently results can be generalised
21. Understanding sample size and variation mainly helps you to:
Judge how much confidence to place in a sample's result as an estimate of the whole population
Assume every sample perfectly represents its population regardless of size
Avoid ever using samples to estimate anything
Treat sample size as irrelevant to the reliability of a result
Answer key (parent copy)
1. Natural differences in results between different random samples
2. Closer to the true population value, with less variation
3. 10 people
4. Small samples have more random variation
5. Selected by chance, giving everyone a fair opportunity to be included
6. Random chance naturally creates some variation between different samples
7. The number of individuals or data points in a sample
8. Larger sample sizes tend to smooth out random variation
9. A small sample size can vary considerably from the true population value just by chance
10. A larger sample reduces random variation, giving a more reliable estimate
11. The sample of size 10
12. Random selection still involves chance, which naturally creates some variation from the true population value
13. The result from 50 students
14. Each sample includes different individuals by chance, introducing natural variation
15. A larger sample averages out more individual random fluctuations, though some chance variation always remains possible
16. Such a small sample size could easily show 80% by chance, even if true population support is quite different
17. Natural sample variation means each poll's random sample won't be identical, even if all methods are valid
18. The relationship between sample size and reduced variation follows a mathematical pattern, not a simple linear one
19. The defect rate found in the sample is an estimate, and the true batch rate could reasonably differ somewhat by chance
20. Such a small sample could show a positive result by chance, even if the drug has little real effect on most people
21. Judge how much confidence to place in a sample's result as an estimate of the whole population