Foundations of Inquiry · Lesson 018 · November 7, 2026
What you will learn
Distinguish random assignment from random sampling; identify confounders; explain why repeated trials and uncertainty matter; and evaluate an experiment without overclaiming causation.
The question
Lesson 017 compared conditions while controlling variables. Lesson 018 asks: when two groups differ, how can we tell whether the intervention caused the difference? A randomized experiment assigns eligible participants to conditions by chance. Random assignment can reduce systematic differences between groups on average, but does not guarantee identical groups in a small study. Random sampling is different: it selects people from a population and affects how far results can generalize.
Worked example
A fictional class tests whether a five-minute retrieval quiz improves next-day recall. Twenty students are randomly assigned: ten receive the quiz and ten spend five minutes rereading. Each group then takes the same 20-question recall test. The quiz group earns 14, 15, 16, 13, 17, 15, 14, 16, 15, 15 points (sum 150, mean 15.0). The rereading group earns 12, 13, 14, 11, 15, 13, 12, 14, 13, 13 points (sum 130, mean 13.0). The observed difference is 2.0 points, or 10 percentage points of the 20-point test. It is not a 10% relative increase: 2/13 ≈ 15.4% relative to the rereading mean. This small illustrative dataset alone cannot establish a reliable effect, because random variation and other limitations remain.
Try it yourself
Write 20 identical slips numbered 1–20. Shuffle and draw ten for Group A; assign the rest to Group B. Plan a safe, simple comparison such as two different study techniques using the same short word list, same study duration and same testing conditions. Record the allocation method before seeing scores. Do not collect identifying information. Repeat the exercise with a fresh allocation or a second list and compare the differences.
Limitations and ethics
Small samples produce noisy estimates. Participants may know which method they use; expectations and prior knowledge can affect results. A single class is not a representative sample of all learners. Avoid withholding necessary educational support. Report both positive and negative outcomes and distinguish the observed result from a general causal conclusion.
Knowledge check
1. Does random assignment guarantee two identical groups? 2. Which matters most for generalizing to a population: random sampling or random assignment? 3. What is the difference between means 15 and 13 on a 20-point test? 4. Why might a one-day experiment fail to establish a long-term learning effect?
Answer key
1. No; it reduces systematic allocation bias on average, but chance imbalances remain. 2. Random sampling, alongside other considerations about representativeness. 3. Two points, or ten percentage points of the maximum possible score. 4. Short duration, small sample, measurement variability, and no long-term follow-up.
Sources and further reading
National Academies of Sciences, Engineering, and Medicine, Reproducibility and Replicability in Science (2019); NIH/NLM, Randomized Controlled Trials (StatPearls, overview); CONSORT reporting guidance for randomized trials.
Continue learning
Review Lesson 017: Controlled Comparisons. Explore the Nova Academy index.
