Nova Academy · Foundations of Inquiry · Lesson 031 · November 20, 2026
Reproducibility and Replication: Can Another Researcher Get the Same Result?
A result becomes more trustworthy when its methods are clear enough to check, rerun, and challenge. Reproducibility and replication are related, but not identical: terminology varies by field, so define which sense you mean.
Learning objectives
- Distinguish computational reproducibility from independent replication.
- Identify the materials, methods, code, and decisions needed for another researcher to evaluate a result.
- Calculate and compare results from repeated measurements.
- Recognize why a failed replication calls for investigation rather than an automatic verdict of fraud.
Key ideas
Reproducibility: In the computational sense used by the US National Academies, obtaining consistent computational results using the same input data, computational steps, methods, code, and conditions. Replicability: obtaining consistent results across studies designed to answer the same scientific question, each collecting its own data. Other disciplines sometimes reverse these labels. Specify definitions when reporting research.
A well-documented experiment records the research question, instrument calibration, sample selection, exact procedure, raw observations, exclusions, analysis code or formulas, software versions, uncertainties, and limitations. Independent teams can then investigate whether the result survives a new sample and context.
Worked example: repeated measurements
Team A measures the same reference length three times: 10.1, 10.2, and 10.0 cm. Its mean is (10.1 + 10.2 + 10.0)/3 = 10.1 cm. Team B independently measures 10.0, 10.1, and 10.2 cm. Its mean is also 10.1 cm. Each team’s sample standard deviation is sqrt(((0)^2+(0.1)^2+(-0.1)^2)/(3−1)) = 0.1 cm.
The matching means are encouraging, but three readings per team are too few to establish strong generalizability. Both teams could share a calibration error. Agreement is not proof of accuracy, and neither the sample standard deviation nor the range should be casually equated with a full measurement-uncertainty statement.
Activity: independent paper-airplane trials
- Write a fixed paper-airplane design and launch procedure, including launch height and how distance will be measured.
- Have two people independently make five launches each. Record every distance in centimeters without deleting inconvenient trials.
- Calculate each team’s mean and range (maximum minus minimum). Compare the two means and list possible sources of variation.
- Swap measurement tools or operators, repeat, and record what changed. Do not claim that five trials prove universal performance.
Reflection
Which instructions were ambiguous? Could two teams follow your written method and still differ? What data or contextual details would help explain the difference? How would you report a result that did not replicate?
Knowledge check
- 1. Can identical output from the same dataset and code establish independent replication?
- 2. Team A records 4, 5, 6 units. What is its mean and range?
- 3. Name two possible reasons why independent teams disagree.
- 4. Why is sharing raw data and analysis code valuable?
Answer key
- 1. No. That supports computational reproducibility; independent replication generally uses newly collected data.
- 2. Mean = (4+5+6)/3 = 5 units; range = 6−4 = 2 units.
- 3. Examples include sampling variation, calibration bias, differing conditions, protocol deviations, or analytical choices.
- 4. It allows others to inspect decisions, detect errors, and rerun calculations, subject to privacy and ethical restrictions.
Limitations and ethics
Replication outcomes depend on sample size, measurement quality, statistical power, context, and definitions of consistency. A single unsuccessful attempt does not necessarily invalidate the original study; a successful repeat does not establish universal truth. Respect participant consent, privacy, intellectual property, and safety when sharing materials.
Sources and further reading
National Academies of Sciences, Engineering, and Medicine (2019), Reproducibility and Replicability in Science; NIST Technical Note 1297, Guidelines for Evaluating and Expressing Measurement Uncertainty; NIST/SEMATECH e-Handbook of Statistical Methods.
Previous: Lesson 030 — Testing Models on New Data · Nova Academy index
