A number is not the same as the thing it represents

Imagine a school celebrates because its students spent 20% more time using a learning website. Did they learn 20% more? Maybe. But time spent on a page is not itself a measure of understanding. The number is real; the conclusion may be unsupported.

In Lesson 014 we asked whether missing people or cases distorted our evidence. Today we look at a different weakness: even the people and cases we count can be measured poorly. We need to know precisely what we are measuring, how it is measured, and which claims that measurement can justify.

1. Start by naming the measurand

In measurement science, a measurand is the quantity intended to be measured. If you want the length of a notebook, “length” is not yet perfectly specific. Do you mean its closed cover from left to right, the page width, or its diagonal? Under which conditions, and in which unit?

The National Institute of Standards and Technology (NIST) describes measurement as an experimental process that produces a value attributable to a quantitative property. The number must have a meaningful reference and a well-defined target.

This method is only an example. A different question might call for the usable writing area, thickness of the book, or another dimension entirely. Better measurements begin with better definitions.

2. Consistency is not the same as correctness

Precision concerns how closely repeated measurements agree. Accuracy concerns closeness to a suitable reference or true value, as far as that value is known. These are related but different features of a measurement process.

A tool that gives nearly the same answer every time may be very repeatable and still consistently too high. Another tool might fluctuate widely around the reference. Looking only at the first few decimals in a reading cannot tell you which tool is trustworthy.

NIST distinguishes bias and accuracy from measurement variability. That distinction matters whenever a number is presented as a measurement.

3. Mini-lab · Two instruments, one reference

15–20 MINUTES · PAPER ONLY · ALL READINGS INVENTED

Imagine a reference object whose length has been independently established as 10.0 cm, with uncertainty small relative to the differences in this classroom example. Two fictional instruments measure it five times each. We are not claiming that any physical rulers were actually tested.

Trial	Instrument A	Instrument B
1	10.0 cm	10.5 cm
2	10.1 cm	10.6 cm
3	9.9 cm	10.4 cm
4	10.0 cm	10.5 cm
5	10.0 cm	10.5 cm

1. Add the five measurements from A and divide by five to get A’s mean.
2. Repeat for B.
3. For each instrument, subtract its lowest reading from its highest reading. This is its observed range.
4. Subtract the reference 10.0 cm from each instrument’s mean.
5. Which instrument is closer to the reference in this example?
6. Which instrument has the smaller range? If the ranges are equal, what does that tell you?
7. Write one sentence describing why repeatability does not, by itself, guarantee accuracy.

Calculate before opening the answer guide. Try it with a calculator or by hand.

4. Worked answer · The consistent error

The range is a simple description of spread, not a complete estimate of measurement uncertainty. Five fictional readings cannot establish every aspect of a real instrument’s performance. Nevertheless, the example shows how systematic offset and variability can be considered separately.

For an optional extension, compute B’s relative mean error: (0.5 cm ÷ 10.0 cm) × 100% = 5%. The units cancel in this ratio. This does not turn a five-reading classroom illustration into an instrument certification.

5. Calibration, resolution, and uncertainty

A measuring instrument may have resolution—the smallest change it displays or can meaningfully distinguish—but fine resolution does not guarantee accurate readings. A display that reports 10.500 cm can still be wrong if the instrument is offset or the procedure is flawed.

Calibration establishes a relationship between instrument indications and suitable reference values under specified conditions. Measurement uncertainty describes the range of values reasonably attributable to what is being measured, using an appropriate evaluation method. Neither is identical to merely writing more decimal places.

NIST’s Technical Note 1297 terminology distinguishes repeatability, reproducibility, precision, and related measurement concepts. These words describe different properties of the process.

6. Reliability and validity: are we measuring the right thing?

So far our measurement target was straightforward: a length with a known reference. Some things we want to investigate—learning, satisfaction, creativity, or trust—are more complicated. They may require several observable indicators and an argument for why those indicators support the intended interpretation.

For example, a website can accurately and consistently count lesson-page visits. That makes visits a useful engagement measure. But it does not make them a direct, valid measure of student understanding. A visitor might open a lesson without reading it or reread it several times while learning something valuable.

The National Academies’ Measuring What Counts explains that assessment validity concerns the support for interpretations and actions based on scores, not simply the existence of a neat number. See also the U.S. Office of Personnel Management’s overview of assessment reliability.

7. Design a better learning measure

Imagine that a free educational program wants to know whether learners understand the difference between selection bias and measurement error. Counting page views could tell the program how often people opened an article. To study understanding, we would need a more directly relevant activity: perhaps a new scenario where learners must identify which problem is present and explain their reasoning.

This is the difference between an indicator being easy to count and evidence being appropriate for a claim. Useful educational evaluation may include several measures, qualitative feedback, and care about who participates.

8. Optional home activity · One object, two methods

5–10 MINUTES · OPTIONAL · NO EQUIPMENT PURCHASES

1. Choose a safe, ordinary object such as a notebook.
2. Write exactly what dimension you intend to measure.
3. Measure it several times using a ruler if available. Do not invent extra decimals beyond the ruler’s sensible resolution.
4. If another suitable measuring method is available, note any differences without assuming either one is automatically correct.
5. Record units, setup conditions, and one plausible source of uncertainty.
6. Finish: “My readings support a claim about _____. They do not establish _____.”

If you have no ruler, complete the instrument comparison above instead. The aim is disciplined observation, not acquiring equipment.

9. Knowledge check · Six questions

1. What is a measurand?
2. Do close-together readings guarantee accuracy?
3. What is the mean of Instrument B’s five invented readings?
4. Do Instruments A and B have different ranges in this example?
5. Why does adding decimal places not establish better accuracy?
6. Why would a precise count of lesson-page visits not automatically measure actual understanding?

---

Reflection · Measure what matters, report what you measured

A trustworthy inquiry is willing to ask two uncomfortable questions: who was never included, and what did we really record about the people or things we did include? Selection and measurement are different vulnerabilities; both deserve attention.

In a learning ecosystem, it is tempting to celebrate whatever number grows most quickly. Growth may be meaningful. But understanding, access, and effective education cannot be inferred automatically from clicks, page counts, or impressive-looking decimals.

Takeaway: Define the target. Choose a justified method. Check the reference. Repeat carefully. Describe the variation. Test whether the measure supports the conclusion.

Completion: You have completed Lesson 015 when you can calculate both means and ranges, explain the systematic offset, and identify one example of a reliable indicator that does not necessarily measure the intended concept. No tuition, grades, enrollment, or certificates are required.

Sources & further learning

• NIST — Measurement Uncertainty: what measurement means and why references and uncertainty matter.
• NIST Engineering Statistics Handbook — Bias and Accuracy: systematic offset and accuracy.
• NIST Engineering Statistics Handbook — Variability: repeatability and spread.
• NIST Technical Note 1297 — Measurement Terminology: repeatability, reproducibility and precision.
• National Academies — Measuring What Counts: validity of interpretations based on educational assessments.

← Review Lesson 014 · Browse the Foundations of Inquiry · Visit the Quantum Atrium

Nova Academy is a free, non-accredited educational initiative of the Electric Icarus Project. All instrument readings and their reference in the classroom experiment are invented for teaching purposes; they are not laboratory measurements or real-world instrument certifications.