A change is not yet an explanation

Suppose a class answers 60% of questions correctly on one assessment and 75% on a later assessment. That is a 15-percentage-point increase in the reported scores. But how similar were the two assessments? Who participated each time? How was the score calculated? What else changed between assessments?

In Lesson 014, we examined who appears in the evidence. In Lesson 015, we asked what our instruments really measure. Today, we introduce another safeguard: compared with what?

1. A baseline is a starting reference

A baseline is an initial or reference measurement used to evaluate later observations of the same variable. It is most useful when we record the definition, units, population, measurement method, date or period, and relevant context.

A baseline is not automatically a timeless standard of truth. A single unusual day may make a poor reference for an activity that varies through the week. The right period and method depend on what we are trying to understand.

The CDC Program Evaluation Framework (2024) distinguishes evaluation designs and emphasizes choosing appropriate measures and comparisons when interpreting program outcomes.

2. A benchmark is a standard or target—not your history

A benchmark can be a defined target, externally established standard, or comparison reference, depending on context. In today’s exercise we will use it as a declared target. Our current baseline might be 60%, while a teacher-chosen benchmark might be 80%. They answer different questions.

A goal can be useful even when it is chosen locally. But an invented 80% classroom target is not a universal scientific standard, and reaching it would not by itself prove deep understanding or causation.

3. Percentage points are not the same as percent change

If a reported score rises from 60% to 75%, the absolute difference is 15 percentage points. The relative percent increase is (75 − 60) ÷ 60 × 100 = 25%. These describe the same change in different units and should not be mixed.

The change from one score to another says nothing on its own about whether the tasks were equally difficult, the participants were comparable, or learning caused the difference.

4. A baseline may need more than one day

Consider comparing this morning’s outdoor temperature with the temperature at the same hour yesterday. That is a valid comparison of two measurements, but it is not automatically the best reference for what is typical at this location and time of year. Different reference periods answer different questions.

For example, NOAA explains Climate Normals as reference values based on a 30-year period for variables such as temperature and precipitation. The current U.S. standard normals period is 1991–2020; the calculations address issues such as missing observations and changes in observing stations. They provide context for current conditions; they do not guarantee what a particular day will be like.

5. Mini-lab · The class that improved

15–20 MINUTES · PAPER ONLY · INVENTED CLASSROOM DATA

Two fictional learning groups are assessed before and after a four-week teaching period. Scores are percentages correct on comparable fictional tasks. Assume, for the arithmetic exercise, that the measurement and scoring rules remain the same. We have not established that the groups were randomly assigned or otherwise equivalent.

Measure	Group A	Group B
Baseline assessment	60%	62%
Later assessment	75%	68%
Declared target benchmark	80%	80%

Try the calculations

1. Calculate how many percentage points Group A changed between the first and second assessments.
2. Do the same for Group B.
3. Subtract B’s change from A’s change. What is the difference between the two changes?
4. Calculate how many points below the 80% goal each group remains at the end.
5. Does the comparison prove that a particular teaching method caused Group A to improve? Explain why not.
6. Write a two-sentence conclusion that states what was measured without overclaiming.

6. Answer key · A difference in changes

This calculation resembles the arithmetic used in some before-and-after comparative designs, but causal interpretation needs further assumptions and evidence. Groups might begin with different characteristics, experience different outside events, or follow different trends even without the program.

7. A comparison group is not automatically a perfect control

A comparison group provides context for how an outcome changes outside the specific intervention being studied. In a well-designed randomized experiment, a control group can help estimate what would have happened otherwise. In observational or quasi-experimental designs, researchers must work harder to address differences between groups.

The CDC Program Evaluation Framework describes experimental, quasi-experimental and observational approaches. Different designs answer different questions and support different levels of causal confidence.

When a design relies on trends being comparable in the absence of an intervention, that assumption should be examined rather than assumed automatically. A difference in changes is a descriptive calculation before it becomes a credible causal estimate.

8. What is a meaningful benchmark in education?

A goal is useful only if it tells us something relevant to the purpose of the activity. A website may set a target of 1,000 visits. Reaching that number would be evidence about visits; it is not automatically evidence that 1,000 people understood a concept.

For a free learning curriculum, a stronger evaluation might include a clear rubric, an unfamiliar practice problem, accessibility feedback, and the opportunity to revise an explanation. Different measures address different aspects of educational quality.

This returns us to Lessons 014 and 015: a carefully chosen benchmark is still unhelpful if the sample excludes key learners or the measure fails to represent understanding.

9. Knowledge check · Six questions

1. What is a baseline, and why does its time period matter?
2. What distinguishes a baseline from a benchmark?
3. How many percentage points does a score rise when it moves from 60% to 75%?
4. What is the difference between Group A’s and Group B’s observed changes?
5. Does a comparison group automatically show what would have happened without the program?
6. Can a project meet a target while still failing to measure its true educational purpose?

---

Reflection · Compared to what?

A result does not stand alone. It is compared to a past value, a target, an alternative, or a carefully defined expectation. Choose the wrong comparison and a perfectly correct number can become a misleading story.

Nova Academy’s educational continuity now connects three safeguards: who was included (Lesson 014), what was measured (Lesson 015), and what that measurement is compared against (Lesson 016). Each improves the questions we can responsibly ask about progress.

Takeaway: Define the measure. Record the baseline. Specify the benchmark. Calculate the change. Choose meaningful comparisons. State uncertainty before attributing cause.

Completion: You have completed Lesson 016 when you can calculate both changes and the nine-point difference, distinguish the 80% goal from the initial scores, and write a qualified conclusion. No tuition, enrollment, grades, or certificate required.

Sources & further learning

• CDC Program Evaluation Framework (2024): evaluation questions, measurement, design and causal interpretation.
• NOAA — Understanding Climate Normals: a real-world example of a defined long-term comparison baseline.
• NOAA National Centers for Environmental Information — U.S. Climate Normals: formal reference periods and methods.
• NIST — Measurement Uncertainty: continuation of the previous lesson’s measurement principles.

← Review Lesson 015 · Browse Nova Academy Lessons · Today’s Electric Icarus Flyer · Explore the Quantum Atrium

Nova Academy is a free, non-accredited educational initiative of the Electric Icarus Project. All class scores and benchmark values are invented teaching data, not observations from real students or evidence that a particular program was effective.