Pilot · L09 · 60 minutes
From data to a justified claim
Use metadata to interpret a fictional journey dataset, apply justified duplicate and missing-value rules, calculate means from cleaned rows, and write a qualified claim that separates association from causation and states the sample’s limits.
Topics: 2.3 Extracting Information from Data · 2.4 Using Programs with Data
Goals and vocabulary
By the end you can:
- Distinguish data, information and metadata, and use metadata to interpret values.
- Apply and justify rules for duplicate rows and missing values.
- Calculate a mean and a filtered mean from cleaned rows.
- Write a claim that states the rules used and the limits of the sample.
- Explain why a pattern in data does not by itself show a cause.
- Data
- Recorded values, such as numbers or words, before they are interpreted.
- Information
- Meaning drawn from data for a purpose.
- Metadata
- Data that describes other data: meaning, units, source and collection method.
- Cleaning
- Finding and handling errors, duplicates and missing values before analysis.
- Missing value
- A field with no recorded value. It is not the same as zero.
- Association
- Two things tend to occur together in the data.
- Causation
- One thing actually produces the other. Data patterns alone rarely establish it.
Data, information and metadata
Data are recorded values, such as 20 or “bus”. Information is what those values tell us once we know what they describe and process them for a purpose, such as “the typical bus journey in this sample took 30 minutes”.
Metadata is data about the data: what each column means, its units, who collected it, when, and how. Without metadata, 20 could be minutes, kilometres or a price.
The journey records
Worked example
| Row | Trip ID | Minutes | Mode |
|---|---|---|---|
| 1 | A | 10 | walk |
| 2 | B | 20 | bus |
| 3 | C | 20 | bus |
| 4 | C | 20 | bus |
| 5 | D | (missing) | walk |
| 6 | E | 50 | bus |
| Field | Meaning |
|---|---|
| Trip ID | One identifier per journey. Each trip should appear once. |
| Minutes | Door-to-door journey time in whole minutes, self-reported. |
| Mode | Main way of travelling: walk or bus. |
| Collected | One fictional morning, from volunteers in one class. |
Cleaning: duplicates and missing values
Worked example
Row 4 repeats trip C, so it was entered twice. Keeping both would count one journey twice. The metadata says each trip appears once, so we keep one row per trip ID.
Row 5 has no minutes. Treating missing as 0 would claim trip D took no time at all, which is false and drags the average down. The justified rule is to exclude rows with missing minutes from time calculations and report how many were excluded.
| Row | Decision | Reason |
|---|---|---|
| 1, 2, 3, 6 | Keep | Valid, unique trips |
| 4 | Remove | Duplicate of trip C |
| 5 | Exclude from time calculations | Minutes missing — not the same as 0 |
Calculating from the cleaned rows
Worked example
| Calculation | Raw (missing as 0, duplicate kept) | Cleaned |
|---|---|---|
| Rows used | 6 | 4 |
| Mean minutes | (10 + 20 + 20 + 20 + 0 + 50) ÷ 6 ≈ 20 | (10 + 20 + 20 + 50) ÷ 4 = 25 |
| Bus mean minutes | (20 + 20 + 20 + 50) ÷ 4 = 27.5 | (20 + 20 + 50) ÷ 3 = 30 |
The cleaning rules changed the answers. That is why a report should say which rules were used, not just the final number.
From a pattern to a justified claim
In the cleaned sample, bus journeys took longer on average than walking ones. That is an association: two things appear together. It does not show that taking the bus causes longer journeys. Students who live further away may be the ones who take the bus.
A justified claim states what the data show, which rows were used, and what the data cannot tell us.
In this fictional sample of 4 valid trips (1 duplicate removed,1 with missing minutes excluded), bus trips averaged 30 minutesand the one walking trip took 10 minutes. The sample is small,from one class on one morning, and does not show that the buscauses longer journeys.Explore each concept
Choose a concept, change its inputs and follow the explanation. Each example models the stated rule or scenario; completing it is practice rather than evidence of mastery.
- Metadata versus primary data
- Descriptive claims versus causes
- Combining records and re-identification
Change it · step through · explain
Change an input, predict what will happen, then advance one step. Read why the state changes, reset and experiment. This fixed teaching model does not save progress or award XP.
Ready · Make a prediction
- QuestionmetadataWaiting
- Time unit metadataminutesWaiting
Ready to explore: Metadata, association and linked data
Why: Change an input, predict the first effect, then advance one step.
Try this: Change one input or choose another listed case. Predict the effect, reset and compare the states and governing rule.
Step through and experiment
Change it · step through · explain
Change an input, predict what will happen, then advance one step. Read why the state changes, reset and experiment. This fixed teaching model does not save progress or award XP.
Ready · Make a prediction
- Input 1deduplicateWaiting
- Input 2excludeWaiting
Ready to experiment
Why: Predict the first change, then advance one step.
Try this: Change only one cleaning policy. Which numerator or denominator changes?
Common misconceptions
Missing means zero
The belief: An empty minutes field can be counted as 0 minutes.
What is actually true: Missing means unknown. Counting it as 0 invents a value and biases the mean down.
Check yourself: Did trip D really take no time?
Keep every row
The belief: Removing rows always throws away good data.
What is actually true: A duplicate row counts one event twice. Removing it makes the data match what happened.
Check yourself: How many real trips does the data describe?
A pattern proves a cause
The belief: Bus trips took longer, so the bus makes trips longer.
What is actually true: This is an association. Distance from school could explain both the mode and the time.
Check yourself: What else could explain the pattern?
Guided practice
Guided: clean and calculate
Use the six fictional journey rows and their metadata.
- Identify each row that should be removed or excluded, and give a reason linked to the metadata.
- Calculate the mean minutes for the cleaned rows, showing the sum and the count.
- Calculate the bus mean for the cleaned rows.
- Recalculate the mean treating the missing value as 0 and the duplicate as kept. Explain why the answer differs.
Independent practice
Independent: a second survey
A fictional second class records: F/15/walk, G/30/bus, G/30/bus, H/(missing)/bus, J/25/walk.
F 15 walkG 30 busG 30 busH (missing) busJ 25 walk- State your cleaning rules and list the rows you keep.
- Calculate the mean minutes and the walking mean from the cleaned rows.
- Write one qualified claim that states the rows used and one limitation.
- A classmate concludes “walking is faster because walking speeds you up.” Explain what is wrong with this conclusion.
Exit check
- Give one reason a missing value should not be replaced by 0 when calculating an average.
- Rewrite “The bus makes journeys slower” as a claim the fictional data can actually support.
Check yourself
Practice only: answers are checked in your browser and nothing is saved or sent to your teacher. Each explanation links back to the section it practises.
Lesson practice
Your practice so far
Check an answer to see its rationale.
Apply it somewhere new
Your practice so far
Check an answer to see its rationale.