C63 · Topic 5.3 · 40 minutes · IOC-1.D
Bias in data and design
By the end you can:
- Locate possible bias mechanisms.
- Explain representation limits in training data.
- Distinguish a group difference from proof of its cause.
Before this lesson: topics 2.3, 5.1.
- Computing bias
- Systematic differences arising through data, design or use of a computing system.
- Error rate
- Errors divided by the relevant number of evaluated cases.
Understand
Bias can enter through problem definition, outcome labels, sampling, design rules and how a system is deployed, not only through an explicitly prejudiced line of code. A dataset can be large yet underrepresent conditions important to the intended use. Examine the mechanism rather than assuming more of the same data fixes the problem.
An observed performance gap is a reason to investigate, not by itself a complete causal explanation or legal determination. Check collection, measurement and deployment context. Evaluation should avoid exposing unnecessary personal data or inferring sensitive characteristics without a justified purpose and safeguards.
Worked example
A fictional recommendation system treats rapid online completion as the label for success.
- Students with shared devices may take longer for access reasons rather than interest or ability.
- The chosen label can underrepresent their successful participation.
- Adding more records from the same process can reproduce that mechanism rather than remove it.
Explore each concept
Choose a concept, change its inputs and follow the explanation. Each example models the stated rule or scenario; completing it is practice rather than evidence of mastery.
- Sampling, labels, rules and deployment
- More data versus better representation
- Evaluate group outcomes after change
Change it · step through · explain
Change an input, predict what will happen, then advance one step. Read why the state changes, reset and experiment. This fixed teaching model does not save progress or award XP.
Ready · Make a prediction
- Bias mechanismsamplingWaiting
- Proposed responsemore-of-same-dataWaiting
Ready to explore: Locate bias before choosing a mitigation
Why: Change an input, predict the first effect, then advance one step.
Try this: Change one input or choose another listed case. Predict the effect, reset and compare the states and governing rule.
Step through and experiment
Change it · step through · explain
Change an input, predict what will happen, then advance one step. Read why the state changes, reset and experiment. This fixed teaching model does not save progress or award XP.
Ready · Make a prediction
- Input 120,80Waiting
- Input 28,4Waiting
Ready to experiment
Why: Predict the first change, then advance one step.
Try this: Change group sizes while keeping very different error rates. Does a high overall accuracy hide a gap?
Check a misconception
Claim: Only training examples can introduce bias.
Human choices about goals, labels, rules and deployment can also introduce it.
Guided practice
Identify a possible design source of bias in the success label.
Work with a partner. Make a prediction, compare your reasoning, then identify the rule that supports your answer.
Independent transfer
A daytime-only image dataset is used at night. Explain the representation issue.
Exit check
Does a group accuracy gap prove exactly which developer choice caused it?
Write your reasoning before comparing with a classmate. If your teacher assigned this task, submit it through My classes to save your work and receive feedback.
Check yourself
Your practice so far
Check an answer to see its rationale.