Scenario Analysis Activity: The School Board's Big Claim

Objective: Your group will critically examine a scenario where data is used to support a claim. You will discuss how the data presented might be interpreted in different ways, potentially leading to misleading conclusions or informing responsible decisions.

Materials:

• This activity sheet
• Your group's critical thinking skills!

Scenario:
The "Academically Advanced Tutoring Program" (AATP) was implemented in the Happyville School District last year. This program, costing $500,000, aims to boost student performance.

At the latest school board meeting, the Superintendent presented the following "data" to show the program's success:

"Since implementing AATP, the average standardized test scores of participating students have increased by a significant 15%! This clearly demonstrates the program's effectiveness and justifies its cost. We recommend expanding AATP to all schools next year."

Discussion Questions:

1. What claim is the Superintendent making based on this data?
   • Think about: What is the main message they want to convey?
     
     
     
     
2. What important questions does this "data" presentation not answer?
   • Think about: What information is missing that would help you form a more complete picture? Consider the students involved, the timing, and other factors.
     
     
     
     
     
     
     
3. How could this presentation of the data be potentially misleading?
   • Think about: Are there alternative explanations for the 15% increase? What assumptions are being made?
     
     
     
     
     
     
     
4. What additional data or context would you need to make a truly informed decision about the AATP program?
   • Think about: What specific questions would you ask the Superintendent or the school board? How would you want the data to be presented to be more trustworthy?
     
     
     
     
     
     
     
     
     
     
     
     
5. Based on your discussion, how does the way data is presented impact decision-making, even without complex charts?
   
   
   
   

Prepare for Presentation: Be ready to share your group's answers and insights with the class.

Are the Numbers Lying? Teacher Script

1. Warm-up: Where do you see statistics? (5 minutes)

Teacher: "Good morning/afternoon everyone! Today, we're going to become data detectives, but with a new lens. To start, I want you to think for a moment: Where in your daily life do you see claims backed by statistics or data? Think about news headlines, social media posts, advertisements, or even conversations where someone uses a number to convince you of something. Don't be shy, just shout out your ideas!"

(Allow students to share responses. Write key ideas on the board, like 'product success rates,' 'political polls,' 'health claims,' 'social trends reported online,' etc.)

Teacher: "Excellent examples! As you can see, statistics and data are used everywhere to support all sorts of claims. But how those claims are told can change everything. That brings us to our big question for today: Are the numbers lying?"

2. Direct Instruction: How data narratives can mislead (15 minutes)

Teacher: "Let's dive deeper into how data can be presented, not just in graphs, but through the story we tell with it. I'm going to show you some examples using our Data Narrative Slides. Pay close attention, because what you hear or read might not always be the full story."

(Display Data Narrative Slides, moving through each slide and using the teacher notes as guidance.)

Teacher (Slide: The Power of the Narrative): "As this slide says, the same underlying facts can tell many stories. The way data is described influences our understanding. Our goal is to be aware of how claims can be misleading."

Teacher (Slide: When Data Narratives Get Tricky): "Data can be described to support very different, even opposing, conclusions. What makes a data-backed claim seem trustworthy or untrustworthy to you? And how can omitting a small piece of information drastically change the story data tells?"

(Facilitate a brief discussion, encouraging students to share their initial thoughts.)

Teacher (Slide: Correlation vs. Causation): "This is still a big one! Just because two things happen at the same time, or seem to be related in a reported statement, it doesn't mean one causes the other! For example, if a headline says, 'Students who eat breakfast every day have higher test scores,' does that mean eating breakfast causes higher scores? Or could there be other factors at play? Always remember: correlation is not causation."

Teacher (Slide: Selective Data & Context): "Sometimes, the trick isn't what data is shown, but what isn't shown. We'll look at how data can be presented through 'selective reporting' – only sharing data that supports a claim – or by 'omitting context' – leaving out crucial background information. These can really change what we think the data is telling us."

Teacher (Slide: Different Stories, Same Situation!): "Now, let's consider some scenarios. Think about: How do different descriptions of data make you feel? What choices did the speaker or writer make with things like the information they included, or the context they provided (or didn't provide), and how do those choices change the message?"

(Guide students through discussing hypothetical examples of how a situation involving data could be framed differently to lead to different conclusions.)

3. Collaborative Work: Unpacking data scenarios (15 minutes)

Teacher (Slide: Your Turn: Scenario Detectives): "Alright, now it's your turn to be the data scenario detectives! I'm going to divide you into small groups."

(Divide students into groups of 3-4.)

Teacher: "Each group will receive the Scenario Analysis Activity instructions. Your task is to carefully read the provided scenario, which describes a claim made using data. Then, work together to answer the discussion questions. You won't be doing any math problems; instead, you'll be critically analyzing the claim, identifying missing information, and discussing alternative interpretations."

Teacher: "Think carefully about your questions: What information is the superintendent not telling us? How could this claim be misleading? What else would you need to know to make a truly informed decision? I'll be circulating to help if you have any questions and to encourage your discussions."

(Circulate among groups, offering guidance, prompting critical thinking about the scenario and their discussion points, and ensuring they are engaging with the activity effectively.)

4. Wrap-up: Group scenario insights (7 minutes)

Teacher: "Time is almost up for your group discussions! Let's bring it back together. Each group will briefly share their key insights and answers to the discussion questions from the Scenario Analysis Activity with the class."

(Call on each group to present. After each presentation, facilitate a brief discussion, inviting other students to ask questions or share their observations on how the data narrative could be misleading or more informative.)

Teacher: "Great job, everyone! It's clear you're starting to see how much power the person presenting data has, even when just telling a story with it."

5. Exit Ticket: Data presentation and conclusions (3 minutes)

Teacher: "To wrap up today's lesson, I have one final question for you. On a piece of paper or digitally, please respond to this prompt: 'How can a seemingly simple statement using data lead to a wrong conclusion if important context is missing? Provide at least one specific example from today's lesson or your own experience.'"

(Allow students 2-3 minutes to write their responses.)

Teacher: "Please hand in your exit tickets as you leave. Thank you all for being such keen data detectives today!"