Week 12 · Data Lab

Week 12 — Data Lab · "Anatomy of a Poll: Audit the ±"

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
Objective: Objective 6 — confidence intervals for a proportion; sample size; the margin of error in the media · SLO A (reason quantitatively from data) · SLO B (communicate results plainly)
Worth 50 points · Data labs group = 15% of the grade · Data Lab 12 · Due: end of Week 12
Format: a hands-on data lab — a real published poll (or the provided one), your spreadsheet as an audit machine, and your own recomputation of the fine print; then you catch the AI's mistakes when it checks your work.

This is the course's signature weekly component. Every instructional week has a Data Lab: real, free, linked public data — nothing to buy, download from a store, or sign up for — worked in your spreadsheet, finished with an AI-critique step. Plan on 60–90 minutes.

Part 1 — The Big Picture

This week you learned that every honest percent ships with a ± — and that the ± is not a vibe, it's a computation: z* × √( p̂(1 − p̂) ⁄ n ). Today you become the auditor. You'll take a poll the way the public actually meets one — a headline, a percent, a sample size, a "margin of error" — and recompute its fine print from its own numbers. Does the reported ± survive your audit? And once it does: what does that ± still not protect you from?

This is the week's ideas doing their real job. Pollsters, market researchers, and public-health agencies publish numbers exactly like these every day; almost nobody checks them. You can now check them in six spreadsheet cells.

Part 2 — The Task

The guiding question: When a poll reports a margin of error, does the math check out against its own sample size — and what claims does the resulting interval actually support?

Choose ONE poll to audit:

Option A — a real published poll. Find one recent poll report that gives you at least a topline percent and a sample size n (a reported margin of error is a bonus). All three sources below are free, no-login:
Short Reads — Pew Research Center (recent findings with methodology)
🔗 https://www.pewresearch.org/short-reads/
Gallup News (current public-opinion findings)
🔗 https://news.gallup.com/home.aspx
Projects — AP-NORC Center (survey reports archive)
🔗 https://apnorc.org/projects/
Option B — the provided poll (always works; also the fallback). Audit the synthetic report below. It is invented for this lab — realistic in shape, not a real survey.

The Plaza Poll (synthetic report, for Option B). City Weekly reports: "Support for converting three downtown blocks into a car-free pedestrian plaza stands at 58%, according to a poll of 600 randomly selected city residents. The poll's margin of error is ±4 percentage points at the 95% confidence level."

Part 3 — Materials & Procedure

You need: a web browser and a spreadsheet — Google Sheets (free) or Excel. Nothing else.

Warm-up (5 min, by hand — no spreadsheet). Suppose a tiny poll asks 100 random people a yes/no question and exactly half say yes (p̂ = 0.50). Compute by hand: SE = √(0.50 × 0.50 ⁄ 100) = √0.0025 = 0.05, so the 95% margin is 1.96 × 0.05 = 0.098 — about ±10 points. Write that down and let it sink in: a 100-person poll is roughly a ±10 instrument. (Every real poll you'll audit today bought its smaller ± with a bigger n.)

Watch one audit first (worked model — read, don't copy).

A regional outlet reports: "39% of 1,111 randomly sampled adults support a new stadium bond; margin of error ±3 points."
Audit: SE = √(0.39 × 0.61 ⁄ 1111) = √0.000214 ≈ 0.0146. Implied ME = 1.96 × 0.0146 ≈ 0.0287 → ±2.9 points. Conservative check: 0.98 ⁄ √1111 ≈ 0.0294 → ±2.9. The reported "±3" is these values rounded — the fine print survives the audit. Interval: 39 ± 3 → 36% to 42%: the bond trails badly; no reading of the interval reaches 50%.

Now your audit:

Pick your poll (Option A or B). For Option A, skim the report until you find the topline percent and the sample size — real reports usually state both near a "methodology" note. If a report hides its n, pick a different poll: a poll that won't tell you n is telling you something.
Record the poll's facts in Table 1 (Part 4): the source and claim, the population it speaks for, n, the topline percent, the reported MOE (or "not reported"), and how people were reached.
Build the six-cell auditor. In a blank sheet: B1 = the topline percent as a decimal (e.g., 0.58) · B2 = n (e.g., 600) · B3 =SQRT(B1*(1-B1)/B2) (the SE) · B4 =1.96*B3 (the implied 95% ME) · B5 =B1-B4 and B6 =B1+B4 (the interval).
Add the conservative check. B7 =0.98/SQRT(B2) — the margin the pollster gets using the planning value p* = 0.5. Reported margins usually match this one.
Fill Table 2 and compare: your implied ME, the conservative check, and the reported MOE.
Price a tighter poll. In B8: =0.25*(1.96/0.02)^2 → the n a ±2 margin requires at 95% (conservative). Record it, and compare it to your poll's actual n.

Fallback (always available): if you can't find a real poll that reports its sample size, Option B is the lab — audit the Plaza Poll and say in your submission that you used Option B. No credit difference.

Part 4 — The Scaffold (complete both tables)

Table 1 records the facts your poll reports about itself.

Poll fact	Your entry
Source + the headline claim	______
Population the poll claims to speak for	______
Sample size n	______
Topline percent (this is p̂)	______
Reported margin of error (or "not reported")	______
How people were reached (phone / online panel / etc.)	______

Table 2 records your audit computations from the six-cell auditor.

Quantity	Value
p̂ as a decimal (B1)	______
Standard error (B3)	______
Implied 95% margin of error (B4), in ± points	______
Implied 95% interval (B5 to B6)	______
Conservative check 0.98 ⁄ √n (B7), in ± points	______
Reported MOE minus your implied MOE	______
n required for ±2 at 95%, conservative (B8)	______

Part 5 — Analysis Questions

Answer in 1–3 sentences each:

Interpret your interval in one correct sentence — all three ingredients: the confidence level, the interval, and the population's proportion. (Not "95% of people…", not "95% chance…".)
Your implied margin and the poll's reported margin probably differ a little. Why? (Hint: which planning value does the conservative convention use, and what does rounding to whole points do?)
Name two sources of error the ± does not cover, and say which Week 1 bias concept each one is. Which of the two worries you more for your poll, and why?
The ±2 planner says a tighter poll needs 2,401 respondents. Compare that to your poll's n and explain, using the 1 ⁄ √n law, why "just make the margin ±1" is a budget-breaking sentence.
Does your poll's interval include 50%? State what the sponsor can honestly claim about majority support — and write, in one plain sentence, the sharper question you'd want answered next. (You're previewing next week: testing whether one specific claimed value is plausible.)

Part 6 — AI-Critique Moment (required — the BYOAI step)

Now bring in your chatbot and be the analyst who checks its work.

Paste your completed Table 1 and Table 2 into your chatbot and ask: "Check my standard error, margin of error, and interval against this poll's numbers. Is the reported margin of error consistent with the sample size? And what exactly does a margin of error cover?"
Check everything it says against your own audit:
- Did it recompute the SE correctly — or drop the square root, or use n where √n belongs? (Redo any arithmetic it disputes yourself.)
- Did it quote a margin from memory ("±3 for a thousand people") instead of computing from your poll's n and p̂?
- Did it slip into a forbidden interpretation — "95% chance the true value is inside" — or claim the ± covers nonresponse or question wording? (It doesn't; the ± prices sampling luck only.)
Write 2–3 sentences reporting what the AI got right and at least one thing you had to correct, question, or verify. If it happened to get everything right, say how you verified each claim — that's the skill.

The habit all term: the tool drafts, you judge. A chatbot will confidently mangle a square root or bless a misreading of "95% confident" — catching it is the point.

Part 7 — What to Submit

One document or text entry containing: your warm-up numbers (SE and margin for the 100-person poll), your completed Table 1 and Table 2, your Part 5 answers (1–5), and your Part 6 AI-critique paragraph — plus one line saying whether you audited a real poll (Option A, with its link) or the Plaza Poll (Option B). Due at the end of Week 12 · 50 points (rubric below the key).

Instructor answer key & model values — REMOVE BEFORE PUBLISHING TO STUDENTS

Option A students audit different real polls, so their numbers are graded on method and interpretation, not on matching one answer key. Everything asserted below is pre-computed and re-verified in the week's math script. The Plaza Poll and the stadium-bond model are synthetic — invented for this lab, clearly labeled as such in the student text; neither is presented as a real survey.

Warm-up (fixed — every student should match): SE = √(0.50 × 0.50 ⁄ 100) = √0.0025 = 0.05; ME = 1.96 × 0.05 = 0.098 ≈ ±10 points. ✓

Worked model (stadium bond, printed in Part 3 — students read, don't submit): SE = √(0.39 × 0.61 ⁄ 1111) = 0.01463; implied ME = 0.02868 → ±2.9; conservative 0.98 ⁄ √1111 = 0.02940 → ±2.9; both round to the reported ±3; interval 36% to 42%, entirely below 50%. ✓

Option B key (the Plaza Poll — fixed numbers):

Quantity	Correct value
p̂	0.58
SE = √(0.58 × 0.42 ⁄ 600)	0.0201 (0.020149…)
Implied 95% ME	1.96 × 0.020149 = 0.0395 → ±3.9 points
Implied interval	(0.5405, 0.6195) → 54.1% to 61.9%
Conservative check 0.98 ⁄ √600	0.0400 → ±4.0 points
Reported − implied	+0.1 point (reported ±4 follows the conservative p* = 0.5 convention)
n for ±2 at 95% (conservative)	0.25 × (1.96 ⁄ 0.02)² = 2,401 exactly

Option A grading: check that p̂ was converted to a decimal; that B3/B4 implement the formulas (spot-recompute one student's SE yourself); that the implied ME is compared to the reported one in points; and that the ±2 planner cell shows 2,401 (it's poll-independent). Grade the analysis, not a specific poll's numbers — and never assert what any real poll "should" say beyond what the student's own screenshotted/linked report states.

Part 5 expected answers:
1. One sentence with all three ingredients, e.g. (Option B): "We are 95% confident that the interval from about 54.1% to 61.9% captures the true proportion of all city residents who support the plaza." Reject "95% of residents…" and "95% chance…" phrasings (that's the point of the question).
2. Reported margins usually use the conservative planning value p* = 0.5 (which can only over-cover) and are rounded to whole points; a p̂-based margin is slightly smaller whenever p̂ ≠ 0.5. Plaza: ±3.9 (p̂-based) vs ±4.0 (conservative).
3. Any two of: nonresponse bias (those who answer differ from those who don't), undercoverage (the frame misses part of the population), response/wording bias (the question pushes an answer), voluntary response (opt-in panels). Full credit requires naming the Week 1 concept, not just "bias," plus a defensible "worries me more" pick tied to their poll's method row.
4. The margin shrinks like 1 ⁄ √n, so precision compounds in cost: from 600 to 2,401 (Plaza) roughly quadruples the interviews to halve the margin; ±1 would take 9,604 — the square-law budget is why media polls stop near ±3.
5. Plaza Poll: the entire interval (54.1% to 61.9%) sits above 50%, so "a majority supports the plaza" is supported at the 95% level — but "support is 58%" overstates the precision. Full credit for a Q-next sentence in the spirit of "Is the true support really above one-half?" — the Week 13 preview. (Option A polls: grade the same logic against wherever their interval falls; intervals containing 50% must be read as "majority undecided, not established.")

AI-critique full credit: names a specific claim checked or corrected — most commonly the chatbot quoting a memorized "±3 for n = 1,000" instead of computing, dropping the square root, or asserting the "95% chance" misread — OR documents verification of each claim. Zero credit for "the AI agreed with everything" with no evidence of checking.

Grading rubric — 50 points

The rubric below maps each part to its points; bands are full / partial / none.

Criterion	Full	Partial	None
Warm-up + poll facts — warm-up SE 0.05 and ME 0.098 correct; Table 1 complete with a legitimate poll (or Option B declared) (5)	5	3	0–1
The six-cell audit — p̂ as a decimal; SE, implied ME, and interval computed correctly from their own poll's numbers (15)	15	8–12	0–6
Reported vs. implied — conservative check computed; the comparison and its explanation (conservative convention + rounding) correct (10)	10	5–8	0–4
Analysis questions — Q1–Q5 hit the expected ideas, especially the correct interpretation sentence (Q1) and what the ± doesn't cover (Q3) (15)	15	8–12	0–6
AI-critique — specific catch or verification reported in 2–3 sentences (5)	5	3	0–2

Quality gate (self-checked): warm-up, worked-model, and Plaza-Poll arithmetic re-verified in the week's math script (SE 0.020149 → ±3.9 vs conservative ±4.0; planner 2,401 exact); all three source links live, free, and no-login (Pew Short Reads, Gallup News, AP-NORC Projects — each fetched and confirmed this build); the synthetic polls are labeled synthetic in the student text; no key number depends on any real poll's unverified content; no graded-item leakage (the quiz, practice, chapter self-checks, and assignment use different scenario surfaces and numbers); rubric sums 5 + 15 + 10 + 15 + 5 = 50. ✓