Title: Week 13 Data Lab — Hypothesis Testing: Foundations
Course: Introduction to Statistics (18-Week Generic Edition)

Course: Introduction to Statistics (18-week generic edition) Objective: Objective 7 — the logic of significance testing: null worlds, p-values, and verdicts · SLO A (reason quantitatively from data) Worth 50 points · Data labs group = 15% of the grade · Data Lab 13 · Due: end of Week 13 Format: a hands-on data lab — this week you build the null hypothesis with your own hands: 200 sets of 20 fair coin flips in a spreadsheet, then you locate a suspicious result inside that world and deliver the verdict; then you catch the AI's mistakes when it interprets your work.

This is the course's signature weekly component. Every instructional week has a Data Lab: real, free, linked public data — nothing to buy, download, or sign up for — worked in your spreadsheet, finished with an AI-critique step. Plan on 60–90 minutes. (This week the "dataset" is one your own spreadsheet generates — an entire world where the null hypothesis is true by construction.)

Part 1 — The Big Picture

This week's chapter said every significance test asks one question: if nothing were going on, how surprising would my data be? Today you stop imagining that "if" and build it. Your spreadsheet will manufacture 200 honest sets of 20 fair coin flips — a complete world where H₀ ("the coin is fair, heads is no more likely than tails") is true by construction — and then you'll ask that world a pointed question: how often does plain chance produce 15 or more heads in 20 flips?

The setup: at a weekend market, a performer charges a dollar to demonstrate "heads on demand — I can make it land heads." You watch 20 flips and count 15 heads. Impressive… or exactly the kind of streak fair coins produce all the time? By the end of the lab you'll have an empirical p-value (from your simulation), the theoretical p-value (from Week 7's binomial, supplied by a spreadsheet function), a verdict at α = 0.05 — and, just as important, a clear view of what that verdict does and doesn't prove.

Optional 7-minute background read before you start: The Binomial Distribution (Math is Fun) 🔗 https://www.mathsisfun.com/data/binomial-distribution.html

Part 2 — The Task

The guiding question: Where does 15-of-20 sit in a world where the coin is fair — ordinary wobble, or out in the tail where chance almost never goes — and what verdict does that location earn?

The hypotheses on trial (write these into your submission): with p = the true probability the performer's flip lands heads — H₀: p = 0.5 (fair; "heads on demand" is theater) vs. Hₐ: p > 0.5 (heads really is more likely on their flips). The suspicion points up, so the extreme results are high head-counts.

The tools (nothing to buy or download): a spreadsheet — Google Sheets (free) or Excel. Its =RANDBETWEEN(0,1) function is a fair coin: each call produces 0 or 1 (call 1 "heads"), each with probability 0.5, with no memory of other calls.

Part 3 — Materials & Procedure

You need: a spreadsheet and about 60–90 minutes. Nothing else.

Warm-up (5 min, no spreadsheet needed). Three small computations by hand or calculator — write all three down; they're your benchmarks for reading the simulation:

1. The expected number of heads in 20 fair flips: 20 × 0.5 = ______.
2. The standard deviation of that head-count (Week 7's formula √(np(1−p))): √(20 × 0.5 × 0.5) = √5 = ______ (round to two decimals).
3. Using answers 1 and 2: the observed 15 heads sits (15 − 10) ÷ 2.24 ≈ ______ standard deviations above expected (round to one decimal). Keep that number in mind as you watch the simulation.

Now build the null world:

1. Label the columns. In row 1, type flip1 in A1, then fill the label across through T1 (flip20); in U1 type heads.
2. Flip one set of 20. In A2 type =RANDBETWEEN(0,1) and copy it across A2:T2 — twenty fair flips, where 1 = heads.
3. Count the set's heads. In U2 type =SUM(A2:T2).
4. Make 200 sets. Select A2:U2, copy, and paste into rows 3 through 201. Column U now holds the head-counts of 200 honest sets. (They re-flip every time the sheet recalculates — expected; see step 8.)
5. Ask the tail question. In W2 type =COUNTIF($U$2:$U$201, ">=15") — how many of your 200 fair sets produced 15 or more heads? In W3 type =W2/200 — your empirical p-value for the performer's 15.
6. Get the exact answer too. In W4 type =1 - BINOM.DIST(14, 20, 0.5, TRUE) — the theoretical P(X ≥ 15) for a fair coin. It returns 0.0207; your W3 estimates this same tail by brute force.
7. Picture the world. Select U2:U201 → Insert ▸ Chart → histogram (Excel: Insert ▸ Statistic Chart ▸ Histogram). You should see a mound centered near 10, thinning fast toward the edges. Give it a title like "200 sets of 20 fair flips."
8. Freeze your run. Select columns A:U → Copy → Paste special ▸ Values only onto the same cells (Excel: Paste Values). Your 200 sets are now frozen data.
9. Profile the null world. In empty cells: =MIN($U$2:$U$201), =MAX($U$2:$U$201), and =COUNTIFS($U$2:$U$201, ">=6", $U$2:$U$201, "<=14") — the count of "ordinary" sets (6–14 heads).
10. Run the companion case. A classmate watched a different performer flip 12 heads in 20. In X2: =COUNTIF($U$2:$U$201, ">=12"), in X3: =X2/200 (empirical p-value for 12), and in X4: =1 - BINOM.DIST(11, 20, 0.5, TRUE) — the theoretical P(X ≥ 12) = 0.2517.

Plausibility note (read before judging your run): with 200 sets, the count reaching 15+ heads is usually 1 to 8 sets (empirical p-value roughly 0.005–0.04), and about 1 run in 65 shows zero such sets — if that's yours, your empirical p-value is 0, read as "rarer than 1-in-200," and nothing is broken. The 12+ count typically lands around 38–63 sets. Your histogram's mound should sit near 10 with almost every set between 6 and 14.

Fallback (no spreadsheet available): flip a real coin 20 times and tally heads; repeat for as many sets as you reasonably can (aim for 10) and say you used the physical fallback. For the 200-set questions, use this model run (a frozen run of the same simulation; counts sum to 200): 3 heads × 1 set · 5 × 1 · 6 × 5 · 7 × 8 · 8 × 22 · 9 × 44 · 10 × 33 · 11 × 24 · 12 × 35 · 13 × 14 · 14 × 7 · 15 × 6. From it: sets with ≥ 15 heads = 6 (empirical p-value 0.03); sets with ≥ 12 heads = 62 (empirical p-value 0.31); sets with 6–14 heads = 192; min 3, max 15.

Part 4 — The Scaffold (complete both tables)

Table 1 records the trial of the performer's 15-of-20, from hypotheses to verdict.

Item | Your entry
H₀ (symbols and words) | ______
Hₐ (symbols and words, with direction) | ______
Observed result on trial | 15 heads in 20 flips
Sets (of your 200) with ≥ 15 heads | ______
Empirical p-value (count ÷ 200) | ______
Theoretical p-value (from =1-BINOM.DIST(14,20,0.5,TRUE)) | ______
Decision at α = 0.05 (with the template sentence) | ______

Table 2 profiles your null world and runs the companion case.

Quantity | Value
Expected heads per set (warm-up 1) | ______
Smallest head-count in your 200 sets (=MIN) | ______
Largest head-count in your 200 sets (=MAX) | ______
Sets with 6–14 heads ("ordinary" sets, =COUNTIFS) | ______
Sets with ≥ 12 heads | ______
Empirical p-value for the companion's 12-of-20 | ______
Theoretical P(X ≥ 12) (from =1-BINOM.DIST(11,20,0.5,TRUE)) | ______
Decision for the companion performer at α = 0.05 | ______

Part 5 — Analysis Questions

Answer in 1–3 sentences each:

1. Describe your histogram — where's the mound, and how far out do the edges reach? Explain why this picture deserves the name "the null world": what, exactly, is true in it by construction?
2. Your empirical p-value and the theoretical 0.0207 are close but (almost certainly) not equal. Are they estimates of the same thing or different things — and why don't they match exactly? (Your Week 5 long-run ideas apply.)
3. Deliver the verdict on "heads on demand" at α = 0.05 using the course template sentence. Then the careful part: does rejecting H₀ prove the performer has a skill? Name at least one explanation other than skill or plain chance that your test cannot rule out (think about the coin itself, or how the flip is thrown).
4. The companion performer's 12-of-20 gives a p-value around 0.25. State that verdict with the template sentence — and explain why this result does not prove that performer's flips are fair.
5. (Connect forward.) Suppose the market ran this same α = 0.05 test on 100 honest performers whose coins are all perfectly fair. About how many would be wrongly "convicted," what is that error called — and why does this number explain the course rule that α is chosen before the data arrive? (Next week this exact logic runs on means, with the t-test doing the tail arithmetic.)

Part 6 — AI-Critique Moment (required — the BYOAI step)

Now bring in your chatbot and be the analyst who checks its work.

1. Paste your completed Table 1 into your chatbot, along with this claim: "My empirical p-value came out a bit different from the theoretical 0.0207, so I think my simulation is broken. Also, since the p-value is small, there's only about a 2% chance the coin is fair — right? Please explain my results."
2. Check everything it says against your own work. This week's three named AI failure modes: - Endorsing the misread — agreeing (or half-agreeing, to be agreeable) that a small p-value is "the chance the coin is fair." It isn't: the p-value assumes fairness and reports how often chance looks this extreme — P(data | H₀), never P(H₀ | data). - Noise read as a bug — calling your simulation "broken" because the empirical and theoretical p-values differ. With only 200 sets, empirical counts of ≥ 15 anywhere from about 1 to 8 are ordinary; both numbers estimate the same tail. - Arithmetic slips — a COUNTIF miscount, or an empirical p-value not equal to (count ÷ 200). Re-do the division yourself.
3. Write 2–3 sentences reporting what the AI got right and at least one thing you had to correct, question, or verify. If it happened to get everything right, say how you verified each claim — that's the skill.

The habit all term: the tool drafts, you judge. A chatbot will confidently call your honest simulation broken — or soothe you with the p-value misread. Catching it is the point.

Part 7 — What to Submit

One document or text entry containing: your three warm-up values, your completed Table 1 and Table 2, a screenshot or one-sentence description of your histogram, your Part 5 answers (1–5), and your Part 6 AI-critique paragraph. Due at the end of Week 13 · 50 points (rubric below the key).

Grading rubric — 50 points

The rubric below maps each part to its points; bands are full / partial / none.

Criterion | Full | Partial | None
Warm-up + build — 10, 2.24, and ≈2.2 correct; the 20-column × 200-row null world constructed with the specified formulas (8) | 8 | 4–6 | 0–3
The trial (Table 1) — hypotheses correct in symbols and words with direction justified; empirical p-value = own count ÷ 200; theoretical 0.0207 recorded; correct verdict in template language (12) | 12 | 6–10 | 0–5
Histogram + null-world profile (Table 2) — histogram present and read correctly; min/max/ordinary-count filled from own run; companion case (12-of-20) completed with correct fail-to-reject verdict (10) | 10 | 5–8 | 0–4
Analysis questions — Q1–Q5 hit the expected ideas, especially the null-world meaning (Q1), empirical vs. theoretical (Q2), and the two "doesn't prove" traps (Q3–Q4) (15) | 15 | 8–12 | 0–7
AI-critique — specific catch or verification reported in 2–3 sentences (5) | 5 | 3 | 0–2