← Back to the Introduction to Statistics outline The Course Maker
← Introduction to Statistics outline
Week 13 · Lecture outline

Week 13 — Lecture Outline · Hypothesis Testing: Foundations

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
Objectives covered: Objective 7 — Carry out and interpret hypothesis tests (this week: the logic — null and alternative hypotheses, p-values, significance levels, and error types; the t and z machinery arrive in Weeks 14–15).
SLOs touched: A (reason quantitatively from data) · B (communicate results to a non-technical audience)
Meeting pattern: planned as 2 sessions × ~75 min = ~150 min. Segment minutes below total ~150; scale them to your own pattern.

Week at a Glance
The week's big question "A claim says one thing; your data say another. How surprising does the data have to be before you're allowed to call the claim wrong?"
By the end of the week, students can… (1) write a null and an alternative hypothesis (H₀ and Hₐ) for a claim, in symbols and words, with the right direction (<, >, ≠); (2) say in one sentence what a p-value means — and what it does not mean; (3) make the reject / fail to reject decision by comparing a p-value to a significance level α, and state the conclusion in context; (4) describe a Type I and a Type II error in context and explain the trade-off; (5) explain why "fail to reject" ≠ "accept" and why statistical significance ≠ practical importance.
Key vocabulary null hypothesis (H₀), alternative hypothesis (Hₐ), one-sided vs. two-sided alternative, test statistic, p-value, significance level (α), statistically significant, reject H₀, fail to reject H₀, Type I error (false alarm), Type II error (miss), statistical vs. practical significance
Materials slides (Deck 13), the Week 13 chapter, the week's readings + video links, a spreadsheet (Google Sheets or Excel), the student's chatbot for the AI-critique moment and the tutorial
Timing note 8 segments, ~150 min total. Session 1 = Segments 1–4 (~75). Session 2 = Segments 5–8 (~75).

Segment 1 — Hook & the Promise (8 min) · Session 1 opens

Hook. "In the last 24 hours, how many claims did the world make at you? 'Charges in 90 minutes.' 'Our new formula works better.' 'Lasts twice as long.' Somebody measured something once — and now it's printed in bold type." Give them ten seconds to recall one. Then: "Here's the uncomfortable part. Even when a claim is true, your data will almost never match it exactly. Samples wobble — we've known that since Week 10. So when your measurement disagrees with the claim, there are always two suspects: the claim is wrong, or chance did it."

  • Hypothesis testing is the machinery for deciding between those two suspects — with a number attached.
  • Frame it now, use it all week: every significance test is a tiny courtroom. The claim goes on trial, it gets the benefit of the doubt, and the data are the evidence.

The promise (write it on the board): "By the end of this week you can take any claim — a package label, a marketing boast, a 'studies show' — put it on trial properly, read the p-value the evidence produces, and deliver the verdict in plain English, including what the verdict does NOT prove."

Why it matters line (memory hook): "A hypothesis test never asks 'is the claim true?' It asks: 'if the claim were true, how surprising would my data be?'"

Segment 2 — Two Rival Hypotheses: H₀ and Hₐ (20 min)

Plain language first. Every test starts by naming the two suspects from the hook as two rival explanations:

  • The null hypothesis (H₀) is the dull explanation: nothing's going on — the claim is as stated, there's no change, no effect, no difference; anything odd in the data is just sampling chance. H₀ is the defendant, and it gets the benefit of the doubt.
  • The alternative hypothesis (Hₐ) is the suspicion: something IS going on — the thing we need convincing evidence to conclude.

Three rules for writing them (say all three, then drill them):
1. Hypotheses are about parameters — μ or p, the population truth — never about statistics (never x̄ or p̂). You don't put your own sample on trial; you already know what it says.
2. H₀ always gets the equals sign (μ = 90, p = 0.5). It's the specific, testable "nothing's going on" value.
3. Hₐ points where the suspicion points: < (suspect too low), > (suspect too high), or ≠ (suspect different, either direction — the two-sided alternative). The direction comes from the question, chosen before looking at the data.

One fully worked example (setup — this example runs all week).

A phone-charger maker advertises: "Charges your phone from empty to full in 90 minutes, on average." A tech-review site suspects real-world charging runs slower than advertised, so it plans to time 25 full charges.
- The parameter: μ = the true mean charge time (in minutes) across all charges of this model.
- H₀: μ = 90 — the claim is as stated; any slow-looking sample is just chance.
- Hₐ: μ > 90 — charging genuinely takes longer than advertised. (One-sided, pointing right: "slower" means more minutes. The classic wrong-tail slip is writing μ < 90 because "slower" sounds like "less.")
- Not allowed: "H₀: x̄ = 90" (that's the sample, not the population) or deciding the direction after peeking at the 25 measurements.

Quick second setup (two-sided). An oven's display claims the true oven temperature matches the 350° setting. A repair technician doesn't suspect a direction — the sensor could run hot or cold. H₀: μ = 350 vs. Hₐ: μ ≠ 350. No suspicion of direction → two-sided.

Memory hook: "H₀ is the dull explanation, and the dull explanation gets the benefit of the doubt."

Segment 3 — The Logic of a Test: The Tea Test That Started It All (25 min)

Tell the story (the week's signature — every beat matters).

Cambridge, England, in the 1920s. At an afternoon tea, a scientist named Muriel Bristol declares she can taste whether the milk or the tea was poured into the cup first. The statistician Ronald Fisher, also at the table, doesn't believe her — and instead of arguing, he invents the modern significance test on the spot.
The design: 8 cups — 4 poured milk-first, 4 tea-first — presented in random order. She knows there are four of each; she must say which four are which.
The logic (walk it slowly — this is the whole week in four steps):
1. Assume the dull explanation. H₀: she's guessing — no tasting ability at all.
2. Ask what guessing would produce. A guesser is choosing 4 cups out of 8 to call "milk-first." There are 70 equally likely ways to choose 4 of 8, and only one is completely right. So P(a guesser gets all 8 cups correct) = 1/70 ≈ 0.014.
3. Look at what actually happened. As the story is told: she got all eight right.
4. Deliver the verdict. If she were guessing, a result this good happens about 1.4% of the time. That's too surprising to shrug off — we reject "she's guessing." The evidence says she can taste the difference.

Now name what you just computed. That 0.014 is a p-value:

The p-value = the probability, computed assuming H₀ is true, of getting data at least as extreme as what was actually observed.

Say the definition twice, because every word is load-bearing: assuming H₀ (it's a what-if, not a fact about the world) and at least as extreme (a tail, not a single outcome).

The scale to read it with (write on the board):
- Small p-value → data this extreme are rare if H₀ is true → the data are hard to blame on chance → evidence against H₀.
- Large p-value → data like this are ordinary if H₀ is true → chance explains it fine → no case against H₀.

And the test statistic (light touch — one sentence, machinery arrives in Week 14): before the p-value, software computes a test statistic — a standardized score measuring how far the sample landed from what H₀ expects (the z-scores of Weeks 8 and 10 were this idea). Far from expected → big statistic → small p-value. This week we read p-values; next week we compute the statistics ourselves.

Memory hook: "The p-value is the courtroom question in a number: could plain chance have done this?"

Segment 4 — Misconceptions + Decision Drill (22 min) · Session 1 closes (~75)

First, the decision rule (needed for the drill). Before testing, we choose a significance level α — the "surprising enough" line, usually 0.05. Then: p ≤ α → reject H₀ (the result is statistically significant); p > α → fail to reject H₀. Depth on α next session; for now it's the pre-set bar the evidence must clear.

Name the misconceptions out loud, then cure each:

  • ❌ "The p-value is the probability that H₀ is true." (The #1 misread in all of statistics.)
    ✅ Cure: the p-value is computed assuming H₀ is true — it can't turn around and tell you the probability of the thing it assumed. The tea test's 0.014 is not "a 1.4% chance she was guessing"; it's "guessing produces a result this good 1.4% of the time." P(data | H₀), never P(H₀ | data).
  • ❌ "A large p-value proves H₀ — the claim is confirmed."
    ✅ Cure: a large p-value means the data are compatible with H₀, not that H₀ is true. "Not enough evidence to convict" is not "proven innocent." We say fail to reject, never accept.
  • ❌ "We test whether x̄ equals the claim."
    ✅ Cure: hypotheses are about the population (μ, p). The sample is the evidence, not the defendant.
  • ❌ "p = 0.049 is a discovery and p = 0.051 is nothing."
    ✅ Cure: α is a line we chose; evidence just barely over or under it is nearly identical evidence. Report the p-value, not just the verdict — and treat borderline results with humility.

Interaction — Think-Pair-Share (decision drill, ~10 min):
Put five results on a slide; students decide reject or fail to reject solo (30 sec), compare with a neighbor (1 min), then vote by show of hands. Use α = 0.05 for the first four:
(1) p = 0.003 · (2) p = 0.21 · (3) p = 0.049 · (4) p = 0.62 · (5) p = 0.03, but this lab uses α = 0.01.
(Answers: reject · fail to reject · reject — barely, worth saying out loud · fail to reject · fail to reject at α = 0.01, though the same p would reject at α = 0.05.)
Debrief (5): "significant" depends on the α you set before looking — and 0.049 vs. 0.051 is a line-call, not a chasm.

Segment 5 — α and the Two Ways to Be Wrong (25 min) · Session 2 opens

Hook back in: "Last session we set a bar: p ≤ 0.05, reject. Today's question: where does 0.05 come from — and what happens when the verdict is wrong?"

Plain language first — α is a risk you choose. Even a perfect test run on perfect data can convict an innocent H₀: if H₀ is true, a p-value ≤ 0.05 still happens 5% of the time — that's what p-values do. So:

  • α = the false-alarm risk you accept. Setting α = 0.05 means: "I'm willing to wrongly reject a true H₀ about 1 time in 20." It is chosen before the data are examined — a standard, not a law of nature.

The two errors (name them with the courtroom):
- Type I error — rejecting a true H₀. Convicting the innocent. The false alarm. Probability = α.
- Type II error — failing to reject a false H₀. Letting the guilty walk. The miss.

Modern example (worked in context — spam filter):

An email spam filter tests every incoming message with H₀: "this message is legitimate."
- Type I: a real message from your instructor lands in the junk folder — the filter "convicted" an innocent email. (False alarm.)
- Type II: a scam email sails into your inbox — the filter missed a guilty one.
Ask the class: which error should the filter's designers fear more? There's no free lunch: make the filter less trigger-happy (lower α) and fewer real emails get junked — but more spam gets through (more Type II). Tightening one error loosens the other, at a fixed sample size.

The design question that follows: consequences pick α. Screening aircraft parts for hidden cracks? A miss is catastrophic — tolerate more false alarms. Deciding whether to reformulate a snack flavor? A false alarm just wastes a taste test — α = 0.05 is fine. There is no universally correct α; there are consequences.

Memory hook: "Type I cries wolf when there's no wolf. Type II sleeps through the real one."

Segment 6 — The Full Walk + Writing the Verdict (18 min)

Now run the charger trial end to end (all five steps, out loud).

The claim: "charges in 90 minutes, on average." The suspicion: slower.
1. Hypotheses: H₀: μ = 90 vs. Hₐ: μ > 90.
2. Significance level: α = 0.05, chosen before timing anything.
3. Evidence: 25 timed charges average x̄ = 96 minutes — 6 minutes over the claim. Suspicious… or ordinary wobble?
4. p-value: the site's software reports p = 0.021. Meaning, in words: if the true average really were 90 minutes, only about 2.1% of samples of 25 charges would average this high or higher.
5. Verdict: 0.021 ≤ 0.05 → reject H₀. Conclusion in context: "There is convincing evidence that this charger's true mean charge time exceeds the advertised 90 minutes."

The two sentence templates (post them; require them all term):
- Reject: "We reject H₀. There is convincing evidence that [Hₐ, in context]."
- Fail to reject: "We fail to reject H₀. There is not convincing evidence that [Hₐ, in context]."
- Banned phrasings: "we accept H₀," "H₀ is true," "the claim is proven."

The companion case (run it immediately — same site, different product):

The site also tests the maker's travel charger, advertised at 150 minutes. 25 charges average x̄ = 153, and the software reports p = 0.38. Verdict: 0.38 > 0.05 → fail to reject H₀ — "there is not convincing evidence that the travel charger's true mean exceeds 150 minutes."
Now the trap question: did we just prove the travel charger's claim is accurate? No. Maybe the claim is fine — or maybe it's off by a little and 25 charges couldn't see it. "Not guilty" is not "innocent"; it's "the evidence didn't clear the bar."

Memory hook: "Absence of evidence is not evidence of absence."

Segment 7 — Statistically Significant vs. Actually Important (15 min)

Plain language first. "Significant" is the most misleading word in statistics: in a test it means the data are hard to explain by chance — it does not mean the effect is big or matters.

Worked example (the one they'll remember):

A running-shoe brand tests a new insole on 80,000 runners and finds marathon times improve by 4 seconds on average, with p = 0.001.
- Statistically significant? Absolutely — p = 0.001 means chance is a terrible explanation for the 4 seconds. With 80,000 runners, the test can detect whisper-thin differences.
- Practically important? Four seconds across a race that takes hours — imperceptible. Would you pay extra for it?
The lesson: with a huge sample, even a trivial true difference produces a tiny p-value. The p-value measures surprise, not size. Always ask two questions: "Is it significant?" and "How big is the effect?"

The mirror image (one line, sets up honesty about small studies): a small sample can miss a real, important effect — a big Type II risk — so "not significant" in a tiny study is weak information, not a debunking.

Memory hook: "Statistical significance measures surprise, not size."

Segment 8 — Technology Workflow + AI-Critique, Callback & Hand-off (17 min) · Session 2 closes (~75)

Technology workflow — build a "nothing's going on" world in a spreadsheet (exact steps; this is the Data Lab's engine):
1. One simulated set of 20 coin flips: in A2 type =RANDBETWEEN(0,1) and fill across to T2 (1 = heads). In U2: =SUM(A2:T2) — the number of heads in that set.
2. Fill the whole block down to row 201 — 200 sets of 20 fair flips: an entire world where H₀ ("the coin is fair") is true by construction.
3. Ask a p-value-flavored question with =COUNTIF(U2:U201, ">=15") — in how many of 200 honest sets did chance alone produce 15 or more heads? (Very few — that's the point.)
4. The exact tail, without simulation: =1 - BINOM.DIST(14, 20, 0.5, TRUE) returns P(X ≥ 15) = 0.0207 — Week 7's binomial, now working as a p-value. Google Sheets and Excel: identical functions.

AI-critique moment (students verify, not consume):

Paste this to your chatbot: "My hypothesis test gave a p-value of 0.21, so there's a 21% chance the company's claim is true. Also, since we failed to reject the null hypothesis, we've shown the claim is correct. Please explain my results."
Both sentences are wrong — and chatbots asked to "explain" often play along instead of objecting. Your job: catch both. (The p-value is P(data at least this extreme | H₀ true), never P(claim true); and failing to reject proves nothing.) If the bot does catch them, verify its corrected wording against the course definition. The tool drafts, you judge.

Callback + tease:
- Callback: "Week 12 ended with a confidence interval — the range of plausible parameter values. A hypothesis test asks the matching question from the other side: is this one claimed value plausible? Same engine (sampling distributions, Week 10), new verdict. The two meet officially next week."
- Tease next week: "This week every p-value was handed to you. Week 14: you compute them — the one-sample t-test, with the same t-table you met in Week 11. The logic never changes again; only the machinery."

Hand-off (the week's work):
- Chapter 13 (the primary reading) if they haven't read it yet — then Lecture Tutorial 13 (AI tutor, share-link submission) — hypotheses, p-values, α, error types.
- Data Lab 13 (simulate the null: 200 sets of 20 flips — where does 15/20 sit?) · Quiz 13 (end of week) · Discussion 13 (the 0.05 question) · Assignment 13 (AI-coached).

Instructor FAQ — Common Stumbles

Student says / does Quick cure
"So the p-value is the chance the null is true?" No — it's computed assuming H₀ is true, so it can't report the probability of H₀. It answers: "how often would chance alone produce data this extreme?" Re-run the tea example: 0.014 is a fact about guessers, not about her.
Writes H₀: x̄ = 90. Hypotheses are about the population (μ, p). The sample is the evidence, not the defendant. One-line check: "could you have written this hypothesis before collecting any data?"
"We accept the null." Banned phrase. A big p-value means the data are compatible with H₀, not that H₀ is proven — "not guilty," never "innocent." Say fail to reject.
Picks the wrong tail ("slower, so μ < 90"). Translate the suspicion into the measurement's direction first: slower charging = more minutes → μ > 90. Have them say what a suspicious sample would look like before writing Hₐ.
"Why 0.05?" Convention and history, not physics — a default false-alarm budget of 1-in-20. Consequences should move it: catastrophic misses → raise α (or better, more data); costly false alarms → lower α.
Mixes up Type I and Type II. Courtroom + wolf: Type I cries wolf when there's no wolf (reject a true H₀, probability α); Type II sleeps through a real wolf (miss a false H₀).
"p = 0.001, so the effect is huge." The p-value measures surprise, not size — with n = 80,000, a 4-second improvement gets p = 0.001. Always ask for the effect size next.
"My p-value is 0.06, so the study proves there's no effect." Two errors at once: 0.06 is weak-but-real evidence (barely over a chosen line), and "no evidence of effect" ≠ "evidence of no effect" — especially in a small study (Type II risk).

Scope flag

This outline stays within Objective 7's logic-first scope: p-values are given (by software or a story's arithmetic), never computed from formulas — the t/z mechanics are Weeks 14–15. The 1920s tea-test story, the spam-filter error framing, and the one-line CI-duality callback are added context (not strictly required by the objective) — kept because they make the logic and the error types stick; cut them for a leaner session. Statistical power is intentionally not introduced (one optional video in the readings previews it).