← Back to the Introduction to Statistics outline The Course Maker
← Introduction to Statistics outline
Week 15 · Lecture outline

Week 15 — Lecture Outline · Testing Proportions & Two-Sample Inference

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
Objectives covered: Objective 7 — Conduct and interpret hypothesis tests for means and proportions (this week: the one-proportion z-test and the two-sample procedures for means and proportions, plus choosing the right procedure).
SLOs touched: A (reason quantitatively from data) · B (communicate results to a non-technical audience)
Meeting pattern: planned as 2 sessions × ~75 min = ~150 min. Segment minutes below total ~150; scale them to your own pattern.

Week at a Glance
The week's big question "Half the internet is a running experiment — version A for some users, version B for others. How do we test a claim about a percentage, and how do we tell whether two groups really differ or just wobbled apart by chance?"
By the end of the week, students can… (1) run a complete one-proportion z-test (hypotheses, conditions, z, p-value from the friendly z-table, conclusion in context); (2) read a two-proportion z printout — including why the SE pools — and conclude correctly; (3) read a two-sample t printout (df technology-reported) and conclude correctly; (4) tell paired from independent designs; (5) choose the right procedure on sight — one-prop z, two-prop z, one-sample t, two-sample t, or paired t.
Key vocabulary one-proportion z-test, hypothesized proportion p₀, standard error under H₀, two-sample inference, two-proportion z-test, pooled proportion, two-sample t-test, technology-reported degrees of freedom, independent vs. paired samples, statistical vs. practical significance, power (informal), A/B test
Materials slides (Deck 15), the Week 15 chapter (it re-prints the friendly z-table), the week's readings + video links, a spreadsheet (Google Sheets or Excel), the student's chatbot for the AI-critique moment and the tutorial
Timing note 8 segments, ~150 min total. Session 1 = Segments 1–4 (~75). Session 2 = Segments 5–8 (~75).

Segment 1 — Hook & the Promise (8 min) · Session 1 opens

Hook. "Sometime this week, an app almost certainly ran an experiment on you. A shopping site showed you one checkout page and showed your neighbor another. A streaming service tried two thumbnails. You were in group A or group B, and nobody asked." That practice has a name — the A/B test — and it is nothing more than this course's machinery pointed at two groups at once.

  • Week 13 gave us the courtroom logic of testing; Week 14 aimed it at means with the t. Two gaps remain: claims about percentages, and the everyday question "are these two groups actually different?"
  • "By the end of this week you will be able to run the test behind every A/B headline you'll ever read."

The promise (write it on the board): "By the end of this week you can test any claimed percentage with one sample, read any two-group printout — proportions or means — and, given a study description, name the right procedure in ten seconds."

Why it matters line (memory hook): "One sample asks 'is the claim true?' Two samples ask 'are these the same?' — and most real questions are the second kind."

Segment 2 — The One-Proportion z-Test (22 min)

Plain language first. Week 13's four-step logic, unchanged: assume the claim (H₀), measure how surprising the data would be under it, and let the p-value speak. The only new part is the statistic. Claims about a percentage are claims about a population proportion p — and Week 12 already gave us p̂'s behavior. The test statistic is a z:

z = (p̂ − p₀) ⁄ SE, where SE = √( p₀(1 − p₀) ⁄ n ) — "how many SEs is the sample proportion from the claimed proportion?"

  • p₀ is the claimed (hypothesized) value — the number in H₀.
  • The SE is built from p₀, not p̂: the test lives inside H₀'s world, so H₀ supplies the yardstick. (Contrast with Week 12's interval, which used p̂ — the interval assumes nothing, the test assumes the claim.)
  • p-value comes from the friendly z-table (Week 8's, re-printed in the chapter); off-table z's come from technology.

One fully worked example (do every step out loud).

A wildflower seed packet claims "50% of seeds sprout." A community garden plants a random 100 seeds from a large seed lot; 60 sprout.
- Hypotheses (two-sided — we'd care either way): H₀: p = 0.50 · Hₐ: p ≠ 0.50. (About p, the true sprouting rate — never about p̂.)
- Sample proportion: p̂ = 60 ÷ 100 = 0.60.
- SE under H₀: √(0.50 × 0.50 ⁄ 100) = √0.0025 = 0.05.
- z = (0.60 − 0.50) ⁄ 0.05 = 2.0 — the sample landed 2 SEs above the claim.
- Two-sided p-value from the table: 2 × (1 − 0.9772) = 2 × 0.0228 = 0.0456 ≈ 0.046.
- At α = 0.05: 0.046 < 0.05 → reject H₀. Say it in words: "If the packet's 50% were true, a sample this far out would happen only about 5 times in 100 — convincing evidence the true sprouting rate isn't 50%, and our sample points higher."

Land the connection: same z-recipe as Week 8, same logic as Week 13, same SE family as Week 12. Nothing new is hard here; it's assembly.

Segment 3 — Conditions & the Proportion-Test Traps (15 min)

Check before you z (the conditions, in plain words):

  1. Random sample (or random assignment) — Week 1 never stops mattering.
  2. Large-count condition using p₀: np₀ ≥ 10 and n(1 − p₀) ≥ 10 — at least 10 expected successes and 10 expected failures under the claim. (Seed packet: 100 × 0.5 = 50 and 50 ✓.)
  3. Sample less than 10% of the population when sampling without replacement (100 seeds from a warehouse lot ✓).

Misconceptions + cures:

  • ❌ Writing hypotheses about p̂ ("H₀: p̂ = 0.50").
    ✅ Cure: hypotheses are always about the population (p). p̂ is the evidence, not the suspect. If your hypothesis has a hat, take the hat off.
  • ❌ Building the SE from p̂ in a test.
    ✅ Cure: the test assumes H₀ is true, so p₀ supplies the SE. (In the seed example the wrong SE gives a slightly different z — and on a close call, "slightly different" flips decisions.) One line to keep: interval → p̂; test → p₀.
  • ❌ Checking np ≥ 10 with the sample's p̂.
    ✅ Cure: same principle — conditions for the test are checked in H₀'s world, with p₀.
  • ❌ "n = 100 is big, so nothing else matters."
    ✅ Cure: with p₀ = 0.02, even n = 100 expects only 2 successes — the large-count condition fails. It's about expected counts, not raw n.

Quick check (cold-call): a claim says 80% of a strip mall's gift cards get redeemed; you sample 25. Does the test qualify? (np₀ = 20 ✓ but n(1 − p₀) = 5 ✗ — fails. Cure options: bigger sample or technology's exact methods — beyond our scope, but knowing it fails is the skill.)

Segment 4 — Two Samples: The Two-Proportion z-Test (30 min) · Session 1 closes (~75)

Plain language first. New question shape: not "is it 50%?" but "are these two rates the same?" Two independent random samples, one from each group.

  • H₀: p₁ = p₂ — "one shared rate; any gap we saw is chance." Hₐ: p₁ ≠ p₂ (or one-sided when only one direction matters and was declared in advance).
  • Evidence: the gap between sample proportions, p̂₁ − p̂₂.
  • The pooling move (teach it as H₀ speaking): if H₀ is true, both groups share ONE rate — so the best estimate of it uses everyone: p̂_pool = (both groups' successes combined) ⁄ (both sample sizes combined). The SE is built from p̂_pool. Pooling isn't a fussy detail; it is the null hypothesis doing the arithmetic.
  • Depth check (course convention): students compute p̂₁, p̂₂, and p̂_pool by hand; the SE, z, and p-value arrive on a printout they read. Show the formula once so the printout isn't magic — then read output like working statisticians.
  • Conditions: both samples random and independent of each other; at least 10 successes and 10 failures in each sample (here we can use the observed counts).

One fully worked example (interpret-the-output — walk every line).

An online retailer has two customer-support teams. From this month's rated tickets it samples independently: Team A — 90 of 120 customers satisfied (p̂₁ = 0.75). Team B — 72 of 120 satisfied (p̂₂ = 0.60). Are the teams' true satisfaction rates different?
- Counts check: 90/30 and 72/48 successes/failures — all ≥ 10 ✓.
- Pooled proportion: (90 + 72) ⁄ (120 + 120) = 162 ⁄ 240 = 0.675.
- Technology output: difference = 0.15 · SE (pooled) = 0.0605 · z = 2.48 · two-sided p = 0.013
- Read it: 0.013 < 0.05 → reject H₀ — convincing evidence the two teams' satisfaction rates differ, with Team A's higher.
- The caution that earns full credit: nobody randomly assigned tickets. Team A may handle easier products or different hours. The test says the rates differ; it does not say the team's skill is why. (Correlation-handshake callback, Week 1.)

Interaction (2 min): "Why 162/240 and not the average of 0.75 and 0.60?" (Trick: with equal n's they coincide — 0.675 both ways. Ask what breaks if n₁ = 300, n₂ = 50. The bigger sample deserves more say — pooling by totals does that automatically.)

Segment 5 — Two-Sample t: Comparing Two Means (25 min) · Session 2 opens

Hook back in: "Last session, two rates. Today, two averages — same story, t instead of z."

Plain language first. Two independent groups, a measured amount (dollars, minutes, grams) in each:

  • H₀: μ₁ = μ₂ — one shared mean. Hₐ: μ₁ ≠ μ₂ (or one-sided, declared in advance).
  • t = (x̄₁ − x̄₂) ⁄ SE, with SE = √( s₁²⁄n₁ + s₂²⁄n₂ ) — each group contributes its own uncertainty.
  • Degrees of freedom: technology reports them (course convention). The two-sample df formula is genuinely ugly; software computes it, we read it. No pooling here — pooling was the proportions move, where H₀ implied one shared rate for the SE.
  • Conditions: both samples random and independent; each roughly normal or n's comfortably large (Week 11's shape rules, twice).

One fully worked example (interpret-the-output — every line).

A kitchen-supply site runs a true A/B test: visitors are randomly routed to a redesigned checkout page or the current one. Among completed orders: redesign — n = 40, x̄ = $63, s = $10; current — n = 40, x̄ = $57, s = $10.
- SE (shown once, then trust the printout): √(10²⁄40 + 10²⁄40) = √(2.5 + 2.5) = √5 ≈ 2.24.
- Technology output: t = 2.68 · df = 78 · two-sided p = 0.009
- Read it: 0.009 < 0.05 → reject H₀ — convincing evidence mean order value differs, higher on the redesign (a $6 average lift).
- And here's the payoff of Week 1: visitors were randomly assigned — this is an experiment — so "the redesign caused the lift" is finally a licensed sentence. Compare with the support teams: same arithmetic, different design, different conclusion rights.

Misconception + cure:
- ❌ "Different-looking x̄'s prove different μ's."
✅ Cure: every pair of samples differs somewhat — that's Week 10's sampling variability. The test asks whether the gap is bigger than chance's usual wobble.

Segment 6 — Paired or Independent? Choosing the Right Procedure (20 min)

Plain language first. Week 14 taught the paired t (same individuals measured twice → analyze the differences). The classic error is running a two-sample t on paired data, or vice versa. The tell: "Could I match each measurement in group 1 to exactly one in group 2 for a reason?" Yes → paired. No → independent.

The two-question procedure map (the collection slide — students photograph this):

  1. What kind of data? Counting successes (a percentage) → z for proportions. Measuring amounts (means) → t for means.
  2. How many samples? One sample vs. a claimed value → one-sample (one-prop z this week; one-sample t, Week 14). Two independent samples → two-sample (two-prop z; two-sample t). Same individuals twice → paired t (Week 14).

Interaction — Think-Pair-Share (rapid-fire, ~10 min). Six scenarios on a slide; solo 30 sec, pair 1 min, vote. Name the procedure:
1. A juice brand claims 40% of shoppers can taste the difference; one random sample of 250 shoppers is tested → one-prop z
2. A bakery's loaves should average 500 g; a random sample of 30 loaves is weighed → one-sample t (Week 14)
3. Reaction times of two independent random samples of gamers, one per controller design → two-sample t
4. The same 30 runners' 5K times in their old shoes and again in new shoes → paired t (Week 14)
5. Two cafeteria lines: the share of diners choosing the veggie option, from a random sample in each line → two-prop z
6. A streaming service asks whether the true share of trial users who convert differs from the claimed 25%, from one random sample of 400 → one-prop z
Debrief the two that split the room: #4 (same runners twice → paired) and #5 (counting choices, two groups → proportions, not means).

Segment 7 — The Grand Misconception Tour + the Power Moment (18 min)

Name them out loud, then cure each (this is midcourse-inference hygiene):

  • ❌ "p = 0.08, so the two groups are the same."
    ✅ Cure: fail to reject ≠ accept (Week 13's law, now at two-sample stakes). No evidence of a difference is not evidence of no difference. Say the licensed sentence: "the data don't provide convincing evidence of a difference."
  • ❌ "p < 0.001, so the difference matters."
    ✅ Cure: statistical ≠ practical significance. With enormous samples, trivially small gaps earn tiny p-values. Always report the size of the difference in real units next to the p-value ("a $0.04 average lift, p < 0.001" should embarrass someone).
  • ❌ "Significant two-sample result → group 1's label explains it."
    ✅ Cure: causal sentences need random assignment (the A/B test has it; the support teams don't). The test compares rates; the design decides what the comparison means.
  • ❌ "We didn't find it, so it isn't there."
    ✅ The power moment (plant the lab): even when a difference is REAL, an honest sample can miss it — that miss is Week 13's Type II error, and how often you catch a real difference is called power (informally this course: bigger samples and bigger real gaps → more power). In Data Lab 15 the class simulates an A/B test where the truth IS different — and about 1 run in 5 will miss it anyway. A non-significant result is a shrug, not a verdict.

Mini-debate (genuinely arguable, ~5 min): "A streaming app A/B tests two home screens on two million users and finds a statistically significant +0.4-second increase in average watch time (p < 0.001). Should the team celebrate and ship it?" Let both sides run; land on: real effect, probably — worth anything, probably not. Significance answers "is it chance?", never "is it worth it?" (This is Discussion 15's territory.)

Segment 8 — Technology Workflow + AI-Critique, Callback & Hand-off (12 min) · Session 2 closes (~75)

Technology workflow — the two-proportion z in a spreadsheet (exact steps, live):
1. Put the counts in cells: successes₁, n₁, successes₂, n₂ (say B1, B2, B3, B4).
2. Pooled proportion: =(B1+B3)/(B2+B4) → for the support teams, 162/240 = 0.675.
3. SE: =SQRT(pool*(1-pool)*(1/B2+1/B4)) (reference the pooled cell) → 0.0605.
4. z: =(phat1-phat2)/SE → 2.48. p-value: =2*(1-NORM.S.DIST(ABS(z),TRUE)) → 0.013. (NORM.S.DIST(z, TRUE) is the spreadsheet's z-table — the left-tail area. Google Sheets and Excel are identical here.)
5. For means, the shortcut: =T.TEST(range1, range2, 2, 3) returns the two-sample p-value directly (2 = two-sided, 3 = the unequal-variance test this course uses). Technology also prints the df — that's where our "df = 78" comes from.

AI-critique moment (students verify, not consume):

Paste this to your chatbot: "Team A: 90 of 120 customers satisfied. Team B: 72 of 120. Is the difference statistically significant at the 5% level? Show your work."
Then check it against the printout (pooled z = 2.48, p = 0.013): chatbots often skip the pooling (their SE gives z ≈ 2.51 — close here, but a different recipe that will disagree on close calls), assert significance without checking the 10-successes/10-failures condition, or describe the p-value as "a 1.3% chance the teams are equal" — a meaning error you can now name. The tool drafts, you judge.

Callback + tease:
- Callback: "Week 13 built the courtroom; Week 14 put means on trial; this week proportions and pairs of groups took the stand — and you got the map that says which trial to run."
- Tease next week: "One-prop z handles two categories: sprout or don't. But what if checkout ends four ways — paid, saved cart, abandoned, error? Next week categorical data get a test of their own: chi-square."

Hand-off (the week's work):
- Chapter 15 (primary reading; friendly z-table re-printed) — then Lecture Tutorial 15 (AI tutor; share link + summary).
- Data Lab 15 ("Split the Traffic: Your Own A/B Test" — you'll run one) · Quiz 15 (end of week) · Discussion 15 ("The Everyday A/B Test") · Assignment 15 (AI-coached).

Instructor FAQ — Common Stumbles

Student says / does Quick cure
Writes "H₀: p̂ = 0.5". Hypotheses indict the population: H₀: p = 0.5. If the hypothesis wears a hat, take it off — p̂ is the evidence, not the claim.
"Week 12 used p̂ in the SE — why p₀ now?" The interval assumes nothing, so it uses the data (p̂). The test assumes H₀, so H₀ supplies the yardstick (p₀). One line: interval → p̂; test → p₀.
"Why pool the two samples?" Because H₀ says the groups share one rate — pooling is the null doing its own arithmetic. (No pooling for the two-sample t: no shared-σ claim there in our course.)
"What's the df formula for two-sample t?" Technology reports it (course convention). Read df off the printout; never hand-compute it here.
Runs a two-sample t on before/after data. The tell: can each value in one group be matched to exactly one in the other? Same people twice → paired t on the differences (Week 14).
"p = 0.078, so the groups are equal." Fail to reject ≠ accept. Licensed sentence: "not convincing evidence of a difference." The lab shows real differences going undetected — that's Type II, not proof of equality.
"p < 0.001 — huge effect!" p measures surprise under H₀, not size. Report the difference in real units beside it; giant samples make microscopic gaps "significant."
"Team A is significantly higher, so Team A is better at the job." Only random assignment buys causal sentences. A/B test: yes. Observational team comparison: the rates differ — the why is unproven.
"My z = 2.48 isn't on the friendly table." By design — the table teaches the logic; printouts and NORM.S.DIST handle any z. Off-table values are always supplied.

Scope flag

This outline stays within Objective 7 at the course profile's stated depth: two-sample SE/z/p arrive on printouts (students hand-compute only p̂'s and the pooled proportion), and two-sample df are technology-reported — the Welch formula is deliberately omitted. The informal power moment (the lab's 1-in-5 miss) and the A/B-testing causal-license framing are added context beyond the objective's letter — kept because they cure the week's two worst misconceptions ("no evidence = no difference" and "significant = better"); cut them for a leaner session. Confidence intervals for a difference are mentioned in the chapter in one line only (intervals were Weeks 11–12; tests are this week's job).