Week 15 · Chapter

Chapter 15 — Testing Proportions & Two-Sample Inference

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
The module's primary reading · ~30–40 minutes · Objective 7 · Read this before (or alongside) the week's lecture — everything else in the module builds on it.

Why you already care about this

Sometime this week, an app almost certainly ran an experiment on you. A shopping site showed you one version of its checkout page and showed the next visitor another. A streaming service tried two different thumbnails on two halves of its audience. You were in group A or group B, a counter ticked somewhere, and nobody asked. The tech industry calls this an A/B test, runs millions of them a year, and decides what half the internet looks like based on the results.

Here is the good news: after fourteen weeks, you already own every part it's built from. Week 1 taught you why random assignment matters. Week 10 taught you that samples wobble. Week 13 gave you the courtroom logic of hypothesis testing, and Week 14 aimed that logic at means. This week closes the two remaining gaps — testing a claim about a percentage, and deciding whether two groups genuinely differ — and then hands you the map for choosing the right tool on sight.

By the end of this chapter you will be able to: run a complete one-proportion z-test from hypotheses to conclusion; explain why the two-proportion test pools its samples, and read a two-proportion printout correctly; read a two-sample t printout (with its technology-reported degrees of freedom) and say exactly what it does and does not license you to conclude; and, given any study description, name the right procedure — one-prop z, two-prop z, one-sample t, two-sample t, or paired t — in about ten seconds.

One line to carry through the week: one sample asks "is the claim true?" — two samples ask "are these the same?" — and most real questions are the second kind.

Where we are

Weeks 11–12 built confidence intervals (the "plausible values" tool); Week 13 built the logic of significance testing — null and alternative hypotheses, p-values, α, Type I and II errors; Week 14 ran that logic on means with the t machinery, including paired data. This week the same four-step skeleton gets its last two everyday attachments: proportions, and comparisons of two independent groups. Nothing here is brand new — it's assembly.

1. The one-proportion z-test

Start with the plain idea. Somebody publishes a percentage — "50% of seeds sprout," "90% of repairs finish same-day" — and you have one random sample to check it against. A claim about a percentage is a claim about a population proportion p, so the hypotheses are about p:

H₀: p = p₀ — the claimed value is right; any gap between it and your sample is chance.
Hₐ: p ≠ p₀ (two-sided), or p > p₀ / p < p₀ if only one direction matters and you chose it before seeing data.

Here p₀ (read "p-naught") is the hypothesized proportion — the number the claim asserts. Notice what is not in the hypotheses: p̂. Hypotheses indict the population; the sample proportion is the evidence, never the suspect.

The test statistic asks the question every z has asked since Week 8 — how many standard errors from expected did the data land?

z = (p̂ − p₀) ⁄ SE, where SE = √( p₀(1 − p₀) ⁄ n )

Look closely at that SE: it is built from p₀, not from p̂. The test's whole logic runs inside H₀'s world — "assume the claim, measure the surprise" — so the claim supplies its own yardstick. This is the one place this week genuinely differs from Week 12, where the confidence interval used p̂ in its standard error. The interval assumes nothing, so it leans on the data; the test assumes the claim, so the claim does the arithmetic. Memory hook: interval → p̂; test → p₀.

The p-value then comes from the normal curve, using the same friendly z-table you met in Week 8 (areas to the LEFT):

The table below re-prints the course's friendly z-table of left-tail areas.

z	area to the left	z	area to the left
−2.5	0.0062	0.5	0.6915
−2	0.0228	1	0.8413
−1.5	0.0668	1.25	0.8944
−1.25	0.1056	1.5	0.9332
−1	0.1587	2	0.9772
−0.5	0.3085	2.5	0.9938

(z = 0 sits at 0.5000; symmetry pairs each negative row with a positive one. Any z not on this table, technology supplies.)

Worked example. A wildflower seed packet claims "50% of seeds sprout." A community garden plants a random 100 seeds from a large seed lot, and 60 sprout. Test the claim at α = 0.05.

Hypotheses. H₀: p = 0.50 · Hₐ: p ≠ 0.50 (we'd care about a miss in either direction).
Sample proportion. p̂ = 60 ⁄ 100 = 0.60.
Standard error under H₀. SE = √(0.50 × 0.50 ⁄ 100) = √0.0025 = 0.05.
Test statistic. z = (0.60 − 0.50) ⁄ 0.05 = 2.0. The sample landed two standard errors above the claim.
p-value. Two-sided, so both tails: the area beyond z = 2 on the right is 1 − 0.9772 = 0.0228; doubled, 2 × 0.0228 = 0.0456 ≈ 0.046.
Conclusion, in words. 0.046 < 0.05, so reject H₀: if the packet's 50% were true, a sample this far out would occur only about 5 times in 100 — convincing evidence that the true sprouting rate isn't 50%, and our sample points higher.

Misconception check. "H₀: p̂ = 0.50." No — hypotheses are always about the population. If your hypothesis is wearing a hat, take the hat off. And the same discipline applies inside the SE: a test's standard error uses p₀. On close calls, the wrong SE flips decisions.

2. Check before you z: the conditions

Every z-based proportion method leans on the normal shape of p̂'s sampling distribution (Week 10), and that shape has entry requirements. Before running a one-proportion z-test, check:

Random. The data come from a random sample (or random assignment). Week 1 never stops mattering — no test can repair a biased sample.
Large counts, in H₀'s world: np₀ ≥ 10 and n(1 − p₀) ≥ 10 — at least ten expected successes and ten expected failures, computed with the hypothesized proportion. (Seed packet: 100 × 0.5 = 50 and 50 ✓.)
The 10% condition. When sampling without replacement, the sample should be under 10% of the population (100 seeds from a warehouse-sized lot ✓).

The large-count condition is about expected counts, not raw sample size. A claim of p₀ = 0.02 checked with n = 100 expects only 2 successes — the test doesn't qualify, no matter how "big" 100 feels. Knowing a test doesn't qualify is a graded skill in this course; the honest cures are a larger sample or the exact small-count methods technology offers (beyond our scope).

3. Two samples: the two-proportion z-test

Now the question most real studies actually ask — not "is it 50%?" but "are these two rates the same?" Two support teams. Two seed treatments. Two versions of a page. The design: two independent random samples, one from each group, with a success count in each.

H₀: p₁ = p₂ — the two populations share one rate; the gap we observed is chance wobble.
Hₐ: p₁ ≠ p₂ (or one-sided, declared in advance).
The evidence is the gap between sample proportions, p̂₁ − p̂₂.

Here is the one genuinely new idea of the week, and it is the null hypothesis doing arithmetic. If H₀ is true, both groups share one common rate — so the best estimate of that one rate should use everyone:

p̂_pool = (successes in both groups combined) ⁄ (n₁ + n₂)

The test's standard error is built from p̂_pool. Pooling isn't a computational nicety — it's what "assume H₀" looks like for two proportions. (Why not just average the two p̂'s? With equal sample sizes the two recipes agree, but if one sample were 300 and the other 50, the bigger sample deserves more say — pooling by combined counts grants it automatically.)

Course depth convention: you compute p̂₁, p̂₂, and p̂_pool by hand; the SE, z, and p-value arrive on a printout (or from your spreadsheet), and your job is to read it like a working statistician. The chapter shows the formula once — SE = √( p̂_pool(1 − p̂_pool)(1/n₁ + 1/n₂) ) — so the printout is never magic. Conditions: both samples random and independent of each other, with at least 10 successes and 10 failures in each sample (for two-sample tests we check the observed counts).

Worked example — interpret the output. An online retailer samples this month's rated support tickets independently from its two teams: Team A, 90 of 120 customers satisfied (p̂₁ = 0.75); Team B, 72 of 120 satisfied (p̂₂ = 0.60). Are the true satisfaction rates different?

Counts check: successes/failures are 90/30 and 72/48 — all at least 10 ✓.
Pooled proportion: (90 + 72) ⁄ (120 + 120) = 162 ⁄ 240 = 0.675.
Technology output: difference = 0.15 · SE (pooled) = 0.0605 · z = 2.48 · two-sided p = 0.013
Read it: 0.013 < 0.05 → reject H₀ — convincing evidence the teams' true satisfaction rates differ, with Team A's higher.
The sentence that separates A-students: nobody randomly assigned tickets to teams. Team A may handle easier products or calmer hours. The test established that the rates differ; the design decides whether anyone may say why. Correlation is still a handshake, not a push (Week 1).

In practice (your tools). The printout is four spreadsheet cells. With the counts in cells: pooled rate =(90+72)/240 → 0.675; standard error =SQRT(pool*(1-pool)*(1/120+1/120)) → 0.0605; z =(0.75-0.60)/SE → 2.48; p-value =2*(1-NORM.S.DIST(ABS(z),TRUE)) → 0.013. NORM.S.DIST(z, TRUE) is the friendly z-table on demand — the left-tail area for any z. Google Sheets and Excel are identical here. (A confidence interval for the difference p₁ − p₂ also exists and appears on the same printouts; this course reads tests this week and leaves that interval to technology.)

Misconception check. "The two sample proportions are different, so the populations differ." Every pair of honest samples differs somewhat — that is Week 10's sampling variability, not news. The test asks whether the observed gap is larger than chance's usual wobble. Respect the p-value, in both directions.

4. Two means: the two-sample t-test

Swap "success rate" for "average" and the story repeats with t in place of z. Two independent groups, a measured amount in each — dollars, minutes, grams:

H₀: μ₁ = μ₂ · Hₐ: μ₁ ≠ μ₂ (or one-sided, declared in advance).
t = (x̄₁ − x̄₂) ⁄ SE, with SE = √( s₁²⁄n₁ + s₂²⁄n₂ ) — each group contributes its own uncertainty.
No pooling here. Pooling was the proportions move, where H₀ itself implied one shared rate for the SE. Our two-sample t lets each group keep its own spread.
Degrees of freedom: technology reports them. The two-sample df formula is genuinely ugly; in this course, software computes it and you read it off the printout. That is a course convention, not a dodge — it is how working analysts do it.
Conditions: both samples random and independent; each group roughly normal or comfortably large (Week 11's shape rules, applied twice).

Worked example — interpret the output. A kitchen-supply site runs a true A/B test: visitors are randomly routed to a redesigned checkout page or the current one. Among completed orders: redesign — n = 40, x̄ = $63, s = $10; current — n = 40, x̄ = $57, s = $10.

The SE, once, so the printout isn't magic: √(10²⁄40 + 10²⁄40) = √(2.5 + 2.5) = √5 ≈ 2.24.
Technology output: t = 2.68 · df = 78 · two-sided p = 0.009
Read it: 0.009 < 0.05 → reject H₀ — convincing evidence that mean order value differs between the pages, about $6 higher on the redesign.
And the Week 1 payoff: visitors were randomly assigned, so this is an experiment — the sentence "the redesign caused the lift" is finally licensed. Put this beside the support teams: same arithmetic, different design, completely different conclusion rights. That contrast is the week in miniature.

In practice (your tools). For two-sample means your spreadsheet skips straight to the answer: =T.TEST(range1, range2, 2, 3) returns the two-sided p-value (the 2 means two-sided; the 3 selects the unequal-variance test this course uses), and any statistics add-on prints the t and df alongside it — that's where "df = 78" comes from.

Misconception check. "p = 0.009 means there's a 0.9% chance the pages are equal." The p-value is the probability of data this extreme assuming the pages are equal — a statement about the data under H₀, never the probability that H₀ is true. Week 13's law, still in force.

5. Paired or independent? Choosing the right procedure

Week 14 gave you the paired t — same individuals measured twice, analyze each individual's difference. The classic end-of-course error is running a two-sample t on paired data (or vice versa). The tell is one question: could I match each measurement in group 1 to exactly one measurement in group 2, for a reason? Yes → paired. No → independent.

That completes the toolbox, and the toolbox needs a map. Two questions pick the tool:

What kind of data? Counting successes — a percentage — points to z for proportions. Measuring amounts — a mean — points to t for means.
How many samples? One sample against a claimed value → one-proportion z (this week) or one-sample t (Week 14). Two independent samples against each other → two-proportion z or two-sample t (this week). The same individuals measured twice → paired t (Week 14).

The table below is the week's procedure map — the one to photograph.

Procedure	Use it when	Example
One-proportion z-test	one sample vs. a claimed percentage	the seed packet
Two-proportion z-test	success counts from two independent samples	the support teams
One-sample t-test (W14)	one sample's mean vs. a claimed value	a stated average weight
Two-sample t-test	means of two independent samples	the checkout A/B test
Paired t-test (W14)	the same individuals measured twice	before/after anything

Misconception check. "It has two columns of numbers, so it's a two-sample t." Two columns of the same people's before-and-after values are one sample of differences. Ask the matching question before touching any formula.

6. When tests mislead: three warnings worth the whole week

Fail to reject ≠ accept. A p-value of 0.078 does not prove two groups are equal — it says the evidence didn't clear the bar. The licensed sentence is "the data do not provide convincing evidence of a difference," never "the groups are the same." Absence of evidence is not evidence of absence.

Statistically significant ≠ important. With enormous samples, microscopic differences earn spectacular p-values. A two-million-user A/B test can make a 0.4-second average difference "highly significant" — and still worthless. The p-value answers "is it chance?"; only the size of the difference, in real units, answers "does it matter?" Report both, always.

Even real differences hide. When a difference truly exists, an honest sample can still miss it — that miss is Week 13's Type II error, and how often a test catches a real effect is called its power. Informally: bigger samples and bigger true differences mean more power. This week's data lab makes power personal: you'll simulate an A/B test where the truth is different by construction — and about one run in five will fail to find it anyway. A non-significant result is a shrug, not a verdict.

Key terms

One-proportion z-test — the significance test for a claim about a single population proportion.
Hypothesized proportion (p₀) — the claimed value of p asserted by H₀; it supplies the test's standard error and its large-count check.
Large-count condition (test form) — np₀ ≥ 10 and n(1 − p₀) ≥ 10: at least ten expected successes and failures under H₀.
Two-sample inference — procedures comparing a parameter across two independent groups.
Two-proportion z-test — the test of H₀: p₁ = p₂ for success counts from two independent samples.
Pooled proportion (p̂_pool) — combined successes over combined sample sizes; the single-rate estimate H₀ implies, used in the two-proportion SE.
Two-sample t-test — the test of H₀: μ₁ = μ₂ for means of two independent samples; SE = √(s₁²/n₁ + s₂²/n₂).
Technology-reported degrees of freedom — this course's convention for the two-sample t: software computes df; you read it off the printout.
Independent samples — no measurement in one group is matched to any measurement in the other.
Paired data — the same individuals measured twice; analyzed as one sample of differences (Week 14).
Statistical vs. practical significance — "unlikely to be chance" vs. "big enough to matter"; large samples can produce the first without the second.
Power (informal) — how often a test detects a difference that really exists; the flip side of Type II error.
A/B test — a randomized experiment on two versions of something; random assignment gives its conclusions causal license.

Summary

A claim about a percentage gets the one-proportion z-test: H₀: p = p₀; z = (p̂ − p₀)/SE with SE built from p₀; p-value from the z-table. Interval → p̂; test → p₀.
Conditions live in H₀'s world: random sample, np₀ ≥ 10 and n(1 − p₀) ≥ 10, sample under 10% of the population.
Two groups' rates get the two-proportion z-test: H₀: p₁ = p₂; pool the samples because the null itself claims one shared rate; read SE, z, and p off the printout.
Two groups' means get the two-sample t-test: no pooling, each group keeps its own spread; df are technology-reported; read the printout.
Paired ≠ independent: same individuals twice → paired t on the differences. The map: proportions vs. means, then one sample / two samples / same-twice.
Three warnings: fail to reject ≠ accept; significant ≠ important; and even real differences hide — power is how often you catch them.
Causal sentences still require random assignment: the A/B test earns them, the observational comparison never does.

Check your understanding

Work these before looking at the answers — they're ungraded, and the answers (with reasoning) are at the bottom of the page.

A phone-repair shop advertises that 90% of screen repairs are finished the same day. A consumer group checks a random 225 recent repairs and finds 198 finished same-day. Using α = 0.05, run the full test: hypotheses, the large-count check, z, the two-sided p-value (from the friendly z-table), and a one-sentence conclusion in context.
Two car-wash locations track return customers. Independent random samples: North — 96 of 150 customers returned within a month (p̂ = 0.64); South — 108 of 150 returned (p̂ = 0.72). Technology reports: pooled p̂ = 0.68 · z = 1.49 · two-sided p = 0.137. State the hypotheses, the conclusion at α = 0.05, and — carefully — what this result does not prove.
A makerspace compares two 3-D printers on the same part file, printing independent batches: Printer A — n = 16 prints, x̄ = 44 min, s = 8; Printer B — n = 16, x̄ = 38, s = 8. Technology reports: t = 2.12 · df = 30 · two-sided p = 0.042. Which procedure is this, what is the conclusion at α = 0.05, and why does the printout — not you — supply the df?
A friend reads that a study comparing two stretching routines found "no significant difference (p = 0.21)" and announces the routines are "proven identical." In two or three sentences, correct them — and name the additional concept from this chapter that explains how a real difference could have been missed.

Answers (worked reasoning — read after attempting)

H₀: p = 0.90 · Hₐ: p ≠ 0.90. Large counts under H₀: np₀ = 225 × 0.9 = 202.5 and n(1 − p₀) = 22.5 — both ≥ 10 ✓ (and 225 repairs is under 10% of all repairs). p̂ = 198 ⁄ 225 = 0.88. SE = √(0.90 × 0.10 ⁄ 225) = √0.0004 = 0.02. z = (0.88 − 0.90) ⁄ 0.02 = −1.0. Two-sided p-value = 2 × 0.1587 = 0.3174 ≈ 0.317. Since 0.317 > 0.05, fail to reject H₀: a sample rate of 88% is entirely consistent with a true 90% same-day rate — no convincing evidence against the ad. (Not proof the ad is right — just no evidence it's wrong.)
H₀: p_N = p_S · Hₐ: p_N ≠ p_S. With p = 0.137 > 0.05, fail to reject H₀: the data don't provide convincing evidence that the two locations' true return rates differ. What it does not prove: that the rates are equal. The 8-point sample gap is within chance's usual wobble for samples this size — the difference could be zero, or it could be real and modest; this test simply didn't have the evidence to tell.
This is a two-sample t-test — a measured amount (minutes), two independent groups (separate batches, not matched prints). Since p = 0.042 < 0.05, reject H₀: convincing evidence the printers' true mean print times differ for this part file, with Printer B faster by about 6 minutes on average. The df comes from the printout because the two-sample df formula is messy and this course's convention — like working practice — is that technology computes it and you read it.
"No significant difference" means the evidence didn't clear the bar — fail to reject is not accept, so nothing was "proven identical"; the honest sentence is that the study found no convincing evidence of a difference. And a real difference could easily have been missed if the study had low power (small samples, modest true effect): a miss like that is a Type II error, not a demonstration of equality.

Next up in this module: the slide deck if you want the lecture's visual walk-through, then the Readings & Resources page for optional deeper dives — and when you're ready to practice, Lecture Tutorial 15 turns your chatbot into your personal Week 15 tutor.