Lecture: Tests for Means & Proportions
Lecture script · about 15–20 minutes spoken · Week 14
Opening
A hospital quality team changes its check-in process and wants to know whether average waits really got shorter. A marketing analyst runs two versions of a web page and wants to know whether one truly wins a bigger share of clicks. A field biologist wants to show that two penguin species really differ in weight, and that the gap isn't the luck of one sample. All three are asking the same kind of question, and today you'll learn to answer it.
Here's where we are. Weeks 1 to 10 gave you description, probability, the normal model, and the standard error. Weeks 11 and 12 turned the standard error into confidence intervals. Last week, in Week 13, you met the logic of a hypothesis test on a guessing quiz: assume the boring explanation, then ask how surprising your data would be. Today that logic goes to work on means and proportions. Next week, in Week 15, you'll use the same logic on the slope of a line.
We'll go in nine steps. First, the one-sample t test, and why one ordinary bird can't vouch for a mean. Then two means, and how to read software output. Then one proportion, the most misread sentence in testing, and two proportions. We'll finish by joining tests to intervals and by choosing the right procedure.
1. The one-sample t test: count standard errors to the claim
Suppose someone tells you that Adelie penguins average 4,000 grams. That's a hypothetical claim, chosen to teach the test. You have 151 weighed birds, and their mean is 3,700.66 grams. The gap is about 299 grams. Is that gap real, or is it wobble?
Here's the idea in plain words. You measure the gap in standard errors, the typical wobble of a mean. A small count means the gap is ordinary wobble. A big count means the claim is hard to believe.
The claimed value is called the null value: the specific value of a parameter that the null hypothesis claims. For a mean we write it mu-naught. The test is the one-sample t test: a hypothesis test for a population mean when sigma is unknown. It takes the sample mean minus the null value and divides by the standard error. The result is the t statistic, the number of standard errors between the sample result and the null value.
Let's run it. The null hypothesis says mu equals 4,000 grams. The alternative says mu is not 4,000 grams, because a mean that's too high or too low would both count against the claim. Check the conditions out loud. The birds come from a field study, not a random draw, so we treat them as representative and say so. Each bird is weighed separately. And 151 is far above the rule of thumb of about 30.
The standard error is 37.32 grams. The gap, 3,700.66 minus 4,000, is minus 299.34 grams. Divide by 37.32 and t is about minus 8.02. The degrees of freedom are 151 minus 1, which is 150. The critical value for 95% with 150 degrees of freedom is 1.976. Our t is far past it, and the p-value is far below 0.001. We reject the null hypothesis.
Pause and try this. The t statistic is negative. Does the minus sign mean there's no evidence against the claim?
Think about what the sign is telling you.
Here's the answer. No. The sign only says the sample mean fell below 4,000 grams. For a two-sided test, distance is what counts, and 8.02 standard errors is a long way.
2. One bird versus the mean
Now a puzzle. Back in Week 8 you met a single Adelie penguin that weighed 4,000 grams. Its z-score was about 0.65. A little heavy, but ordinary. So how can a mean of 4,000 grams be so far out?
The key is the difference between two yardsticks. Single birds vary a lot: the standard deviation is 458.57 grams. Averages of 151 birds vary much less: the standard error is 37.32 grams. That's about a 12-fold difference. The same 299-gram gap is less than one standard deviation for one bird. For a mean, it's about 8 standard errors.
Think of a bathroom scale that wobbles a fair amount from one weighing to the next. One odd reading means nothing. But if you average many readings, the wobbles cancel out. If the average still sits far from the claim, something real is going on.
Pause and try this. Which yardstick belongs in the t statistic, the standard deviation of 458.57 grams or the standard error of 37.32 grams?
Picture what each one describes.
Here's the answer. The standard error, 37.32 grams. The test is about a mean, and the standard error describes how much means wobble. Using the standard deviation would make the gap look about 12 times smaller than it is.
3. Comparing two means: set up the two-sample t test
Gentoo penguins look bigger than Adelie penguins. But "look bigger" isn't evidence. The fair question is whether the gap between the two sample means is bigger than the wobble you'd expect if the two populations had the same mean.
Two groups bring a new requirement: independent samples, meaning the individuals in one group have no link to the individuals in the other. Gentoo and Adelie birds are different birds, weighed separately, so that holds. The two-sample t test compares the means of two independent groups. It divides the difference in sample means by the standard error of a difference in means, which is how much that difference typically wobbles from sample to sample. The null value is a difference of 0.
Here are the numbers. There are 123 Gentoo birds with a mean of 5,076.02 grams and a standard deviation of 504.12 grams. There are 151 Adelie birds with a mean of 3,700.66 grams and a standard deviation of 458.57 grams. From the unrounded means, the difference is 1,375.35 grams.
For the standard error, square each standard deviation and divide by its sample size. That gives about 2,066.15 for the Gentoo birds and 1,392.63 for the Adelie birds. Add them to get 3,458.78, and take the square root. The standard error of the difference is about 58.81 grams.
Pause and try this. To get the standard error of a difference, do the two groups' wobbles add or cancel?
Recall the order of the steps.
Here's the answer. They add. Each mean wobbles on its own, so the difference wobbles more than either one. You square, divide, add, and then take the square root.
4. Reading software output
In practice, you'll read a two-sample test from a spreadsheet or from software. The skill is knowing which number answers which question.
For the penguins, the output reports the difference, 1,375.35 grams, and its standard error, 58.81 grams. It reports Welch's t, which is 23.386: the difference divided by the standard error. It reports about 249.6 degrees of freedom. And it reports a p-value far below 0.001.
That 249.6 isn't a typo. Welch's degrees of freedom are the degrees of freedom for a two-sample procedure that doesn't assume the two population standard deviations are equal. Software computes them from both samples, and they often land on a decimal. In this course you read them from the output; you never compute them by hand. In a spreadsheet, the T.TEST function with the fourth input set to 3 gives you this Welch p-value.
So the verdict: reject the null hypothesis. A gap of about 23 standard errors can't be explained by sampling wobble. One honest caveat: this is an observational study. It shows the species differ in mean mass among birds like those studied. It doesn't say why.
Pause and try this. In that output, which number tells you how many standard errors the gap spans?
Scan the list in your head: difference, standard error, t, degrees of freedom, p-value.
Here's the answer. Welch's t, 23.386. The difference is in grams, the standard error is the yardstick, and t is the gap measured in that yardstick.
5. Testing one proportion
Switch from grams to categories. Of the 333 penguins with a recorded sex, 165 are female. That's a little under half. Is the gap from an even split real?
The one-proportion z test is a hypothesis test for a population proportion. It takes the sample share minus the null value and divides by a standard error built from the null value. Why the null value? Because a test assumes the null hypothesis is true, so the claim sets the wobble. Week 12's interval used the sample share instead, because an interval assumes nothing.
The null hypothesis says the share is 0.5, and the alternative says it isn't. Check the conditions: under the null, you'd expect 166.5 females and 166.5 males, both at least 10. The sample share is 165 out of 333, about 0.4955. The null standard error is about 0.0274. Put them together and z is about minus 0.164. The sample share sits about 0.16 standard errors below one half.
The two-sided p-value is about 0.87. Here's how to say that out loud: if the true split were even, a sample share at least this far from 0.5 would happen about 87% of the time. That's far above 0.05, so we fail to reject the null hypothesis. The data are consistent with an even split.
Pause and try this. In this test's standard error, do you use the sample share, 0.4955, or the null value, 0.5?
Think about what a test assumes.
Here's the answer. The null value, 0.5. A test assumes the null hypothesis is true, so the wobble comes from the claimed share.
6. "Not significant" is not "no difference"
Here's the sentence that trips people up. The sex test failed to reject. It's tempting to say, "So the test proved half the penguins are female." That sentence is wrong.
Go back to Week 12's interval for the same 333 birds. It runs from 0.4418 to 0.5492. Values like 0.47 and 0.52 sit inside it, right next to 0.5. So the data are consistent with an even split, and they're also consistent with a population that's 47% female, or 52%. Failing to reject leaves a whole range of values standing.
Think of a smoke alarm that stays quiet. Silence means no smoke was detected. It doesn't prove the air is perfectly clean. A little smoke might sit below what the alarm can sense. A test works the same way. A quiet result means the data didn't detect a difference. And with a small sample, the alarm is even less sensitive.
Pause and try this. Name one value other than 0.5 that these data can't rule out.
Use the interval's two ends.
Here's the answer. Any value between 0.4418 and 0.5492, such as 0.47 or 0.52. That's why "fail to reject" never means "accept".
7. Comparing two proportions
The last test compares two shares. The study worked on several islands. On Biscoe, 44 of 168 penguins are Adelie, about 26.2%. On Dream, 56 of 124 are Adelie, about 45.2%. That's a gap of 19.0 percentage points. Real, or wobble?
The two-proportion z test compares the proportions of two independent groups. It divides the difference by a standard error built from the pooled proportion: the share of successes when the two samples are combined. Why pool? Because the null hypothesis says the two shares are equal. If they're equal, your best estimate of that one shared value uses every bird from both islands.
So you add the successes and add the sample sizes: 100 Adelie birds out of 292, about 34.2%. Check the conditions: the counts 44, 124, 56, and 68 are all at least 10. The standard error comes out to about 5.62 percentage points. Divide the gap of minus 19.0 points by 5.62, and z is about minus 3.38.
Beyond 3 standard errors on both sides, a normal model leaves only about 0.27% of its area. So the p-value is below 0.003, and we reject the null at the 0.05 level and at the 0.01 level. One caveat: the team chose which colonies to study, so this compares the monitored colonies, not the islands as a whole.
Pause and try this. To pool the shares, should you average 26.2% and 45.2%?
Think about how many birds came from each island.
Here's the answer. No. The islands have different sample sizes, so you add the counts: 44 plus 56 successes out of 168 plus 124 birds. Pool the counts, not the percentages.
8. Tests and intervals: two views of one question
You've now seen the same pattern four times. A test counts standard errors from a claimed value. An interval reaches a set number of standard errors around the estimate. They're built from the same pieces, so they agree. That's the duality of tests and intervals: a two-sided test at level alpha rejects a null value exactly when that value lies outside the interval at level 1 minus alpha.
Check it on the Adelie mean. The 95% interval runs from 3,626.92 to 3,774.40 grams. The claim of 4,000 grams sits outside it, and the test rejected. Now test a claim of 3,750 grams. The t statistic is about minus 1.32, inside 1.976, so we fail to reject. And 3,750 sits inside the interval.
Intervals do one more job that a p-value never can: they show size. Using 1.976 as a safe critical value, the Gentoo gap runs from about 1,259.14 to 1,491.56 grams. Even the low end is about a third of a typical Adelie's mass. That difference is statistically significant and practically large.
Now a poll. Suppose 1,000 adults are polled and 54% support a proposal. The z statistic against an even split is about 2.53. At the 0.05 level, that's beyond 1.960, so we reject, and the 95% interval, 50.9% to 57.1%, leaves out 50%. At the 0.01 level, 2.53 is inside 2.576, so we fail to reject, and the 99% interval, 49.93% to 58.07%, includes 50%.
Pause and try this. Without any new arithmetic, would a two-sided test at the 0.05 level reject a Gentoo-minus-Adelie difference of 0 grams?
Ask where 0 sits.
Here's the answer. Yes. Zero lies far outside the interval from 1,259.14 to 1,491.56 grams, so the matching test rejects it, as the software did.
9. Choosing the right procedure
Four tests can feel like four recipes. They're really one recipe with two switches.
The first switch is the type of variable. A quantitative variable, like a mass in grams, calls for a mean. A categorical variable, like female or male, calls for a proportion. The second switch is the number of groups: one group against a claimed value, or two independent groups against each other. A mean with one group is a one-sample t test. A mean with two groups is a two-sample t test. A proportion with one group is a one-proportion z test. A proportion with two groups is a two-proportion z test.
Then the conditions decide whether you're allowed to use it. Remember last week's guessing quiz. With 10 questions, the expected number correct is only 2.5, below 10, so a z test doesn't fit, and you use the binomial instead. With 100 questions, the expected counts are 25 and 75, so the z test fits.
Pause and try this. Is the share of female penguins different from one half? Which of the four procedures answers that?
Flip the two switches.
Here's the answer. A one-proportion z test. The variable is categorical, and there's one group compared with a claimed value.
Closing
Let's put the week together. A one-sample t test counts standard errors from a claimed mean, and the Adelie claim of 4,000 grams fell about 8 of them away. One ordinary bird can't vouch for a mean, because means wobble far less than birds. A two-sample t test compares two means, and you read Welch's t and its degrees of freedom from the output.
A one-proportion z test puts the claimed share inside the standard error, and the penguin sex split came out consistent with even. "Not significant" never means "no difference". A two-proportion z test pools the counts, and the Adelie share differs between Biscoe and Dream. A two-sided test and its matching interval always agree, and the interval tells you how big. To choose, check the variable type, then the number of groups, then the conditions.
Next in this module come the slide deck, the Readings and Resources page, the lecture tutorial, the practice set, and the quiz. And next week, in Week 15, you'll fit a line through the penguin data and test whether its slope is real.
Built with The Course Maker · samples.thecoursemaker.com