Lecture: Linear Regression & Inference for the Slope
Lecture script · about 15–20 minutes spoken · Week 15
Opening
Think about the last time an app gave you a number before anything happened. A ride-share fare before the ride. A delivery time before the food left the kitchen. Somebody measured one thing they could see and used it to guess another thing you cared about. That guess came from a line, or something very like one. In this lecture I want to show you how that line is built, how to read it, and when not to trust it.
You already have most of the pieces. In Week 4 you drew a scatterplot of 342 penguins, flipper length across the bottom and body mass up the side, and you found a strong, positive, straight-line pattern with r = 0.8712. In Weeks 11 to 14 you learned to build intervals and run tests. In this lecture those two halves meet. We'll draw a line through the penguin cloud, use it to predict, measure how far it misses, and then ask whether its tilt is real or a fluke of one sample.
This is the last module of new material. After it comes finals week, where you'll use everything from the course at once.
1. The least-squares line
Picture that scatterplot again, and imagine laying a ruler across it. You could tilt the ruler a lot of ways. To pick the best one, look at each dot and measure how far it sits above or below the ruler. Some misses are up, some are down, so square each one to make it positive. Then add them all up. The best ruler is the one that makes that total as small as it can be.
That line has a name. The least-squares regression line is the straight line ŷ = b₀ + b₁x that makes the sum of the squared residuals as small as possible. The little hat on the y matters. Plain y is a bird's real, measured mass. The predicted value, written ŷ and read "y hat", is the value the line gives for a chosen x. It's an estimate, not an observation.
For the penguins, x is flipper length in millimetres and y is body mass in grams. Software reports the line as ŷ = −5,780.8314 + 49.6856x. That first number is where the line starts, and the second is how steeply it climbs. One more fact is worth holding on to: the line always runs through the point of averages. Plug in the average flipper, 200.9152 millimetres, and the line gives about 4,201.76 grams, which is the average mass up to rounding.
Pause and try this. A friend says the best line is the one that passes through the most dots. What's wrong with that?
Think about 342 scattered birds for a moment.
Here's the answer. Almost no line touches more than a handful of dots, so counting hits ignores most of the birds. Least squares judges a line by all of its misses at once.
2. Reading the slope
Now the tilt. In plain words, the slope answers one question: if x goes up by one, how much does the prediction go up? The slope is the number b₁ that gives the predicted change in y for each one-unit increase in x.
For the penguins, that's about 49.69 grams per millimetre. Say it as a sentence, because the sentence is what matters: each additional millimetre of flipper length predicts about 49.69 grams more body mass. Ten more millimetres predict about 496.86 grams more, roughly half a kilogram.
Here's where people trip. The slope and the correlation describe the same cloud, so they feel like the same thing. They aren't. You can rebuild the slope from r by multiplying r by the spread of y and dividing by the spread of x. With r = 0.8712, a body-mass spread of 801.9545 grams, and a flipper spread of 14.0617 millimetres, you land on about 49.69. That last step is what gives the slope its units. The correlation has no units and never leaves the range from −1 to 1. The slope has units and can be any size.
Pause and try this. If you measured every flipper in centimetres instead of millimetres, which number would change: the slope, or r?
Think about which one carries units.
Here's the answer. The slope would change, and r would not. The slope is in grams per unit of flipper length, so a bigger unit makes it ten times bigger, while r stays 0.8712 because it has no units.
3. Reading the intercept, then distrusting it
Every line has to start somewhere. The intercept is the number b₀ that gives the predicted y when x is 0. It anchors the line.
Sometimes it means something. If x were hours of practice, a student who didn't practice at all is a real person. For the penguins, though, the intercept is a warning. Set the flipper length to 0 and the line predicts −5,780.83 grams. A negative mass is impossible. And no penguin in the data had a flipper anywhere near 0; the shortest was 172 millimetres.
So the honest reading is this: the intercept sets the line's height so it fits the birds we have. It has no physical meaning here.
Pause and try this. Before you interpret any intercept, what one question should you ask?
Give it a few seconds.
Here's the answer. Is x = 0 inside, or at least near, the data? If it isn't, the intercept has no meaning in context, and you say so.
4. Predicting inside the data
Now let's use the line for its real job. A field team measures a flipper of 200 millimetres. What mass should they expect? Multiply 49.6856 by 200 and you get 9,937.12. Add the intercept, which means taking away 5,780.8314, and you get a predicted mass of 4,156.29 grams.
Before you trust that, check where 200 sits. The flippers in the data ran from 172 to 231 millimetres, so 200 is comfortably inside. That makes this interpolation: using a regression line to predict y at an x value inside the range of the x values the line was built from.
A second prediction shows the slope at work. At 210 millimetres the line predicts about 4,653.14 grams. Ten more millimetres, and the prediction climbed by almost exactly the 496.86 grams the slope promised.
Pause and try this. Is 4,156.29 grams what a penguin with a 200 millimetre flipper weighs?
Careful with the wording here.
Here's the answer. No. It's the line's estimate of the average mass for birds with that flipper. Real birds scatter above and below it, which is the next idea.
5. Residuals and the typical miss
A prediction is a guess, so it misses. Think of a weather app that called for a mild afternoon and got a warm one. The miss has a direction and a size. In regression, that miss has a name. A residual is the observed value of the response minus its predicted value, y − ŷ, for one individual. Observed minus predicted, always in that order. Then a positive residual means the point sits above the line, and a negative one means it sits below.
Let's try two hypothetical birds, teaching cases rather than rows of the data. Suppose a team weighs a new penguin with a 200 millimetre flipper at 4,500 grams. Its residual is 4,500 minus 4,156.29, which is 343.71 grams. That bird is about 344 grams heavier than the line predicts. Now suppose a bird with a 210 millimetre flipper weighs 4,400 grams. Its residual is −253.14 grams, about 253 grams lighter than predicted.
One bird is one story. For the whole cloud you want the residual standard error, the typical size of a residual in the units of y. For the penguins it's about 394.28 grams. Compare that with guessing every bird at the overall average, which misses by about 801.95 grams, the standard deviation of body mass. Knowing the flipper cuts the typical miss roughly in half.
Pause and try this. A bird's residual is negative. Is it heavier or lighter than the line predicts?
Say it to yourself as observed minus predicted.
Here's the answer. Lighter. A negative residual means the observed mass is smaller than the predicted mass, so the bird sits below the line.
6. What r² measures
Body masses vary a lot from bird to bird. Some of that variation lines up with flipper length. Some of it doesn't. The coefficient of determination, written r², is the square of the correlation coefficient. It gives the share of the variation in the response that the least-squares line on x accounts for.
For the penguins, square 0.8712 and you get about 0.76; software reports 0.759. So flipper length accounts for about 75.9% of the variation in body mass, and about 24.1% is left to other things the line doesn't see.
Here's the classic wrong turn. People read 75.9% as a share of birds, as if three out of four penguins sat right on the line. Almost none of them do. It's a share of variation, not a share of points.
Pause and try this. Finish the sentence correctly: "An r² of 0.759 means that flipper length…"
Try the whole sentence in your head.
Here's the answer. "…accounts for about 75.9% of the variation in body mass." Never "75.9% of the birds are on the line."
7. Reading a residual plot
A line can score well on r² and still be the wrong shape. To check, you take the scatterplot and remove the line's tilt, so each dot shows only its miss. That's a residual plot: a scatterplot of the residuals against x, with a horizontal reference line at 0. Three patterns matter. A shapeless horizontal band says the straight line fits. A curve says the form is wrong. A fan that widens says the spread isn't equal.
Here's a clean example of the curve, from a single die. Score each face by its squared distance from 3.5. Faces 1 and 6 score 6.25, faces 2 and 5 score 2.25, and faces 3 and 4 score 0.25. In Week 4 you found r = 0 for these scores. So the least-squares line is flat, predicting about 2.92 for every face. The residuals run about 3.33, then −0.67, then −2.67, −2.67, −0.67, and back up to 3.33. Picture those six dots across the faces: high at both ends, low in the middle. That's a U.
Pause and try this. The die scores follow a perfect pattern, yet r is 0. What does the U in the residuals tell you?
Think about the shape, not the number.
Here's the answer. It tells you a straight line is the wrong form for these data. The pattern is curved, so the line misses it completely, whatever r² says.
8. Testing the slope
The 342 penguins are a sample. Another field season would catch other birds and give a slightly different line. So the slope wobbles, the way a sample mean does. The real question is about the population.
The population slope, β₁, read "beta one", is the slope of the population regression line, the parameter that a test or interval about a regression slope is about. The sample slope b₁ estimates it. Before any test, you check four conditions out loud. The form is linear, and Week 4's scatterplot showed that. The observations are independent, since each bird was measured separately; they come from a field study, not a random draw, so you treat them as representative of penguins like those monitored. The residuals should be roughly normal, and they should have about equal spread. You'll check those last two in the Data Lab.
Then the test. The null hypothesis says β₁ = 0: no linear relationship between flipper length and mean body mass. The alternative says β₁ isn't 0. The standard error of the slope is how much the sample slope typically wobbles from sample to sample; software reports it as 1.5184 grams per millimetre. The t statistic divides the slope by that standard error, and 49.6856 divided by 1.5184 is about 32.72. The degrees of freedom are 342 minus 2, which is 340, because the line estimates two numbers. The p-value is far below 0.001, so we reject the null hypothesis at the 0.05 level.
Pause and try this. Put the p-value into words. If flipper length had no linear relationship with body mass, what would a slope this far from 0 be?
Lean on Week 13's sentence.
Here's the answer. It would almost never happen. If the null were true, data this extreme would occur far less than 0.001 of the time.
9. An interval for the slope
A test asks whether there's a slope. An interval asks how big it is. The confidence interval for the slope has the same shape as every interval since Week 11: the estimate, plus or minus a critical value times a standard error.
The course table doesn't list t* for 340 degrees of freedom. But you know t* shrinks toward z* as the degrees of freedom grow, so it sits between 1.960 and 1.976, the value for 150 degrees of freedom. We use the safer 1.976. Multiply it by 1.5184 and the margin of error is about 3.00 grams per millimetre. So the interval runs from about 46.69 to 52.69 grams per millimetre. You're 95% confident that, for penguins like these, each extra millimetre of flipper goes with between about 46.69 and 52.69 grams more mean body mass. The interval leaves out 0, which matches the test.
Now one caution. A slope this strong tempts people to say longer flippers make heavier birds. The penguins were observed, not assigned their flippers, so this is association. And species may be lurking: Gentoo average about 5,076 grams against about 3,701 for Adelie, so one line across all three species can mix species differences into the slope.
Pause and try this. Does this interval prove that a longer flipper causes a heavier body?
Think about how the data were collected.
Here's the answer. No. It's observational data, so the slope shows an association. Only a randomized experiment supports a cause-and-effect claim.
10. Where the line stops
The penguin line was built from flippers between 172 and 231 millimetres. Inside that range it's been checked. Outside it, nobody knows. Plug in 300 millimetres and the arithmetic hands you 9,124.85 grams. But 300 is 69 millimetres beyond the longest flipper in the data. That's extrapolation: using a regression line to predict y at an x value outside the range of the observed x values. The data give no warrant that the straight-line pattern continues there. The intercept was the same trap at the other end.
The second danger sits inside the data. An influential point is a single point whose removal would change the regression line a lot. Use the calendar. Pair each month's position, 1 through 12, with its number of days. For all 12 months, the slope is about 0.07 days per month position, tilting slightly up. Now drop February, month 2, with its 28 days. The slope for the other 11 months is about −0.02. One point flipped the sign.
Pause and try this. Why does February pull so hard on the calendar line?
Picture where it sits on the plot.
Here's the answer. It sits low and near the left end, so it drags that end of the line down and tips the whole line upward. Take it away and the line goes nearly flat.
Closing
Let's put the week together, in the order we built it. The least-squares line makes the squared misses as small as possible, and it runs through the point of averages. The slope is the predicted change in y per unit of x, about 49.69 grams per millimetre, and it isn't the correlation. The intercept means nothing when x = 0 is far from the data. Predict inside the data, and call anything outside it extrapolation.
A residual is observed minus predicted, and the typical miss is about 394 grams. An r² of 0.759 is a share of variation, not a share of birds. A residual plot should look like a shapeless band. Check four conditions, then the t test for the slope gives about 32.72 on 340 degrees of freedom, and the interval runs from about 46.69 to 52.69. Watch for influential points, and say "associated with," not "causes."
Next in this module come the slide deck, the Readings and Resources page, the lecture tutorial, the practice set, and the quiz. Then comes finals week. Every question this course has asked, from where the data came from to whether a line can be trusted, comes together there.
Built with The Course Maker · samples.thecoursemaker.com