← Back to the Introduction to Statistics outline The Course Maker
← Introduction to Statistics outline
Week 17 · Chapter

Chapter 17 — Linear Regression with Inference + Course Synthesis

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
The module's primary reading · ~30–40 minutes · Objective 8 (with a synthesis of Objectives 6–9) · Read this before (or alongside) the week's lecture — everything else in the module builds on it.

Why you already care about this

Your energy bill arrives and it's ugly, and you mutter the explanation before you've finished opening the envelope: it was a cold month. Congratulations — you just ran a regression in your head. You used one number you know (temperature) to explain another number you care about (energy use). People do this constantly: more experience should mean higher pay, more practice should mean better performance, a bigger engine should burn more fuel. The hunch is free. What statistics adds is the machinery: a line that turns the hunch into an actual prediction, a number (r²) that says how much of the story the line captures, and — because this is the inference half of the course — a test that says whether the line's slope is real or just a lucky tilt.

This is the last week of new material, and it was saved for last on purpose, because regression is where the whole course shakes hands with itself. The line comes from Week 4's scatterplots and correlation. The idea that a sample slope wobbles around a true slope comes from Week 10. The test and the interval run on the same t-machinery as Weeks 11 and 14. And the week ends by zooming out: with every tool now on the table, the final skill is choosing the right one — which is exactly what next week's cumulative final (a low-stakes checkpoint, like the midterm) rewards.

By the end of this chapter you will be able to: interpret the slope and intercept of a least-squares line in plain English; read r² as a share of variation explained and a residual as an actual-minus-predicted miss; use a residual plot to judge the model and refuse to extrapolate beyond the data; run the t-test and confidence interval for the slope straight off computer output; and, handed any scenario from the second half of the course, name the procedure it calls for.

Where we are

Weeks 1–4 taught you to describe data — including, in Week 4, two-variable data: scatterplots, and the correlation r that measures the strength and direction of a linear relationship. Weeks 5–8 taught you how chance behaves. Weeks 10–16 built the inference toolkit: confidence intervals for means and proportions, hypothesis tests for means and proportions, and chi-square tests for categorical tables. This week — Week 17, the last instructional week — the describing half and the inference half meet: we fit the line Week 4 only gestured at, and then we put its slope on trial. One reminder worth carrying in: Week 4's law — correlation is a handshake, not a push — is not repealed by anything in this chapter.

1. The least-squares line: turning a cloud into a prediction

Start with the plain idea. A scatterplot with a roughly linear pattern can be summarized by one straight line, written

ŷ = a + bx

where ŷ ("y-hat") is the predicted value of y for a given x. The hat means predicted, not actual — the same hat, with the same warning, that p̂ wore in Week 12. In this equation, x is the explanatory variable (the one you know, the one doing the predicting) and y is the response variable (the one you're predicting). That choice is not decorative: swap x and y and the slope, intercept, and every prediction change. Decide what predicts what before you fit anything.

Many lines could be drawn through a cloud of points. The least-squares line is the one that makes the sum of the squared vertical misses as small as possible — call it the line of least regret. Its two numbers are the whole story:

  • The slope b is the predicted change in y for each one-unit increase in x.
  • The intercept a is the predicted y when x = 0 — meaningful only when x = 0 makes sense and sits near the data.

You will not compute least-squares lines from raw sums in this course — technology does that. But two facts about the line are worth owning, because they let you build the line from summary statistics and they explain its behavior:

  1. b = r · (s_y ⁄ s_x) — the slope is the correlation times the ratio of the standard deviations. Correlation sets the direction; the SDs set the exchange rate.
  2. The line always passes through (x̄, ȳ) — the average day sits on the line — which gives the intercept: a = ȳ − b·x̄.

Worked example. A homeowner logs 30 heating-season days: x = the day's average outdoor temperature (in °C, ranging from 2 to 18), y = the home's daily energy use (in kWh). The summaries: x̄ = 10 °C, s_x = 4 °C; ȳ = 30 kWh, s_y = 8 kWh; and the correlation r = −0.9 (a strong negative linear relationship — colder days, more energy).

  • Slope: b = r · (s_y ⁄ s_x) = (−0.9) × (8 ÷ 4) = (−0.9) × 2 = −1.8 kWh per °C.
  • Intercept: a = ȳ − b·x̄ = 30 − (−1.8)(10) = 30 + 18 = 48 kWh.
  • The least-squares line: ŷ = 48 − 1.8x.

Now say the slope out loud, because the sentence is the skill. Every slope interpretation has four required parts — per one unit of x · predicted · on average · in context units: "Each additional degree of outdoor temperature predicts, on average, 1.8 kWh less energy use." Not "causes." Predicts, on average.

The intercept: at 0 °C the line predicts 48 kWh. Here 0 °C sits just below the coldest observed day (2 °C), so it's a reasonable near-data anchor. In other problems the intercept is nonsense — a 0-kg dog, a 0-mm flipper — and then it's just where the line happens to cross the axis, not a statement about the world.

One quick check that the machinery hangs together: predict the average day, x = 10. ŷ = 48 − 1.8(10) = 48 − 18 = 30 — which is exactly ȳ. The line really does pass through (x̄, ȳ).

Misconception check. "The intercept is always meaningful." No. The intercept is a prediction at x = 0, and x = 0 is often absurd or far outside the data. Interpret the intercept only when zero is a sensible, near-data value of x; otherwise call it what it is — the line's anchor point.

2. How good is the line? r², residuals, and the extrapolation trap

Fitting a line is easy; the discipline is asking how much it actually explains, where it misses, and where it has no right to speak.

r² — the share of variation explained. Square the correlation and it earns a new name: the coefficient of determination. r² is the fraction of the variation in y that the linear relationship with x accounts for. In the energy example, r = −0.9, so r² = (−0.9)² = 0.81: 81% of the day-to-day variation in energy use tracks temperature. The other 19% is everything else — laundry day, guests, thermostat fiddling. Memory hook: "r² is a share, not a grade."

Misconception check. "r² = 0.81 means the line's predictions are 81% accurate." No. r² is a share of variation explained, not an accuracy rate or a percent of correct predictions. Individual predictions still miss — the residuals, next, are exactly those misses. (And notice r² can't tell direction: 0.81 could come from r = +0.9 or r = −0.9. Report r when direction matters.)

Residuals — the misses, one point at a time. A residual is actual minus predicted: residual = y − ŷ. A point above the line has a positive residual (the line under-predicted); below the line, negative.

Worked example, continued. Predict a 5 °C day: ŷ = 48 − 1.8(5) = 48 − 9 = 39 kWh. One actual 5 °C day used 42 kWh. Its residual = 42 − 39 = +3 kWh — that home used 3 kWh more than the line predicted; the point sits above the line.

The residual plot — the model's X-ray. Plot the residuals against x and read it like a scan: random, patternless scatter around zero means the linear model is doing its job. A curve means the relationship isn't actually a line — the model is wrong in shape. A fan (spread growing along x) means predictions are less reliable at one end. A stray extreme residual is an outlier worth investigating. This plot is also where the inference conditions get checked in Section 3.

Extrapolation — where the line has no right to speak. The line was fitted on days from 2 to 18 °C. Ask it about a 35 °C summer day anyway: ŷ = 48 − 1.8(35) = 48 − 63 = −15 kWh. Negative energy use — the line predicts the power company pays you. Impossible. And the failure is instructive: at 35 °C the relationship the line captured doesn't just weaken, it reverses — air conditioning kicks in, and hot days raise energy use. Extrapolation — predicting outside the range of x used to fit the line — isn't "a bit less precise." It can be completely wrong, because the pattern itself can change out there. Memory hook, worth saying twice: "Inside the data, the line predicts. Outside the data, the line is fiction."

In practice (your tools). In Google Sheets or Excel, the whole fit is three cells: =SLOPE(y-range, x-range), =INTERCEPT(y-range, x-range), =RSQ(y-range, x-range) (and =CORREL(y-range, x-range) for r). The y-range comes first — swapping the arguments is the classic silent error, and the machine won't warn you. To see the fit: select both columns → Insert ▸ Chart ▸ Scatter → Customize ▸ Series ▸ Trendline, and check "Show R²." You'll do exactly this — on real penguins — in Data Lab 17.

3. Is the slope real? Inference for β

Here is the uncomfortable fact that makes this section necessary: two columns of pure random numbers will almost never produce a slope of exactly zero. Luck alone tilts lines. So a fitted slope b — even a plausible-looking one — always faces the question the second half of this course was built to answer: is this tilt more than luck?

The setup mirrors everything since Week 10. The fitted slope b came from one sample; behind it stands the population slope β — the true slope you'd get from the whole population. b varies from sample to sample, exactly as x̄ and p̂ did. And the skeptic's hypothesis is wonderfully concrete here:

H₀: β = 0 — the flat line. If β = 0, then x tells you nothing about y, and whatever tilt you fitted is noise. Hₐ: β ≠ 0 (two-sided is this course's default).

The test statistic is the same shape as every t you've computed:

t = (b − 0) ⁄ SE(b), with df = n − 2.

SE(b), the standard error of the slope, measures how much slopes like yours wobble sample to sample — in this course it is always supplied by technology output; you interpret it, never hand-compute it. The degrees of freedom drop by two because a fitted line spends two df — one for the slope, one for the intercept. (One-sample t procedures used n − 1; same table, different row rule.)

Conditions first — the LINE check. Inference borrows trust from assumptions, so check them: Linear pattern in the scatterplot · Independent observations · Normal-ish residuals (no wild skew or extreme outliers) · Equal spread of residuals across x (no fan). L and E are read straight off the residual plot; N off a quick histogram of the residuals. The mnemonic conveniently spells LINE.

The critical values come from the same course t-table you've used since Week 11 — only the df rule is new:

The table below gives the course's two-sided critical values t*.

df t* (90%) t* (95%) t* (99%)
9 1.833 2.262 3.250
15 1.753 2.131 2.947
24 1.711 2.064 2.797

Problems in this course land on these rows; for any other df, technology supplies the value.

Worked example — reading real output. An HR analyst at one company regresses annual salary (in thousands of dollars) on years of experience for n = 26 employees, and the software prints:

Predictor     Coef    SE Coef      T        P
Constant     41.50      2.30     18.04    0.000
Experience    2.50      1.25      2.00    0.057
R-Sq = 14.3%

Read it row by row. The fitted line is ŷ = 41.50 + 2.50x: each additional year of experience predicts, on average, 2.50 thousand dollars ($2,500) more salary. Now the trial of the slope:

  • t = Coef ÷ SE Coef = 2.50 ÷ 1.25 = 2.00 (always recompute this ratio as a habit — it's how you catch garbled output).
  • df = n − 2 = 26 − 2 = 24, so the 5% two-sided critical value is t* = 2.064.
  • |t| = 2.00 < 2.064 → fail to reject H₀ at the 5% level. The printed p-value agrees: P = 0.057 > 0.05.
  • Conclusion in words: "With 26 employees, this positive-looking slope is not statistically convincing at the 5% level — a flat line (β = 0) remains plausible."

Misconception check. "So the test proved experience doesn't matter at this company." It did no such thing — fail to reject ≠ accept, Week 13's law. The data are merely inconclusive: p = 0.057 sits achingly close to 0.05, and a slightly larger sample might well have tipped it. What you may say is that these data couldn't rule out the flat line. What you may not say is that the flat line is true.

The confidence interval — and the duality. The interval answers the better question: how big might β plausibly be? The recipe is the one you know: b ± t*·SE(b), df = n − 2.

For the salary slope, at 95%: 2.50 ± 2.064 × 1.25. The margin of error is 2.064 × 1.25 = 2.58, so the interval runs 2.50 − 2.58 = −0.08 up to 2.50 + 2.58 = 5.08: we are 95% confident that each year of experience is associated with between −$80 and +$5,080 in predicted salary. Notice the interval (barely) contains 0 — which is exactly why the test (barely) failed to reject at 5%. Interval contains 0 ⇔ two-sided test fails to reject at the matching α. One computation, two readings — the same duality that closed Week 14 closes the course.

One more reading habit: R-Sq = 14.3% is r² — experience explains a modest 14.3% of salary variation at this company (so even a real slope would be one storyline among many). The correlation itself is √0.143 ≈ 0.38.

Misconception check. "The slope is significant, so x causes y." A significant slope says the tilt is unlikely to be luck. It says nothing about who's pushing whom — a lurking third variable can drive both x and y, and the arithmetic never checks the arrow. Still a handshake, not a push. Causation needs random assignment (Week 1's law), not regression.

4. The whole toolkit: choosing the right procedure

You now own every tool this course teaches. The final skill — the one the cumulative final actually rewards — is choosing, and choosing is one question asked well: what shape is the answer?

  • The answer is one mean (an average) → one-sample t: an interval to estimate it, a test to judge a claimed value; df = n − 1. Paired before/after data → one-sample t on the differences.
  • The answer is one proportion (a yes/no percent) → one-proportion z: interval or test.
  • The answer compares two groups → two-sample t for means, two-proportion z for percents.
  • The answer is counts across categories → chi-square: goodness-of-fit (one categorical variable vs. claimed percentages) or independence (a two-way table).
  • The answer predicts one number from another → regression: the line, r², and the t-test / CI for the slope; df = n − 2.

Memory hook: "Mean, proportion, counts, or a line? Name the answer's shape, and the procedure names itself." Then confirm with two follow-ups: how many groups? and quantitative or categorical?

And the course in one breath, since this is the last chapter of new material. Weeks 1–4: describe — get honest data (who was measured, how they were picked) and summarize it. Weeks 5–8: chance — learn how randomness behaves so you can recognize its fingerprints. Weeks 10–17: infer — use one sample plus the logic of chance to say something disciplined about a world you didn't fully measure, always with a margin of humility attached. Every procedure in the map above is that one sentence wearing different clothes.

Key terms

  • Explanatory variable (x) — the variable used to predict; the response variable (y) is the one being predicted.
  • Least-squares line — the line ŷ = a + bx minimizing the sum of squared vertical misses.
  • ŷ (y-hat) — the line's predicted value of y at a given x; predicted, not actual.
  • Slope (b) — predicted change in y per one-unit increase in x; b = r·(s_y ⁄ s_x).
  • Intercept (a) — predicted y at x = 0; a = ȳ − b·x̄; meaningful only when x = 0 is sensible and near the data.
  • Coefficient of determination (r²) — the share of variation in y explained by the linear relationship with x.
  • Residual — actual − predicted (y − ŷ); positive above the line, negative below.
  • Residual plot — residuals vs. x; patternless scatter = model adequate; curve or fan = trouble.
  • Extrapolation — predicting outside the range of x used to fit the line; untrustworthy because the pattern itself may change.
  • Population slope (β) — the true slope behind the fitted b; H₀: β = 0 is "the flat line."
  • SE(b) — the standard error of the slope, reported by technology; measures sample-to-sample wobble in b.
  • t-test / confidence interval for the slope — t = b ⁄ SE(b) and b ± t*·SE(b), both with df = n − 2.
  • LINE conditions — Linear, Independent, Normal-ish residuals, Equal spread: the checklist before slope inference.
  • Choosing the right procedure — matching a question's answer-shape (mean, proportion, counts, line) to the tool that fits it.

Summary

  • The least-squares line ŷ = a + bx minimizes squared misses; b = r·(s_y ⁄ s_x) and the line passes through (x̄, ȳ). Interpret the slope with all four parts: per one unit of x, predicted, on average, in context units.
  • r² is the share of variation explained — not an accuracy rate. A residual is actual − predicted; a patternless residual plot is the model's all-clear.
  • Inside the data the line predicts; outside the data, the line is fiction. Refuse to extrapolate — the pattern itself can change beyond the data (the −15 kWh lesson).
  • Slope inference runs on familiar machinery: check LINE, then t = b ⁄ SE(b) and b ± t*·SE(b), both with df = n − 2, with SE(b) always read from output. Interval contains 0 ⇔ the two-sided test fails to reject.
  • Fail to reject ≠ accept, and a significant slope is still a handshake, not a push.
  • Choosing the procedure = naming the answer's shape: mean, proportion, two groups, counts in categories, or a line.

Check your understanding

Work these before looking at the answers — they're ungraded, and the answers (with reasoning) are at the bottom of the page.

  1. A dog-walking service prices walks with the fitted line ŷ = 8 + 0.5x, where x = the dog's weight (kg) and y = the fee charged ($), fitted on dogs from 5 to 40 kg. (a) Interpret the slope in a full four-part sentence. (b) Predict the fee for a 20 kg dog. (c) The intercept is 8 — your friend says "so walking a weightless dog costs $8." What's the better way to describe the intercept here?
  2. An escape-room company finds the correlation between a team's number of previous rooms played (x) and its escape time in minutes (y) is r = −0.6. (a) What is r², and what does it say in context? (b) One team was predicted to finish in 52 minutes and actually finished in 47. Compute the residual and say what its sign means.
  3. A plant researcher regresses leaf growth (mm) on weekly watering amount for n = 17 houseplants. Output: Coef = 2.40, SE Coef = 0.80 for the watering slope, P = 0.009. (a) Compute t and the df. (b) Using the course t-table at 95% (df 15: t* = 2.131), state the test's conclusion at α = 0.05. (c) Build the 95% confidence interval for the slope and check that it agrees with your test conclusion.
  4. A choir director wants to know whether members who attend the early rehearsal and members who attend the late rehearsal differ in average number of pieces memorized. Name the procedure this calls for, and explain in one sentence why — using the "shape of the answer" question — it isn't a chi-square test or a regression.

Answers (worked reasoning — read after attempting)

  1. (a) "Each additional kilogram of dog weight predicts, on average, $0.50 more in walking fee." (Per one unit of x ✓ predicted ✓ on average ✓ context units ✓.) (b) ŷ = 8 + 0.5(20) = 8 + 10 = $18. (c) A 0 kg dog doesn't exist, and x = 0 sits below the smallest dog in the data (5 kg) — so the intercept isn't a statement about weightless dogs; it's the line's anchor point: the base level the weight charge builds on.
  2. (a) r² = (−0.6)² = 0.36 — about 36% of the variation in escape times is explained by the linear relationship with experience; the other 64% is everything else (team chemistry, puzzle luck). Note r² is a share of variation, not "36% accurate." (b) Residual = actual − predicted = 47 − 52 = −5 minutes. Negative residual → the team finished faster than the line predicted — the point sits below the line.
  3. (a) t = 2.40 ÷ 0.80 = 3.0; df = n − 2 = 17 − 2 = 15. (b) |t| = 3.0 > 2.131 → reject H₀ at the 5% level (the output agrees: P = 0.009 < 0.05): the data give convincing evidence of a linear relationship between watering and leaf growth (β ≠ 0). (c) 95% CI: 2.40 ± 2.131 × 0.80 = 2.40 ± 1.7048 → (0.70, 4.10) mm per unit of watering (rounded to two decimals). The interval excludes 0, agreeing with the rejection — interval and test are one computation with two readings. (Whether more watering causes the growth is a design question — this was observational unless watering was randomly assigned.)
  4. Two-sample t-test (comparing the means of two independent groups). Shape of the answer: the question asks about an average (pieces memorized — quantitative), compared across two groups — so it's a two-sample means procedure. It isn't chi-square because nothing is being counted into categories, and it isn't regression because neither variable is predicting a number from another number — the "explanatory variable" here is just a two-group label.

Next up in this module: the slide deck if you want the lecture's visual walk-through, then the Readings & Resources page for optional deeper dives — and when you're ready to practice, Lecture Tutorial 17 turns your chatbot into your personal regression (and whole-toolkit) tutor.