← Back to the Introduction to Statistics outline The Course Maker
← Introduction to Statistics outline
Week 5 · Chapter

Chapter 5 — Probability Foundations

Introduction to Statistics Generic evergreen edition

Course: Introduction to Statistics (18-week generic edition)
The module's primary reading · ~30–40 minutes · Objective 4 (probability-rules portion) · Read this before (or alongside) the week's lecture — everything else in the module builds on it.

Why you already care about this

One night in 1913, at a roulette table in Monte Carlo, the ball landed on black. Then black again. Then again — twenty-six times in a row, the most famous streak in casino history. As the run grew, bettors crowded the table and piled increasingly desperate sums on red, all reasoning the same way: after this many blacks, red is overwhelmingly due. The wheel, which had no idea what it had just done, kept paying the house. Reportedly, millions of francs were lost to a single sentence of bad probability.

You run the same mental machinery every day, at lower stakes: a weather app says 70% rain and you decide whether that's a promise; a game has you convinced your luck "has to turn"; a headline says a screening test is "95% accurate" and you wonder what that even means. For four weeks this course has described data you already had. This week it starts computing the chances of things that haven't happened yet — which is the machinery every inference tool in the second half of the course runs on.

By the end of this chapter you will be able to: say what a probability actually is (and what it never promises); list a sample space and compute probabilities when outcomes are equally likely; use the complement, addition, and multiplication rules — and know when each one is legal; tell disjoint from independent (they are not synonyms); and read a conditional probability out of a two-way table without falling for the swap that fools headline writers. Five rules, one scale, two famous traps.

Where we are

Weeks 1–4 built the describing half of the course: where trustworthy data come from (Week 1), pictures and shapes (Week 2), center and spread (Week 3), and relationships between two variables — including the two-way tables you counted your way through (Week 4). This week those same tables come back with a new superpower, and the course pivots from what did we see? to what are the chances? Nothing here needs more math than careful counting and fractions.

1. What probability is: the long run

Start with the plain idea. A random phenomenon is a process where individual outcomes are unpredictable, but the long-run pattern is completely stable. Nobody on Earth can call your next roll of a fair die. Anybody who's taken this course can tell you what 10,000 rolls will look like: each face appearing about one-sixth of the time.

That long-run pattern is what probability is:

The probability of an event is the proportion of times the event would occur in a very long run of repetitions.

P(six) = 1/6 means: roll forever, and the fraction of rolls showing six settles at one-sixth. It does not mean "one six in every six rolls," and it says nothing at all about your next roll. Carry this as the week's motto: probability is a long-run promise, not a short-run guarantee.

The settling-down itself has a name — the law of large numbers: as the number of repetitions grows, the running proportion of an event drifts toward its true probability. Short runs are wild; long runs are tame.

Worked example. A student simulates die rolls in a spreadsheet and tracks the running proportion of sixes:

  • After 10 rolls: 3 sixes → 3 ÷ 10 = 0.300. Nearly double the "true" value — and nothing is wrong.
  • After 100 rolls: 21 sixes → 21 ÷ 100 = 0.210.
  • After 1,000 rolls: 172 sixes → 172 ÷ 1,000 = 0.172.
  • The theoretical value is 1 ÷ 6 ≈ 0.1667, and each checkpoint lands closer than the last: 0.300 → 0.210 → 0.172 → … That narrowing is the law of large numbers, live.

Misconception check. "P(six) = 1/6, so in six rolls I'll get exactly one six." No — six rolls can easily produce zero sixes, or three. The 1/6 describes the process over the long run, not any short stretch of it. (You will watch this yourself in this week's Data Lab, where your own 500 simulated rolls almost certainly won't give exactly 0.1667 — and that's the lesson working.)

In practice (your tools). The fastest way to feel the law of large numbers is to simulate it. In Google Sheets or Excel, =RANDBETWEEN(1,6) produces one virtual die roll; fill it down 500 rows, count the sixes so far with =COUNTIF($B$2:B2,6), divide by the roll number, and chart the running proportion. Early chaos, then the settle toward 0.1667. That is exactly this week's Data Lab.

2. Sample spaces, events, and equally likely outcomes

Before you can compute any chance, list what can happen.

  • The sample space (written S) is the set of all possible outcomes of a random phenomenon. One die: S = {1, 2, 3, 4, 5, 6}. One card from a standard 52-card deck: 52 outcomes.
  • An event is any collection of outcomes you care about. "Roll an even number" is the event {2, 4, 6}.
  • Probability lives on a fixed scale: every probability is between 0 and 1 (impossible → 0, certain → 1), and the whole sample space has probability 1. A "probability" of 1.4 or −0.2 is not a bold claim; it is an arithmetic error, and later in this chapter you'll see which error it usually is.

When all outcomes in the sample space are equally likely, probability is pure counting:

P(A) = (number of outcomes in A) ÷ (total number of outcomes in S).

Worked example. Roll two fair dice — picture one red and one blue so you can tell them apart. Every (red, blue) pair is one outcome, so the sample space has 6 × 6 = 36 equally likely outcomes. What's the probability the sum is 7? Count the ways: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) — six ways. So P(sum is 7) = 6 ÷ 36 = 1/6 ≈ 0.167. Same recipe with one card: P(heart) = 13 ÷ 52 = 1/4 = 0.25. Count the favorable, divide by the total.

One warning before the counting recipe goes to your head: equally likely must be earned. A die is symmetric, so its six faces genuinely share the probability equally. A tossed thumbtack lands point-up or point-down — two outcomes — and they are nowhere near 50/50, because nothing about a thumbtack is symmetric.

Misconception check. "There are two outcomes, so it's 50/50." This is the equiprobability bias, and it's everywhere: "I'll either pass or fail — 50/50." Two outcomes only split the probability evenly when symmetry (or evidence) says they should. Count outcomes only after checking that each one carries equal weight.

3. The complement rule: the "not" shortcut

The complement of an event A — written "not A" — is everything in the sample space that A doesn't include. Since something must happen, A and not-A split the total probability of 1 between them:

Complement rule: P(not A) = 1 − P(A).

Worked example. A prize spinner has 10 equal sectors, 3 of them red. P(red) = 3/10 = 0.3. What's the probability of not landing on red? You could count the seven non-red sectors — or just take the shortcut: P(not red) = 1 − 0.3 = 0.7. Done.

The rule looks almost too small to name, but it quietly does two big jobs. First, it's often the fast route: "what's the chance of at least one …" questions in later weeks are usually solved by computing the complement ("none at all") and subtracting from 1. Second, it doubles as a sanity check on everything else you compute: if your probabilities of A and not-A don't add to 1 — or any answer escapes the 0-to-1 scale — something upstream broke.

4. The addition rules: "or," and the double-count trap

Statistics reads "A or B" inclusively: A happens, B happens, or both do. How do you compute it?

First, a definition. Two events are disjoint (also called mutually exclusive) if they cannot both happen on the same trial. One card cannot be both a seven and a king.

Addition rule (disjoint events): P(A or B) = P(A) + P(B).
General addition rule (any two events): P(A or B) = P(A) + P(B) − P(A and B).

The general rule exists because plain adding counts the overlap twice. Watch it happen:

Worked example. Draw one card from a standard deck. What is P(heart or face card)? Hearts: 13 of the 52. Face cards (jacks, queens, kings): 12 of the 52. Add them: 13/52 + 12/52 = 25/52 — and that answer is wrong. The jack, queen, and king of hearts are both hearts and face cards, and the plain sum counted each of those three cards twice. Subtract the overlap once: P(heart and face) = 3/52, so

P(heart or face) = 13/52 + 12/52 − 3/52 = 22/52 = 11/26 ≈ 0.423.

Memory hook: "OR adds — then subtracts the overlap." When events are disjoint the overlap is zero, and the general rule collapses into the plain one; checking "can they both happen?" before adding is the entire skill.

Misconception check. "You can always just add probabilities for 'or.'" Adding overlapping events inflates the answer — sometimes past 1, which is your smoke alarm. If a probability calculation ever hands you 1.2, hunt for the double-count.

5. Independence, the multiplication rule, and the gambler's fallacy

Now "and." Two events are independent if knowing that one happened does not change the probability of the other. Two spins of a spinner: what the first spin did tells you literally nothing about the second.

Multiplication rule (independent events): P(A and B) = P(A) × P(B).

Worked example. That same 10-sector spinner (3 red sectors). P(red on any one spin) = 0.3. Spin it twice; the spins are independent. P(red then red) = 0.3 × 0.3 = 0.09 — about a 9% chance. Notice the direction: the "and" answer is smaller than either ingredient. Demanding two things is harder than demanding one. ("Or" grows; "and" shrinks — a free sanity check on every answer.)

Two boundaries make this rule safe to use:

Boundary one: disjoint and independent are different words. Disjoint is an overlap fact — the events can't both happen. Independent is an information fact — one happening tells you nothing about the other. Here's the proof they aren't synonyms: disjoint events are never independent (unless one was impossible to begin with). If "seven" and "king" are disjoint and I tell you the card is a seven, the probability it's a king just crashed to zero — knowing one event completely changed the other. That's maximal dependence.

Boundary two: independence must be plausible, not assumed. Dice, spinners, and well-shuffled decks are memoryless machines — independence fits. Weather on consecutive days, a streaky athlete, back-to-back readings on a warming machine — these have memory, and multiplying plain probabilities across them is fiction. (When events aren't independent, the honest "and" computation uses conditional probability — Section 6 — or a two-way table, which handles it for you.)

And now the trap that built Monte Carlo's fortune:

Misconception check — the gambler's fallacy. "No six in twenty rolls — a six is due." No. The die has no memory: on every roll, P(six) = 1/6, whatever just happened. Independence is exactly the statement that history doesn't lean on the next trial. But wait — doesn't the law of large numbers say the proportion must return to 1/6? It does, and here is the reconciliation that dissolves the paradox: the long run repairs proportions by swamping, not by compensating. A drought of sixes never gets "paid back" with extra sixes; it gets diluted into insignificance by thousands of ordinary rolls. The wheel owed those Monte Carlo bettors nothing, and it paid them exactly that.

6. Conditional probability: shrinking the world

One more idea, and it's the week's most valuable one. Often you want a probability in light of something you already know: the chance a widget is defective given which machine made it; the chance a text gets answered given it was sent at 2 a.m.

The conditional probability of A given B, written P(A | B), is the probability that A occurs given that B has occurred. The vertical bar reads "given."

The move behind the symbol is one you already made in Week 4, when you computed conditional distributions from a two-way table: shrink your world to B, then re-count inside it. Whatever follows the bar is your new denominator.

Worked example — the week's signature table. A small factory's end-of-day quality check covers all 200 widgets made that day on its two machines:

The table below shows one day's production counts by machine and inspection result.

Defective Not defective Total
Machine A 6 114 120
Machine B 10 70 80
Total 16 184 200

Three questions, three different worlds:

  1. P(defective) — world: all 200 widgets. 16 ÷ 200 = 0.08. The plant-wide defect rate.
  2. P(defective | Machine B) — world: B's row only, 80 widgets. 10 ÷ 80 = 0.125. For comparison, P(defective | Machine A) = 6 ÷ 120 = 0.05. Machine B's output fails at two and a half times A's rate.
  3. P(Machine B | defective) — a different question entirely: a defective widget just turned up in the reject bin; how likely did it come from B? World: the defective column, 16 widgets. 10 ÷ 16 = 0.625.

There is also a formula version, for when you have probabilities instead of counts:

P(A | B) = P(A and B) ÷ P(B).

Check it against the table: P(defective | B) = (10/200) ÷ (80/200) = 0.05 ÷ 0.40 = 0.125. Same answer — the formula is the shrink-and-re-count, written as a fraction. Rearranged, it gives the general multiplication rule, P(A and B) = P(B) × P(A | B), which collapses to Section 5's rule exactly when A and B are independent — because independence means P(A | B) = P(A).

That last sentence hands you a practical test. Is "defective" independent of "machine"? Compare P(defective | B) = 0.125 with the overall P(defective) = 0.08. Not equal — so no: defect status depends on machine, and it's Machine B that needs the maintenance visit. When the conditional probability matches the overall probability, the "given" told you nothing, and that is what independence looks like in a table.

Misconception check — the conditional swap. "P(defective | B) and P(B | defective) are the same thing." They are not even close: 0.125 versus 0.625. The first lives in B's row (of B's output, what fraction fails?); the second lives in the defective column (of the failures, what fraction is B's?). Same table, same two events — different denominators, different questions. Headlines run on this swap ("most accidents happen near home" describes where the driving is, not how dangerous your street is). The habit that prevents it forever: say the "given" world out loud in words before you divide.

In practice (your tools). Spreadsheets make the counting side of this painless: =COUNTIF tallies one condition, =COUNTIFS tallies a joint condition ("Machine B" and "defective"), and dividing the two is a conditional probability. When this week's tutorial or lab hands you a table, resist the urge to memorize which number goes where — rebuild the shrink-the-world logic each time, and the denominator picks itself.

Key terms

  • Random phenomenon — individual outcomes unpredictable; long-run pattern stable.
  • Probability — the proportion of times an event would occur in a very long run of repetitions.
  • Law of large numbers — as repetitions grow, the running proportion of an event settles toward its true probability.
  • Simulation — using a random mechanism (like =RANDBETWEEN) to imitate a chance process many times.
  • Sample space (S) — the set of all possible outcomes.
  • Event — any collection of outcomes from the sample space.
  • Complement (not A) — all outcomes not in A; P(not A) = 1 − P(A).
  • Disjoint (mutually exclusive) events — events that cannot both happen on the same trial.
  • Addition rule (disjoint) — P(A or B) = P(A) + P(B).
  • General addition rule — P(A or B) = P(A) + P(B) − P(A and B).
  • Independent events — knowing one happened doesn't change the probability of the other.
  • Multiplication rule (independent events) — P(A and B) = P(A) × P(B).
  • Conditional probability P(A | B) — the probability of A given that B occurred; equals P(A and B) ÷ P(B).
  • Gambler's fallacy — the false belief that a run of one outcome makes the other outcome "due" in an independent process.

Summary

  • Probability is a long-run promise, not a short-run guarantee: P(A) is the proportion of times A would happen over endless repetitions, and the law of large numbers is the settling-down you can watch in a simulation.
  • List the sample space first; when outcomes are genuinely equally likely, P(A) = favorable ÷ total — and "equally likely" must be earned, never assumed from "there are two outcomes."
  • NOT: P(not A) = 1 − P(A). Every probability lives between 0 and 1 — answers outside the scale mean an upstream error.
  • OR: add — and subtract the overlap unless the events are disjoint (can't both happen).
  • AND: multiply — but only when the events are independent (one tells you nothing about the other). Disjoint and independent are different facts, and disjoint events are never independent.
  • GIVEN: P(A | B) shrinks the world to B and re-counts; P(A | B) ≠ P(B | A), and saying the "given" world in words first prevents the swap. A conditional that matches the overall rate is what independence looks like in a table.
  • The die has no memory: streaks don't create debts (gambler's fallacy), and the long run fixes proportions by swamping, not compensating.

Check your understanding

Work these before looking at the answers — they're ungraded, and the answers (with reasoning) are at the bottom of the page.

  1. A carnival wheel has 12 equal sectors: 5 green, 7 yellow. For one spin, find P(green) and P(not green), and state the rule you used for the second one.
  2. One card is drawn from a standard 52-card deck. Find P(the card is a number card (2 through 10) or a spade). (Careful: can both happen at once?)
  3. A friend rolls a fair die 20 times and gets 7 sixes — a proportion of 0.35, more than double 1/6. She concludes the die must be loaded. Using this week's vocabulary, explain what's actually going on, and what would happen to that proportion over 2,000 more rolls of a fair die.
  4. A pottery studio's kiln log shows 120 bowls fired last month: 70 with Glaze A (7 cracked) and 50 with Glaze B (5 cracked). Find P(cracked), P(cracked | Glaze A), and P(cracked | Glaze B) — then say whether cracking appears independent of glaze, and how you know.

Answers (worked reasoning — read after attempting)

  1. The 12 sectors are equally likely, so P(green) = 5/12 ≈ 0.417. For the second: complement rule — P(not green) = 1 − 5/12 = 7/12 ≈ 0.583. (Counting the 7 yellow sectors gives the same answer; the rule is the shortcut that skips the counting.)
  2. The events overlap — a card can be both a number card and a spade (the 2–10 of spades, 9 cards) — so use the general addition rule. Number cards: 9 per suit × 4 suits = 36. Spades: 13. Overlap: 9. P(number or spade) = 36/52 + 13/52 − 9/52 = 40/52 = 10/13 ≈ 0.769. Plain adding would have double-counted those 9 spade number cards.
  3. Nothing needs explaining away — 7 sixes in 20 rolls is ordinary chance variation in a short run; probability is a long-run promise, and 20 rolls isn't the long run. By the law of large numbers, over 2,000 more rolls the running proportion would drift toward 1/6 ≈ 0.1667 — not because the die "owes" non-sixes (that's the gambler's-fallacy direction), but because thousands of ordinary rolls swamp the early surplus. Concluding "loaded" from 20 rolls is reading a short run as if it were the long run.
  4. World: all 120 bowls → P(cracked) = 12/120 = 0.10. Shrink to Glaze A's 70 bowls: P(cracked | A) = 7/70 = 0.10. Shrink to Glaze B's 50: P(cracked | B) = 5/50 = 0.10. Every conditional matches the overall rate — knowing the glaze changes nothing — so cracking and glaze look independent in these data. (That is the table signature of independence: the "given" carries no information.)

Next up in this module: the slide deck if you want the lecture's visual walk-through, then the Readings & Resources page for optional deeper dives — and when you're ready to practice, Lecture Tutorial 5 turns your chatbot into your personal probability tutor.