Lecture: Probability Foundations
Lecture script · about 15–20 minutes spoken · Week 5
Opening
Picture a friend who has lost a coin toss 5 times in a row. They're sure the next toss is due to go their way. Are they right? By the end of this lecture you'll be able to say no, and say exactly why. That small question opens up a whole new way of thinking about chance.
Chance runs through daily work. A pharmacist weighs the risk of a side effect. A travel planner judges whether a connecting flight will run late. A field biologist asks which species a tagged bird on one island is likely to be. All of them reason with the same few rules, and those rules are what we build today.
In Weeks 1 through 4, you learned to describe data you already had: where it came from, what it looks like, its center and spread, and how two variables move together. This week we change direction. Instead of describing what did happen, we put a number on what might happen. Week 4's conditional distribution comes back today, too, as a conditional probability. Next week, in Week 6, Random Variables, we'll attach numbers to outcomes and find the average a chance process produces.
We'll take seven ideas in order: the long run, sample spaces, the complement rule, the addition rule, the multiplication rule, conditional probability, and independence. Our examples come from a pair of dice, the calendar, a guessed quiz, and the 344 penguins you've known since Week 1.
1. Randomness: the long run settles, the short run swings
Hold a fair die in your hand. Before you roll it, you can't say which face will land on top. But roll it 600 times, and you can say a lot. That mix of "unpredictable once" and "predictable over many tries" has a name. Randomness is the property of a process whose single outcomes cannot be predicted in advance but whose outcomes follow a stable pattern over many repetitions, like the roll of a fair die.
A probability is a number from 0 to 1 that gives the long-run share of repetitions in which an event happens. Zero means it never happens, and 1 means it always does. A die has 6 faces, and only one of them shows a six. So the probability of a six is 1 out of 6, about 16.7 percent. In 600 rolls, you'd expect about 100 sixes.
Here's the catch. After 6 rolls, you could easily see no sixes at all, or two of them. Those shares are 0 percent and about 33.3 percent, both far from 16.7. After 600 rolls, the share almost always sits near 16.7 percent. That settling has a name. The law of large numbers is the fact that, as a random process is repeated more and more times, the share of repetitions in which an event happens settles toward that event's probability, even though short runs can swing widely.
Notice what the law does not say. It says nothing about the next roll. The die has no memory. The long run evens out because hundreds of new rolls swamp an old streak, not because the die pays you back. Believing otherwise is the gambler's fallacy: the mistaken belief that an outcome becomes more likely after a streak without it.
Pause and try this. You've rolled a fair die five times with no six. What is the chance of a six on the next roll?
Take a moment and commit to a number.
Here's the answer. It's still 1 out of 6, about 16.7 percent. The die can't remember the streak, so the chance doesn't change, and our coin-tossing friend is wrong for the same reason.
2. Sample spaces and events: list every way it can land
Now roll two dice and add them, the way a board game does. How likely is a total of 7? Before you can answer, you need a full list of what can happen. An outcome is one possible result of a single repetition of a random process. The sample space is the complete list of every possible outcome. And an event is a set of outcomes from the sample space that you name and ask about.
Think of one die as red and one as blue, so a red 2 with a blue 5 is a different outcome from a red 5 with a blue 2. The red die has 6 faces and the blue die has 6, so there are 6 times 6, or 36, outcomes. Each one is equally likely, meaning each has the same chance of happening. When outcomes are equally likely, a probability is a count divided by a count: the outcomes in your event, over all the outcomes.
So count the ways to make 7. Red 1 and blue 6, red 2 and blue 5, and so on up to red 6 and blue 1. That's 6 outcomes out of 36, which is 1 out of 6. Now count the totals of 10 or more: red 4 and blue 6, red 5 and blue 5, and four more pairs, 6 outcomes in all. That's also 6 out of 36.
Pause and try this. A classmate says the totals run from 2 to 12, so each total has the same chance. Why is that wrong?
Think about how many ways a total of 2 can happen.
Here's the answer. The totals aren't equally likely outcomes. A total of 2 needs a 1 on both dice, which is one way, while a total of 7 happens six ways. Count the 36 equally likely pairs first, then group them into totals.
3. The complement rule: count what does not happen
If a weather app says there's a 30 percent chance of rain, you already know the chance of no rain: 70 percent. Rain and no rain cover every possibility, so their chances add to the whole. The complement of an event is the event that it does not happen. The complement rule says that the probability an event does not happen equals 1 minus the probability that it does happen.
This rule saves work. Say you want the chance that two dice total 9 or less. Counting every total from 2 to 9 is slow. But "9 or less" fails exactly when the total is 10 or more, and you already know that chance: 6 out of 36. So the chance of 9 or less is 1 minus 6 out of 36, which is 30 out of 36, or 5 out of 6. That's about 83.3 percent.
The calendar works the same way. Pick a month of a common year at random. Seven of the 12 months have 31 days, so the chance is 7 out of 12. The chance of a shorter month is 1 minus 7 out of 12, which is 5 out of 12. You can check that by counting: February and the four 30-day months make 5.
Pause and try this. Someone writes the complement of "the total is 10 or more" as "the total is 10 or less." What went wrong?
Look closely at the total of exactly 10.
Here's the answer. A total of 10 sits in both events, so they overlap. An event and its complement never share an outcome, so the true complement is "9 or less".
4. The addition rule: add the chances, then subtract the overlap
Suppose a game pays you if the dice total 7 or if they total 10 or more. In probability, "or" means at least one of them happens. Two events are mutually exclusive when they share no outcomes, so they can't both happen on the same roll. A total of 7 and a total of 10 or more are like that. The addition rule says that for mutually exclusive events, you add the two probabilities. So 6 out of 36 plus 6 out of 36 is 12 out of 36, which is 1 out of 3.
Many events do overlap, though. Take "the first die shows 6" and "the total is 10 or more". The first event has 6 outcomes. The second has 6 outcomes. But three outcomes are in both: a 6 then a 4, a 6 then a 5, and a 6 then a 6. If you add 6 and 6, you count those three twice. The general addition rule works for any two events: add the two probabilities, then subtract the probability that both happen, so outcomes in the overlap are counted only once. That gives 6 plus 6 minus 3, or 9 outcomes out of 36. That's 0.25, one roll in four.
Here's an analogy. Picture two guest lists for one party, and a few friends are on both. To count the guests, you add the two lists and then take out the names you counted twice. The general addition rule does the same thing with chances.
Pause and try this. If you forgot the overlap, what wrong answer would you get for "first die 6 or total 10 or more"?
Try it before you check.
Here's the answer. You'd get 12 out of 36 instead of 9 out of 36. The three shared outcomes would be counted twice, once for each event.
5. The multiplication rule: multiply when one event leaves the other alone
Roll the red die and the blue die at the same moment. What the red die shows has no effect on the blue one. Independent events are two events for which learning that one happened leaves the probability of the other unchanged. And the multiplication rule says that the probability two independent events both happen equals the product of their separate probabilities.
So the chance of two sixes is 1 out of 6 times 1 out of 6, which is 1 out of 36. Check it against the list of outcomes: only one pair, a 6 and a 6, gives double sixes. The rule and the count agree.
Now a classroom case. A student guesses at random on a four-option multiple-choice question. One option of the four is right, so the chance is 1 out of 4, which is 0.25. Guess on two questions, and one guess has no effect on the other. Multiply: 1 out of 4 times 1 out of 4 is 1 out of 16, or 6.25 percent. Guessing gets worse fast as the questions pile up.
But the rule only works when the events really leave each other alone. Every one of the 124 Gentoo penguins was logged on the island of Biscoe. So the chance a random penguin is Gentoo and on Biscoe is the full 124 out of 344. Multiplying by the chance of Biscoe would shrink that number, and it would be wrong.
Pause and try this. Two dice are rolled. Is the chance that both show a six bigger or smaller than the chance that one die shows a six?
Think about what "both" asks for.
Here's the answer. It's smaller: 1 out of 36 compared with 1 out of 6. Asking for both events to happen is a stricter demand, so its chance shrinks.
6. Conditional probability: shrink the world to what you know
Imagine a field biologist standing on Biscoe who spots a penguin far down the beach. Which species is it? News changes the guess. Knowing the island shrinks the world to that island's penguins. A conditional probability is the probability that one event happens once you know that a second event has happened. You keep only the outcomes in the second event and find the share of them that are also in the first.
Here's the table in words. Biscoe holds 44 Adelie and 124 Gentoo, which is 168 penguins. Dream holds 56 Adelie and 68 Chinstrap, which is 124. Torgersen holds 52 Adelie. Together that's 344.
Now ask for the chance a penguin is Gentoo, given that it's on Biscoe. We read the vertical bar in the symbol as "given". Keep only the 168 Biscoe penguins. Of those, 124 are Gentoo. So the chance is 124 out of 168, about 0.7381. With no news at all, the chance a penguin is Gentoo is 124 out of 344, about 0.3605. One fact, "this bird is on Biscoe", roughly doubles the chance of Gentoo.
Watch the order, though. Flip the question: given that a penguin is Gentoo, what's the chance it's on Biscoe? Every Gentoo is on Biscoe, so that chance is 124 out of 124, which is 1. The chance of Gentoo given Biscoe is only 124 out of 168. Swapping the condition changed the answer.
Pause and try this. Which number do you divide by to find the chance a penguin is on Biscoe, given that it is Gentoo?
Say the word "given" out loud as you think.
Here's the answer. You divide by 124, the number of Gentoo penguins, because Gentoo is the group you were given. Always divide by the group you were given, not by everyone.
7. Independence: check it, and do not confuse it with mutually exclusive
Now we can test independence instead of assuming it. Compare a conditional probability with a plain one. If the news changes the chance, the events are linked. Dependent events are two events for which learning that one happened changes the probability of the other.
Try it with dice. Suppose you know the first die shows 6. Only one of the six pairs, a 6 then a 1, totals 7. So the chance of a 7, given a first die of 6, is 1 out of 6. That matches the plain chance of a 7, so those two events are independent. Now test a total of 10 or more. Three of the six pairs reach 10, so the chance is 3 out of 6, which is 1 out of 2. The plain chance was only 1 out of 6, so these events are dependent.
The penguins fail the test too. About 0.7381 is nowhere near 0.3605, so species and island are not independent.
Here's the trap. Students often think "independent" means "can't happen together". It's nearly the opposite. No penguin is both Adelie and Gentoo, so those events are mutually exclusive. But tell me a penguin is Adelie, and the chance it's Gentoo drops to 0. The news changed the chance, so the events are dependent.
Pause and try this. "The first die shows 6" and "the second die shows 6": are these independent, mutually exclusive, or both?
Ask whether they can happen on the same roll.
Here's the answer. They're independent but not mutually exclusive. One die doesn't affect the other, and both can land on 6 at once, with a chance of 1 out of 36.
Closing
Let's pull it together in the order we built it. A probability is a long-run share; short runs swing, long runs settle, and a streak never makes an outcome due. List the equally likely outcomes, then count the ones you want. The chance something doesn't happen is 1 minus the chance it does. For "or", add the chances and take the overlap out once. For "and", multiply only when the events are independent. A conditional probability divides by the group you were given, and swapping the condition changes the answer. And mutually exclusive events are never independent.
Next in this module come the Readings & Resources page, the lecture tutorial, the practice set, and the quiz, where you'll use these rules yourself, and then the discussion and the assignment. Next week, in Week 6, Random Variables, every outcome gets a number, and you'll find the average a chance process produces in the long run.
Built with The Course Maker · samples.thecoursemaker.com