Week 13 Data Lab: Could a Guesser Score 7?

Part 1 — The Big Picture

This week you learned the logic of a significance test: assume the null hypothesis is true, then ask how surprising the data would be. In this lab you measure that surprise yourself. A student claims they guessed on all 10 questions of a four-option quiz and scored 7. You build the guessing model in a spreadsheet with BINOM.DIST, chart it as bars, mark the tail at 7 or more, and find the p-value two ways. Then you simulate 1,000 pure guessers to see how rarely guessing alone reaches 7, compare two rules for flagging a student by their Type I error rates, and check a chatbot's reasoning about the same test.

Time budget: Plan for about 90 minutes.

Part 2 — The Task

Guiding question: If the student really guessed on every question, how surprising is a score of 7 out of 10, and is that surprise strong enough to reject the guessing claim at α = 0.05?

Workshop data: Guessing-quiz null model and 1,000 simulated guessers

Source: A model and a simulation you build yourself in a spreadsheet: the binomial chances for a pure guesser on 10 questions with four options each, alongside 1,000 pure guessers whose answers the spreadsheet draws at random.

• Rows: 1,000 simulated guessers, one per spreadsheet row; the first scaffold table lists the null model in 12 rows, one per score from 0 to 10 plus a total row
• Columns: ten question columns, each 1 for a correct guess and 0 for a wrong one, and the guesser's score
• Model: binomial with n = 10 questions and p = 0.25, the chance of guessing one question right

Part 3 — Materials & Procedure

Tools and materials

• Google Sheets (free) or Excel
• A document or notebook for your written answers

Ways of working

• interpret-real-data
• compute-and-plot
• structured-observation

Procedure

1. Open a blank sheet and name the first tab Model. In A1 type Score, in B1 type P(X = k), and in C1 type P(X ≥ k). Then type the scores 0 through 10 in cells A2 to A12. Tool: Google Sheets (free) or Excel. Check: Column A lists the eleven possible scores, from 0 to 10.
2. In B2 type =BINOM.DIST(A2,10,0.25,FALSE) and fill it down to B12; each cell is the chance that a pure guesser scores exactly that many. In B13 type =SUM(B2:B12). Format column B to four decimals and copy the eleven chances into the first scaffold table. Tool: Google Sheets (free) or Excel. Check: Eleven chances between 0 and 1 appear, and B13 shows 1.
3. In C2 type =SUM(B2:B$12) and fill it down to C12; each cell adds the chances from that score up to 10, so it is the chance of that score or higher. Then, in E2, type =1-BINOM.DIST(6,10,0.25,TRUE), which finds the chance of 7 or more with the complement rule, and compare it with C9. Copy the C column into the first scaffold table. Tool: Google Sheets (free) or Excel. Check: C2 shows 1, the column shrinks toward 0 as the score rises, and E2 matches C9 to four decimals.
4. Select A1:B12 and use Insert, then Chart, and choose a column chart, so each score gets one bar. Title it Scores of a pure guesser on 10 four-option questions, and label the horizontal axis Score (correct answers out of 10) and the vertical axis Chance if H₀ is true. Mark the tail: in Google Sheets use Customize, then Series, then Format data point to give the bars for 7, 8, 9, and 10 a different fill; in Excel click one bar twice to select it alone, then use Format Data Point. Add a text note on the chart that names the bars for 7 or more as the tail, so the chart does not rely on color alone. Tool: Google Sheets (free) or Excel. Check: Eleven bars rise to a peak at the low scores and fall away to the right; the tail bars for 9 and 10 are too short to see, and your note names the tail.
5. Add a tab named Simulation. In B2 type =IF(RANDBETWEEN(1,4)=1,1,0), which gives 1 for a correct guess (one chance in four) and 0 for a wrong one, and copy it across to K2 for the ten questions. In L2 type =SUM(B2:K2) for that guesser's score. Select B2:L2 and fill down to row 1001, so 1,000 pure guessers each take the quiz. Tool: Google Sheets (free) or Excel. Check: Column L holds 1,000 whole-number scores from 0 to 10, and every edit to the sheet draws new guesses.
6. Copy L2:L1001 and paste it as values into N2 (in Google Sheets, Edit, then Paste special, then Values only; in Excel, Home, then Paste, then Paste Values), so the scores stop changing. Then type =AVERAGE(N2:N1001), =COUNTIF(N2:N1001,">=5"), and =COUNTIF(N2:N1001,">=7"), and record the three results in the second scaffold table beside what the model expects: np for the average, and 1,000 times the model chance for each count. Tool: Google Sheets (free) or Excel. Check: The average sits near the null mean np, and far fewer of your guessers reach 7 than reach 5.
7. Fill the third and fourth scaffold tables: state the two hypotheses, record the test statistic and the p-value both ways, write the p-value in the course sentence, decide at α = 0.05, and compare the two rejection rules. Check: Every row of both tables has an answer, and each number traces to a cell in your sheet or to your first table.
8. Answer the analysis questions in your document, then paste your tables into an approved chatbot with the prompt in Part 6 and check its answer against the scripted checks. Tool: A document or notebook for your written answers. Check: You have five written answers and a list of the chatbot's errors with corrections.

Fallback route

When to use it: Use this route if no spreadsheet is available or the formulas will not run.

Use a calculator and the binomial formula: the chance of exactly k correct is the number of ways to place k correct answers among the 10 questions, times 0.25 multiplied by itself k times, times 0.75 multiplied by itself 10 − k times. The model data below gives the number of ways for each score. Add the chances from the top score down to fill the P(X ≥ k) column. Draw the bar chart by hand on grid paper, shade the bars for 7 to 10, and label them as the tail in words. In place of your own simulation, use the counts below from one completed run of 1,000 simulated guessers. Complete every scaffold table, every analysis question, and the critique step from these records, and submit the same items for equal credit.

This route earns the same credit as the primary route.

The number of ways for each score lets you compute the model by hand, and one completed run of 1,000 guessers stands in for your own simulation.

Fallback model data

Score k	Ways to place k correct answers among 10 questions	Guessers with that score in one run of 1,000
0	1	59
1	10	190
2	45	276
3	120	228
4	210	164
5	252	55
6	210	25
7	120	2
8	45	1
9	10	0
10	1	0
Total	1,024	1,000

Part 4 — The Scaffold

Build the guessing model score by score, because every p-value and error rate in this lab is read from it.

Student scaffold table 1

Score k	P(X = k), four decimals	P(X ≥ k), four decimals
0
1
2
3
4
5
6
7
8
9
10
Total of the P(X = k) column

Set your simulated guessers beside the model, because a p-value is a statement about how often guessing alone does this well.

Student scaffold table 2

Quantity	Expected from the null model	From your 1,000 simulated guessers
Average score
Guessers scoring 5 or more, out of 1,000
Guessers scoring 7 or more, out of 1,000

Carry out the test one step at a time, from the hypotheses to a conclusion about the student's claim.

Student scaffold table 3

Step of the test	Your answer
Null hypothesis H₀, in symbols and in words
Alternative hypothesis Hₐ, in symbols and in words
Why the alternative is one-sided
Test statistic: the observed score, and how many standard deviations it sits above np
p-value from adding the tail of your first table
p-value from =1-BINOM.DIST(6,10,0.25,TRUE)
The p-value in the course sentence
Decision at α = 0.05, chosen before looking at the data
Conclusion about the student's claim, in context

Compare two rules for flagging a student, because the cutoff you choose sets how often a pure guesser is wrongly flagged.

Student scaffold table 4

Rejection rule	Type I error rate from the model, four decimals	Your simulated guessers wrongly flagged, out of 1,000	Would the rule flag a student who scores 6?
Reject H₀ when the score is 5 or more
Reject H₀ when the score is 7 or more

Part 5 — Analysis Questions

1. From your first table, report P(X = 7) and the p-value P(X ≥ 7), each to four decimals, and show that adding the tail and the complement formula give the same p-value. Then compute (7 − 2.5) ⁄ 1.3693, using the course's null standard deviation 1.3693, and say in words how far the score of 7 sits from what guessing predicts.
2. Describe your bar chart in words, without relying on color: where the tallest bar sits, which way the long, thin tail runs, and where the score of 7 falls. Then compare your simulated guessers with the model in your second table: how close is each simulated value to the model's, and in which direction does each miss?
3. Write the p-value in the course sentence, "if the null were true, data this extreme would occur about … of the time". State the decision at α = 0.05 and the conclusion about the student's claim. Then explain in two sentences what the p-value is not: it is not the chance that the student was guessing, and it is not the chance that the score was a fluke.
4. Use your fourth table. How many times smaller is the Type I error rate of the rule "reject when the score is 7 or more" than that of the rule "reject when the score is 5 or more"? Then explain, without computing a number, why the stricter rule makes more Type II errors, using a student who knows some answers and scores 6, and what that does to the power, 1 − β.
5. Choose one and answer it in 3–4 sentences. (a) A score of 5 has a p-value of 0.0781, above α = 0.05, so you fail to reject H₀: explain why that does not prove the student was guessing, using the jury picture of "not guilty" versus "innocent". (b) Your null model assumes every question has four equally tempting options and that each guess is independent of the others: name one way a real quiz could break that model, and say how that would change what your p-value can tell you.

Part 6 — AI-Critique Moment

Choose one equal-credit critique route. Route A: ask an approved AI tool to critique your completed work, then check its response against your own evidence. Route B is the no-AI equivalent printed below.

Approved tools: Claude, Gemini, ChatGPT, Grok, Microsoft Copilot.

Prompt to paste:

I've pasted my completed Week 13 Data Lab tables above. A student says they guessed at random on all 10 questions of a quiz with four options per question, and they scored 7. I tested H₀: p = 0.25 against Hₐ: p > 0.25, where p is the student's true chance of a correct answer on each question, and I also simulated 1,000 pure guessers in a spreadsheet. Please do five things. First, give the probability that a pure guesser gets exactly 7 questions right, showing your arithmetic. Second, give the p-value for a score of 7, and tell me how many of my 1,000 simulated guessers scored 7 or more. Third, describe the shape of the distribution of a pure guesser's scores and name any skew. Fourth, tell me what the p-value means and whether it is the chance that the student was guessing; as a look back at Week 11, also say whether a 95% confidence interval means there is a 95% chance that the true mean is inside it. Fifth, say whether a small p-value proves that studying caused the student's high score.

Test the response for these likely failures:

• Asked for the chance that a pure guesser gets exactly 7 right, chatbots often multiply the chance of one arrangement and stop, reporting 0.25⁷ × 0.75³ ≈ 0.0000257, as if the 7 correct answers had to be the first 7 questions.
• Asked about your simulation, chatbots often state how many of your 1,000 simulated guessers scored 7 or more as if they had seen your sheet, or give a p-value they never computed from the binomial model.
• Asked about the shape of a pure guesser's scores, chatbots often call the distribution left-skewed because most of the bars sit at the low scores, naming the skew by where the pile is.
• Asked whether a small p-value proves that studying caused the high score, chatbots often say yes, reading a cause into a result that only rules out pure guessing.
• Chatbots often read the p-value 0.0035 as a 0.35% chance that the student was guessing, the same slip as reading a 95% confidence interval as a 95% chance that the true mean is inside that one interval.

Write 3–5 sentences naming one thing the chatbot got right and at least two places it went wrong or overreached, quoting its words and correcting each with evidence from your own tables or the course's pinned values.

Equivalent route (no AI required)

No AI account is required for this lab. Instead of the chatbot, critique this flawed analysis, printed here in full: "The chance that a pure guesser gets exactly 7 right is 0.25⁷ × 0.75³ ≈ 0.0000257. Of my 1,000 simulated guessers, 35 scored 7 or more, so the p-value is 35 ⁄ 1,000 = 0.035. A pure guesser's scores are left-skewed, because most of the bars sit at the low scores. The p-value means there is only a 0.35% chance that the student was guessing, and likewise a 95% confidence interval means a 95% chance that the true mean is inside it. Because the result is significant, studying caused the student's high score." Mark it up: quote each flawed sentence, name what is wrong (a dropped number of ways, an invented statistic, a flipped skew direction, a misread probability statement, or cause read from a test result), cite the evidence from your tables or the course's pinned values, and write a corrected sentence beneath it. This route assesses the same objectives (Objective 7 and Objective 4) as the chatbot critique, takes comparable effort (about 30 minutes and 200–300 words), is scored by the same scoring criteria for the same credit, and its own submission artifact is the marked-up flawed analysis, posted in place of the chatbot share link and report.

Part 7 — What to Submit

• All four completed scaffold tables
• The bar chart of the null model with the tail marked and named, as a chart, a labelled sketch, or a written description
• Written answers to all five analysis questions
• The chatbot critique report with its share link, or the equivalent route's marked-up flawed analysis

Deliverable:

A short Data Lab report on the guessing quiz: four completed tables (the null model's eleven chances and its tail chances; your 1,000 simulated guessers beside the model; the test from the hypotheses to a conclusion; and two rejection rules compared by their Type I error rates), a bar chart of the null model with the tail at 7 or more marked and named in words, or a labelled sketch or a written description of it, written answers to five analysis questions, and a critique of a chatbot's reasoning about the same test, or, on the equivalent route, a marked-up correction of the printed flawed analysis. Submit in any accessible file type or as a link; a recorded response may be submitted as audio or video with a transcript, or as text. Choose a written, recorded, or visual form for your answers; every form is graded by the same scoring criteria.

Rubric

The rubric below shows how each submitted part is evaluated; bands are full, partial, or none.

Workshop evaluation criteria

Criterion	Full	Partial	None
Building and checking the null model	All eleven chances P(X = k) and the P(X ≥ k) column are recorded to four decimals from the BINOM.DIST formulas or the binomial formula with the given ways, and the chances are checked to add to 1.	The model is built, but one or two chances or tail values are wrong or missing, or the check that the chances add to 1 is left out.	No model table is recorded, or its chances are invented rather than computed.
Picturing the null model and its tail	The bar chart, labelled sketch, or written description shows one bar per score from 0 to 10, names what each axis measures, and marks and names the tail at 7 or more in words.	The picture is present, but an axis is unnamed or the tail is not marked and named.	No chart, sketch, or description of the null model is submitted.
Simulated guessers compared with the model	1,000 guessers are simulated and frozen as values, or the fallback run is used; the average and both counts are recorded beside the model's expected values, and analysis question 2 describes how close each is and the direction of each miss.	The simulation is summarized, but one quantity or the comparison with the model is missing.	No simulation summary or comparison is submitted.
Carrying out and interpreting the test	The third table and analysis questions 1 and 3 state H₀ and Hₐ about the parameter p, give the test statistic and the p-value both ways, write the p-value in the course sentence, reject H₀ at α = 0.05 with a conclusion in context that stops short of proof, and name two things the p-value is not.	The test is mostly complete, but the hypotheses are written about the sample, the p-value is computed one way only, the decision is missing, or the interpretation calls the p-value the chance that the student guessed.	The test steps are missing, or the p-value is not computed from the guessing model.
Weighing the two kinds of error	The fourth table and analysis question 4 give both Type I error rates, say the stricter rule's rate is about 22 times smaller, and explain in words why the stricter rule misses more students who know some answers, so β rises and power falls.	The error rates are recorded, but the comparison or the Type II explanation is missing or reverses the trade-off.	The error comparison is missing.
Critique of the chatbot or of the printed flawed analysis	On either route, at least two errors are quoted and corrected with evidence from the student's own tables or the course's pinned values, including either the dropped number of ways or the misread p-value, and one correct point is acknowledged; both routes earn this band equally.	On either route, errors are named but corrected without evidence, or only one error is caught.	No critique appears, or pasted output is submitted without any checking.
Reflection on what the test can and cannot show	The chosen reflection option is answered in 3–4 sentences that name a specific limit, such as fail to reject not proving guessing or a broken assumption of the guessing model, and tie it to this lab's test.	The reflection names a limit in general terms without tying it to this lab's test.	No reflection is submitted.

Quality Gate

• Every chance, count, and p-value comes from your own formulas, the binomial formula with the given ways, or the fallback run, never from memory or a chatbot.
• The eleven model chances add to 1, and the p-value found by adding the tail matches the complement formula to four decimals.
• The simulated scores are pasted as values, so they match the tables you submit.
• The hypotheses are about the parameter p, the decision compares the p-value with α = 0.05 chosen in advance, and no sentence calls the p-value the chance that the student was guessing.
• The chart, sketch, or description names both axes and the tail at 7 or more in words.
• No statistic, count, or source is invented anywhere in the submission.
• The critique quotes the chatbot or the flawed analysis and corrects at least two errors with evidence from your tables or the course's pinned values.