Week 13 — Was the Student Guessing? A Short Significance-Test Report

Assignment 12 of the term — find p-values from the guessing-quiz table, decide at a significance level chosen in advance, describe the two ways a test can be wrong, separate statistical from practical significance, and correct a chatbot's misreadings of a p-value.

This adaptive assignment uses a coaching dialogue to help you turn a first attempt into a clear, evidence-based report. You remain the author. The coach asks questions, checks the reasoning you show, and waits for you to revise before it produces the required report. In Week 7 you found binomial chances for a student guessing on a 10-question quiz; this week the same table becomes a test of a claim. This week's lecture tutorial, quiz, discussion, and Data Lab practise the same ideas from other angles; this assignment is where you put them together.

The task

Several students each say they guessed at random on every question of a quiz with four options per question. If a student is guessing, then p, the student's true chance of a correct answer on each question, is 1 ⁄ 4 = 0.25. On a 10-question quiz the count of correct answers X is then binomial with n = 10 and p = 0.25. Its null mean is np = 10 × 0.25 = 2.5 correct answers, and its standard deviation is about 1.37 correct answers. The guessing-quiz table (Dataset D) gives P(X = k) for k = 0, 1, 2, …, 10 correct answers, each rounded to four decimals: 0.0563, 0.1877, 0.2816, 0.2503, 0.1460, 0.0584, 0.0162, 0.0031, 0.0004, 0.0000, 0.0000. For every test below, use the significance level α = 0.05, chosen before anyone looked at a quiz.

Work these four problems, showing your arithmetic:

1. A student who says they guessed on all 10 questions of a four-option quiz scored 7. In the guessing-quiz table, P(X = 7) = 0.0031, P(X = 8) = 0.0004, and P(X = 9) and P(X = 10) both round to 0.0000. What is the p-value P(X ≥ 7), to four decimals?
2. A second student who says they guessed scored 5 on the same quiz. In the guessing-quiz table, P(X = 5) = 0.0584, P(X = 6) = 0.0162, P(X = 7) = 0.0031, P(X = 8) = 0.0004, and P(X = 9) and P(X = 10) both round to 0.0000. What is the p-value P(X ≥ 5), to four decimals?
3. A third student who says they guessed got 4 of the 10 questions right. In the guessing-quiz table, P(X ≤ 2) = 0.5256 and P(X = 3) = 0.2503. What is the p-value P(X ≥ 4) = 1 − (P(X ≤ 2) + P(X = 3)), to four decimals?
4. On a 100-question version of the four-option quiz, a pure guesser's count of correct answers has null mean 25 and standard deviation 4.3301. What count sits 3 standard deviations above the null mean, 25 + 3 × 4.3301?

Beside each answer, add one brief interpretation:

• Problem 1. State H₀ and Hₐ in symbols and in words about p, and say why the alternative is one-sided. Give your decision at α = 0.05 and a conclusion in context. Write the course's p-value sentence: "if the student were guessing, a score this high would occur about … of the time". Then correct Claim 1 below.
• Problem 2. Give your decision at α = 0.05 and say what it does and does not show about the second student, correcting Claim 2 below. Then take the rule "reject H₀ when the quiz has 5 or more correct answers". Describe a Type I error and a Type II error for that rule in the quiz's own terms, say which error your problem 2 value is the probability of, and explain what happens to each error if the cutoff rises to 7.
• Problem 3. Give your decision at α = 0.05, write the course's p-value sentence for this student, and name one limit of a 10-question quiz for telling a small edge apart from pure guessing.
• Problem 4. A fourth student gets 40 of the 100 questions right. Both np = 25 and n(1 − p) = 75 are at least 10, so a normal model for the count is reasonable. The empirical rule says about 99.7% of a normal model lies within 3 standard deviations of its mean. Compare 40 with the count you found, and use the empirical rule to say roughly how rare 40 or more correct answers would be for a guesser. Then explain why 4 right out of 10 and 40 right out of 100, the same proportion of correct answers, lead to such different p-values, and say whether that proportion is a large edge over guessing. Correct Claim 3 below.

Check a chatbot's answer

A classmate asked a chatbot to explain these results. Its answer sounds confident, but each of its three claims misreads a significance test. Quote each claim in your report, say what is wrong with it, and rewrite it so it is correct.

• Claim 1: "The p-value for the score of 7 is the probability that the student was guessing. It is tiny, so the student almost certainly studied."
• Claim 2: "The second student's p-value is above 0.05, so the test proves that student was guessing."
• Claim 3: "The fourth student's p-value is far smaller than the third student's, so the fourth student knows far more of the material."

The American Statistical Association warns that a p-value is not the probability that the null hypothesis is true and does not measure the size or importance of an effect. Use that warning, and your own numbers, as evidence.

Explain it to a non-expert

A volunteer at a campus tutoring center has never studied statistics. After hearing about the fourth student, the volunteer says, "A p-value that small means this student really knows the material, right?" Write a 3–5 sentence message that answers the volunteer in everyday words, gives a reason tied to the quiz numbers, and names one limit — for example, that a p-value does not measure how much a student knows, or that a test cannot show why a student did well. Explain any technical term, such as "p-value" or "statistically significant", in everyday words. More than one good answer is possible.

For a different, ungraded example, imagine a game-night host who reads that a test of a die suspected of being loaded, with H₀: P(six) = 1 ⁄ 6, found no significant evidence against a fair die, and who asks whether the die is fair. Model answer: "Not necessarily. The test checked whether the rolls were too unusual for a fair die, and they were not, so the rolls are consistent with a fair die. A die that is only a little off could give rolls like these too, so the test cannot prove the die is fair. Rolling it many more times would give a sharper answer."

Use that pattern for the volunteer, with your own explanation and evidence.

Part 1 — Student instructions

1. Draft all four answers, the four brief interpretations with your corrections of the three claims, and your message to the volunteer before opening a chatbot.
2. Open any course-approved chatbot. Copy the entire Coach Prompt box, including its first and last marker lines, and paste it as one message. The coach will ask for your draft and then work one question at a time.
3. Answer in your own words. Show calculations when asked. Use the coach's feedback to revise at least one calculation or explanation, even if your first attempt was correct.
4. When the coach asks you to reflect on your revision, choose one of two options of equal weight: (a) explain which change made your report more accurate and what evidence prompted it, or (b) explain which of the chatbot's three claims you might have believed before this week and what in your own work now shows it is wrong.
5. Continue until the prompt's exit condition is met.
6. Save the chat's share link and copy the final report exactly as the coach gives it.

The chatbot is a coach, not a ghostwriter. It must draw out your reasoning rather than write the report for you.

Coach Prompt

COPY EVERYTHING BELOW THIS LINE
ROLE: You are a careful learning coach. Draw out and challenge MY thinking about null and alternative hypotheses, p-values from a binomial table, decisions at a significance level, Type I and Type II errors, and statistical versus practical significance. Do not write my assignment for me and do not invent work I did not show.

ASSIGNMENT: Work through these four problems in order, one at a time. Do not invent, replace, or reorder a problem. The data: if a student guesses on a 10-question quiz with four options per question, the count of correct answers X is binomial with n = 10 and p = 0.25, with null mean 2.5 and standard deviation about 1.37. The guessing-quiz table gives P(X = k) for k = 0 to 10, rounded to four decimals: 0.0563, 0.1877, 0.2816, 0.2503, 0.1460, 0.0584, 0.0162, 0.0031, 0.0004, 0.0000, 0.0000. On a 100-question version the null mean is 25 and the standard deviation is 4.3301. The significance level is alpha = 0.05, chosen in advance.
PROBLEM P1: A student who says they guessed on all 10 questions of a four-option quiz scored 7. In the guessing-quiz table, P(X = 7) = 0.0031, P(X = 8) = 0.0004, and P(X = 9) and P(X = 10) both round to 0.0000. What is the p-value P(X ≥ 7), to four decimals?
PROBLEM P2: A second student who says they guessed scored 5 on the same quiz. In the guessing-quiz table, P(X = 5) = 0.0584, P(X = 6) = 0.0162, P(X = 7) = 0.0031, P(X = 8) = 0.0004, and P(X = 9) and P(X = 10) both round to 0.0000. What is the p-value P(X ≥ 5), to four decimals?
PROBLEM P3: A third student who says they guessed got 4 of the 10 questions right. In the guessing-quiz table, P(X ≤ 2) = 0.5256 and P(X = 3) = 0.2503. What is the p-value P(X ≥ 4) = 1 − (P(X ≤ 2) + P(X = 3)), to four decimals?
PROBLEM P4: On a 100-question version of the four-option quiz, a pure guesser's count of correct answers has null mean 25 and standard deviation 4.3301. What count sits 3 standard deviations above the null mean, 25 + 3 × 4.3301?
After the four answers, I must briefly interpret each one. For P1: H0 and Ha in symbols and words about p, why Ha is one-sided, the decision at alpha = 0.05, a conclusion in context, the course sentence "if the student were guessing, a score this high would occur about ... of the time", and a correction of Claim 1. For P2: the decision at alpha = 0.05 with a correction of Claim 2, then a Type I error and a Type II error for the rule "reject H0 when the quiz has 5 or more correct answers" in the quiz's own terms, which error the P2 value is the probability of, and what happens to each error if the cutoff rises to 7. For P3: the decision, the course sentence for this student, and one limit of a 10-question quiz. For P4: a fourth student gets 40 of 100 right; I compare 40 with my P4 count, use the empirical rule (about 99.7 percent of a normal model lies within 3 standard deviations of its mean) to say how rare 40 or more would be for a guesser, explain why 4 of 10 and 40 of 100 give such different p-values, say how large an edge over guessing that proportion is, and rewrite Claim 3. The three claims I must correct are: Claim 1, "The p-value for the score of 7 is the probability that the student was guessing. It is tiny, so the student almost certainly studied." Claim 2, "The second student's p-value is above 0.05, so the test proves that student was guessing." Claim 3, "The fourth student's p-value is far smaller than the third student's, so the fourth student knows far more of the material." I must then write a 3-5 sentence message to a campus tutoring-center volunteer who is new to statistics and thinks a p-value that small means the fourth student really knows the material, answering in everyday words, giving a reason tied to the quiz numbers, and naming one limit. I must explain technical terms in everyday words.

SCORING CRITERIA: Score only evidence I supply: correct p-values and counts (40 points), test reasoning, decisions, and corrections (32 points), clarity for a non-expert reader (16 points), and showing work so it can be checked (12 points). For each of P1-P4, give 2, 1, or 0 evidence units per criterion; one unit is worth 5 computation points, 4 reasoning points, 2 clarity points, or 1.5 showing-work points. Sum all four problems within each criterion. Count my final message only with P4, not again with P1-P3.
COMPUTATION UNITS: A correct p-value to four decimals, or a correct count including a correct rounding to two decimals, = 2; the right values and operation with only an arithmetic slip = 1; a missing result or the wrong values or operation = 0. Do not penalize an alternative valid method, such as a spreadsheet BINOM.DIST check, or equivalent correct notation.
REASONING UNITS: For P1-P3 separately, an accurate account of the test step the problem asks for plus its paired correction or limit = 2; exactly one of the two = 1; neither = 0. For P1 the account is the hypotheses, the decision, and the course sentence, paired with the correction of Claim 1. For P2 the account is the decision with the correction of Claim 2, paired with the Type I and Type II errors for the rule and the trade-off when the cutoff rises. For P3 the account is the decision and the course sentence, paired with a limit of a 10-question quiz. For P4, BOTH a comparison of 40 with the P4 count that separates statistical from practical significance and corrects Claim 3 AND a final message with a reason tied to the quiz numbers and a named limit = 2. If full P4 evidence is absent but at least one accurate substantive part appears, give 1. No accurate reasoning = 0; naming a rule alone does not earn a unit. Calling a p-value the probability that H0 is true or the chance the score was a fluke, saying fail to reject proves H0, saying a smaller p-value means a bigger effect, choosing alpha after seeing the data, or giving beta a number does not count as accurate. An arithmetic slip alone is not a reasoning error.
CLARITY UNITS: For each problem, labeled evidence understandable to someone new to statistics, with technical terms explained = 2; intelligible evidence that needs rereading because of ambiguity or jargon = 1; absent or unintelligible evidence = 0. Include the final message with P4 if submitted. Missing reasoning alone does not lower clarity for otherwise clear submitted work.
SHOWING-WORK UNITS: For each problem, checkable values and steps = 2; a relevant but incomplete value or step = 1; a result alone or no method = 0. A visible method stays checkable despite an arithmetic slip.

Do NOT invent grading rules.

DIALOGUE RULES: Ask exactly one question per message and wait for my answer. Start by asking for my complete first attempt. Build each next question from my exact words. Probe an unsupported claim and ask me to examine its evidence. Keep the conversation focused on questions rather than completed work.

ENGAGEMENT GUARDS: A greeting, copied prompt, yes/no reply, or one-word answer is not a substantive exchange. Do not flatter or accept agreement as evidence. Praise only a concrete move I made. If evidence is missing, say what is missing and ask for it. Require a real revision plus a short reflection on it; let me choose either (a) which change made my report more accurate and what evidence prompted it, or (b) which of the three claims I might have believed before this week and what in my work now shows it is wrong. Do not issue the report early.

EXIT CONDITION: Continue until at least five substantive exchanges have occurred AND I have supplied all four answers and their brief interpretations AND corrected all three claims AND written my message to the volunteer AND revised at least one calculation or explanation AND reflected on the revision. Stop only after whichever requirement happens last.

REPORT: After the exit condition is satisfied, output exactly these labelled lines in this order. Replace X with the exact earned score; it may end in .5 and must not be rounded to a whole number. Base every line only on evidence in this chat; write "Not supplied" for missing evidence and never raise the score for polished chatbot prose.
STUDENT'S SCORE: X/100
EVIDENCE FOR THE SCORE: [specific evidence from my calculations and reasoning]
REVISION MADE: [what I changed and why]
STRENGTH: [one earned strength]
NEXT STEP: [one concrete improvement]
Then tell me to submit this report, my revised significance-test report, and the share link.
COPY EVERYTHING ABOVE THIS LINE

Deliverables

Submit these three items in the assignment entry:

• your revised significance-test report with all four answers, their brief interpretations, your corrections of the three claims, and your message to the volunteer — written, or recorded as audio or video with a transcript, or as a labelled sketch of the guessing-quiz distribution with each tail you used shaded and a short note; every form is graded by the same scoring criteria;
• the coach's five-line report; and
• the conversation share link.

Submit in any accessible file type or as a link; a recorded response may be submitted as audio or video with a transcript, or as text.

Equivalent route (no AI required)

This route assesses the same Week 13 skills — stating a null and an alternative hypothesis, finding a p-value from the guessing-quiz table, deciding at a significance level chosen in advance, describing Type I and Type II errors in context, telling statistical from practical significance, and correcting misreadings of a p-value — for comparable time and effort, about 60 to 90 minutes, the same as the coached route. Complete the same four problems, the four brief interpretations, the corrections of the three claims, and the message to the volunteer. Then audit your own first draft against the attached scoring criteria, criterion by criterion, and write a second draft. Add a 150–200 word revision memo that names one calculation or explanation you changed and the evidence that prompted the change, and then answers one of two options of equal weight: (a) which change made your report more accurate and why, or (b) which of the three claims you might have believed before this week and what in your work now shows it is wrong. Submit the revised report, your completed criteria checklist, and the revision memo in the assignment entry; together they replace the share link and the coach's report. This route uses the same scoring criteria and the same points, with no penalty for choosing it.

How you'll be graded

The full analytic rubric is attached to this assignment in Canvas.

Academic integrity and AI use

The calculations, reasoning, corrections, revision, and final report must be your own. Do not ask the chatbot to create the assignment or add evidence you did not produce. Submit an honest share link from your own dialogue. The equivalent route keeps the same workload and evidence and requires no AI account.

Built with The Course Maker · samples.thecoursemaker.com