Week 9 — How Far Does the Survey Reach? Testing a Sample and an Analogy in an Ad

Assignment 8 of the term — check who a survey actually reached, judge a generalization as strong or weak, catch a story standing in for a trend, test a comparison, and explain to someone new why a glowing survey and a catchy comparison do not show the plan works for most people.

This adaptive assignment uses a coaching dialogue to help you turn a first attempt into a clear, evidence-based evaluation. You remain the author. The coach asks questions, checks the reasoning you show, and waits for you to revise before it produces the required report. In Week 3 you learned to judge an inductive argument as strong or weak and as cogent or not; this week you learned to ask who was in a sample, how they got there, and whether a comparison's likeness bears on its conclusion. This week's lecture tutorial, quiz, discussion, and Argument Workshop practise the same ideas from other angles; this assignment is where you put them together on one short text.

The task

Your text is an illustrative ad for a fitness app, with no real app, company, or member behind it: "Nine in ten of our members love the new workout plan! We emailed a survey to all 4,000 members, 400 replied, and 360 of them said they love it. One member finished her first marathon after three months on the plan, so it will get anyone race-ready. Using our app is like hiring a personal trainer: a trainer plans your workouts, and so does our app, so you'll get the results a trainer gets."

Work these four problems, showing your arithmetic and quoting the sentences you use:

1. The ad says it emailed a survey to all 4,000 members and 400 replied. What percentage of the members replied?
2. Of the 400 members who replied, 360 said they love the plan. What percentage of the replies said they love it?
3. Of the 4,000 members the survey was emailed to, how many did not reply?
4. Of the 400 replies, how many did not say they love the plan?

Beside each answer, add one brief interpretation:

• Problem 1: name the population and the sample the ad's first sentence is about, name the trap in how the sample came together, and explain why the members who replied might differ from those who did not.
• Problem 2: write the ad's first sentence as an inductive generalization (what was observed, and what it concludes about all members), judge it strong or weak, and say whether it could be cogent if its premise is true.
• Problem 3: name the fallacy in the marathon sentence, count how many cases it rests on, and describe the kind of evidence that would bear on whether the plan gets people race-ready.
• Problem 4: run the three-step analogy test on the personal-trainer comparison: list a similarity that bears on the conclusion, name a disanalogy, and give your verdict, then write the message to the friend described below.

Explain it to a non-expert

A friend who has never studied logic sees the ad and asks, "Nine in ten members love it and it's like having a personal trainer, so I should sign up, right?" Write a 3–5 sentence message that explains in everyday words why the survey does not show that nine in ten of all members love the plan and why the trainer comparison is not proof, reports the percentage of members who replied and how many members did not reply, and names one limit: nobody knows how the silent members feel, so the plan could be better or worse for them than the ad suggests. Explain any technical term in everyday words. More than one good answer is possible.

For a different, ungraded example, imagine a café sign that says "Our online reviewers give us top marks, so you'll love it here." Model answer: "Those reviews come only from people who chose to write one, and the happiest and the angriest customers are the most likely to bother. That makes the reviewers a self-selected group, not a fair picture of every customer. Before deciding, try to find out how many customers never left a review, or ask a few friends who have eaten there."

Use that pattern for the friend, with your own wording and evidence.

Part 1 — Student instructions

1. Draft all four answers, the four brief interpretations, and your message to the friend before opening a chatbot.
2. Open any course-approved chatbot. Copy the entire Coach Prompt box, including its first and last marker lines, and paste it as one message. The coach will ask for your draft and then work one question at a time.
3. Answer in your own words. Show calculations, write each trap and fallacy name out in full, and quote the sentences you use when asked. Use the coach's feedback to revise at least one calculation or explanation, even if your first attempt was correct.
4. Continue until the prompt's exit condition is met.
5. Save the chat's share link and copy the final report exactly as the coach gives it.

The chatbot is a coach, not a ghostwriter. It must draw out your reasoning rather than write the work for you. If it says 400 replies are enough to speak for every member because 400 is a large number, or treats the trainer comparison as proof, say so and ask it why.

Coach Prompt

COPY EVERYTHING BELOW THIS LINE
ROLE: You are a careful learning coach for an introductory critical thinking course. Draw out and challenge MY thinking about samples and generalizations: the population and the sample, the three ways a sample can mislead, nonresponse, strength and cogency, the hasty generalization, and arguments from analogy. Do not write my assignment for me, do not invent work I did not show, never invent a statistic, a study, a survey figure, or a quotation, never name a real person, town, school, company, app, or product, and never use logic symbols. Do not teach correlation and causation or percentage points.

ASSIGNMENT: Work through these four problems in order, one at a time. Do not invent, replace, or reorder a problem. The text is an illustrative fitness-app ad: "Nine in ten of our members love the new workout plan! We emailed a survey to all 4,000 members, 400 replied, and 360 of them said they love it. One member finished her first marathon after three months on the plan, so it will get anyone race-ready. Using our app is like hiring a personal trainer: a trainer plans your workouts, and so does our app, so you'll get the results a trainer gets."
PROBLEM P1: The ad says it emailed a survey to all 4,000 members and 400 replied. What percentage of the members replied?
PROBLEM P2: Of the 400 members who replied, 360 said they love the plan. What percentage of the replies said they love it?
PROBLEM P3: Of the 4,000 members the survey was emailed to, how many did not reply?
PROBLEM P4: Of the 400 replies, how many did not say they love the plan?
After the four answers, I must briefly interpret each one: for P1, name the population and the sample, name the trap in how the sample came together, and explain why the repliers might differ from those who did not reply; for P2, write the ad's first sentence as an inductive generalization, judge it strong or weak, and say whether it could be cogent if its premise is true; for P3, name the fallacy in the marathon sentence, count its cases, and describe evidence that would bear on the claim; for P4, run the three-step analogy test on the personal-trainer comparison with a similarity, a disanalogy, and a verdict. I must then write a 3-5 sentence message to a friend explaining why the survey does not show nine in ten of all members love the plan and why the trainer comparison is not proof, reporting the percentage of members who replied and the number who did not reply, and naming one limit. I must explain technical terms in everyday words.

SCORING CRITERIA: Score only evidence I supply: correct computations (20 points), sampling and analogy evaluation (48 points), clarity for a non-expert reader (16 points), and showing the evidence so it can be checked (16 points). For each of P1-P4, give 2, 1, or 0 evidence units per criterion; one unit is worth 2.5 computation points, 6 evaluation points, 2 clarity points, or 2 evidence points. Sum all four problems within each criterion. Count my message to the friend only with P4, not again with P1-P3.
COMPUTATION UNITS: A correct percentage of members who replied, percentage of replies that said they love the plan, count of members who did not reply, or count of replies that did not say they love it = 2; the right values and operation with only an arithmetic slip = 1; a missing answer or the wrong values or operation = 0. Do not penalize an alternative valid method or equivalent correct notation.
EVALUATION UNITS: For P1-P3 separately, an interpretation that meets every part named for that problem = 2; at least one part but not all = 1; none = 0. For P4, BOTH the three-step analogy test run on the trainer comparison with a similarity, a disanalogy that bears on the results, and a verdict AND a message to the friend that explains why the survey and the comparison do not show the plan works for most members, reports the percentage and the count, and names a limit = 2. If full P4 evidence is absent but at least one accurate substantive part appears, give 1. No accurate evaluation = 0; naming a term alone does not earn a unit, calling the 400 replies a random sample is not accurate, and calling the generalization valid is not accurate. An arithmetic slip alone is not an evaluation error.
CLARITY UNITS: For each problem, labeled evidence understandable to someone new to critical thinking, with technical terms explained = 2; intelligible evidence that needs rereading because of ambiguity or jargon = 1; absent or unintelligible evidence = 0. Include the message to the friend with P4 if submitted. Missing reasoning alone does not lower clarity for otherwise clear submitted work.
EVIDENCE UNITS: For each problem, checkable values and steps, with every quoted sentence matching the ad as printed = 2; a relevant but incomplete value, step, or quotation, or a quotation that drifts from the text = 1; an answer alone or no method = 0. A visible method stays checkable despite an arithmetic slip.

Do NOT invent grading rules.

DIALOGUE RULES: Ask exactly one question per message and wait for my answer. Start by asking for my complete first attempt. Build each next question from my exact words. Probe an unsupported claim and ask me to examine its evidence. When I name a trap, ask me who had a chance to end up in the sample and who did not, and when I judge the trainer comparison, ask me whether the difference I named bears on the results the ad promises. Keep the conversation focused on questions rather than completed work.

ENGAGEMENT GUARDS: A greeting, copied prompt, yes/no reply, or one-word answer is not a substantive exchange. Do not flatter or accept agreement as evidence. Praise only a concrete move I made. If evidence is missing, say what is missing and ask for it. Require a real revision plus a short reflection on why the revision is better. Do not issue the report early.

EXIT CONDITION: Continue until at least five substantive exchanges have occurred AND I have supplied all four answers and their brief interpretations AND written my message to the friend AND revised at least one calculation or explanation AND reflected on the revision. Stop only after whichever requirement happens last.

REPORT: After the exit condition is satisfied, output exactly these labelled lines in this order. Replace X with the exact earned total; it may end in .5 and must not be rounded to a whole number. Base every line only on evidence in this chat; write "Not supplied" for missing evidence and never raise the total for polished chatbot prose.
STUDENT'S SCORE: X/100
EVIDENCE FOR THE SCORE: [specific evidence from my calculations, quotations, and reasoning]
REVISION MADE: [what I changed and why]
STRENGTH: [one earned strength]
NEXT STEP: [one concrete improvement]
Then tell me to submit this report, my revised work, and the share link.
COPY EVERYTHING ABOVE THIS LINE

Deliverables

Submit these three items in the assignment entry:

• your revised work with all four answers, their brief interpretations, and your message to the friend — written, or recorded as audio or video with a transcript, or as a labelled sketch that shows the 4,000 members, the 400 who replied, and the 360 who said they love the plan as nested groups, with a short note that says each verdict in words; every form is graded by the same scoring criteria;
• the coach's five-line report; and
• the conversation share link.

Submit in any accessible file type or as a link; a recorded response may be submitted as audio or video with a transcript, or as text.

Equivalent route (no AI required)

This route assesses the same Week 9 skills — naming a population and a sample, spotting nonresponse, judging a generalization as strong or weak and as cogent or not, naming a hasty generalization from one vivid anecdote, testing an argument from analogy, and explaining to someone new why a self-selected survey is not proof — for comparable time and effort, about 60 to 90 minutes, the same as the coached route. Complete the same four problems, the four brief interpretations, and the message to the friend. Then audit your own first draft against the attached scoring criteria, criterion by criterion, and check every quoted sentence against the ad as printed. Write a second draft and add a 150–200 word revision memo that names one calculation or explanation you changed, the evidence that prompted the change, and one question you still have. Submit the revised work, your completed criteria checklist, and the revision memo in the assignment entry; together they replace the share link and the coach's report. This route uses the same scoring criteria and the same points, with no penalty for choosing it.

How you'll be graded

The full analytic rubric is attached to this assignment in Canvas.

Academic integrity and AI use

The calculations, reasoning, revision, and final report must be your own. Do not ask the chatbot to create the assignment or add evidence you did not produce, and check any sentence it quotes against the ad as printed here. Submit an honest share link from your own dialogue. The equivalent route keeps the same workload and evidence and requires no AI account.

Built with The Course Maker · samples.thecoursemaker.com