Week 12 — Political Analysis Workshop · "Reading a Real National Poll"

Course: Introduction to Political Science (POLS 1) · Silver Oak University (fictional sample) · Prof. Halloran
Objective: Objective 6 — read and evaluate real political data (sampling, margin of error, correlation vs. causation) · SLO A (political analysis & source/data evaluation)
Worth 50 points · Political Analysis Workshops group = 15% of the grade · Workshop 12
Mode this week: political data. (Last week you read election results; this week you read a poll. Either way you'll end by catching an AI's mistakes.)

This is the course's signature weekly component. Every instructional week has one Political Analysis Workshop. This week's dataset is a real, current Pew Research Center release — the exact kind of national poll you'll see cited in the news for the rest of your life. All sources are links to external, authoritative reports — nothing to buy or download.

Part 1 — The Big Picture

This week you learned how public opinion is measured — random sampling, the margin of error, and the difference between MoE and outright bias. Now you'll run all of it on a real, current national poll, published just weeks before this term began.

The guiding question:

"What does this poll actually show about how Americans use and view AI — and how much should we trust any single number in it?"

A poll is powerful and precise about its own limits: a responsible pollster tells you exactly how confident to be. Your job is to read the release for what it actually says — sample, dates, numbers, and stated uncertainty — not for the headline alone.

Part 2 — The Data Source (read it first)

Release: Pew Research Center, "Americans and AI 2026: Chatbots, Smart Devices and Views on Impact." Publication date: June 17, 2026. Authors: Jeffrey Gottfried, William Bishop, Monica Anderson, Michelle Faverio, Eugenie Park, and Colleen McClain. Type: a nationally representative public-opinion survey report, published by a nonpartisan, nonadvocacy research organization.

Read the release at its authoritative source (links only):
- 🔗 Main report — Pew Research Center: https://www.pewresearch.org/internet/2026/06/17/americans-and-ai-2026-chatbots-smart-devices-and-views-on-impact/
- 🔗 Methodology (sample size, field dates, stated margin of error): https://www.pewresearch.org/internet/2026/06/17/americans-and-ai-methodology/
- 🔗 Full report PDF: https://www.pewresearch.org/wp-content/uploads/sites/20/2026/06/PI_2026.06.17_Americans-and-AI_REPORT.pdf

The core facts — pre-verified live 2026-07-02, directly against the pages above:

Two topline figures you'll analyze (quoted/paraphrased exactly from the release):
- Figure A: "A little under half of U.S. adults (44%) now report using [ChatGPT], up from 34% last year." (2025 → 2026 comparison.)
- Figure B: "Americans largely think AI is moving too fast. About two-thirds say this. Only 2% say it's advancing too slowly."

A polished poll release still has a purpose: Pew is a nonpartisan fact tank, but every release is also written to be read and shared — headlines are chosen, some findings get more emphasis than others. Reading it as a political scientist means checking the methodology section, not just the headline.

Part 3 — Read-the-Data Scaffold (fill this in)

Complete each box in a sentence or two. This is the heart of the workshop.

Part 4 — Analysis Questions

Answer in a few sentences each:
1. The margin of error, computed and applied: Using MoE = 1.96 × √(0.25/n), compute the margin of error for a simple random sample of n = 5,119 (show your arithmetic). How does your computed value compare to Pew's own stated ±1.6 points? If they differ, what's the most likely reason (hint: Segment 3's "honest complication")?
2. Reading a close number vs. a wide one: Figure B reports "about two-thirds" say AI is advancing too quickly, against just 2% who say too slowly. Given the ±1.6-point margin of error, is this finding one where the margin of error meaningfully changes how you'd interpret the result? Contrast this with a hypothetical close finding (say, 51% vs. 49%) — why would the margin of error matter more there?
3. Correlation vs. causation, applied: The release shows chatbot use rising alongside widespread public worry about AI's pace and its effect on data security. Would it be accurate to say the data proves rising use is causing rising worry, or worry is causing something else? What kind of evidence (beyond this one release) would you need to make a causal claim?
4. Sampling method: This poll used Pew's American Trends Panel, recruited through address-based sampling (a random method). Contrast this with an online click-poll on a news website that asks "vote now: is AI moving too fast?" Which one can compute a trustworthy margin of error, and why?
5. The reach and the limits: This release describes attitudes of U.S. adults nationally as of one week in February 2026. What is one way public opinion on a fast-moving topic like AI could plausibly look different by the time you read this — and what would a political scientist need to do to find out (rather than assume)? (Answer analytically — the fact that opinion CAN shift is documented survey methodology; predicting the DIRECTION of any future shift is a genuinely different, much harder claim, and thoughtful people should be cautious about it.)

Part 5 — AI-Critique Moment (required — this is the BYOAI step)

Now bring in your approved chatbot (Gemini, Claude, or ChatGPT) and be the political scientist who checks its work.

1. Ask it: "Tell me about the most recent Pew Research Center survey on how Americans view artificial intelligence — give me its sample size, its margin of error, its field dates, and its main findings."
2. Check everything it says against the real release linked in Part 2:
   - Did it give the correct sample size (5,119) and field dates (Feb. 17–23, 2026) — or invent plausible-sounding but wrong numbers? Chatbots frequently produce a confident, precise-sounding figure that appears nowhere in the actual methodology page.
   - Did it state the correct margin of error (±1.6 points, from Pew's own methodology page) — or did it just recompute the textbook formula (≈±1.4) and present that as "the" margin of error, skipping the real release's own stated figure?
   - Did it quote a topline percentage accurately (44% ChatGPT use, up from 34%; "about two-thirds" say AI is advancing too quickly) — or round/inflate it in a way that changes the meaning (e.g., "over half" instead of "a little under half")?
   - Did it slide from correlation into causation anywhere — for instance, claiming rising chatbot use is "because" trust in AI companies is rising (the release actually shows confidence in companies and government is falling, not rising, alongside use) — or otherwise assert a cause that the poll's design (a single cross-sectional snapshot) cannot establish?
3. Write 3–4 sentences reporting what the AI got right and at least one thing you had to correct, verify, or flag — a wrong sample size or margin of error, a misquoted percentage, or a correlation-as-causation slip. (If it happened to get everything right, explain how you verified each claim against the source — that's the skill.)

The habit all term: the tool drafts, you verify against the source. A chatbot will hand you a "margin of error" or a poll percentage that sounds exactly right and doesn't match the actual release — catching it is the point.

Part 6 — What to Submit

Submit a single document (or text entry) with: your completed Part 3 scaffold (all six moves), your Part 4 answers (including your Question 1 arithmetic, shown step by step), and your Part 5 AI-critique paragraph (naming the specific thing you checked). Due Sunday, Nov 22, 11:59 p.m. (50 points).

Instructor answer key & model responses — REMOVE BEFORE PUBLISHING TO STUDENTS

Every fact and figure below is verified against Pew Research Center's own report and methodology pages (verified live 2026-07-02), and the margin-of-error arithmetic is re-run in Python.

Part 3 scaffold (model):
- ① What is measured? Self-reported attitudes and behavior — whether respondents say they use AI chatbots, how they think AI will affect them and society, and their confidence in government/companies to handle it responsibly. This is not a direct behavioral measurement (like server logs of actual chatbot use) — it's what people report about themselves, which is a real (and standard, and honestly labeled) limitation of any self-report survey.
- ② Over what population and period? U.S. adults nationally (via Pew's American Trends Panel, not just tech-forward users or any single age group), surveyed over a one-week field period, Feb. 17–23, 2026 — a snapshot, not a permanent fact; opinion on a fast-moving topic like AI can shift.
- ③ What does it show? Figure A: 44% of U.S. adults report using ChatGPT, up from 34% the year before. Figure B: roughly two-thirds (about 67%, per the "about two-thirds" framing) say AI is advancing too quickly, versus only 2% who say too slowly.
- ④ What does it NOT show? It does not show that this pattern will persist unchanged; it does not show WHY usage rose (see ⑤); and it does not, by itself, tell us whether the public's judgment that AI is "moving too fast" is correct — that's a normative-adjacent claim the poll can describe attitudes about but can't settle.
- ⑤ Correlation or causation? The release shows chatbot use rising in the same period that worry about AI's pace and skepticism of government/corporate AI regulation are also elevated — but a single cross-sectional survey cannot establish that one caused the other, or rule out a third factor (e.g., broader media coverage of AI driving both increased exposure/use AND increased awareness of risks simultaneously). Claiming a cause would require additional evidence: a study tracking the same individuals over time (panel/longitudinal data), or a design that can rule out confounding factors — not just two numbers moving together in one snapshot.
- ⑥ Margin of error / how sampled? Stated MoE = ±1.6 percentage points at 95% confidence (Pew's own methodology page). The sample was randomly recruited via address-based sampling into the American Trends Panel — not self-selected — which is exactly what makes a trustworthy margin of error computable in the first place.

Part 4 (expected):
1. MoE arithmetic: MoE = 1.96 × √(0.25/5,119) = 1.96 × √0.0000488... = 1.96 × 0.00699 ≈ 0.0137 → ±1.4 percentage points (re-verified in Python: 1.370%, rounds to ±1.4). Pew's own stated figure is ±1.6 points — slightly larger than the simple-formula value. The most likely reason (per Segment 3's "honest complication"): real-world design effects — weighting to population benchmarks, an oversample of Asian adults that gets weighted back, and mixed-mode (online + phone) data collection all add some imprecision beyond what the simple random-sample formula captures. The lesson: always cite the release's own stated MoE, not just your own recomputation of the textbook formula.
2. Close vs. wide numbers: with a ±1.6-point margin of error, a finding of "about two-thirds vs. 2%" is nowhere close to being affected by that margin — the gap (roughly 65 percentage points) utterly swamps ±1.6 points either way, so this is a genuinely robust, high-confidence finding. A hypothetical 51%-vs-49% split, by contrast, sits ENTIRELY inside a ±1.6-point (or even a larger) margin of error — the "true" population value could plausibly be a tie, or even reversed, and reporting it as a clear majority would be misleading. Full credit requires students to draw exactly this contrast, not just restate the margin of error's definition.
3. Correlation vs. causation: No — the release's data does not prove a causal direction. Rising use and rising worry appearing together is a correlation in one time period; establishing causation would require, at minimum, tracking the same respondents over time (to see which change preceded the other) or a research design that can rule out a common third cause (e.g., overall media saturation about AI driving both more exposure/use and more awareness of downsides at once).
4. Sampling method: Pew's American Trends Panel is a randomly recruited panel (address-based sampling gives every sampled household a known chance of selection) — a trustworthy margin of error CAN be computed for it. An online click-poll samples only people who chose to click and vote — a self-selected, non-random sample — for which no formula can produce a meaningful margin of error, regardless of how many people click.
5. Strong answers note that opinion on a fast-moving technology topic is well-documented to shift as the technology, its coverage, and personal experience with it all change — this is a general, well-documented property of survey research on emerging topics, not a specific prediction about which direction AI attitudes will move. A political scientist wanting to know the CURRENT state of opinion, rather than assume continuity from a Feb. 2026 snapshot, would need to find a more recent release (checking Pew's topic page or field dates) rather than treat any one poll as permanently current. All positions on how confidently to extrapolate get graded on reasoning, not verdict.

Part 5 (AI-critique): full credit for a specific catch — most commonly the AI inventing or misstating the sample size or field dates, stating the simple-formula MoE (≈±1.4) as if it were Pew's own reported figure instead of the real, larger, design-effect-adjusted ±1.6, misquoting a topline percentage, or sliding into an unsupported causal claim about why chatbot use or AI-related worry rose. Full credit also if the student verified each AI claim against the linked release and methodology page and reported how.

Grading rubric — 50 points

Quality gate (self-checked) — Fact-and-source-accuracy gate: PASS. The release's title ("Americans and AI 2026: Chatbots, Smart Devices and Views on Impact"), publication date (June 17, 2026), field dates (Feb. 17–23, 2026), sample size (n = 5,119, American Trends Panel, address-based-sampling recruitment), and stated margin of error (±1.6 percentage points at 95% confidence) are all verified directly against Pew Research Center's own report and methodology pages (pewresearch.org/internet/2026/06/17/americans-and-ai-2026-chatbots-smart-devices-and-views-on-impact/ and .../americans-and-ai-methodology/), verified live 2026-07-02. Both cited topline figures (44%/34% ChatGPT use; "about two-thirds"/2% on AI's pace) are quoted/paraphrased exactly from the release text. The margin-of-error arithmetic (1.96 × √(0.25/5,119) ≈ ±1.4 points) was independently re-computed in Python and matches the key's stated value; the gap to Pew's own reported ±1.6 is correctly attributed to real-world design effects, not an error. No fabricated statistic, "study," or source appears anywhere in this workshop. Evenhandedness check — PASS: the workshop reports the poll's documented findings plainly (as data, not both-sidesed), while explicitly keeping the normative question ("is the public right to worry?") and any causal story separate from what the empirical release itself establishes; the AI-critique targets fabricated figures and correlation-as-causation — not any partisan framing, since AI attitudes were selected as a non-hot-button topic per this build's brief.