Week 10 — AI Build Studio · "Hallucination Hunt"

Course: Using Artificial Intelligence (AI 101) · Silver Oak University (fictional sample) · Prof. Quinn
Objective: Objective 4 — run a systematic verification workflow; recognize the five hallucination shapes; counter sycophancy · SLO A (produce well-verified results with AI) · SLO B (evaluate and verify AI critically)
Worth 50 points · AI Build Studios group = 15% of the grade · Studio 10
Format: a hands-on build — you'll deliberately elicit hallucinations, document what you find, run the full four-step verification workflow on each, and produce a verified Hallucination Hunt report.

This is the signature verification Studio for this course. Every week you've caught one AI mistake. This week you go hunting on purpose — the goal is to find hallucinations, name their shapes, and demonstrate that you can systematically verify AI output. The deliverable is a report that proves you know the difference between real and fabricated.

---

Part 1 — The Build Goal

By the end of this Studio you will have:
1. Elicited at least three hallucinations from an AI on a niche topic of your choice.
2. Documented the shape of each hallucination (from the Week 10 family: invented citation, fabricated statistic, fake case law, wrong arithmetic, fabricated quote).
3. Run the four-step verification workflow on each one and reported what was real versus fabricated.
4. Produced a short Hallucination Hunt Report summarizing your findings.

This is not about catching the AI being "bad" — it is about demonstrating that you can audit AI output systematically and know the difference. That skill travels to every professional and academic context you will enter.

Open two approved assistants to work in (you'll need both for the cross-check step):
- ChatGPT: https://chatgpt.com
- Claude: https://claude.com
- Gemini: https://gemini.google.com
- Copilot: https://copilot.microsoft.com

Free accounts are fine. Use any two of the four.

---

Part 2 — Choose Your Hunting Ground

Pick a niche topic you're genuinely curious about — something specific enough that you'd be unlikely to know the exact citations, statistics, or case law off the top of your head. This makes the hunt real: you're not testing what you already know, you're catching what the AI generates in an area where you'll need to verify.

Good hunting grounds (pick one, or choose your own):
- The history of a specific law or court ruling in your intended career field.
- Research on a specific health topic (e.g., the effects of a specific supplement, a specific therapy technique).
- Citation-heavy claims in your major's core debates (e.g., a landmark study in sociology, economics, or psychology).
- A historical claim from a period you don't know in depth.
- Technical specifications or standards in a field you find interesting.

Write your topic here: My hunting ground for this Studio is: ______.

---

Part 3 — Elicit the Hallucinations (the Hunt itself)

How to prompt for hallucinations. Hallucinations are most common when you ask for:
- Specific citations (author, title, journal, year)
- Specific statistics (percentages, counts, findings)
- Specific case law (case name, court, date, holding)
- Specific quotes attributed to real people
- Specific technical details in niche areas

Step 3a — Ask the primary AI for three things:
Use the prompting template below (adapt for your topic). Send it to your primary AI (your first assistant):

"I'm researching [your topic]. Give me: (1) two or three academic citations with full details (author, title, journal, year); (2) one or two statistics or research findings with percentages or numbers; and (3) one specific quote from a recognized expert or historical figure in this area."

Capture the full response. You'll verify each piece.

Step 3b — Record what you received. List each item:
- Citation 1: [paste what the AI gave you]
- Citation 2: [paste what the AI gave you]
- Statistic 1: [paste what the AI gave you]
- Statistic 2 (if given): [paste]
- Quote: [paste what the AI gave you — including who it's attributed to]

---

Part 4 — Run the Full Verification Workflow

For each item you received, run all four steps. Document what you find as you go — this documentation is your deliverable.

Step 1 — Ask for sources and check them.
For each item:
- Ask the AI: "What is your source for [this specific claim]? Are you certain this exists?"
- Capture its response.
- Then search Google Scholar (https://scholar.google.com) for each citation — use the exact title in quotation marks.
- For statistics, ask for the specific study or report, then search for it.
- For quotes, search the alleged text in quotation marks on Google.
- Record: found / not found / partially found (real author, wrong title, etc.).

Step 2 — Cross-check in a second model.
- Open your second approved assistant and ask the same question: "I'm researching [your topic]. Give me two or three academic citations with full details, one or two statistics, and a quote from a recognized expert."
- Compare what the second model gives you with what the first gave you.
- Do the citations match? Do the statistics agree? Does the quote appear from the same source?
- Record: where they agree, where they conflict.

Step 3 — Ask the AI to critique itself.
- Return to your primary AI and send: "Looking back at the citations and statistics you gave me — how confident are you that each one exists as you described it? Is any part of your answer something you might have generated rather than retrieved from a real source?"
- Record: whether the AI flags any uncertainty, doubles down, or walks anything back.

Step 4 — Verify in an authoritative external source.
- For each citation you couldn't confirm in Google Scholar, search your library database or try the journal's own search function.
- For statistics, search for the specific study on the organization's or publisher's own website.
- For the quote, search the author's own published works or a reliable biography.
- Record: confirmed real / confirmed fabricated / unclear (and why).

---

Part 5 — Your Hallucination Hunt Report

Write a short report using this structure (no minimum word count — be precise, not long):

Topic: [your niche topic]

What I found:

Item	What the AI gave me	Shape (if fabricated)	Verification result	How I verified
Citation 1	[paste]	[e.g., invented citation / real]	[real / fabricated / partial]	[what you searched, what you found]
Citation 2	[paste]
Statistic	[paste]
Quote	[paste]

Cross-check findings: Did the two models agree? Where did they conflict? What did the disagreements tell you?

Self-critique findings: Did the AI flag any uncertainty when asked? What did it say?

Summary (2–3 sentences): Of the items you verified, how many were real, how many were fabricated, and how many were partially real (e.g., real author, wrong paper)? What was the most common shape of hallucination you encountered?

---

Part 6 — Reflection (2–3 sentences)

What surprised you most about this Hunt — either about what was fabricated, or about how convincingly the AI presented the fabricated items? How will you approach AI-generated citations and statistics differently going forward?

---

Part 7 — What to Submit

Submit a single document (or text entry) with:
1. Your topic and the three prompting queries you used.
2. The raw AI responses (what each model gave you).
3. Your complete verification documentation (what you checked, where, and what you found for each item).
4. Your Hallucination Hunt Report table (Part 5).
5. Your reflection (Part 6).

Due Sunday, Nov 8, 11:59 p.m. (50 points)

---

Instructor answer key & model deliverable — REMOVE BEFORE PUBLISHING TO STUDENTS

Students use their own topics, so deliverables vary. Grade the process (quality of the hunt + verification rigor + accuracy of shape identification + honest reporting), not the specific items found. The model below shows what full credit looks like.

Model deliverable (illustrative — this exact scenario was not pre-run; this describes the general pattern):

• Topic: The history of compulsory education laws in the United States.
• Primary AI response: Gave three citations including one with a real-sounding author (a genuine education historian), a plausible title, and a real journal — but searching the title in Google Scholar returned no matching article. The author published on education history but this specific paper did not appear. Also gave a statistic: "A 2019 Education Week analysis found that 87% of states had enacted compulsory attendance laws by 1920." Could not trace to a specific Education Week report.
• Quote: Attributed a sentence to Horace Mann. Searching the quote in quotation marks returned no primary source; the sentiment is consistent with Mann's documented views but the specific wording appears generated.
• Cross-check: Second model gave different citations on the same topic — one matched a real, verifiable book (found via library catalog); the others didn't match. This mismatch flagged both sets for further scrutiny.
• Self-critique: Primary AI, when asked, acknowledged: "I should note that while those citations reflect what I know about the scholarly literature, I can't guarantee the exact title and publication details are accurate — I'd recommend verifying in a database."
• Shape summary: 2 invented citations (real author, wrong paper), 1 fabricated statistic, 1 fabricated quote (sentiment consistent, wording generated). 1 cross-check citation turned out to be a real verifiable book.
• Reflection: "I was surprised that one of the 'fabricated' citations was a real scholar — just a paper that didn't seem to exist. That's much harder to catch than a completely made-up author. Going forward I'll search the exact title, not just the author name."

Why the verification step can't be faked: a student who lists AI outputs without actually running verification searches earns the low end of the verification row — the rubric rewards documented evidence of checking (what you searched, what you found), not just claiming you verified.

Grading rubric — 50 points

Criterion	Full	Partial	None
Elicited at least three verifiable items (citations, statistics, or quotes) using appropriate hunting prompts (8)	8	4–6	0–3
Documented each item's shape with correct hallucination-family terminology (10)	10	5–8	0–4
Ran all four verification steps with evidence of actually having done them (e.g., "I searched X on Google Scholar and found Y") (20)	20	10–16	0–9
Honest, accurate reporting — distinguishes real, fabricated, and partial; acknowledges where results were unclear (7)	7	4–5	0–3
Reflection — a genuine insight about what surprised them or changed their practice (5)	5	3	0–2

Quality gate (self-checked):
- The tools and links named (approved assistants at their official homepages; Google Scholar at scholar.google.com) are real and current (verified live).
- No fabricated sources are presented as real anywhere in this file — any AI output presented as an example is clearly labeled as a hallucination to catch.
- The activity requires the student to verify AI output externally, not just assess it — verification-as-content is the assignment itself.
- The rubric rewards judgment, verification rigor, and accurate reporting — not the AI's prose.
- Product-accuracy gate: PASS.