Week 6 — AI Build Studio · "Run a Simulation"

Course: Using Artificial Intelligence (AI 101) · Silver Oak University (fictional sample) · Prof. Quinn
Objective: Objective 2 — design and run an AI simulation; verify its output; catch fabrications or errors · SLO A (produce a quality result with AI) · SLO B (catch the AI's mistakes and verify)
Worth 50 points · AI Build Studios group = 15% of the grade · Studio 6
Format: a hands-on build — you'll design a simulation for a real scenario in your own life, run it, critically evaluate its output, catch what the AI fabricates or overstates, and improve it.

This is the course's signature weekly component. Every instructional week has one Studio — a real thing to build, a required step where you verify and improve the AI's work, and a short reflection. All tools are free; everything is links to external sites.

---

Part 1 — The Build Goal

By the end of this Studio you'll have produced four things:
1. A complete simulation prompt (role, goal, exit condition — specific and labeled).
2. The simulation output — the conversation you ran, or the relevant excerpt.
3. A verification / AI-critique write-up naming at least one thing the AI fabricated, overstated, or got wrong — and what you found when you checked.
4. A short reflection on what made the simulation useful and what to improve next time.

This is the Week 6 lesson in miniature: design a simulation well (role, goal, exit condition) AND audit it for fabrications (generated ≠ verified).

Open one approved assistant to build in: ChatGPT (https://chatgpt.com), Claude (https://claude.com), Gemini (https://gemini.google.com), or Copilot (https://copilot.microsoft.com). A free account is enough.

---

Part 2 — Choose a Simulation Type and a Real Scenario

Pick ONE simulation type that is genuinely useful to you right now. Your scenario should be something you actually care about — simulations with real stakes are more instructive than made-up ones.

Option A — Difficult-conversation or difficult-customer simulation. A hard conversation you have coming up (or recently had): telling a colleague their work needs improvement, dealing with an upset customer, asking your landlord to fix something, negotiating a grade with a professor, having a conflict with a roommate.

Option B — Pre-mortem simulation. A plan or project you are actively working on: a group project, a job application, a club event, a personal goal with a deadline. The AI will imagine the project failed and reason backward.

Option C — Decision role-play / multi-stakeholder. A proposal or decision you need to make where hearing other perspectives would help: a business idea, a policy argument for a class, a club funding request, a personal major or career decision.

Option D — Adaptive-tutor simulation. A topic or concept from any of your courses (or your job/life) that you are currently struggling with. The AI will teach you one-on-one at your pace.

Write your choice here: I am doing a [TYPE] simulation on [SPECIFIC SCENARIO].

---

Part 3 — Design and Run the Simulation

Step 1 — Draft your simulation prompt. Include and label all three required parts:
- Role: who the AI is playing — be specific (not "an interviewer" but the exact type, company, scenario, and personality).
- Goal: what the simulation is designed to accomplish for you.
- Exit condition: when and how the simulation ends (after N exchanges; when I've done X; break character and give me feedback).

Save your prompt before you run it.

Step 2 — Run the simulation. Engage with the AI for at least 5 exchanges. Push back on the AI's moves; escalate the difficulty; follow up. A simulation you only skim the surface of is not good rehearsal.

Step 3 — Deepen it with a follow-up. Ask one follow-up that forces a different angle:
- For difficult-conversation: "Now escalate — the other person is more upset."
- For pre-mortem: "Which of these risks was most preventable, and why?"
- For decision role-play: "Which stakeholder's concern is hardest to resolve?"
- For adaptive tutor: "Give me a harder version of the last practice question."

Step 4 — Trigger your exit condition and, if you built it in, ask the AI for feedback on your performance.

---

Part 4 — The Verification / AI-Critique Step (required — this is the load-bearing step)

Now audit the simulation for fabrications, unsupported claims, and overreach.

Somewhere in any simulation, the AI is likely to produce at least one of the following:
- A specific statistic presented as fact ("studies show 70% of projects fail due to X")
- A historical fact or quote if your simulation touched on a real person or event
- A claim about how things work (legally, medically, financially, organizationally) presented confidently
- AI-generated dialogue from a real person that it implies is real or verified

Run this audit sequence:
1. Identify one specific, checkable claim the AI made inside the simulation.
2. Ask the AI to back it up: "What is your source for that specific claim? Are you certain it's accurate? If you're not sure, say so." Note whether it doubles down, walks back, or admits uncertainty.
3. Verify it yourself — a quick search, an official source, or a second assistant. Confirm whether the claim is true, approximately true, fabricated, or unverifiable.
4. If your simulation involved a historical figure: explicitly check whether any "quote" or specific statement is verifiable in a real historical record. It almost certainly is not. Name what you found.
5. Watch for the confidence trap: confident wording is not evidence of accuracy. The AI is equally fluent when it is wrong.

Write 4–6 sentences reporting:
- The specific claim you checked (quote it)
- What the AI said when you asked for a source
- What you found when you verified independently
- At least one specific thing the AI fabricated, overstated, or got wrong — or, if everything checked out, exactly how you verified each claim (that is the skill)
- How you would fix or guard against this in future simulations

The central rule of this Studio: AI-generated dialogue — especially from historical figures, but also from any simulated persona — is generated text, not verified truth. A convincing simulation is not a reliable simulation. Catching the gap between those two things is the whole point of this step.

---

Part 5 — Improve the Simulation Prompt

Based on what you learned — from running it and from the audit step — rewrite your simulation prompt with at least two specific improvements. Common improvements:

• Make the role more specific (add personality, stakes, constraints)
• Add a stronger exit condition with a feedback request
• Add an instruction to flag any uncertain claims ("if you are unsure of a specific fact inside the simulation, say so rather than stating it confidently")
• Adjust the goal to reflect what you actually needed from the rehearsal

Label what you changed and why.

---

Part 6 — Reflection (2–3 sentences)

What surprised you most — about how the simulation went, or about what the AI produced that you had to check? What will you do differently next time you design a simulation?

---

Part 7 — What to Submit

Submit a single document (or text entry) with: your scenario choice and the simulation type, your original simulation prompt (labeled: role, goal, exit condition), the relevant simulation output (conversation excerpt or key exchanges), your improved prompt (with a note on what changed), your Part 4 AI-critique write-up (the claim, the source check, what you found, the fix), and your Part 6 reflection. Due Sunday, Oct 18, 11:59 p.m. (50 points).

---

Instructor answer key & model deliverable — REMOVE BEFORE PUBLISHING TO STUDENTS

Students use their own scenarios, so deliverables vary. Grade the process (simulation design quality + verification + reflection), not a specific answer. The model below shows what full credit looks like.

Model deliverable (illustrative — pre-mortem):

• Scenario: Pre-mortem on a club fundraiser. Type: pre-mortem simulation.
• Original simulation prompt:
• Role: You are a post-event analyst reviewing a student environmental club fundraiser that took place three weeks ago and raised only $47 of a $500 goal. Almost no one came. You have the basic details below: 60-student club, campus venue, social media-only promotion, set during midterm week, no RSVP process, no follow-up emails.
• Goal: Reason backward from the failure. Identify the five most likely causes, ranked by impact on attendance and fundraising.
• Exit condition: After presenting and explaining all five causes, ask me which I find most surprising and give me two actionable fixes for that one.
• Simulation output excerpt: [student's actual conversation]
• Part 4 (model critique): "The AI said: 'Research consistently shows that events promoted only on social media reach 12–18% of potential attendees on average.' I asked for the source; the AI admitted it was 'drawing on general patterns in event management literature' and could not provide a specific citation. I searched for this statistic and could not find a verified source for the 12–18% figure — it appears to be a plausible-sounding number the AI generated. I would not cite this statistic as fact. Fix: I will add a line to future simulation prompts: 'Do not cite specific statistics as fact unless you can name the exact source — if uncertain, say so.' The AI's five-cause analysis was useful without the fake statistic."
• Improved prompt: [adds: "If at any point you would cite a specific statistic or research finding, flag that it is illustrative, not verified — I will look up real data myself."]
• Reflection: "The most surprising thing was how confident the AI sounded giving a specific percentage that turned out to be unverifiable. The fluency of the claim made me almost miss it. Next time I will specifically ask for sources after every specific claim, not just at the end."

Why the verification step can't be faked: a student who submits a simulation with no audit, no checked claim, and no catch earns the low end of the verification row. The rubric rewards the judgment and the catch, not the polished simulation output.

Grading rubric — 50 points

Criterion	Full	Partial	None
Simulation design — role, goal, and exit condition all present and labeled; role is specific (not generic); scenario is real and substantive (12)	12	6–10	0–4
Simulation engagement — ran at least 5 meaningful exchanges; used a follow-up to deepen or escalate; triggered exit condition (10)	10	5–8	0–4
Verification / AI-critique (Part 4) — identified a specific checkable claim; asked the AI for a source; verified independently; named what was fabricated, overstated, or unverifiable (or showed clear verification for each claim); gave a fix (18)	18	9–14	0–7
Improved prompt (Part 5) — at least two labeled improvements with reasons; improvements are specific, not cosmetic (5)	5	3	0–2
Reflection (Part 6) — a thoughtful takeaway about simulation design and/or the verification step (5)	5	3	0–2

Quality gate (self-checked): the tool links named (ChatGPT/Claude/Gemini/Copilot homepages) are real and current (verified live); no fabricated product features; the activity requires the student to catch the AI, not trust it; no student-produced output is asserted as "the" answer — the key grades the process. The model critique explicitly demonstrates catching a fabricated statistic. Product-accuracy gate: PASS.