Analysis Workshop: Two Ways to Read One Dial

Part 1 — The Big Picture

This week you learned that the Big Five describe every person on five continuous dials, and that a description of personality becomes scientific only when it has an operational definition and a measure that is reliable and valid. A dial reading such as "high on conscientiousness" sounds clear, but how would you check it? In this workshop you measure one dial, conscientiousness, two different ways for the same characters: once as an overall 1-to-7 rating and once as a count of observable behaviors. You put one number on how well the two measures agree using the correlation coefficient r, and you design the experiment that could test a claim about changing the dial. A chatbot then answers questions about your work, and you catch where it goes wrong.

Time budget: Plan for about 90 minutes.

Part 2 — The Task

Guiding question: Do two different ways of measuring conscientiousness agree about the same characters, how would you measure the trait honestly, and what would it take to test whether a planner app can change it?

Before you begin — hypothesis: Before you rate anyone, write one sentence in this form: If I rate 8 characters on the conscientiousness dial and also count their conscientious behaviors, then characters with higher dial ratings will ___ because ___.

Workshop data: your-character-dial-ratings

Source: Your own two measurements of conscientiousness for 8 characters from shows, books, or games you know well, recorded in this workshop; no outside file is needed.

• Rows: 8 characters per run, one row each; the definition table has 6 rows, and the data table has 9 rows, the 8 characters plus one row for r
• Columns: The character, a dial rating from 1 to 7, and a checklist count from 0 to 5

Part 3 — Materials & Procedure

Tools and materials

• Google Sheets (free) or Excel
• A document or notebook for your written answers

Ways of working

• operationalize-a-construct
• design-a-correlational-or-experimental-check
• compute-and-interpret-r

Procedure

1. Fill in the first scaffold table, your operational definition, before you rate anyone: the trait, the 1-to-7 dial scale with its two end labels, and five observable conscientious behaviors you will count, such as keeping promises on time or planning ahead. Each behavior must be something a viewer or reader could see the character do. Tool: A document or notebook for your written answers. Check: The first table is complete before you choose any characters.
2. Choose 8 characters from shows, books, or games you know well, and write them in the second scaffold table. Use fictional characters only, so no real person is rated. Pick a mix you expect to land at different places on the dial. Tool: A document or notebook for your written answers. Check: Eight character names in the first column.
3. For each character, give one overall conscientiousness dial rating from 1 to 7 from your general impression, and write it in the second table. Do this for all 8 before you start the checklist. Tool: A document or notebook for your written answers. Check: Eight dial ratings, one beside each character.
4. Now go through your five checklist behaviors for each character and count how many the character regularly shows, from 0 to 5. Write each count in the second table. Tool: A document or notebook for your written answers. Check: Eight checklist counts, each a whole number from 0 to 5.
5. Open a blank sheet. Type Dial in cell A1 and Checklist in cell B1, then type your eight dial ratings in A2 through A9 and your eight checklist counts in B2 through B9, keeping each character on one row. Tool: Google Sheets (free) or Excel. Check: Two columns of eight numbers, one row per character.
6. In an empty cell, type =CORREL(A2:A9,B2:B9) and record the result to two decimals in the last row of the second table. If every dial rating or every checklist count is the same, the formula shows an error because r cannot be computed when one variable never changes; record that and explain it in your first analysis answer. Tool: Google Sheets (free) or Excel. Check: A value between negative 1 and positive 1, or an error message if one column never changes.
7. Select A1 through B9 and insert a scatter chart with the dial rating across the bottom and the checklist count up the side. Title both axes. Tool: Google Sheets (free) or Excel. Check: Eight dots, one per character, that you can describe in words.
8. Fill in the third scaffold table, your experiment design, then answer the analysis questions and run the critique step in Part 6. Tool: A document or notebook for your written answers. Check: Three completed tables, five written answers, and a critique.

Fallback route

When to use it: Use this route if you cannot run the character ratings, for example if no spreadsheet is available, if you do not know 8 characters well enough to rate, or if you prefer not to rate anyone.

Write the operational definition in the first table, and fill in the second table for as many characters as you can without computing r. Then use the model data below in place of your own ratings: compute r for the ten pairs with the same CORREL formula, make the scatter chart, and answer the first analysis question about these data. For questions 2 and 3, describe the pattern you would predict for two measures of conscientiousness and explain it with reliability and validity. Complete the design table, the remaining questions, and the critique step. Submit the same items for equal credit.

This route earns the same credit as the primary route.

An illustrative paired dataset this course uses for practicing r, constructed for teaching rather than collected from real students.

Fallback model data

Hours of sleep	Quiz score (of 20)
5	11
6	14
6.5	12
7	15
7	13
7.5	16
8	17
8	14
8.5	18
9	17

Part 4 — The Scaffold

Write the operational definition first, because a trait you have not defined as a procedure is not yet measurable.

Student scaffold table 1

Part of the definition	Your entry
The trait I am measuring	Conscientiousness, the Big Five dial that runs from spontaneous and disorganized to careful, organized, and dependable
The dial scale and its two end labels	1 means very spontaneous and disorganized; 7 means very careful, organized, and dependable
Five observable behaviors I will count
What one dial rating measures, in one sentence
What one checklist count measures, in one sentence
Why I rate the dial before I count the checklist

Record both measurements for each character on one row, because r pairs the two numbers in each row.

Student scaffold table 2

Character	Dial rating (1 to 7)	Checklist count (0 to 5)
1
2
3
4
5
6
7
8
Correlation r (two decimals)

Plan the experiment that could test a cause, because your character ratings can only show an association.

Student scaffold table 3

Design element	Your plan
The claim being tested	Using a planner app every day for a month raises a person's conscientiousness
Independent variable (IV) and its two conditions
Dependent variable (DV) and how it is measured
How participants are assigned to conditions
How participants are chosen, and who the results could apply to
One third variable you would hold the same in both conditions
What the result would be allowed to claim

Part 5 — Analysis Questions

1. Report your r to two decimals. Describe its direction and strength in words, and write one sentence that uses the phrase associated with. If r could not be computed, explain why in terms of your two columns.
2. Describe the pattern in your eight dots: did characters with higher dial ratings tend to show more checklist behaviors, which character broke the pattern most, and do the dots look closer to a straight line or a scattered cloud?
3. Explain your result with this week's ideas. Why is a 1-to-7 rating a reading on a continuous dial rather than a type, and what does it mean when two different measures of the same trait agree, or disagree, about the same characters? Use the words reliability and validity correctly.
4. Your character ratings are correlational: you measured two things but manipulated nothing. Name one third variable, such as how much you like a character or how much of their life the story shows, that could push both measures in the same direction, and explain how the experiment in your third table would rule out a third variable like that.
5. Choose one and answer it in 3–4 sentences. (a) If you rated the same 8 characters again next week, would you expect the same dial ratings and checklist counts? Say what that tells you about the reliability of your measures, and whether a reliable measure is automatically valid. (b) Could your checklist count capture something other than conscientiousness, such as how much screen time a character gets or the kind of story they are in? Explain the validity problem and suggest one change that would make the measure more valid.

Part 6 — AI-Critique Moment

Choose one equal-credit critique route. Route A: ask an approved AI tool to critique your completed work, then check its response against your own evidence. Route B is the no-AI equivalent printed below.

Approved tools: Claude, Gemini, ChatGPT, Grok, Microsoft Copilot.

Prompt to paste:

I chose 8 characters from shows, books, or games I know well and measured each one's conscientiousness two ways: a 1-to-7 dial rating I gave from my overall impression, and a count of how many of my 5 written conscientious behaviors the character shows, from 0 to 5. I've pasted my completed tables above, including my correlation r between the two measures and my plan for an experiment on whether a planner app raises conscientiousness. Please answer five things in short sentences. First, does my correlation prove that being conscientious causes the behaviors on my checklist? Second, in my planned experiment, which is the independent variable and which is the dependent variable? Third, if I want my results to apply to all college students in my country, should I use random assignment or random sampling? Fourth, my dial ratings came out almost the same when I rated the characters again a day later, so is my dial rating valid? Fifth, is the Big Five model a theory or a hypothesis, and is my prediction that characters with higher dial ratings will show more checklist behaviors a theory or a hypothesis?

Test the response for these likely failures:

• Chatbots often say a positive r proves that the trait causes the checklist behaviors, or that the behaviors cause the dial rating.
• Chatbots often call the conscientiousness score the independent variable because it is the trait being studied, or call the planner app the dependent variable.
• Chatbots often say random assignment to the app and no-app groups is what lets the results apply to all college students in the country.
• Chatbots often say a dial rating that comes out the same twice is therefore valid.
• Chatbots often call the student's specific prediction a theory, or call the Big Five model only a hypothesis.

Write 3–5 sentences naming one thing the chatbot got right and at least two places it went wrong or overreached, quoting its words and correcting each with evidence from your own tables or this week's course definitions.

Equivalent route (no AI required)

No AI account is required for this workshop. Instead of the chatbot, critique this flawed analysis, printed here in full: "My dial ratings and checklist counts had a strong positive correlation, which proves that being conscientious causes the behaviors on my checklist. In my experiment, the conscientiousness score is the independent variable and the planner app is the dependent variable. I will randomly assign students to the app and no-app groups so that my results apply to every college student in the country. When I rated my characters again the next day, my dial ratings came out almost the same, so my dial rating is valid. The Big Five model is only a hypothesis, while my prediction that characters with higher dial ratings will show more checklist behaviors is a theory." Mark it up: quote each flawed sentence, name what is wrong (cause read from a correlation, the IV and DV swapped, random assignment confused with random sampling, reliability confused with validity, or theory confused with hypothesis), cite the evidence from your tables or the course definitions, and write a corrected sentence beneath it. This route assesses the same objectives (Objective 7 — describe cognitive, social, and emotional development across the lifespan and compare the major families of personality theory and their assessment) as the chatbot critique, takes comparable effort (about 30 minutes and 200–300 words), is scored by the same scoring criteria for the same credit, and its own submission artifact is the marked-up flawed analysis, posted in place of the chatbot share link and report.

Part 7 — What to Submit

• All three completed scaffold tables
• Your scatter chart with titled axes, or a labelled sketch of it
• Your hypothesis sentence and written answers to all five analysis questions
• The chatbot critique report with its share link, or the equivalent route's marked-up flawed analysis

Deliverable:

A short Analysis Workshop report: three completed tables (your operational definition of conscientiousness, your eight characters with two measurements each and r, and an experiment design), a scatter chart with titled axes or a labelled sketch, a hypothesis sentence and written answers to five analysis questions, and a critique of a chatbot's answers about your work, or, on the equivalent route, a marked-up correction of the printed flawed analysis. Submit in any accessible file type or as a link; a recorded response may be submitted as audio or video with a transcript, or as text. Choose a written, recorded, or visual form for your answers; every form is graded by the same scoring criteria.

Rubric

The rubric below shows how each submitted part is evaluated; bands are full, partial, or none.

Workshop evaluation criteria

Criterion	Full	Partial	None
Operational definition of conscientiousness	The first table names the trait, a dial scale with both end labels, five observable behaviors to count, and one-sentence statements of what the dial rating and the checklist count each measure, with a reason for rating the dial first.	A definition is given, but the end labels, one or more observable behaviors, or one of the one-sentence statements is missing or vague.	No operational definition is written.
Data, r, and its interpretation	Eight characters with two measurements each, or the fallback data, are recorded; r is computed and reported to two decimals or its error is explained; the scatter chart or sketch has titled axes; and the interpretation says associated with, gives direction and strength, and names a third variable.	The data and r are present, but the chart, the direction and strength, or the third variable is missing, or the sentence slips into cause.	No ratings or correlation are recorded, or the values are invented.
Experimental design check	The design table puts the manipulated app condition as the IV and the measured conscientiousness score as the DV, uses random assignment for the conditions and random sampling for who could be studied, holds a third variable constant, and states what the result could claim.	The design is attempted, but the IV and DV are swapped, or assignment and sampling are confused, or the claim goes beyond the design.	No experiment design is submitted.
Dial, reliability, and validity reasoning and the limitation	Questions 2 to 5 describe the pattern, explain the rating as a position on a continuous dial rather than a type, treat agreement between two measures as validity evidence, connect the third variable to the experiment, and answer the chosen limitation question with an accurate account of reliability or validity.	Most questions are answered, but the rating is treated as a type, reliability and validity are blurred, or a reason is missing.	The analysis questions are missing or do not address the measurement.
Critique of the chatbot or of the printed flawed analysis	On either route, at least two errors are quoted and corrected with evidence from the student's own tables or the course definitions, and one correct point is acknowledged; both routes earn this band equally.	On either route, errors are named but corrected without evidence, or only one error is caught.	No critique appears, or pasted output is submitted without any checking.

Quality Gate

• The operational definition is written before any character is rated, and it names the trait, the dial scale with both end labels, and five observable behaviors.
• Only fictional characters are rated; no real person is rated, labeled, or diagnosed.
• Every rating, count, and the correlation come from your own run and formula, or from the fallback model data, never from memory or a chatbot.
• The scatter chart puts the dial rating across and the checklist count up, with both axes titled.
• Every sentence about r says associated with and names a third variable; no sentence claims that the trait causes the behaviors or that the behaviors cause the rating.
• The design table names the IV, the DV, random assignment, and random sampling, each in its correct role.
• No statistic, rating, or source is invented anywhere in the submission.
• The critique quotes the chatbot or the flawed analysis and corrects at least two errors with evidence from your tables or the course definitions.