Analysis Workshop: Two Kinds of Smart Across a Lifetime

Part 1 — The Big Picture

This week you learned that intelligence does not go downhill across the board with age: fluid intelligence, fast reasoning about new problems, tends to decline in later adulthood, while crystallized intelligence, the knowledge and vocabulary built up over a lifetime, holds steady. Psychology only counts a claim like that as knowledge once it is measured. In this workshop you turn both kinds of intelligence into operational definitions, read a small illustrative dataset of ten adults with the correlation coefficient r, and design the experiment that could test a popular claim about keeping a mind sharp. A chatbot then answers questions about your work, and you catch where it goes wrong.

Time budget: Plan for about 90 minutes.

Part 2 — The Task

Guiding question: In a set of adults of different ages, does a timed-puzzle score fall with age while a vocabulary score holds steady, how would you measure each kind of intelligence honestly, and what would it take to test whether a daily puzzle habit changes fluid reasoning?

Before you begin — hypothesis: Before you look closely at the numbers, write one sentence in this form: If older adults in the dataset are compared with younger ones, then their puzzle scores will ___ and their vocabulary scores will ___ because ___.

Workshop data: ten-adults-puzzles-and-vocabulary

Source: An illustrative dataset of ten adults, constructed for this course to practise reading development data and printed in full in this workshop; it is not real data from real people, and no outside file is needed.

• Rows: 10 adults, one row each, aged 25 to 70 in steps of 5 years; the data table has 12 rows, the 10 adults plus one row for each r
• Columns: Age in years, timed-puzzle score out of 20, and vocabulary score out of 20

Part 3 — Materials & Procedure

Tools and materials

• Google Sheets (free) or Excel
• A document or notebook for your written answers

Ways of working

• operationalize-a-construct
• design-a-correlational-or-experimental-check
• compute-and-interpret-r

Procedure

1. Fill in the first scaffold table, your operational definitions, before you open the data: write what a timed-puzzle score of 0 to 20 and a vocabulary score of 0 to 20 would each measure, and one way you could check whether each test gives consistent scores. Tool: A document or notebook for your written answers. Check: The first table is complete before you look at the numbers.
2. Open a blank sheet. Type Age in cell A1, Puzzle in cell B1, and Vocabulary in cell C1, then type the ten rows from the data table into A2 through C11, one adult per row. Tool: Google Sheets (free) or Excel. Check: Column A runs 25, 30, 35, and so on up to 70.
3. In an empty cell, type =CORREL(A2:A11,B2:B11) to find r for age and puzzle score, and in another cell type =CORREL(A2:A11,C2:C11) to find r for age and vocabulary score. Record both results to two decimals in the last two rows of the data table. Tool: Google Sheets (free) or Excel. Check: Two values between negative 1 and positive 1, one for each kind of score.
4. Select A1 through B11 and insert a scatter chart with age across the bottom and puzzle score up the side; then make a second chart for age and vocabulary score. Title both axes on each chart. Tool: Google Sheets (free) or Excel. Check: Two charts with ten dots each that you can describe in words.
5. Fill in the third scaffold table, your experiment design, then answer the analysis questions and run the critique step in Part 6. Tool: A document or notebook for your written answers. Check: Three completed tables, five written answers, and a critique.

Fallback route

When to use it: Use this route if no spreadsheet is available to you, or if a disability or device limit makes building the charts in a spreadsheet unsuitable for you.

Plot the ten adults by hand on paper or describe each chart in words, one dot per adult, with age across the bottom. Instead of a two-decimal r, write the direction (positive or negative) and the strength (strong, moderate, or weak) you judge from the dots, and label it as judged by eye. Complete the definition table, the design table, all five analysis questions, and the critique step with the same data. Submit the same items for equal credit.

This route earns the same credit as the primary route.

Part 4 — The Scaffold

Write the operational definitions first, because a kind of intelligence you have not defined as a procedure is not yet measurable.

Student scaffold table 1

Part of the definition	Your entry
Fluid intelligence, as measured here	The number of brand-new pattern puzzles solved correctly in 3 minutes, from 0 to 20
What one puzzle score measures, in one sentence
Crystallized intelligence, as measured here	The number of 20 vocabulary words defined correctly, with no time limit
What one vocabulary score measures, in one sentence
One way to check that each test gives consistent scores

Read each adult's two scores side by side, because r pairs age with each kind of score separately.

Student scaffold table 2

Age (years)	Puzzle score (of 20)	Vocabulary score (of 20)
25	17	12
30	18	13
35	16	14
40	15	13
45	15	15
50	13	16
55	12	15
60	12	17
65	10	16
70	9	17
r for age and puzzle score (two decimals)
r for age and vocabulary score (two decimals)

Plan the experiment that could test a cause, because the ten-adult dataset can only show an association.

Student scaffold table 3

Design element	Your plan
The claim being tested	Doing a daily puzzle app for eight weeks raises older adults' timed-puzzle scores
Independent variable (IV) and its two conditions
Dependent variable (DV) and how it is measured
How participants are assigned to conditions
How participants are chosen, and who the results could apply to
One third variable you would hold the same in both conditions
What the result would be allowed to claim

Part 5 — Analysis Questions

1. Report both values of r to two decimals. For each, describe its direction and strength in words, and write one sentence that uses the phrase associated with.
2. Describe the pattern in each chart: does the puzzle score fall steadily across the ages, does the vocabulary score rise, fall, or stay level, and which adult breaks each pattern most?
3. Explain the two patterns with this week's ideas. Which pattern fits fluid intelligence and which fits crystallized intelligence, and why would a lifetime of experience help one kind of score but not the other? Connect your answer to the idea that development is lifelong, with gains and losses side by side.
4. These ten adults were each measured once, at different ages, so the data are correlational. Name one third variable, such as being born in a different decade with different schooling or more practice with screens, that could make younger adults score higher on the puzzle test, and explain how the experiment in your third table would handle a third variable like that.
5. Choose one and answer it in 3–4 sentences. (a) If the same ten adults took the puzzle test again next week, would you expect the same scores? Say what that tells you about the reliability of the test, and whether a reliable test is automatically valid. (b) Could the timed-puzzle score capture something other than fluid intelligence, such as eyesight, comfort with screens, or nerves about being timed? Explain the validity problem and suggest one change that would make the test more valid.

Part 6 — AI-Critique Moment

Choose one equal-credit critique route. Route A: ask an approved AI tool to critique your completed work, then check its response against your own evidence. Route B is the no-AI equivalent printed below.

Approved tools: Claude, Gemini, ChatGPT, Grok, Microsoft Copilot.

Prompt to paste:

I worked with a small illustrative dataset of ten adults aged 25 to 70, each with a timed-puzzle score out of 20 and a vocabulary score out of 20, and I've pasted my completed tables above, including my two correlations r (age with puzzle score, and age with vocabulary score) and my plan for an experiment on a daily puzzle app. Please answer five things in short sentences. First, does my negative r between age and puzzle score prove that getting older causes fluid intelligence to drop? Second, in my planned experiment, which is the independent variable and which is the dependent variable? Third, if I want my results to apply to all older adults in my country, should I use random assignment or random sampling? Fourth, people scored almost the same when they took my puzzle test twice, so is my puzzle test valid? Fifth, is the idea that intelligence has a fluid part and a crystallized part that age differently a theory or a hypothesis, and is my prediction that the 70-year-old will solve fewer puzzles than the 25-year-old a theory or a hypothesis?

Test the response for these likely failures:

• Chatbots often say a strong negative r proves that aging itself lowers fluid intelligence, forgetting that the ten adults were measured once, at different ages.
• Chatbots often call the puzzle score the independent variable because it is the main measure, or call age the independent variable in the correlational dataset.
• Chatbots often say random assignment is what lets the results apply to all older adults in the country.
• Chatbots often say a puzzle test that gives the same scores twice is therefore valid.
• Chatbots often call the broad fluid-and-crystallized account a hypothesis and the specific prediction a theory, or treat the two words as interchangeable.

In 2–3 sentences per point, quote what the chatbot got right, quote at least two answers it got wrong or only half right, and correct each one using your tables or the course definitions.

Equivalent route (no AI required)

No AI account is required for this workshop. Instead of the chatbot, critique this flawed analysis, printed here in full: "My ten adults show r = −0.98 between age and puzzle score, which proves that getting older causes fluid intelligence to drop. In my experiment, the puzzle score is the independent variable and the daily puzzle app is the dependent variable. I will randomly assign the volunteers from my apartment building to the app group and the crossword group, so my results apply to every older adult in the country. When I gave the same puzzle test twice, people got almost the same scores, so the puzzle test is valid. Horn and Cattell's idea that intelligence has a fluid part and a crystallized part that age differently is only a hypothesis, while my prediction that the 70-year-old will solve fewer puzzles than the 25-year-old is a theory." Mark it up: quote each flawed sentence, name what is wrong (cause read from a correlation, the IV and DV swapped, random assignment confused with random sampling, reliability confused with validity, or theory confused with hypothesis), cite the evidence from your tables or the course definitions, and write a corrected sentence beneath it. This route assesses the same objectives (Objective 6 and Objective 7) as the chatbot critique, takes comparable effort (about 30 minutes and 200–300 words), is scored by the same scoring criteria for the same credit, and its own submission artifact is the marked-up flawed analysis, posted in place of the chatbot share link and report.

Part 7 — What to Submit

• All three completed scaffold tables
• Your two scatter charts with titled axes, or labelled sketches or word descriptions of them
• Your hypothesis sentence and written answers to all five analysis questions
• The chatbot critique report with its share link, or the equivalent route's marked-up flawed analysis

Deliverable:

A short Analysis Workshop report: three completed tables (your operational definitions of fluid and crystallized intelligence, the ten adults with both values of r, and an experiment design), two scatter charts with titled axes or labelled sketches or descriptions, a hypothesis sentence and written answers to five analysis questions, and a critique of a chatbot's answers about your work, or, on the equivalent route, a marked-up correction of the printed flawed analysis. Submit in any accessible file type or as a link; a recorded response may be submitted as audio or video with a transcript, or as text. Choose a written, recorded, or visual form for your answers; every form is graded by the same scoring criteria.

Rubric

The rubric below shows how each submitted part is evaluated; bands are full, partial, or none.

Workshop evaluation criteria

Criterion	Full	Partial	None
Operational definitions of fluid and crystallized intelligence	The first table states in one sentence each what a puzzle score and a vocabulary score measure, keeps fast reasoning on new problems apart from stored knowledge, and names a concrete way to check that each test gives consistent scores.	Definitions are given, but one of the two sentences is missing or vague, the two kinds of intelligence are blurred, or the consistency check is missing.	No operational definitions are written.
Data, r, and its interpretation	Both values of r are computed and reported to two decimals, or judged by eye and labelled that way on the fallback route; both charts or descriptions have titled axes; and each interpretation says associated with, gives direction and strength, and names a third variable.	The values of r are present, but a chart, a direction or strength, or the third variable is missing, or a sentence slips into cause.	No correlation is reported, or the values are invented.
Experimental design check	The design table puts the manipulated app or crossword condition as the IV and the measured puzzle score as the DV, uses random assignment for the conditions and random sampling for who could be studied, holds a third variable constant, and states what the result could claim.	The design is attempted, but the IV and DV are swapped, or assignment and sampling are confused, or the claim goes beyond the design.	No experiment design is submitted.
Development reasoning and the limitation	Questions 2 to 5 describe both patterns, match the puzzle pattern to fluid intelligence and the vocabulary pattern to crystallized intelligence with a reason, treat development as lifelong with gains and losses, connect the third variable to the experiment, and answer the chosen limitation question with an accurate account of reliability or validity.	Most questions are answered, but fluid and crystallized intelligence are swapped, aging is treated as decline across the board, reliability and validity are blurred, or a reason is missing.	The analysis questions are missing or do not address the data.
Critique of the chatbot or of the printed flawed analysis	On either route, at least two errors are quoted and corrected with evidence from the student's own tables or the course definitions, and one correct point is acknowledged; both routes earn this band equally.	On either route, errors are named but corrected without evidence, or only one error is caught.	No critique appears, or pasted output is submitted without any checking.

Quality Gate

• The operational definitions are written before you look at the data, and they say what each score measures and how you would check consistency.
• Every value of r comes from your own formula on the printed data, or is judged by eye and labelled that way on the fallback route, never from memory or a chatbot.
• Each chart puts age across the bottom and the score up the side, with both axes titled.
• Every sentence about r says associated with and names a third variable; no sentence claims that aging alone causes a score to change.
• The design table names the IV, the DV, random assignment, and random sampling, each in its correct role.
• No statistic, score, or source is invented anywhere in the submission, and the dataset is described as illustrative.
• The critique quotes the chatbot or the flawed analysis and corrects at least two errors with evidence from your tables or the course definitions.