Step by step
The question
Question: 40 teachers from a school signed up for the study.
They chose for themselves whether to use a new reading plan.
The plan group saw more progress.
What can the study say?
1 Two questions
To judge a study, we ask two questions.
How were the teachers picked?
And how were the groups made?
2 How they were picked
The story says the teachers signed up.
They chose to join.
So: no, not at random.
People who join by themselves can be different from the others.
Careful!
A big group is not enough.
Volunteers may be different from the rest of the school.
So what we find is true only for people like these, not for the whole school.
3 How the groups were made
The teachers chose for themselves.
So: no, not at random.
People who choose to use a new reading plan may be different in other ways too.
So the plan and the better result go together.
But maybe the plan did not cause it.
4 What the study can say
So the study can say: the plan and the result go together, only for people like these.
Maybe the plan is not the cause.
5 Check
Let's check with the story.
The first sentence says: signed up.
That matches.
The second sentence says: they chose.
That matches too.
So the answer is right!
Let's look back.
First: two questions.
Then: how they were picked.
Then: how the groups were made.
Then: what the study can say.
Last: check.