Question 1: Authors assigned treatment status at the level of the high school. What
does that mean in practical terms, and why did the authors opt for that type of
randomization?
In practical terms, assignment of the treatment status at the level of the high school
means that the treatment was applied to the whole group of students (who were
considered eligible for this study: high achieving and low-income) within a high
school in Michigan, instead of randomly selecting individual students to receive
treatment or be in a control group. So, if there was a certain number of high achieving
and low-income students in a certain high school, all of them were assigned with a
treatment status. That concludes, that if high school is in the treatment group, all the
eligible students would receive the intervention. The other way around, if the school is
in the control group, all the eligible students in it won`t receive the intervention.
The authors chose that type of intervention because there might be possible treatment
spillovers within schools, so basically the treated student may affect the others, who
are in a control group. If only some of the eligible students within one school received
personalised communication, related to the HAIL Scholarship, the information they
received might affect the decisions of their peers in the control group, so as the result
it would be hard to measure the real impact of the intervention due to this spillover
effect within the same school.
Question 2: If the authors had found no effect of the intervention, a natural follow-up
question would be - did participants receive the materials? And if so, did they read it?
However, there is no clear way of physically confirming this. So, authors try to find an
alternative measure for HAIL awareness. Please briefly describe their approach and
what they find.
To understand if the outcomes of the treatment would result in no effect due to the
fact, that participants didn`t receive the materials, authors implemented an alternative
measure for HAIL awareness. Their approach involved assigning each student a
personalized web address and including it in the student packets along with an
encouragement to visit the website for more information about the HAIL Scholarship
and the University of Michigan. Additionally, they offered students a free University
of Michigan t-shirt to incentivize them to log on to the website.
Based on their approach, they found the following results:
•
Approximately 40 percent of students who were offered the HAIL Scholarship
visited the website at least once. This percentage provides a lower bound on the
number of students who read the packet, as some students may have read the packet
but not visited the website.
•
Among the students who visited the website, the average number of views was
5.5, with a median of 3. This suggests that a significant portion of the students who
visited the website engaged with the content multiple times.
•
The first-time visits to the website were concentrated in the days following the
mailing of the student and parent letters, indicating that the materials in the packets
prompted students and, in some cases, parents to log onto the website. Notably, only
the student packet contained login instructions, but both students and parents were
involved in the process.
These findings suggest that a substantial proportion of students did engage with the
intervention materials and read the information provided, both in the initial mailing
and on the website. This alternative measure allowed them to assess the extent to
which students and parents interacted with the intervention materials, which could be
informative in understanding the impact of the HAIL Scholarship intervention.