Session 3: Trial management
Sarah Miller (Queens, Belfast)
Laura Dunne (Queens, Belfast)
RCT MANAGEMENT: KEY ISSUES AND
LESSONS LEARNED…
Dr Laura Dunne & Dr Sarah Miller
Introduction:
“An ounce of prevention is worth a pound of cure”
Related issues:
• Recruitment
– achieving required sample and power
• Retention
– Treatment attrition
– Measurement attrition
• Communicating with schools and developers
Centre for Effective Education | SCHOOL OF Education
Two examples…
• RCT evaluation of Barnardo’s Ready To Learn
afterschool programme: 16 schools and 8 data
collection sweeps over three years
• RCT evaluation of a classroom based social
emotional learning programme: 74 schools and 5
data collection sweeps over four years
Centre for Effective Education | SCHOOL OF Education
RCT Evaluation of Barnardo’s Ready To
Learn Afterschool Programme
The programme:
• Aims to raise achievement by focusing on
children’s early literacy skills and enhance
parental engagement in children’s learning
• Programme runs during term time for one hour
after school on Tuesday, Wednesday and
Thursday afternoons
• Activities in the programme include story telling,
drawing, singing, puppets, talking and listening
Centre for Effective Education | SCHOOL OF Education
RCT Evaluation of Barnardo’s Ready To
Learn Afterschool Programme
The evaluation:
• A cluster randomised design combining outcome
data with experiences of participants (over three
years, 8 data sweeps)
• 16 schools, 564 Year 1 pupils (9 intervention; 7
control)
Measures
• Battery of age-appropriate tests , focus groups
and parental interviews
• Principal interviews
Centre for Effective Education | SCHOOL OF Education
RCT evaluation of the Roots of Empathy
programme
The Programme:
• A universal programme delivered on a whole-class
basis for one academic year
• 27 lessons based around 9 themes delivered by a
trained RoE instructor
• Lessons revolve around a monthly classroom visit by
an infant and parent
• The parent-infant visit serves as a springboard for
discussions about understanding feelings and infant
development and effective parenting practices
Centre for Effective Education | SCHOOL OF Education
RCT evaluation of the Roots of Empathy
programme
The Evaluation:
• Cluster randomized controlled trial: 74 primary
schools randomly allocated to control and
intervention groups
• Schools in the intervention group receive the
programme in their selected Year 5 class for one
academic year
• Schools in the control group do not receive the
programme but receive it the following year
Centre for Effective Education | SCHOOL OF Education
Key issues
School recruitment
•
Actively engage with schools re: both the programme
and the methodology e.g. attend information sessions,
meetings with service and Principals.
•
Make it clear to schools what their participation will
involve (MoU)
•
Meetings with service and Principals
•
Consider a wait list control
•
Financially compensate for teacher time
•
Consent prior to randomisation
Centre for Effective Education | SCHOOL OF Education
Key issues
Retention
• Regular communication: newsletters each term,
study updates, Christmas cards, annual principal
interviews, school visits
• Minimise disruption and be flexible - pilot measures,
testing P1-3, stickers
• Document reasons for dropout and monitor attrition
between intervention and control groups
• If a school drops out of the intervention, continue to
collect data (ITT)
• Process evaluation: observation and field notes
Centre for Effective Education | SCHOOL OF Education
Key issues
Communication
•
Make clear to schools who in which team is responsible
for each element of the study (MoU), provide a
timetable of events
•
Identify a key contact to liaise with in the school
•
Minimise the number of different people the school
has to deal with
•
Agree a research plan and be clear who is responsible
for what
•
Regular (and efficient) communication between
programme developer(s) and research team (Skype,
email etc)
• Set realistic expectations of the possible outcomes of
the evaluation
Centre for Effective Education | SCHOOL OF Education