RESEARCH QUESTIONS for SLOPE i3 EVALUATION
I. Impact Study
Research Question
Sample
Potential Measure(s)
1. On average, for student cohort 2, does performance on
the state’s annual standardized assessment in
mathematics differ for students who participate in the
project-based drop-in units and students who do not
participate in the project-based drop-in units?
Student Cohort 2 California Standards Test (CST)
in mathematics from grade 8
administered after second year
(2014) of implementation
2. On average, for student cohort 2, does performance on
the state’s annual standardized assessment in
mathematics differ for students who participate in the
project-based drop-in units plus Summer CAMP and
students who do not participate in these two components
of the intervention?
Student Cohort 2 California Standards Test (CST)
in mathematics from grade 8
administered after second year
(2014) of implementation
II. Exploratory Questions
Research Question
Sample
Potential Measure(s)
1. On average, for student cohort 1, does performance on
the state’s annual standardized assessment in
mathematics differ for students who participate in the
project-based drop-in units and students who do not
participate in the project-based drop-in units?
Student Cohort 1 California Standards Test (CST)
in mathematics from grade 8
administered after first year
(2013) of implementation
2. On average, for student cohort 1, does performance on
the state’s annual standardized assessment in
mathematics differ for students who participate in the
project-based drop-in units plus Summer CAMP and
students who do not participate in these two components
of the intervention?
Student Cohort 1 California Standards Test (CST)
in mathematics from grade 8
administered after first year
(2013) of implementation
Notes
A 2-level HLM that examines the impact of project-based
drop-in units on CST scores (based on approximately twothirds of student sample who are only exposed to the projectbased drop-in units)
Scores analyzed for about two-thirds of student sample (with
parent consent) taught by treatment and control teachers
A 2-level HLM that examines the impact of drop-in units and
Summer Academy on CST scores (based on approximately
one-third of student sample who are identified as needing
extra support in the summer in order to succeed in Algebra I)
Scores analyzed for about one-third of student sample (with
parent consent) who are taught by treatment and control
teachers and who also attend Summer Academy before
taking Algebra I in the subsequent fall semester
Notes
A 2-level HLM that examines the impact of project-based
drop-in units on CST scores (based on approximately twothirds of student sample who are only exposed to the projectbased drop-in units)
Scores analyzed for about two-thirds of student sample (with
parent consent) taught by treatment and control teachers
A 2-level HLM that examines the impact of drop-in units and
Summer Academy on CST scores (based on approximately
one-third of student sample who are identified as needing
extra support in the summer in order to succeed in Algebra I)
Scores analyzed for about one-third of student sample (with
parent consent) who are taught by treatment and control
teachers and who also attend Summer Academy before
taking Algebra I in the subsequent fall semester
Updated August 9, 2011
1
Research Question
Sample
Potential Measure(s)
Notes
3. On average, does the impact of intervention on the
state’s annual standardized assessment in mathematics
differ by gender for students who participate in the projectbased drop-in units and students who do not participate in
the project-based drop-in units?
Student Cohort 1
Student Cohort 2
(separate cohort
analyses)
California Standards Test (CST)
in mathematics from grade 8
administered after first year
(2013) and second year (2014)
of implementation
A 2-level HLM that examines the impact of project-based
drop-in units on CST scores by gender (i.e., differential
treatment impact by gender)
Scores analyzed for either two-thirds or one-thirds of student
sample (with parent consent) taught by treatment and control
teachers
4. On average, does the impact of intervention on the
state’s annual standardized assessment in mathematics
differ by gender for students who participate in the projectbased drop-in units plus Summer CAMP and students who
do not participate in these two components of the
intervention?
Student Cohort 1
Student Cohort 2
(separate cohort
analyses)
California Standards Test (CST)
in mathematics from grade 8
administered after first year
(2013) and second year (2014)
of implementation
A 2-level HLM that examines the impact of project-based
drop-in units on CST scores by gender (i.e., differential
treatment impact by gender)
Scores analyzed for either two-thirds or one-thirds of student
sample (with parent consent) taught by treatment and control
teachers
III. Implementation Fidelity Study
Research Question
1. How many hours did treatment teachers spend participating in professional development activities associated with the Algebra 1 curriculum?
2. On average, how many hours of coaching during Algebra 1 did treatment teachers receive?
3. In general, in what activities were teachers involved during coaching and collaboration time?
4. Did the treatment teachers implement the full Algebra 1 intervention as intended?
5. On average, how much time did teachers spend on each Algebra 1 unit? On each lesson? On steps within each lesson?
6. What types of challenges did teachers experience during implementation of each of the Algebra 1 units
7. Overall, to what degree did the treatment teachers implement the Algebra 1 curriculum as recommended?
8. Is there any difference in degree of implementation of the Algebra 1 curriculum between the first year and the second year?
9. Which CAC activities did teachers use to reinforce instruction during each drop-in unit?
10. How much time (total number of minutes) did students spend engaged in CAC activities for each drop-in unit?
11. What types of challenges did teachers experience when teaching the CAC?
Updated August 9, 2011
2
12. How many hours did teachers participate in professional development activities associated with Summer CAMP?
13. Did the treatment teachers implement the full CAMP intervention?
14. On average, how much time did teachers spend on each CAMP unit? On each lesson? On steps within each lesson?
15. Overall, to what degree did the treatment teachers implement the CAMP intervention as recommended?
16. Is there any difference in degree of CAMP implementation between the first year and the second year?
17. During CAMP, which CAC activities did teachers use to reinforce instruction during each drop-in unit
18. During CAMP, how much time (total number of minutes) did students spend engaged in CAC activities for each drop-in unit?
19. During CAMP, what types of challenges did teachers experience when teaching the CAC?
Updated August 9, 2011
3