June 2014 Physics Modeling at the University of Northern Iowa Grant# XXXXXXXXX, Carver Charitable Trust Grant amount: $ XXXXXXX FINAL REPORT to the Carver Charitable Trust Board of Trustees Overview This report addresses the University of Northern Iowa implementation of the Iowa Regents Modeling Workshops in physics, which received funding through the Carver Charitable Trust. It is a subsequent grant related to previous workshops receiving funding January of 2011 for the workshop held at Iowa State University and January 2012 for the workshop held at the University of Iowa. This report will communicate the final evaluation of the modeling workshop held at the University of Northern Iowa in the summer of 2013. Comparatively, this workshop was slightly different than the previous two workshops through the inclusion of PRISIMS resources and activities along with the modeling materials. 2012 Profile of UNI Participants The UNI workshop hosted 22 participants distributed throughout the eastern two thirds of the state. The makeup of participants differed from previous workshops in several ways. First the distribution of teaching experience was much different than the previous workshops at U of I and ISU. The previous workshops were fairly evenly distributed in experience while the UNI workshop had a significant number of participants 12/22 in their first five years of teaching experience. Only three had between five and ten years of experience and the remaining seven had more than 15 years of teaching experience. Additionally, there were fewer in field participants than the previous workshops with only four having significant physics or engineering experience as part of their background. There were a number of participants that did not currently hold a physics teaching position but were preparing to instruct physics at a school with increasing enrollment or who were pursuing endorsement to offer physics at a rural school for the first time. Several participants were encouraged to attend by physics teachers who attended previous workshops with the hope of establishing more continuity in instructional practices and/or for curriculum articulation purposes. Impact Assessment: Participant FCI Data The 2013 UNI participants produced similar trends to the 2011 ISU group and the 2012 U of I group. The gains for the UNI group were figured in similar manner as the previous participant groups and are summarized comparatively as follows. Aggregate UNI participant scores generated an average normalized gain of .43, which is comparable to the mean of the first two workshops. Averages of participants’ pre and post-test averages generate a similar normalized gain of .42. These normalized gains are less than the .55 average of the previous workshop participants figure in a similar manner. Another average was completed excluding participants with initial perfect scores for the reasons outlined in previous reports. When completing this average, the UNI group fairs better with a normalized gain of .65 compared to an average of .54 for the previous groups. This is likely due to the 5 participants well above mastery that finished with perfect scores yielding a normalized gain of 1 thus raising the overall average of this subgroup. Excluding the participants above mastery that finished with perfect scores, produces a result of .55 which is comparable to the average of the previous workshops. As expected, all three workshops have generated participant data consistent with published results from the ASU NSF and MSN Modeling workshops. Tables of all three years of participant FCI scores are presented in appendix 1. To further evaluate participant data in relation to the grant goal of producing well qualified physics instructors, an evaluation of participants below mastery entering the workshop was conducted. The UNI group started with 10 participants below mastery which is similar to the previous workshops. The major difference between the UNI below mastery group compared to the previous workshops participants was a significantly lower pre-test average. The UNI group started with pre-test average of 14 compared to an average of 18.4 for the previous workshops participants. This is likely due to the greater number of participants who mainly instruct physical science and had less overall experience with physics content. While the UNI group finished with 7 total participants below the mastery threshold, this group began with a significantly lower initial understanding. The group finished with only one participant below the threshold for understanding with most participants in this group finishing just shy of the mastery cut off. Participant scores for this subgroup for all three years follow the FCI scores for all participants in the appendix. Impact Assessment: Reflective comments from participants implementation journals Previous reports for the Iowa Regents Modeling Workshop emphasized weekly reflections during the workshop and mid-year reflections. The UNI participants kept an implementation journal throughout the year that was shared with faculty. The excepts below were cut from participant journals at the conclusion of implementation. In general, similar themes run through each of the three regents modeling workshops. The majority of participants have indicated significant positive change in their professional growth and noticed significant changes in their students’ attitudes, behaviors, and academic abilities. Examples from participant implementation journals follow for those areas. Reflective comments related to professional growth: I thought I entered the workshop “knowing” what it was about, but that was not true. I learned so much from the workshop and the professional conversations we have had this year. My thinking and questioning continued to develop as I worked through the physics curriculum this year. The most significant professional change I have experienced that I can directly relate to my participation in this workshop is my transformation from a teacher centered classroom in physics to a student centered classroom. Being accepted to the workshop has given me more credibility within my department. It is rather tough to make suggestions for change with a 43 year career teacher. The workshop gave me the experience, deployments, and lab equipment necessary to persuade my colleagues to try something different than they had done before. I feel my most professional change has been the growth in my approach to teaching as well as becoming more content specific. I pose more questions to students in order to have them become more accountable for their learning. When in the workshop, everything was put back onto me (as a student) to gauge what I was thinking and I had to learn to become an accountable learner. This really opened my eyes t what I was I doing in class and how that needed to change. The workshop led to a mind shift for me about my approach to teaching and helped me establish a more student-led classroom in which observations, data analysis, critical thinking, and peer review are much more prevalent than they were in my classroom in previous years. Reflective comments related to student attitudes and behaviors: Much better understanding as evidenced by FCI. Much better at thinking … like being aware of assumptions and simplifications; like using representation tools; getting away from thinking operation first, and if they do, at least do unit analysis or arithmetical reasoning; like considering the relationships, imagining the physical situation — math less. I have never felt more refreshed from teaching or as excited about teaching as this year with the modeling curriculum. I am excited to read/learn more to improve my instructional practices. In addition, our district received the TLC grant; I will be able to work with instructional coaches, curriculum coaches, and model teachers to improve upon my instruction. The students’ perception of science was thoroughly challenged and for some they may not value this rigorous year of science boot camp, but hopefully the thinking and reasoning skills will help them understand how to think for the sake of thinking. By struggling through some of the labs and having whiteboard discussions, the students were given an opportunity to take collaborative ownership of their work. In past years students showed ownership of work by turning in problem sets, but the whiteboard discussions took the ownership to a new level. Students could be heard defending their work to other groups and engaging in meaningful discussions with each other. Granted it took a lot of battling to get the students to use the whiteboards and truly look at their work, but once the routine was set for our last quarter of classes, the students started to really engage in the discussions. I’m seeing confidence in some of my lower performing students just in the last week that I can directly attribute to the modeling and PRISMS atmosphere. Implementing what we learned from the workshop last summer was both exciting and overwhelming. The workshop led to a mind shift for me about my approach to teaching and helped me establish a more student-led classroom in which observations, data analysis, critical thinking, and peer review are much more prevalent than they were in my classroom in previous years. I think the biggest difference for the Physics students was their approach to learning. I got them to buy into the idea of developing problem-solving tools that can be used in a variety of situations. I think the students demonstrated their mastery of this when we got to the projectile motion unit. I simply presented them with a scenario of an object moving in two directions at one time and told them to explain the problem. Each group came up with some ideas, and after a discussion they were able to come to a conclusion as a group, with basically no input from me. After they solved the problem, I pointed out the growth they had demonstrated over the course of the year and made sure they were aware of their problem solving strategies and usefulness of them. I saw great growth in my physical science student’s ability to think, communicate, and apply their knowledge to different situations from the beginning of the year to the present. Their ability to think like scientists improved greatly! Also, I saw improvement in nearly all of my student’s conceptual exams (Scientific Reasoning and Physical Science Concepts Inventory). Overall, I would say that the PRISMS and Modeling materials, along with the workshop last summer, proved to be very effective! Impact Assessment: 2013 Participant TPI Results The Teaching Perspectives Inventory (TPI) was taken prior to the workshop and at the conclusion of the first year of implementation. The delay was intentional to provide an appropriate time frame for actual changes in any perspective to occur. The TPI identifies 5 perspectives in three areas, beliefs, actions, and intentions. For any given perspective the highest score possible is a 45 and the lowest possible is a 9. In general averages for any category are in the mid-thirties and dominant perspectives are those with higher scores. A general descriptor of each perspective follows: Transmission- Effective teaching requires a substantial commitment to subject matter or content. Apprenticeship- Effective teaching enculturates students into a set of social norms and habits of mind. Developmental- Effective teaching is planned and conducted from a learner’s point of view. Nurturing- Effective teaching assumes long term persistent effort to achieve comes from the heart as well as the mind. Social Reform- Effective teaching seeks to change society in substantial ways. The TPI also checks for agreement between a teacher’s beliefs, intentions, and actions within each category but for the purposes of this report, positive changes in these areas are indicative of teachers bringing their beliefs, intentions, and actions into closer alignment. The UNI group was the first group to produce a near 100 percent return on the pre and post TPI. Previous participants struggled with returning the correct information and or neglected to take the final TPI. When examining the average change in each category we get what we might expect from the content and design of a modeling or interactive engagement workshop. Significant changes in both the apprenticeship and developmental categories occurred for the UNI group. This is expected because the instructional design is based on developing a deep and robust understanding of physics content in relation to what is known psychologically about how people learn. This trend is generally apparent for all three workshops to varying degrees and suggests the designs of the workshops are affecting teachers in areas where you would predict significant effects to occur. When examining UNI participants individual score changes across the apprenticeship and developmental categories, many participants had shifts bringing these two perspectives into a better balance with other perspectives. For a number of the UNI participants, these perspectives went from being a less dominant perspective to the most dominant perspective, with the majority of participants having their most dominant perspective either of these two on their post TPI. Interestingly, the UNI group did show a significant positive change in the nurturing perspective while the other two workshop groups posted a negative and no change in this category. We might speculate this is due to the number of candidates in their first few years of experience but this is not corroborated with individual changes. A number of more experienced teachers brought their nurturing perspectives from being less dominant into balance with other perspectives. A more rigorous comparison of workshop variables and participant make-up would be necessary to address the differences between workshops. The transmission and social reform categories had the least average change amongst UNI participants. For the transmission perspective, positive and negative changes seem to occur as a result of bringing that particular perspective into balance with the other perspectives. The social reform category remained the least dominant perspective for the UNI participants with the positive change being accounted for by several participants with substantial positive changes balanced out by several large negative changes causing a smaller overall positive change. Again, accounting for the differences between each of the workshop groups in areas such as transmission and social reform would require analysis that goes beyond the scope of this report, yet noting the differences seems important in relation to the overall evaluation of the workshops as a whole. A table with all three workshops changes in each category is provided for reference. A secondary analysis of TPI results consisted of examining the belief, intention, and action changes that occurred in participants. The average change for all three categories was positive and for the majority of participants resulted in a better alignment between all three categories. It can be inferred that the large positive change in beliefs is likely due to the number of participants in their first few years of teaching taking the workshop. This claim is corroborated by significant positive changes in beliefs for participants in their first few years of teaching along with less change for more experienced participants. Perhaps attending the workshop and working with other participants helped affirm their beliefs in relation to any teaching perspective. Changes in intentions and actions cannot be attributed to any subgroup of the participants and seem to be mostly attributed to alignment of beliefs, intentions, and actions as a result of attending the workshop. In general, the workshop and subsequent attempts at implementation likely affect participants’ perspectives on teaching. The data seem to support a positive effect in both the developmental and apprenticeship perspectives and a better overall alignment of participants’ teaching perspectives and their beliefs, intentions, and actions in relation to effective teaching practices. A table comparing the average changes in teachers TPI scores for all three workshops is provided (Below or in appendix) TPI Aggregate Comparisons Average Average Change Change TPI category UNI ISU Transmission 0.86 2.10 Apprenticeship 3.43 1.90 Developmental 2.43 2.50 Nurturing 2.95 -1.40 Social Reform 1.57 0.60 Belief Total 4.24 3.00 Intention Total 3.05 4.14 Action Total 3.62 2.71 Average Change U of I 0.00 1.00 1.50 0.00 2.80 NA NA NA Impact assessment: Student FCI Data and VASS Profiling To be consistent with first year modelers completing a modeling workshop at ASU, a high quality workshop should average a student post-test percentage of 52% or a normalized gain of 0.35. The graphic below is a summary of ASU’s findings for over 7,500 students and serves as the standard for comparing the effectiveness of the Iowa Regents’ Modeling Workshops. FCI mean score (%) 80 69 60 40 42 20 26 Post-test 52 26 29 Pre-test Instruction Novice Expert type Modelers Modelers FCI mean scores under different instruction types Traditional UNI Student FCI Scores: The UNI participants on average achieved a final post-test average of 50% and an average normalized gain of .32. These results are comparable to results generated by novice modelers attending ASU’s nationally recognized workshops and master’s degree program. However, there were 4 UNI participants producing results that fared worse than traditional instruction with final post-test percentages less than 42% and a normalized gain of less than .21. These participants were from both ends of the participant experience spectrum and were from schools ranging in their diversity and socioeconomic status. Factoring these four individuals out leads to an improvement of the average normalized gain for the remainder of UNI participants to .36 which is slightly greater than what would be expected from published information out of ASU’s Modeling Instruction program. Additionally, 12% of students scored at or above mastery in understanding Newtonian concepts with 32% of students achieving the threshold for having a strong conceptual understanding of Newtonian concepts. The two highest achieving participants were very diverse in experience and demographics. The highest achieving participant had 10 years of teaching experience, a 2% minority and 8% Free and reduced population, with a .61 normalized gain. The second highest achieving participant had 1 year of experience, a 22% minority and 62% free and reduced population, with a .55 normalized gain. There was a substantial difference in the number of students taking physics and the overall school population that might explain the differences. Similar instances can be claimed for participants with average and low normalized student gains. In general, there are instances of successful and unsuccessful implementation for the UNI participants in rural, urban, and mid-sized districts as well as districts that vary greatly in their minority and free and reduced population. This generalization is evident in all three years of data generated by the Iowa Regents’ Modeling Workshops. Student data summaries for all three workshops have been provided in the appendix. When comparing the workshops longitudinally, we have to keep in mind the UNI workshop included PRISIMS interactive engagement materials along with modeling materials. UNI participants self-reported in their reflection journals whether they implemented a Modeling approach or a mixed Modeling and PRISIMS approach during the academic year. Almost all participants who reported a Modeling approach provided evidence of using PRISIMS activities in their implementation journals and those that reported a mix of approaches varied in their use depending the concept or other instructional decision making factors like student abilities, math background, number of students, etc. Throughout the workshop participants were encouraged to think about activities from either approach in relation to meeting their students’ needs. Data was evaluated for those who self-reported consistently using a modeling approach. This sub group of participants generated a .41 normalized gain. This is consistent with the average gains of ISU and U of I modeling workshop participants which were .40 for all participants regardless of level of implementation. Evaluating a mixed approach with all remaining participants is a bit unfair given all four participants producing results with normalized gains worse than traditional instruction are in that sub group. The entire mixed sub group produced a normalized gain of .26. A calculation was also completed without the four participants with the lowest student normalized gains. An average normalized gain of .33 is figured indicating the majority of participants with a mixed approach still produce results consistent with interactive engagement methods yet slightly less than ASU’s modeling workshop data. It is also important to note that participants producing normalized gains less than those documented with traditional instruction occurred in all three workshops with two at ISU and one at U of I. VASS student profiling: The Views About Science Survey was also given to students by some participants. Some chose to give the survey to all students while others gave it to representative groups of students. While the survey can be quite informative about students’ attitudes and beliefs about learning and understanding the nature of science, the data reported have turned out to be far too general to substantially contribute to evaluating the Regents’ Modeling Workshops. For the data returned, there was some consistency in student FCI scores and VASS profiles. Generally the greater the VASS profile, the greater the post FCI score. However, there seems to be a much greater range of VASS profiles that produce post FCI scores above mastery than anticipated from the research. Additionally it was speculated that a correlation might exist between teachers’ average student VASS profile and the post FCI average. Initially it appeared to be the case with a small set of data but trend did not continue. Unfortunately no real insight was gained as a result of having participants give the VASS for the purposes of evaluating the Regents’ Modeling Workshops; however, it is possible individual teachers gained insight to their students’ attitudes and beliefs. Unfortunately, there was no follow-up with individual teachers giving the VASS to see if any insights were gained. Summary of Findings: The UNI workshop has clearly demonstrated the same level of success as the previous ISU and U of I Modeling workshops. The majority of participants indicated significant professional growth and changes in students’ attitudes and behaviors. The TPI provided evidence of greater balance in teachers’ perspectives of teaching and greater alignment of beliefs, intentions, and actions. Both participant and student FCI scores generated comparable results to ASU’s Modeling Instruction workshops. The UNI workshop brings to a close three successful years of Modeling workshops that would not have occurred without the support of the Carver Charitable Trust. We are grateful to have your support. Your support has provided approximately 75 teachers with some of the best science professional development available and will continue to affect students for many years into the future. Regards, Iowa Modeling Workshop Consortium Craig Ogilvie, Ph.D., Professor of Physics, Iowa State University Mary Hall Reno, Ph.D., Professor and Department Chair, Physics, University of Iowa Larwence Escalada, Ph.D., Professor of Physics, Chair Science Education Department, University of Northern Iowa Shannon McLaughlin, Physics Instructor, Norwalk High School Jeff Weld, Ph.D., Director, Iowa Math & Science Education Partnership Appendix 1 2011 ISU Modeling Workshop Participant FCI Scores 2012 U of I Modeling Workshop Part Number FCI Pre-Test FCI Post-Test Normalized Gain 1 30 30 0 2 30 30 0 3 30 30 0 4 30 30 0 5 29 29 0 6 28 28 0 7 18 18 0 8 11 14 0.16 9 15 19 0.27 10 20 23 0.3 11 26 28 0.5 12 20 26 0.6 13 14 24 0.63 14 27 29 0.67 15 23 28 0.71 16 23 28 0.71 17 18 29 0.92 18 19 30 1 19 27 30 1 20 28 30 1 21 29 30 1 22 29 30 1 AVERAGES 22.44 26.28 0.48 0.51 Normalized gain with Average Pre and Post Number FCI Pre-Test FCI Post-Test 1 22.00 NA 2 30.00 30.00 3 30.00 30.00 4 30.00 30.00 5 30.00 30.00 6 12.00 4.00 7 28.00 27.00 8 29.00 29.00 9 29.00 29.00 10 23.00 24.00 11 25.00 26.00 12 23.00 25.00 13 28.00 29.00 14 26.00 28.00 15 22.00 27.00 16 20.00 27.00 17 26.00 29.00 18 18.00 27.00 19 21.00 28.00 20 19.00 28.00 21 12.00 27.00 22 23.00 29.00 23 16.00 28.00 24 18.00 29.00 25 16.00 29.00 26 29.00 30.00 Averages 23.27 27.16 Average Normalized Gain N=5 through N=22 0.58 Normalized gain with Average Pre and Post Average gain N=6 through N=26 Average gain N=7 through N=26 Average gain N=8 through N=26 2013 UNI Participant FCI Scores Number FCI-PRE FCI-Post 27 25 1 25 24 2 30 30 3 30 30 4 29 29 5 27 27 6 18 18 7 10 12 8 18 21 9 19 22 10 15 20 11 14 21 12 21 25 13 6 20 14 27 29 15 11 24 16 26 29 17 11 29 18 29 30 19 27 30 20 29 30 21 29 30 22 27 30 23 Averages 21.96 25.43 Noralized gain with average pre/post Average of N=8 to N=23 Average if N=8 to N=18 Normalized Gain -0.67 -0.20 0.00 0.00 0.00 0.00 0.00 0.10 0.25 0.27 0.33 0.44 0.44 0.58 0.67 0.68 0.75 0.95 1.00 1.00 1.00 1.00 1.00 0.43 0.42 0.65 .55 ISU Participants Starting Below Mastery on FCI FCI Pre-Test FCI Post-Test Normalized Gain 18 18 0 11 14 0.16 15 19 0.27 20 23 0.3 20 26 0.6 14 24 0.63 23 28 0.71 23 28 0.71 18 29 0.92 19 30 1 18.1 23.9 0.53 Normalized Gain from averages 0.49 U of I Participants Starting Below Ma FCI Pre-Test FCI Post-Te 23.00 24.00 23.00 25.00 22.00 27.00 20.00 27.00 18.00 27.00 21.00 28.00 19.00 28.00 12.00 27.00 23.00 29.00 16.00 28.00 18.00 29.00 16.00 29.00 12.00 4.00 18.69 25.54 without the highlighted anomaly 19.25 27.33 Normalized Gain from averages UNI Participants Starting Below Mastery on the FCI FCI-Pre FCI-Post Normalized Gain 6 20 0.58 10 12 0.10 11 24 0.68 11 29 0.95 14 21 0.44 15 20 0.33 18 18 0.00 18 21 0.25 19 22 0.27 21 25 0.44 14.3 21.2 0.44 Average N gain 0.44 2013 University of Northern Iowa Student FCI, Vass, and Demographic Average Average N Teacher 1 N (students) 60 FCI Pre 7.40 FCI Post 14.75 Normalized Gain 0.32 Average VASS Profile NA % minority 6 %F& R 18 Notes on U/R implementation U Modeling 2 16 7.67 13.27 3 11 10.9 18.1 4 46 7.08 10.98 5 29 6.83 9.52 6 36 8.08 16.64 7 20 11.40 21.00 8 41 11.00 16.00 9 67 8.37 15.27 10 35 8.09 21.18 11 12 6.08 19.33 12 12 8.42 16.00 13 80 10.06 16.34 14 36 7.20 11.40 15 10 8.49 10.47 16 10 6.60 12.60 Averages 521 8.35 15.18 N gain average pre and post Weighted N gain based on N (students) Average N gain Full Modeling Implementaion Average N gain Mixed PRISIMS/Modeling 0.26 0.40 0.17 0.11 0.40 0.53 0.21 0.34 0.61 0.55 0.38 0.33 0.18 0.10 0.26 0.32 0.32 0.32 NA 150 NA 148 NA 130 NA NA NA NA 149 149 NA 153 NA 146.55 4 11 49 11 P 9 15 11 2 22 10 23 31 P 16 43 22 70 55 P 41 17 22 8 62 32 49 52 P 9 R R U M R R U U M M R U M M M Mixed Modeling Mixed Mixed Mixed Modeling Modeling Mixed Modeling Modeling Modeling Mixed Mixed Mixed Modeling Student Above Mastery Students at threshold for understanding 0.41 0.24 62 or 12% 165 or 32% 2013 University of Iowa Student FCI, Vass, and Demographic T# N Pre ave Post ave N gain ave N>24 VASS Weigh ted N Imple menta % Minori % F&R U/R tion 1 17 9.4 9.2 0.00 0 2 16 7.8 18.8 0.50 5 3 141 7.6 14.7 0.32 8 4 6 10 25 67 17 9.4 9.8 8.7 20.1 23.0 18.4 0.50 0.65 0.47 7 34 4 12 30 6.5 14.5 0.34 3 14 9 7.7 14.0 0.29 1 139.00 18 5 8.6 15.6 0.33 1 149.00 19 18 8.9 15.6 0.34 3 148.00 20 22 79 10 7.8 6.0 16.3 15.2 0.39 0.38 6 0 146.00 141.00 28 14 9.2 14.4 0.26 0 27 66 6.7 15.1 0.37 7 29 112 7.3 15.5 0.37 9 N=15 626 8.1 16.0 0.37 88 Stundets achieving Mastery 15% 136.00 146.00 127.00 ty Curr. 0.00 Constr 40 NA U aints 8.00 Full 20 15 U Most/ 45.12 short NA NA U time 12.50 Full NA NA Mid Size 43.55 Full 10 13 U 7.99 Most <5 45 R Most/l ow 10.20 NA NA R resour ces Most/l ow 2.61 NA NA R resour ces 1.65 Full <5 22 R Most/ Time 6.12 NA NA U constr aint 30.81 Full 20 15 U 3.80 Full 30 50 R Most 3.64 short NA NA R time Most 24.42 short NA NA U time Most 41.44 short NA NA U time 0.39 Average Weighted Normalized Gain 0.40 Average Weighted N gain - T#1 0.40 Average Weighted N gain - T#1 For Funded 2011 ISU Student FCI, Vass, and Demographic Teacher Number 14598 AP 56742 78574 84747 14598 Regular 14879 45668 57832 32585 42789 87888 N 60 41 6 16 61 4 12 4 22 49 73 FCI Post 24.47 20.00 21.00 18.61 18.43 17.50 17.25 17.75 17.09 14.20 12.45 FCI Pre 11.93 8.00 10.80 7.89 8.61 6.00 8.33 8.00 10.41 6.35 6.73 % Gain 41.80 40.00 34.00 35.74 32.73 38.30 29.79 32.50 22.27 26.19 19.06 N Gain 0.70 0.55 0.53 0.49 0.48 0.48 0.43 0.44 0.37 0.34 0.25 85784 97845 98766 9 11 15 15.00 17.73 11.27 10.22 10.91 6.53 15.93 22.73 15.78 0.29 0.37 0.19 Full through unit 6 Full through Unit 7 Full through Unit 6 0 2 9 56788 68522 11 93 487 13.91 12.10 7.00 7.92 23.02 14.56 0.31 0.20 Curricular constraints Low time/curriculum 57 14 16.80 8.48 27.78 0.39 0.40 18.07 8.46 32.04 0.45 0.46 Cohort Averages All Participants Average of N gain Cohort Averages of full Implemnt Average of N gain Percent of students achieving mastery on FCI= 15% VASS 150 150 150 147 144 148.06 Implementation Full 80% w/open inquiry Full Full Full 80%-competitions Full Full Full Full Full % minority 25 2 0 44 25 NA 7 1 0 18 5