Assessing Partnership Evolution K-12 Achievement, Supervisor and Grad Surveys, and TPAs (plus some very interesting data regarding selectivity) Our Context 2/26/2014 Fresno is the 5th largest city in CA Fresno is the 34th largest city in USA Fresno County is the richest agricultural county in the U.S. with $5.5B in agricultural production/year Brookings Institute ranks Fresno #5 for concentrated urban poverty, three years ago we were #1.. Fresno includes CA 18 of Rep. Jim Costa \ 436/435 + DC in mean family income Local Schools are about 35% English Learners Fresno State enrolls But Fresno has the lowest 23,000 students, graduates college completion rate of any 3700 students, and awards 900 graduate degrees each year…SMSA in the United States. 2/26/2014 CSU Teacher Education Post-Baccalaureate only Until 2013, must be able to be completed in a year C Must be subject matter competent through exam Emphasizes preparation for ELD and SpEd Collect data on all program completers Kremen Post Bac and Grad Enrollment 2/26/2014 Partner Schools Vision Vision - 2005 To enhance educational achievement in the Central Valley by preparing future teachers for the classrooms of the 21st century by building collaborative partnerships between university and public school faculty to strengthen the opportunities for careerlong learning and professional growth. Began 2006 with 1 Partnership IN 2011-12 8 Partnerships IN 2013-14 7 K-12 teaching 8 administration includes dual and Teacher Residency Distinguishing Features: Classes held on district school sites Teacher candidates are cohorted Daytime classes that are planned around the district schedule Partnership liaison faculty member assigned Field experiences are primarily in that district and are more closely connected to coursework Some faculty team teach courses with district partners (teachers, administrators) Partnership for Exemplary Teachers “Partner Schools” More Features: Courses taught with the district “lens” Co-teaching utilized, teachers are trained in unison with candidates Master teacher conference Character Education Conference Tried assigning candidates to schools’ PLCs, works better to have them be part of PLC Partnership for Exemplary Teachers “Partner Schools” Fresno State and PK-12 faculty revising program and syllabi Many courses team taught w/ K-12 Faculty Model Lessons in Partnership . Co Teaching Workshops • Provide substitute or, if on Saturday, pay master teacher a stipend (sub pay) • Master teacher conference •Walk through with District and Fresno Pacific 2/26/2014 Prepare teachers for the valley context Urban/Rural Diverse & linguistically rich Poverty All students achieve Be seen as an indispensible partner for districts/COEs Supt Hanson specifically requested a partnership to prepare upper grade and middle school teachers for success with all students with a particular emphasis in math and science. Spring 2012 Partnership Study: Qualitative Component • Purpose: To learn about the effects of the Partnership Program over time • Theoretical framework: transformative paradigm (Mertens, 2005) Evolution- originally MS only 3 semesters…then 12 month… Then Ed Administration Then Single Subject Then Dual Multiple and SpEd (two year) Then Teacher Residency Next tchr residency/masters and credential Tchr Residency Math/Science basic credential Next Linked Learning Secondary 2/26/2014 evolution From us going to them to them coming to us and requesting e.g. Dual tcher residency Porterville Washington Union rural Fresno Unified/Fresno State Bechtel Grant Teacher Residency Program FALL 2013 With partner Fresno Unified Bechtel granted a $470,000 grant to support the preparation of teachers for grades 4-8 with emphasis on Science and Math. FUSD Bechtel Grant Teacher Residency Teacher Candidates commit to the 14 month program and being on sites all day. Committed to increased educational experiences in math and science earning a foundational credential in math and/or science. If hired commit to work for FUSD for 3 years. Are provided a stipend $12,500. FUSD Bechtel Grant Teacher Residency 2014-2015 Masters and Credential (four terms) Bechtel has tentatively committed to 3 more years at up to $300,000/year Assessment of Partnership Pathways Effect on program changes on specific skills K-12 Achievement Qualitative TPAs Selectivity..unanticipated outcome 2/26/2014 Survey of Employers and Graduates Employment supervisors and graduates at the end of the graduates first year of professional teaching. 110 items 5 broad areas 26 composites 26 Teacher Preparation Composites Teacher Composite B-1: First-Year Multiple-Subject Teachers Assess Their CSU Preparation for Reading-Language Arts Instruction (K-8) Where and When Teachers Were Prepared Well Prepared Adequately Prepared Somewhat Prepared 53% Not-at-All Prepared 32% 12% CSUFr 07-08 85% CSUFr 06-07 79% CSUFr 05-06 81% CSUFr 04-05 82% CSUFr 03-04 85% CSUFr 02-03 88% CSUFr 99-02 83% System 07-08 85% System 06-07 84% System 05-06 81% System 04-05 83% 53% 30% 13% 4% System 03-04 83% 52% 31% 13% 4% System 02-03 49% 46% 77% 0% 17% 35% 48% 34% 33% 47% 34% 51% 25% 50% 3% 12% 3% 14% 37% 19% 75% Percentages of Teaching Graduates Whose CSU Preparation Was Assessed One Year After It was Completed 2% 12% 15% 33% 40% 0% 14% 31% 51% 2% 12% 34% 54% 5% 13% 42% 49% 2% 13% 30% 46% 4% 18% 55% 84% System 99-02 30% 3% 4% 3% 5% 100% What demonstrates the validity of the data? • Quality of preparation is not related to number of units required in the credential program. • Quality of preparation is not related to size of program • Employment supervisor ratings are not related to: – Percent eligible for free or reduced lunch – Percent of English Learners – Achievement decile of the school – Number of emergency credentialed teachers in the school No Effect on Results Student SES/poverty level Student EL status Graduates’ Parent Education or Income Number of courses/units in credential program Number of students in the program (boutique vs large) P-12 school achievement level at which first year teacher teaches Specific Attempts to Improve Preparation Resulted in Improved Ratings 2/26/2014 Well Prepared in Technology 2/26/2014 Teach English Learners 2/26/2014 Areas Without Improvement Plan 2/26/2014 Pathways to Teaching Traditional Campus-Based Partner Schools Internships All 3 have exactly the same courses, texts, & pool of instructors Traditional Campus-Based Program Classes held on campus Usually at night Students are not cohorted Field experiences are spread across various districts and only loosely connected to coursework Internships District has hired teacher candidate prior to completing preliminary credential Courses are the same as full time teacher candidates Courses are taken in the evenings Interns are assigned a supervisor Interns attend monthly workshops (Friday evening and Saturday) Survey Composites Advantage of Partnerships Overall effectiveness of basic teaching credential program in the CSU system. +10 Preparation to understand and teach core subjects of school curriculum at distinct levels. +10 Preparation of general pedagogical principles and practices. +17 Preparation to teach California’s students in diverse groups and stages of development. +20 Overall quality of the CSU teacher preparation. +10 Cohorted MS Programs and Non-Cohorted MS Program Graduates at CSU Fresno Rate Their Preparation Overall Prep C 38% Overall Prep NC 41% 24% 42% 35% Value Fieldwork NC Assess & Reflct C 6% 24% 41% 26% 50% 5% 8% 13% 41% 25% 2% 16% 38% 25% 5% 21% 34% 43% 2% 11% 43% 30% 0% 21% 42% 45% 5% 12% 40% 30% 2% 12% 36% 45% Manage Clsrm C 8% 23% 50% Motivate Stds C 4% 23% 34% 3% 18% 60% Plan Instruction NC Assess & Reflct NC 13% 67% Plan Instruction C 7% 34% Value Fieldwork C 4% 26% 50% Value of Courses NC Manage Clsrm NC 17% 41% Value of Courses C Motivate Stds NC Not-at-All Prepared Somewhat Prepared Adequately Prepared Well Prepared 75% Percentages of Teaching Graduates Whose CSU Preparation Was Assessed One Year After It was Complete 4% 7% 100% Cohorted MS Programs and Non-Cohorted MS Program Graduates at CSU Fresno Rate Their Preparation 26% Prep Sp Lrners NC Somewhat Prepared Adequately Prepared Well Prepared 40% 28% 38% Prep Sp Lrners C 40% 26% Prep Engl Lrnrs NC 41% 41% Use Technology NC Use Technology C 0% 40% 25% 33% 34% 30% 50% 6% 16% 34% 25% 4% 26% 41% Use Mix Strat C 7% 13% 41% 27% 4% 26% 43% Use Mix Strat NC 6% 16% 41% Prep Assess/Refl C 4% 26% 41% 25% Prep Assess/Refl NC 9% 16% 41% 40% Prep Equity/Div C 4% 25% 45% 27% Prep Equity/Div NC 5% 13% 38% 35% Prep Middle Gr C 5% 25% 45% 25% 7% 17% 43% Prep Engl Lrnrs C Prep Middle Gr NC Not-at-All Prepared 4% 11% 24% 75% Percentages of Teaching Graduates Whose CSU Preparation Was Assessed One Year After It was Complete 8% 100% • Cohort model • Liaison support • Investment in the culture and language of a school district • Stronger bridge between theory and practice – especially through team teaching and modeling lessons TPA • TPA Outcomes • TPA with survey 2/26/2014 Achievement 2/26/2014 2/26/2014 2/26/2014 2/26/2014 http://edresults.org/ccag/visits/centralvalley/a chievequadv2.php How the California State University Examines K-12 Student Learning as One Outcome of CSU Teacher Education Measuring K-12 Student Learning as a Teacher Education Outcome in California Measured Progress by California Students in Learning Important Ideas and Abilities 1. Factors Beyond the Control of Policymakers and Educators in California: • Parent Education Levels • Family Income and Assets • Students’ English Proficiency when Instruction Begins • Students’ Other Special Needs CSU-CTQ Slide 2 Measuring K-12 Student Learning as a Teacher Education Outcome in California Measured Progress by California Students in Learning Important Ideas and Abilities 1. Factors Beyond the Control of Policymakers and Educators in California: 2. Factors that California Policymakers and Educators Influence: • Parent Education Levels • Family Income and Assets • Students’ English Proficiency when Instruction Begins • Students’ Other Special Needs • Learner Populations Previously Underserved by Education • Teacher-Student Ratios and Class Sizes • Cultures of Poor Performance in Many Schools CSU-CTQ Slide 3 Measuring K-12 Student Learning as a Teacher Education Outcome in California Measured Progress by California Students in Learning Important Ideas and Abilities 1. Factors Beyond the Control of Policy- 2. Factors that California Policymakers and Educators in California: makers and Educators Influence: • Parent Education Levels • Family Income and Assets • Students’ English Proficiency when Instruction Begins • Students’ Other Special Needs CSU-CTQ Slide 4 • Learner Populations Previously Underserved by Education • Teacher-Student Ratios and Class Sizes • Cultures of Poor Performance in Many Schools 3. Factors that Policymakers and Educators Influence or Control in California: • Traditional and Alternative Pathways into Teaching • Alternative Ways for Candidates to Demonstrate Subject-Matter Competence • Differences Between Bilingual and Monolingual Preparation Programs • Focused or Infused Treatment of Significant Pedagogical Topics Measuring K-12 Student Learning as a Teacher Education Outcome in California Measured Progress by California Students in Learning Important Ideas and Abilities 1. Factors Beyond the Control of California Policymakers & Educators: 2. Factors that California Policymakers and Educators Influence: 3. Factors that Policymakers and Educators Influence or Control in California: • Parent Education Levels • Family Income and Assets • Students’ English Proficiency When Instruction Begins • Students’ Other Special Needs • Student Populations Previously Underserved by Education • Teacher-Student Ratios and Class Sizes • Cultures of Poor Performance in Many Schools • Traditional and Alternative Pathways into Teaching • Alternative Ways for Candidates to Demonstrate Subject-Matter Competence • Differences Between Bilingual and Monolingual Preparation Programs • Focused or Infused Treatment of Significant Pedagogical Topics CSU Assesses the Effects of Teacher Education’s Features on K-12 Student Learning by . . . 1.Compiling student data that includes all of the measured factors in Sets 1, 2 and 3 above, so each student’s record includes data on the features of preparation experienced by each teacher who taught core subject(s) to that student in a given year. 2.Focusing on academic “learning” and not “status” by measuring each student’s academic growth in each year. 3.Reporting the statistical effects of each factor in Sets 1-3 while statistically minimizing the effects of every other factor in Sets 1-3 on annual growth in math, science, reading-language arts, and history-social studies by large numbers of pupils. 4.Providing each university with statistical results that have “actionable implications” for teacher education decision-makers. 5.Providing each university with a comprehensive file of linked student–teacher data for faculty researchers to analyze. CSU-CTQ Slide 5 Value-Added Research What is the value of having a teaching credential from the CSU? First large scale study of the value-added by teacher preparation programs CSU versus non-CSU teacher preparation programs Sample = 25,530 students and 925 teachers All between-group contrasts are significant at the .01 level. The Relative Importance of 12 Educational and Demographic Factors that Account for Year-to-Year Gains in Mathematics by Students in Urban Middle Schools and High Schools Class Size Asian-White Gap Parent Education Level Family Income Level Student's Gender Student's English Proficiency Level Years of Teacher's Experience In urban secondary schools, students learned more mathematics when taught by CSU first- and second-year teachers than by other first- and second-year teachers. The orange bar represents the relative importance of this factor in student learning. (1) For urban students trying to learn mathematics in grades 7-12, the difference between CSU teacher preparation and non-CSU teacher preparation (orange bar) was: ● 27 times more influential than class size, ● five times more influential than parent education level, and ● four times more influential than family income level. (2) Also for students in 7-12, the greater effectiveness of CSU preparation (orange) was: ● 1.6 times more influential than student English proficiency level, and ● 1.2 times more than the gap between first-year and second-year teachers. CSU Teachers and Other Teachers Latino-White Gap Black-White Gap Student's Disability Status (3) The CSU difference was also equivalent to: ● 81 percent of the Latino-white gap, ● 52 percent of the black-white gap, and ● 44 percent of the gap between students with and without disabilities. Student's Prior Level of Math Learning Intellectual Property of -0.1 the CSU Center for Teacher Quality 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Standard Deviations in Students’ Year-to-Year Gains in Secondary Math for College and Career Readiness Alternative Pathways into Teaching as a Factor in K-12 Student Learning: The Effects of Alternative Pathways Compared with Eleven Other Educational and Demographic Factors Class Size Asian-White Gap In urban secondary schools, students learned more mathematics when taught by former CSU student teachers than by former CSU intern teachers. The blue bar represents the relative importance of these alternative pathways for student learning. (1) The greater effectiveness of CSU supervised teaching over CSU intern teaching (blue) was: Parent Education Level ● eighteen times more influential than class size, ● three times more influential than parent education level, ● two times more influential than family income level, and ● 1.1 times more influential than student English proficiency level. Student's Gender Family Income Level Student's English Proficiency Level Former Interns, Former Student Teachers Years of Teacher's Experience (2) The greater effectiveness of student teaching also was: ● 72% as influential as the difference between first-year and second-year CSU teachers, ● 45% as influential as the gap between Latino students and white students, ● 39% as influential as the black-white gap, and ● 31% as influential as the gap between students with and without disabilities. Latino-White Gap Black-White Gap Student's Disability Status Student's Prior Level of Math Learning 0.000 CSU-CTQ Slide 6 0.120 0.240 0.360 0.480 0.600 Standard Deviations in Students’ Year-to-Year Growth in Secondary Mathematics for College & Career Readiness Qualitative 2/26/2014 Data Collection: Focus Groups Spring 2006 • Student Teachers 6 focus groups 3 districts 26 student teachers • Master Teachers 2 focus groups 2 districts 13 master teachers • Principals 1 focus groups 2 districts 3 principals • University Director 2 co-directors Fall 2011, Spring and Fall 2012 • Student Teachers 3 focus groups 3 districts 32 student teachers • Master Teachers 2 focus groups 2 districts 13 master teachers • Principals 1 focus groups 1 district 4 principals • University Director 1 director Student Teacher Reasons for Choosing the Partnership • Initial reasons Convenience of location Job and children’s after school schedules • Later reasons – support “Because I had two friends. One didn’t do a partnership and one did. That was just a whole big deal of difference, all the things that they experienced. And you notice it once you’re done, once you look back on it. Like, ‘What did I learn? What did I take from it?’ And I can say that when I was in the Partnership cohort, I took out more, had more to say, had more to look back on positively. And the friend, I mean I’ve never seen him teach or anything, but he just regrets it. He wishes he could have done the Partnership because there was that time when he had questions, and none were answered . . . And he succeeded, but not the way he wanted.” “I wish everyone could see the benefit of the Partnership, so hopefully we were of some assistance so people will know to take advantage of it.” • All student teachers in 2006 and 2011 would choose the Partnership Program again Student Teacher Perspectives • • • • • • • Collaboration, networking and professional partnerships Did everything together, learned from each other “We learned a lot from each other. We just took each other’s ideas and strategies, and we implemented it in our own work and classroom, and that really helped us. When we would have conversations about what to do with this kid, what should I do with this project, and so we really collaborated even outside of class. Emailed each other, ‘Hey, can you help me with this?’ We’ve all learned to work with each other and I feel like just having this positive experience is going to benefit us.” Student teacher bonding “We’ve grown as one big family.” Vicariously exposed to multiple teachers’ approaches, skills and resources “If I had a problem in my classroom, I can go to a one of us and ask, ‘How does your teacher handle this? This is how my teacher handled it.’ Even though we were only in one placement, we’re actually getting experience from all the other placements and teachers because we had the time to share and talk about it.” Value other’s solutions, anticipating PLCs Value more than one person in the classroom, anticipating co-teaching Developed strong sense of commitment to all students regardless of poverty or ethnicity Master Teacher Perspectives • Initial perspectives Work to organize Teachers initially unclear of what was expected of them Scheduling issues • Later perspectives More comfortable with system More willing to have student teachers & utilize them strategically Appreciate university liaisons Teachers greatly affected by PLCs, more flexible & student-focused, less autonomous Both initial and later perspectives Student teachers were prepared Occasionally think a student teacher should not enter the profession Expect student teachers to be willing to work Student teachers aren’t ready to take over initially “They were very willing to learn, but they were not quite competent to take charge,” but they are when they finish Responsibility to help train the next generation of teachers Want student teachers to experience start of school year • Principal Perspectives Principals not as personally involved with student teachers Student teachers are better prepared than they used to be Student teachers willingly help wherever they are needed Start on the first day View student teachers as potential hires “They’ll say anything to get a job interview, but it’s different when you get to see who they are over the course of a semester. I’ll tell you, at the job fair, those who have been placed in the cohort, even not in a placement at my school, you can tell them above the others.” • Partnership placements connect student teachers to the schools “When I did student teachers before, they just went to the classroom and the university. They were more isolated. You just closed the door and taught. That was it.” • PLCs have improved teacher performance which benefits student teachers “Five years ago, the way we’d support a student teacher would be fairly basic compared to today, but I think that is because of the capacity of our teachers in our district has grown so much, not to just internalize innovative instruction, but being a part of a PLC, and how to respond when kids aren’t learning.” • • • • • Theme: Institutionalization of Collaboration • Change from individualistic to collective perspectives • Family-like support and collaboration • Student teacher initiation • Reward systems and professional development • Accountability to non-supervisors Change from Individualistic to Collective Perspectives • • • • Initially, teachers spoke in first person singular “I have an hour to do science and history.” Later, teachers spoke in first person plural “We’ll be there to help them.” Initially, teachers spoke about autonomy in their classrooms “I made my student teachers do everything. I made them come on Fridays sometimes. I made them take over the morning work. I made them just do lessons. I would model it, and I would watch them do it.” Later, collective responsibility for all students “They’re all mine. All the 3rd graders are mine. I know all the 3rd graders. We (3rd grade teachers) all take care of them.” • Student teachers experience collective perspective “We feel comfortable. So I’m comfortable asking any teacher around here for help with anything, not just go to my master teacher.” • Teachers model collaboration, possibly from working together in PLCs Family-like Support and Collaboration • • Great camaraderie at all levels Deep sense of support is like a “family” Student teacher “We’ve bonded. We’ve grown as one big family. We’re all here to help each other.” Student teacher “If any one of us got a job. We’d be like, ‘That’s awesome for you! Great job! Let’s go out and celebrate!’” Master teacher “Those student teachers have lunch together every day, and they’re just like a big family.” Principal “You see them taking that ownership of being part of the school family and environment, showing up to carnivals and Friday night movie nights with their families, and they’re part of that school. That’s something they’re not asked to do or have to do, but they take pride and when they come back and their job duties are over, those kids model them like they’re rock stars because they miss them.” Student Teacher Initiation • Partnership student teachers experience accelerated growth in intensive context Exposed to many ideas from multiple teachers, Learn more through synergy of sharing “We’re actually, you could say, placed in a lot of different classrooms because we were able to share our experiences . . .because some things that happened in the classroom are similar and some of it is different, and then to know that the teachers were different themselves. . . hearing how other teachers, other master teachers would handle a classroom management situation ‘my teacher did the best thing today,’ and then we’d tell everybody and then we’d all make a little note, and we have this list of good classroom ideas, and it’s impossible for me to go sit in 20 different classrooms, but I got to observe what other teachers did.” • Master teachers view their tasks more collectively Strategically “deployed” “If the 4th grade teachers need help, we deploy student teachers there. We use them strategically because we can always use extra help. They are always busy, and they learn more about different grade levels that way.” Student teachers fully participate in the Professional Learning Communities, “Planning was really done together.” Teachers value new ideas and information “They have new ideas and a different way of saying it.” • Student teachers assume a teacher’s identity earlier “I really liked this semester. We did a lot more team teaching rather than, ‘I’m the master teacher. You’re the student teacher. We don’t really work together.’ This semester we’re both up there, we’re both teaching, we’re both pulling students, doing the exact same work. You know, ‘you take the low students, and I take the high students’ and really working together using two teachers rather than ‘I’ll sit in the back and watch you.’ I really, really loved it!” Reward Systems and Professional Development • Initial ideas about rewards as externals Some teachers expected payment, principals opposed this University faculty offer teachers course work Teachers feel responsible for training next generation of teachers • Later ideas about what was valuable is internal Improving is rewarding, “I don’t want a pat on the back. I want to know, ‘OK, How can I make this better? How can it make me a better teacher?’ because that’s my goal.” Sharing is rewarding, “Because we’re together all the time, we’ve learned who is strong in what and who is weak in what, and so we were able to pair up . . You’re good in doing this and I am good in doing this so we can work together.” Student teachers honored by grade level teachers choosing their lesson plans University faculty modeling lessons in teachers’ classes Teachers like extra help “I’m walking around helping and then I have suddenly 10 hands in the air. It helped to have 3 adults walking around answering questions, giving that individual help.” Teachers and principals like new ideas. “I also like the ideas that she has. We always try to stay fresh and on top of things.” Teachers still feel responsible for training next generation of teachers • Affective rewards: Quality was a source of pride for all Data Collection: Focus Groups Spring 2006 • Student Teachers 6 focus groups 3 districts 26 student teachers • Master Teachers 2 focus groups 2 districts 13 master teachers • Principals 1 focus groups 2 districts 3 principals • University Director 2 co-directors Fall 2011, Spring and Fall 2012 • Student Teachers 3 focus groups 3 districts 32 student teachers • Master Teachers 2 focus groups 2 districts 13 master teachers • Principals 1 focus groups 1 district 4 principals • University Director 1 director Sanger’s success can be attributed, unequivocally, to our partnership with Fresno State Superintendent Marcus Johnson 2011 Superintendent of the Year selectivity 2/26/2014 Contextual Demographics Some unanticipated findings… Family Income of Fresno State Graduates (Indicated when applying to a CSU system university) No differences in rating among the groups. Correlation <.10 between income level and principal rating. Parental Education Level for Fresno State Elementary Graduates No differences in rating among the groups. Correlation <.10 between education level and principal rating of graduate. 2/26/2014 2/26/2014 2/26/2014 Supervisor Rating to SAT & GPA 2/26/2014 Supervisor Rating and CBEST 2/26/2014 Lessons Learned District Partnerships School Partnerships Cohorts Co-Teaching Spiraling Curriculum Systematic Assessment Supervision/Coordinator University Liaisons Connection of coursework with fieldwork 2/26/2014