Middle Grades Education 2012‐13 Page 1 of 10 Middle Grades Education – Initial Preparation Annual Program Report Academic Year 2012‐13 Rebecca Stobaugh November 1, 2013 1. Continuous Assessment Results a. Admission Data Table 1 provides the average admission test scores and admission grade point average (GPA) of Middle Grades candidates approved by the Professional Education Council (PEC) for admission into initial teacher preparation programs during this academic year. Before the Office of Teacher Services submits their names for review and approval by the PEC, candidates must meet minimum requirements established by the state and/or the WKU Professional Education Unit. Table 1. Approved Candidate Test Score Averages Program Middle Grades ACT N Mean 51 24 PPST Math N Mean 3 180 PPST
Reading N Mean
4
178
PPST
Writing N Mean
3
176
SAT N Mean
GRE Composite N Mean Admission GPA N Mean
66
3.26
b. Course Based Assessment Data Table 2 provides the percentage of Middle Grades candidates (N = 233 scoring at each level of proficiency on critical performances within education courses for this academic year. Proficiency levels are based on a scale of 1 – Standard Not Met, 2 – Standard Partially Met, 3 – At Standard, and 4 – Above Standard. Table 2. CP Proficiency Level Percentages Course MGE‐275 MGE‐385 MGE‐475 MGE‐481 1 Grand Total 2 0.00%
5.98%
2.38%
6.06%
3 32.35%
29.06%
97.62%
84.85%
4 67.65% 64.96% 0.00% 9.09% 9.875 60.97 35.425 Compared to the unit‐wide results, Middle Grades candidates are typically performing above average with around 90% meeting or exceeding proficiency on the CPs. Middle Grades Education 2012‐13 Page 2 of 10 Table 3 indicates the level of Middle Grades candidates (N = 190) proficiency across critical performances related to the Kentucky Teacher Standards (KTS). Candidates receiving an overall rating of 3 or 4 on a CP are considered to have demonstrated proficiency on the standards associated with the CP. Across all standards, Middle Grades students performed exceptionally well. The lowest standard was KTS 9 with 94% of MGE candidates scoring proficient. Table 3. Percent of Middle Grades Candidates Scoring Proficient on CPs by KTS Program Middle Grades Unit‐Wide 1 95% 97% 2 97% 97% 3 97% 97% Kentucky Teacher Standards 4 5 6 7 99% 96% 97% 98% 98% 97% 97% 97% 8 96% 98% 9 94% 97% 10 98% 99% *KTS Key: 1 – Content Knowledge, 2 – Designs/Plans Instruction, 3 – Maintains Learning Climate, 4 – Implements/ Manages Instruction, 5 – Assessment/Evaluation, 6 – Technology, 7 – Reflection, 8 – Collaboration, 9 – Professional Development, 10 – Leadership Table 4 indicates the number of Middle Grades candidates (N = 20) who have scored 2 or lower (below proficiency) on critical performances during this academic year. Low scores on CPs have dropped significantly as faculty provided remediation for those struggling on CPs. Table 4. Middle Grades Candidates Scoring Below Proficient on CPs Student ID 800587622 800356916 800588695 800487294 800595265 800488013 800715262 800508244 800533157 800618078 800537935 800683410 800523607 800688249 800486666 800701802 800534246 800712162 800501824 800666180 Score 1 Student Count 2 1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 Middle Grades Education 2012‐13 Page 3 of 10 800775600 Grand Total 2
18
4
2 22 c. Clinical Experiences Data Middle Grades uses the following courses and experiences to evaluate candidate dispositions: MGE 275, MGE 385, and MGE 475/481. The program has identified the following courses and experiences where candidates report the diversity of their field experiences: MGE 385. MGE 385 has been designated as the experience where candidates must work in settings at or above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 5 reports how Middle Grades candidates performed on dispositions as they entered and progressed through their program (N = 148) and during their student teaching experience (N = 61). Students are considered “proficient” who average a 3 or higher on each disposition category. Table 5. Middle Grades Proficiency Rates on Unit‐Wide Dispositions Period a. Prior to Student Teaching b. During Student Teaching Values Learning WKU Professional Education Dispositions Values Personal Values Values Values Integrity Diversity Collaboration Professionalism 97.74% 100.00%
100.00%
100.00% 100.00%
100.00% 100.00%
100.00%
100.00% 100.00%
Over this academic year, Middle Grades candidates (N = 148) reported demographic information on field placements with an average of 51.1% ethnically diverse students, 49.4% students on free/reduced lunch, and 11.4% student with disabilities (based on National Center for Education Statistics and Kentucky Department of Education). Table 6 reveals the percentages of field experiences with various characteristics. Note that candidates could choose all the characteristics that applied for any given experience. Table 6. Percentages of Field Experience by Category Types Working with Student With Special Needs % Candidates working with Students with Physical Impairments % Candidates working with Students with Learning Disabilities % Candidates working with Students with Moderate/Severe Disabilities % Candidates working with Students with Emotional/Behavioral Disorders 36%
67%
7%
20%
Middle Grades Education 2012‐13 Page 4 of 10 % Candidates working with Gifted Students % Candidates working with English Language Learners % Candidates working with Students with Visual Impairments % Candidates working with Students with Hearing Impairments % Candidates working with Students with Speech/Language Delays % Candidates working with Students with Development Delays % Candidates working with Students with Autism Spectrum Disorder % Candidates working with Students with Other Impairments Working with Diverse Students % Candidates working with African American Students % Candidates working with Native American/American Indian Students % Candidates working with Latino/Hispanic Students % Candidates working with Asian Students % Candidates working with Students with Special Needs (Aggregate) % Candidates working with Diverse Students (Aggregate) 65%
39%
5%
3%
8%
7%
6%
2%
91%
12%
81%
65%
87%
96%
Overall, as can be seen in Table 6, in 87% of their field experiences Middle Grades candidates reported working with at least one student with special needs and in 96% of their field experiences candidates reported working with at least one student from a diverse ethnic group. d. Culminating Assessment Data As Component 4 of the WKU Professional Education Unit Continuous Assessment Plan (CAP) strategy, all initial preparation candidates complete a culminating assessment of professional and pedagogical knowledge and skills, the Teacher Work Sample (TWS). This assessment is also used to demonstrate candidates’ ability to impact P‐12 student learning. In particular, candidate performances on Assessment Planning and Analysis of Student Learning have been identified as key indicators of candidates’ ability related to student learning. Although in spring 2008 the Professional Education Council agreed that candidates who score a holistic score of at least “2 – Developing” are able to exit the program, for program evaluation purposes our goal is that at least 80% of program candidates will achieve “3 – Proficient” or higher. Table 7 presents the proficiency rate for Middle Grades candidates (N = 34). MGE students had slightly lower proficiency rates that other program areas. Table 7. Initial Preparation TWS Proficiency Rates Program Middle Grades Unit‐Wide % Proficient 86% 93% Because the faculty also scores TWS at the indicator level, we are able to use these scores to ascertain candidate success in meeting each component of the TWS. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale Middle Grades Education 2012‐13 Page 5 of 10 (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 8 depicts the percentage of Middle Grades candidates who averaged at least 2.5 on the indicators for each TWS Factor: CF – Contextual Factors, LG – Learning Goals, DFI – Design for Instruction, ASL – Analysis of Student Learning, and ROT – Reflection on Teaching. The ASL and ROT sections were the lowest areas with 74% and 77% passage rates. These are the last two sections of the TWS and classes prior to student teaching do not practice these sections. Table 8. Initial Preparation TWS Proficiency Rates of Middle Grades Candidates Program Middle Grades Unit‐Wide CF 80% 89% LG 94% 91% DFI 86% 85% ASL 74% 91% ROT 77% 88% Because the TWS indicators have been aligned to Kentucky Teacher Standards, we can use these scores to ascertain candidate success in meeting each standard related to the TWS.
Table 9 reports these scores as they relate to Kentucky Teacher Standards. Table 9. Percentage of Middle Grades Candidates who “Passed” each Teacher Standard Program Middle Grades Unit‐Wide 1 82.86%
89% 2 94.29%
94% 3 88.57%
92% 5 97.14%
90% 6 88.57%
92% 7 74.29%
91% 9 77.14%
88% Additionally, all candidates are assessed during their student teaching experience using the Student Teaching Evaluation form. Table 10 reports the percentages of Middle Grades student teachers (N = 61) successful on each standard. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 10. Middle Grades Proficiency Rates by Kentucky Teacher Standards Program Middle Grades Unit‐Wide 1 97% 97% 2 90% 92% 3 94%
96% Kentucky Teacher Standards 4 5 6 7 92%
87%
93%
92%
91% 87% 90% 86% 8 95% 93% 9 99%
97% 10 91%
89% e. Exit and Follow Up Data Table 11 delineates the Educational Testing Services reports of the pass rates on the Praxis II content exams of candidates who completed the program in the 2011‐12 academic year (the Middle Grades Education 2012‐13 Page 6 of 10 most recent year with complete data). The last column allows for pass rate comparison of our candidates to our 2010‐11 results. Table 11. Pass Rates on Content Tests for Initial Teacher Preparation Program/Type of Assessment Candidate N (2011‐12) WKU Pass Rate (2011‐12) 24 31 28 12 88% 100% 82% 100% MIDDLE SCHOOL ENGLISH LANGUAGE ARTS MIDDLE SCHOOL MATHEMATICS MIDDLE SCHOOL SOCIAL STUDIES MIDDLE SCHOOL SCIENCE WKU Pass Rate (2010‐11) 94% 100% 89% 93% Annually, the WKU Teacher Survey is sent to student teachers and alumni who potentially have been teaching one or more years. For the 2012‐13 academic year, out of a possible 406 student teachers, 375 (92%) completed the survey. Below are the results for Middle Grades student teachers, 56 of whom responded. Survey items requested the respondent’s perception of WKU preparation on each of the Kentucky Teacher Standards using a scale of 1 “Poor,” 2 “Fair,” 3 “Good,” and 4 “Excellent.” Standards with average scores of 3 or better across items were considered to demonstrate acceptable program quality. Table 12 reports Middle Grades survey results. Results from MGE candidates are slightly lower in most areas excluding KTS 6. Table 12. Average Scores on Teacher Standards Questions for Middle Grades Respondents Program Middle Grades Unit‐Wide 1 3.34 2 3.25 3 3.48 Kentucky Teacher Standards 4 5 6 7 3.30 3.17 3.42 3.07 8 2.96 9 3.24 10 2.97 3.49 3.45 3.66 3.43 3.14 3.36 3.18 3.29 3.37 3.30 Respondents were also able to provide comments. Table 13 presents Middle Grades respondent comments. Table 13. Middle Grades Respondent Comments As far as implementing a plan for engaging in leadership activities, I feel that nothing was addressed, except when someone said something about needing to engage in a leadership activity (or maybe that was on a paper somewhere?). I don't feel I've been prepared in any way to take a leadership role or even what that entails. I currently help with chess club. Is that enough? Should I be a leader in the teaching community instead? I don't know; this really hasn't been addressed much in the four years I've been at WKU. Standard 8 was treated in much the same way: the only time it has been addressed is briefly in my most recent teacher work sample and during some of the work I completed for LTCY 421. Develop a plan for engaging in leadership activities, implement a plan for engaging in leadership activities, and analyze data to evaluate the results of planned and executed leadership efforts were all marked poor because in the WKU courses taken there was very Middle Grades Education 2012‐13 Page 7 of 10 little discussion about providing leadership within the school. None of my classes had an assignment where a plan for engaging in leadership activities was developed and without a plan it is impossible to implement and analyze data from that plan. In the future students can be given an assignment where they can work either as an individual or as a group in creating a plan for leadership. If the students cannot implement the plan at the prospective school/grade level then the students can implement the plan at WKU. This would give potential teachers experience in creating, implanting, and analyzing a leadership plan as well as making the WKU community a better place. Having a leadership assignment prior to student teaching would beneficial! I believe that if WKU were to implement the use of data in earlier classes, it would have been more beneficial. I also think that it would be wonderful if WKU were to have students complete a leadership activity before their student teaching semester. I believe there should be more than just one online class to address students with special needs. I don't think that we had an actual class that really prepared us for student teaching in general. I felt like I had to learn a lot of this on my own through experience. I think that WKU should really look into making a class that specifically lets you know what you are stepping into when you student teach. I don't feel like I was prepared at all and was really overwhelemed at one point during my student teaching experience. I personally do not feel that WKU prepared me at all for teaching. Instead of learning ways to become a better teacher, I heard too many stories of old teaching experiences that did not relate to what I need to learn. I wasn't prepared to create lessons in the A2 format for student teaching. I wasn't prepared to create assessments for every student teaching observation and I wasn't prepared to implement higher level thinking questions in my lesson. If it wasn't for excellent teachers that I had this semester during my student teaching, I would not have been able to do this on my own! I wish that I could have been dual‐certified in Math and Science. I LOVE the SKyTeach philosophy ‐ it is definitely prepared me in ways that I feel I would have lacked otherwise. While I appreciate the idea of having teachers specialize in their content area, many schools around the area are currently looking for teachers who are dual certified. I'm not sure how WKU could improve Standard 10, but I wasn't prepared for the amount of engagement with the community. It's hard to facilitate that when Student Teaching because you don't really have very many rights as a teacher, but it would have been nice to know it was expected. My lesson plans could have required me to use different types of technology in each plan so that I learned how to use multiple types of technology. Through our education process there needs to be opportunities, if not a whole class, devoted to integrating technology in the classroom. Especially students use of technology. When developing a plan of action for areas needing improvement, more instruction could have been used. The TWS is a detailed instrument and I feel needs more than two days of physical in‐person instruction in Bowling Green. Leading up to the final TWS, each method course requires a "TWS". The one completed in these classes pales in comparison to the requirements necessary during student teaching. A more formal approach should be taken in these methods courses. A class was taken in teaching students in diverse situations. The class could have been more beneficial had it represented a true city classroom. Middle Grades Education 2012‐13 Page 8 of 10 Summary of Results by Kentucky Teacher Standards and Other Key Conceptual Framework Values
Below is a summary based on student teacher performance which is considered an essential
part of our evaluation. It is at this point we can tell if all the pedagogy knowledge is actually
transferable to the classroom experience. Since our students have little opportunity to teach in a
regular classroom prior to student teaching, this is a critical area to receive feedback from
university supervisors, cooperating teachers, and instructors on their performance.
Standard 1 (Content Knowledge)
A large number of Middle Grade Education (MGE) candidates scored proficient or higher (97%)
on this content knowledge standard (Table10).
Standard 2 (Designs/Plans Instruction)
A large number of Middle Grade Education (MGE) candidates are scoring proficient or higher
(94%) on their TWS for this standard (Table 9).
Standard 3 (Maintains Learning Climate)
Middle Grade Education (MGE) candidates are scoring proficient or higher (94%) on their
student teacher evaluation for this standard. This is slightly below the unit‐wide level of 96%
(Table 10).
Standard 4 (Implements/Manages Instruction)
Middle Grade Education (MGE) candidates had a higher average on CPs for this standard with
99% passing their CP compared to 98% in the unit (Table 3). In addition based on data from the
student teacher evaluations, 92% met the target for KTS 4 compared to 91% for the unit (Table
10).
Standard 5 (Assessment/Evaluation)
Ninety-three percent of Middle Grade Education (MGE) candidates are scoring proficient or
higher on their student teaching evaluations for KTS 5. This is well above the unit‐wide level of
88% (Table 10). While this seems to be a difficult area for many teacher candidates, as a whole,
MGE candidates are able to design effective assessments.
Standard 6 (Technology)
Interestingly, 93% of the MGE candidates mastered KTS 6 on their student teacher evaluation
compared to 90% in the unit (Table 10). This percentage is rising from previous years as faculty
are modeling technology use and requiring candidates to incorporate technology in their lessons.
Standard 7 (Reflection)
A large majority (97%) of MGE candidates passed KTS 7 CPs (Table 3). This does seem to be a
more challenging area on the TWS as it was a low area with 74% of MGE candidates passing the
Reflection section compared to 91% in the unit (Table 9).
Standard 8 (Collaboration)
Middle Grades Education 2012‐13 Page 9 of 10 MGE candidates demonstrated effectiveness on KTS 8 as shown in their student teacher
evaluations with 95% passing this standard above the 93% passage for the unit (Table 10).
Standard 9 (Professional Development)
On the student teacher evaluations, 99% of MGE candidates mastered KTS 9 above the 97% for
the unit (Table 10). However, on the TWS this standard was a low point. Only 77% of the MGE
candidates passed this standard on the TWS compared to the unit passage rate of 88% (Table 9).
Perhaps this standard can be more authentically measured in a clinical setting as assessed in the
student teacher evaluations.
Standard 10 (Leadership)
Student teacher evaluations indicate that 91% of MGE candidates mastered this standard
compared to 89% in the unit (Table 10). Despite negative comments in a student survey, MGE
candidates seem to be demonstrating leadership in the clinical setting (Table 13).
3. Efforts to Report and Disseminate Results
This report was circulated among all MGE faculty for review and comment. In addition the
report is posted electronically for faculty to have available for review as needed when making
decisions regarding program analysis and recommendations for approval.
4. Key Discussions and/or Decisions Made Based on Assessment Results
a. Assessment or Data Collection Changes Based on Assessment Results
As MGE and Secondary move to more clinical based field assignments, program areas are
examining critical performances and other course requirements to carefully align to teaching
standards and increase the level of rigor in our program. Faculty who teach the MGE courses
need to continue to work with our Field Placement Coordinator and examine ways to improve
the field experiences so students will be allowed to have more teaching experiences to improve
competencies and confidence prior to student teaching.
b. Program Curriculum or Experiences Changes Based on Assessment Results
As the last two sections of the TWS were challenging areas for students, the program area needs
to design similar experiences that replicate these tasks earlier in the program. MGE is continuing
to require candidates that receive a level 1 or 2 to revise their CP before the end of the course.
This ensures candidates gain mastery of the key concepts taught in each course.
c. Decisions about Group/Individual Student Progress Based on Assessment Results
There is little significant change in our scores from the previous year. Data shows that
leadership and assessment are areas that are improving. Instructors have reviewed Critical
Performances and course content to strengthen content and performances related to classroom
management, technology, assessment, and leadership standards. In addition, we need to review
quality of field experiences our students are receiving.
5. Discuss trends in assessment results over the last few years.
The data from these assessments indicate that our candidates in Middle Grades Education
continue to be successful. Their performance on course critical performances remains
stable overall. However, they did improve in their critical performances accomplished during
Middle Grades Education 2012‐13 Page 10 of 10 their methods courses. The CP for that course is the Teacher Work Sample. One possible reason
for the improvement may be that the instructors teaching those courses last year are more
familiar with the TWS than in the previous year. Since the TWS is now again under review, we
need to monitor that we don’t confuse our students by instructors trying to learn the new version
along with the students. It appears that with each change, there is a slight bump down for a
semester or two. In analysis of the Critical Performances as aligned with the KTS, there has been
an improvement across all but one area (Professional Development) which remained the same.
There were several areas in the TWS that were low areas compared to the previous year
including Contextual Factors, Analysis of Student Learning, and Reflection of the Learning.
Adjunct faculty were instructors of this score which potentially caused differences in scoring. It
is important that all faculty that assess the TWS are reliable scorers and are fully trained on the
instrument.
With the increased required number of field hours required by EPSB in spring 2014, we are
planning field experiences that are meaningful and hopefully enhance student learning.
Designated types of field experiences are identified for each course to ensure that students meet
all the field experience categories.