Middle Grades Education (131203) 2011‐12 Page 1 of 11 Middle Grades Education (131203) – Initial Preparation Annual Program Report Academic Year 2011‐12 Dr. Judy Davison November 27, 2011 1. Continuous Assessment Results a. Admission Data Table 1 provides the average admission test scores and admission grade point average (GPA) of Middle Grades Education candidates approved by the Professional Education Council (PEC) for admission into initial teacher preparation programs during this academic year. Before the Office of Teacher Services submits their names for review and approval by the PEC, candidates must meet minimum requirements established by the state and/or the WKU Professional Education Unit. Table 1. Approved Candidate Test Score Averages Program Middle Grades Education ACT N Mean 44 23 PPST Math N Mean 2 177 PPST
Reading N Mean
PPST
Writing N Mean
N Mean
2 2 1 177 177 GRE Composite N Mean 2 1175 SAT 1370 Admission GPA N Mean
51 3.26 b. Course Based Assessment Data Table 2 provides the percentage of Middle Grades Education candidates (N =208 scoring at each level of proficiency on critical performances within education courses for this academic year. Proficiency levels are based on a scale of 1 – Standard Not Met, 2 – Standard Partially Met, 3 – At Standard, and 4 – Above Standard. Table 2. CP Proficiency Level Percentages Course 1 2 3 4 MGE275 0% .81% .33.47% 65.73% MGE385 0% 0% 32.93% 67.07% MGE475 0% 0% 11.11% 88.89% MGE477 0% 0% 100% 0% MGE479 0% 0% 100% 0% MGE481 0% 0% 72.73% 27.27% Grand Total 0% 3% 61% 36% Table 3 indicates the level of Middle Grades Education candidates (N = 208) proficiency across critical performances related to the Kentucky Teacher Standards (KTS). Candidates receiving an overall rating of 3 or 4 on a CP are considered to have demonstrated proficiency on the Middle Grades Education (131203) 2011‐12 Page 2 of 11 standards associated with the CP. Compared to the unit‐wide results, Middle Grades Education candidates are typically performing near average. Table 3. Percent of Middle Grades Education Candidates Scoring Proficient on CPs by KTS Program Middle Grades Education Unit‐Wide 1 2 3 95% 97% 96% 97% 96% 97% Kentucky Teacher Standards 4 5 6 7 97% 98% 96% 96% 96% 96% 96% 97% 8 9 97% 97% 90% 95% 10 99% 98% *KTS Key: 1 – Content Knowledge, 2 – Designs/Plans Instruction, 3 – Maintains Learning Climate, 4 – Implements/ Manages Instruction, 5 – Assessment/Evaluation, 6 – Technology, 7 – Reflection, 8 – Collaboration, 9 – Professional Development, 10 – Leadership Table 4 indicates the number of Middle Grades Education candidates (N = 23) who have scored 2 or lower (below proficiency) on critical performances during this academic year. Table 4. Middle Grades Education Candidates Scoring Below Proficient on CPs Student ID 800510949 800027316 800526321 800108440 800537885 800273232 800539859 800376804 800539996 800438119 800575706 800500032 800580707 800259921 800595308 800433706 800688586 800103416 800689162 800439692 800713357 800363843 800713969 Grand Total Score 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 23 Student Count 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 23 Middle Grades Education (131203) 2011‐12 Page 3 of 11 c. Clinical Experiences Data Middle Grades Education uses the following courses and experiences to evaluate candidate dispositions: The program has identified the following courses and experiences where candidates report the diversity of their field experiences: The program has identified the following courses and experiences where candidates report the diversity of their field experiences: MGE 275, MGE 385, MGE 475, MGE 477, MGE 479, MGE 481. Fieldwork in the listed courses has been designated as the experience where candidates must work in settings at or above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 5 reports how Middle Grades Education candidates performed on dispositions as they entered and progressed through their program (N =176 and during their student teaching experience (N = 56). Students are considered “proficient” who average a 3 or higher on each disposition category. Table 5. Middle Grades Education Proficiency Rates on Unit‐Wide Dispositions Period a. Prior to Student Teaching b. During Student Teaching Values Learning WKU Professional Education Dispositions Values Personal Values Values Values Integrity Diversity Collaboration Professionalism 98.14% 99.69% 100% 99.13% 97.8% 98.21% 100% 100% 100% 100% Over this academic year, Middle Grades Education candidates (N = 160) reported demographic information on 222 field placements with an average of 16% ethnically diverse students, 53% students on free/reduced lunch, and 12% student with disabilities (based on National Center for Education Statistics and Kentucky Department of Education). This ethnic diversity percentage continues to be above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 6 reveals the percentages of field experiences with various characteristics. Note that candidates could choose all the characteristics that applied for any given experience. Table 6. Percentages of Field Experience by Category Types Working with Student With Special Needs % Candidates working with Students with Physical Impairments % Candidates working with Students with Learning Disabilities % Candidates working with Students with Moderate/Severe Disabilities % Candidates working with Students with Emotional/Behavioral Disorders % Candidates working with Gifted Students 13%
65%
6%
33%
60%
Middle Grades Education (131203) 2011‐12 Page 4 of 11 % Candidates working with English Language Learners % Candidates working with Students with Visual Impairments % Candidates working with Students with Hearing Impairments % Candidates working with Students with Speech/Language Delays % Candidates working with Students with Development Delays % Candidates working with Students with Autism Spectrum Disorder % Candidates working with Students with Other Impairments Working with Diverse Students % Candidates working with African American Students % Candidates working with Native American/American Indian Students % Candidates working with Latino/Hispanic Students % Candidates working with Asian Students % Candidates working with Students with Special Needs (Aggregate) % Candidates working with Diverse Students (Aggregate) 32%
8%
9%
8%
6%
9%
2%
92%
17%
79%
47%
91%
96%
Overall, as can be seen in Table 6, in 100% of their field experiences Middle Grades Education candidates reported working with at least one student with special needs and in 100% of their field experiences candidates reported working with at least one student from a diverse ethnic group. d. Culminating Assessment Data As Component 4 of the WKU Professional Education Unit Continuous Assessment Plan (CAP) strategy, all initial preparation candidates complete a culminating assessment of professional and pedagogical knowledge and skills, the Teacher Work Sample (TWS). This assessment is also used to demonstrate candidates’ ability to impact P‐12 student learning. In particular, candidate performances on Assessment Planning and Analysis of Student Learning have been identified as key indicators of candidates’ ability related to student learning. Although in spring 2008 the Professional Education Council agreed that candidates who score a holistic score of at least “2 – Developing” are able to exit the program, for program evaluation purposes our goal is that at least 80% of program candidates will achieve “3 – Proficient” or higher. Table 7 presents the proficiency rate for Middle Grades Education candidates (N = 32). Table 7. Initial Preparation TWS Proficiency Rates Program Middle Grades Education Unit‐Wide % Proficient 83% 96% Because the faculty also scores TWS at the indicator level, we are able to use these scores to ascertain candidate success in meeting each component of the TWS. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 8 Middle Grades Education (131203) 2011‐12 Page 5 of 11 depicts the percentage of Middle Grades Education candidates who averaged at least 2.5 on the indicators for each TWS Factor: CF – Contextual Factors, LG – Learning Goals, DFI – Design for Instruction, ASL – Analysis of Student Learning, and ROT – Reflection on Teaching. Table 8. Initial Preparation TWS Proficiency Rates of Middle Grades Education Candidates Program Middle Grades Education Unit‐Wide CF LG DFI ASL ROT 94% 94% 92% 91% 73% 89% 83% 92% 91% 88% Because the TWS indicators have been aligned to Kentucky Teacher Standards, we can use these scores to ascertain candidate success in meeting each standard related to the TWS.
Table 9 reports these scores as they relate to Kentucky Teacher Standards. Table 9. Percentage of Middle Grades Education Candidates who “Passed” each Teacher Standard Program Middle Grades Education Unit‐Wide 1 2 3 5 6 7 9 73% 83% 91% 91% 91% 92% 85% 88% 67% 83% 67% 76% 91% 88% Additionally, all candidates are assessed during their student teaching experience using the Student Teaching Evaluation form. Table 10 reports the percentages of Middle Grades Education student teachers (N = 32) successful on each standard. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 10. Middle Grades Education Proficiency Rates by Kentucky Teacher Standards Program Kentucky Teacher Standards 4 5 6 7 1 2 3 Middle Grades Education 96% 95% 98% 95% 93% 91% Unit‐Wide 94% 89% 94% 89% 88% 82% 95% 87% 8 9 10 93% 97% 95% 90% 88% 90% e. Exit and Follow Up Data Table 11 delineates the Educational Testing Services reports of the pass rates on the Praxis II content exams of candidates who completed the program in the 2010‐11 academic year (the most recent year with complete data). The last column allows for pass rate comparison of our candidates to our 2008‐09 results. Middle Grades Education (131203) 2011‐12 Page 6 of 11 Table 11. Pass Rates on Content Tests for Initial Teacher Preparation Program/Type of Assessment Middle Grades Education Praxis II Test (1) Middle Grades Education Praxis II Test (2) Candidate N (2010‐11) 118 WKU Pass Rate (2010‐11) 94% WKU Pass Rate (2009‐10) 98% Annually, the WKU Teacher Survey is sent to student teachers and alumni who potentially have been teaching one or more years. For the 2011‐12 academic year, 488 student teachers, 44 completed the survey. Below are the results for Middle Grades Education student teachers, 44 of whom responded. Survey items requested the respondent’s perception of WKU preparation on each of the Kentucky Teacher Standards using a scale of 1 “Poor,” 2 “Fair,” 3 “Good,” and 4 “Excellent.” Standards with average scores of 3 or better across items were considered to demonstrate acceptable program quality. Table 12 reports Middle Grades Education survey results. Table 12. Average Scores on Teacher Standards Questions for Middle Grades Education Respondents Program Kentucky Teacher Standards 4 5 6 7 1 2 3 Middle Grades Education 3.26 3.1 3.34 3.15 2.95 3.12 Unit‐Wide 3.44 3.42 3.61 3.37 3.25 3.40 8 9 10 2.95 2.8 3.04 2.76 3.32 3.06 3.31 3.09 Respondents were also able to provide comments. Table 13 presents Middle Grades Education respondent comments. Table 13. Middle Grades Education Respondent Comments WKU needs to have more instruction on classroom management and using technology in the classroom. WKU could have actually walked us through a professional growth plan and had us do one in our classes to practice. I don't feel like I ever learn how to actually adapt assessments or assignments based on diverse student needs. I just feel like we lightly skimmed this subject in classes and never really learned practical ways to modify things for students or when to modify. I only feel confident in preparing a KTIP lesson plan. I did not feel confident in other areas that were listed. Standards 3, 4, 5, 6, 7, 8, 9, 10 were never practiced outside the classroom during my experience at WKU. Prior to student teaching I had only taught one lesson, and therefore addressing many of these standards was very new to me! I really think students need to be in the classroom and get to know students to be able to plan these lessons more! The use of technology in instruction is not something that is taught "hands on". Analyzing data/using data to reflect on instruction, evaluate outcomes, and results is another area that is learned "after" you are in the classroom. If students could experience these in a classroom setting prior to student teaching, (perhaps during their observations), I think that they would Middle Grades Education (131203) 2011‐12 Page 7 of 11 have a better understanding of these concepts. Middle Grade educators seem to lack instruction pertaining to Teacher Work Samples, using effective strategies, and knowing what is available to actually use in the classroom. For many, these skills are learned during student teaching. However, students are so immersed in their Teacher Work Samples during the student‐teaching period, they may not be reaping the full benefits student‐teaching has to offer. There are too many areas that instruction would be beneficial to the education student. The methods courses need to have more instruction about developing assessment modes and pre and post assessments. The instructors need to be at the remote campuses to provide one on one support. There are a lot of holes that the education program leaves for students to fall into. I feel that collaboration was not a large focus during my time at WKU. I dont' know that a whole course should be about collaboration alone, but maybe one devolted to collaboration, leadership, and professional growth would be beneficial to future students. If I answered poor on any choice it was because I felt no support from WKU personnel in this area. A good example of this would be in the leadership assignment during student teaching. In my experience I had no guidance nor explanation of what was expected of me and my project. This was very difficult when I did not know what was expected. However, the same person guided me through my growth needs and I excelled in technology because of her. Most of the excellence scores came from the leadership & support I received this semester from my student teaching seminar instructor. She set the precedent of what a great teacher should expect. At first I thought she was setting expectations unattainably high. However, after completing the assignments I now see that she expected the best and by doing so she brought that out in my teaching. I understood more, reflected more, and became a better teacher because of her during this spring. If there is a way to include more classroom management tools or classes, that would be helpful. In addition, if there is a way to practice instruction in a classroom earlier in the program, that would be helpful. Additional and earlier exposure to the teacher work sample would be helpful. Most things that happen in a "normal" school day are never addressed in the training. Talking to parents, assessing learning, developing professional growth plans, etc had never crossed my mind until I had to do it student teaching. At least an overview or some tips would have been helpful. The only real parts of this survey that I judged as "Poor" had to do with data analysis and providing leadership. We did not really learn either of these things until student teaching. I see now how important both aspects would be to becoming a great teacher, and wish we had learned more about them during the semesters prior to student teaching. Data analysis is extremely important to a teacher and we had no real opportunity to explore this during EDU courses. We also did not learn about many leadership opportunities within the school, such as SBDM councils, and intern projects. Knowledge of these would help us not only in student teaching, but also during our first year of teaching when we would need to complete actual leadership projects. Middle Grades Education (131203) 2011‐12 Page 8 of 11 2. Summary of Results by Kentucky Teacher Standards and Other Key Conceptual Framework Values Below is a summary based on student teacher performance which is considered an essential part of our evaluation. It is at this point we can tell if all the pedagogy knowledge is actually transferable to the classroom experience. Since our students have little opportunity to teach in a regular classroom prior to student teaching, this is a critical area to receive feedback from university supervisors, cooperating teachers, and instructors on their performance. Standard 1 (Content Knowledge) 96 % of Middle Grade Education (MGE) program students are scoring proficient or higher on this content knowledge standard. This is above the unit‐wide level of 94%. (Table10) Standard 2 (Designs/Plans Instruction 95 % of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluations. This is significantly above the unit‐wide level of 89%. (Table 10) Standard 3 (Maintains Learning Climate) 98% of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluation. This is above the unit‐wide level of 94%. (Table 10) Standard 4 (Implements/Manages Instruction 95 % of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluations. This is above the unit‐wide level of 89%. (Table 10) Standard 5 (Assessment/Evaluation) 93 % of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluations. This is well above the unit‐wide level of 88%. (Table 10) Standard 6 (Technology) 91 % of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluations. This is above the unit‐wide level of 82%. (Table 10) Even though the students have commented that they don’t get as much hands on experience with new technologies in schools prior to student teaching, with the guidance received during student teaching, they are gaining proficiency. Standard 7 (Reflection) 95 % of Middle Grade Education (MGE) program students are scoring proficient or higher on critical performances. This is above the unit‐wide level of 87%. (Table 10) Reflection is emphasized by university instructors and supervisors during student teaching semester, especially as they do their work for their seminar course. Apparently this shows in this Middle Grades Education (131203) 2011‐12 Page 9 of 11 comparison. Some other candidates do not meet as often personally so our belief is that this personal interaction is of great benefit to them. Standard 8 (Collaboration) 93 % of Middle Grade Education (MGE) program students are scoring proficient or higher on their student teaching evaluations. This is above the unit‐wide level of 90%. (Table 10) While they haven’t had as much experience in this area as preferred, they are showing much growth in their last semester. Standard 9 (Professional Development) 97% of Middle Grade Education (MGE) program students are scoring proficient or higher on student teaching evaluations. This is above the unit‐wide level of 88%. (Table 10) It is during their student teaching semester that they truly start to understand the importance of PD. Standard 10 (Leadership) 95% of Middle Grade Education (MGE) program students are scoring proficient or higher on student teaching evaluations. This is above the unit‐wide level of 90%. (Table 10) 3. Efforts to Report and Disseminate Results This report was circulated among all MGE faculty for review and comment. In addition the report is posted electronically for faculty to have available for review as needed when making decisions regarding program analysis and recommendations for approval. 4. Key Discussions and/or Decisions Made Based on Assessment Results a. Assessment or Data Collection Changes Based on Assessment Results It is suggested that faculty be more involved in the development of survey instruments provided for alumni and students so we know the specific questions that are asked. In addition, we request clarity on the process of survey dissemination and if follow up reminders are done to increase the number of alumni responses. Fourteen percent return is too low to make a well informed judgment for program revisions. While is critical to know how our student teachers perceive their preparation, that group may be struggling with their first semester of teaching and less objective in their views. We strongly encourage the survey prompt to invite all comments (positive and negative) rather just appealing for expanding on the lower scores. Overall, the only score below 3.0 was in Standard 10 (Leadership) so despite some negative comments, the majority felt their preparation was “good”. However, we strive to increase numbers answering “excellent.” b. Program Curriculum or Experiences Changes Based on Assessment Results Middle Grades Education (131203) 2011‐12 Page 10 of 11 Faculty who teach the MGE courses need to continue to work with our Field Placement Coordinator and examine ways to improve the field experiences so students will be allowed to have more teaching experiences to improve competencies and confidence before their student teaching semester. The new KTS Leadership standard was the lowest scoring response at 2.76. Collaboration (Standard 8) is also low at 2.8. Faculty need to meet to be more definitive on where these are addressed in the program. A goal to improve that score to above a 3.0 for the next year’s review is reasonable. Since we have lost some MGE faculty through departures from WKU or in support of the SKYTeach Program, we have used adjuncts or one year appointments to teach courses while we appeal for more consistency through hiring long term tenure track positions for our program. In addition, we need more full time support for the extended campus sites and cooperation with public schools in allowing our students to be more engaged with students and the school community to provide hands on experiences for our students. c. Decisions about Group/Individual Student Progress Based on Assessment Results There is little significant change in our scores from the previous year and it is apparent that our students feel we need to strengthen our classroom management, technology, and assessment areas. We need to have instructors who teach the MGE 275, MGE 385 and MGE Methods to review Critical Performances and course content to strengthen content and performances related to classroom management, technology, assessment, and leadership standards. In addition, we need to review quality of field experiences our students are receiving. Since the 2010‐2011 review, new policies are being implemented to address any Critical Performance scores of 2 or less. We will be evaluating the success of those policies later this year. 5. Discuss trends in assessment results over the last few years (Please refer back to your 2009‐10 and 2010‐11 APRs which are posted to the College of Education Professional Education Unit website) The data from these assessments indicate that our candidates in Middle Grades Education continue to be successful. Their performance on course critical performances remains stable overall. However, they did improve substantially in their critical performances accomplished during their methods courses. The CP for that course is the Teacher Work Sample. One possible reason for the improvement may be that the instructors teaching those courses last year are more familiar with the TWS than in the previous year. Since the TWS is now again under review, we need to monitor that we don’t confuse our students by instructors trying to learn the new version along with the students. It appears that with each change, there is a slight bump down for a semester or two. In analysis of the Critical Performances as aligned with the KTS, there has been an improvement across all but one area (Professional Development) which remained the same. Middle Grades Education (131203) 2011‐12 Page 11 of 11 Surprisingly, standards 4 (Management) and 10 (Leadership) increased greatly. With the increased required number of field hours required by EPSB in spring 2014, we will be preparing for making those hours meaningful and hopefully enhance student learning, rather than create any confusion on their part during the adjustment period.