EXED IP Undergraduate APR 2008-09 Page 1 of 9 Exceptional Education – Initial Preparation (Undergraduate) Annual Program Report Academic Year 2008-09 February 2, 2011 1. Continuous Assessment Results a. Admission Data Table 1 provides the average admission test scores and admission grade point average (GPA) of Exceptional Education candidates approved by the Professional Education Council (PEC) for admission into initial teacher preparation programs during this academic year. Before the Office of Teacher Services submits their names for review and approval by the PEC, candidates must meet minimum requirements established by the state and/or the WKU Professional Education Unit. Table 1. Approved Candidate Test Score Averages ACT Program EXED N 18 Mean 22 PPST Math N Mean 3 177 PPST Reading N Mean 3 177 PPST Writing N Mean 3 173 SAT N Mean GRE Composite N Mean Admission GPA N Mean 85 3.21 b. Course Based Assessment Data Table 2 provides the percentage of Exceptional Education candidates (N = 66) scoring at each level of proficiency on critical performances within education courses for this academic year. Proficiency levels are based on a scale of 1 – Standard Not Met, 2 – Standard Partially Met, 3 – At Standard, and 4 – Above Standard. Table 2. CP Proficiency Level Percentages Course EDU-250 EDU-489 ELED-345 ELED-355 ELED-365 ELED-407 EXED-330 EXED-331 EXED-332 EXED-333 EXED-334 EXED-415 1 2% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 2 13% 0% 20% 0% 0% 0% 0% 7% 9% 0% 0% 0% 3 50% 100% 80% 50% 50% 83% 63% 14% 27% 69% 76% 100% 4 35% 0% 0% 50% 50% 17% 38% 79% 64% 31% 24% 0% EXED IP Undergraduate APR 2008-09 Page 2 of 9 Course EXED-417 EXED-418 EXED-422 EXED-430 EXED-431 EXED-432 EXED-434 LME-318 LME-407 LME-445 LME-448 LTCY-320 LTCY-420 MGE-275 PSY-310 SEC-351 SEC-352 Grand Total 1 2 0% 0% 0% 0% 0% 7% 0% 0% 0% 0% 0% 0% 0% 0% 3% 0% 0% 1% 3 0% 0% 0% 0% 0% 14% 0% 8% 33% 0% 15% 14% 0% 20% 11% 0% 0% 7% 8% 33% 45% 0% 21% 14% 73% 92% 67% 100% 46% 57% 50% 20% 23% 0% 0% 45% 4 92% 67% 55% 100% 79% 64% 27% 0% 0% 0% 38% 29% 50% 60% 63% 100% 100% 47% Table 3 indicates the level of Exceptional Education candidate (N = 62) proficiency across critical performances related to the Kentucky Teacher Standards (KTS). Candidates receiving an overall rating of 3 or 4 on a CP are considered to have demonstrated proficiency on the standards associated with the CP. Compared to the unit-wide results, Exceptional Education candidates are typically performing at average. Table 3. Percent of Exceptional Education Candidates Scoring Proficient on CPs by KTS Program EXED Unit-Wide 1 94% 94% 2 89% 95% Kentucky Teacher Standards 4 5 6 7 98% 90% 85% 93% 94% 95% 93% 96% 3 91% 95% 8 100% 96% 9 97% 95% 10 100% 97% *KTS Key: 1 – Content Knowledge, 2 – Designs/Plans Instruction, 3 – Maintains Learning Climate, 4 – Implements/ Manages Instruction, 5 – Assessment/Evaluation, 6 – Technology, 7 – Reflection, 8 – Collaboration, 9 – Professional Development, 10 – Leadership Table 4 indicates the number of Exceptional Education candidates (N = 16) who have scored 2 or lower (below proficiency) on critical performances during this academic year. Table 4. Exceptional Education Candidates Scoring Below Proficient on CPs Student ID 6271 3375 Score 1 Student Count 2 1 1 2 1 3 EXED IP Undergraduate APR 2008-09 Page 3 of 9 9042 7717 1091 1515 0532 3984 5406 5311 7469 3220 4138 1717 9597 5434 Grand Total 4 1 1 3 1 1 2 2 1 1 1 1 2 1 2 1 23 4 1 1 1 2 2 1 1 1 1 2 1 3 1 26 c. Clinical Experiences Data The Exceptional Education program uses the following courses and experiences to evaluate candidate dispositions: EDU 250, EXED 334, EXED 416, and EXED 490. The program has identified the following courses and experiences where candidates report the diversity of their field experiences: EXED 334 and EXED 416. EXED 334 has been designated as the experience where candidates must work in settings at or above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 5 reports how Exceptional Education candidates performed on dispositions as they entered and progressed through their program (N = 16) and during their student teaching experience (N = 12). Students are considered “proficient” who average at 3 or higher on each disposition category. Table 5. Exceptional Education Proficiency Rates on Unit-Wide Dispositions Period Prior to Student Teaching During Student Teaching Values Learning 100% 92% WKU Professional Education Dispositions Values Personal Values Values Values Integrity Diversity Collaboration Professionalism 100% ---92% 100% 100% 100% Over this academic year, Exceptional Education candidates (N = 7) reported demographic information on 14 field placements with an average of 11% diversity (based on National Center for Education Statistics). This diversity percentage meets the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 6 reveals the percentages of field experiences with various characteristics. Note that candidates could choose all the characteristics that applied for any given experience. EXED IP Undergraduate APR 2008-09 Page 4 of 9 Table 6. Percentages of Field Experience by Category Types Physical Disability 36% Learning Disability 93% African American 79% Working With Students With Special Needs Mod/Sev Visual EBD Gifted ELL Disability Impair 43% 43% 21% 43% 0% Hearing Impair 29% Working with Diverse Students Native American Latino/Hispanic 0% 64% Develop Delay 21% Autism 36% Asian American 57% Overall, in 93% of their field experiences Exceptional Education candidates reported working with at least one student with special needs and in 79% of their field experiences candidates reported working with at least one student from a diverse ethnic group. d. Culminating Assessment Data As Component 4 of the WKU Professional Education Unit Continuous Assessment Plan (CAP) strategy, all initial preparation candidates complete a culminating assessment of professional and pedagogical knowledge and skills, the Teacher Work Sample (TWS). This assessment is also used to demonstrate candidates’ ability to impact P-12 student learning. In particular, candidate performances on Assessment Planning and Analysis of Student Learning have been identified as key indicators of candidates’ ability related to student learning. Although in spring 2008 the Professional Education Council agreed that candidates who score a holistic score of at least “2 – Developing” are able to exit the program, for program evaluation purposes our goal is that at least 80% of program candidates will achieve “3 – Proficient” or higher. Table 7 presents the proficiency rate for Exceptional Education candidates (N = 11). Table 7. Initial Preparation TWS Proficiency Rates Program Exceptional Education Unit-Wide % Proficient 100% 89% Because the faculty also scores TWS at the indicator level, we are able to use these scores to ascertain candidate success in meeting each component of the TWS. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 8 depicts the percentage of Exceptional Education candidates who averaged at least 2.5 on the indicators for each TWS Factor: CF – Contextual Factors, LG – Learning Goals, AP – Assessment Plan, DFI – Design for Instruction, IDM – Instructional Decision Making, ASL – Analysis of Student Learning, and RSE – Reflection and Self-Evaluation. EXED IP Undergraduate APR 2008-09 Page 5 of 9 Table 8. Initial Preparation TWS Proficiency Rates of Exceptional Education Candidates Program EXED Unit-Wide CF 100% 86% LG 100% 98% Teacher Work Sample Components AP DFI IDM ASL 100% 100% 92% 100% 78% 95% 82% 78% RSE 92% 75% Because the TWS indicators have been aligned to Kentucky Teacher Standards, we can use these scores to ascertain candidate success in meeting each standard related to the TWS. Table 9 reports these scores as they relate to Kentucky Teacher Standards. Table 9. Percentage of Exceptional Education Candidates who “Passed” each Teacher Standard Program EXED Unit-Wide Kentucky Teacher Standards (Measured by TWS) 2 4 5 6 7 100% 92% 100% 83% 92% 93% 85% 77% 87% 76% 1 100% 94% 9 83% 76% Additionally, all candidates are assessed during their student teaching experience using the Student Teaching Evaluation form. Table 10 reports the percentages of Exceptional Education student teachers (N = 12) successful on each standard. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 10. Exceptional Education Proficiency Rates by Kentucky Teacher Standards Program EXED Unit-Wide 1 100% 96% 2 92% 93% 3 100% 95% Kentucky Teacher Standards 4 5 6 7 92% 92% 100% 100% 88% 88% 85% 88% 8 100% 92% 9 92% 93% 10 100% 87% e. Exit and Follow Up Data Table 11 delineates the Educational Testing Services reports of the pass rates on the Praxis II content exams of candidates who completed the program in the 2007-08 academic year (the most recent year with complete data). The last column allows for pass rate comparison of our candidates to our 2006-07 results. Table 11. Pass Rates on Content Tests for Initial Teacher Preparation Program/Type of Assessment SE STUDENTS W/MENTAL RETARDATION EDUC. EXCEPTIONAL STUDENTS: CK ED EXCEPT STUDENTS: MILD/MOD. DISAB. ED EXCEPT STUDENTS: PROFND. DISAB. Candidate N (2007-08) 8 32 18 5 WKU Pass Rate (2007-08) 88% 100% 94% 100% WKU Pass Rate (2006-07) 100% 100% 100% -- EXED IP Undergraduate APR 2008-09 Page 6 of 9 Annually, the WKU Teacher Survey is sent to student teachers and alumni who potentially have been teaching one or more years. For the 2008-09 academic year, out of a possible 410 student teachers 354 (86%) completed the survey; out of a possible 480 alumni, 106 (22%) completed the survey. Below are the results for Exceptional Education student teachers and alumni, 21 of whom responded. Survey items requested the respondent’s perception of WKU preparation on each of the Kentucky Teacher Standards using a scale of 1 “Poor,” 2 “Fair,” 3 “Good,” and 4 “Excellent.” Standards with average scores of 3 or better across items were considered to demonstrate acceptable program quality. Table 12 reports Exceptional Education survey results. Table 12. Average Scores on Teacher Standards Questions for EXED Respondents Program EXED Unit-Wide 1 3.3 3.3 2 3.1 3.3 3 3.5 3.5 Kentucky Teacher Standards 4 5 6 7 3.2 3.1 3.3 3.2 3.2 3.1 3.3 3.2 8 3.2 3.0 9 3.1 3.3 10 3.1 3.0 Respondents were also able to provide comments if they answered “poor” for any item. Table 13 presents Exceptional Education respondent comments by years of experience (0 = Student Teaching). Table 13. Exceptional Education Respondent Comments tch exp 0 0 3 Comments I feel that it is a difficult task to prepare teachers, and the best way to prepare them is by letting them experience teaching on their own. I do feel that we could have been better prepared in a lot of areas. I feel that there needs to be less paper work and more hands on experience! In regards to behavior management I feel that I was not prepared to teach, based on what I was taught during my two behavior classes. Need more instruction on planning for a diverse group with multiple levels of skill while keeping the standards in mind. 2. Summary of Results by Kentucky Teacher Standards and Other Key Conceptual Framework Values Candidates in the 2008-2009 graduating class in Exceptional Education LBD/MSD performed well on the KY Teaching Standards and were consistently proficient in Kentucky Teacher Standards and Critical Performance Indicator Scores. According to the assessment data collected unit wide on performance on KY Teacher Standards, 2008-2009 Exceptional Education teacher candidates performed above the unit wide average on 4 of the 10 standards. On 5 of the 10 standards, the percentage of candidates scoring proficient on Critical Performances by KY Teacher Standards was lower than the unit wide percentage. On 1 of the 10 standards, the percentage of Exceptional Education candidates was equal to the unit wide percentage. Areas in which the Exceptional Education candidates performed below the unit wide averages were in Designing and Planning Instruction; Maintaining Learning Climate; Assessment/Evaluation; Technology; and Reflection. It should be noted that there are many more students in other areas of Teacher Education than in Exceptional Education. This means that only a very few students performing poorly puts our EXED IP Undergraduate APR 2008-09 Page 7 of 9 percentages at a lower level below the unit wide averages easily in most cases. There are some conflicting conclusions to be made using only the data collected unit wide. For example, Exceptional Education candidates were overwhelmingly performing above the unit wide percentages if looking at data from Table 8 (TWS Proficiency Rates); Table 9 (Percentage of Candidates who “passed” each teacher standard) and Table 10 (Proficiency rates by Ky Teacher Standards). However, if looking at the data from Table 3 (Percentage of candidates scoring proficient on Critical Performances by Kentucky Teacher Standards), our students scored below the unit wide percentages in 5 out of 9 standards. Contrary to Table 3 data, on the same 5 standards in Tables 9 and 10 our candidates performed above the unit wide percentages on those same standards. On all of the TWS indicators shown on Table 8, our candidates were above the unit wide rates in 7 out of 7 indicators. Several explanations for this discrepancy are plausible. First, the variation of faculty and staff scoring the different performances may indicate a problem with reliability of scoring the standards. Second, the critical performances may be flawed in that they are not truly assessing the standards intended. A third possible explanation may be that students improved their performance with time and became proficient throughout the course of their program on the standards that were, previously, below unit wide percentages. That being said, there was only one standard that was consistently below the unit wide percentages: Standard 2: Designs and Plans Instruction, (Table 3 and Table 10). It is believed that the lower averages in designing and planning instruction may be explained by the fact that personnel who do not have extensive experience in special education classrooms often supervise student teachers in the special education program. Because of this fact, the assessment of student skills in designing and planning instruction may not be based upon what is appropriate and best practice in a classroom serving students with moderate to severe disabilities. The Exceptional Education program at WKU is strong in providing students with opportunities to work with students in diverse environments and requires field experience hours that is far above other programs in the state and region. Before our candidates begin student teaching, they have logged well over 300 hours in the field in a variety of settings including those serving children with Learning and Behavior Disorders, Moderate and Severe Disabilities, Collaborative Inclusive Classrooms, Clinical Settings, Early Childhood Settings, and Alternative Educational Settings. As evidenced by the data in Table 6, our students are provided opportunities to observe and/or work directly with children with a variety of disabilities and ethnicities. It is believed that this is a clear strength of our program. Another strength of our program is the dispositions displayed by our candidates by the time they reach student teaching. The Exceptional Education faculty believes that professionalism and ethical practice are a priority and these dispositions are emphasized and taught in all classes, in particular in field experience courses. Our disposition ratings (Table 5) are a reflection of this emphasis. EXED IP Undergraduate APR 2008-09 Page 8 of 9 If evaluating our program through the Praxis II scores of teacher candidates (Table 11), our 2007-2008 scores are varied according to which of the three test’s data is examined. For the Core Knowledge for Exceptional Education (0353), 100% of our 07-08 graduates passed on the first attempt. For the Mild/Moderate Disabilities test (0542), 94% of graduates passed on the first attempt. For the Severe/Profound test (0544), 100% of graduates passed on the first attempt. 3. Efforts to Report and Disseminate Results An Exceptional Education coordinator was appointed and program faculty members were involved in reviewing the data. Each Exceptional Education faculty member examined the data and made suggestions as to how to improve the program. Informal discussions were held among Exceptional Education members. Final results will be shared with the School of Education and the college. The School of Teacher Education faculty members meet on a regular basis each semester of the academic year to review and analyze program area assessment results. Generally, the faculty is responsible for examining data from within their respective classes, especially critical performances, to complete a Unit Action Plan. The Unit Action Plan relates to NCATE documents. All reports are usually reviewed by the elementary faculty before submitting to the department head and other responsible parties within the college and university. Each year, program assessment data is reviewed and evaluated for information which warrants program changes and course modification. 4. Key Discussions and/or Decisions Made Based on Assessment Results a. Assessment or Data Collection Changes Based on Assessment Results It is believed that some of the areas that appear as weaknesses according to the Unit Wide data may not be reflective of other indicators of program evaluation such as student reports, intern observations etc. Changes that should be made include making sure all data is reported accurately by the program and that Disposition ratings be collected more frequently rather than just at student teaching. Other areas in the unit collect disposition ratings at certain points prior to student teaching and the Exceptional Education program should begin this as well. In addition, the program will continue to collect data on pass rates of individual tests to determine the need for curriculum changes. b. Program Curriculum or Experiences Changes Based on Assessment Results Due to assessment results along with changes required of special education teachers in the field, it will be proposed that the faculty begin to research revising the program to better serve our students, their future students, and the districts that hire them. EXED IP Undergraduate APR 2008-09 Page 9 of 9 c. Decisions about Group/Individual Student Progress Based on Assessment Results Students not meeting proficiencies based upon Critical Performances in individual courses need to be targeted at an earlier point in their program. Students not meeting proficiencies prior to student teaching may be provided with remediation and personal mentoring to bring them up to the proficient levels required.