Modern Languages 2010‐11 Page 1 of 7 Modern Languages – Initial Preparation Annual Program Report Academic Year 2010‐11 Person Completing Report April 1, 2012 1. Continuous Assessment Results a. Admission Data Table 1 provides the average admission test scores and admission grade point average (GPA) of Modern Languages candidates approved by the Professional Education Council (PEC) for admission into initial teacher preparation programs during this academic year. Before the Office of Teacher Services submits their names for review and approval by the PEC, candidates must meet minimum requirements established by the state and/or the WKU Professional Education Unit. Table 1. Approved Candidate Test Score Averages Program Spanish ACT N Mean 6 27 PPST Math N Mean PPST
Reading N Mean
PPST
Writing N Mean
SAT N Mean
GRE Composite N Mean 3 1023 Admission GPA N Mean
9
3.62
b. Course Based Assessment Data Table 2 provides the percentage of Modern Languages candidates (N = 18) scoring at each level of proficiency on critical performances within education courses for this academic year. Proficiency levels are based on a scale of 1 – Standard Not Met, 2 – Standard Partially Met, 3 – At Standard, and 4 – Above Standard. Table 2. CP Proficiency Level Percentages Course EDU‐250 EDU‐489 EXED‐330 PSY‐310 SEC‐351 SEC‐352 SEC‐453 Grand Total 2 0% 0% 0% 0% 13% 0% 0% 5% 3 61% 100% 60% 0% 71% 0% 25% 55% 4 39% 0% 40% 100% 16% 100% 75% 40% Total 100% 100% 100% 100% 100% 100% 100% 100% Table 3 indicates the level of Modern Languages candidates (N = 18) proficiency across critical performances related to the Kentucky Teacher Standards (KTS). Candidates receiving an overall rating of 3 or 4 on a CP are considered to have demonstrated proficiency on the standards associated with the CP. Compared to the unit‐wide results, Modern Languages candidates are typically performing slightly below or slightly above average on nearly all items. Modern Languages 2010‐11 Page 2 of 7 Table 3. Percent of Modern Languages Candidates Scoring Proficient on CPs by KTS Program MLE Unit‐Wide 1 93 96% 2 93 95% 3 100 96% Kentucky Teacher Standards 4 5 6 7 88 100 94 95 96% 96% 96% 96% 8 100 96% 9 94 94% 10 100 99% *KTS Key: 1 – Content Knowledge, 2 – Designs/Plans Instruction, 3 – Maintains Learning Climate, 4 – Implements/ Manages Instruction, 5 – Assessment/Evaluation, 6 – Technology, 7 – Reflection, 8 – Collaboration, 9 – Professional Development, 10 – Leadership Table 4 indicates the number of Modern Languages candidates (N = 1) who have scored 2 or lower (below proficiency) on critical performances during this academic year. Table 4. Modern Languages Candidates Scoring Below Proficient on CPs Student ID Score 1 2 3 3 Grand Total Student Count 3 3 c. Clinical Experiences Data Modern Languages uses the following courses and experiences to evaluate candidate dispositions: EDU 250 and EDU 490. The program has identified the following courses and experiences where candidates report the diversity of their field experiences: EDU 250, EDU 490, and SEC 351. SEC 351 has been designated as the experience where candidates must work in settings at or above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 5 reports how Modern Languages candidates performed on dispositions as they entered and progressed through their program (N = 9) and during their student teaching experience (N =3). Students are considered “proficient” who average a 3 or higher on each disposition category. Table 5. Modern Languages Proficiency Rates on Unit‐Wide Dispositions Period a. Prior to Student Teaching b. During Student Teaching Values Learning WKU Professional Education Dispositions Values Personal Values Values Values Integrity Diversity Collaboration Professionalism 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% Over this academic year, Modern Languages candidates (N = 12) reported demographic information on 12 field placements with an average of 16% ethnically diverse students, 50% Modern Languages 2010‐11 Page 3 of 7 students on free/reduced lunch, and 14% student with disabilities (based on National Center for Education Statistics and Kentucky Department of Education). This ethnic diversity percentage continues to be well above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 6 reveals the percentages of field experiences with various characteristics. Note that candidates could choose all the characteristics that applied for any given experience. Table 6. Percentages of Field Experience by Category Types Working with Student With Special Needs % Candidates working with Students with Physical Impairments % Candidates working with Students with Learning Disabilities % Candidates working with Students with Moderate/Severe Disabilities % Candidates working with Students with Emotional/Behavioral Disorders % Candidates working with Gifted Students % Candidates working with English Language Learners % Candidates working with Students with Visual Impairments % Candidates working with Students with Hearing Impairments % Candidates working with Students with Speech/Language Delays % Candidates working with Students with Development Delays % Candidates working with Students with Autism Spectrum Disorder % Candidates working with Students with Other Impairments Working with Diverse Students % Candidates working with African American Students % Candidates working with Native American/American Indian Students % Candidates working with Latino/Hispanic Students % Candidates working with Asian Students % Candidates working with Students with Special Needs (Aggregate) % Candidates working with Diverse Students (Aggregate) 20%
73%
7%
20%
53%
53%
7%
13%
7%
13%
13%
7%
93%
13%
87%
47%
93%
100%
Overall, as can be seen in Table 6, in 100% of their field experiences Modern Languages candidates reported working with at least one student with special needs and in 100% of their field experiences candidates reported working with at least one student from a diverse ethnic group. d. Culminating Assessment Data As Component 4 of the WKU Professional Education Unit Continuous Assessment Plan (CAP) strategy, all initial preparation candidates complete a culminating assessment of professional and pedagogical knowledge and skills, the Teacher Work Sample (TWS). This assessment is also used to demonstrate candidates’ ability to impact P‐12 student learning. In particular, candidate performances on Assessment Planning and Analysis of Student Learning have been identified as key indicators of candidates’ ability related to student learning. Modern Languages 2010‐11 Page 4 of 7 Although in spring 2008 the Professional Education Council agreed that candidates who score a holistic score of at least “2 – Developing” are able to exit the program, for program evaluation purposes our goal is that at least 80% of program candidates will achieve “3 – Proficient” or higher. Table 7 presents the proficiency rate for Modern Languages Education candidates (N = 3). Table 7. Initial Preparation TWS Proficiency Rates Program Modern Languages Education Unit‐Wide % Proficient 100% 94% Because the faculty also scores TWS at the indicator level, we are able to use these scores to ascertain candidate success in meeting each component of the TWS. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 8 depicts the percentage of Modern Languages candidates who averaged at least 2.5 on the indicators for each TWS Factor: CF – Contextual Factors, LG – Learning Goals, DFI – Design for Instruction, ASL – Analysis of Student Learning, and ROT – Reflection on Teaching. Table 8. Initial Preparation TWS Proficiency Rates of Modern Languages Education Candidates Program MLE Unit‐Wide CF 100% 96% LG 100$ 94% DFI 67% 90% ASL 100% 91% ROT 67% 96% Because the TWS indicators have been aligned to Kentucky Teacher Standards, we can use these scores to ascertain candidate success in meeting each standard related to the TWS. Table 9 reports these scores as they relate to Kentucky Teacher Standards. Table 9. Percentage of Modern Languages Education Candidates who “Passed” each Teacher Standard Program MLE Unit‐Wide 1 67% 80% 2 100% 93% Kentucky Teacher Standards (Measured by TWS) 3 5 6 7 8 100% 100% 67% 67% 100% 97% 87% 73% 85% 92% 9 67% 95% Additionally, all candidates are assessed during their student teaching experience using the Student Teaching Evaluation form. Table 10 reports the percentages of Modern Langugaes Education student teachers (N = 3) successful on each standard. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 10. Modern Languages Education Proficiency Rates by Kentucky Teacher Standards Modern Languages 2010‐11 Page 5 of 7 Program MLE Unit‐Wide 1 2 3 100% 100% 100% 97% 91% 96% Kentucky Teacher Standards 4 5 6 7 100% 100% 100% 67% 91% 89% 91% 86% 8 9 100% 100% 94% 90% 10 100% 93% e. Exit and Follow Up Data Table 11 delineates the Educational Testing Services reports of the pass rates on the Praxis II content exams of candidates who completed the program in the 2009‐10 academic year (the most recent year with complete data). The last column allows for pass rate comparison of our candidates to our 2008‐09 results. Table 11. Pass Rates on Content Tests for Initial Teacher Preparation Candidate N (2009‐10) 0 Program/Type of Assessment Spanish Content Knowledge WKU Pass Rate (2009‐10) ‐‐ WKU Pass Rate (2008‐09) 100% Annually, the WKU Teacher Survey is sent to student teachers and alumni who potentially have been teaching one or more years. For the 2010‐11 academic year, out of a possible 2 student teachers, 2 (100%) completed the survey. Below are the results for MLE student teachers, both of whom responded. Survey items requested the respondent’s perception of WKU preparation on each of the Kentucky Teacher Standards using a scale of 1 “Poor,” 2 “Fair,” 3 “Good,” and 4 “Excellent.” Standards with average scores of 3 or better across items were considered to demonstrate acceptable program quality. Table 12 reports MLE survey results. Table 12. Average Scores on Teacher Standards Questions for MLE Respondents Program MLE Unit‐Wide 1 2 3 2.75 3.43 2.20 3.41 3.30 3.52 Kentucky Teacher Standards 4 5 6 7 2.50 3.33 2.70 3.23 2.00 3.29 2.50 3.35 8 9 10 2.38 3.14 3.25 3.36 2.13 3.12 Respondents were also able to provide comments if they answered “poor” for any item. Table 13 presents MLE respondent comments. Table 13. Modern Languages Education Respondent Comments No comments reported for the program. 2. Summary of Results by Kentucky Teacher Standards and Other Key Conceptual Framework Values a. Admission Data: Table 1 shows that mean admission test scores (ACT) and admission grade point averages (GPA) for the Modern Languages Education candidates tracked for the time Modern Languages 2010‐11 Page 6 of 7 period were well above minimum requirements established by the state and/or the WKU Professional Education Unit. b. Course Based Assessment Data: 95% of Modern Languages Education candidates (N = 18) generally scored a 3 or a 4 on a 4 point scale for all CPs. c. The percent of MLE candidates scoring proficient on CPs by KTS had been higher on average in last year’s report than unit‐wide percentages. This advantage dropped by a few percent for many KTS, especially KTS 4: Implements/Manages Instructions. d. Clinical Experiences Data: Unit‐wide disposition scores are high. The percentage of diversity experiences by MLE candidates is higher than unit‐wide averages. e. Teacher Work Sample: Table 8 shows TWS averages for MLE and unit‐wide. MLE candidates scored higher than unit‐wide for all TWS items except DFI and ROT (Design for Instruction, Reflection on Teaching), which were substantially lower at 67%. f. Exit and Follow Up Data: No candidates took the Praxis II exam in the content area. WKU Teacher Survey Results from MLE respondents are lower on average than unit‐wide results. Scores for the years 2008‐09 and 2009‐10 were also lower than unit‐wide averages, but this cycle shows a further drop, so that the differential ranges from .22 to 1.28 lower than unit‐wide averages. It bears asking: Is the sample large enough to be reliable? What could the causes be? 3. Efforts to Report and Disseminate Results: This report is being shared among the ML faculty, and discussed at a faculty meeting. Data in the report can and will be used to inform (a) the types of in‐house professional development offered to faculty, (b), the advising of teacher candidates in the department, and (c) the types of outreach and professional development experiences offered as part of courses or extracurricular activities. 4. Key Discussions and/or Decisions Made Based on Assessment Results a. Assessment or Data Collection Changes Based on Assessment Results The report does not indicate a need to make substantive changes in assessment or data collection at this time. A larger data set (higher N) would of course provide a more reliable basis for change. b. Program Curriculum or Experiences Changes Based on Assessment Results The lower scores Table 3 item 4, and on Table 8 ‐ TWS (Design for Instruction, Reflection on Teaching, both seem to indicate a need for more training specifically related to Modern Languages 2010‐11 Page 7 of 7 foreign language pedagogies. Teacher Survey Results show a drop in graduates’ perception of the effectiveness of preparation they received in the program. As the number of language teachers produced grows, it is worth examining if the department should hire a professor who specializes in foreign language methodologies, or if the problem can be solved through more effective curricular coordination between the Department of Modern Languages and the College of Education faculty. c. Decisions about Group/Individual Student Progress Based on Assessment Results None appear indicated at this time.