English for Secondary Teachers 2011‐12 Page 1 of 10 English for Secondary Teachers – Initial Preparation Annual Program Report Academic Year 2011‐12 Alex Poole November 28, 2012 1. Continuous Assessment Results a. Admission Data Table 1 provides the average admission test scores and admission grade point average (GPA) of English for Secondary Teachers candidates approved by the Professional Education Council (PEC) for admission into initial teacher preparation programs during this academic year. Before the Office of Teacher Services submits their names for review and approval by the PEC, candidates must meet minimum requirements established by the state and/or the WKU Professional Education Unit. Table 1. Approved Candidate Test Score Averages Program English for Secondary Teachers ACT N Mean 18 25 PPST Math N Mean PPST
Reading N Mean
PPST
Writing N Mean
SAT N Mean
GRE Composite N Mean Admission GPA N Mean
18
3.42
b. Course Based Assessment Data Table 2 provides the percentage of English for Secondary Teachers candidates (N = 102) scoring at each level of proficiency on critical performances within education courses for this academic year. Proficiency levels are based on a scale of 1 – Standard Not Met, 2 – Standard Partially Met, 3 – At Standard, and 4 – Above Standard. Table 2. CP Proficiency Level Percentages Course 1 2 3 4 AMS‐333 0% 0% 0% 100% EDU‐250 0% 0% 77.91% 22.09% EDU‐489 0% 0% 83.33% 16.67% EXED‐330 0% 0% 100% 0% LME‐407 0% 0% 0% 100% LTCY‐421 0% 0% 84.21% 15.79% MGE‐275 0% 0% 36.36% 63.64% MGE‐385 0% 0% 60% 40% MGE‐475 0% 0% 0% 100% MGE‐481 0% 0% 100% 0% PSY‐310 0% 0% 13.58% 86.42% English for Secondary Teachers 2011‐12 Page 2 of 10 Course 1 2 3 4 SEC‐351 0% 0% 45.81% 54.19% SEC‐352 .89% 0% 13.39% 85.71% SEC‐453 0% 0% 41.56% 58.44% SEC‐475 0% 0% 0% 100% SMED‐320 41.03% 0% 58.97 0% Grand Total 1.95% 0% 41.84% 56.22% Table 3 indicates the level of English for Secondary Teachers candidates (N = 102) proficiency across critical performances related to the Kentucky Teacher Standards (KTS). Candidates receiving an overall rating of 3 or 4 on a CP are considered to have demonstrated proficiency on the standards associated with the CP. Compared to the unit‐wide results, English for Secondary Teachers candidates are typically performing above average. Table 3. Percent of English for Secondary Candidates Scoring Proficient on CPs by KTS Program English for Secondary Teachers Unit‐Wide 1 98.35
% 97% 2 3 97.44
% 97% 98.59
% 97% Kentucky Teacher Standards 4 5 6 7 96.16
% 98% 97.36
% 96% 97.5%
96% 97.99
% 97% 8 9 100% 97% 98.23
% 95% 10 100% 98% *KTS Key: 1 – Content Knowledge, 2 – Designs/Plans Instruction, 3 – Maintains Learning Climate, 4 – Implements/ Manages Instruction, 5 – Assessment/Evaluation, 6 – Technology, 7 – Reflection, 8 – Collaboration, 9 – Professional Development, 10 – Leadership Table 4 indicates the number of English for Secondary Teachers candidates (N = 2) who have scored 2 or lower (below proficiency) on critical performances during this academic year. Table 4. English for Secondary Teachers Candidates Scoring Below Proficient on CPs Score Student ID 800534020 800593363 Grand Total 1 2 1 1 1 1 2
c. Clinical Experiences Data Student Count 2 English for Secondary Teachers 2011‐12 Page 3 of 10 English for Secondary Teachers uses the following courses and experiences to evaluate candidate dispositions: EDU 250 and EDU 490. The program has identified the following courses and experiences where candidates report the diversity of their field experiences: EDU 250, EDU 480, and SEC 352. SEC 352 has been designated as the experience where candidates must work in settings at or above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 5 reports how English for Secondary Teachers candidates performed on dispositions as they entered and progressed through their program (N = 86) and during their student teaching experience (N =12). Students are considered “proficient” who average a 3 or higher on each disposition category. Table 5. English for Secondary Teachers Proficiency Rates on Unit‐Wide Dispositions Period a. Prior to Student Teaching b. During Student Teaching Values Learning WKU Professional Education Dispositions Values Personal Values Values Values Integrity Diversity Collaboration Professionalism 99.36% 100% 100% 100% 100% 91.67% 100% 100% 100% 100% Over this academic year, English for Secondary Teachers candidates (N = 79) reported demographic information on 103 field placements with an average of 18% ethnically diverse students, 46% students on free/reduced lunch, and 10% student with disabilities (based on National Center for Education Statistics and Kentucky Department of Education). This ethnic diversity percentage continues to be above the average 11% diversity of the schools in the 30+ counties that represent our service area. Table 6 reveals the percentages of field experiences with various characteristics. Note that candidates could choose all the characteristics that applied for any given experience. Table 6. Percentages of Field Experience by Category Types Working with Student With Special Needs % Candidates working with Students with Physical Impairments % Candidates working with Students with Learning Disabilities % Candidates working with Students with Moderate/Severe Disabilities % Candidates working with Students with Emotional/Behavioral Disorders % Candidates working with Gifted Students % Candidates working with English Language Learners % Candidates working with Students with Visual Impairments % Candidates working with Students with Hearing Impairments % Candidates working with Students with Speech/Language Delays % Candidates working with Students with Development Delays 12
61
11
37
51
37
8
8
22
13
English for Secondary Teachers 2011‐12 Page 4 of 10 % Candidates working with Students with Autism Spectrum Disorder % Candidates working with Students with Other Impairments Working with Diverse Students % Candidates working with African American Students % Candidates working with Native American/American Indian Students % Candidates working with Latino/Hispanic Students % Candidates working with Asian Students % Candidates working with Students with Special Needs (Aggregate) % Candidates working with Diverse Students (Aggregate) 18
6
87
12
69
44
90
94
Overall, as can be seen in Table 6, in 90% of their field experiences, English for Secondary Teachers candidates reported working with at least one student with special needs and in 94% of their field experiences candidates reported working with at least one student from a diverse ethnic group. d. Culminating Assessment Data As Component 4 of the WKU Professional Education Unit Continuous Assessment Plan (CAP) strategy, all initial preparation candidates complete a culminating assessment of professional and pedagogical knowledge and skills, the Teacher Work Sample (TWS). This assessment is also used to demonstrate candidates’ ability to impact P‐12 student learning. In particular, candidate performances on Assessment Planning and Analysis of Student Learning have been identified as key indicators of candidates’ ability related to student learning. Although in spring 2008 the Professional Education Council agreed that candidates who score a holistic score of at least “2 – Developing” are able to exit the program, for program evaluation purposes our goal is that at least 80% of program candidates will achieve “3 – Proficient” or higher. Table 7 presents the proficiency rate for English for Secondary Teachers candidates (N = 12). Table 7. Initial Preparation TWS Proficiency Rates Program English for Secondary Teachers Unit‐Wide % Proficient 100% 96% Because the faculty also scores TWS at the indicator level, we are able to use these scores to ascertain candidate success in meeting each component of the TWS. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 8 depicts the percentage of English for Secondary Teachers candidates who averaged at least 2.5 on the indicators for each TWS Factor: CF – Contextual Factors, LG – Learning Goals, DFI – Design for Instruction, ASL – Analysis of Student Learning, and ROT – Reflection on Teaching. English for Secondary Teachers 2011‐12 Page 5 of 10 Table 8. Initial Preparation TWS Proficiency Rates of English for Secondary Teachers Candidates Program English for Secondary Teachers Unit‐Wide CF LG DFI ASL ROT 100% 94% 100% 91% 100% 89% 100% 92% 100% 88% Because the TWS indicators have been aligned to Kentucky Teacher Standards, we can use these scores to ascertain candidate success in meeting each standard related to the TWS.
Table 9 reports these scores as they relate to Kentucky Teacher Standards. Table 9. Percentage of English for Secondary Teachers Candidates who “Passed” each Teacher Standard Program English for Secondary Teachers Unit‐Wide 1 2 3 5 6 7 9 91.67% 83% 91.67% 91% 91.67% 92% 100% 88% 91.67% 83% 83.33% 76% 91.67% 88% Additionally, all candidates are assessed during their student teaching experience using the Student Teaching Evaluation form. Table 10 reports the percentages of English for Secondary Teachers student teachers (N = 12) successful on each standard. For program evaluation purposes, candidates are considered successful who average at least 2.5 on a three point scale (1 – Not Met, 2 – Partially Met, and 3 – Met) on indicators aligned to a standard. Table 10. English for Secondary Teachers Proficiency Rates by Kentucky Teacher Standards Program English for Secondary Teachers Unit‐Wide Kentucky Teacher Standards 4 5 6 7 1 2 3 91.7
% 83.3
% 91.7%
91.7%
83.3%
75% 94% 89% 94% 89% 88% 82% 8 9 10 91.7%
75% 83.3%
100% 87% 90% 88% 90% e. Exit and Follow Up Data Table 11 delineates the Educational Testing Services reports of the pass rates on the Praxis II content exams of candidates who completed the program in the 2010‐11 academic year (the English for Secondary Teachers 2011‐12 Page 6 of 10 most recent year with complete data). The last column allows for pass rate comparison of our candidates to our 2009‐2010 results. Table 11. Pass Rates on Content Tests for Initial Teacher Preparation Candidate N (2010‐11) 27 25 Program/Type of Assessment ENG LANG LIT COMP CONTENT KNOW ENG LANG LIT COMP ESSAYS WKU Pass Rate (2010‐11) 96% 96% WKU Pass Rate (2009‐10) 89% 83% Annually, the WKU Teacher Survey is sent to student teachers and alumni who potentially have been teaching one or more years. For the 2011‐12 academic year, out of a possible 12 student teachers, 12 (100%) completed the survey. Below are the results for English for Secondary Teachers student teachers, 12 of whom responded. Survey items requested the respondent’s perception of WKU preparation on each of the Kentucky Teacher Standards using a scale of 1 “Poor,” 2 “Fair,” 3 “Good,” and 4 “Excellent.” Standards with average scores of 3 or better across items were considered to demonstrate acceptable program quality. Table 12 reports English for Secondary Teachers survey results. Table 12. Average Scores on Teacher Standards Questions for English for Secondary Teachers Respondents Program English for Secondary Teachers Unit‐Wide Kentucky Teacher Standards 4 5 6 7 1 2 3 3.42 3.40 3.60 3.22 3.44 3.42 3.61 3.37 3.10 3.23 3.08 8 9 10 2.71 3.60 3.06 3.25 3.40 3.32 3.06 3.31 3.09 Respondents were also able to provide comments. Table 13 presents English for Secondary Teachers respondent comments. Table 13. English for Secondary Teachers Respondent Comments While I am confident in the material that I was taught in the education department, I would have liked to have been more well informed about the process of entering the College of Education. I knew nothing going in, and wasn't offered much help (with classes, requirements and Praxis). When we as future educators go into the field, all we are told is to get 15 hours per class. We need to have higher expectations for our field work. We should be expected to teach, to grade, to do administrative tasks, and to get involved with extracurriculars. We should also be exposed to assessing data in conjunction with using technology and be expected to create a plan from the results. We don't have an opportunity to do this in LTCY 421 because we are expected to have a plan before we even know the student, and we can't change our plan. English for Secondary Teachers 2011‐12 Page 7 of 10 We need to have higher expectations set on us as future educators, and we also need more diverse experiences in teaching before our student teaching experience. The English Department's Quality Enhancement Plan (QEP) includes five "Intended Educational (Student) Outcomes" which are examined as students near the completion of their initial preparation program. Below are the outcomes, means of assessment and criteria (where available), and most recent results. Outcome 1: Students will demonstrate the ability to critically analyze a literary text (NOTE: “literary” can be defined broadly to include film, pop culture, and other texts that are formally studied in English courses). Method of Assessment: Portfolio of writing assignments Criteria: 75% score 4 or higher (on five‐point scale), aggregate, none below 3 Results: 73% (11/15) scored 4 or higher; none below 3 Outcome 2: Students will show the ability to use secondary sources to support an argument about a literary text. Method of Assessment: Portfolio of writing assignments Criteria: 75% score 4 or higher (on five‐point scale), aggregate, none below 3
Results: 60% (9/15) scored 4 or higher; none below 3 Outcome 3: Students (self‐selected) will understand the process and protocol of applying to graduate/professional school OR students (self‐selected) will understand the process and protocol of applying for a job related to their field of study. Method of Assessment: Portfolio of writing assignments Criteria: 75% score 4 or higher (on five‐point scale), aggregate, none below 3 Results: 67% (12/18) scored 4 or higher, 1/18 below 3 Outcome 4: English Majors will be aware of study abroad opportunities. English for Secondary Teachers 2011‐12 Page 8 of 10 Method of Assessment: Senior exit survey asked students to rate their knowledge of study abroad opportunities on a scale of 1 (lowest) to 5 (highest). Results: 5 = 7; 4 = 15; 3 = 6; 2 = 3; 1 = 0 71% 4 or higher; 10% Below 3 Outcome 5: English majors will have studied minority and non‐Western literature. Measurement: Senior exit survey asked students the following: “As an English major, I have formally studied minority and non‐Western literature” Method of Assessment: Senior exit survey asked students the following: “As an English major, I have formally studied minority and non‐Western literature.” Results: Strongly agree= 9; Agree=18; Neutral =3; Disagree=1; Strongly Disagree=0 87% Agree or Strongly Agree; 3% Disagree or Strongly disagree English for Secondary Teachers 2011‐12 Page 9 of 10 2. Summary of Results by Kentucky Teacher Standards and Other Key Conceptual Framework Values a. Admission Data: The EST ACT mean (25) was slightly higher than the mean of all teaching candidates (24). The EST GPA mean (3.42) was also higher than the unit‐wide mean(3.29). Incoming candidates bring strong academic qualifications. b. Course Based Assessment Data: CP proficiency level percentages were all in the upper 90’s, and all but one were higher than unit‐wide averages. c. Clinical Experiences Data: The EST proficiency rates on the Unit‐Wide Dispositions do not differ prior to and during student teaching, except on the “values learning” category, which dropped almost .8% from prior to student teaching to during student teaching. The reasons for this drop are unknown, but we need to monitor this category in order to see if this discrepancy is a long‐term trend (i.e., year‐to‐year) or just a one‐time phenomenon. The field experience data indicate that students have experiences in diverse settings. d. Culminating Assessment Data: Figures for the Teacher Work Sample show excellent performance by our students. Specifically, they scored 100% in all five categories. The EST proficiency rates for all Kentucky Teacher Standards are also good, with five being at 90% or above, three being between 80% and 90%, and two being between 70% and 80%. Average scores for standards six and eight were the lowest (75% for each). On all but two standards (4 and 10), averages were lower than the previous year’s. However, the number of participants is quite small (N=12), so overall averages can easily be skewed by outliers. e. Exit and Follow‐up Data: EST students continue to improve their performance on Praxis II exams. On the "Content Knowledge" test, 2010‐2011 students had a 96% pass rate. On the "Essays" test, the pass rate was also 96%. These are impressive improvements over last year’s pass rates, which were 89% and 83%, respectively. The results of the WKU Teacher Survey showed that students feel slightly less prepared than average. However, average scores on all standards were higher than last year’s, except for one (8). This again shows steady improvement. 3. Efforts to Report and Disseminate Results: This report was forwarded to the English Department's English for Secondary Teachers Committee for reading and discussion. At the beginning of the semester, Praxis and English capstone course results were distributed to and discussed by the department as a whole. English for Secondary Teachers 2011‐12 Page 10 of 10 4. Key Discussions and/or Decisions Made Based on Assessment Results a. Assessment or Data Collection Changes Based on Assessment Results. We will continue to differentiate the EST majors from all other English majors. By doing this, we will get a more accurate picture of the former’s accomplishments and needs. It may also help us determine whether or not unique assessment goals need to be created for EST majors. b. Program Curriculum or Experiences Changes Based on Assessment Results We made significant changes to our curriculum four years ago. This assessment cycle is only the second one since those changes have been made; however, it still included students in the old curriculum. Therefore, it would be premature to make any decisions. In addition, a more long‐term approach to program assessment is necessary given the small numbers of participants. This year’s assessment pool only included 12 participants; as a result, scores could easily be skewed. c. Decisions about Group/Individual Student Progress Based on Assessment Results No significant changes have been made. As reported above, we made significant changes to our curriculum four years ago. This assessment cycle is only the second one since those changes have been made; however, it still included students in the old curriculum. Therefore, it would be premature to make any decisions. 5. Discuss trends in assessment results over the last few years (Please refer back to your 2009‐10 and 2010‐11 APRs which are posted to the College of Education Professional Education Unit website) Our students continue to generally perform well on all assessments. Of course, there are fluctuations on certain specific assessments from year‐to‐year, yet given the relatively small numbers of students in each assessment cycle, this is statistically probable.