School of Education Undergraduate Unit Assessment Submitted by: Brian McAlister, Director & Lesley Voigt, Assessment Coordinator 2010 Submitted October, 2011 Table of Contents Introduction ............................................................................................................................................................................................................. 1 PRAXIS I: Pre-Professional Skills Test ................................................................................................................................................................. 2 PPST Reading ..................................................................................................................................................................................................... 3 PPST Writing ...................................................................................................................................................................................................... 5 Writing Test Category......................................................................................................................................................................................... 6 Points................................................................................................................................................................................................................... 6 Type of Test ........................................................................................................................................................................................................ 6 UW-Stout ............................................................................................................................................................................................................ 6 State..................................................................................................................................................................................................................... 6 National ............................................................................................................................................................................................................... 6 Average Percent Correct (percentage of items correctly per category) .................................................................................................................. 6 PPST Mathematics .............................................................................................................................................................................................. 7 PRAXIS II: Content Test ........................................................................................................................................................................................ 9 Art Education .................................................................................................................................................................................................... 11 Business Education ........................................................................................................................................................................................... 11 Elementary Education ....................................................................................................................................................................................... 12 Family & Consumer Sciences Education ......................................................................................................................................................... 12 Health Education ............................................................................................................................................................................................... 13 Marketing Education ......................................................................................................................................................................................... 14 Middle School Subjects – Special Education ................................................................................................................................................... 15 Science Education ............................................................................................................................................................................................. 15 Technology Education ...................................................................................................................................................................................... 15 Student Teacher Performance Ratings .................................................................................................................................................................. 20 Educational Benchmarking Inventory (EBI) for Exiting Student Teachers ......................................................................................................... 23 EBI Factor Means Highest Difference to Lowest Difference ........................................................................................................................... 23 Trend Analysis of All EBI Factors ................................................................................................................................................................... 24 EBI SOE Specific Questions Related to Wisconsin Teacher Standards........................................................................................................... 25 EBI - Institution Specific Questions ................................................................................................................................................................. 25 Mean Data; Scale (1-Not at all, 4-Moderately, 7-Extremely) .......................................................................................................................... 25 School of Education Undergraduate Follow-Up Survey Results ..................................................................................................................... 26 Communicating Assessment Data with Constituencies .................................................................................................................................... 28 Utilization of Assessment Data to Improve Courses and the Unit ................................................................................................................... 28 School of Education’s Unit Assessment Report 2010 Introduction This report is a summary of the University of Wisconsin-Stout’s School of Education (SOE) assessment data for undergraduates and teacher education candidates gathered from the fall semester 2003 through December 2010. In the School of Education, data is gathered from several sources to inform unit and program decisions. Data from this report is used to develop unit and program goals, inform curriculum changes, and enhance course delivery in order to improve teacher education candidate learning. This report contains data from the PRAXIS I: PreProfessional Skills Test, PRAXIS II: Content Test, Student Teacher Performances, Benchmark Interview Ratings (new in 2008) the Educational Benchmarking Inventory (EBI). Program Specific Reports Program specific reports submitted are supplemental to this summary and provide data and narrative descriptions and analyses of Graduate (and Employer) Follow-up Surveys, Student Teacher Exit Surveys, Student Teaching Seminar Surveys, and other program specific data which aid Program Directors in making program and curricular decisions. The program specific reports also describe how this assessment data is used to improve the program, program curriculum, and delivery of courses. In addition, Program Directors use information from the program-specific Assessment in the Major reports to identify and describe program goals for the upcoming year. Assessment Data Uses The unit and program assessment reports are shared with internal and external constituents of the School of Education, the SOE Advisory Board, and individual Program Advisory Committees. Advisory committee members discuss trends and make recommendations for improvement to Program Directors, the SOE Director, and the Dean of the College of Education, Health and Human Sciences. The Director meets regularly each semester with individual Program Directors to discuss program data, yearly goals, and progress toward achieving short-term and long-term goals. SOE unit and program goals are informed by the data and are developed in alignment with university goals and priorities as well as external standards developed by the Wisconsin Department of Public Instruction and various external accrediting agencies. Organization of Assessment Report This report is organized into sections based on the source of the data. The Table of Contents may be used to navigate to a specific section or subsection of the report. To navigate without scrolling, go to the Table of Contents page. When viewing the data tables throughout the report, the current year data column is shaded and the text is bolded. 1 PRAXIS I: Pre-Professional Skills Test Educational Testing Service Institutional Report The PRAXIS I: Pre-Professional Skills Test (PPST) is required for teacher certification by the Wisconsin Department of Public Instruction. Passing the PPST is required as part of meeting SOE’s Benchmark I: Acceptance into Teacher Education Program and Pre-student Teaching Experiences. Teacher candidates are not admitted into the teacher education program until they pass the PPST. The PPST is administered through the Educational Testing Service (ETS). The PPST consists of three tests: Reading, Writing and Mathematics. All three tests must be passed to meet the SOE Education’s Benchmark I requirements. These tests can be taken in a handwritten format in traditional testing settings at designated sites and times or by computer at designated sites. ETS provides an annual institutional academic year summary report on all students attempting the PPST and passing rates. It also compares scores of UW-Stout students to those at the state level and the national level. Note that according to PI34.14 (1) (b) Exceptions under par. (a) relating to the established passing scores on standardized tests or SCD designed or approved assessments, or the minimum cumulative grade point average may be granted to no more than 10% of the total number of students admitted to the initial or advanced programs for each admission period. During 2010, an exception to the PPST cut score was granted for 1 candidate representing .7% of those that were accepted into the program during 2010. In other words, 99.3% of teacher education candidates that were accepted to teacher education programs in 2010 passed the PPST. 2 PPST Reading The PPST Reading test results based on the ETS institutional report indicates that UW-Stout scores have remained consistent over the past few years. PPST Reading Number of UW-Stout Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Passing Score: Number of attempts with WI Passing Score: Percentage of attempts with WI Passing Score: Type of Test Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp 05/06 06/07 07/08 08/09 09/10 101 174 185 186 159 158 175 176 169-180 172-181 125 146 187 186 157 160 176 176 171-179 172-179 79 114 185 185 156 162 176 176 172-179 172-180 84 122 185 185 157 162 177 178 173-180 175-182 83 103 183 186 160 164 177 177 173-180 175-181 175 175 175 175 175 58/101 113/174 57% 65% 77/125 89/146 62% 61% 49/79 66/114 62% 58% 59/84 93/122 70% 76% 53/83 78/103 64% 77% 3 Average Percent Correct (percentage of correct answers by category on paper and pencil format) Reading Test Category Literal Points Available 17-21 Comprehension Critical and Inferential Comprehension 18-23 Type of Test Paper 05/06 74% 06/07 75% 07/08 70% 08/09 68% Comp NA NA NA Paper 69% 72% Comp NA NA UW-Stout State National 09/10 70% 09/10 09/10 75% 72% NA 75% 77% 74% 65% 67% 66% 72% 69% NA NA 76% 81% 78% UW-Stout’s average percent correct (percentage of items answered correctly) on the two reading test categories of Literal Comprehension and Critical/Inferential Comprehension increased from 2005/06 to 2006/07 for the teacher education candidates. UW-Stout’s teacher candidate average scores in Literal Comprehension dropped from 06/07-08/09, but that appears to be consistent with the trend at the State and National Level. However, Critical and Inferential Comprehension average scores improved. Until 2010, we were unable to see State and National comparisons at the detail level. Percent correct scores are shown to be much higher when taken via computer as is consistent with State and National level. 4 PPST Writing The PPST Writing test results based on the ETS institutional report shows that the lowest observed score has increased dramatically over the past few years. This shows improvement in those students who were scoring lowest have greatly increased their writing skills. We are anticipating that these scores will again increase due to implementing an elective 1-credit writing class to assist students struggling in this area and by continuing to support our PRAXIS tutor support system. Type of Test Paper Number of UW-Stout Examinees: Comp Paper Highest Observed Score: Comp Paper Lowest Observed Score: Comp Paper Median: Comp Paper Average Performance Range: Comp Paper WI Passing Score: Comp Paper Number with WI Passing Score: Comp Paper Percent with WI Passing Score: Comp PPST Writing 05/06 06/07 07/08 08/09 09/10 104 211 181 183 163 151 174 174 172-175 171-176 124 159 183 182 161 166 174 175 172-177 173-176 68 102 182 185 168 166 174 175 172-176 172-177 77 116 182 184 165 152 174 175 172-176 173-177 84 124 182 184 168 166 174.5 175 172-177 173-177 174 174 174 174 174 57/104 128/211 55% 61% 79/124 104/159 64% 65% 35/68 68/102 51% 67% 43/77 84/116 56% 72% 50/84 86/124 60% 69% The Computer PPST Writing test results based on the ETS institutional report indicates that our candidates had a better pass rate on 4 out of the past 5 years on the computerized version of the test vs. the paper and pencil version. 5 Average Percent Correct (percentage of items correctly per category) Writing Test Category Points Available Grammatical Relationships 11-13 Structural Relationships 14-16 Idiom/Word Choice Mechanics 10-12 Essay 12 Type of Test Paper Comp Paper Comp Paper Comp Paper Comp State UW-Stout National 05/06 06/07 07/08 08/09 09/10 09/10 09/10 48% NA 49% 52% NA 54% 59% NA 57% 56% NA 53% 57% 58% 56% 60% 64% 64% 58% 62% 62% NA 52% NA 66% NA NA 55% NA 67% NA NA 50% NA 63% NA NA 56% NA 61% NA 62% 52% 60% 63% 62% 67% 58% 64% 64% 64% 65% 56% 62% 62% 62% In all four writing test categories, UW-Stout students continue to score below the state level. However, UW-Stout teacher candidate average scores are typically within a few percentage points of the national averages. 6 PPST Mathematics The PPST Mathematics test results based on the ETS institutional report shows that pass rates of UW-Stout teacher candidates continues to decrease, however the success rate is still higher than Reading and Writing. In 2007/08, the Mathematics test was updated. The results below reflect the new test. PPST Mathematics Number of UW-Stout Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Passing Score: Number with WI Passing Score: Percent with WI Passing Score: Type of Test 07/08 08/09 09/10 Paper 68 76 76 Comp Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp Paper Comp 16 187 189 160 170 176 183 171-182 178-185 105 188 188 162 163 179 178 172-184 173-182 107 188 188 162 161 179 178 174-182 173-182 173 173 173 46/68 15/16 54/76 83/105 56/76 86/107 Paper 68% 71% 74% Comp 94% 79% 80% The Computer PPST Mathematics test results based on the ETS institutional report indicates that this is definitely our strongest area. We have fewer candidates taking this test due to the percent pass rate being so high. This chart also shows that our candidates do better on the computerized version vs. the paper and pencil version. 7 Average Percent Correct (percentage of correct answers by category) Type UW-Stout Wisconsin Mathematics Points Test Category Available of Test 07/08 08/09 09/10 09/10 Paper 48% 54% 57% 62% Number and 11-13 Comp operations NA NA 65% 73% Paper 51% 57% 57% 63% Algebra 7-8 Comp NA NA 72% 77% Paper 58% 53% 62% Geometry and 66% 7-9 Comp Measurement NA NA 71% 74% Paper 55% 61% 60% 65% Data Analysis 10 Comp and Probability NA NA 77% 81% National 09/10 56% 68% 56% 71% 58% 68% 60% 77% UW-Stout teacher candidates scored higher than the national average percent correct in all four math test categories every year. 8 PRAXIS II: Content Test Benchmark II: Admission to Student Teaching requires candidates to pass PRAXIS II, the content test for a specific teacher certification area. As of 8/31/2004, all Wisconsin teacher education students must pass the content test to be eligible to student teach. No exceptions are granted by the School of Education for the PRAXIS II. Therefore, we have a 100% pass rate for all candidates that enter student teaching. Praxis II report data is general and does not clearly differentiate among undergraduate teacher education students, graduate teacher education students, teachers who want to add-on an additional teacher certification or the “certification only” category. Note that the number of examinees taking a designated content test varies by content test area. 9 Note: All teacher education candidates are required to pass the Praxis II to be admitted to student teaching as a part of Benchmark II Requirements Marketing and Business Education candidates must pass #10100 and #10560 to be eligible for student teaching and licensure Science and Technology Education candidates must pass #10050 and #10435 to be eligible for student teaching and licensure 10 Art Education Praxis Test Code - 10133 The Art Education has a history of passing this test pretty consistently on the first attempt. Art Education data from the ETS report is as follows: Content Test from ETS 05/06 06/07 07/08 08/09 09/10 8 13 9 18 14 Highest Observed Score: 186 194 173 183 187 Lowest Observed Score: 155 156 155 147 158 164 158-168 155 175 165-180 155 167 160-168 155 168.5 157-174 155 171.5 168-175 155 8/8 13/13 9/10 16/18 14/14 100% 100% 90% 89% 100% Number of Examinees: Median: Average Performance Range WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: Business Education Praxis Test Code - 10100 The data below shows that Business Education candidates consistently have a 100% pass rate on the Business Education Test. Content Test from ETS 05/06 06/07 07/08 08/09 09/10 Number of Examinees: 15 7 17 18 18 Highest Observed Score: 730 670 770 670 710 Lowest Observed Score: 610 620 580 590 610 Median: 660 620 630 635 660 Average Performance Range: 650-680 620-660 610-650 620-650 650-680 WI Score Needed to Pass: 580 580 580 580 580 Number with WI Passing Score: 15/15 7/7 17/17 18/18 18/18 Percent with WI Passing Score: 100% 100% 100% 100% 100% 11 Elementary Education Praxis Test Code - 10014 Elementary Education candidates pass rate has rebounded since a minor dip the previous two years. Content Test from ETS Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Scores: 05/06 49 189 124 160 148-168 147 06/07 67 195 129 161 151-168 147 07/08 68 197 132 157 150-167 147 08/09 50 191 134 159.5 149-170 147 09/10 50 189 133 158 149-170 147 40/49 61/67 48/68 43/50 44/50 82% 91% 71% 86% 88% Family & Consumer Sciences Education Praxis Test Code – 10121 (New test as of 9/01/08) Candidates felt the need to hurry and take the FCSE Content test early in 2008 before the new test was introduced. It is obvious by the results that these students were not ready. The new test appears to be a much more accurate assessment as all but one candidate passed this test on the first attempt. Content Test from ETS (0120) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 05/06 18 740 600 665 620-710 590 18/18 100% 06/07 14 740 590 655 630-680 590 14/14 100% 07/08 13 730 530 640 600-710 590 11/13 85% * - scores from new test #10121 have been added to the totals scores could not be reported due to format changes in the new test Content Test from ETS (0121) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 08/09 18 197 151 166.5 159-171 159 15/18 83% 09/10 4 NA NA NA NA 159 ¾ 75% 12 Health Education Praxis Test Code - 20550 Health Education data from the ETS report is as follows: ????? Content Test from ETS Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 05/06 06/07 07/08 08/09 9 830 630 660 710750 610 9/9 100% 10 790 640 660 650780 610 10/10 100% 4 - 7 700 520 680 600690 610 5/7 71% 610 - 13 Marketing Education Praxis Test Code – 10561 (New test as of 9/01/08) The new Marketing Education Praxis II test was not administered during the Fall 2008 semester. ETS does not offer that test during every testing date. The first date this test was offered during the 2008-2009 school year was in January. Candidate performance rebounded to 100% during 2009-2010. Content Test from ETS (0560) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 05/06 06/07 07/08 15 820 610 720 6 720 610 705 15 780 590 630 660-750 690-720 610-720 600 600 600 15/15 6/6 13/15 100% 100% 87% Content Test from ETS (0561) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 08/09 19 191 145 162 156-175 153 17/19 89% 09/10 18 185 156 171.5 166-177 153 18/18 100% 14 Middle School Subjects – Special Education Praxis Test Code – 20146 Special Education candidates take the Middle School Subjects content test in the state of Wisconsin. Content Test from ETS Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: Score Needed to Pass: Number with Passing Score: Percent with WI Passing Score: Science Education Praxis Test Code – 10435 Content Test from ETS (0435) Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: 05/06 06/07 07/08 08/09 09/10 15 185 128 148 143-159 146 9 60% 27 177 134 151 148-162 146 21/27 78% 34 174 122 152 147-158 146 28/34 82% 28 181 128 152 143-162 146 20/28 71% 28 174 131 148.5 139-157 146 18/28 64% 08/09 09/10 5 197 161 164 7 187 142 163 Average Performance Range: WI Score Needed to Pass: 163-173 161-176 154 154 Number with WI Passing Score: 5/5 6/7 Percent with WI Passing Score: 100% 86% Technology Education Praxis Test Code – 10050 15 Content Test from ETS Number of Examinees: Highest Observed Score: Lowest Observed Score: Median: Average Performance Range: WI Score Needed to Pass: Number with WI Passing Score: Percent with WI Passing Score: 05/06 56 750 580 655 630-680 590 55/56 98% 06/07 38 720 560 650 630-680 590 35/38 92% 07/08 39 720 550 670 630-700 590 37/39 95% 08/09 29 710 620 670 650-680 590 29/29 100% 09/10 30 730 590 670 640-690 590 30/30 100% 16 Benchmark I Interview Results SOE Unit Number Question 1 Explain personal and professional growth between your initial resume and updated resume. Explain your philosophy of education. 2 3 4 5 6 Unsatisfactory Satisfactory Unsatisfactory Satisfactory Explain three personal characteristics that will make you an effective teacher. Unsatisfactory Describe yourself as a learner and how that will impact your future teaching. Unsatisfactory Describe experiences that have impacted your understanding of diversity and human relations and how these might aid you as you work with students and families Explain two subject matter/content artifacts and how these examples illustrate your understanding of the content you will be teaching Completed Alignment Summary 7 Response Satisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory SOE UNIT 2008 2009 2010 N=133 N=96 N=80 0% 3% 1% 100% 97% 99% 0% 1% 0% 100% 0% 99% 100% 1% 0% 100% 99% 100% 0% 1% 100% 100% 99% 99% 0% 2% 0% 100% 98% 100% 0% 5% 100% 100% 95% 99% 0% 2% 0% 100% 98% 100% 17 Benchmark II Interview Results SOE Unit 2008 2009 2010 Response N=147 N=129 N=80 Unsatisfactory 1% 2% 1% 36% 29% 41% Describe your Philosophy of Education and how it Emerging has evolved Basic 62% 69% 58% Not Observed 1% 0% 0% Unsatisfactory 1% 2% 1% Emerging 31% 22% 34% Describe what it means to be a "Reflective Practitioner" Basic 66% 76% 64% Not Observed 1% 0% 0% Unsatisfactory 1% 2% 0% Emerging 26% 19% 31% Describe the WI Teacher Standard and Domain you feel most competent in Basic 72% 79% 69% Not Observed 1% 0% 0% Unsatisfactory 0% 2% 0% Emerging 32% 26% 32% Describe the WI Teacher Standard and Domain you have experienced the greatest growth Basic 66% 72% 68% Not Observed 2% 0% 0% Unsatisfactory NA NA 0% Provide Portfolio evidence (signed copy of the Emerging NA NA 52% Instructional Technology Utilization rubric) of your Basic NA NA 48% competence in current instructional technology Not Observed NA NA 0% Reviewers choose 2 of the following; discuss portfolio evidence that: Unsatisfactory 0% 2% 2% Emerging 11% 14% 27% demonstrates your content knowledge Basic 34% 31% 71% Not Observed 55% 53% 0% Unsatisfactory 0% 8% 3% Emerging 18% 8% 40% demonstrates your ability to create instructional opportunities adapted to diverse learners Basic 31% 34% 57% Not Observed 51% 50% 0% Unsatisfactory 0% 0% 4% Emerging 10% 3% 25% demonstrates your ability to teach effectively Basic 7% 12% 71% Not Observed 83% 85% 0% Unsatisfactory 0% 1% 2% Emerging 17% 17% 41% demonstrates your ability to assess student learning Basic 23% 36% 57% Not Observed 60% 46% 0% Question 18 Benchmark III Interview Results SOE Unit Number Question Artifacts from student teaching, reflection ratings 1 2 3 Final Student Teaching Assessments and Recommendations from Cooperating Teachers Disposition ratings from student teaching from cooperating & University Supervisors Instructional Technology Utilization Rubric 4 5 Alignment Summary of artifacts meeting all 10 Wisconsin Teaching Standards & 4 Domains/ Components & reflections/ reflection ratings Response Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a 2008 N=143 0% 5% 39% 53% 3% 0% 5% 33% 62% 0% 0% 3% 32% 63% 2% NA NA NA NA NA 0% 6% 13% 81% 0% SOE UNIT 2009 2010 N=127 N=138 0% 0% 2% 0% 24% 24% 74% 76% 0% 0% 0% 1% 2% 1% 24% 20% 72% 78% 2% 0% 0% 0% 1% 1% 20% 20% 43% 76% 36% 3% NA 0% NA 1% NA 19% NA 77% NA 3% 0% 0% 9% 14% 15% 8% 75% 75% 1% 3% 19 Student Teacher Performance Ratings The School of Education uses Danielson’s domains and the Wisconsin Teacher Standards as components of the framework for student teacher competency evaluations. Cooperating teachers utilize these competencies to rate student teachers on their final performance evaluation. Student teaching placements vary among programs. The numbers have been tabulated by averaging the scores per item per candidate rather than on each experience. • Early Childhood Education candidates complete three student teaching placements at the preschool, kindergarten and primary levels. • Art Education student teachers complete two nine-week student teaching placements at the elementary and secondary levels. • Family and Consumer Sciences Education, Marketing and Business Education, Science Education, and Technology Education student teachers complete two nine-week student teaching placements at the middle school and high school levels. • Special Education candidates complete a semester long student teaching placement at either the elementary, middle or high school level depending on their individual licensure needs. Student Teacher Evaluations SOE Unit Rating Scale: 1=Unsatisfactory, 2=Emerging, 3=Basic, 4=Proficient SOE UNIT 2008 2009 2010 N=135 N=152 N=120 Mean Mean Mean Teachers know the subjects they are teaching 3.71 3.80 3.78 Teachers know how children grow 3.61 3.73 3.82 Teachers understand that children learn differently 3.68 3.76 3.73 Teachers know how to teach 3.61 3.78 3.84 Teachers know how to manage a classroom 3.54 3.71 3.65 Teachers communicate well 3.61 3.71 3.78 Teachers are able to plan different kinds of lessons 3.64 3.80 3.77 Teachers know how to test for student progress 3.65 3.65 3.75 Teachers are able to evaluate themselves 3.69 3.80 3.78 Teachers are connected with other teachers and the community 3.68 3.64 3.70 Teachers make effective use of instructional technologies to enhance student learning. NA NA 3.91 20 21 Each of the program/certification areas has been inspected to determine patterns in student teacher competency ratings from cooperating teachers. The highest and lowest component rating means and the Wisconsin Teacher Standards rating means for each program for calendar years 2008 through 2010 are displayed below. In 2010, an additional item was added “Teachers make effective use of instructional technologies to enhance student learning”, to help evaluate students use of instructional technologies in the field. This item was significantly our highest rating component at 3.91. Wisconsin Teacher Standard 1: Know the subjects teaching Highest mean 9: Reflection 2nd highest mean Calendar Year 2008 3.71 3.69 Calendar Year 2009 Wisconsin Teacher Standard 1: Know subjects teaching Tied - Highest mean 9: Reflection Tied - Highest mean 7: Able to plan different kinds of lessons Tied – Highest mean 3.80 3.80 Wisconsin Teacher Standard 4: Know how to teach Highest Mean 2: Know how children grow 2nd Highest Mean Calendar Year 2010 3.84 3.78 3.80 It is interesting to see that different standards emerge as being the highest each year. Reflection continues to stay among the top three highest rated components. The trend overtime shows in incline in the mean scores of student teachers across all standards. Trends are also looked at for the lowest rated standards. These have considerably changed over the past year. However Classroom Management continues to be rated below eight other standards. The lowest teacher standard means for 2008-2010 are as follows: Wisconsin Teacher Standard 5: Classroom management lowest mean 2: Know how Children Grow 2nd lowest mean 4: Know how to Teach 2nd lowest mean 6: Communicate Well 2nd lowest mean Calendar Year 2008 3.54 3.61 Wisconsin Teacher Standard 10: Connected with other teachers Lowest mean 8: Know to test for student progress 2nd lowest mean Calendar Year 2009 3.64 3.65 Wisconsin Teacher Standard 5: Classroom management lowest mean 10: Connected with other teachers 2nd lowest mean 3.61 3.61 22 Calendar Year 2010 3.65 3.70 Educational Benchmarking Inventory (EBI) for Exiting Student Teachers (Ratings are on a 7 point scale with “1” indicating either strong disagreement or being very dissatisfied and “7” indicating either strong agreement or being very satisfied) The Educational Benchmarking Inventory (EBI) of exiting student teachers is administered via computer at the end of student teaching for the purpose of unit assessment. Of the 149 student teachers attempted to survey, 80 surveys were returned. This is an overall response rate of 54% for 2008/09 which was down from 65% in 2007/8. In 2009/10, the SOE will send these surveys closer to the actual end date of the student teaching placements in an attempt to raise our response rate. We will also make sure to include our summer placements as these candidates were not asked to participate in the survey in 2008/9. A longitudinal analysis was performed to compare the UW-Stout EBI results on the 14 factors from year. Differences in factor mean scores between 2008/9 and 2009/10 are noted in the table below and are sorted from highest difference to lowest difference. EBI Factor Means Highest Difference to Lowest Difference (Rating Scale “1” disagreement/very dissatisfied –“7” strong agreement/ very satisfied) EBI Factor Factor 2: Learning Theories, Teaching Pedagogy/Techniques Factor 11: Fellow Students in Program Factor 7: Assessment of Student Learning Factor 3: Research Methods, Professional Development, Societal Implications Factor 5: Classroom Equity and Diversity Factor 13: Career Services Factor 4: Aspects of Student Development Factor 6: Management of Educational Constituencies Factor 1: Quality of Instruction Factor 14: Overall Program Effectiveness Factor 10: Support Services Factor 9: Administration Services Factor 8: Satisfaction with Faculty and Course Factor 12: Student Teaching Experiences 2006/7 2007/8 2008/9 2009/10 2010/11 or from 2009/10 to 2010/11* 4.90 5.04 5.00 4.74 5.22 5.41 5.06 5.43 5.23 5.35 5.29 5.54 5.12 5.91 5.48 4.36 4.65 4.51 4.38 4.74 4.70 4.23 5.05 4.24 4.65 4.51 5.27 4.74 5.36 5.78 4.93 4.25 5.18 4.19 4.93 4.80 5.54 5.11 5.50 5.69 4.97 4.06 5.16 4.20 4.90 4.48 5.29 4.89 5.44 5.58 4.81 3.77 5.02 4.11 4.83 4.41 5.52 5.15 5.58 5.89 5.15 4.11 5.34 4.40 5.09 4.63 5.74 5.36 5.71 5.82 *The up or down arrow only indicates direction of mean score differences compared to the previous year and does not indicate if the differences are statistically or meaningfully different. 23 Trend Analysis of All EBI Factors (Rating Scale “1” disagreement/very dissatisfied –“7” strong agreement/ very satisfied) The above graph indicates SOE teacher education candidates continue to rate their Fellow students in the Program and Student Teaching Experiences most highly. Career Services and Management with Educational Contingencies although increasing, are rated lowest out of all EBI categories over time. While career services is rate low, we still have a very high employment rate. EBI SOE Specific Questions Related to Wisconsin Teacher Standards (Rating Scale “1” disagreement/very dissatisfied to “7” strong agreement/ very satisfied) Each EBI participating institution can add ten questions to the EBI Teacher Education Exit Assessment which are institution specific. UW-Stout adds 10 questions closely related to the Wisconsin Teaching Standards. Those results are provided in the table below. Our questions were updated in the 2009/10 school year. EBI - Institution Specific Questions Mean Data; Scale (1-Not at all, 4-Moderately, 7-Extremely) SOE 09/10 10/11 N=127 N=87 To what degree were you prepared to create meaningful learning experiences for students based on your content knowledge? To what degree were you prepared to provide instruction that fosters student learning and intellectual, social and personal development? To what degree were you prepared to create instructional experiences adapted for students who learn differently? To what degree were you prepared to use a variety of learning strategies including the use of technology to encourage critical thinking and problem solving? To what degree were you prepared to manage classroom behavior and create a learning environment that encourages positive social interaction, active engagement in learning and self-motivation? To what degree were you prepared to use instructional technology and media to foster active inquiry, collaboration and interaction in the classroom? To what degree were you prepared to plan instruction based on knowledge of subject matter, students, the community and curriculum goals? To what degree were you prepared to use formal and informal assessment strategies to evaluate student progress? To what degree were you prepared to reflect on teaching and evaluate the effects of choices and actions on pupils, parents and others? To what degree were you prepared to foster relationships with colleges, families and the community to support student learning and well-being? 5.42 5.48 5.29 5.37 5.21 5.48 5.32 5.51 4.91 5.08 5.05 5.21 5.35 5.43 5.14 5.14 5.60 5.47 5.16 5.38 School of Education Undergraduate Follow-Up Survey Results Rating Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, 4 = Strongly Agree Questions HOW DID STOUT CONTRIBUTE TO: WRITING EFFECTIVELY SPEAK/PRESENT IDEAS LISTENING EFFECTIVELY UTILIZATION OF TECHNOLOGIES USING ANALYTIC REASONING CREATIVE PROBLEM SOLVING CRITICALLY ANALYZING INFO MAINTAINING SENSE OF PHYSICAL WELL BEING APPRECIATING AND UNDERSTANDING DIVERSITY DEVELOPING A GLOBAL PERSPECTIVE APPRECIATING THE VALUE OF LITERATURE AND THE ARTS APPRECIATING NATURAL OR PHYSICAL SCIENCES APPRECIATING SOCIAL, ECONOMIC AND POLITICAL FORCES APPRECIATING HISTORY IN CONTEXT TO CURRENT ISSUES ORGANIZING INFO MAKING DECISIONS MAKING DECISIONS ETHICALLY WORKING IN TEAMS LEADERSHIP THINKING CREATIVELY MAINTAINING A SENSE OF MENTAL WELL BEING RATE ASPECTS OF EDUCATION: GENERAL EDUCATION INFORMATION PROGRAM INSTRUCTION AVAILABILITY OF FACULTY IN GEN ED COURSES AVAILABILITY OF FACULTY IN PROGRAM COURSES COURSE AVAILABILITY ACADEMIC ADVISING LAB FACILITIES AND EQUIPMENT DIGITAL ENVIRONMENT How would you rate the overall effectivess of your program/major? How valuable/senior coursework: Promoting meaningful connections Preparation for community, civic and political roles Financial mgmt Continuing education Finding employment How well did the activities prepare you: your classes Experiential learning experience Co-curricular and extra-curricular acitivities Current job title Employer/company name Is your employment directly related to your UW-Stout major? What is your annual full-time salary? Graduated in 2002 Graduated in 2006 3.55 3.93 3.61 3.73 3.5 3.88 3.59 3.3 3.52 3.18 3.23 3.16 3.13 3.14 3.93 3.79 3.68 4.25 4.02 4.02 3.66 3.51 3.75 3.69 4.04 3.59 3.32 3.86 3.5 3.82 4.02 3.6 2.82 3.43 3.74 3.49 3.95 3.57 3.31 3.76 3.49 3.86 3.61 3.81 3.55 3.15 3.74 3.47 3.31 3.17 3.24 3.17 3.9 3.76 3.73 4.07 3.98 4.03 3.47 3.22 3.63 3.29 3.75 3.34 3.07 3.53 3.6 3.83 4.11 3.45 2.91 3.24 3.09 3.51 3.85 3.45 4.31 $41,394.63 4.47 $31,726.79 26 Are you employed full or part time? May we ask your employer to participate in our employer survey? How did your education at Stout compare to education of people hired from other colleges? If unemployed, please indicate current status: student active military service full-time homemaker unemployed and seeking a job unemployed and not seeking a job Other Other blank How would you rate the value of your education? How would you rate your overall Stout experience in the development of interpersonal skills? If you had to do it over again: Would you attend Stout? Would you enroll in the same program 1.07 1.51 3.79 1.04 1.71 3.65 0.02 0 0.04 0.02 0 0.02 0 0 0.02 0.08 0 0.03 3.84 4.11 3.41 3.72 4.39 3.6 4.19 3.77 27 Communicating Assessment Data with Constituencies Data will be communicated to faculty members and other stakeholders through informal and formal means. The SOE Council and SOE Advisory Board are designed to guide and support continuous improvement at the unit level. These stakeholders meet on a regular basis for the purpose of improving the unit and its programs, reviewing policies and procedures to make recommendations to the SOE Director related to revisions, updates, and to meet the ever changing needs of the community, PK-12 schools, and technical colleges. In addition, the Assessment in the Major findings are shared with other stakeholders including technical content instructors and Wisconsin’s Department of Public Instruction (DPI). Utilization of Assessment Data to Improve Courses and the Unit • Share a copy of the assessment report with the Dean, Provost, SOE Advisory Board, SOE Council, SOE faculty and staff, and the DPI liaison during the annual liaison visit. • Continue to monitor candidate pass rates on the PPST and PRAXIS to determine the need for curricular or programmatic changes. • Create a system for tracking student use of the PAXIS Tutors and their success rate after receiving services. Integrate this data into the annual report. • Create a RTI task force to make meaningful recommendations for professional development for Teacher Education faculty and to guide curricular revisions in the areas of classroom management, inclusion and professional education courses within each major. • Work with UW-Stout’s Career Services office to improve services for candidates in the School of Education. • Integrate enrollment data into the assessment report so that they can be used to inform marketing needs. • Create a Integrated Marketing Task Force with the charge to create a SOE marketing plan that will help guide marketing initiatives for the school and its programs. • Create a Technology Task force with a charge to create a SOE Technology Plan to help guide curricular and resource investments. • Create a system for the regular assessment of the assessment system so that it can be responsive to impending changes in teacher education such as the implementation of the Teacher Performance Assessment that is being mandated in Wisconsin. 28