B.S. in Science Education Assessment in the Major Report By Dr. Kevin Mason, Program Director 2014-15 Submitted: October 2015 Table of Contents Introduction ..............................................................................................................................................................................................................2 Overview ..................................................................................................................................................................................................................2 Previous Goals and Evidence...................................................................................................................................................................................4 PRAXIS I: Pre-Professional Skills Test .................................................................................................................................................................6 PRAXIS II: General Science Content Knowledge Exam Summary .......................................................................................................................9 PRAXIS I and II First Time Test Takers ...............................................................................................................................................................12 Benchmark Ratings ................................................................................................................................................................................................14 Student Teaching Performance Ratings .................................................................................................................................................................20 edTPA ....................................................................................................................................................................................................................22 Educational Benchmarking Inventory (EBI) .........................................................................................................................................................23 Alumni Follow-up Survey .....................................................................................................................................................................................23 Communicating Assessment Data with Constituencies .........................................................................................................................................23 Utilization of Assessment Data to Improve Courses and the Program ..................................................................................................................23 Goals for 2015-16 ..................................................................................................................................................................................................25 Science Education AIM Report 2014-15 Page 1 Introduction The B.S. in Science Education program officially began in the fall of 2009. Prior to the B.S. in Science Education, the science education program existed as a concentration within the BS in Applied Science program. The Wisconsin Department of Public Instruction (DPI) approved the first science certification, in broad field science, at UW-Stout in 2004. The first course in science education was taught in the spring of 2006. In the School of Education, data is gathered from several sources to inform unit and program decisions. Data in this report is used to develop program goals, inform curriculum changes, and enhance course delivery in order to improve teacher candidate learning. This report contains data from the PRAXIS I: Pre-Professional Skills Test, PRAXIS II: Content Test, Student Artifact Reflections, Candidate Dispositions, Teacher Candidate Performances, Benchmark Interviews, and the Educational Benchmarking Inventory (EBI). This report also describes how assessment data is used to set programmatic goals and improve the program, program curriculum, and delivery of courses. Overview The enrollment in the B.S. in Science Education during the 2014-2015 academic year was 13 candidates. This included nine pre-science education majors and four science education majors. Historically, the enrollment of the B.S. in Science Education grew dramatically in its first two years, from 16 candidates in 2009-2010 to 30 candidates during the 2010-2011 academic years. Since that time, the enrollment numbers have steadily declined to 27 candidates in 2011-2012, 23 in 2012-2013,18 in 2013-2014, and now 13 in 2014-2015. This follows a state and national trend of decreasing enrollments in teacher education programs. In collaboration with the marketing office at the UW-Stout, SOE has developed a marketing strategy for the next three years to address these enrollment declines. In regard to gender, the balance of men and women enrolled in the program continues to be good, especially compared to other programs in the science, technology, engineering, or mathematics (STEM) fields. In 2014-2015, there were five female candidates and eight male candidates. In 2013-2014, there were 9 male and 9 female candidates enrolled. In previous years, the female enrollment was slightly higher than the male enrollment. In 2012-2013, there were 11 male and 12 female candidates enrolled. In 2011-2012, there were 12 male and 15 female candidates enrolled. In 2010-2011, there were 12 male and 18 female candidates. In 2009-2010, there were seven male and nine female candidates enrolled. In 2014-2015, there were no minority candidates enrolled in the program. This is also similar to previous years. There were no minority candidates in 2013-2014, one in 2012-2013, one in 2011-2012, none in 2010-2011, and none in 2009-2010. To remedy this issue, the School of Education supports a multicultural recruitment and retention coordinator to recruit underrepresented groups into the education programs at UW-Stout, including minorities and women in science, technology, engineering, and mathematics (STEM) education. Three candidates graduated from the program in 2014-2015. Five candidates graduated from the program in 2013-2014, six candidates in 20122013, six candidates in 2011-2012, four candidates in 2010-2011, and one candidate in 2009-2010. The program admitted three freshmen in 2014-2015. Four freshmen were admitted in 2013-2014, one in 2012-2013, two in 2011-2012, five in 20102011, and four in 2009-2010. It should be noted that the number of freshmen compose only a portion of the new candidates admitted to the program Science Education AIM Report 2014-15 Page 2 each year. Some of the new candidates enter the B.S. in Science Education as change of majors, transfer students, or students already holding a bachelor’s degree. In 2014-2015, there were seven new science education candidates enrolled in the Introduction to Math and Science Education (STMED 101) course. Six new science education candidates were enrolled in STMED 101 in 2013-2014, 10 in 2012-2013, 10 in 2011-2012, and 15 in 2010-2011. The School of Education and the science education program have engaged in multiple recruiting efforts during the past several years in an attempt to offset the declining enrollments in education across the state and nation. First, the program collaborates with the university admissions office as they engage in multiple recruiting efforts at local events to market the science education program. In fact, faculty members in the School of Education have attended several of these events, including a STEM night at a local high school and a booth at the Northern Wisconsin State Fair. Second, the School of Education supports a multicultural recruitment and retention coordinator to recruit pre-service teachers to critical shortage areas in education, such as science, technology, mathematics, and special education. Third, the faculty members in the School of Education have conducted mailings to secondary teachers and guidance counselors to promote the program. Fourth, the faculty members in the School of Education visit schools on a regular basis for the supervision of teacher candidates. This provides an opportunity to talk with teachers and bring marketing materials, such as posters or safety glasses with the Stout logo, to the schools and teachers. Fifth, faculty members in the School of Education have engaged in grant writing and professional development summer academies with in-service teachers in the state of Wisconsin to increase the awareness and reputation of the program locally. Sixth, the faculty members have also published and presented at multiple state and national conferences to increase the awareness and reputation of the program nationally. At each conference, marketing materials such as banners, posters, brochures, and pens are displayed or distributed. Finally, the Program Director of the B.S. in Science Education received Hobson’s training in the fall of 2014 to develop automated responses to students who express an interest in the science education program. Science Education AIM Report 2014-15 Page 3 Previous Goals and Evidence 2014-2015 Program Goals Ongoing Goals 1 Refer candidates to the CEHHS writing guidelines and the writing center in science, technology, and math education (STMED) courses for support and feedback on their writing. 2 Refer candidates to the PPST tutor and enroll in the elective course, EDUC 010 Praxis I: Writing, as needed. 3 4 5 6 7 8 9 Refer candidates to the PRAXIS II exam preparation books located in the Library Learning Center. Monitor the PRAXIS II exam scores by category and identify long term trends in candidate performance. Emphasize philosophies of education and the theories and practices of classroom management across the curriculum, in a variety of different courses and contexts. Continue to compile teacher candidate evaluation data to obtain larger sample sizes. Educate candidates in the introduction to math and science education (STMED 101) course about the e-portfolio artifacts and help lab to improve candidates’ ability to use an e-portfolio to demonstrate their knowledge and skills. Embed learning activities and assignments in science, technology, engineering, and math education course to address all three categories of the instructional technology utilization rubric, including administrative software, presentation software, and interactive whiteboards. To prepare teacher candidates for the successful completion of the Educative Teacher Performance Assessment (edTPA). Science Education AIM Report 2014-15 Evidence Towards Meeting Goals CEHHS guidelines have been utilized and made available to candidates enrolled in STMED 101, 185, 360, 390, and 401. Dang Yang was a guest speaker in the Introduction to Math and Science Education (STMED 101) course in the fall of 2014. This has occurred in group and individual advisement meetings. This continues to be done in the Assessment in the Major report. The philosophy of education artifact in Foundations of Education (EDUC 326) has been revised. A new Embedded Signature Assessment (ESA) has been developed to include theories of classroom management and will be piloted in fall of 2015. This continues to be done in the Assessment in the Major report. Frode Larsen was a guest speaker in the Introduction to Math and Science Education (STMED 101) course in the fall of 2014. These instructional technologies were embedded in the Lab and Classroom Management (STMED 390) course in the fall of 2012, 2013, and 2014. The faculty of the School of Education have developed seven Embedded Signature Assessments (ESA) that are aligned to the edTPA Rubrics and INTASC standards. Several ESA’s were piloted in the spring of 2015 and all seven will be implemented in courses in fall of 2015. Page 4 Science Education AIM Report 2014-15 Page 5 PRAXIS I: Pre-Professional Skills Test All education majors must pass all three sections of the PRAXIS I: Core Academic Skills for Educators (CORE). The three sections consist of reading, writing and mathematics. Prior to the fall 2013 semester, candidates would take the Pre-Professional Skills Test (PPST) which was available until the end of the 2013-14 academic year. Also, candidates can meet their Benchmark I requirements through scores on their college entrance exam. Note that the pass rates in the table reflect attempts by all candidates prior to being formally accepted into the School of Education, all candidates are required to pass the CORE as part of Benchmark I, therefore the pass rate of candidates accepted is 100%. PPST Attempts and Pass Rates Teacher Education Program SCIED SOE 2010 PPST Test Math Math Exemption Writing Writing Exemption Reading Reading Exemption Math Math Exemption Writing Writing Exemption Reading Reading Exemption 2011-12 2012-13 2013-14 # test attempts 7 NA 9 # (and %) passed 7 (100%) NA 7 (78%) # test attempts 5 NA 7 # (and %) passed 5 (100%) NA 4 (57%) # test attempts 4 NA 7 # (and %) passed 4 (100%) NA 4 (57%) # test attempts 2 NA 9 # (and %) passed 1 (50%) NA 2 (22%) NA NA NA NA NA NA NA NA 7 7 (100%) 6 5 (83%) 6 4 (67%) 2 2 (100%) NA NA NA NA NA NA NA NA 198 NA 213 150 (76%) NA 141 (66%) 210 NA 287 142 (68%) NA 138 (48%) 114 1 175 86 (75%) NA 95 (54%) 70 NA 104 53 (76%) NA 46 (44%) NA NA NA NA 1 NA 1 NA 243 138 (57%) 240 135 (56%) 157 86 (55%) 109 48 (44%) NA NA NA NA 2 NA 3 NA Beginning in 2013-2014, candidates who received a minimum composite score of 23 on the American College Testing (ACT) exam with a minimum score of 20 in math, reading, and English are exempted from taking the PPST. As a result, the number of candidates attempting the PPST decreased in 2013-2014 to only two candidates. Furthermore, because the candidates who were more successful on the ACT are removed from the test taking sample, the pass rates have decreased significantly in 2013-2014. In 2013-2014, the pass rate was 50% on the math section (1 out of 2 attempts), 22% on the writing section (2 out of 9 attempts), and 100% on the reading section (2 out of 2 attempts). All of these attempts were made by two candidates. Because of the small sample size, this data is not a good representation of the performance or abilities of all science education candidates. Rather, it may be more meaningful to examine the trends over past five years. Science Education AIM Report 2014-15 Page 6 Examining the trends over multiple years, the mean scores for science education candidates have generally exceeded the average scores for the School of Education in all three categories. In 2010-2011, 100% of candidates passed the math section (7out of 7 candidates), 78% passed the writing section (7 out of 9 candidates), and 100% passed the reading section (7 out of 7 candidates). In 2011-2012, 100% of candidates passed the math section (5 out of 5 candidates), 57% passed the writing section (4 out of 7 candidates), and 83% passed the reading section (5 out of 6 candidates). In 2012-2013, 100% of candidates passed the math section (4 out of 4 candidates), 57% passed the writing section (4 out of 7 candidates), and 67% passed the reading section (4 out of 6 candidates). The data shows that both science education candidates and School of Education candidates, in general, are weakest in the area of writing. The sample sizes are typically larger for the writing section because it includes candidates who had previously failed and then attempt to re-take the writing section. The College of Education, Health, and Human Development has developed a set of writing guidelines to improve and foster writing development for all students in the college. In addition, the School of Education offers an elective course, EDUC 010 Praxis I: Writing, to prepare education candidates for the writing section of the Pre-Professional Skills Test. Improving the writing of candidates in the B.S. in Science Education program is an ongoing goal of the program. In 2013-2014, the Core exam was offered as an alternative to the PPST as the assessment of basic skills in reading, writing, and mathematics. However, there were no science education candidates who attempted the Core exam in 2013-2014. In 2014-2015, the Core exam replaced the PPST for pre-service teachers in the state of Wisconsin who are not exempted by their ACT scores. In 2014-2015, one science education candidate attempted the CORE exam and passed the math, reading, and writing sections. Science Education AIM Report 2014-15 Page 7 CORE Attempts and Pass Rates Teacher Education Program SCIED SOE 2013-14 CORE Test CORE Math Math Exemption CORE Writing Writing Exemption CORE Reading Reading Exemption CORE Math Math Exemption CORE Writing Writing Exemption CORE Reading Reading Exemption 2014-15 # test attempts NA # (and %) passed NA # test attempts 1 # (and %) passed 1 (100%) NA NA NA NA NA NA 1 1 (100%) NA NA NA NA NA NA 1 1 (100%) NA NA NA NA 15 6 (46%) 122 67 (55%) NA NA 1 NA 13 7 (54%) 136 56 (41%) 1 NA 1 NA 14 10 (71%) 113 75 (66%) 3 NA NA NA Science Education AIM Report 2014-15 Page 8 PRAXIS II: General Science Content Knowledge Exam Summary Science education candidates are required to pass the Praxis II General Science Content Knowledge Exam prior to student teaching. It should be noted that all candidates are required to pass the Praxis II to be admitted to student teaching as part of Benchmark II so the pass rate is 100% upon Benchmark II approval. In 2014-2015, three out of six candidates passed the PRAXIS II General Science Content Knowledge Exam, resulting in a 50% pass rate. Like the PRAXIS I exam, the candidates who did not pass the PRAXIS II exam are likely to retake the exam until they receive a passing score. Although some candidates have taken multiple attempts, 100% of B.S. in Science Education candidates have passed the PRAXIS II exam to become eligible for student teaching since the program began. Over the past five years, the science education candidates have performed very well on the Praxis II General Science Content Knowledge Exam, often exceeding national averages in many categories. In 2008-2009, five out of five candidates passed the Praxis II exam, resulting in a 100% pass rate. In 2009-2010, six out of seven candidates passed the Praxis II exam, resulting in an 86% pass rate. In 2010-2011, ten out of eleven candidates passed the Praxis II exam, resulting in a 91% pass rate. In 2011-2012, 5 out of 7 candidates passed the Praxis II exam, resulting in a 71% pass rate. In 2012-2013, 5 out of 10 candidates passed the Praxis II exam, resulting in a 50% pass rate. In 2013-2014, four out of six candidates passed the PRAXIS II exam, resulting in a 67% pass rate. Due to the small number of candidates taking the exam in any given year, the pass rates have varied from year to year. Science Education Praxis Test Code – 10435 Content Test from ETS 08/09 09/10 (0435) 7 5 Number of Examinees: 187 197 Highest Observed Score: 142 161 Lowest Observed Score: 154 154 WI Score Needed to Pass: 10/11 11/12 12/13 13/14 14/15 11 200 146 154 7 177 144 154 10 177 144 154 6 200 139 154 6 170 141 154 Number with WI Passing Score: 5/5 6/7 10/11 5/7 5/10 4/6 3/6 Percent with WI Passing Score: 100% 86% 91% 71% 50% 67% 50% Science Education AIM Report 2014-15 Page 9 In 2014-2015, the candidates scored highest in the areas of Scientific Principles, Physical Science, and Science, Technology, and Society. The mean score in the Physical Science section exceeded the national averages. The lowest scores occurred in the areas of Life Science and Earth and Space Science. Due to the small sample sizes, the scores on each section vary from year to year. Percent Correct (percentage of items answered correctly) by Area UW-Stout Points SCIED Test Category Available 08/09 09/10 10/11 11/12 12/13 Scientific Principles 12 82 82 73 66 63 Physical Science 48 74 63 67 61 57 Life Science 23-24 72 75 74 66 69 Earth and Space Science 24 72 66 67 57 60 Science Technology & Society 12 80 75 83 77 59 13/14 82 70 65 61 14/15 67 68 60 60 National 14/15 70 66 74 67 76 71 72 The trends over the past five years indicate that candidates generally score the highest on Scientific Principles and Science, Technology, and Society The section with the lowest scores has varied from one year to the next, indicating that the program provides a good balance of curriculum and instruction in all disciplines of science. Typically, the candidate’s test scores are strongest in the science subject areas in which they will be certified to teach and weakest in the areas they will not be certified to teach, which would be expected. These trends will continue to be monitored each year to identify long-term trends. Science Education AIM Report 2014-15 Page 10 Science Education AIM Report 2014-15 Page 11 PRAXIS I and II First Time Test Takers Beginning in 2013-14, all teacher preparation programs in the state of Wisconsin are required to report on “the passage rate on the first attempt of students and graduates of the program on examinations administered for licensure” (Act 166, section 17). In 2014-2015, the science education program had one first-time test taker attempt the CORE exam. This candidate passed the math, writing, and reading section, resulting in a 100% pass rate for all three sections. In 2013-2014, the science education program had two first time test takers attempt the PPST exam. One of two candidates passed the math section (50%), one of two candidates passed the writing section (50%), and two of two candidates passed the reading section (100%). Teacher Education Program SCIED SOE Teacher Education Program SCIED SOE Spring 2013 PPST Test Math Writing Reading Math Writing Reading # first time test takers 2 2 2 63 65 63 2013-14 # (and %) passed 2 (100%) 2 (100%) 1 (50%) 52 (83%) 40 (62%) 40 (63%) # first time test takers 2 2 2 56 53 56 2013-14 CORE Test CORE Math CORE Writing CORE Reading CORE Math CORE Writing CORE Reading # first time test takers NA NA NA 13 11 13 # (and %) passed NA NA NA 6 (46%) 5 (45%) 9 (69%) # (and %) passed 1 (50%) 1 (50%) 2 (100%) 48 (86%) 25 (47%) 28 (50%) 2014-15 # first time test takers 1 1 1 87 92 97 # (and %) passed 1 (100%) 1 (100%) 1 (100%) 56 (64%) 41 (45%) 68 (70%) In 2014-2015, the science education program had two first time test takers attempt the PRAXIS II General Science Content Knowledge Exam. One of two candidates passed, resulting in a 50% pass rate. In 2013-2014, three of five first time test takers passed the PRAXIS II exam, resulting in a 60% pass rate. In the spring of 2013, four of six first time test takers passed the PRAXIS II exam, resulting in a 67% pass rate. This data will continue to be monitored each year to identify trends that might be more useful and reliable for informing programmatic decisions. Science Education AIM Report 2014-15 Page 12 Spring 2013 SCIED Content Test 2013-14 2014-15 # first time test takers # (and %) passed # first time test takers # (and %) passed # first time test takers # (and %) passed 6 4 (67%) 5 3 (60%) 2 1 (50%) Science Education AIM Report 2014-15 Page 13 Benchmark Ratings Benchmark ratings are reported by benchmark level (I, II, and III) for each requirement or question. On Benchmark I, 100% of science education candidates passed the benchmark I interview by passing all seven interview questions from 2008 to 2011. Beginning in the spring of 2012, the benchmark I process was changed to eliminate the interview. Now, candidates in the School of Education submit an application for review, in order to gain admission into the School of Education. The application is first reviewed by staff in the School of Education office. If the application is complete, the Benchmark I materials are cleared for review by the program director in each program area. Beginning Spring 2015, further adjustments were made to the Benchmark I rubric. In 2014-2015, five of eight science education candidates were cleared for Benchmark I review. Two candidates were denied due to failing scores on the PPST or CORE exams. One candidate was denied for not having completed enough credits to be eligible. Similarly, in 2013-2014, one of the two candidates who applied was cleared for review. One candidate was denied because they did not have a passing PPST or Core exam score. In 2012-2013, seven of the ten candidates who applied were cleared for review by the program director in science education. The three candidates with incomplete applications were denied for insufficient credits, failing PPST scores, and an incomplete e-portfolio, respectively. Benchmark I Applications Benchmark I Applications Cleared for Benchmark I Review Cleared for Benchmark I Review based on score from college entrance test Denied: No passing PPST/CORE score Denied: Low GPA Denied: Insufficient credits/course work Denied: Did not receive “C” or higher in English, Speech, Intro, or Foundation of Education courses Denied: Missing background check Denied: Other Reasons 201213 10 7 N/A 1 N/A 2 SCIED 201314 2 1 N/A 1 N/A N/A 201415 8 5 N/A 2 N/A 1 SOE 201415 156 103 29 27 5 28 N/A N/A N/A 2 N/A 1 N/A N/A N/A N/A 8 N/A *Individual candidates who apply multiple times per academic year are counted twice or more. There might also be multiple reasons for candidates not clearing for Benchmark I review. Some candidates might also have been cleared for BM I review based on PPST/CORE scores, even though they would have been cleared based on scores from their college entrance test. Being cleared for Benchmark I review does not necessarily mean that a candidate went through with it. Science Education AIM Report 2014-15 Page 14 After being cleared for review, the candidates’ Benchmark I application materials are then forwarded to their program director. In the fall of 2014, four of four science education candidates passed the Benchmark I review, resulting in a 100% pass rate. Using a new rubric in the spring of 2015, one of one science education candidates passed the Benchmark I review, resulting in a 100% pass rate. Since the spring of 2012, the science education candidates have had a 100% pass rate on the Benchmark I review. Benchmark I Rubric Results (new rubric) ePortfolio Review Rubric Foundations of Education (EDUC-326) Final Project or Program Equivalent Response Deficiency Satisfactory Deficiency Satisfactory Deficiency Satisfactory Dispositional Review Rubric Response Commitment to Learning: The candidate will demonstrate a commitment to his/her own and his/her students’ continuous learning Respect for Oneself and Others: The candidate will demonstrate respect for himself/herself and others through thoughtful and responsive communication, showing respect and collaboration Commitment to Excellence: The candidate recognizes his/her professional responsibility for engaging in and supporting appropriate professional practices for self and others Deficiency Signed Statement of Values and Dispositions Resume Science Education AIM Report 2014-15 Satisfactory Deficiency Satisfactory Deficiency Satisfactory SCIED Spring 2015 N=1 0% 100% 0% 100% % 100% SOE Spring 2015 N=40 0% 100% 0% 100% 2% 98% 0% 2% 100% 98% 0% 2% 100% 98% 0% 2% 100% 98% Page 15 Benchmark I Review Results (old rubric) Artifact Name Signed Statement of Values and Dispositions Resume Philosophy Statement Two (2) artifacts related to the subject matter the candidate will teach Disposition Area Commitment to Learning: The candidate will demonstrate a commitment to their own and their students continuous learning. Respect for Oneself and Others: The candidate will demonstrate respect for others through thoughtful and responsive communication, showing respect and collaboration. Commitment to Excellence: The candidate recognizes his/her professional responsibility for engaging in and supporting appropriate professional practices for self and others. SCIED 2012-13 2013-14 SOE Fall 2014 N=39 0% 100% 3% 97% 0% 100% 3% 97% N=5 0% 100% 0% 100% 0% 100% 0% 100% N=2 0% 100% 0% 100% 0% 100% 0% 100% Fall 2014 N=4 0% 100% 0% 100% 0% 100% 0% 100% 0% 0% 0% 0% 0% 100% 100% 100% 100% 100% 0% 0% 0% 0% 0% No Deficiency Deficiency 100% 0% 100% 0% 100% 0% 100% 0% 100% 0% No Deficiency 100% 100% 100% 100% 100% Response Incomplete Complete Incomplete Complete Incomplete Complete Incomplete Complete Spring 2012 N=3 0% 100% 0% 100% 0% 100% 0% 100% Response Deficiency No Deficiency Deficiency The science education candidates have experienced similar success on the Benchmark II interview. Since 2008, 100% of science education candidates have passed the Benchmark II interview. In 2014-2015, three of three science education candidates passed the Benchmark II interview. The candidates received the highest scores on their philosophy of education, instructional technology utilization, and demonstrating the ability to teach effectively. The candidates received their lowest scores in describing what it means to be a reflective practitioner and demonstrating the ability to create instructional opportunities adapted to diverse learners. Science Education AIM Report 2014-15 Page 16 From 2008-2015, the highest scores have typically been on describing a reflective practitioner, instructional technology utilization, demonstrating content knowledge, and demonstrating the ability to teach effectively. Conversely, the candidates have traditionally scored lowest on describing their philosophy of education. The philosophy of education will continue to be an area of emphasis in the program. Since 2010, it has been recommended that all instructors in science, technology, and math education courses discuss and reinforce philosophies of education as it relates to the content of the course. The philosophy of education portfolio artifact produced in Foundations of Education (EDUC 326) was also revised in the spring of 2015. Benchmark II Interview Results Question Describe your Philosophy of Education and how it has evolved. Describe what it means to be a "Reflective Practitioner." Describe the WI Teacher Standard and Domain you feel most competent in. Describe the WI Teacher Standard and Domain you have experienced the greatest growth. Provide Portfolio evidence (signed copy of the Instructional Technology Utilization rubric) of your competence in current instructional technology. Science Education AIM Report 2014-15 Response Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a 2008 2009 2010 N=3 0% 33% 67% 0% 0% 0% 100% 0% 0% 0% 100% 0% 0% 33% 67% 0% NA NA NA NA N=3 0% 33% 67% 0% 0% 0% 100% 0% 0% 0% 100% 0% 0% 0% 100% 0% NA NA NA NA N=6 0% 33% 67% 0% 0% 0% 100% 0% 0% 17% 83% 0% 0% 17% 83% 0% 0% 17% 83% 0% SCIED 2011- 20122012 2013 N=5 0% 80% 20% 0% 0% 60% 40% 0% 0% 20% 80% 0% 0% 60% 40% 0% 0% 20% 80% 0% N=6 0% 33% 67% 0% 0% 0% 100% 0% 0% 33% 67% 0% 0% 50% 50% 0% 0% 0% 100% 0% 20132014 N=6 0% 36% 64% 0% 0% 18% 82% 0% 0% 55% 45% 0% 0% 18% 82% 0% 0% 0% 100% 0% 20142015 SOE 20142015 N=3 N=103 0% 3% 0% 40% 100% 57% 0% 0% 0% 2% 100% 33% 0% 65% 0% 0% 0% 1% 0% 27% 100% 72% 0% 0% 0% 3% 33% 34% 67% 63% 0% 0% 0% 0% 0% 29% 100% 71% 0% 0% Page 17 Reviewers choose two of the following; discuss portfolio evidence that: Demonstrates your content knowledge. Demonstrates your ability to create instructional opportunities adapted to diverse learners. Demonstrates your ability to teach effectively. Demonstrates your ability to assess student learning. Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a Unsatisfactory Emerging Basic n/a 0% 33% 67% 0% 0% 0% 67% 33% 0% 0% 0% 100% 0% 0% 0% 100% 0% 0% 33% 67% 0% 0% 0% 100% 0% 0% 33% 67% 0% 0% 33% 67% 0% 0% 0% 100% 0% 17% 83% 0% 0% 0% 100% 0% 0% 20% 80% 0% 0% 0% 100% 0% 0% 33% 67% 0% 0% 0% 100% 0% 0% 0% 100% 0% 0% 0% 100% 0% 0% 25% 75% 0% 0% 0% 100% 0% 0% 50% 50% 0% 0% 0% 100% 0% 0% 100% 0% 0% 0% 0% 100% 0% 0% 36% 64% 0% NA NA NA NA 0% 50% 50% 0% 0% 0% 100% 0% 0% 33% 67% 0% 3% 45% 53% 0% 1% 41% 58% 0% 18% 50% 32% 0% 3% 31% 66% 0% The Benchmark III Interview occurs at the end of the student teaching semester. Science education candidates have had a 100% pass rate on the Benchmark III interview since 2008. In 2014-2015, three of three science education candidates passed the Benchmark III interview. The science education candidates scored highest on the portfolio artifacts from student teaching, the utilization of technology, and the alignment of artifacts to Charlotte Danielson’s four domains and the ten Wisconsin Educator Standards. In these three categories, 100% of science education candidates received proficient ratings, the highest rating possible. The lowest ratings on Benchmark III during the 2013-2014 school year occurred on the final teacher candidate assessment and disposition ratings from the cooperating teacher. On the final teacher candidate assessment, one candidate (33%) received a proficient rating (the highest possible rating) and two candidates (67%) received a basic rating. Likewise, on the disposition rating, one candidate (33%) received a proficient rating (the highest possible rating) and two candidates (67%) received a basic rating. These results are consistent with past years and compare favorably with other candidates in the School of Education on the Benchmark III assessment, as shown in the right-hand column of the table provided. Professional dispositions and values will continue to be a point of emphasis in the introduction to math and science education course and the field experiences. Science Education AIM Report 2014-15 Page 18 Benchmark III Interview Results Question Artifacts from student teaching, reflection ratings. Final Student Teaching Assessments and Recommendations from Cooperating Teachers. Disposition ratings from student teaching from cooperating & University Supervisors. Instructional Technology Utilization Rubric. Alignment Summary of artifacts meeting all 10 Wisconsin Teaching Standards & 4 Domains/ Components & reflections/ reflection ratings. Science Education AIM Report 2014-15 Response Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a Unsatisfactory Emerging Basic Proficient n/a 2008 N=0 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% NA NA NA NA NA 0% 0% 0% 0% 0% 2009 2010 N=3 N=4 0% 0% 0% 0% 0% 25% 100% 75% 0% 0% 0% 0% 0% 0% 0% 0% 100% 100% 0% 0% 0% 0% 0% 0% 0% 0% 0% 100% 100% 0% NA 0% NA 0% NA 75% NA 25% NA 0% 0% 0% 0% 0% 0% 0% 100% 100% 0% 0% SCIED SOE 2011- 2012- 2013- 2014- 20142012 2013 2014 2015 2015 N=7 N=7 N=5 N=3 N=117 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 9% 100% 100% 100% 100% 91% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 14% 20% 0% 1% 0% 14% 0% 67% 24% 100% 71% 80% 33% 75% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 29% 29% 20% 67% 15% 71% 71% 80% 33% 85% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 14% 14% 0% 0% 4% 86% 86% 100% 100% 96% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 100% 100% 100% 100% 99% 0% 0% 0% 0% 0% Page 19 Student Teaching Performance Ratings Teacher candidates are rated by their cooperating teachers during the student teaching capstone experience in the science education program. The following data shows how teacher candidates performed relative to each of the ten Wisconsin Educator Standards and on the use of instructional technology. There were three science education candidates who completed student teaching during the 2014-2015 academic year. Science education candidates met or exceeded the School of Education average on four of eleven items during the 2014-2015 academic year. Student Teacher Evaluations Rating Scale: 1=Unsatisfactory, 2=Emerging, 3=Basic, 4=Proficient SCIED 20112008 2009 2010 2012 N=0 N=3 N=3 N=7 Mean Mean Mean Mean Teachers know the subjects they are teaching. 0.00 3.75 3.67 3.86 Teachers know how children grow. 0.00 3.63 3.93 4.00 Teachers understand that children learn differently. 0.00 3.63 3.67 4.00 Teachers know how to teach. 0.00 3.63 4.00 4.00 Teachers know how to manage a classroom. 0.00 3.75 3.93 3.64 Teachers communicate well. 0.00 3.63 3.93 3.71 Teachers are able to plan different kinds of lessons. 0.00 3.88 3.73 4.00 Teachers know how to test for student progress. 0.00 3.88 4.00 3.79 Teachers are able to evaluate themselves. 0.00 3.88 3.87 3.86 Teachers are connected with other teachers and the community. 0.00 3.38 3.87 3.86 Teachers make effective use of instructional technologies to enhance student learning. NA NA 3.93 4.00 20122013 N=7 Mean 3.71 3.57 3.71 3.86 3.43 3.71 3.71 3.57 3.86 3.71 20132014 N=5 Mean 3.73 3.80 3.80 3.90 3.43 3.80 3.53 3.70 3.80 3.80 20142015 N=3 Mean 3.83 3.50 3.83 3.67 3.50 3.67 3.50 3.83 3.67 3.50 SOE 20142015 N=120 Mean 3.75 3.75 3.75 3.75 3.58 3.77 3.73 3.69 3.82 3.77 3.86 3.70 3.83 3.76 In 2014-2015, the science teacher candidates scored highest in knowing the subjects they are teaching, understanding that children learn differently, knowing how to test for student progress, and the use of instructional technologies to enhance learning. The teacher candidates scored lowest in knowing how children grow, knowing how to manage a classroom, being able to plan different kinds of lessons, and connecting with other teachers and the community. Due to the small sample sizes and individual differences, the strengths and weaknesses of teacher candidates vary each year. Over the past five years, classroom management has been one of the lower rated areas for science education teacher candidates, and all teacher candidates in the School of Education. This continues to be an area of emphasis in the program. Research shows that this is a difficult skill for novice teachers to master. The lab and classroom management course (STMED 390) will continue to address both the theory and practices of Science Education AIM Report 2014-15 Page 20 classroom management. The pre-student teaching and student teaching field experiences provide candidates with opportunities to apply and refine their classroom management skills. In addition, it will be recommended that all instructors in science, technology, and math education (STMED) courses discuss and reinforce concepts of classroom management in courses throughout the program, as it relates to the content of each course. Finally, a new Embedded Signature Assessment (ESA) has been developed by the faculty of the School of Education to measure the candidate’s ability to analyze the classroom environment and apply theories of classroom management. This ESA will be implemented in the Lab and Classroom Management (STMED 390) course beginning in the fall of 2015. Science Education AIM Report 2014-15 Page 21 edTPA The Educative Teacher Performance Assessment or edTPA is an assessment process completed during student teaching. It is designed by educators to answer the essential question: "Is a new teacher ready for the job?" The edTPA includes a review of teaching strategies such as lesson plans, video clips of teaching, and assessment strategies used in teaching. The edTPA will measure the new teacher's ability to effectively teach to all students. Starting in the 2015-16 academic year, all Wisconsin teacher candidates are required to complete the edTPA. Passing the edTPA will be a requirement for a Wisconsin teacher license starting in 2016-17. Science Education AIM Report 2014-15 Page 22 Educational Benchmarking Inventory (EBI) Science education candidates complete the EBI (Scale= 1-7 with 1= not at all, 4= moderately and 7= extremely) at the conclusion of their student teaching experience. The Science Education program did not have enough respondents (>5) to disaggregate any meaningful data from this survey. Alumni Follow-up Survey UW-Stout surveys graduates every two years. The next survey will be sent in 2014 for graduates in 2012 and 2008. The executive summary and full report from the Alumni Follow-Up Study are online at the following site: http://www.uwstout.edu/static/bpa/ir/afu/2010index.html The Science Education program does not yet have sufficient data from alumni follow-up studies conducted by the BPA office. See the full detail of the report in Appendix B. Communicating Assessment Data with Constituencies Data will be communicated to content and core professional education faculty members through informal and formal means. Formally, the data and findings will be shared with members of the science education advisory board, which includes science content faculty, school of education faculty, and local K-12 teachers. In addition, science, technology, and mathematics education (STMED) program faculty will meet during scheduled discipline area work group meetings (DAWG), which are specifically designed to support ongoing program improvement. The agenda and minutes from these meetings will be kept as an artifact of this collaborative effort to improve the program effectiveness. Utilization of Assessment Data to Improve Courses and the Program In conclusion, science education candidates have continued to demonstrate outstanding knowledge and skills based on the assessment data collected from 2006 to 2015. Candidates have historically performed very well on assessments required for science teacher licensure in the state of Wisconsin. These include the PRAXIS I Pre-Professional Skills Test (PPST) or CORE Exam and the PRAXIS II General Science Content Knowledge Exam. The average scores for science education candidates continue to exceed the average scores for the School of Education in most assessment areas. The data shows that both science education candidates and School of Education candidates, in general, are weakest in the area of writing. To address this area, the College of Education, Health, and Human Development has developed a set of writing guidelines to improve and foster writing development for all students in the college. In addition, our faculty and staff have encouraged candidates in each course to take advantage of the writing center on campus to complete their writing assignments and receive feedback and guidance for improving their writing. Finally, the School of Education offers an elective course, EDUC 010 Praxis I: Writing, to prepare education candidates for the writing section of the Pre-Professional Skills Test. Science education candidates have also performed very well on the assessment data collected by the School of Education. From 2008-2014, the data shows that 100% of science education candidates have passed Benchmark I, 100% have passed Benchmark II, and 100% have passed Benchmark III. These results compare favorably with other candidates in the School of Education. Although 2014-2015 was an exception, the long term trend has shown that science education candidates historically scored the lowest in response to the Benchmark II interview question asking them to describe Science Education AIM Report 2014-15 Page 23 their philosophy of education. It was recommended in 2010 that all instructors in science, technology, and math education courses discuss and reinforce philosophies of education as it relates to the content of the course. For the Benchmark III interview, 100% of science education candidates have passed all of the requirements and were recommended for teaching licensure since 2008. The lowest ratings during the 2014-2015 school year occurred in the student teacher evaluation and disposition ratings. Once again, these results are comparable to the performance of other candidates in the School of Education on the Benchmark III assessment. Professional dispositions and values will continue to be a point of emphasis in the introduction to math and science education course and field experience courses. Teacher candidates are rated by their cooperating teachers during the student teaching capstone experience in the science education program. Over the past five years, candidates have scored highest in their ability to teach and lowest in their ability to manage the classroom management. Therefore, classroom management continues to be an area of emphasis in the program. Research shows that this is a difficult skill for novice teachers to master. The lab and classroom management course (STMED 390) will continue to address both the theory and practices of classroom management. In addition, the pre-student teaching and student teaching field experiences provide candidates with opportunities to apply and refine their classroom management skills. Finally, it was recommended two years ago that all instructors in science, technology, and math education courses discuss and reinforce the theory and practice of classroom management, as it relates to the content of each course. The following program goals were established based on the assessment data displayed in this report. Although all of the goals will be retained and ongoing, there is also evidence that progress has been made toward completing these goals. The newest program goal is to prepare teacher candidates for the successful completion of the Educative Teacher Performance Assessment (edTPA). The edTPA will be required during student teaching in the state of Wisconsin beginning in the fall of 2015. It will be consequential for teacher licensure beginning in the fall of 2016. To accomplish this goal, I have participated on the edTPA committee and provided leadership in the School of Education to prepare for the implementation of the edTPA. I have attended state and national conferences and lead professional development workshops for faculty and staff in the School of Education. In the fall of 2014, all teacher candidates in the School of Education will be required to complete the edTPA. From this experience, we hope to learn how to best prepare and support our candidates for success on this national performance evaluation. Science Education AIM Report 2014-15 Page 24 Goals for 2015-16 2015-16 Goals 1 2 3 4 5 6 7 8 9 Refer candidates to the CEHHS writing guidelines and the writing center in science, technology, and math education (STMED) courses for support and feedback on their writing. Refer candidates to the PPST tutor and enroll in the elective course, EDUC 010 Praxis I: Writing, as needed. Refer candidates to the PRAXIS II exam preparation books located in the Library Learning Center. Monitor the PRAXIS II exam scores by category and identify long term trends in candidate performance. Emphasize philosophies of education and the theories and practices of classroom management across the curriculum, in a variety of different courses and contexts. Continue to compile teacher candidate evaluation data to obtain larger sample sizes. Educate candidates in the introduction to math and science education (STMED 101) course about the e-portfolio artifacts and help lab to improve candidates’ ability to use an e-portfolio to demonstrate their knowledge and skills. Embed learning activities and assignments in science, technology, engineering, and math education course to address all three categories of the instructional technology utilization rubric, including administrative software, presentation software, and interactive whiteboards. To prepare teacher candidates for the successful completion of the Educative Teacher Performance Assessment (edTPA). Science Education AIM Report 2014-15 Page 25