Assessment Report College of Education and Human Services Educator Preparation Programs July 1, 2011-June 30, 2012 PROGRAMS ASSESSED Adolescent to Young Adult Education, M.Ed. Integrated Language Arts Integrated Mathematics Integrated Social Studies Integrated Science Career & Technical Education, B.S.Ed., M.Ed. Early Childhood Education, B.S.Ed. Health Education, B.S.Ed. Intervention Specialist, M.Ed. Early Childhood Mild-Moderate Moderate-Intensive Educational Needs Library/Media Education, M.Ed. Middle Childhood Education, M.Ed. Multi-Age Education, M.Ed. Foreign Language (French, Spanish, German) Visual Arts Physical Education, B.S.Ed. Principal, M.Ed. Reading Education, M.Ed. Superintendent, Ed.S. Teacher Leader, M.Ed. ASSESSMENT COORDINATORS James Tomlin, Chair, Department of Teacher Education; Jill Lindsey, Chair, Department of Leadership Studies; Drew Pringle, Chair, Department of Kinesiology and Health; Tammy Kahrig, Director of Assessment and Accreditation ASSESSMENT MEASURES EMPLOYED All educator preparation programs in the College of Education and Human Services are aligned with the Ohio Board of Regents and Ohio Department of Education standards and requirements for Ohio educator licensure programs and are accredited by the National Council for the Accreditation of Teacher Education (NCATE) through fall 2015. In addition, the following programs are nationally recognized for demonstrating through key assessments that their programs meet the standards for their respective Specialized Professional Association (SPA): Adolescent to Young Adult Education, M.Ed. Integrated Language Arts: National Council of Teachers of English (NCTE) Integrated Mathematics: National Council of Teachers of Mathematics (NCTM) Integrated Social Studies: National Council for the Social Studies (NCSS) Integrated Science: National Science Teachers Association (NSTA) Early Childhood Education, B.S.Ed.: National Association for the Education of Young Children (NAEYC) Health Education, B.S.Ed.: American Association for Health Education (AAHE) Intervention Specialist, M.Ed.: Council for Exceptional Children (CEC) Early Childhood Mild-Moderate Moderate-Intensive Educational Needs Library/Media Education, M.Ed.: American Library Association (ALA) Middle Childhood Education, M.Ed.: National Middle School Association (NMSA) Multi-Age Education, M.Ed. Foreign Language (French, Spanish, German): American Council on the Teaching of Foreign Languages (ACTFL) Physical Education, B.S.Ed.: National Association for Sport and Physical Education (NASPE) Principal, M.Ed.; Superintendent, Ed.S.; Administrative Specialist, Ed.S: Educational Leadership Constituent Council (ELCC) Reading Education, M.Ed.: International Reading Association (IRA) Measures. The educator preparation programs in CEHS employ a number of direct and indirect measures to assess student learning and ensure that students attain the competencies necessary for meeting the state, NCATE, and SPA standards for the profession. Direct Measures include: a. Student Academic Portfolio b. Praxis II Licensure Examination c. Critical Transition Points and Benchmarks determined in each program of study: admission, midpoint, clinical/student teaching practicum, exit or program completion d. Clinical field experience measures and rubrics incorporating pedagogy, diversity, technology, professionalism, and emotional intelligence Page 2 of 5 e. National Teacher Performance Assessment (TPA) portfolio f. Observed performances in clinical practice including concern conference records Indirect Measures include: a. Student Course Evaluations b. Program Completer/Alumni Survey c. Site Supervisor Surveys d. Employer Survey All faculty in the college received a USB drive at the September 2011 retreat that contained data for their program that was compiled in the CEHS Dean’s Office during the 2011-2012 academic year (e.g. program completer survey, portfolio and assessments from the electronic portfolio system, Tk20). Faculty were encouraged to review and discuss assessment data with other faculty in their program to determine actions. Program directors were asked to complete an annual assessment report in which they reflected on their assessment findings and identified actions to be taken based on assessment findings as well as any changes they planned to make in their assessment process (e.g. modifications in rubrics, changes in key assessments, modifications to transition points/gateways, changes in portfolio, etc.). ASSESSMENT FINDINGS Student learning outcomes. All educator preparation programs in the college use and continually assess the student learning outcomes listed below. Programs use the student learning goals for their respective Specialized Professional Association (SPA) to further define and add levels of specificity to the student learning outcomes. Teacher candidates and candidates for professional roles: a. Are knowledgeable in their chosen fields. b. Demonstrate appropriate pedagogical content knowledge to help all students learn. c. Are knowledgeable, competent, and sensitive in working with diverse populations and in diverse settings. d. Apply appropriate technology to add value to the learning process. e. Understand and demonstrate the qualities and dispositions of professionals. f. Demonstrate emotional intelligence and are cognizant of its significance as a positive disposition in teaching, learning, leading, and adjusting in life. Findings. All educator preparation programs use an electronic assessment management system (Tk20) to facilitate the collection of student artifacts, scoring of student work, and management of data to identify areas of improvement. Portfolios, key assessments, and transition points are built into the Tk20 system. 1. The Student Academic Portfolio assesses the key content knowledge, pedagogical content knowledge, technology, emotional intelligence, professionalism, and diversity competencies for each program. Data indicate that the majority of students achieve the target or acceptable scores on the rubrics. 2. Praxis II Subject Assessments for state licensure measure knowledge of specific subjects that K12 educators will teach, as well as general and subject-specific teaching skills and knowledge. Praxis II scores for the preceding year become available each March. Wright State continues to demonstrate effective level results for all program completers, reflective of the state average pass rate. For the 2010-2011 academic year, 95% was the pass rate for WSU program Page 3 of 5 3. 4. 5. 6. 7. 8. 9. completers compared to 96% for the state. Dissaggregated scores for each specific subject area were reviewed by the program directors and faculty to determine areas of improvement. Critical transition points and benchmarks are used to determine whether or not students have acquired the minimum knowledge, skills, and abilities necessary to participate in the field. Students not meeting the gateways were provided with supplemental instruction or coaching and their entry to the field was delayed until the gateway thresholds were met. Clinical field experience measures and rubrics indicated that the majority of students are demonstrating acceptable or target levels of learning on the assessments. 2011-2012 marked the third year of full implementation of the National Teacher Performance Assessment (TPA) completed during student teaching/clinical practice. This national performance assessment includes a 3-5 day planning, instruction, and assessment of P-12 student learning and utilizes 15 rubrics developed by educators across the nation, including university, school, and accreditation representatives. During the 2011-2012 academic year, over 200 portfolios were submitted for national scoring in the spring field test. Data from this assessment is currently being compiled and will be reviewed and discussed at upcoming faculty meetings. Observation of performance in clinical practice identified a number of students needing additional development. Student course evaluations were administered at the conclusion of each course. Instructors used the feedback to identify areas of strength and areas of opportunity and make modifications at the course level. The Program Completer/Alumni survey indicates that students perceive that they have acquired the key content knowledge, pedagogical content knowledge, technology, emotional intelligence, professionalism, and diversity competencies for their program (majority rate each as “strong” or “adequate”). The limitation of this data, however, is the poor response rate to the survey. Data from the site supervisor and employer surveys indicate that site supervisors and employers perceive that students/graduates are achieving satisfactory levels of competency in the key knowledge, skills, and abilities for their programs. ACTIONS/RESPONSE TO ASSESSMENT FINDINGS Based on the annual review of the data for their program, faculty directors identified a number of areas of improvement. Below are select examples of actions that were taken or planned by program faculty: The conversion from quarters to semesters provided faculty an opportunity to critically review all courses in their programs. Extensive revision of courses and program occurred across the college. Concern conferences were held with students identified as needing additional attention/development and plans were made to help students achieve higher levels of performance. Provide more experiences for candidates in the areas of writing higher-level lesson objectives and helping candidates design diversified lesson assignments that required students to use application, analysis, evaluation, and synthesis skills. To address the two areas of Praxis II with the lowest scores, we have created a more comprehensive course in that area, which will be offered online and taught only by full-time faculty to provide continuity in the expertise of the instructors. Page 4 of 5 Continue to develop our relationships with mentor teachers through professional development activities. We have redesigned our technology course to better comply with the state requirements. The current assessment rubrics will be revised. The resource kit rubric has been adjusted to be clearer about the type of assessments candidates should complete. The rubric has also been made clearer about the types of instruction needed to address this national standard. Rubrics will be revised to add a “not applicable/not observable” category to clarify the difference between this and a low score. Provide more experiences for candidates to write IEPs. Increase the instruction of and use of technology in program coursework. Due to the small number of students in the program, our data sets are small. Need to look at data across multiple years to determine areas for improvement. APA style needs to be addressed much earlier in the program. We will identify specific classes and assignments for this to occur. Of particular concern to several program directors is the extremely low response rate to the program completer/alumni survey. In order to improve the response rates from students, a number of steps will be taken, including 1. Discuss the importance of the survey in key classes, emphasizing how the survey is used to improve the program 2. Explore online administration of the survey 3. Consider revising the survey to minimize barriers to the survey 4. Seek input from the Advisory Board ASSESSMENT ACTIVITIES FOR COMING YEAR Due to the state and national standards that impact the educator preparation programs, all of our learning outcomes are assessed annually through the measures described above. As the national Teacher Performance Assessment evolves and takes root, we plan to explore the possibility of using the TPA in place of some of our other key assessments. This year we also plan to conduct a thorough review of assessment practices and processes across the college to identify opportunities for improving and supporting meaningful assessment. We plan to review and potentially revise the CEHS Program Completer Survey, the CEHS Summary of Program Review of Assessment Data form, and CEHS Employer Survey. We also plan to reformulate the college’s Unit Assessment Committee to further the aim of assessment as a means to improve student learning. UNIVERSITY LEARNING OUTCOMES ASSESSMENT We expect that the university learning outcomes will be assessed in ED 2600 Introduction to Education and/or ED 2700 Educational Psychology. Page 5 of 5