Program Name: Elementary Education CIP Code: 131202 Department of Education School of Education and Behavioral Sciences Elementary Education CIP Code 131202 Program Code 350 1 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Department of Education Elementary Education Program Goals and Objectives (Aligned with the Association for Childhood Education International (ACEI) and the Oklahoma 15) 1. Graduates will demonstrate competence in basic skills and indepth knowledge of the subject matter they teach. (Knowledge, Comprehension, and Application) 2. Graduates will demonstrate professional knowledge and effective pedagogical skills. (Knowledge, Comprehension, and Application) 3. Graduates will demonstrate understanding of student development and create appropriate learning environments. (Knowledge, Comprehension, Application, Synthesis) 4. Graduates will demonstrate the ability to think reflectively and critically about the teaching/learning process. (Evaluation) 5. Graduates will demonstrate ability to interact effectively with diverse students, parents/families, and communities. (Knowledge, Comprehension, Application) 6. Graduates will demonstrate ability to use a variety of strategies to assess, analyze, and modify teaching/learning. (Knowledge, Comprehension, Application, Synthesis, Evaluation) 2 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Alignment Among Program Outcomes, CU Mission Statement & Strategic Plan 2013 Program Outcomes CU Mission Statement CU Strategic Plan 2013 1. Graduates will demonstrate competence in basic skills and in-depth knowledge of the subject matter they teach. “The university desires to assist its students and other persons living in its service area in acquiring the skills, knowledge, values, and attitudes that will enable them to lead creative, productive and self-fulfilling lives.” “Maintain and enhance Cameron’s commitment to providing programs of the highest quality in instruction, research and service to better meet the needs of the citizens of the region.” 2. Graduates will demonstrate professional knowledge and effective pedagogical skills. “The university desires to assist its students and other persons living in its service area in acquiring the skills, knowledge, values, and attitudes that will enable them to lead creative, productive and self-fulfilling lives.” “Capitalize on Cameron’s multiple levels of academic programs – associate, baccalaureate, and master’s – to enhance student learning and to increase opportunities for educational advancement.” 3. Graduates will demonstrate understanding of student development and create appropriate learning opportunities. “The university recognizes that the educational process includes the development of the intellectual, cultural, social, physical, moral, and occupational capacities of persons who participate in its programs and activities.” “Assure efficient, effective course delivery in multiple formats.” “Develop a centralized reporting system to monitor and expand student opportunities for service learning, to include internships, civic involvement, and off-campus educational activities.” Fall 2010 Program Quality Improvement Report 2010-2011 3 Program Name: Elementary Education CIP Code: 131202 Alignment Among Program Outcomes, CU Mission Statement & Strategic Plan 2013 Program Outcomes CU Mission Statement CU Strategic Plan 2013 4. Graduates will demonstrate ability to think reflectively and critically about the teaching/learning process. “The university desires to assist its students and other persons living in its service area in acquiring the skills, knowledge, values, and attitudes that will enable them to lead creative, productive and self-fulfilling lives.” “Maintain and enhance Cameron’s commitment to providing programs of the highest quality in instruction, research and service to better meet the needs of the citizens of the region.” 5. Graduates will demonstrate ability to interact effectively with diverse students, parents/families, and communities. “The university desires to assist its students and other persons living in its service area in acquiring the skills, knowledge, values, and attitudes that will enable them to lead creative, productive and self-fulfilling lives.” “Enhance and promote an active campus life by focusing on athletics, cultural events, student programs, and organizations.” 6. Graduates will demonstrate ability to use a wide variety of strategies to assess, analyze, and modify teaching/learning. “The university recognizes that the educational process includes the development of the intellectual, cultural, social, physical, moral, and occupational capacities of persons who participate in its programs and activities.” “Maintain and enhance Cameron’s commitment to providing programs of the highest quality in instruction, research and service to better meet the needs of the citizens of the region.” “Expand the number of educational, cultural, and social opportunities for the region.” 4 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Alignment Among Program Outcomes and Department of Education Mission Program Outcomes Department of Education Mission 1. Graduates will demonstrate competence in basic skills and in-depth knowledge of the subject matter they teach. Educators who are competent use pedagogical and content knowledge to support learning for all, provide instruction based on standards and student needs, and use assessment and strategies for active engagement so that all can learn. 2. Graduates will demonstrate professional knowledge and effective pedagogical skills. Educators who are competent use pedagogical and content knowledge to support learning for all, provide instruction based on standards and student needs, and use assessment and strategies for active engagement so that all can learn. 3. Graduates will demonstrate understanding of student development and create appropriate learning opportunities. Educators who are caring are responsive to individual needs and create learning environments that promote positive social interactions and motivation. 5 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Alignment Among Program Outcomes and Department of Education Mission Program Outcomes Department of Education Mission 4. Graduates will demonstrate ability to think reflectively and critically about the teaching/learning process. Educators who are committed collaborate with others by using effective communication skills while being reflective decision makers and lifelong learners who are willing to change in order to continuously improve. 5. Graduates will demonstrate ability to interact effectively with diverse students, parents/families, and communities. Educators who are caring are responsive to individual needs and create learning environments that promote positive social interactions and motivation. 6. Graduates will demonstrate ability to use a wide variety of strategies to assess, analyze, and modify teaching/learning. Educators who are committed collaborate with others by using effective communication skills while being reflective decision makers and lifelong learners who are willing to change in order to continuously improve. 6 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Action Plan 2008-2009 Outcome Action Plan Implementation #1 Graduates will demonstrate competence in basic skills and in-depth knowledge of the subject matter they teach. •Candidates encouraged to take Introduction to Teaching prior to taking OGET. •Implement a tutorial writing session for the constructed response item. Implemented by advisor Not implemented, but keep monitoring the scores. #2 Graduates will demonstrate professional knowledge and effective pedagogical skills. •Implement a study skills seminar at RSU •Continue to monitor the data at both sites in terms of reliability •Implemented through SOEA •Held meeting with Lawton and RSU supervisors to discuss what evidence should be seen when observing student teachers. Mentor teachers were all part of the discussion during mentor training. •Rubric was created for the student teaching assessment to be used by supervisors and mentor teachers. #4 Graduates will demonstrate the ability to think reflectively and critically about the teaching/learning process. •Add content of analyzing assessment results into either Practicum or Classroom Assessment course •Added content in Practicum and Classroom Assessment. 7 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Direct Measures of Assessment with Remediation Major Assessments (direct) Entry to Teacher Education Grade Equivalent of 12.0 or higher on the Nelson Denny Passing score on OGET Passing score on EDUC 3003 lesson plan rubric Entry to student teaching Passing score on OSAT Passing score on State Phonics Test Satisfactory portfolio check: Passing scores for a minimum of 5 OK competencies as evidenced by course artifacts and instructor-completed rubrics Exiting student Passing scores for remaining 10 OK competencies as evidenced by teaching course artifacts and instructor-completed rubrics Completed Teaching Styles checklist One year Completion of Follow-up Survey postgraduation Remediation Reading lab activities and resources Attend SOEA workshop(s) Candidates submit draft(s) of lesson plan in course prior to final portfolio submission; Resubmit/rescore OSAT study guide; consult content area textbooks; Study Island Provide with list of resources on phonics; Retake test Candidates submit draft(s) of assignments in course prior to final portfolio submission; Resubmit/rescore Candidates submit draft(s) of assignments in course prior to final portfolio submission; Resubmit/rescore NONE NONE 8 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Indirect Measures of Assessment with Remediation Major Assessments (indirect) Entry to Teacher Education 3 satisfactory recommendation forms with dispositions (1from faculty within dept., 2 from faculty outside dept.) Grade of “C” or better in the following courses: ENGL 1113, ENGL 1213, COMM 1113, MATH 1513 or higher, HIST 1483 or 1493, PS 1113, 2 Humanities, and EDUC 3003 Grade of “S” in EDUC 1800 Completion of Application for Admittance to Teacher Education Retention GPA of 2.5 Concurrent enrollment or a grade of C or better in EDUC 3733 and Bio or Phys Science Satisfactory interview including assessment of critical thinking skills Remediation Letter of appeal to Admission & Retention Committee (Teacher Education Council); Recommendation letter from faculty member; 3 new recommendation forms with dispositions (1from faculty within dept., 2 from faculty outside dept.) Retake Course(s) Receive a “U” and attend or retake NONE Academic Probation Retake Course(s) Provide with list of resources (websites, books, articles, etc.) on developing critical thinking skills Entry to Submission of completed application for student teaching NONE student Satisfactory Practicum dispositional assessments in each of the following: Faculty member, Chair, and candidate conference; Plan of teaching Math Methods, Reading Diagnosis, and Practicum (school-based mentor) Improvement created Satisfactory transcript check - minimum GPA of 2.5 Academic Probation Minimum of 80 clock hours of field experiences with two diverse field Additional field experience placements completed Exiting student Satisfactory student teaching evaluations University Supervisor, , and Intern Conference; Develop a Plan teaching of Improvement Minimum retention GPA of 2.5 Academic Probation Program Completion of exit survey NONE completion Minimum retention GPA of 2.5 Academic Probation Grade of “C” or better in all elementary education major courses, Retake Course(s) professional education courses, and courses that fulfill the state 4x12 requirement Satisfactory completion of foreign language proficiency requirements NONE 9 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Student-Learning Outcome and Measurements MEASUREMENTS OF STUDENT LEARNING PROGRAM OUTCOME 1 Graduates will demonstrate competence in basic skills and indepth knowledge of the subject matter they teach. CURRICULUM AREA OR TARGET AUDIENCE Elementary education majors Measurements OGET-Direct OSAT-Direct Student Teaching Evaluation ResultsIndirect Follow-up survey resultsIndirect Methods used to determine validity of measurement instruments Methods used to determine reliability of measurements Use of multiple assessors Use of multiple assessors Use of multiple assessors Use of multiple assessors Schedule for measurements 5 times per year 5 times per year 2 times per year Annually 10 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 1: OGET Oklahoma General Education Test Criterion-referenced and competency-based state teacher exam • Assesses general knowledge • A passing score of 240 required for admission to the Teacher Education program, along with other criteria such as an overall GPA of 2.5. 11 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 285 280 275 270 265 260 255 250 245 240 Multi-Year Trend Chart OGET Critical Thinking: Reading and Communication Overall Scores CU RSU State 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 12 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OGET Communication Overall Scores 290 280 270 260 CU RSU State 250 240 230 220 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 13 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OGET Critical Thinking: Mathematics Overall Scores 300 290 280 270 CU RSU State 260 250 240 230 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 14 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 300 Multi-Year Trend Chart OGET Mathematics Computation Overall Scores 290 280 270 CU RSU State 260 250 240 230 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 15 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 290 Multi-Year Trend Chart OGET Science, Art & Literature, Social Sciences Overall Scores 280 270 260 CU RSU State 250 240 230 220 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 16 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OGET Critical Thinking: Writing Overall Scores 255 250 245 240 CU RSU State 235 230 225 220 06-07 07-08 CU=30 CU=67 State=5000 State=5475 08-09 09-10 CU=50 CU=32 RSU=5 RSU=14 State=5338 State=4942 17 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 1: OGET Analysis of Data: • RSU candidates maintained the 100% pass rate. • 91% of CU candidates passed. This is a 15% increase over 2008-2009. • In 2008 – 2009 Cameron candidates only exceeded the state average in three areas: Computation Skills, Liberal Studies: Science, Art & Literature, Social Sciences, and Critical Thinking Skills: Writing. In 2009-2010 candidates exceeded the state average in Communication, Liberal Studies: Science, Art & Literature, Social Sciences, and Critical Thinking Skills: Writing. • Even though the writing scores exceeded the state average, they are still the lowest as compared to the other areas. 18 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 1: OGET Action Plan: • In order to help students improve in the areas of Critical Thinking Skills in Mathematics, Reading and Communication, the Department of Education faculty will work in partnership with the General Education Faculty to review the test result data and the test objectives in order to create an action plan. This could be done by inviting them to be members of the Assessment Advisory Board. (2010-2011 AY) 19 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 2: OSAT Oklahoma Subject Area Test Criterion-referenced and competency-based state teacher exam • Composed of two sub-tests assessing subject content knowledge 1. evaluates through 55 selected response items & 1 constructed response item 2. evaluates through 55 selected response items • Passing score of 240 required for admission to student teaching, along with other criteria such as an overall GPA of 2.5. 20 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 1 Reading Overall Scores 290 280 270 260 CU RSU State 250 240 230 220 06-07 CU=21 State=2582 07-08 08-09 09-10 CU=43 CU=81 CU=53 RSU=4 RSU=6 RSU=7 State=2090 State=2314 State=2392 21 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 1 Language Arts Overall Scores 270 265 260 255 250 CU RSU State 245 240 235 230 06-07 CU=21 State=2582 07-08 08-09 09-10 CU=43 CU=81 CU=53 RSU=4 RSU=6 RSU=7 State=2090 State=2314 State=2392 22 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 1 Social Studies Overall Scores 270 265 260 255 250 245 240 235 230 225 220 CU RSU State 06-07 CU=21 State=2582 07-08 08-09 09-10 CU=43 CU=81 CU=53 RSU=4 RSU=6 RSU=7 State=2090 State=2314 State=2392 23 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 1 Constructed Response Overall Scores 255 250 245 240 CU RSU State 235 230 225 220 06-07 CU=21 State=2582 07-08 08-09 09-10 CU=43 CU=81 CU=53 RSU=4 RSU=6 RSU=7 State=2090 State=2314 State=2392 24 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 2: OSAT Subtest 1 Analysis of Data: • In 09 – 10, 79% of Cameron’s candidates and 100% of RSU’s candidates passed Subtest 1. This is a decline of 5% over last year for CU and an increase of 17% for RSU. CU candidates scored above the state average in but one are (SS) of the test. RSU candidates scored above the state averages in all but one area (LA). • The writing scores for the CU campus declined while the RSU campus scores’ increased. This could be because the action plan was implemented by adding tutorial writing sessions through SOEA at the RSU campus. 25 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 2: OSAT Subtest 1 Action Plan: (Recommended by Assessment Advisory) • Take Reading Diagnosis before taking the Elementary OSAT. Add additional writing component to Reading Diagnosis writing exercises that include multistep directions. Emphasize strategy file in Reading Diagnosis course. (Spring 2011) • Focus on (emphasize) objectives in the OCTP study guide to help improve test results. This could be accomplished by revisiting alignment among the methods courses’ objectives. (Spring 2011) 26 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 2 Math Overall Scores 265 260 255 250 CU RSU State 245 240 235 06-07 CU=31 State=2224 07-08 08-09 09-10 CU=47 CU=80 CU=51 RSU=4 RSU=6 RSU=5 State=1793 State=2028 State=2129 27 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 2 Science Overall Scores 275 270 265 260 CU RSU State 255 250 245 06-07 CU=31 State=2224 07-08 08-09 09-10 CU=47 CU=80 CU=51 RSU=4 RSU=6 RSU=5 State=1793 State=2028 State=2129 28 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 275 Multi-Year Trend Chart OSAT Subtest 2 Health & Fitness Overall Scores 270 265 CU RSU State 260 255 250 06-07 CU=31 State=2224 07-08 08-09 09-10 CU=47 CU=80 CU=51 RSU=4 RSU=6 RSU=5 State=1793 State=2028 State=2129 29 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OSAT Subtest 2 Fine Arts Overall Scores 280 270 260 250 CU RSU State 240 230 220 210 06-07 CU=31 State=2224 07-08 08-09 09-10 CU=47 CU=80 CU=51 RSU=4 RSU=6 RSU=5 State=1793 State=2028 State=2129 30 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 2: OSAT Subtest 2 Analysis of Data: • 80% of the RSU candidates and 84% of Cameron’s candidates passed Subtest 2. This was a decline of 20% over last year for RSU. This was an increase of 5% for CU. Cameron candidates scored below the state average in math, science, and Fine Arts. RSU candidates scored above the state averages in two areas (science and fine arts). • For the CU campus, math continues to be the area with the lowest scores. 31 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 2: OSAT Subtest 2 Action Plan: • Continue to evaluate the alignment of course objectives in the methods courses with the test objectives. Math test objectives focus on problem-solving, so emphasize in Math Methods course. (2010-2011 AY) 32 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 3: Student Teaching Evaluations • Revised beginning Fall 2007 • Content addendum added Spring 2008 & scored once per placement • Completed by mentor teacher at the end of each of two placements; by university supervisor twice during each placement • Two evaluators ensures reliability and validity of scores. • RSU students complete student teaching during spring semester only. • Remediation provided if needed 33 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 3: Student Teaching Evaluations - Lawton 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 07-08 N=51 08-09 N=57 09-10 N=54 Language Arts 2.78 2.75 2.67 Science 2.75 2.76 2.63 Mathematics 2.76 2.82 2.69 Social Studies 2.76 2.74 2.68 The Arts 2.73 2.69 2.71 Health Education 2.82 2.70 2.69 Physical Education 2.83 2.72 2.76 34 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 3: Student Teaching Evaluations – RSU 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 08-09 n=8 09-10 N=13 Language Arts 2.72 2.81 Science 2.80 2.56 Mathematics 2.79 2.80 Social Studies 2.69 2.76 The Arts 2.86 2.60 Health Education 2.67 2.50 Physical Education 2.57 2.45 35 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 3 Student Teaching Evaluations Analysis of Data: • Candidates are scoring well above passing in all areas Action Plan: • Continue to monitor 36 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 1 Measure 4: Follow-Up Survey • Completed by school administrator, mentor teacher, and university representative at the end of candidate’s first year of teaching • Used to assess how well the program prepared candidates • Initial administration in 06-07 year, compared to 0708, 08-09, 09-10 data in following chart (Lawton only) • Have no data from RSU graduates because Cameron does not supervise resident teachers in NE Oklahoma 37 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 4: Follow-up Survey Data Mean Scores for CU Lawton Students Only (selected items,07-08, N=49; 08-09, N=43, 09-10, N=40) 1=Not Acceptable 2=Emerging 3=Competent 4=Exemplary Passing = 3 or above Understands the central concepts and methods of inquiry of the subject matter discipline(s) he or she teaches. Administrators Mentors University Reps. Overall 06 - 07 3.65 3.70 3.48 3.61 07- 08 3.33 3.70 3.48 3.51 08 – 09 3.79 3.79 3.51 3.70 09 – 10 3.85 3.68 3.73 3.75 Analysis of Data: • Administrators and University Representatives’ ratings increased from 08-09 to 09-10. • Mentors rated candidates lower from 08-09 to 09-10. Action Plan: • Consider more in-depth training for mentors so that they clearly understand what they are evaluating in regards to this specific item and how it relates to the candidate’s preparation and experience. Fall 2010 Program Quality Improvement Report 2010-2011 38 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey 4.00 3.90 Score 3.80 3.70 Administrators Mentors Univ. Reps Overall 3.60 3.50 3.40 3.30 06-07 07-08 08-09 09-10 Year 39 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Student-Learning Outcome and Measurements MEASUREMENT OF STUDENT LEARNING PROGRAM OUTCOME 2 Graduates will demonstrate professional knowledge and effective pedagogical skills. CURRICULUM AREA OR TARGET AUDIENCE Elementary education majors Measurements 1. OPTEDirect 2. Teacher Work SampleDirect 3. Student Teaching Evaluation ResultsIndirect 4. Follow-up survey resultsIndirect Methods used to determine validity of measurement instruments Methods used to determine reliability of measurements Use of multiple assessors Use of multiple assessors Use of multiple assessors Use of multiple assessors Schedule for measurements 5 times per year 2 times per year 2 times per year 2 times per year Annually 40 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 1: OPTE Oklahoma Professional Teacher Exam Criterion-referenced and competency-based state teacher exam • Assesses professional knowledge & skills • 75 selected-response items • Constructed-response items consist of 3 written performance modules – Critical Analysis: Candidates analyze an educational issue related to learners and the learning environment. – Student Inquiry: Candidates describe an instructional activity which would help students achieve a specific learning goal. – Teacher Assignment: Candidates apply professional knowledge to evaluate a school or classroom situation and recommend a course of action. • Passing score of 240 required to obtain an Oklahoma Teaching License 41 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: Learners in the Learning Environment Overall Scores 285 280 275 270 265 260 255 250 245 240 CU RSU State 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 42 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: Instruction and Assessment Overall Scores 270 268 266 264 262 260 258 256 254 252 250 CU RSU State 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 43 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: The Professional Environment Overall Scores 270 265 260 255 CU RSU State 250 245 240 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 44 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: Critical Analysis Module Overall Scores 270 265 260 255 250 CU RSU State 245 240 235 230 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 45 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: Student Inquiry Module Overall Scores 242 240 238 236 234 232 230 228 226 224 222 CU RSU State 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 46 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart OPTE: Teacher Assignment Module Overall Scores 255 250 245 240 CU RSU State 235 230 225 220 06-07 CU=53 State=2224 07-08 08-09 09-10 CU=48 CU=69 CU=56 RSU=4 RSU=6 RSU=12 State=1941 State=1966 State=1725 47 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2 Measure 1: OPTE Analysis of Data: • In 09 – 10, 100% of RSU candidates passed, while 96% of CU candidates passed. This was consistent for RSU; however, it’s a 3% increased for CU over last year. • Also, Cameron candidates scored above the state average in all areas except two. RSU candidates scored above the state average in four areas and fell below state average in the areas the Professional Environment, and the Teacher Assignment Module. This is consistent with last year; however, scores increased in Instruction and Assessment over last year. Action Plan: (Recommended by Assessment Advisory ) • Develop writing exercises that include multi-step directions; Focus on objectives in the OCTP Study Guide. (Spring 2011) 48 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 2: Teacher Work Sample • Multi-week thematic unit created by the teacher candidate • Taught in a public school classroom under guidance of a public school mentor teacher • Organized into 6 “factors” or sections • Emphasis on using assessment to guide instruction & achieving student learning gains 49 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Teacher Work Sample Factor 2: Unit Goals and Objectives 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% CU Not Met RSU Not Met CU Met RSU Met CU Exceeds RSU Exceeds 06-07 CU=48 07-08 CU=49 08-09 CU=72 RSU=8 09-10 CU=50 RSU=15 50 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Teacher Work Sample Factor 4: Instructional Design 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% CU Not Met RSU Not Met CU Met RSU Met CU Exceeds RSU Exceeds 06-07 CU=48 07-08 CU=49 08-09 CU=72 RSU=8 09-10 CU=50 RSU=15 51 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2: Measure 2 Teacher Work Sample Data Analysis of Data: • Scores indicate students are competent in professional knowledge and skills • More candidates have exceeding scores in Unit Goals and Objectives as compared to Instructional Design. • Candidates who have “not met” scores are given an opportunity to reach the competent level. Action Plan: • None at this time 52 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2: Measure 3 Student Teaching Evaluations – Lawton Mentor Teacher Ratings 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 07-08 N=51 08-09 N=57 09-10 N=58 Follows Oklahoma Criteria for Teaching Performance 2.81 2.82 2.78 Reviews and revises decisions based on evidence 2.85 2.75 2.83 Is sensitive to student needs 2.89 2.87 2.88 Willingness to use or familiarity with technology 2.86 2.81 2.84 Takes advantage of professional growth opportunities 2.81 2.8 2.8 53 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2: Measure 3 Student Teaching Evaluations - Lawton University Supervisor Ratings 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 07-08 N=51 08-09 N=57 09-10 N=58 Follows Oklahoma Criteria for Teaching Performance 2.81 2.82 2.79 Reviews and revises decisions based on evidence 2.85 2.75 2.51 Is sensitive to student needs 2.89 2.87 2.8 54 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2: Measure 3 Student Teaching Evaluations - RSU Mentor Teacher Ratings 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 08-09 N=8 09-10 N=13 Follows Oklahoma Criteria for Teaching Performance 2.26 2.75 Reviews and revises decisions based on evidence 2.30 2.75 Is sensitive to student needs 2.60 2.84 Willingness to use or familiarity with technology 2.87 2.88 Takes advantage of professional growth opportunities 2.77 2.76 55 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2: Measure 3 Student Teaching Evaluations - RSU University Supervisor Ratings 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Item 08-09 N=8 09-10 N=13 Follows Oklahoma Criteria for Teaching Performance 2.26 2.67 Reviews and revises decisions based on evidence 2.28 2.63 Is sensitive to student needs 2.57 2.94 56 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2 Measure 3 Student Teaching Evaluations Analysis of Data: • Candidates are well above passing. • Overall data shows a slight dip in scores for Lawton students candidates and a slight increase for RSU students. This may be due to the “leveling” meeting that occurred in fall 2009. Action Plan: • Continue to monitor the data at both sites in terms of reliability. 57 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 4: Follow Up Survey Data Mean Scores for CU Students Only 1=Not Acceptable Year 06 - 07 07 – 08 08 – 09 09 - 10 (selected items, N=49; 08-09, N=43; 09-10, N=40) 2=Emerging 3=Competent 4=Exemplary Passing = 3 or above Rating Party 8 10 11 Administrators Mentors University Reps. Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall 3.82 3.71 3.56 3.70 3.75 3.73 3.75 3.72 3.86 3.84 3.63 3.78 3.92 3.83 3.80 3.85 3.59 3.61 3.46 3.55 3.44 3.64 3.54 3.54 3.72 3.67 3.51 3.64 3.80 3.73 3.75 3.76 3.53 3.69 3.48 3.57 3.52 3.72 3.52 3.58 3.88 3.79 3.63 3.77 3.79 3.72 3.70 3.74 Item 8. Creates learning experiences that make the subject matter meaningful to students. Item 10. Uses a variety of communication techniques to foster active inquiry Item 11. Plans instruction based upon curriculum goals 58 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 4: Follow Up Survey Data Mean Scores for CU Students Only 1=Not Acceptable (selected items, N=49; 08-09, N=43; 09-10, N=40) 2=Emerging 3=Competent 4=Exemplary Passing = 3 or above Year 06 - 07 07 – 08 08 – 09 09 - 10 Rating Party 17 19 22 Administrators Mentors University Reps. Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall 3.73 3.71 3.52 3.66 3.54 3.67 3.54 3.58 3.81 3.84 3.58 3.74 3.82 3.80 3.60 3.74 3.76 3.65 3.51 3.64 3.60 3.77 3.48 3.62 3.86 3.77 3.64 3.76 3.81 3.78 3.73 3.77 3.69 3.73 3.54 3.66 3.38 3.52 3.32 3.41 3.81 3.79 3.63 3.74 3.82 3.80 3.83 3.82 Item 17. Understand the process of continuous, life-long learning Item 19. Actively seeks opportunities for continuous professional growth. Item 22. Understands and develops instructional strategies/plans based on OK core curriculum Fall 2010 Program Quality Improvement Report 2010-2011 59 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 8 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 60 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 10 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 61 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 11 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 62 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 17 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 63 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 19 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 64 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 22 4.00 3.90 3.80 Administrators 3.70 Mentors 3.60 Univ. Reps Overall 3.50 3.40 3.30 06-07 07-08 08-09 09-10 65 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 2 Measure 4: Follow-up Survey Results Analysis of Data: • Students scored well above passing on all items. • Administrators rated graduates higher in nearly all items during 09-10 • Steady gains indicated on item 10 over the years may indicate that 09-10 action plan for that item was successfully implemented Action Plan: • Continue modeling how to find and use PASS objectives for students in Intro to Teaching and methods courses where students create lesson plans and units of instruction. • Bring in professional development opportunities for students on campus such as OEA teaching consultants or other guest speakers. 66 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Student-Learning Outcome and Measurements MEASUREMENTS OF STUDENT LEARNING PROGRAM OUTCOME 4 Graduates will demonstrate the ability to think reflectively and critically about the teaching/learni ng process. CURRICULUM AREA OR TARGET AUDIENCE Elementary education majors Measurements 1. Teacher Work SampleDirect 2. Student Teaching Evaluation ResultsIndirect 3. Follow-up survey resultsIndirect Methods used to determine validity of measurement instruments Methods used to determine reliability of measurements Schedule for measurements 2 times per year Use of multiple assessors Use of multiple assessors 2 times per year Use of multiple assessors Use of multiple assessors Annually 67 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Measure 1:Teacher Work Sample Factor 5: Analysis of Learning 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% CU Not Met RSU Not Met CU Met RSU Met CU Exceeds RSU Exceeds 06-07 CU=48 07-08 CU=49 08-09 CU=72 RSU=8 09-10 CU=50 RSU=15 68 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Measure 1:Teacher Work Sample Factor 6: Reflection on Teaching and Learning 120% 100% 80% CU Not Met RSU Not Met CU Met RSU Met CU Exceeds RSU Exceeds 60% 40% 20% 0% 06-07 CU=48 07-08 CU=49 08-09 CU=72 RSU=8 09-10 CU=50 RSU=15 69 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4: Measure 1 Teacher Work Sample Data Analysis of Data: • Most candidates exceed the criteria on this assessment, therefore indicates the program outcome is being met. • Candidates who have “not met” scores are given an opportunity to reach the competent level. Action Plan: • None at this time. 70 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4: Measure 2 Student Teaching Evaluations - Lawton 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Mentor Teacher Ratings Is a reflective decision maker 07-08 N=51 08-09 N=57 09-10 N=58 2.85 2.83 2.81 University Supervisor Ratings Item 07-08 N=51 08-09 N=57 09-10 N=58 Is a reflective decision maker 2.79 2.68 2.59 71 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4: Measure 2 Student Teaching Evaluations - RSU 1=Does not meet expectations 2=Meets expectations Passing = 2 or above 3=Exceeds expectations Mentor Teacher Ratings 08-09 N=8 09-10 N=13 2.42 2.84 Item 08-09 N=8 09-10 N=13 Is a reflective decision maker 2.39 2.62 Is a reflective decision maker University Supervisor Ratings 72 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4 Measure 2 Student Teaching Evaluations Analysis of Data: • Candidates are well above passing. • Overall data shows a slight dip in scores for Lawton students candidates and a slight increase for RSU students. This may be due to the “leveling” meeting that occurred in fall 2009. Action Plan: • Continue to monitor the data at both sites in terms of reliability. 73 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Measure 3: Follow Up Survey Data Mean Scores for CU Lawton Students Only (selected items,07-08, N=49; 08-09, N=43; 09-10, N=40) 1=Not Acceptable 2=Emerging 3=Competent 4=Exemplary Passing = 3 or above Year 06 - 07 07 – 08 08 – 09 09 - 10 Rating Party Administrators Mentors University Reps. Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall Administrators Mentors University Reps Overall 16: Evaluates the effects of 18:Displays the willingness his/her choices and actions to change when change will on others and modifies lead to greater student those actions when needed. learning and development. 3.61 3.60 3.52 3.58 3.46 3.67 3.49 3.54 3.81 3.86 3.53 3.74 3.75 3.65 3.70 3.70 3.86 3.82 3.70 3.80 3.52 3.57 3.55 3.55 3.91 3.86 3.67 3.81 3.90 3.83 3.85 3.86 74 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 16 4.00 3.90 Title 3.80 3.70 Administrators 3.60 Mentors Univ. Reps 3.50 Overall 3.40 3.30 06-07 07-08 08-09 09-10 Title 75 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Multi-Year Trend Chart Follow-Up Survey-Item 18 4 3.9 Title 3.8 3.7 Administrators 3.6 Mentors Univ. Reps 3.5 Overall 3.4 3.3 06-07 07-08 08-09 09-10 Title 76 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4: Measure 3 Follow-up Survey Results Analysis of Data: • Students scored well above passing on all items. • Administrators’ and mentors’ ratings were consistent in 06-07. • Administrators rated candidates the lowest in 07-08. • University Reps rated candidates the lowest in 08-09. Action Plan: • Consider implementing the Fred Jones model into the Classroom Management course. 77 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Outcome 4: Measure 3 Follow-up Survey Results Analysis of Data: • All ratings well above average for each item rated • Administrator and Mentor ratings were down in 09-10 from 08-09 in both items, while University Rep. ratings increased. Action Plan: • The residency committees are no longer a state requirement, we will be using a new way to gather the data only from graduates, university faculty, and principal (as available). 78 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Published information on graduates Lawton 09-10* Entered Graduate School Working In Discipline Unknown Fall 2009 (n=30) 2 24 4 Spring 2010 (n=27) 1 21 0 Total (n=57) 3 45 4 RSU Entered Graduate School Working In Discipline Unknown Spring 2010 (n=13) 0 6 6 Total 0 6 6 09-10 *Status of students not included above is known: awaiting certification, job hunting, or electing not to work. 79 Fall 2010 Program Quality Improvement Report 2010-2011 Program Name: Elementary Education CIP Code: 131202 Questions? 80 Fall 2010 Program Quality Improvement Report 2010-2011