2014‐2015 ACADEMIC ASSESSMENT MA in Reading Not accepting applications in 14‐15 Program Mission Statement The College of Education’s mission, contributing to the mission of USFSM, is to prepare outstanding educators who will learn, lead, inspire, and transform their schools and communities. Student Learning Outcomes: 1. Content Knowledge: Candidates are knowledgeable about literacy, reading and writing as content areas, instruction with assessment in reading and writing. 2. Reflective and Ethical Practice: Candidates are reflective, demonstrated through self‐evaluation of their teaching practices. 3. Evaluation and Decision‐making: Candidates are analytical of data, theoretical underpinnings, research, curriculum and instruction, and use analysis to effect instruction and staff development. 4. Educational Design a. Candidates create effective instructional programs that build knowledge and skill while supporting learners’ as active agents in their learning process. b. Candidates use technology effectively in teaching and assessment, both in terms of materials and processes that engage multiple modalities. 5. Learner as an Individual in Community a. Candidates tailor plans to the individual, taking into consideration context (social – cultural), and different experiences, abilities, and learning modes of the individual. b. Candidates understand and respect diversity demonstrated in fairness and sensitivity with the expectation that all students learn. 1 Methods of Assessment The College of Education uses a 5 point rubric scale for all critical tasks and most other assessments, unless otherwise noted. Performance descriptors are found in the College of Education Performance Level Indicators (COEPLIs). These are summarized into the 5 point scale: 1 unacceptable; 2 Marginal; 3 Meets Standard (target); 4 Advanced (target 2); 5 Exceptional. The first target is 3, meets standard, the second target is 4 (advanced) and a score of 5 is reserved to truly exceptional demonstrations of knowledge and skill. Candidates must meet standard (3) on all their Critical Tasks because they must demonstrate they hold the knowledge, skills, and dispositions expected for their educational role. The standards and requirements used to define the content knowledge, content pedagogy, pedagogy, dispositions, and other skills in the Elementary Education Initial Teacher Preparation Programs (ITPs) include, but are not limited to: Florida Educator Accomplished Practices (FEAPs); ESOL Standards; Competencies & Skills assessed in the Florida Teacher Certification Exams (FTCE); Reading Endorsement Competencies (REC); and the College of Education Candidate Proficiencies. Florida Teacher Certification Examinations for K‐12 Reading: The FTCE is composed of three tests: Professional Education, General Knowledge, and Subject Area Exams. Basic skills are evaluated through the General Knowledge Test that has 4 subtests: Essay, English Language Skills, Reading, Mathematics). Candidates must pass this exam to be eligible for admission. Professional Education knowledge (pedagogy, professional practices) is assessed through the Professional Education Test. Candidates must pass this exam to be eligible to graduate. Candidates must also pass the Subject Area Examination pertinent to their area of certification (e.g., Elementary Education K‐6 for the initial teacher preparation programs). Candidates receive either a passing or not‐passing score. Course‐based Critical Tasks: Course‐based performance assessments called critical tasks provide data on candidate progress toward meeting standards and on program effectiveness in preparing candidates to meet standards. Candidates are required to add critical tasks to an electronic portfolio housed in Taskstream where these assignments are evaluated using a content rubric that follows the COE policy on Performance Level Indicators. Candidates must earn a minimum rating of 3 (meets standard) on the following 5‐point scale: 1(unacceptable), 2(marginal), 3(meets standard), 4 (advanced), and 5 (exceptional) on the critical task in order to show adequate competency in each proficiency. Please note overall average for Critical Tasks do not include Dispositions and IP3 data which are reported separately. Dispositions Assessment: The purpose of this assessment is to provide a tool and process for formative evaluation and improvement of candidate dispositions. Dispositions are values, attitudes, beliefs, temperament, and character elements held by educators that contribute to educators’ development as professionals and support student learning. The assessment supports NCATE/ELCC Standards/other applicable professional standards; Florida certification standards (FPLS, FEAPs) including Florida Code of Ethics and the Principles of Professional Conduct for the Education Profession in Florida, program learning outcomes, and USFSM College of Education Candidate Proficiencies. Candidates self‐assess and are assessed by faculty three times during their program (near the beginning, toward the middle but prior to final internship, and during final internship). 2 Wayfind Technology Assessment: The Wayfind Assessment is an externally developed examination of candidate proficiency in education‐based technology that assesses candidate proficiencies relative to the National Educational Technology Standards for Teachers (NETS‐T) standards of the International Society for Technology in Education. The Wayfind assessment uses scaled scores. Florida Value‐added Model Impact on Student Learning: Florida’s Value Added Model (VAM) was designed to contribute data to be used in educator performance evaluations, per requirement by the FL legislature via the Student Success Act, section 1012.34, F. S., which requires evaluation systems include measurement of student learning growth in order to improve “instructional practice and school leadership.” The performance of students represents 50% of a teacher’s evaluation, with performance based on student learning growth. Student learning growth is to be assessed “annually and measured by statewide assessments or, for subjects and grades not measured by statewide assessments, by districts assessments as provided in s. 1008.22(8), F. S.” This model is also part of Race to the Top requirements. The USFSM College of Education is required to use these data, from our program completer’s first year of employment, for program improvement. Scores are reported as percentages and represent the amount of change above or below expected growth. Transition Point Projects: Similar to critical tasks are Transition Point Projects that candidates must complete successfully (score of 3 or higher) before moving into the next level of their program. These are also assessed using a content rubric with performance level indicators. Surveys: Survey data for AY12‐13 and AY13‐14 are aggregated across all programs. Beginning AY14‐15, data will be disaggregated. Exit Survey: The Exit survey is given to candidates in the Initial Teacher Preparation programs just before graduation. Through it, candidates rate their preparation on each of the Florida Educator Accomplished Practices (FEAPs), and their satisfaction with various elements of their program, such as overall preparation, opportunities to use technology, feedback and guidance from mentor teachers, supervisors, and relevance the program. Completer (Alumni) Survey: This survey is sent to all graduates of the College of Education for the first three years after graduation. Completers provide feedback on their preparation relative to the COE Candidate Competencies. The survey uses a five‐point rating of preparedness. Employer Survey: This survey is sent to all principals in the counties served by the College of Education. It seeks feedback on completer preparation in each of the COE Candidate Proficiencies. The survey uses a five‐point rating of preparedness. 3 Previous Cycle Analysis of Results and Planned Changes 1. Content Knowledge: 2. Reflective and Ethical Practice: 3. Evaluation and Decision‐making: 4. Educational Design a. Candidates design educational experiences that result in successful learning. b. Candidates demonstrate proficiency in educational technology aligned to the NETS‐T standards. 5. Learner as an Individual in Community a. Candidates construct learning environments that reflect the diversity of experiences, perspectives, and cultures of their students and the larger world. b. Candidates communicate in ways that demonstrate fairness, respect, and sensitivity to diversity, setting high academic expectations for all students. Summary of Program Improvement Initiatives (to be pursued in the next academic year): Dean’s Response: 4 USF SARASOTA‐MANATEE ACADEMIC ASSESSMENT PLANNING RECORD ACADEMIC PROGRAM: MA in Reading Student Learning Outcomes 1. Content Knowledge: Elementary education candidates demonstrate depth and breadth of content knowledge and content pedagogical knowledge across subject areas. Methods of Assessment Course‐based Critical Tasks Transition Point Project FTCE Reading K‐
12 CIP Code: 13.1315 Target All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. Employer Survey Completer Survey 2. Reflective and Ethical Practice: Elementary education candidates demonstrate self‐
reflection, professional growth, and ethical practice. Course‐based Critical Tasks Dispositions Assessment Employer Survey Completer Survey All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. Assessment Results 12‐13
Course‐based Critical Task
RED6545 RED6748 13‐14
14‐15
4.28 N=5 3.65 N=3 3.98
N=4 100% passed
No data 4.28 N=5 3.74 N=5 Transition Point Project
4.1
Literacy Curr. Prof. Dev. N=6 FTCE
100%
Reading K‐12 N=4 Institution v. State Scaled Score 236.3 v. 227.5 Employer Survey*
4.0
3.3
N=29 N=40 Completer Survey*
4.1
3.7
N=52 N=20 *Refer to Methods of Assessment (page 3) for detailed information. 12‐13
Course‐based Critical Task
RED6540 Dispositions
Employer Survey*
0
NA 4.59
N=6 4.0
N=29 13‐14
4.14
N=7 Not Avail.
14‐15
3.5
N=40 5 Student Learning Outcomes Methods of Assessment Target Assessment Results Completer Survey*
4.5
4.2
N=52 N=20 *Refer to Methods of Assessment (pages 3 & 4) for detailed information. 3. Evaluation and Decision‐making: Elementary education candidates use research‐based practices and data to make instructional decisions. Course‐based Critical Tasks Dispositions Assessment Transition Point Project All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. Employer Survey Completer Survey 4 a. Educational Design: Elementary education candidates design educational experiences that result in positive impact on student academic achievement. Course‐based Critical Tasks Transition Point Project Exit Survey Employer Survey Completer Survey All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. 12‐13
Course‐based Critical Task
RED6545 RED6748 RED6846 13‐14
14‐15
4.28 N=5 3.65 N=3 4.12 N=1 Not Avail.
‐
4.28 N=5 3.74 N=5 4.16 N=14 Dispositions
Not Avail.
Transition Point Project
3.97
Research in Literacy Instruction N=15 Employer Survey*
3.4
3.1
N=29 N=40 Completer Survey*
4.1
3.6
N=52 N=20 *Refer to Methods of Assessment (page 3) for detailed information. 12‐13
Course‐based Critical Task
RED6540 RED6846 RED6545 0 Na 4.12 N=1 4.28 N=5 13‐14
14‐15
4.10 N=7 4.16 N=14 4.28 N=5 6 Student Learning Outcomes Methods of Assessment Target Assessment Results Transition Point Project
Research in Literacy Instruction Employer Survey*
3.98
3.97
N=4 N=15 3.4
3.3
N=29 N=40 Completer Survey*
4.1
4.1
N=52 N=20 *Refer to Methods of Assessment (page 3) for detailed information. 4 b. Educational Design: Elementary Education candidates are proficient in integrating technology to enhance academic achievement. Course‐based Critical Tasks Wayfind Technology Assessment Exit Survey Employer Survey Completer Survey All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. Note: The matrix did not show any methods used under “Key Assessments”. 12‐13
13‐14
Course‐based Critical Task
RED6540 0 4.10 Na N=7 4.16 RED6846 4.12 N=1 N=14 Wayfind
‐
‐
RED6449 Employer Survey*
3.9
3.6
N=29 N=40 Completer Survey*
4.1
3.8
N=52 N=20 14‐15
*Refer to Methods of Assessment (page 3) for detailed information. 7 Student Learning Outcomes 5a. Learner as an Individual in Community: Elementary Education candidates enhance learning environments to meet needs of the diverse experiences, perspectives, and cultures of their students. Methods of Assessment Course‐based Critical Tasks Dispositions Assessment Transition Point Project Target All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. Exit Survey Assessment Results 12‐13
Course‐based Critical Tasks
RED6540 RED6545 Employer Survey Completer Survey 4.14 N=7 4.28 N=5 Transition Point Project
Employer Survey*
Completer Survey*
3.7
N=29 4.4
N=52 13‐14
14‐15
4.14 N=7 4.28 N=5 4.10
N=6 3.2
N=40 4.2
N=20 *Refer to Methods of Assessment (page 3) for detailed information. 5b. Learner as an Individual in Community: Elementary Education candidates communicate in ways that demonstrate fairness, respect, and sensitivity to diversity, setting high academic expectations for all students. Course‐based Critical Tasks Transition Point Project Exit Survey Employer Survey Completer Survey All candidates demonstrate proficiency at a 3 or higher on a 5‐
point scale. 12‐13
Course‐based Critical Tasks
RED6846 Dispositions Assessment
Employer Survey*
Completer Survey*
4.12 N=1 5.00
N=6 3.8
N=29 4.4
N=52 13‐14
14‐15
4.16 N=14 NA
3.4
N=40 4.1
N=20 *Refer to Methods of Assessment (page 3) for detailed information. 8 Current Cycle Analysis of Results and Planned Changes 6. Content Knowledge: 7. Reflective and Ethical Practice: 8. Evaluation and Decision‐making: 9. Educational Design a. Candidates design educational experiences that result in successful learning. b. Candidates demonstrate proficiency in educational technology aligned to the NETS‐T standards. 10. Learner as an Individual in Community a. Candidates construct learning environments that reflect the diversity of experiences, perspectives, and cultures of their students and the larger world. b. Candidates communicate in ways that demonstrate fairness, respect, and sensitivity to diversity, setting high academic expectations for all students. Summary of Program Improvement Initiatives (to be pursued in the next academic year): Dean’s Response: 9