Central Washington University Assessment of Student Learning Department and Program Report Please enter the appropriate information concerning your student learning assessment activities for this year. Academic Year of Report: 2012-2013 Department: Advanced Programs College: CEPS Program: MEd School Administration/Instructional Leadership 1. What student learning outcomes were assessed this year, and why? The student learning outcomes assessed this year are as follows: 1) leader promotes the success of every student by facilitating the development, articulation, implementation, and stewardship of a vision of learning that is shared and supported by all stakeholders. 2) leader promotes the success of every student by advocating, nurturing and sustaining a school culture and instructional program conducive to student learning and staff professional growth. 3) leader promotes the success of every student by ensuring management of the organization, operation, and resources for a safe, efficient, and effective learning environment. 4) leader promotes the success of every student by collaborating with faculty and community members, responding to diverse community interests and needs, and mobilizing community resources. 5) leader promotes the success of every student by acting with integrity, fairness, and in an ethical manner. 6) leader promotes the success of every student by understanding, responding to, and influencing the political, social, economic, legal, and cultural context. These outcomes are related to College goals II, V, University goals and I, II, V, and VI, and Washington Standard-based benchmarks. 2. How were they assessed? A) What methods were used? The methods used to assess candidates were: 1) 2) 3) 4) 5) Livetext Portfolio (M.Ed. & Residency Certificate) Pre and Post Self-Inventory Assessment (Completed by Principal Interns) Mid-Quarter Measurement of Knowledge & Skills Completed by (Principal Interns) Building Supervisors’ quarterly evaluation of Principal Interns. Assessment of Online Learning 1 B) Who was assessed? All candidates (35) in the School Administration and Program Administrator Certification programs were assessed based on the Interstate School Leaders Licensure Consortium (ISLLC) and the knowledge and skills standards. C) When was it assessed? All of the students in the school administration program were assessed in the 2012-2013 academic year. Their LiveText portfolios were reviewed by all educational administration faculty members, summarized, and discussed. A pre and post self-assessment inventory on the ISLLC standards was completed by the interns. A survey on standard V – Knowledge and Skills was also administered to the interns. See results below: EDAD 577- Diversity Leadership - Performance Assessment –Fall 2012 Candidates’ portfolios in EDAD 577 were assessed using a rubric that evaluates their leadership/vision, disposition, and knowledge. 95% of the candidates met the ISLLC standards. Leadership 64 (72%) Disposition 64 (72%) Knowledge 64 (72%) Target 21 (23%) Acceptable 22 (25%) Unacceptable 2 (2%) 3 (3%) 2 (2%) EDAD 578 – Readings in School Leadership - Performance Assessment –Fall 2012 Candidates’ portfolios in EDAD 578 were assessed using a rubric that evaluates their ability to evaluate, reflect, and make connections regarding a leadership situation. 100% of the candidates met the ISLLC standards. Describe the Situation Relevant ISLLC Reflect the Situation Evaluate the Situation Connections 22 (25%) 1 (11%) 8 (88%) 1 (12%) 7 (87%) 1 (12%) 7 (87%) 1 (12%) 7 (87%) 1 (12%) 7 (87%) Unsatisfactory Emerging Proficient Exemplary EDAD 579- School Personnel Performance Assessment – Summer 2012 Candidates’ portfolios in EDAD 579 were assessed using a rubric that evaluates their leadership, knowledge and skills, resource allocations and legal and regulatory application. 100% of the candidates met the ISLLC standards. Leadership 40 (97%) 1 (2%) Knowledge and Skills 40 (95%) 42 (100%) 40 (97%) 2 (4%) Resource Allocation Legal and Regulatory Application Target 1 (2%) Acceptable Unacceptable EDAD 580-Educational Administration Performance Assessment - Winter 2013 Candidates’ portfolios in EDAD 580 were assessed using a rubric that evaluates their leadership, decision-making, volunteering, and extended learning opportunities. 94% of the candidates met the ISLLC standards. Describe the Situation Relevant ISLLC Reflect the Situation Evaluate the Situation Connections 1 (1%) 4 (4%) 40 (45%) 1 (1%) 4 (4%) 40 (45%) 43 (48%) 1 (1%) 4 (4%) 40 (45%) 1 (1%) 4 (4%) 40 (45%) 43 (48%) 1 (1%) 4 (4%) 40 (45%) Unsatisfactory Emerging 43 (48%) 43 (48%) Proficient Exemplary EDAD 581-School Finance Performance Assessment – Spring 2013 Candidates’ portfolio in EDAD 581 were assessed using a rubric that evaluates their leadership and student learning, management and effective learning environment, collaboration and mobilizing community resources, and demonstrating integrity and ethics. 100% of the candidates met the ISLLC standards. Leadership and Student Learning ISLLC-1, ISLLC-2, ISLLC-6 Management and Effective learning Enviroment ISLLC-3, ISLLC-4 Collaborating and mobilizing Community Resources ISLLC-1, ISLLC-4 Demonstration of Integrity and Ethics ISLLC-1, ISLLC-5, ISLLC-6 25 (100%) 25 (100%) 25 (100%) 25 (100%) Target 43 (48%) Acceptable Unacceptable EDAD 582 - School Curriculum Performance Assessment – Spring 2013 Candidates’ performance in EDAD 582 was assessed using a rubric that evaluates leadership, information collection and problem analysis, curriculum design, and measurement and evaluation on the course rubric. 100% of the candidates met the ISLLC standards. Leadership ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4, ISLLC-5, ISLLC-6 Information Collection and Problem Analysis ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4, ISLLC-5, ISLLC-6 Curriculium Design ISLLC-1, ISLLC-3, ISLLC-6 Measurement and Evaluation ISLLC-2, ISLLC-4, ISLLC-6 26 (81%) 6 (18%) 26 (81%) 6 (18%) 26 (81%) 6 (18%) 26 (81%) 6 (18%) Target Acceptable Unacceptable 3 EDAD 583 - School & Community Relations Performance Assessment – Fall 2012 In EDAD 583 candidates’ performance was assessed using a rubric that evaluates leadership, decision-making, extended learning opportunities, and volunteering. 91% of the candidates met the ISLLC standards. Content 45 (70%) Organization 45 (72%) Development 44 (72%) 12 (19%) 5 (8%) Use of Language 44 (72%) 12 (19%) 5 (8%) 7 pts. 5 pts. 3 pts 1 pt. EDAD 584 - School Supervision Performance Assessment – Winter 2013 Candidates’ performance in EDAD 584 was assessed using a rubric that evaluates leadership, problem analysis, instructional program and staff development, and legal regulatory application. 100% of the candidates met the ISLLC standards. Leadership ISLLC-1, ISLLC-2, ISLLC-3 Problem Analysis ISLLC-1, ISLLC-3, ISLLC-5, ISLLC-6 Instructional Program and staff Development ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4, ISLLC-5, ISLLC-6 Legal Regulatory Application ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4, ISLLC-5, ISLLC-6 7 (100%) 7 (100%) 7 (100%) 7 (100%) Target 13 (20%) 5 (7%) 1 (1%) 12 (19%) 5 (8%) Acceptable Unacceptable EDAD 586 – The Principalship Performance Assessment – Fall 2013 For EDAD 586 candidates were assessed using a rubric that evaluates vision for success, culture of teaching and learning, management of learning, relationship with the broader community to foster learning, integrity, fairness and ethics, and the political, social, economic, legal and cultural context of learning. 100% of the candidates met the ISLLC standards. A Vision for Success ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4, ISLLC-5, ISLLC-6 Culture of Teaching and Learning ISLLC-1, ISLLC-2 Management of Learning ISLLC-1, ISLLC-2, ISLLC-3 Relationships with the Broader Community to foster Learning ISLLC-1, ISLLC-2, ISLLC-3, ISLLC-4 Integrity, Fairness, and Ethics of Learning ISLLC-3, ISLLC-4, ISLLC-5 The political, Social, Economic, Legal and Cultural Context of Learning ISLLC-4, ISLLC-5, ISLLC-6 58 (100%) 58 (100%) 58 (100%) 58 (100%) 58 (100%) 58 (100%) Target Acceptable Unacceptable EDAD 589 - School Law Performance Assessment – Winter 2013 Candidates’ performance in EDAD 589 School Law was assessed using a rubric that evaluates state and federal policies, organizational oversight, school policies and procedures, and legal regulatory application. 66% of the candidates met the ISLLC standards at the exemplary level and 33% of the candidates were at the emerging level. Describe the Situation Relevant ISLLC Reflect the Situation Evaluate the Situation Connections 15 (17%) 65 (77%) 1 (50%) 1 (33%) 1 (50%) 1 (50%) Unsatisfactory 2 (66%) Emerging 1 (50%) 1 (50%) Proficient Exemplary EDAD 692/693 - Internship Performance Assessment – Fall 2011 Eight candidates’ portfolios in EDAD 692/693 were assessed using a rubric that evaluates their leadership vision, decision-making, volunteering, and extended learning opportunities. Candidates’ expectation of achievement is stated at the target level. 91% of the candidates met the ISLLC standards. School Vision of Learning Promote a Positive School Culture Sustaining Student learning Management of Organization Collaborating with Stakeholders Acts with Integrity Understand the larger Educational Context 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 9 (81%) 9 (81%) 9 (81%) 9 (81%) 9 (81%) 9 (81%) 9 (81%) Proficient 4 (4%) 1 (50%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) 1 (9%) Emerging Unsatisfactory EDAD 692/693 - Internship Performance Assessment – Winter 2013 Eight candidates’ portfolios in EDAD 692/693 were assessed using a rubric that evaluates their leadership vision, decision-making, volunteering, and extended learning opportunities. Candidates’ expectation of achievement is stated at the target level. 100% of the candidates met the ISLLC standards. Ethical Leadership ISLLC-7 Learner-Centered Leadership ISLLC-7 Human Resource and Management ISLLC-7 Communication and community Relations ISLLC-7 Instructional Leadership ISLLC-7 1 (25%) 3 (75%) 1 (25%) 3 (75%) 1 (25%) 3 (75%) 1 (25%) 3 (75%) 1 (25%) 3 (75%) Target Acceptable Unacceptable 5 EDAD 692/693 - Internship Performance Assessment – Spring 2013 Eight candidates’ portfolios in EDAD 692/693 were assessed using a rubric that evaluates their leadership vision, decision-making, volunteering, and extended learning opportunities. Candidates’ expectation of achievement is stated at the target level. 100% of the candidates met the ISLLC standards. Ethical Leadership ISLLC-7 Learner-Centered Leadership ISLLC-7 Human Resource and Management ISLLC-7 Communication and community Relations ISLLC-7 Instructional Leadership ISLLC-7 6 (100%) 6 (100%) 5 (83%) 1 (16%) 6 (100%) 5 (100%) Target Acceptable Unacceptable Pre-Post Self-Assessment Inventory Report Purpose To provide school leaders with the opportunity to examine their own school leadership practices with respect to the knowledge, dispositions, and performances contained within the ISLLC School Leader Standards. This self-examination should lead to greater familiarity with the Standards and provide a starting point to assist the candidate in identifying potential areas of focus for professional development planning. The Pre-Autumn internship orientation was held on July 27, 2012 and 8 candidates from the Principal Residency Program attended the orientation. During the orientation, a self-inventory designed to provide a personal profile of the school leadership assets based on the ISLLC Standards for School Leaders was given to the candidates. The self-inventory consists of statements that describe the knowledge, dispositions, and performances contained within the ISLLC Standards for School Leaders. The candidates were asked to respond to each statement by reflecting on what they have learned, what they believe and value, and what they are accomplishing as a school leader. Data collected was analyzed using SPSS. The results of the pre-evaluation are shown in Table 1. During the final internship meeting held April 26th, 2013 the same self-inventory survey was given to the candidates again. Eight of the eight candidates completed the self-inventory. These results are in Table II. Note: There is a direct correlation in ID’s from Table I to Table II. Based on the instrument, the principal candidates were to evaluate themselves on the following scale: 1 represents Little extent; 2 represents Some extent; 3 represents Sufficient extent; 4 represents Exemplary extent. Table 1 Pre-Evaluation Below is the average score of the individual candidates on the 31 items pre self-inventory. Entry fall, 2012 ID 1 2 3 4 5 6 7 8 Group Ave. ISLLC ISLLC 1 2 1.6 1.56 3 2.89 1.8 2 2 2.67 2.2 2.56 2.25 2.67 2.4 2.33 3 2.33 2.28 2.38 ISLLC ISLLC ISLLC ISLLC 3 4 5 6 1.8 1.75 2 2.34 2.8 2.5 2.8 2.67 1.6 1.75 2.2 2 1.2 1.25 1.8 2.33 3.4 3.75 3.8 3.5 2.4 2.5 3 2.67 1.6 1.75 2.4 2 2.2 1.75 3 2.33 2.13 2.13 2.63 2.48 Individual Average 1.84 2.78 1.90 1.90 3.2 2.6 2.1 2.45 2.34 Table II Post-Evaluation Below is the average score of the individual candidates on the 31 items post self- inventory. Exit spring 2013 ID 1 2 3 4 5 6 7 8 Group Ave. ISLLC ISLLC 1 2 3.1 3.61 4 4 4 4 3.6 3.72 4 4 3.6 3.67 3.8 3.78 4 3.44 3.76 3.78 ISLLC ISLLC ISLLC ISLLC 3 4 5 6 2.9 3.25 3.3 3.3 4 4 4 4 4 4 4 4 3.6 3.5 3.4 3.3 4 4 4 4 4 3.75 4 3.67 3.4 3.75 4 3.3 3.4 3.25 3.6 3 3.66 3.69 3.79 3.58 Individual Average 3.25 4 4 3.53 4 3.78 3.68 3.45 3.71 Comparing the group average of pre-evaluation, fall, 2012 (2.34) and post evaluation, spring 2013, (3.71), it can be concluded that there is a significant difference in the self-inventory. The candidates perceived themselves as prepared to work in the field of school administration. Mid-Quarter Measurement of Knowledge & Skills Standard V To determine whether the 2012-13 interns are having a positive impact on students learning, we adopted Standard 5 Knowledge and Skills statements to assess their leadership abilities. Assessed Spring 2013, first 7 by the candidates and then two weeks later by their building supervisors, they were asked to “Choose how frequently you engaged in the behavior described by the statement during the internship experience” with a scale of 1= Never, 2 = Seldom, 3 = Occasionally, 4 – Often and 5 = Always. Table VI Creating a Culture: Influence, establish and sustain a school culture conducive to continuous improvement for students and staff: 1 - Articulate and model a shared belief vision for continuous improvement and lifelong learning within the entire school. Q2 Promote and model an atmosphere of inclusiveness, equity and respect among students, staff and community. Q3 - Develop and model an atmosphere of personal responsibility, trust and collaboration among students and staff. Q4 - Understand the needs and strengths of the individuals and groups in the school. Q5 - Use authentic assessment data, including self-appraisal, to measure culture and inform action of individuals, group and the school system. Candidates Mean Scores Supervisors Mean Scores 4.63 5.00 4.75 5.00 4.88 4.67 4.63 4.67 4.38 4.33 Ensuring School Safety: Lead the development and annual update of a comprehensive safe schools plan that includes prevention, intervention, crisis response and recovery. Q6 - Supervise teachers’ instructional practices, classroom management and discipline to ensure a safe classroom environment. Q7 - Develop open communication systems that allow for proactive identification and intervention of potential incidents. Q8 - Communicate so that students, parents and community members are confident the school is safe. 4.57 4.00 4.43 4.67 4.29 5.00 Planning with Data: Lead the development, implementation and evaluation of the data-driven plan for improvement of student achievement: Q9 - Clearly communicate the elements of the school improvement plan with constituent groups (staff, parents, board and community). Q10 - Collaborate, problem-solve and build consensus with individuals and groups. 4.50 4.00 4.63 5.00 Aligning Curriculum: Assist instructional staff in aligning curriculum, instruction and assessment with state and local learning goals: Q11 - Use a variety of assessments to diagnose learning needs and to design appropriate learning interventions to improve success on state, local and classroom measures. 4.75 4.67 Q12 - Assist in adjusting to and accepting curriculum, instruction and assessment changes. 4.63 4.67 Q13 - Engage instructional staff in frequent conversations/reflection about classroom practice. 4.63 4.33 Improving Instruction: Monitor, assist and evaluate staff implementation of the school improvement plan, effective instructing and assessment practices. Q14 - Engage each staff member in their appropriate implementation of 4.13 the School Improvement Plan (SIP). 4.00 Q15 - Use a variety of data to inform and support decisions about instruction and assessment of learning. 4.33 4.38 Managing Resources: Manage human and fiscal resources to accomplish student achievement goals: Q16 - Possess knowledge of laws related to special areas (e.g. ASB, special programs, grants). 4.38 4.33 Q17 - Have knowledge of effective practices as well as rules and regulations related to staff supervision, coaching and evaluation. 4.88 4.67 Q18 - Effectively address difficult issues and conduct difficult discussions. 4.63 5.00 Engaging Communities: Communicate and partner with school community members to promote student learning: Q19 - Partner and collaborate with administrative colleagues. 4.50 5.00 Q20 - Share leadership based upon situation and need. 4.50 5.00 Q21 - Use a variety of vehicles and tools to communicate about student learning with constituents, both within and outside of the school. 4.38 4.00 Q22 - Gather and use relevant data regarding community responses to student learning and their attitudes about the school. 4.00 3.33 Based on the results from this mid-year survey, the internship candidates were able to participate in all of the 22 components of Standard V. The candidates’ responses indicate that they were engaged in Standard V activities Always (60.86%), Often (28.3%), Occasionally (10.6%), and Seldom (3.1%). The candidates building supervisors responses indicate that they were engaged in Standard V activities Always (66.4%), 9 Often (24.7%), Occasionally (1.6%), Seldom (6%) and Never (1.1%). It can be concluded that the interns are having a positive impact in their place of assignment. Building Supervisors’ quarterly evaluation of Principal Interns At the end of each quarter, the building principals are required to evaluate the interns on the ISLLC standards. The evaluation instrument has 31 variables, with a Likert scale of 1 to 4, and 4 being the highest score. The ISLLC data was collected from building supervisors in Pre-Autumn, fall, winter and spring. We can generally conclude from the result that the principal interns performed above average. See Table VII below. Table VII ISLLC Pre-Autumn Fall Winter Spring 1 2.78 3.11 3.7 3.9 2 2.94 3.20 3.63 3.85 3 2.63 3.11 3.39 3.55 4 2.79 3.30 3.53 3.75 5 3.08 3.46 3.6 3.8 6 2.77 3.38 3.46 3.5 Assessment of Online Learning Spring 2013, Advanced Programs offered three courses in the completely online format and one as a hybrid class that met 50% of the time and 50% of the coursework was to be completed online. Mid-quarter assessment of Online Teaching, Social, & Cognitive Presence was administered to each student in each class. The results are as follows: The following statements relate to your perceptions of “Teaching Presence” – you instructor’s course design, facilitation of discussion and direct instruction – in the course. Please indicate both your agreement or disagreement with each statement and how important you think it is. 3.27 The instructor provided feedback in a timely fashion. The instructor provided feedback that helped me understand my strengths and weaknesses relative… 4.24 The instructor helped to focuse discussion on relevant issues in a way that helped me to learn. 4.27 Instructor actions reinforced the development of a sense of community among course participants. 4 The instructor encouraged course participants to explore new concepts in the course. 4.18 The instructor helped keep the course participants on task in way that helped me to learn. 4.09 The instructor helped to keep course participants engaged and participating in productive dialogue. 4.09 The instructor was helpful in guiding the class towards udnerstanding course topics in a way that… 4.05 The instructor was helpful in identifying areas of agreement and disagreement on course topics th… 4 The instructor clearly communicated important due dates/time frames for learning activities. 4.09 The instructor provided clear instructions on how to participate in course learning activities. 4.1 4.18 The instructor clearly communicated important course goals. 4.32 The instructor clearly communicated important course topics. 0 1 2 3 4 5 11 The following statements relate to your perceptions of “Social Presence” – the degree to which you feel socially and emotionally connected with others – in your course. Please indicate both your agreement and disagreement with each statement and how important you think it is. Online discussions help me to develop a sense of collaboration. 4 I felt that my point of view was acknowledged by other course participants. 4.23 I felt comfortable disagreeing with other course participants while still maintaining a sense of trust. 4.24 I felt comfortable interacting with other course participants. 4.36 I felt comfortable participating in the course discussions. 4.36 I felt comfortable conversing through the online medium. 4.36 Online or web-based communication is an excellent medium for social interaction. 4.36 I was able to form distinct impressions of some course participants. 3.91 Getting to know other course participants gave me a sense of belonging in the course. 3.91 0 1 2 3 4 5 The following statements relate to your perceptions of “Cognitive Presence” – the extent to which you are able to construct and confirm meaning – in this course. Please indicate both your agreement and disagreement with each statement and how important you think it is. I can apply the knowledge / created in this course to my work or other non-class related activities. 4.32 I have developed solutions to course problems / that can be applied in practice. 4.05 I can describe ways to test and apply the / knowledge created in this course. 4.23 Reflection on course content and discussions / helped me understand fundamental concepts in this class... 4.09 Learning activities helped me construct / explanations / solutions. 4 Combining new information helped me answer / questions raised in course activities. 4.27 Discussing course content with my classmates was / valuable in helping me appreciate different perspec... 4.41 Brainstorming and finding relevant information / helped me resolve content related questions. 4.18 I utilized a variety of information sources to / explore problems posed in this course. 4.23 I felt motivated to explore content related / questions. 4.05 Course activities piqued my curiosity. 3.95 Problems posed increased my interest in course / issues. 4.05 0 1 2 3 4 5 13 3. What was learned? Livetext Portfolio: Based on the rubrics created for each of the courses on LiveText, it can be concluded that 100% of the faculty members are repeatedly evaluating candidates’ portfolios. All the course portfolios reviewed show that the majority of students met the standards. The reason for the candidates meeting the 100% is due to the fact that students are allowed many opportunities to demonstrate competence in each of the ISLLC standards and that they are measured repeatedly. Mid-Quarter Measurement of Knowledge & Skills: The mean for this outcome was four and above for both building supervisors and interns. The data results showed continuing progress since the 2011-12 assessment. The result was shared with the interns for their input, their response was satisfactory. The knowledge and skills results reflected in the interns’ Professional Growth plan as assessed by their building supervisors and university supervisor showed 95% of the PGPs met the target. The ISLLC standards: The assessment rubrics on LiveText met the expected outcomes. All portfolios were reviewed two or more times to achieve the target variables mentioned for student success. Pre-Post Self-Assessment Inventory Report on the ISLLC standards: This met the expected outcome as shown above. Assessment of online learning: The data results for the online learning -“Teaching Presence” is 4.07, “Social Presence” is 4.19, and “Cognitive Presence” is 4.15 out of 5 points. The online learning survey results showed a significant difference when compared to 2011-12 “Teaching Presence” 3.78, “Social Presence” – 3.87, and “Cognitive Presence” – 3.77 results. As the college classroom changes, we have the opportunity to monitor and modify the teaching/learning process within the classroom. 4. What will the department or program do as a result of that information? The department continues to revise the six broad standards of the ISLLC standards that represent the most essential expectations for today’s school administrators. With the ELCC (Educational Leadership Constituent Council) the program is referencing course rubrics and syllabi in teaching and learning. 5. What did the department or program do in response to last year’s assessment information? All assessment methods are described in terms of population assessed, number assessed, and survey response rate. 6. Questions or suggestions concerning Assessment of Student Learning at Central Washington University: None at this time.