Central Washington University Assessment of Student Learning Department and Program Report Please enter the appropriate information concerning your student learning assessment activities for this year. Academic Year of Report: 2010-2011 Department: Advanced Programs College: CEPS Program: School Administration 1. What student learning outcomes were assessed this year, and why? The student learning outcomes assessed this year are as follows: 1) leader promotes the success of every student by facilitating the development, articulation, implementation, and stewardship of a vision of learning that is shared and supported by all stakeholders. 2) leader promotes the success of every student by advocating, nurturing and sustaining a school culture and instructional program conducive to student learning and staff professional growth. 3) leader promotes the success of every student by ensuring management of the organization, operation, and resources for a safe, efficient, and effective learning environment. 4) leader promotes the success of every student by collaborating with faculty and community members, responding to diverse community interests and needs, and mobilizing community resources. 5) leader promotes the success of every student by acting with integrity, fairness, and in an ethical manner. 6) leader promotes the success of every student by understanding, responding to, and influencing the political, social, economic, legal, and cultural context. These outcomes are related to College goals II, V, University goals and I, II, V, and VI, and Washington Standard-based benchmarks. 1 2. How were they assessed? A) What methods were used? The methods used to assess candidates are: 1) Livetext Portfolio (M.Ed. & Residency Certificate) 2) Pre and Post Self-Inventory Assessment (Completed by Principal Interns) 3) Mid-Quarter Measurement of Knowledge & Skills Completed by (Principal Interns) 4) Building Supervisors’ quarterly evaluation of Principal Interns. B) Who was assessed? All candidates in the School Administration and Program Administrator Certification programs were assessed based on the Interstate School Leaders Licensure Consortium (ISLLC) and the knowledge and skills standards. C) When was it assessed? All of the students in the school administration program were assessed in 2010-2011 academic year. Their LiveText portfolios were reviewed by all educational administration faculty members, summarized, and discussed. A pre and post selfassessment inventory on the ISLLC standards was completed by the interns. A survey on standard V – Knowledge and Skills was also administered to the interns. See results below. Candidates’ portfolios in EDAD 580 were assessed using a rubric that evaluates their leadership, decision-making, volunteering, and extended learning opportunities. Candidates’ expectation of achievement is stated at the target level. Based on data results, 60% of the candidates met the expectation at the target level while 40% met the expectation at the acceptable level. Candidates’ portfolio in EDAD 581 were assessed using a rubric that evaluates their leadership and student learning, management and effective learning environment, collaboration and mobilizing community resources, and demonstrating integrity and ethics. Data results show that 97% of the candidates met the expectations at the target level, and 3% at an acceptable level. Candidates’ performance in EDAD 582 was assessed using a rubric that evaluates leadership, information collection and problem analysis, curriculum design, and measurement and evaluation on the course rubric. Based on data results, 14% of the candidates met the expectations at the target level, 80% at the acceptable level, and 6% at the unacceptable level. 2 In EDAD 583 candidates’ performance was assessed using a rubric that evaluates leadership, decision-making, extended learning opportunities, and volunteering. Based on data results 58% of the candidates met the expectations at the target level, and 42% at the acceptable level. Candidates’ performance in EDAD 584 was assessed using a rubric that evaluates leadership, problem analysis, instructional program and staff development, and legal regulatory application. Based on data results, 79% of the candidates met expectations at the target level, and 21% at the acceptable level. For EDAD 586 candidates were assessed using a rubric that evaluates vision for success, culture of teaching and learning, management of learning, relationship with the broader community to foster learning, integrity, fairness and ethics, and the political, social, economic, legal and cultural context of learning. Based on the data results, 14% of the candidates met the expectations at the target level, and 86% at the acceptable level. Candidates’ performance in EDAD 589 School Law was assessed using a rubric that evaluates state and federal policies, organizational oversight, school policies and procedures, and legal regulatory application. Based on data results, 93% of the candidates met expectations at the target level, and 7% at the acceptable level. For EDSE 512 candidates were assessed using a rubric that evaluates State and Federal policies, organizational oversight, school policies and procedure, and legal regulatory application. Based on the data results, 75% of the candidates met the expectations at the target level, and 25% at the acceptable level. Pre-Post Self-Assessment Inventory Report Purpose To provide school leaders with the opportunity to examine their own school leadership practices with respect to the knowledge, dispositions, and performances contained within the ISLLC School Leader Standards. This self-examination should lead to greater familiarity with the Standards and provide a starting point to assist the candidate in identifying potential areas of focus for professional development planning. The Pre-Autumn internship orientation was held on July 30, 2010 and 10 candidates from the Principal Residency Program attended the orientation. During the orientation, a selfinventory designed to provide a personal profile of the school leadership assets based on the ISLLC Standards for School Leaders was given to the candidates. The self-inventory consists of statements that describe the knowledge, dispositions, and performances contained within the ISLLC Standards for School Leaders. The candidates were asked to respond to each statement by reflecting on what they have learned, what they believe and 3 value, and what they are accomplishing as a school leader. Data collected was analyzed using SPSS. The results of the pre-evaluation are shown in Table 1. During the final internship meeting held April 29th, 2011 the same self-inventory survey was given to the candidates again. All 10 candidates completed the self-inventory. These results are in Table II. Note: There is a direct correlation in ID’s from Table I to Table II. Based on the instrument, the principal candidates were to evaluate themselves on the following scale: 1 represents Little extent; 2 represents Some extent; 3 represents Sufficient extent; 4 represents Exemplary extent. Table 1 Pre-Evaluation Below is the average score of the individual candidates on the 31 items pre self-inventory. Entry fall, 2010 Id ISLLC 1 1 2 3 4 5 6 7 8 9 10 Group Ave. 1.8 2 2.5 1.8 2.2 1.9 2.8 2.4 3 2.6 2.3 ISLLC 2 1.33 2.33 2.33 2.22 1.44 1.83 1.78 2.11 2.22 2.56 2.02 ISLLC 3 1.4 2.4 2 1.6 1.2 1.2 1.2 2 2.6 3 1.86 ISLLC 4 1.25 2.25 2.25 1.75 1.25 1 2.25 3 3 3.25 2.13 ISLLC 5 1.8 2.6 3 2.2 1.6 1.2 2.8 3 3 3.4 2.46 ISLLC 6 1 2.33 2 1.33 1.67 1.33 2.33 2.67 2.67 2.67 2.0 Individual Average 1.43 2.32 2.35 1.82 1.56 1.41 2.20 2.53 2.75 2.91 2.13 4 Table II Post-Evaluation Below is the average score of the individual candidates on the 31 items post self- inventory. Exit spring 2011 Id ISLLC 1 1 2 3 4 5 6 7 8 9 10 Group Ave. 4 3 3.8 3 3.9 3.2 3.7 3.9 3 3.6 3.5 ISLLC 2 3.7 3 3 3.7 4 3.3 3.4 3.7 3 3.9 3.5 ISLLC 3 3.6 3 2.8 3.8 4 3.6 3.6 3.8 3 3.8 3.5 ISLLC 4 4 3 3 4 4 2.8 3.8 3.8 3 4 3.5 ISLLC 5 4 3 3.4 3.8 4 3.2 4 3.8 3.4 4 3.7 ISLLC 6 3.7 3 3.7 3.3 4 3.7 3.8 3.7 3.3 4 3.6 Individual Average 3.8 3 3.3 3.6 4 3.3 3.7 3.8 3.1 3.9 3.6 To determine whether there is a significant difference between the pre self-evaluation and post self-evaluation on the knowledge, dispositions, and performances on the ISSLC standards, a paired-sample t test was used to compare the means of the scores from the related samples. The assumption for using the paired-sample t test is that both variables are the interval levels and are normally distributed. The pre and post self-evaluations individual mean scores were entered in a SPSS data format for analysis. Table III Paired Samples Statistics Mean N Std. Deviation Std. Error Mean Entry 2.1280 10 .54483 .17229 Exit 3.5500 10 .35040 .11081 Pair 1 Table IV Paired Samples Correlations N Pair 1 Entry & Exit Correlation 10 -.187 Sig. .604 5 Table V Paired Samples Test Paired Differences 95% Confidence Interval of the Difference Mean Std. Std. Error Deviation Mean Lower Upper Sig. t df (2-tailed) Entry Pair 1 -1.42200 .70084 .22162 -1.92335 -.92065 -6.416 9 .000 Exit Reading the output A paired sample t test was calculated to compare the mean entry score to the mean exit score. The mean on the entry score was 2.13 (sd = .54), and the mean on the exit was 3.55, (sd =.35). A significant increase from entry to exit was found (t(9) = - 6.461, p < 001. See Tables above. At the end of each quarter, the building principals are required to evaluate the interns on the ISLLC standards. The evaluation instrument has 31 variables, with a Likert scale of 1 to 4, and 4 being the highest score. The results of the evaluation were entered into SPSS for a descriptive statistics analysis. “N” represents pre-autumn, fall, winter, and spring. Table VI Building Supervisors’ quarterly evaluation of Principal Interns Descriptive Statistics N Minimum Maximum Mean Std. Deviation Isllc1 4 2.80 3.70 3.2000 .39158 Isllc2 4 2.70 3.50 3.1250 .35000 Isllc3 4 2.60 3.60 3.1250 .42720 Isllc4 4 2.70 3.50 3.1500 .36968 Isllc5 4 3.30 3.70 3.5500 .17321 Isllc6 4 2.80 3.60 3.2000 .32660 Valid N (listwise) 4 We can generally conclude from the result that the principal interns performed above average. See Table VI above. 6 Mid-Quarter Measurement of Knowledge & Skills Standard V To determine whether the 2010-11 interns are having a positive impact on students learning, we adopted Standard 5 Knowledge and Skills statements to assess their leadership abilities. Assessed Spring 2011 the candidates were asked to “Choose how frequently you engaged in the behavior described by the statement during the internship experience:” The survey was completed through Qualtrics with the responses as follows: Creating a Culture: Influence, establish and sustain a school culture conducive to continuous improvement for students and staff: Table VII Table VIII Table VII Responses: Always (20%), Often (40%), Occasionally (40%) Table VIII Responses: Always (50%), Often (50%) Ensuring School Safety: Lead the development and annual update of a comprehensive safe schools plan that includes prevention, intervention, crisis response and recovery: Table IX Table IX Responses: Always (10%), Often (50%), Occasionally (10%), Seldom (30%) Table X Table X Responses: Always (40%), Often (60%) Table XI Table XI Responses: Always (70%), Often (30%) 7 Planning with Data: Lead the development, implementation and evaluation of the datadriven plan for improvement of student achievement: Table XII Table XIII Table XII Responses: Always (20%), Often 40%), Occasionally (40%) Table XIII Responses: Always (50%), Often 50%) Aligning Curriculum: Assist instructional staff in aligning curriculum, instruction and assessment with state and local learning goals: Table IXV Table XV Table IXV Responses: Always (40%), Often (50%), Occasionally (10%) Table XV Responses: Always (30%), Often (40%), Occasionally (30%) Table XVI Table XVI Responses: Always (40%), Often (40%), Occasionally (20%) 8 Improving Instruction: Monitor, assist and evaluate staff implementation of the school improvement plan, effective instruction and assessment practices: Table XVII Table XVIII Table XVII Responses: Often (80%), Occasionally (10%), Never (10%) Table XVIII Responses: Always (20%), Often (60%), Occasionally (20%) Managing Resources: Manage human and fiscal resources to accomplish student achievement goals: Table IXX Table IXX Responses: Always (20%), Often (60%), Occasionally (20%) Table XX Table XX Responses: Always (20%), Often 60%), Occasionally (20%) Table XXI Table XXI Responses: Always (20%), Often (50%), Occasionally (30%) 9 Engaging Communities: Communicate and partner with school community members to promote student learning: Table XXII Table XXIII Table XXII Responses: Always (60%), Often (40%) Table XXIII Responses: Always (50%), Often (50%) Table IXXV Table IXXV Responses: Always (40%), Often (60%) Table XXV Table XXV Responses: Always (30%), Often (20%), Occasionally (30%), Seldom (10%) Based on the results from this mid-year survey, the internship candidates were able to participate in all but one of the 19 components of Standard V. The candidates responses indicate that they were engaged in Standard V activities Always (38.4%), Often (45.3%), Occasionally (14.2%), Seldom (2%), and Never (.5%). It can be concluded that the interns are having a positive impact in their place of assignment. 3. What was learned? Livetext Portfolio: Based on the rubrics created for each of the courses on LiveText, it can be concluded that 100% of the faculty members are repeatedly evaluating candidates’ portfolios. 100% met the standards. The reason for the candidates meeting the 100% is due to the fact that students are allowed many opportunities to demonstrate competence in each of the ISLLC standards and that they are measured repeatedly. The ISLLC standards: The assessment rubrics on LiveText met the expected outcomes. All portfolios were reviewed two or more times to achieve the target variables mentioned for student success. Pre-Post Self-Assessment Inventory Report on the ISLLC standards: This met the expected outcome as shown in Table V. 10 Mid-Quarter Measurement of Knowledge & Skills –Standard V: Although the expected outcomes are met on the Knowledge and skills standards, the results in Table IX were closely reviewed by faculty members and PEAB. Since 30% of the candidates reported they are seldom offered the opportunity to interact with the teacher in instructional practice, classroom management, and discipline to ensure a safe classroom environment, some interventions was considered on the part of the supervisors. Building supervisors were informed of the findings, and they agreed to have the candidates sit in during teacher observations. The results in Table XXI – XXV “engaging communities: Communicate and partner with school community members to promote student learning is impressive. The candidates are engaged in working with diverse communities. 4. What will the department or program do as a result of that information? The department plan is to continue assessing program goals and their impact on student learning. The department, in collaboration with the School Administration PEAB, decided to revise the syllabus for all the courses in the program, and align its objective and rubrics with the ISLLC standards. This document will be presented to PEAB at our February 3, 2012 meeting for review and recommendation. This assessment report will be distributed to department faculty and discussed at our faculty meeting on December 9, 2011. In case of any concerns arising from the discussion, faculty will be asked to address the issue and report their findings at our next faculty meeting in 2012. 5. What did the department or program do in response to last year’s assessment information? The department continued to revise the six broad standards of the ISLLC standards that represent the most essential expectations for today’s school administrators. The program is referencing in the course rubrics and syllabi how the standards are addressed. The department has had a renewed focus on diversity in all the courses. Students were required to download the rubric from Livetext for class discussion. Their input for course delivery and assessment was requested. Suggestions from students to offer online or hybrid courses have been implemented. 6. Questions or suggestions concerning Assessment of Student Learning at Central Washington University: None at this time. 11