Exhibit 2.4.g.25 School Psychology MS & CAS Comprehensive Data Analysis Report (DAR) Summary 2011-2013 AY 12 1. Have the change(s) in response to data that you documented last year had the desired effect on your program? Please provide specifics referencing prior changes that you submitted in [AY 2010-2011]. Goal: Develop more clearly delineated rubrics for grading intern portfolios to increase scoring reliability and clarify expectations: Results of Change: Using the new rubrics assisted interns in the design of their required case studies and other projects and enhanced the final products. Number of individuals exceeding expectations increased from prior years. Goal: Align/reinforce curriculum across related courses. Results of change: Dr. Bartels took over teaching PSYC 625 Functional Behavioral Assessment (from adjunct faculty) based on student feedback that the course did not address issues facing school psychologists. Course was aligned with PSYC 761. Both courses received high ratings; students were better prepared for next course in series: PSYC 651 taught summer 2012. Goal: Based on Practicum supervisor feedback in PSYC 773 (Domain 2.2) plus internship supervisor feedback, feedback– candidates need to increase practice in facilitating teams. Results of Change: Added “mandated” goal on internship plan starting Fall 2011 related to effective team participation and taking leadership role on a team (IEP, SST, etc); increase practice during seminar for practicum. Saw an increase in the following two items on Practicum Supervisor Ratings Scale (on scale of 1-5): Contributes actively and effectively during team meetings 4.36 Takes a beginning role facilitating team meetings 4.17 2. What significant findings emerge from your examination of these data? Diversity: Need to better assess Diversity in portfolio rubrics; need to better document candidate involvement with K-12 students from diverse backgrounds per NASP National Recognition Report Internship: Need to better assess intern professional dispositions; continue emphasizing team participation in internship; need to improve consistency in reporting supervision hours and hours spent in consultation during internship - while every candidate exceeded the required number of hours, numbers reported varied greatly, suggesting inconsistency in what is being defined as supervision and consultation. Practicum: Continue emphasizing team participation in practicum based on supervisor feedback All students chose Comps question related to identifying LD versus ED; is this reflective of a lack of confidence in making the ED determination? 3. How have you involved faculty in your identification of the implications of these data? Data from all six assessments shared and analyzed at annual summer retreat. Specific activities for each faculty member delineated; progress will be reviewed throughout the 2012-2013 year at monthly faculty meetings and during Summer 2013 retreat. 4. What specific actions will you take in response to these data? Diversity: Collect data on candidate involvement with K-12 students from diverse backgrounds at practicum and intern levels Add rubric items assessing competency in this area to portfolio rubrics Internship: Operationalize “consultation” with interns to increase reliability in reporting- this is necessary so that we can ascertain that candidates are engaging in formal consultation activities with school staff and families Clarify what is meant by “supervision” during internship to increase reliability of data Add Professional Dispositions to Internship Supervisor Evaluation Form Continue requiring specific item in Internship Plan related to team participation Practicum: Expand coverage of ED definition to ensure adequate competency AY 13 1. Have the change(s) in response to data that you documented last year had the desired effect on your program? Please provide specifics referencing prior changes that you submitted in [AY 2011-2012]. Goal based on 2011-2012 data: Emphasize team participation in internship Results of change, measured by data collected 2012-2013: For the past two years (2010-2011; 20112012), ratings from practicum and internship field supervisors have averaged 3.2 on a scale of 1-4 on items that assessed candidates’ ability to participate meaningfully in team meetings (both general education and special education). Candidates, themselves, identified this as an area of weakness during class discussions. As a result, additional simulations of team meetings were instituted during practicum seminar and all second year candidates were required to develop a personal goal, using Goal Attainment Scaling, at the end of practicum related to team participation during internship. Ratings on all items related to team participation on the internship field supervisor rating form increased and all were at or above 3.8 on a scale of 1-4. Goal based on 2011-2012 data: Diversity: Better assess diversity in portfolio rubrics; better document candidate involvement with K-12 students from diverse backgrounds Results of Change, measured by data collected 2012-2013: Disaggregated data from portfolio counseling case rubrics related to candidates’ ability to reflect on the impact of a student’s background on his or her development and outcomes of counseling indicated that, while generally satisfactory, it was a weaker area. This is based on data indicating that 3 of 14 candidates earned ratings of “Needs Development” on this item. Because we had set the goal to add this item to our rubric, we were able to identify this as an area in need of further attention. (Please see goals for 2013-2014 for plans to address this issue). We were also able to determine the extent to which interns engaged with students from diverse backgrounds and were pleased that our interns’ involvement with k-12 students who were African-American was greater than involvement with Caucasian (non-Hispanic) students. This is an accurate reflection of our state’s demographics and reflects that our interns are obtaining necessary experiences with students from diverse backgrounds. Goal: Need to improve consistency in reporting supervision hours and hours spent in consultation during internship - while every candidate exceeded the required number of hours, numbers reported varied greatly, suggesting inconsistency in what is being defined as supervision and consultation. Results of change: A form was developed, disseminated, and reviewed that describes activities completed during internship (including what constitutes supervision) in observable terms. As a result, data across candidates appeared more consistent and thus more accurate and reliable. This allows us, as a faculty, to ensure that candidates have the opportunity to practice the full range of responsibilities of a school psychologist during their internship. 2. What significant findings emerge from your examination of these data? Data collected across assessments in the 2012-2013 academic year suggest improvement needed in the following areas: Develop increased competency in selecting appropriate assessment instruments when conducting psychological evaluations during practicum (data source: rating of 3.45 on a scale of 1 to 4 on Field Supervisor Practicum Evaluation) Develop increased competency in determining if students meet IDEA criteria for emotional disability and learning disability (data source: 3 ratings of Fail on Comprehensive Exam question related to ED; 3 ratings of “weak pass” on exam question related to LD) Need to increase focus on legal/ethical issues in spite of excellent field supervisor ratings in this area (data source: lowest area on Praxis exam, although all passed, was in this area) Need to increase awareness of how to address needs related to students’ race/ethnicity/SES during counseling (data source: lower ratings on this item on Portfolio case studies). 3. How have you involved faculty in your identification of the implications of these data? Data from all six assessments was shared and analyzed at annual summer retreat. Specific activities for each faculty member delineated; progress will be reviewed throughout the 2013-2014 year at monthly faculty meetings and during summer 2014 retreat. 4. What specific actions will you take in response to these data? Develop increased competency in selecting appropriate assessment instruments when conducting psychological evaluations during practicum o Action: Inform practicum supervisors of need to allow candidates to do this with feedback and on their own; present more case scenarios in practicum seminar Develop increased competency in determining if students meet IDEA criteria for emotional disability and learning disability o Mandate that at least one case during practicum be an initial evaluation of a student suspected of having LD; meet with instructor for Social/Emotional Assessment course to discuss findings and brainstorm solutions; continue to review LD and ED criteria in practicum seminar Need to increase focus on legal/ethical issues in spite of excellent field supervisor ratings in this area o More case studies to be presented in PSY 790; make exam in ethics course more similar to what is expected on Praxis (e.g. multiple choice versus essay to cover more topics) Need to increase awareness of how to address needs related to students’ race/ethnicity/SES during counseling o Full-time faculty is taking over Multicultural Counseling course from adjunct instructor