M.S. & Ed.S. in School
Psychology
Assessment in the Major Report
By Dr. Christine Peterson, Program Director
2014-15
Submitted: October 2015
Table of Contents
Description of Methods ..........................................................................................................................................................................................2
Program Disposition Review..................................................................................................................................................................................3
Practicum and Internship Student Evaluations .......................................................................................................................................................5
PRAXIS II: Content Test Summary or National School Psychologist Exam Results .........................................................................................10
PRAXIS II: Content Test Summary .....................................................................................................................................................................11
Praxis II First Time Test Takers ...........................................................................................................................................................................13
Other Information .................................................................................................................................................................................................14
Plan for Communicating Assessment Results ......................................................................................................................................................15
Conclusions and Action Plan to Improve Teaching and/or Advisement in the School Psychology Program .....................................................16
Goals for 2014-15 .................................................................................................................................................................................................17
School Psychology AIM Report 2014-15
Page 1
Description of Methods
The following assessment tools and strategies were utilized to determine student performance, progress, and levels of satisfaction in the
major:
Benchmark I
 Faculty review/selection of program applicants
 Transcripts review
 Acceptance into the School Psychology Program
Benchmark II
 Initial portfolio review
 Student disposition reviews
 Review of practicum I evaluations
 Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct
Benchmark III
 Second portfolio review
 Student disposition reviews
 Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct
 Review of practicum II evaluations
 Student scores on the National Certification in School Psychology Exam (NCSP) or Praxis II
Benchmark IV
 Survey of interns regarding their perceptions of program competencies and coursework
 Final portfolio review
 Exit Student Survey
2010-2014 Enrollment Data
Year
Enrolled
2014
14
2013
8
2012
11
2011
12
2010
10
Completed
10
12
10
12
10
School Psychology AIM Report 2014-15
Page 2
Program Disposition Review
Disposition reviews are conducted at Benchmark II (end of first year in the program) and III (mid-term during second year in the program)
for all school psychology students. Domain scores rated as: 1 (unsatisfactory), 2 (minimal), 3 (satisfactory) to 4 (proficient). Total
composite scores range from ≤ 15 (unsatisfactory) to 32 (proficient).
Figure 1 – Comparative Benchmark II Means
School Psychology AIM Report 2014-15
Commented [PC1]: Can we cut both Disposition Tables to 3
most recent years comparison and put most recent year at the top
(just to be consistent across all tables)?
Page 3
Figure 2 – Comparative Benchmark III Means
School Psychology AIM Report 2014-15
Page 4
Outcome of the Disposition Review Results
As a whole, the 2014-15 school psychology students displayed satisfactory to proficient attitudes and behaviors within the program (see Fig.
1 and 2). Scores from 2014-15 Benchmark II data exceeded an average of 3.6 (out of 4.0) or higher across all domains. Benchmark III data
averaged 3.7 (out of 4.0) or higher across all domains.
Conclusion(s):
Benchmark II and III Dispositions allow faculty and students to engage in reflective dialogue regarding each student’s progress and
development toward the mastery of critical skills necessary for success as a grad student and as a professional. Data from 2014-15 suggests
that students are progressing appropriately in these important skills domains.
Practicum and Internship Student Evaluations
Graduate students in the school psychology program complete two (2) supervised field practica totaling 600 hours during their second year
in the program, and a culminating 1200 hour internship that is their third and final year in the program. Graduate students completing both
Practicum I and Practicum II are evaluated through the use of The Field Practicum Evaluation tool. This newly revised tool utilizes the
same items, along a developmental scale, in order to demonstrate and review growth as students advance through each experience. Scoring
metrics for each tool are as follows:
Unestablished- Observer1
2
Practicum II
Observer2
Novice-3
Novice-3
Internship
School Psychology AIM Report 2014-15
Advanced
Novice-4
Practicum I
Advanced
Novice-4
Emerging
Practitioner-5
Advanced
Novice-4
Emerging
Practitioner-5
Established
Practitioner-6
Advanced
Practitioner-7
Page 5
Outcomes of the Practicum and Internship Student Evaluations
a) Practicum Evaluations
Results from the Fall 2014 evaluations indicate Practicum I students area scores (means) ranged from 2.9 (Preventive and Responsive
Services) to 3.6 (Data-Based Decision Making and Accountability) on the 4-point developmental scale Evaluation Guide for School
Psychology. Scores 3.0 and above are rated in the Novice to Advanced Novice range, which is considered in the accepted standard
for Practicum I students on the developmental scale.
Practicum I students obtained an average score of 2.9 on the 3.0 Professional Work Characteristics scales.
Results from the Spring 2015 evaluations indicate Practicum II students rating scores ranged from 4.6 (Intervention and Support
Academic Learning) to 5.0 (Research and Program Evaluation) on the 5.0 point developmental scale. Results indicate that practicum
students earned above satisfactory scores (4.0 or higher) in all NASP Domains. Professional Work Characteristics ranged 2.9 out of a
3.0 point scale.
Figure 3 – Practicum I Evaluation Means
School Psychology AIM Report 2014-15
Commented [PC2]: For Practicum I, II and Internship tables,
can we separate out the
Page 6
Figure 4 - Practicum II Evaluation Means
School Psychology AIM Report 2014-15
Page 7
Evaluation Guides - Practicum II
Domain Means: Scale 2-5 (Prof. Work Characteristics: Scale 1-3)
Professional Work Characteristics
NASP Domain 10: Legal, Ethical, and Professional Practice
NASP Domain 9: Research and Program Evaluation
NASP Domain 8: Diversity in Development and Learning
NASP Domain 7: Family and School Collaborations
2014-15 N=7
NASA Domain 6: Preventative and Responsive Services
2013-14 N=8
NASP Domain 5: School-Wide Practices to Support Learning
2012-13 N=12
NASP Domain 4: Interventions and Mental Health Services to…
NASPA Domain 3: Interventions and Instructional Support to…
NASP Domain 2: Consultation and Collaboration
NASP Domain 1: Data-Based Decision Making and Accountability
2.6
School Psychology AIM Report 2014-15
2.8
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
Page 8
b) Internship Evaluations
Results indicate that internship area mean scores ranged from 6.3 (Interventions and Instructional Support to Develop Academic
Skills) to 6.7 (Legal, Ethical and Professional Practice) on the 7-point Evaluation Guide for School Psychology for the spring 2015
interns. Professional Work Characteristics ratings averaged 2.9 out of a 3.0 point scale. Results indicate that interns earned above
satisfactory to highly satisfactory scores (scores of 6.0 and above) in all Evaluation Guide areas (see Figure 5). The revised Intern
Evaluation scale is designed to demonstrate growth from the Emerging Practitioner during Practicum II, to the Established and
Advanced Practitioner during Internship.
Figure 5 – Internship Evaluation Means
School Psychology AIM Report 2014-15
Commented [PC3]: Can we cut this to most recent 3 years
comparison?
Page 9
Results indicate that interns scored in the Established Practitioner or above range on all Evaluation Guide areas (Data-based Decision
Making; Consultation and Collaboration; Effective Instruction and Development of Cognitive/Academic Skills; Socialization and
Development of Life Skills; Student Diversity in Development and Learning; School and Systems Organization, Policy
Development, and Climate; Prevention, Crisis Intervention, and Mental Health; Home/School/Community Collaboration; Research
and Program Evaluation; School Psychology Practice and Development; Professional Work Characteristics; and Communication
Skills).
Conclusion(s):
Reviews of the Evaluation Guide for School Psychology suggest that UW-Stout graduate students are adequately prepared to enter
the field of practice with professional skills for success as practicing school psychologists. We are pleased with the accuracy of the
data that our recently revised evaluation tools have provided.
PRAXIS II: Content Test Summary or National School Psychologist Exam Results
It is a program requirement to take the National Certification in School Psychology (NCSP) exam (PRAXIS II) used by the National
Association of School Psychologists (NASP) as requirement for national certification. The passing score established by NASP and
recognized by the Wisconsin Department of Public Instruction (DPI) is 147. This passing score reflects a major revision to the Praxis metric,
implemented in Fall of 2014 (previous to that, passing score was 165).
In addition to the total score received by examinees’, the PRAXIS II exam also report scores addressing the following content areas:
Professional Practices
Direct and Indirect Services
Systems Level Services
Foundations of School Psychological Service Delivery
Outcomes of the NCSP Results
In 2014-15, 100% of the students obtained scores of 147 or higher on their first attempt at the Praxis. As such, every student who took the
PRAXIS Exam met the passing criteria of NASP and DPI.
School Psychology AIM Report 2014-15
Page 10
PRAXIS II: Content Test Summary
School Psychology Praxis Test Code – 10402
Content Test from ETS
14/15
(0402)
8
Number of Examinees:
179
Highest Observed Score:
151
Lowest Observed Score:
147
WI Score Needed to Pass:
Number with WI Passing
Score:
8/8
Percent with WI Passing
Score:
100%
Percent Correct (percentage of items answered correctly) by Area
UWNational
Points
Stout
School Psychology Test Category
Available
14/15
14/15
Professional Practices
30-31
75%
73%
Direct and Indirect Services
24-25
77%
82%
Systems Level Services
17-17
77%
77%
Foundations of School
33-35
65%
72%
Psychological Service Delivery
School Psychology AIM Report 2014-15
Page 11
Figure 7 – Praxis II Domain Percentages
School Psychology AIM Report 2014-15
Commented [PC4]: Can we add the national average data to
this table, for comparison?
Page 12
Praxis II First Time Test Takers
Beginning in the 2013-14 school year, all teacher preparation program in the state of Wisconsin are required to report on “the passage rate on
the first attempt of students and graduates of the program on examinations administered for licensure” (Act 166, section 17).
Spring 2013
SPSY
Content
Test
2013-14
2014-15
# first time
test takers
# (and %)
passed
# first time
test takers
# (and %)
passed
# first time
test takers
# (and %)
passed
10
9 (90%)
8
8 (100%)
7
7 (100%)
Conclusions
Overall results indicate UW-Stout’s School Psychology program is preparing students to achieve the foundational knowledge necessary to
pass the NCSP exam. Analysis of Stout performance trends across Praxis domains is unavailable at this time, due to recent revisions to the
test. First time test takers of the newly revised exam performed best on the Systems and Preventive/Responsive Services domains. Lowest
scores were observed on the Legal/Ethics domain. Information from PRAXIS II data review will be used to inform on-going program
review with focus on continuous improvement in identified areas, as well to inform program development in general.
School Psychology AIM Report 2014-15
Page 13
Other Information
a) Portfolio Assessment (Benchmark II and Benchmark III)
Students initiate and complete components of a portfolio over the course of the program. The completed portfolio includes evidences
or artifacts demonstrating professional competence in all ten (10) NASP’s Domains of School Psychology training and practice
(same as the Department of Public Instruction content guidelines). A partially completed portfolio (Benchmark II) demonstrating
competency in 8 out the 10 domains is submitted to the program faculty for review after completion of the M.S.Ed. and prior to
admittance to the Ed.S. in School Psychology Program.
At Benchmark III, students re-submit their portfolio to demonstrate competency in all ten (10) domains. Faculty members rate the
portfolio materials. Based on a review of the portfolio ratings, academic performance, and professional conduct, a student is either
recommended for, or denied admittance to, the Ed.S. degree in School Psychology (for Benchmark II) or Internship (Benchmark III).
Outcomes of the Portfolio Assessment
A review of the portfolio ratings in 2015 revealed all students produced satisfactory portfolios at Benchmarks II and III. Based on
faculty consensus, there is however, a need to further highlight the relevancy of the portfolio for candidates, particularly during the
internship year. Faculty has discussed a number of ways to motivate students to sustain their efforts to update and enhance their
professional portfolio during internship.
b) Survey of Intern Result
In 2013-14, as in previous years, school psychology interns were surveyed to determine their perceptions regarding their previous
coursework. Each intern was asked to respond to questions on a 5-point Likert-style survey (1 = No knowledge or Skill gained to 5
= A lot of knowledge or Excellent skills gained) to assess how much knowledge or skill the student gained in each course. Further,
interns are asked to identify the five most important courses in terms of how helpful they were in preparing them for their internship
year. They also are asked to identify which courses were not helpful or had overlapping content.
Outcomes of the Intern Survey
Results from the spring of 2015 survey indicate students believed that they gained above average to a lot of knowledge and skill (x =
4.29) in most program courses. However, students continue to indicate that they gained below average knowledge and skills in the
Research Foundations (EDUC 740, x = 2.29). Variable opinions were expressed about the courses related to counseling skills and
theory, with some students considering it a value add to the program (x=4.10), while others questioned the relevance of this
coursework to their jobs in the field (x=2.6). Additional comments indicated that most students reported they believe they were well
prepared for the field of school psychology after taking all program requirements. Students did have additional feedback related to
the nature of research expectations and timelines in the program. Revisions based in part on this feedback, are underway.
School Psychology AIM Report 2014-15
Page 14
c) Program Advisory Committee (PAC)
The program faculty members meet annually with the School Psychology Program’s Advisory Committee (PAC). Colleagues from
related programs on campus, as well as practitioners and stakeholders in the community meet to discuss updates in the program and
solicit information from the committee about program needs and future goals. PAC membership consists of colleagues and related
stakeholders, including practitioners in the field, representatives from higher education, and the state department (DPI). The PAC has
been consulted as we have continued to revise and update the program in order to align with the new NASP standards. Their input
continues to be valuable as we further develop and evolve our program.
Outcome(s) of the Program Advisory Committee Results:
Most recent PAC focus continued to relate to revisions to the program. Focus of recent meetings has been on NASP accreditation
efforts. New discussions will include program revisions and recommendations from the NASP report.
Plan for Communicating Assessment Results
The School Psychology Program faculty meets on a bi-weekly basis to discuss program issues and plan for the future. This Assessment in
the Major report will be disseminated to all members of the program faculty, and the results will be discussed by the program faculty at an
upcoming meeting. This guide is a valuable resource for faculty to use in guiding program revisions. Furthermore, this report will be
disseminated to the Program Advisory Committee (PAC) in the fall of the 2015-16 academic year.
School Psychology AIM Report 2014-15
Page 15
Conclusions and Action Plan to Improve Teaching and/or Advisement in the School Psychology
Program
Summary and review of all available data related to the School Psychology programs at UW-Stout, the following goals are targeted for the
2013-14 year:
Targeted Goals for 2014-15
1. Based a variety of sources (NAPS review, program advisory, student feedback), program faculty will work to implement a program
revision that aims for continuous improvement and efficiency.
a. Revisions were developed and courses revised throughout the Spring and Summer of 2015. Approval process will begin Fall
of 2015
2. In coordination with Goal #1, program faculty will work in collaboration to comprehensively review research expectations, timelines
and associated coursework for School Psychology, and revise accordingly.
a. This goal is on-going, in conjunction with a comprehensive program revision.
3. Faculty will work with Grad School and Stout Marketing to develop a more concrete and tangible marketing plan for the School
Psychology program.
a. Program PD met with both Grad School and Marketing reps in spring of 2015 to develop a strategic marketing plan,
including revisions to our web presence.
4. Program faculty will work with SOE support members to transition to an E-portfolio process for students.
a. This goal is on-going.
School Psychology AIM Report 2014-15
Page 16
Goals for 2014-15
1.
2.
3.
4.
A comprehensive program revision (including revised and new coursework), will be approved for a Fall 2016 implementation.
Marketing and Recruitment efforts will help us achieve a higher number of quality applicants to the program.
Program faculty will work with SOE support members to transition to an E-portfolio process for students.
Program will review and renew efforts to embed applied activities across relevant coursework, with a focus on Year One.
Respectfully submitted,
Christine R. Peterson, Ph.D.
Associate Professor
Program Director for School Psychology
University of Wisconsin-Stout
School Psychology AIM Report 2014-15
Page 17