1 Program Director Self-Study Report For Program: M.S.Ed/Ed.S School Psychology Submitted by Program Director Name: Christine R. Peterson Year: Fall, 2012 Planning and Review Committee 1. UW-STOUT’S STRATEGIC PLAN 1.1 Describe how the program relates to UW-Stout's Strategic Plan. 1.1.1 Describe early and ongoing experiential learning opportunities to students within the program. Experiential learning opportunities play a large role, and represent a strength for the School Psychology program. A 3 year program in total, the first year consists primarily of courses designed to build and practice key professional skills. Virtually all 1st year coursework includes applied requirements such as school-based academic benchmarking and progress monitoring (CBM) in local districts, service learning requirements in community settings, and early childhood screening activities. During the 2nd year of the program, students complete upwards of 750 hours of site-based practicum experiences, including a 600 hour school-based practicum and a campus-based clinic practicum in the Clinical Services Center. Students spend an average of 3 days per week on applied activities this year. The 3rd and final year of the program is spent entirely on site, completing a 1200-hour fulltime, school-based internship. In total, at completion of the program, students will have spent well over 2000 hours engaging in applied activities. 2 1.1.2 What are the initiatives used to increase and support program enrollment, student retention and graduation rates? Given a recent downward trend nationally in graduate program applications and enrollments, increasing our recruitment efforts has been an identified as a high priority for the school psychology program here at Stout. Working with institutional supports provided through the graduate school and marketing departments, we are currently working to identify new recruitment efforts to supplement those already in place. Current initiatives include publication and dissemination of our program brochure to undergrad psychology and education programs across the nation, with a particular focus on WI, MN, IA, SD, IL, and MI. Addition recruitment efforts are made via participation on social media sites (i.e., Facebook) and a variety of online graduate school information sites (i.e., Gradschool.com). Program information is also published on the NASP website devoted to NASP accredited school information. 1.1.3 Respond to the program facts and your program’s creative endeavors related to the diversity aspects of Inclusive Excellence: “UW-Stout’s plan to intentionally integrate diversity efforts into the core aspects of everything we do. Diversity is broadly defined and includes, but is not limited to, race/ethnicity, gender, sexual orientation, age and disability status.” The School Psychology program at UW-Stout has a proud history of promoting diversity within both the program as well as in the field itself. School psychology as a field has historically struggled to represent the diverse population of school children it serves, remaining a largely Caucasian, female dominated profession. By contract, UW-Stout’s program has graduated 8 males (38%) and 4 students from underserved populations (18%) over the course of the last 3 years. These percentages far outperform national averages. We will continue to work to maintain and improve upon those figures into the future. 1.1.4 Describe the environmental sustainability initiatives of your program: “UW-Stout’s attempt to make students, faculty, and staff more aware of the importance of sustaining our environment through energy conservation, waste reduction, and other measures that will not bring harm to the environment, and to provide students with innovative research opportunities in these areas.” The School Psychology program strives to uphold and follow practices consistent with UW-Stouts overall sustainability plan. We are moving toward digitizing our recordkeeping practices, in an effort to minimize use of paper. We are also looking to move toward a digital portfolio process. We support sustainability across campus and strive to encourage responsible practices on the part of our students at school, at home and in the community. 3 1.1.5 List various training and development opportunities of core faculty teaching within your program. School Psychology faculty at UW-Stout engage and participate in a variety of professional development and training activities. As a general rule, all faculty attend the annual conference of the National Association of School Psychologists (NASP), a weeklong conference that offers both workshop as well as more intensive professional training opportunities. Our attendance serves multiple purposes: to allow for our own professional development endeavors, to present our research and expertise to a national audience, and to model for and support our students, whom we strongly encourage to attend and present with us. Additionally, our faculty engages in a broad array of professional develop activities, including attendance at the 2012 UW-system sponsored Higher Education Institute, as well as various state level conferences and workshops such as the Wisconsin School Psychology Association’s fall and spring conferences, the summer conference for the Wisconsin Applied Behavior Analysis association. 2. DESCRIPTION OF THE PROGRAM 2.1 Curriculum Design 2.1.1 State the program objectives. UW-Stout School Psychology Program goals and objectives are consistent with current NASP Standards for Training and Professional Practice. The M.S.Ed./Ed.S. School Psychology program strives to achieve the following program goals and associated student outcome objectives: Goal #1: Provide students with a solid understanding of the psychological and educational principles underlying the field of school psychology. These principles include, but are not limited to human learning and development (typical and atypical), as well as human diversity. Students apply this foundational knowledge to problems of learning and behavior through appropriate decision-making, competent intervention planning and implementation, and effective communication and consultation (NASP Domains 3, 4, 5, 6, & 7). Objectives: 1. Students will have knowledge of human learning processes, techniques to assess these processes, and direct/indirect services applicable to the development of cognitive and academic skills. In practice, students will collaborate with 4 others to develop appropriate cognitive and academic goals for students with different abilities, disabilities, strengths, and needs; implement interventions to achieve those goals; and evaluate effectiveness of interventions. 2. Students will have the knowledge of human learning processes, techniques to assess these processes, and direct/indirect services applicable to the development of behavioral, affective, adaptive, and social skills. In practice, students will collaborate with others to develop appropriate behavioral, affective, adaptive, and social goals; and evaluate the effectiveness of these interventions. 3. Students will have knowledge of individual differences, abilities, and disabilities and of the potential influence of biological, social, cultural, ethnic, experiential, socioeconomic, gender-related, and linguistic factors in development and learning. In practice, students will demonstrate sensitivity and skills needed to work with individuals of diverse characteristics and to implement strategies selected and/or adapted based on individual characteristics, strengths, and needs. Goal #2: Prepare students to follow a problem solving approach to the practice of school psychology with children and youth, use multiple sources of data to facilitate the best decision-making, regardless of whether it involves an individual child or an entire program. Students competently conduct psychological assessments that are relevant to student problems and use their findings for decision making and program planning. Data gathering is used to evaluate the effectiveness of programs and to continually improve one’s practice (NASP Domains 1 & 9). Objective: 1. Students will have knowledge of varied models and methods of assessment that yield information useful in identifying strengths and needs, in understanding problems, and in measuring progress and accomplishments. Students will have the knowledge of the underlying research and statistics of these models as well as methods of program evaluation. In practice, students will use such models and methods as part of a systematic process to collect data and other information, translate assessment results into empirically-based decisions about service delivery, and evaluate outcomes of services across different levels (i.e. individual, system). Goal #3: Prepare students to use a systemic perspective to view children’s development and to understand the contexts in which this development occurs. An emphasis on consultation and collaboration with families as well as other professionals is integral to the role of the school psychologist. Students work with others to support the academic, social, and emotional development of children and youth (NASP Domains 2, 6, & 7). 5 Objectives: 1. Students will have knowledge of different models of consultation and collaboration and of their application to family, school, and community systems. This knowledge base will include an understanding of family systems, including family strengths and influences on student development, learning, and behavior, and effective methods to involve families in education and service delivery. In practice, students will collaborate effectively with families, educators, and others in the community in planning and decision-making at the individual, group, and system levels. 2. Students will understand the systemic nature of schools and other settings as well as specific knowledge of general education, special education, and other educational and related services. In practice, students will work with individuals and groups to facilitate policies and practices that create and maintain a safe, supportive, and effective learning environment for children and youth. Goal #4: Prepare skilled interventionists who have knowledge about various intervention strategies that are associated with positive outcomes, as well as skills in implementing these different interventions. In order to do so effectively, students must have knowledge of relevant research and be able to translate this knowledge into practice by adapting interventions to meet the needs of the client and the system (NASP Domain 3 & 4). Objective: 1. Students will have knowledge of evidence-based prevention and intervention approaches across universal, selected and targeted levels that are designed to improve the behavioral, socio-emotional, and academic functioning of students. In practice, students will provide or contribute to prevention and intervention programs that promote the mental health and physical well-being of students. These programs include, but are not limited to, individual and group counseling, behavioral support planning, and school-wide programs. Goal #5: Prepare students to engage in scholarly activities and psycho-educational service within a culturally diverse society. In doing so, students are knowledgeable about the impact of biological, social, cultural, ethnic, socioeconomic, and linguistic factors on the learning and development of students and families from diverse backgrounds (NASP Domains 1, 5, & 8). Objectives: 1. Students will have knowledge of varied models and methods of assessment that yield information useful in identifying strengths and needs, in understanding problems, and in measuring progress and accomplishments. Students will have the knowledge of the underlying research and statistics of these models as well as methods of 6 program evaluation. In practice, students will use such models and methods as part of a systematic process to collect data and other information, translate assessment results into empirically-based decisions about service delivery, and evaluate outcomes of services across different levels (i.e. individual, system). 2. Students will have knowledge of individual differences, abilities, and disabilities and of the potential influence of biological, social, cultural, ethnic, experiential, socioeconomic, gender-related, and linguistic factors in development and learning. In practice, students will demonstrate sensitivity and skills needed to work with individuals of diverse characteristics and to implement strategies selected and/or adapted based on individual characteristics, strengths, and needs. 3. Students will have knowledge of up-to-date information sources and technology relevant to their work; and of public policy development applicable to service delivery. In practice, students will incorporate technology to enhance their practice, and demonstrate involvement in their profession and a commitment to on-going professional development. Goal #6: Prepare professionals who will act as child advocates and facilitate the optimization of psycho-educational environment of the child. In doing so, students are knowledgeable of and adhere to the legal and ethical guidelines of the field. Additionally, students present themselves as professionals in all aspects of their functioning and regularly engage in on-going professional development (NASP Domain 10). Objectives: 1. Students will have knowledge of the history and foundations of their profession, up-to-date information sources and technology relevant to their work; of public policy development applicable to service delivery; and of ethical, professional, and legal standards. In practice, students will practice in ways that are consistent with state and professional standards, and demonstrate involvement in their profession and a commitment to on-going professional development. 2.1.2 What are the initiatives used to determine the need for program revision, including but not limited to program enrollment, student retention or student graduation rates. We rely on a variety of important factors to inform our program and to make necessary revisions. A key influence, and foundation upon which our curriculum is based, is the NASP Standards for Professional Practice (aka, “The Practice Model”). This model includes 10 domains of practice, with language specific to graduate training 7 programs, intended to reflect the evolving trends in practice, and designed to ensure the preparation of new professionals, who enter the field ready to engage in best practices. The most current NASP practice model was approved in 2010 for application in 2011. Another important source of information to guide our curriculum and program practices and revisions includes consultation with our Program Advisory Committee (PAC), which includes members representing a cross-section of Stout graduates, practitioners in the field, higher education and state level representatives. Data from our evaluation tools helps faculty to reflect on the program profile, target areas for improvement, and modify our program as needed to ensure our students are comprehensively prepared, as measured by the various evaluation tools in place. Direct student input also plays a critical role in our program review and revision process. As students complete their final semester in the program, they complete an intensive exit survey, in which they are asked a variety of questions related to all coursework, as well the overall program. We review this information annually, as part of both our ongoing faculty meeting agenda, and for review at PAC meetings. 2.1.3 Is your program defined as a distance learning program (yes/no)? – Or delete all together. No. 2.1.4 Give examples and explain the ways in which the program intentionally integrates diversity efforts, functions and contributes to the program. The School Psychology program at UW-Stout has a proud history of promoting diversity within both the program as well as in the field itself. School psychology as a field has historically struggled to represent the diverse population of school children it serves, and remains a largely Caucasian, female dominated profession. By contrast, UW-Stout’s program has graduated 8 males (38%) and 4 students from underserved populations (18%) We will continue to work to maintain and improve upon this trend into the future. 2.1.5 UW-Stout “programs are presented through an approach to learning which involves combining theory, practice and experimentation” (Mission Statement). Briefly describe the components of your program where students participate in scholarly activity such as: research, scholarship, experiential learning and creative endeavor. Opportunities for experiential learning and scholarly activity pervade all aspects of the School Psychology program. Students in our program complete both a Master’s thesis and an EdS capstone project in the form of an EdS thesis or an EdS action research project. 8 Virtually all 1st year coursework includes applied requirements such as school-based academic benchmarking and progress monitoring (CBM) in local districts, service learning requirements in community settings, and early childhood screening activities. It is during the first year in the program, that students begin their Master’s thesis project. During the 2nd year of the program, students complete upward of 750 hours of site-based practicum experiences, including a 600 hour school-based practicum and a campus-based clinic practicum in the Clinical Services Center. Students spend an average of 3 days per week on applied activities this year. Students generally complete their Master’s thesis in the Fall of year two of the program, and are encouraged to disseminate their projects via state and/or national conventions. The 3rd and final year of the program is spent entirely on site, completing a 1200-hour fulltime, school-based internship. In total, at completion of the program, students will have spent well over 2000 hours engaging in applied activities. It is at this time that students begin their capstone research project, with the intention of completion of data collection and analysis by the end of this academic year. Beginning in Spring of 2013, students will be required to submit their research for presentation at a state and/or national conference. 2.1.6 Does your program currently have an accreditation or certification agency that reviews the program? If so, which agency and to what extent do they influence the structure of the curriculum? The School Psychology program at UW-Stout is accredited by the National Association of School Psychology (NASP). NASP influences every aspect of our program, from the curriculum and course sequence, to the faculty profile, to cohort size. NASP represents the gold standard for graduate preparation and training of school psychologists, and accreditation is critical to the on-going health and success of the program. NASP accrediting furthermore, aligns with and is managed by The National Council for Accreditation of Teacher Education (NCATE), the national accreditation body for Schools of Education across the nation, including UW-Stout. 2.2 Faculty/Academic Staff Expertise 2.2.1 List the key people in the curriculum. A key instructor is one who teaches at least one required professional course in your program. Christine Peterson, PhD Amy Zimmerman, ABD Denise Brouillard, PhD 9 Amy Gillett, PhD Ann Brand, PhD John Klem, PhD Kelly Lamon, EdS Jennifer Zaske, EdS (We are currently conducting a search to fill one fulltime vacancy in the program.) 2.2.2 2.3 What additional areas of faculty/academic staff expertise are currently needed? The primary emphasis to promote continued stability in the school psychology program is to hire and retain high quality PhD level faculty in the area of school psychology. It would be preferable to identify a qualified candidate with 5+ years of related professional experience. Our program is assigned adequate coverage in terms of faculty and adjunct support, but the program has struggled to keep a consistent fulltime faculty base, and faculty with prior experience coming into the job. We are in the process of an active faculty search at this time. Preferences will be made for quality applicants with expertise one or more of the following areas: identification and intervention of learning disabilities, service to underserved student populations, identification and intervention of social-emotional disorders. Facilities 2.3.1 What special facilities and or capital equipment currently available are utilized and how do they strengthen this program? What additional facilities (special classrooms, labs, additional space involving minor construction) have been requested and has that been filled? One of the primary resources affiliated with the school psychology program is our Assessment Resource Library. This library houses all assessment measures required for comprehensive training in the program in the areas of cognitive, academic, developmental, social-emotional, adaptive, and behavioral functioning. We maintain a large base of assessment measures that are utilized across a number of different programs, including Special Education and School Counseling. NASP and test publishers require strict security and oversight of such measures, which in turn requires trained graduate assistants to operate the library, under the supervision of qualified faculty. This library must be accessible to students and also locked and secure when not in use. In coordination with the Assessment Resource Library, we also utilize what is known as the “program office” space, that serves a variety of programmatic needs. Principally, the program uses 3 computers to train students on a variety of computer-scoring programs for cognitive and other measures. This space also serves as a contact point between cohort I students and Grad assistants, who meet regularly to review assessment course related mentoring. 10 This space serves as the “hub” for graduate assistants to manage the Assessment Library in close proximity, and for students to check and/out library materials. Another critical space we utilize is the Clinical Service Center, located on 2nd floor of Voc Rehab building. We coordinate with our colleagues in the Mental Health and Marriage and Family Counseling programs to share this space. Many of our students participate in the Play Therapy coursework, which also utilizes space in the Clinical Services Center. These two facilities play critical roles in the implementation of our program. 2.4 Resources for the Program 2.4.1 Evaluate as to currency/up-to-datedness, quality, relevance, and quantity of the library resources to support the program. List or describe any information or service needs created over the past three years by concentration and course changes and include a brief statement as to how these needs have been met by the library. Between the standard library resources offered to programs, the Instructional Resource Center, and support within the SOE to ensure adequate and up to date assessment resources, our program is fortunate to enjoy support to meet our needs on an annual basis. Working with administrators to communicate about upcoming needs, we are able to secure the resources we need. 2.4.2 List any special resources used to meet program and/or student needs such as: Academic Computing, Instructional Technology Services for curriculum materials development, ASPIRE, Research Center, Media Self-Instruction Lab, Academic Skills Center, etc. List or describe any other resources which are needed to meet the program objectives with a brief statement as to how these would enhance or maintain the concentration quality. School Psychology students work closely with the PARQ office, where they often seek assistance with data analysis related to thesis work. With recent an anticipated revisions to our research requirements, we hope to better connect with the Research Center to more frequently present at events like Research Day, etc. 2.5 Assessment in the Major 2.5.1 Attach your most recent Assessment in the Major report. • See attached 11 3. Supply evidence of the quality of the graduates of the program. 3.1 Describe the demand for graduates and anticipated changes or trends in such positions/roles. The School Psychology program at UW-Stout is proud to report a %100 job attainment record in the field, for both internship and post-grad employment. A majority of our students complete their internships in the region, however we have had recent interns in Colorado, Nevada, Texas, and even internationally. Future trends in the field indicate a growing demand for school psychologists, based on both the aging of the field (a larger percentage of retirements) as well as increase in positions. Interestingly, despite the strong indication for strength in the field, there is also a trend in graduate programs across the nation toward lower numbers of applicants and entrants into school psychology programs. Some of the larger and higher-profile programs have reported as much as a %30 decline in both applicants and entrants. The program at UW-Stout has generally remained stable with an average of approx. 35-40 applicants and 9-14 entrants annually. We consistently find that a percentage of our applicant pool is not considered viable or appropriate for consideration, and the program has identified as a priority goal to increase both the number and quality of preparedness of our applicant pool. 3.2 Interpret the data from the Planning, Assessment, Research and Quality (PARQ) office of the alumni follow-up surveys. Recent alumni surveys offered some revealing and important data. Five-year follow up was generally positive in nature, regarding the overall quality and structure of the program. One-year follow-ups provided mixed feedback, consistent with other sources of data. Leadership and milieu in recent years has been identified as an area of concern, though many of the relevant changes of have already been made. The importance of consistent faculty to promote strong and trusting facultystudent relationships was also noted. Though not reflective of the majority of students who move through out program to completion, some of the follow-up data came from a student who was questioning their career decision. This is important input, as it emphasizes our need as a program to do our best to ensure that applicants and students understand the full nature of the role of the school psychologist, and to prepare them with a realistic perspective as to the field itself. The program will look to future alumni survey data to evaluate the effect of recent and significant changes made. 3.3 Interpret the major results from your Program Specific Surveys (students, faculty and advisory committee) conducted by the Planning and Review Committee. Much of the feedback from stakeholders affiliated with the program (students, faculty and PAC) reflects the major transitions that the program has experienced over the last 3 years. Faculty issues, including program director leadership and related concerns, play a significant role in the feedback and performance reviews, particularly from students. This is 12 understandable, knowing the context within which many of the reporting students experienced the program, which undoubtedly influenced their trust and generalized to feelings about the program overall. This on top of a universal recognition that the field of school psychology itself has experienced a virtual seachange over the last 3-4 years, pushing graduate preparation programs across the nation to evolve with the shifting profile of the field. However it is critical to note that many of the concerns voiced most critically are no longer applicable to our program. Significant shifts in leadership, faculty profile, as well as major revisions to the program plan and sequence have already addressed a majority of the concerns voiced by survey respondents. An intense review of all survey data revealed the following themes to be considered and addressed by current program faculty: Student input was reviewed from both the PRC survey as well as a program exit survey, which includes detailed input related to all coursework in our program, as well as the overall program. Summary of all feedback suggests the following themes: • EDUC 740 – Research Foundations • Students consistently report a variety of issues with this course, primarily around 2 themes: 1) the range of learners with statistical backgrounds is too diverse, leaving more advanced students to simply review long-ago mastered concepts and diminishing advanced learning for statistical analysis and/or 2) inconsistency around how much if at all, individual student thesis work is incorporated into the content of the course. • Summary findings suggest a need to re-evaluate, as part of a larger conversation, our research and related coursework requirements within the program (see below) • Research Requirements • Faculty stability – Students have consistently and with vigor expressed the importance of having consistently present and available faculty. The intensity of the program almost necessitates that students have trusted faculty mentors with whom to consult and seek counsel throughout the program. Faculty changes, while unavoidable, erode the trust and strength of the mentor-student relationship Summary findings suggest a need to recruit with vigor and all available resources, high quality faculty who are more likely to commit for the longer, and to build an environment that encourages longevity. • Clear communication of program expectations 4. Supply evident of continuous improvement efforts of the program. 4.1 Describe the strengths and unique features of your program that distinguish it from similar programs. What are the 13 weaknesses of the program? Of particular interest to our program, and an area of identified strength that distinguishes us from other programs is our focus on Counseling. The Stout program not only includes counseling coursework into our program, but also coordinates with the School Counseling program to allow for students who wish to, to seek a dual program leading to the M.S in School Counseling and the EdS in School Psychology. Student feedback has consistently been mixed on the counseling aspect of the program, with some students highlighting their appreciation of this coursework and preparation, while others noting it’s lack of utilization in the field. In response, and also in an effort to better align with the expanding content demands put forth by NASP, our recent program revision has moved 2 counseling courses from required to elective status. In so doing, we have worked to preserve both the counseling identify and affiliation of the program, as well as the ability to complete the dual program for those students who wish to do so. At the same time, in moving these classes from required to elective status, we have been able to realign our total credit requirements closer to NASP standards, and also to make room for expanding and new content expectations as put forth by our accrediting body. Another strength of our program is the emphasis on the applied experiences our students engage in. From their first semester to their final experience, students are engaging in real, applicable experiences. This is critical to student’s mastery of skills, from the conceptual to the applied. The Stout experience prides itself on offering applied practice early in the program, through completion of the internship. A particular area targeted for growth in the coming years includes our research component. We currently have a thesis requirement at both the Master’s and the Ed.S. levels, with accompanying coursework in research foundations. Increasingly, completion of the Ed.S. thesis has created a lag in program completion, as students tend to wait until the end of their internship to begin the thesis project. This results in students putting off completion of the program in full for 1-3 years following internship, which is not preferred. In collaboration with our program advisory board, we are seeking ways in which to revise the overall research requirements within the program, in order to meet the needs of our student population and to encourage timely completion of research. This is also important for the program, as we are losing opportunities to present student research at state and national conferences, given the delay in project completion. Future considerations are under way to investigate development of research coursework designed around the action research project specifically, and related to single-subject design and program evaluation, which is also more relevant to the type of research practitioners do the in the field. 4.2 Submit evidence of program response to the concerns and recommendations in your previous program review. From 2007 Consultant Report: 1.Program students are not getting adequate exposure of ethnic diversity in their course of study. Most students and instructors are themselves not members of a diverse ethnic or racial group; in addition, practicum sites are typically not ideal for gaining experiences with diverse ethnic groups. 14 Update: Content related to serving students from diverse and underserved populations has been infused across our program curriculum. Additionally, coursework related specifically to multi-cultural issues is a required program component, as is a related cross-cultural field experience. Finally, we strongly encourage, and work with our students to secure, study abroad experiences that will allow them to further expand their awareness of diversity issues. 2.There is some course overlap between two courses in the program: SPED-630 Inclusion of Students with Exceptional Needs and SPSY-778 Psychoeducational Disability. Update: This content overlap was eliminated at the time of our most recent program revision, which deleted SPED-630 from the program plan. 3.Program students expressed some concern that two required graduate psychology courses, PSYC-730 Advanced Psychology of Learning, and PSYC-850, were in part, duplicating course material that many students covered in undergraduate courses. Update: Both of these courses have been revised for content, and attempts made to enhance both the rigor and expectations from these two courses, as well as to further distinguish them from one another. While many of our students coming from a psychology background do have similar courses in their undergraduate history, fewer comments have noted regarding this issue. 4.The primary issue of concern for UW-Stout’s M.S.Ed. and Ed.S. programs in School Psychology is the continuing accreditation by the National Association of School Psychologists (NASP). The next accreditation review will occur in Fall, 2006 and NASP has already indicated that institutions that don’t have a 1:10 teacher-student ratio and at least three Ph.D. faculty members who are teaching a nine-hour credit load may lose their accreditation. If Stout were to lose this accreditation, we would have great difficulty attracting students and placing graduates. Update: UW-Stout School Psychology program remains NASP approved, with a full review due in the Fall of 2013. It will be an imperative, to complete a successful faculty search this year, in order to present a fully staffed faculty, following NASP mandates for course coverage, student ratios, and faculty profiles. 15 4.3 In the next seven years, what are the major improvements or changes you plan to implement to improve program quality? Long range planning for the UW-Stout school psychology program will emphasize: 1. Continued efforts to distinguish ourselves clearly from competing programs in the state and region 2. Increasing and expanding recruitment efforts to ensure high quality applicants and enrollees 3. Completing an in-depth review and possible revision to our research requirements and process, in order to better align with program initiatives like increasing student dissemination of research, and timely completion of EdS research projects. Very recent program revisions have been completed, and are anticipated to usher our program into the next generation of best practices in graduate training for several years to come. We now look forward to refining the details of our program, to continue to enhance applied activities with an emphasis on high quality. 5. Attachments-Please include electronic links. 5.1 Submit any other information or documentation that may be helpful to the Planning and Review Committee in reviewing the quality of the program including interpretation of data from Institutional Research and PRC data. 5.2 Links of specific program information to be included: • Program plan sheet o Link to SPSY Program Plan • Current assessment in the major o See attached • Individual program facts o Link to School Psychology Homepage • Current program advisory committee • • • • • • • Kathryn Bush, WI Dept. of Instruction Kelly Lamon, CEHHS/School of Education Tyler Miller, Menomonie, School District Christine Peterson, Program Director Todd Savage, UW-River Falls Tara Stefanson, Student Representative Jackie Weissenburger, Assoc. Vice Chancellor 16 • Amy Zimmerman, CEHHS/School of Education • Other items requested by the consultant 17 M.S. & Ed.S. in School Psychology Assessment in the Major Report By Dr. Christine Peterson, Program Director 2011-12 Submitted: October 2012 18 Table of Contents Description of Methods ........................................................................................................................................................................................................... 19 I. PROGRAM DISPOSITION REVIEW.......................................................................................................................................................................................... 20 II. PRACTICUM AND INTERNSHIP STUDENT EVALUATIONS ..................................................................................................................................................... 22 III. PRAXIS II: CONTENT TEST SUMMARY OR NATIONAL SCHOOL PSYCHOLOGISTS EXAM RESULTS....................................................................................... 27 PRAXIS II: Content Test Summary ............................................................................................................................................................................................ 28 IV. OTHER INFORMATION ........................................................................................................................................................................................................ 30 V. PLAN FOR COMMUNICATING ASSESSMENT RESULTS ......................................................................................................................................................... 31 VI and VII. CONCLUSIONS AND ACTION PLAN TO IMPROVE TEACHING AND/OR ADVISEMENT IN THE SCHOOL PSYCHOLOGY PROGRAM.......................... 32 19 Description of Methods The following assessment tools and strategies were utilized to determine student performance, progress, and levels of satisfaction in the major: Benchmark I • Faculty review/selection of program applicants • Transcripts review • Acceptance into the School Psychology Program Benchmark II • Initial portfolio review • Student disposition reviews • Review of practicum I evaluations • Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct Benchmark III • Second portfolio review • Student disposition reviews • Faculty reviews of student performance, including academic progress, experiential evaluations, and professional conduct • Review of practicum II evaluations • Student scores on the National Certification in School Psychology Exam (NCSP) or Praxis II Benchmark IV • Survey of interns regarding their perceptions of program competencies and coursework • Final portfolio review • Exit Student Survey 2010 Assessment in the Major report data to synthesize and review the above listed data 2009-2012 Enrollment Data Year 2009 2010 2011 2012 Enrolled 12 10 12 9 Completed 10 12 11 20 I. PROGRAM DISPOSITION REVIEW Disposition reviews are conducted at Benchmark II (end of first year in the program) and III (mid-term during second year in the program) for all school psychology students. Domain scores rated as: 1 (unsatisfactory), 2 (minimal), 3 (satisfactory) to 4 (proficient). Total composite scores range from ≤ 15 (unsatisfactory) to 32 (proficient). Figure 1 – Comparative Benchmark II Means Disposition Review - Benchmark II Mean Data: Scale 1-4 4 3.8 3.6 3.4 3.2 3 2010-11 2.8 2011-12 2.6 2.4 2.2 2 Attendance Preparedness Continuous Learning Positive Climate Reflective Thoughtful & Responsive Listener Cooperative & Collaborative Respectful 21 Figure 2 – Comparative Benchmark III Means Disposition Review - Benchmark III Mean Data: Scale 1-4 4 3.5 3 2.5 2 2010-11 1.5 2011-12 1 0.5 0 Attendance Preparedness Continuous Learning Positive Climate Reflective Thoughtful & Responsive Listener Cooperative & Collaborative Respectful Outcome of the Disposition Review Results As a whole, the 2011-12 school psychology students displayed satisfactory to proficient attitudes and behaviors within the program (see Fig. 1 and 2). Scores from 2011-12 Benchmark II data ranged from mean scores of 3.8 for preparedness and continuous learning to 4.0 for collaboration, respect, and reflective practices. At Benchmark III, mean scores ranged similarly from 3.8 for preparedness and continuous learning to 3.8 for collaboration and reflective practices. All mean scores for 2011-12 fall within the proficient range of performance on Disposition Reviews. 22 Conclusion(s): Benchmark II and III Dispositions allow faculty and students to engage in reflective dialogue regarding each student’s progress and development toward the mastery of critical skills necessary for success as a grad student and as a professional. Data from 2011-12 suggests that students are progressing appropriately in these important skills domains. II. PRACTICUM AND INTERNSHIP STUDENT EVALUATIONS Graduate students in the school psychology program complete two (2) supervised field practica totaling 600 hours during their second year in the program, and a culminating 1200 hour internship that is their third and final year in the program. Graduate students completing both Practicum I and Practicum II are evaluated through the use of The Field Practicum Evaluation tool. This newly revised tool utilizes the same items, along a developmental scale, in order to demonstrate and review growth as students advance through each experience. Scoring metrics for each tool are as follows: Unestablished-1 Practicum II Observer-2 Observer-2 Novice-3 Novice-3 Internship Advanced Novice-4 Advanced Novice-4 Advanced Novice4 Practicum I Emerging Practitioner-5 Emerging Practitioner-5 Established Practitioner-6 Advanced Practitioner7 *It is important to note here that evaluation tools related to all practicum and internship underwent a major revision between Spring and Summer of 2012. Thus, some data presented in this AIM report will reflect one application of the data, which reflects our inability to directly compare “old” and “new” data metrics. This transition should take a total of two years to complete , and future AIM reports will be poised to represent a full comparison of all data as implementation allows. Outcomes of the Practicum and Internship Student Evaluations a) Practicum Evaluations Results from the 2011 evaluations indicate Practicum I students area scores (means) ranged from 3.9 (Prevention, Crisis Intervention, and Mental Health) to 4.6 (Professional Work Characteristics) on the 5-point scale Evaluation Guide for School Psychology. Scores 4.0 an dabove are considered proficient in relationship to the developmental level of the student. Results indicate that Practicum I students earned above satisfactory scores in all areas (Data-based Decision Making; Consultation and Collaboration; Effective Instruction and Development of Cognitive/Academic Skills; Socialization and 23 Development of Life Skills; Student Diversity in Development and Learning; School and Systems Organization, Policy Development, and Climate; and Mental Health; Home/School/Community Collaboration; Research and Program Evaluation; School Psychology Practice and Development; Information Technology; Professional Work Characteristics; and Communication Skills). For more information, please refer to Figure 3. Prevention, Crisis Intervention and Mental Health is more commonly scored slightly below other ratings due to less frequent opportunities to demonstrate skills in crisis intervention, but continues to be an area targeted for improvement. Results from the 2010 evaluations indicate Practicum II students area scores (means) ranged from 4.1 (Pupil Service Standards) to 4.8 (Systems and Organizations) on a 5-point Likert scale Evaluation Guide for School Psychology (see See Figure 3). Results indicate that practicum students earned above satisfactory scores(4.0 or higher) in all NASP Domains. Scores on the Stage of Development scale (see Figure 6) indicate that most Practicum II students were between the Practitioner: Initial Stage (5.0) and the Practitioner: Advanced (6.0). Most students were at the Practitioner: Initial Stage (5.0) in all NASP Domains. In several domains practicum students are rated higher on the Stages of Development scale that interns, and higher than anticipated based on their level of training and experience at this point in their programs. This is a trend that has been noticed over several years. Based on previous results and input from field-based supervisors, the format for evaluation tools will be revised over the 2011-2012 school year, due to reported confusion in how to complete this portion of the evaluation tool, which will hopefully remedy these seemingly inconsistent results. 24 Figure 3 – Practicum I Evaluation Means Evaluation Guides - Practicum I Domain Means: Scale 1-5 Pupil Services Standards Communication Skills Professional Work Characteristics Information Technology School Psychology Practice and Development Research and Program Evaluation Home/School/Community Collaboration 2011-12 N=9 Prevention, Crisis Intervention, and Mental Health 2010-11 N=11 School and Systems Organization, Policy Development, and… Student Diversity in Development and Learning Socialization and Development of Life Skills Effective Instruction and Development of Cognitive/Academic… Consultation and Collaboration Data-Based Decision Making and Account 3.6 3.8 4 4.2 4.4 4.6 4.8 5 25 Figure 4 - Practicum II Evaluation Means Evaluation Guides - Practicum II Domain Means: Scale 2-5 Pupil Services Standards Communication Skills Professional Work Characteristics Information Technology School Psychology Practice and Development Research and Program Evaluation Home/School/Community Collaboration 2011-12 N=9 Prevention, Crisis Intervention, and Mental Health School and Systems Organization, Policy Development, and Climate Student Diversity in Development and Learning Socialization and Development of Life Skills Effective Instruction and Development of Cognitive/Academic Skills Consultation and Collaboration Data-Based Decision Making and Account 3.6 3.8 4 4.2 4.4 4.6 4.8 5 26 b) Internship Evaluations Results indicate that internship area mean scores ranged from 5.7 (Interventions and Instructional Support for Academics) to 6.5 (Legal, Ethical and Professional Practices) on the 7-point Evaluation Guide for School Psychology for the Spring 2010 interns. Results indicate that interns earned above satisfactory to highly satisfactory scores (scores of 6.0 and above) in all Evaluation Guide areas (see Figure 5). The revised Intern Evaluation scale is designed to demonstrate growth from the Emerging Practitioner at the Practicum II stage, to the Established and Advanced Practitioner at the Internship stage. Figure 5 – Internship Evaluation Means Evaluation Guides - Internship Domain Means: Scale 3-7 Professional Work Characteristics NASP Domain 10: Legal, Ethical, and Professional Practices NASP Domain 9: Research and Program Evaluation NASP Domain 8: Diversity in Development and Learning NASP Domain 7: Family and School Collaborations NASP Domain 6: Preventative and Responsive Services 2011-12 N=10 NASP Domain 5: School-Wide Practices to Support Learning NASP Domain 4: Interventions and Mental Health Services to… NASP Domain 3: Interventions and Instructional Support to… NASP Domain 2: Consultation and Collaboration NASP Domain 1: Data-Based Decision Making and Accountability 2.8 3 3.23.43.63.8 4 4.24.44.64.8 5 5.25.45.65.8 6 6.26.46.66.8 7 27 Results indicate that interns scored in the Established Practitioner or above range on all Evaluation Guide areas (Data-based Decision Making; Consultation and Collaboration; Effective Instruction and Development of Cognitive/Academic Skills; Socialization and Development of Life Skills; Student Diversity in Development and Learning; School and Systems Organization, Policy Development, and Climate; Prevention, Crisis Intervention, and Mental Health; Home/School/Community Collaboration; Research and Program Evaluation; School Psychology Practice and Development; Professional Work Characteristics; and Communication Skills). Conclusion(s): Reviews of the Evaluation Guide for School Psychology suggest that UW-Stout graduate students are adequately prepared to enter the field of practice with professional skills for success as practicing school psychologists. III. PRAXIS II: CONTENT TEST SUMMARY OR NATIONAL SCHOOL PSYCHOLOGISTS EXAM RESULTS It is a program requirement to take the National Certification in School Psychology (NCSP) exam (PRAXIS II) used by the National Association of School Psychologists (NASP) as requirement for national certification. The passing score established by NASP and recognized by the Wisconsin Department of Public Instruction (DPI) is 165. This new ruling was adopted and implemented starting May 31, 2009. In addition to the total score received by examinees’, the PRAXIS II exam also report scores addressing the following content areas: 1) Data-based Decision Making, 2) Research-based Academic Practices, 3) Research-Based Behavioral and Mental Health Practices, 4) Consultation and Collaboration, 5) Applied Psychological Foundations, and 6) Ethical, Legal, and Professional Foundations. Students and interns can take the national certification exam at any point during their graduate program, preferably prior to the start of their one-year internship. Outcomes of the NCSP Results In 2010-11, 82% of the students obtained scores of 165 or higher on their first attempt at the Praxis, with 100% students passing within 2 attempts. As such, every student who took the PRAXIS Exam met the passing criteria of NASP and DPI. 28 PRAXIS II: Content Test Summary School Psychology Praxis Test Code – 10401 Content Test from ETS 10/11 (0401) 11 Number of Examinees: 179 Highest Observed Score: 162 Lowest Observed Score: 172 Median: Average Performance Range: WI Score Needed to Pass: 168-173 Number with WI Passing Score: 9/11 Percent with WI Passing Score: 82% 165 29 Percent Correct (percentage of items answered correctly) by Area UWNational Points Stout SCOUN Test Category Available 10/11 10/11 Data-Based Decision Making 41-42 67 68 Research-Based Academic Pract. 14-15 84 76 Research-Based Behavioral & Mental 18-19 74 76 Health Practices Consultation & Collaboration 14-15 69 71 Applied Psychological Foundation 15-16 69 69 Ethical/Legal & Prof. Foundations 13-14 59 74 Figure 7 – Praxis II Domain Percentages 90% 80% 70% 60% 50% 40% PRAXIS II: Institution Average % Correct 08/09 30% PRAXIS II: Institution Average % Correct 10/11 20% 10% 0% Data-Based Research-Based Research-Based Consultation Decision Making Academic Behavioral and and Practices Mental Health Collaboration Practices = National Average per Domain Applied Ethical/Legal Psychological and Professional Foundation Foundations 30 Conclusions Overall results indicate UW-Stout’s School Psychology program is preparing students to achieve the foundational knowledge necessary to pass the NCSP exam. Review of domain area trends suggest that our students are making gains and/or demonstrating strengths in knowledge related to Data-based Decision-Making, Research-based Academic Interventions, Research-based Mental Interventions, and Applied Psychological Foundations. Areas identified for growth include Consultation and Collaboration, and Legal and Ethical Professional Foundations. Information from PRAXIS II data review will be used to inform on-going program review with focus on continuous improvement in identified areas, as well to inform program development in general. IV. OTHER INFORMATION a) Portfolio Assessment (Benchmark II and Benchmark III) Students initiate and complete components of a portfolio over the course of the program. The completed portfolio includes evidences or artifacts demonstrating professional competence in all eleven (11) NASP’s Domains of School Psychology training and practice (same as the Department of Public Instruction content guidelines). A partially completed portfolio (Benchmark II) demonstrating competency in 8 out the 11 domains is submitted to the program faculty for review after completion of the M.S.Ed. and prior to admittance to the Ed.S. in School Psychology Program. At Benchmark III, students re-submit their portfolio to demonstrate competency in all eleven (11) domains. Faculty members rate the portfolio materials. Based on a review of the portfolio ratings, academic performance, and professional conduct, a student is either recommended for, or denied admittance to, the Ed.S. degree in School Psychology (for Benchmark II) or Internship (Benchmark III). Outcomes of the Portfolio Assessment A review of the portfolio ratings in 2011 revealed all students produced satisfactory portfolios at Benchmarks II and III. b) Survey of Intern Result In 2010, as in previous years, school psychology interns were surveyed to determine their perceptions regarding their previous coursework. Each intern was asked to respond to questions on a 5-point Likert-style survey (1 = No knowledge or Skill gained to 5 = A lot of knowledge or Excellent skills gained) to assess how much knowledge or skill the student gained in each course. Further, interns are asked to identify the five most important courses in terms of how helpful they were in preparing them for their internship year. They also are asked to identify which courses were not helpful or had overlapping content. 31 Outcomes of the Intern Survey Results from the spring of 2011 survey indicate students believed that they gained above average to a lot of knowledge and skill (x = 4.31) in most program courses. However, students indicated that they gained below average knowledge and skills in the Research Foundations (EDUC 740, x = 2.66). Additional comments indicated that most students reported they believe they were well prepared for the field of school psychology after taking all program requirements. Feedback included commentary about the nature of leadership in the program, and suggestions for enhancing mentorship to students. Conclusion(s): Intern survey responses suggested the need for further review of the EDUC 740 Research Foundations course. Although content related to research is considered foundational knowledge for Masters and Ed.S. Thesis courses, more information is needed to determine whether the course could emphasize more theoretical and research-related information related to be more focused on student research. Some narrative feedback articulated a desire for more statistical analysis content in research foundations coursework. c) Program Advisory Committee Results The program faculty members meet with the School Psychology Program’s Advisory Committee (PAC) two times a year. Internal and external colleagues meet to discuss updates in the program and solicit information from the committee about program needs and future goals. The 2011-12 PAC for school psychology was re-organized, with new members identified. Current program consists of 7 colleagues, including practitioners int he field, representatives from higher education, and the state dept (DPI). Outcome(s) of the Program Advisory Committee Results: Most recent PAC focus was on the revisions to the program. Ongoing discussions related to student research requirements and process were started at the last meeting and will continue this Fall. V. PLAN FOR COMMUNICATING ASSESSMENT RESULTS The School Psychology Program faculty meets on a bi-weekly basis to discuss program issues and plan for the future. This Assessment in the Major report will be disseminated to all members of the program faculty, and the results will be discussed by the program faculty at an upcoming meeting. This guide is a valuable resource for faculty to use in guiding program revisions. Furthermore, this report will be disseminated to the Program Advisory Committee (PAC) in the fall of the 2012 academic year. 32 VI and VII. CONCLUSIONS AND ACTION PLAN TO IMPROVE TEACHING AND/OR ADVISEMENT IN THE SCHOOL PSYCHOLOGY PROGRAM Summary and review of all available data related to the School Psychology programs at UW-Stout, the following goals are targeted for the 2012-13 year: A. Enhanced support for students in preparation for taking the Praxis exam. This may include: providing test prep materials, available though the Resource Library, and/or in coordination with the Praxis Lab. B. Student research requirements for the M.S. and Ed.S. programs needs a thorough review, with suggestions for modifications. This will be a focus of program review, as well as PAC input this Fall of 2012. The pros and cons of EDUC 740 has been a point of consistent feedback from students, both during the program and on the intern exit survey. All elements of student research need to be reviewed and potentially revised to better meet program goals. C. Students benefit most from applied experiences in the field, from the earliest point in their education, that allows them to observe, practice and reflect on the skills they are learning. Faculty will work to increase the number of applied activities available to students. This may include: partnering with district(s) to conduct CBM assessments, FBA assessments, early childhood screening and/or benchmarking, etc. D. Complete and seek full approval during the 2013 NASP review cycle. This may include winterm and/or summer collaborative workgroups to successfully compile and submit all required material for review. NASP approval is a priority for the continued success of the program. E. Program faculty will work with SOE support members to transition to an E-portfolio process for students. F. In coordination with efforts and supports already in place, increase marketing and recruitment efforts for the program, with the goal of a larger pool of high quality applicants to the program, better ensuring full cohorts of qualified graduate students. Respectfully submitted, Christine R. Peterson, Ph.D. Assistant Professor Program Director for School Psychology University of Wisconsin-Stout