Program Report for School Psychology UNIVERSITY OF WISCONSIN-STOUT September, 2008 Table of Contents SECTION ...................................................................................................................... Page List of Appendices ...............................................................................................................2 Letter to Reviewers .............................................................................................................3 Matrix Response to 2006 NASP Review.............................................................................5 Standard 1.2 – Narrative Response ....................................................................................16 Standard 1.4 – Narrative Response ....................................................................................18 Standard 2.1 – Narrative Response ...................................................................................20 Standard 2.2 – Narrative Response ....................................................................................24 Standard 2.3 – Narrative Response ....................................................................................28 Standard 2.4 – Narrative Response ....................................................................................31 Standard 2.5 – Narrative Response ....................................................................................35 Standard 2.6 – Narrative Response ....................................................................................38 Standard 2.7 – Narrative Response ....................................................................................41 Standard 2.8 – Narrative Response ....................................................................................45 Standard 2.9 – Narrative Response ....................................................................................47 Standard 2.10 – Narrative Response ..................................................................................50 Standard 2.11 – Narrative Response ..................................................................................54 Standard 3.2 – Narrative Response ....................................................................................56 Standard 3.5 – Narrative Response ....................................................................................57 Standard 4.3 – Narrative Response ....................................................................................58 1 Appendices Previous Program Manuals and Handbooks ....................................................................... A Syllabi and Rubrics ............................................................................................................. B Candidate Work Samples and Artifacts .............................................................................. C Revised Handbooks and New Syllabi................................................................................. D Supportive Materials ........................................................................................................... E 2 Dear Program Approval Board members, Thank you for taking the time to review and provide feedback to the University of Wisconsin-Stout’s School Psychology Program. Enclosed you will find four copies of our report and appendices for your review. In 2006, our program was granted Conditional Approval for two years. As requested, the enclosed report addresses only those standards considered to be NA (Standard is Not Adequately Addressed) at the time of the last review. Thus, although the report is organized in numerical sequence, only sixteen standards are addressed. Please also note that, unlike our 2006 submission, we have included all course rubrics with our program syllabi to provide evidence that our students are assessed and graded congruent with our program goals. In addition, representative candidate work samples are included to demonstrate the attainment of these goals. To ensure our program meets or exceeds both the letter and intent of the standards set forth by the National Association of School Psychologists, we hired a program consultant (i.e., Andrea Canter), hired a third full time faculty member with a doctorate, and implemented several programmatic changes since the 2006 review. The following is a short summary of the major program changes reflected throughout our report and included the appendices: 1) We have infused more content related to diversity, prevention, academic interventions, behavioral interventions, and methods to evaluate the effectiveness of interventions throughout the curriculum. 2) We have updated both the title and content of one required course (i.e., now SPSY-768 Learning Disabilities: Assessment and Intervention instead of SPSY786 Diagnosis and Remediation of Learning Disabilities and SPSY-786a LD Lab). 3) A new course (SPSY-800 Systems Level Prevention and Intervention) has been created to insure students are prepared in the areas of systems level consultation, prevention, and intervention (including crisis intervention) services. This systems level course will go through our institutional curriculum-approval process and become a required course for all candidates entering the program in the fall of 2009. 4) The practicum and internship syllabi, manuals, and evaluation tools have been modified significantly. We have differentiated the knowledge/skills expected of candidates in their first practicum, second practicum, and internship. We also added practicum/internship requirements to include consultation, behavioral intervention, and academic intervention case studies. As such, our candidates’ 3 field experience expectations and evaluations more accurately reflect their point of training within the program. 5) We have provided documentation of our candidates’ experiences in diverse school-based settings prior to their internship year. Thank you for your consideration of our report. Because our School of Education is a formal candidate for NCATE accreditation (refer to page 110 in Appendix E), we have not submitted a fee. If you have questions or are in need of additional materials, please contact me at weissenburgerj@uwstout.edu or 715-232-1326. We look forward to your review. Sincerely, Jacalyn W. Weissenburger, Ph.D. Program Director, School Psychology 409 McCalmont Hall University of Wisconsin-Stout Menomonie, WI 54701 4 REVIEWER’S GUIDE GENERAL COMMENTS BASED ON THE 2006 NASP REVIEW COMMITTEE FINDINGS Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance (outlined by NASP Review Committee) General Comments General Comments A more systematic approach to preparation in the areas of organization and operation of schools and school systems, prevention at the school or system level and crisis intervention is needed. The program might also consider adding specific activities related to development and implementation of academic interventions that are required for all candidates, and to a requirement that all practicum candidates engage in systematic consultation with teachers. 5 More emphasis in this area has been added to several courses since the last review. In addition, a revision to SCOUN-798 (now COUN-788) covers more content in this area. A new course entitled SPSY 800 Systems Level Prevention & Intervention will be implemented in Spring of 2009 and will be required of all program candidates beginning the program in Fall of 2009. Refer to Appendix E, Course Alignment Grid, p. 8-9. Academic interventions have been an emphasis in SPSY-768/768a since the last review. A recent revision to the course syllabus and the associated rubric makes this emphasis more explicit. The course is now entitled LD: Assessment and Intervention. Furthermore, revisions to our Practicum I and II require systematic consultation and the implementation of an academic intervention for all candidates. Refer to Appendix D, Practicum I and II Handbooks, p. 8, p. 34, p. 54, and p. 80. Refer to Appendix B, COUN 788, pp. 100-116. Refer to Systems Level Prevention & Intervention course proposal in Appendix D, pp. 132-138. Refer to Appendix B, SPSY 768/768a, pp. 89-99. Refer to Appendix D, SPSY 768, pp. 139-149. Areas for Improvement (outlined by NASP Review Committee) General Comments General Comments NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance Rubrics have been developed for some evaluation tools and should be expanded for all tools used to assess candidate progress throughout the program. This is particularly critical for differentiating the depth and breath of knowledge and skills expected for practicum candidates versus interns, given that the same evaluation tools are used for both groups. Practicum and internship evaluation tools have been revised. Although Practicum I, Practicum II and Internship all have similarly organized evaluation tools, different items and different expectations are described for each level of experience. Refer to Appendix A, for previous Practicum I, Practicum II, and Internship Handbooks. Methods for collecting data across all candidates related to assessment of measurable positive outcomes for all clients are also needed. All candidates collect data to assess positive outcomes (behavioral and academic) for candidates in coursework (SPSY 690 and SPSY 768), in practica, and during internship. Examples of courses and field activities are provided as evidence in this area. Refer to Appendix A and Appendix D, Practicum I and II Handbooks, as well as Internship Handbook. 6 Refer to Appendix D for newly revised Practicum I, Practicum II and Internship Handbooks. Refer to Appendix C, SPSY 690, pp. 1-97, SPSY 768, pp. 98-107, SPSY 781/782, pp. 108-123, and SPSY 792, pp. 157-164. I. Program Context/Structure Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance The required course and corresponding field experience in multiculturalism is now noted on all syllabi submitted as evidence of practice. However, the courses listed as integrating attention to diversity in the curriculum, syllabi for several do not include objectives, readings and/or lectures related to diversity. Field practica syllabi do not include objectives related to diversity. The program should consider expanding methods to collect direct and indirect intervention outcome data. Documentation of these efforts would enhance evidence of practice related to policy. The program has made a concerted effort to infuse culturally and linguistically diverse content with related objectives into most program courses and field experiences (i.e., 21). In addition, 11 of our courses have requirements that explicitly assess content through the use of rubrics in this area. In addition, our candidates are placed in a diverse setting for at least one of their applied field experiences. Further, we actively recruited and hired a faculty member with minority status for our third faculty member in 2007. Refer to Appendix B, SPSY 768/768a, pp. 89-95; SCOUN 788, pp. 100-116; SPSY 745, pp. 124139; SPSY 710, pp.140-148; SPSY 775, pp.149-163; and EDUC 536, pp. 244-257. The program hired our third FTE school psychology faculty member in the fall of 2007. Refer to Appendix E, p. 29, for Dr. Carlos Dejud’s vita. (outlined by NASP Review Committee) 1.2 1.4 A successful outcome of the current search for a third FTE school psychology faculty member will adequately address this standard. 7 Refer to Appendix C, SPSY 781/782, pp.117-123; SPSY 792, pp.166-167; SPSY 792, pp.168205. Refer to Appendix E, p. 8 and 12. Refer to Appendix E, p. 29, for Dr. Carlos Dejud’s vita. II. Domains of School Psychology Training and Practice Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance (outlined by NASP Review Committee) General Comments Given that the Evaluation Guide for both Practicum and Internship contain identical items and ratings scales, criteria for determining ratings would be helpful in clarifying the greater depth of knowledge and skills expected at the intern versus practicum level. The evaluation forms for Practicum I, Practicum II, and Internship have been revised. More clarification has been added and practicum and internship fieldwork expectations are differentiated by level. Further, the evaluation tools contain different items at each level. Refer to Appendix D for the newly revised Practicum I, Practicum II, and Internship Handbooks. General Comments There is not a correspondence between several of the NASP domains and items on the Evaluation Guide, although results from the Evaluation Guide are offered of evidence of attainment for each domain. The evaluation forms have been revised to conform to NASP’s domains. The assessment items are organized into each domain area. Refer to Appendix D for the newly revised Practicum I, Practicum II, and Internship Handbooks. General Comments The lack of a rubric that explains how ratings are obtained for the Evaluation Guides also renders this tool less useful as evidence of attainment. While a useful rubric is supplied with the Developmental Appraisal Form for practicum candidates, there is no corresponding rubric at the Intern level. Moreover, items on the form are very general and not very useful in indicating competence of specific skills in a given domain. Descriptors of all evaluation tools for Practicum I, Practicum II, and Internship have been included in each newly revised Handbook. Furthermore, the items have been revised to indicate specific skills required at each level. Refer to Appendix D for the newly revised Practicum I, Practicum II, and Internship Handbooks, p. 20, pp. 43-44, p. 66, pp. 88-89, p. 110, and pp. 130-131. General Comments The Portfolio Assessment Rubric would be strengthened by inclusion of specific criteria for making the determination that a student is working at the observer, novice, or practitioner level. To use grades from course-embedded assignments as evidence of assessment and attainment, the program should provide rubrics or the criteria by which grades are determined. Descriptors of all evaluation tools for Practicum I, Practicum II, and Internship have been included in each Handbook. Furthermore, the items have been revised to indicate specific skills required at each level. Course rubrics have been included in this report. Refer to Appendix D for the Practicum I, Practicum II, and Internship for the criteria (pp. 4344, pp. 88-89, and pp. 130-131). 8 Refer to Appendix B for the syllabi and course rubrics. Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance (outlined by NASP Review Committee) 2.1 Counseling and consultation courses, fieldwork, and practica do not address using data to evaluate the effectiveness of services. In SPSY 781/782, a rubric, with items measuring specific skills, might be used to evaluate the case study to provide evidence of assessment. Assessment skills in data-based decision making as measured by the Developmental Performance Appraisal is limited to test administration and interpretation rather than to a full range of psychological services. The Evaluation Guide for School Psychology Practicum includes an item related specifically to evaluating the effective implementation of intervention plans, but not the evaluation of intervention outcomes. Rubrics for the counseling and consultation courses have been provided in this report. The program’s School Consultation course (SPSY-775) has been revised to address using data to evaluate the efficacy of our candidate’s consultation skills (objective #12). A rubric has been created to evaluate the effectiveness of intervention or consultation services for Practicum I and Practicum II. Furthermore, Practicum I, Practicum II, and Internship have been revised to require all candidates to evaluate case studies to provide evidence of the effectiveness of interventions (academic or behavioral). Examples of courses and field activities are given to demonstrate student opportunities to acquire data-based decision making skills. Further, newly revised evaluation tools for Practicum I, Practicum II, and Internship clearly assess candidates’ skills in evaluating intervention outcomes. 9 Refer to Appendix A syllabi and rubrics. Refer to Appendix B, SPSY 781/782, pp. 174-178. Refer to Appendix C, Behavior Intervention Project, pp. 1-97, Curriculum-based Measurement Project, pp. 98-107; and Psychological Consultation Project, pp. 108-123. Refer to Appendix D, for newly revised field experience evaluation tools. Refer to Appendix C, pp. 1-156, for work samples. Areas for Improvement NASP Review Committee Comments How this Area Addressed In the course outline (SPSY 775), goals and competencies are vague with respect to skills taught. There do not appear to be specific requirements for engaging in consultation during practicum. In regards to assessment, although the syllabus for SPSY 775 mentions a Consultation Paper rating/Grading Criteria form, it was not included with the syllabus. Items on the Alumni and Employer Survey provided as evidence of attainment in this domain, relate to skills only indirectly related to consultation. The program might consider expanding items related to providing systematic consultation services that address candidates’ academic and behavioral difficulties. See General Comments. Since the last review, SPSY 775 was revised to reflect an emphasis on developing specific consultation skills. Evidence of Program Compliance (outlined by NASP Review Committee) 2.2 In addition, consultation paper rating/grading criteria forms for SPSY 775 have been included in this report. Twelve other program courses address content relevant to this area, and new systems level consultation course has been created. Further, Practicum I and Practicum II candidates are now required to engage in a consultation case study and present results, and candidates’ skills in this area are more explicitly assessed. Lastly, course rubrics containing items pertinent to the assessment of communication and consultation skills have been included. 10 Refer to Appendix A, Practicum I and II Handbooks. Refer to Appendix B, SPSY 775, pp. 160-161. Refer to Appendix C, pp. 1-97, pp. 108-123, and pp. 157-164, for work samples. Refer to Appendix D, for newly revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools. Refer to Appendix E, pp. 132-138, for newly revised systems-level consultation course. Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance (outlined by NASP Review Committee) In regards to preparation related to development of instructional and academic interventions, SPSY 768 and its associated lab include an objective related to “understanding effective academic interventions.” It is not clear from the syllabus where candidates are taught to develop an actual academic intervention or are expected to implement an academic intervention. The program’s response to this standard describes an expectation for weekly monitoring of a student’s progress through CBM probes, but it is not clear where this requirement is explained to candidates. The Practicum and Intern Guidelines include items related to behavioral interventions and counseling, but not to academic interventions. SPSY 768’s syllabus and requirements have been revised to more clearly require the implementation of an academic intervention and weekly progress monitoring. Refer to Appendix B, SPSY 768, pp. 89-96 A CBM rubric is included to evaluate candidates’ efficacies in this area. Newly revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools require and assess candidates’ intervention (academic and behavioral) skills. Refer to Appendix E, newly revised Practicum I, Practicum II, and Internship Handbooks, p. 8, p. 23, p. 34, p. 54, p. 69, p. 113, pp. 123124. 2.4 Content knowledge is addresses through required coursework. Evidence of assessment would be enhanced through provision of rubrics used by faculty to evaluate live and videotaped observations of candidate’s counseling skills. See General Comments. Counseling is an emphasis in the program. Coursework rubrics have been included in this report to demonstrate that candidates’ counseling and other skills relevant to Domain 4 are evaluated. Refer to Appendix B for counseling course syllabi and rubrics. 2.5 Content knowledge is addressed through required coursework, although there is limited evidence that biological bases of behavior are covered in the course referenced. See General Comments. Updated syllabi and rubrics for SPSY 778 and PSYC 850 show evidence that the biological bases of behavior are addressed and assessed. Refer to Appendix B, SPSY 778, pp. 28-48 and PSYC 850, pp. 223232. 2.3 11 Refer to Appendix D, Practicum I, Practicum II, and Internship Handbooks. Refer to Appendix E, pp. 139-149. Areas for Improvement NASP Review Committee Comments How this Area Addressed Organization and operation of schools and school systems does not appear to be addressed in any systematic way. SPSY 775 lists managing resistance in systems course as a course objective, but it is not clear from the weekly outline where this is actually addressed. School organization and climate is broader than this, however, and it appears that candidates are expected to develop this knowledge base incidentally, through field experiences, rather than through direct instruction. See General Comments. Issues in question are addressed in 8 courses and with additional emphasis in SPSY 775 and SCOUN 788. It is unclear where candidates are prepared in the area of crisis intervention. The program has expanded attention to prevention in several courses; however, this appears limited to prevention at the individual student level. School or system-wide prevention does not appear to be addressed. Evidence of assessment would be enhanced through provision of rubrics used by faculty to evaluate live demonstrations of candidates’ counseling skills. The Evaluation Guide, offered as evidence of attainment, does not contain items related to prevention or crisis intervention. See General Comments. Issues in question are addressed in courses COUN 750, COUN 788, SPSY 690, SPSY 768, as well as in practicum and internship. Evidence of Program Compliance (outlined by NASP Review Committee) 2.6 2.7 In addition, a new course (SPSY 800 Systems and Crisis Intervention) will be added as a required course after universitylevel approval. In addition, a new course (SPSY 800 Systems and Crisis Intervention) will be added as a required course after approval from the university’s curriculumapproval process. Counseling rubrics have been included to show we evaluate live demonstrations of candidates’ counseling skills. 12 Refer to Appendix B for course syllabi and rubrics. Refer to Appendix D, pp. 132-138 for new course (SPSY 800 Systems and Crisis Intervention). Refer to Appendix B for course syllabi and rubrics. Refer to Appendix D, pp. 132-138 for new course (SPSY 800 Systems and Crisis Intervention). Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance Evidence provided through syllabi continues to suggest that this area is addressed in very limited manner. Course syllabus for SPSY 775 is unclear where the skills necessary to collaborate with families are taught. While the syllabus for SPSY 784 lists all NASP domains as course objectives, the course requirements and evaluation criteria do not suggest emphasis on family collaboration. See General Comments. Syllabi were revised and rubrics are included to demonstrate how this domain is addressed and assessed in several program courses. Further, a new course (SPSY 800 Systems and Crisis Intervention) will be added as a program requirement. Newly developed practicum and internship evaluation tools verify an added emphasis in this area. Refer to Appendix B for course syllabi and rubrics. 2.9 The Evaluation Guide, offered as evidence of attainment, does not contain items related to program evaluation. See General Comments. Newly developed practica and internship evaluation tools clearly assess candidates’ program evaluation skills. Refer to Appendix D for revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools. 2.10 Content knowledge is addressed. See General Comments. Newly developed practica and internship evaluation tools clearly assess candidates’ skills in this area. Refer to Appendix D for revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools. 2.11 Content knowledge is limited to the school psychologist’s use of technology and does not included technology related to providing services to clients. While the summary of ratings from 2004-05 includes Information Technology (Appendix C, p. 45), the completed Developmental Appraisal Guide (Appendix C, p. 60) does not include items that pertain to information technology. Although this domain has been added to the form in the 2006 revision to the Practicum Handbook, it is unclear on what the numerical ratings from 2004-05 are based. See General Comments. Issues in question are addressed in courses SPSY 690, SPSY 768, SPSY 777, SPSY 778, as well as in practicum and internship. Assistive technology will be given more emphasis in SPSY 778 beginning this fall (2008). Refer to Appendix B for course syllabi and rubrics. (outlined by NASP Review Committee) 2.8 13 Refer to Appendix D, page 132138, for SPSY 800. Refer to Appendix D for revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools. Refer to Appendix D for revised Practicum I, Practicum II, and Internship Handbooks and evaluation tools. Refer to Appendix D, p. 158. III. Field Experiences/Internship Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance A single internship summary log, for Feb. 13 – June 19, is provided as evidence of practice. Note: The form used for this summary is the same one used on a weekly basis, and thus is very difficult to interpret. To meet this standard, intern activity summaries for a recent graduating cohort would be helpful. Internship summary logs for 2006 through 2008 cohorts are provided as evidence of this content. Students now complete separate forms to keep track of their weekly activities and to summarize their activities at the end of the semester. Refer to Appendix C, pp.165-167 for a listing of Intern Activity Summaries; pp. 168-179, Fall 2006 Intern Logs; pp. 180-190, Spring 2007 Intern Logs; pp. 191-197, Fall 2007 Intern Logs; and pp. 198-204, Spring 2008 Intern Logs. (In the past, some students used the wrong form when submitting their summaries to the university supervisor at the end of the semester.) Also, refer to newly created log forms, in Appendix D, pp. 45-46, pp. 91-92, and pp. 128-129. Internship summary data and logs from the 2006 through 2008 cohorts are provided for evidence of practice. Refer to Appendix C, pp.165-204. (outlined by NASP Review Committee) 3.2 3.5 A single internship summary log is provided as evidence of practice. 14 IV. Performance-Based Program Assessment and Accountability Areas for Improvement NASP Review Committee Comments How this Area Addressed Evidence of Program Compliance (outlined by NASP Review Committee) 4.3 Documentation that interns engage in delivery of a comprehensive range of services would be engaged by provision of intern activity summaries for a recent graduating cohort. Evidence that services delivered by interns results in “measurable positive impact on children, youth, families, and other consumers” would be satisfied through a summary of the results of a comprehensive, performance-based assessment of candidate abilities (e.g. case studies, portfolio) evaluated by faculty during internship. The two projects offered as evidence of practice are required during courses taken primarily during the first year of the program. The Evaluation Guide items offered as evidence of practice focus on what the candidate does rather than on the outcomes of candidate activities for clients. 15 Recent program cohort activity summaries are provided. Newly revised summary and evaluation forms are included in the new practicum and internship handbooks. Intern portfolios and projects will be assessed at the conclusion of the internship year (Benchmark IV) to determine whether candidates can integrate their skills and make a “measureable positive impact.” Refer to Appendix C, pp. 157-164 and pp.166-167. Refer to Appendix D for new Internship Handbook. 1.2 A commitment to responding to human diversity is articulated in the program’s philosophy/mission, goals and objectives and practiced throughout all aspects of the program including admissions, faculty, coursework, practica, and internship experiences. Human diversity is a strength that is valued and appreciated. 1.2 Policy: The program adheres to the University of Wisconsin-Stout’s non-discrimination policies available online at http://www.uwstout.edu/asls/policies and presented in Appendix E, page 118. In addition, the program adheres to UW-Stout’s Strategic Plan for Diversity that includes a goal to “foster institutional environments and course development that enhance learning and a respect for racial and ethnic diversity.” The School Psychology Program is housed in UW-Stout’s School of Education (SOE) and supports SOE’s mission, goals, and values. SOE goals and values (refer to School of Education’s Goals and Values in Appendix E, page 116) reflect an appreciation for human diversity and an understanding of the importance of educating professional educators to acquire the knowledge and skills to provide services in a multi-cultural, pluralistic society. SOE goals include: 1) provide program curricula that reflect diversity, research, theory, and best practice, and 2) recruit, support, and retain a diverse faculty/staff who model best practice in professional education. SOE values include: 1) value candidate/faculty diversity, 2) model respect for differences, 3) recruit and support a diverse candidate/faculty population, and 4) require candidate experiences that address diversity. The program also adheres to the National Association of School Psychologists (NASP) guidelines for working with diverse populations. In addition, the program has adopted NASP’s Domain 5: Candidate Diversity in Human Development as one of the program’s goals (see the School Psychology Program Handbook in Appendix A, pages 9-10). As such, candidates are expected to value human diversity, as well as develop additional knowledge and skills relevant to the area of human diversity as they progress through the program. 1.4 Practice: All candidates take a required course and participate in a field experience in which the acquisition of knowledge and skills in human diversity is targeted (EDUC-536 Multiculturalism: Issues and Perspectives, SPSY-778 Psychoeducational Disabilities, and EDUC-576 Field Experience: Cross Cultural). These two requirements specifically address candidates’ understanding and appreciation of diversity through required readings, lectures, discussions, presentations, and applied experience (50 hours) in a multicultural/diverse setting (refer to pages 244 to 278 in Appendix B) . Candidates also take 15 additional program courses, five practica, and a two-semester internship in which content relevant to human diversity has been integrated into the curriculum (see Course Alignment grid in Appendix E on pages 8-9 and Appendix B for 16 course syllabi and rubrics). Nine of the 15 courses explicitly address diversity through lectures and readings: • • • • • • • • • COUN 750 COUN 788 SPSY 753b SPSY 778 SCOUN 690 SPSY 743 SPSY 710 SPSY 775 SCOUN 705 Counseling Theory Counseling Process Lab School Psychology Lab Psychoeducational Disabilities Behavioral Interventions in the Schools (now SPSY-690) Cognitive Assessment Psychoeducational Assessment of the Young Child School Consultation Play Therapy Further, 6 program courses both address (lectures and readings) and explicitly assess (rubrics) candidate knowledge and/or skills in the area of diversity: • • SPSY-753 SPSY-777 • SPSY-768/768A • • • SPSY-745 SPSY-701 PSYC-850 Psychometric Theory and Application Legal/Ethical Issues: School Counselors and School Psychologists Diagnosis & Remediation of Learning Disabilities/LD Lab (now SPSY-768 LD: Assessment and Intervention) Assessment of Personality Seminar in School Psychology Psychology of Development The University of Wisconsin-Stout is located in a small community in northwestern Wisconsin. As such, the community’s population is largely homogeneous (i.e., largely White or Caucasian). However, the university is located within driving distance to a large, metropolitan community (Minneapolis/St. Paul, MN) and other rural settings where their populations include economically disadvantaged families and migrant workers from other countries. Therefore, program faculty members work to ensure that candidates have at least one practicum placement in a culturally diverse and/or economically depressed setting (see field experience placements for one cohort in Appendix E, page 12). Further, because of the program values and appreciates human diversity, the program faculty actively recruited one full time school psychology faculty member from a different culture. As such, in spring of 2007, the program’s faculty members were able to hire Dr. Carlos Dejud who is bilingual and has ethnic minority status (see Dr. Carlos Dejud’s vita on page 29 in Appendix E). Furthermore, Dr. Dejud has worked with program faculty to develop a 4-week summer immersion program in Panama to increase students’ global awareness and knowledge/skills related to culturally and linguistic diverse students and other systems of service delivery. This program will be offered for the first time in summer of 2009. The program also has worked to revise their practicum/internship evaluation forms to evaluate candidates’ appreciation of human diversity and their skills in working with diverse populations (see newly revised practicum and internship handbooks in Appendix 17 D, pages 20-33, pages 66-79, and pages 110-122). In sum, our commitment to diversity is reflected by our efforts to: (a) infuse content on human diversity into our coursework, (b) provide candidates with field experiences in diverse settings, (c) hire a faculty member from diverse background, and (d) develop rubrics/evaluation tools to assess candidate knowledge/skills in the area of human diversity. 1.4 The program possesses at least three full-time equivalent faculty. At least two faculty members (including the program administrator) shall hold the doctorate with a specialization in school psychology and be actively engaged in school psychology as a profession (e.g., by possessing state professional and/or national credentials, having experience as a school psychologist, participating in professional school psychology associations, and/or contributing to research, scholarly publications, and presentations in the field). Other program faculty possess the doctoral degree in psychology, education, or a closely related discipline with a specialization supportive of their training responsibilities in the school psychology program. 1.4 Policy: The School Psychology Program follows the University of WisconsinStout’s School of Education policy and hires those who have credentials that allow them to be successful in teaching, supervision, research and service. In 2004, three full-time faculty members (i.e., 100% assignment) were allocated to the program by the newly established School of Education by the University of Wisconsin-Stout. 1.4 Practice: Due to the loss of one core tenure-track school psychology faculty member to another university in July of 2004 and an unexpected administrative leave of another core school faculty member in December of 2004, UW-Stout’s School of Education underwent searches to replace these two fulltime tenure-track school psychology faculty positions during the 2004-2005, the 2005-2006, and the 2006-2007 academic years. Although the 2004-2005 search proved unsuccessful, the School of Education was able to fill one of the two open school psychology positions in spring of 2006 (Dr. Crystal Cullerton-Sen) and the other open school psychology position in spring of 2007 (Dr. Carlos Dejud). Beginning in the fall of 2004 until the fall of 2007, affiliate faculty and adjunct faculty filled in to teach required courses, supervise practicum/interning candidates, and advise theses while we have conducted our faculty searches. During the 2007-2008 academic year, the program’s full-time school psychology faculty included: Dr. Jacalyn Weissenburger (Program Director and Associate Professor), Dr. Crystal Cullerton-Sen (Assistant Professor), Dr. Carlos Dejud (Assistant Professor), and Ms. Kelly Lamon (Associate Lecturer). Additional affiliate faculty members teach program coursework and provide thesis advisement. Affiliate faculty hold doctorates in school psychology counseling, education, or psychology. Core Faculty: Dr. Jacalyn Weissenburger (Associate Professor) has an Ed.S. in School Psychology from the University of Kansas and a Ph.D. in Educational Psychology (with a 18 concentration in Special Education and subprogram specializations in School Psychology and School Counseling) from the University of Minnesota. Dr. Weissenburger is a Nationally Certified School Psychologist (NCSP) and is state certified in Wisconsin and Iowa as a School Psychologist. Dr. Weissenburger has 16 years experience as a school psychologist in the public schools, has a presentation record at NASP, and has published in the field. Dr. Crystal Cullerton-Sen (Assistant Professor) has a Ph.D. in Educational Psychology with a concentration in School Psychology and a subprogram specialization in Child Psychology from the University of Minnesota. Dr. Cullerton-Sen has experience as a school psychologist in the public schools in Massachusetts. Dr. Cullerton-Sen has a record of contributing to publications and presentations in the field. Dr. Carlos Dejud (Assistant Professor) has a Ph.D. in Bilingual School Psychology with a double minor in Emotional/Behavioral Disorders and Bilingual Special Education from the University of Arizona. Dr. Dejud has experience as a school psychologist in Arizona. Dr. Dejud has publication and presentation record demonstrating his contributions to the field of school psychology. Ms. Kelly Lamon (Associate Lecturer) was hired to temporarily fill the third full-time open faculty position during the 2006-2007 year due to a failed search for a doctorallevel faculty member. Ms. Lamon continues to supervise candidates in the program. Ms. Lamon has an Educational Specialist degree (Ed.S.) in Counseling and Psychological Services. She has 7 years experience as a practitioner in the field of school psychology, and Ms. Lamon is state certified in Wisconsin as a School Psychologist. 2007-2008 Course Responsibilities of Core Faculty: Fall 2007 Jacalyn Weissenburger, Ph.D. SPSY-753b School Psychology Lab SPSY-753 Psychometric Theory and Application SPSY-784/785 Clinical Practicum School Psychology Program Director 1 credit 2 credits 3 credits 3 credits Crystal Cullerton-Sen, Ph.D. SPSY-775 School Consultation SPSY-745 Assessment of Personality SPSY-781/782 Field Practicum Supervision 2 credits 2 credits 5 credits Carlos Dejud, Ph.D SPSY-778 Psychoeducational Disabilities SCOUN-690 (now SPSY-690) Behavioral Interventions SPSY-781/782 Field Practicum Supervision 2 credits 2 credits 5 credits 19 Kelly Lamon, Ed.S. SPSY-792 Intern in School Psychology Supervision 4 credits Spring 2008 Jacalyn Weissenburger, Ph.D. SPSY-786/786A Diagnosis of LD/LD Lab (now SPSY-768 LD: Assessment and Intervention) SPSY-784/785 Clinical Practicum School Psychology Program Director 3 credits 3 credits Crystal Cullerton-Sen, Ph.D. PSYC-850 Psychology of Development SPSY-743 Cognitive Assessment SPSY-781/782 Field Practicum Supervision 3 credits 3 credits 3 credits Carlos Dejud, Ph.D SPSY-775 School Consultation SPSY-690 Behavioral Interventions EDUC-536 Multiculturalism SPSY-781/782 Field Practicum Supervision 2 credits 2 credits 2 credits 3 credits Kelly Lamon, Ed.S. SPSY-792 Intern in School Psychology Supervision 4 credits 3 credits Affiliate Faculty: Dr. Amy Gillett, Dr. Mary Beth Tusing, Dr. Denise Zirkle Brouillard, Dr. Barbara Flom, Dr. Stephen Shumate, Dr. Ed Biggerstaff, Dr. Fred Weissenburger and Dr. Helen Swanson contribute to the program through teaching courses, providing thesis advisement, and/or serving on candidates’ research committees. All affiliate faculty members have doctoral degrees in counseling, education, psychology, or school psychology. Core and affiliate faculty vitas are included in Appendix E, pages 23-100. Refer also to Appendix E, page 115. 2.1 Data-Based Decision Making and Accountability: School psychologists have knowledge of varied models and methods of assessment that yield information useful in identifying strengths and needs, in understanding problems, and in measuring progress and accomplishments. School psychologists use such models and methods as part of a systematic process to collect data and other information, translate assessment results into empirically based decisions about service delivery, and evaluate the outcomes of services. Data-based decision-making permeates every aspect of professional practice. 20 2.1 Addressed: Data-Based Decision Making and Accountability is one of the four core elements of the program. As such, content relevant to this domain is integrated into all aspects of the program. Program goals (see the School Psychology Program Handbook in Appendix A, pages 3-4) and required curriculum/field experiences address this domain (see the School Psychology Program Handbook in Appendix A, page 25-27, for the Program Plan Sheet and Appendix B for the program syllabi and rubrics). Data-Based Decision-Making and Accountability knowledge and skills are key elements in 13 courses (see also Course Alignment grid in Appendix E, pages 8-9). • • • • SPSY 753 SPSY 753b SPSY 778 SCOUN 690 • • EDUC 740 SPSY 743 • SPSY 768/768A • • • • • • SPSY 745 SPSY 775 SPSY 710 SPSY 701 SCOUN 705 SPSY 870 Psychometric Theory and Application School Psychology Lab Psychoeducational Disabilities Behavioral Interventions in the Schools (previously SCOUN 760 Theories and Techniques of Behavior Modification and now SPSY-690) Research Foundations Cognitive Assessment (formerly SPSY-743 Advanced Mental Testing) Diagnosis and Remediation of Learning Disabilities & LD Lab (now SPSY 768 LD: Assessment and Intervention) Assessment of Personality School Consultation Psychoeducational Assessment of Young Children Seminar in School Psychology Play Therapy Specialist Thesis In the above courses, the development of data-based decision-making knowledge and skills are addressed through readings, lectures, applied practice, group presentations, and research. Specific training in the administration, scoring, and interpretation of normreferenced and curriculum-based measurement techniques are included in the assessment courses where candidates practice their assessment skills in classrooms and laboratory settings. Further, content about formulating data-based recommendations and applying empirically-based academic and behavioral interventions is addressed through readings, lectures, and presentations in SPSY-768/768a and SCOUN-690. In addition, in EDUC740, SPSY-735, and SPSY-870, candidates learn the importance of using research data as the basis for all decision-making as a practitioner. Data-based decision-making and accountability skills and practice are further developed and applied in practica and internship settings (see Course Alignment grid in Appendix E, pages 8-9). 21 • SCOUN 798 • • • SPSY 781/782 SPSY 784/785 SPSY 792 Field Experience in School Counseling (now SCOUN-798 School Counseling Practicum) Field Practica in School Psychology Clinical Practica in School Psychology Internship in School Psychology In the above field experiences, candidates learn to develop their skills through such activities as generating recommendations and evaluating the effectiveness of academic/behavioral/counseling interventions in school-based and clinical settings. 2.1 Assessed: Tests, graded papers, and graded psychoeducational reports are components of the assessment and research classes, and candidate performance in this domain is further assessed through faculty evaluations of live and video-taped observations of candidates’ assessment skills (see rubrics in Appendix B). Further, candidates’ abilities to evaluate the effectiveness of academic and behavioral interventions is explicitly assessed through faculty evaluations (rubrics) of candidatedesigned and implemented single case design behavioral interventions developed in SCOUN-690 Behavioral Interventions in the Schools (now SPSY-690) and curriculumbased measurement profiles developed in SPSY-768/768A Diagnosis and Remediation of Learning Disabilities/LD Lab (now SPSY-768). Additionally, graded thesis papers (SPSY-735 Problems in School Psychology and SPSY-870 Specialist Thesis) provide indications of candidate knowledge and skills related to empirically-based practice. The assessment of Data-Based Decision-Making and Accountability is also implemented through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, pages 35-36). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (or Mentor) to assess candidate progress in Data-Based Decision-Making and Accountability through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) The assessment of candidates’ Data-Based Decision-Making and Accountability skills is further evaluated by the practicum and internship evaluation forms. Knowledge and skill in this area was assessed through two separate evaluation forms completed by field supervisors until spring of 2008. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ Assessment Skills, including their ability to “evaluate the effectiveness of behavioral intervention plans,” on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ 22 knowledge and skills in Data-Based Decision-Making along a 4-stage developmental continuum ranging from Observer to Mastery. (The Developmental Performance Appraisal form was developed in 2002 to monitor the progress of candidates’ knowledge and skills through the course of the program.) Finally, candidates’ data-based decision-making and accountability skills are assessed through the PRAXIS II School Psychology exam’s Diagnosis and Fact-Finding area scores. In response to the NASP Program Review of 2006, candidate’s Data-Based Decision Making and Accountability knowledge/skills will now be assessed through separate and newly revised evaluation tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D, page 21, page 67, and page 111, respectively) beginning this fall (2008). Further, candidate’s Data-Based DecisionMaking and Accountability will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.1 Attained: Data indicate program candidates attain solid competencies in Data-Based DecisionMaking and Accountability. Since 1996, all program graduates but one obtained satisfactory grades (B or better) in coursework or practica relevant to the domain. The one candidate who failed to do so was provided with feedback, and a remediation plan was developed and implemented. The candidate then satisfactorily completed a second attempt at SPSY-782 Field Practicum in School Psychology. Similarly, 2004-2008 summary portfolio evaluations indicate candidates make developmental progress in the area of Data-Based Decision-Making and Accountability from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 3.3 (Novice), whereas an average rating of a 5.3 (Practitioner) was found at Benchmark III (see Portfolio Review Data in Appendix E, page 112). Further, portfolio work samples (or artifacts) demonstrate our candidate’s skills in applying their Data-BasedMaking and Accountability skills to inform their practice and to generate research-based or scientifically-validated interventions and recommendations for particular clients (please refer to work samples in Appendix C, pages 1 to 164). Practicum and evaluations indicate our candidates perform within expectation in this area, as well. Over the past three years, practicum and internship means for Assessment ranged from 4.18 to 4.66 on the 5-point assessment items on the Evaluation Guide (please refer to Evaluation Guide data in Appendix, pages 19-22). Further, means ranged from 4.08 to 4.66 on the item assessing how well candidates “evaluate the effectiveness of behavioral intervention plans.” On the Developmental Performance Appraisal form, practicum and internship ratings indicated candidates’ knowledge and skills in Data-Based Decision-Making and 23 Accountability ranged from Novice to Practitioner when assessed at Practicum I (means of 2.58 and 2.38), to nearly Practitioner at Practicum II (means of 2.98 and 2.87), and to Practitioner when assessed as interns (means of 3.27 and 3.24). (Refer to Developmental Appraisal summary data in Appendix E, page 10.) Further, results from the PRAXIS II School Psychology Exam indicate most program candidates have acquired Diagnosis and Fact-Finding skills. Over the past three years, 22 out of 31 (71%) candidates scored in the average or above average performance range (i.e., scores of 21 or above in this area). Sources of attainment data for 2.1: • • • • • • • Candidate transcript reviews by Program Faculty Committee (see Appendix C, pages 206-210, for blinded transcripts) Assessment in the Major Report (Appendix E, pages 1-7) Portfolio Work Samples (Appendix C, pages 1 to 156) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Data (Appendix E, page 10) School Psychology Exam or PRAXIS II Data (Appendix E, page 113) 2.2 Consultation and Collaboration: School psychologists have knowledge of behavioral, mental health, collaborative, and/or other consultation models and methods and of their application to particular situations. School psychologists collaborate effectively with others in planning and decision-making processes at the individual, group, and system levels. 2.2 Addressed: Our program goal, Consultation and Collaboration, is one of the four core elements of the program. As such, it pervades all aspects of the program. Program goals (see pages 3-6 of the School Psychology Program Handbook in Appendix A) and required curriculum/field experience address this domain (see course syllabi in Appendix B). Candidates take 9 courses, five 5 practica, and a two-semester internship in which content relevant to consultation, communication, and collaboration skills have been integrated into the curriculum (see Course Alignment grid on pages 8-9 in Appendix E and Appendix B for course syllabi/rubrics). The following nine courses explicitly address consultation, communication, and collaboration knowledge/skills through lectures, applied practice exercises, and readings: • • • SPSY 788 SPSY 778 SPSY 690 • SPSY 745 Counseling Process Lab Psychoeducational Disabilities Behavioral Interventions in the Schools (previously SCOUN-690) Assessment of Personality 24 • • • • • SPSY 710 SPSY 775 SPSY 701 SCOUN 705 COUN 752 Psychoeducational Assessment of the Young Child School Consultation Seminar in School Psychology Play Therapy Group Dynamics These skills also are both addressed and assessed through applied experiences in practica and internship settings (SCOUN-798 (now SCOUN-788), SPSY-781/782, SPSY 784/785, EDUC-576, and SPSY-792) through such activities as collaborating with teachers, participating on school teams, interviewing parents, and consulting with professionals in the field. In SPSY-781/782 Practicum I and Practicum II, for example, candidates are required to conduct a behavioral case consultation (see Appendix B, page 176 and 178). Further, in response to the NASP Program Review of 2006, candidate’s Consultation and Collaboration knowledge/skills also will be addressed through an additional course emphasizing the development of candidates’ systems-level consultation, prevention, and intervention skills (see Appendix D, pages 132-138). This course will go through our university’s course approval process with the intent of adding it to the program requirements beginning in fall of 2009. Newly revised Practicum I and Practicum II Handbooks also require candidates to engage in consultation services (see Appendix D, pages 8, 35-36. 54, and 81-82). 2.2 Assessed Interpersonal communication skills have long been evaluated for program applicants through reviewing letters of reference and applicant interviews at Benchmark I (see Pupil Service Programs – Performance Based Assessment Requirements Checklist brochure in Appendix E, page 111). In addition to tests and other graded written work in the courses germane to the development of consultation/collaboration knowledge and skills, candidate performance is explicitly assessed through role-play demonstrations in SPSY-775 School Consultation, as well as through faculty evaluations of live and video-taped observations of candidates’ communication, collaboration, and/or consultation skills in COUN-788 Counseling and Process Lab, COUN-798 School Counseling Field Experience, SPSY781/782 Field Practicum, SPSY-784/785 Clinical Practicum, and SPSY-792 Internship (see course and field experience rubrics in Appendix B). In 2003, we developed a Disposition Review form (see Disposition Review form in Appendix A, page 37-38) to be used by faculty to evaluate candidate performance on non-academic skills at Benchmark II (after 30+ credits and prior to practica) and Benchmark III (after 60 credits and prior to internship). The Disposition Review form assesses candidates’ listening skills, respectful attitudes, and cooperation/collaborative skills. 25 The assessment of consultation and collaboration skills is also addressed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, pages 35-36). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (now Mentor) to assess candidate progress in the area of consultation collaboration through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) Further, the assessment of consultation and collaboration skills is a component of the practicum and internship evaluation forms. Knowledge and skill in this area was assessed through two separate evaluation forms completed by field supervisors. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ “Interpersonal Relationships, Communication, and Consultation Skills” on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another practicum and internship evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), assesses candidates’ knowledge and skills in Consultation and Collaboration along a 4-stage developmental continuum ranging from Observer to Mastery (now Mentor). (The Developmental Performance Appraisal form was developed in 2002 to monitor the developing knowledge and skills of candidates through the course of the program.) Further, candidate’s consultation skills were explicitly assessed during their school-based practica (see Appendix B, page 178, and Appendix D, pages 22, 35-36, 68, 81-82, and 112). Finally, program graduates’ consultation and collaboration skills are assessed through alumni 4-point Likert-style self-evaluation surveys and 5-point employer surveys at 1year and 5-years post graduation. In response to the NASP Program Review of 2006, candidates’ Consultation and Collaboration knowledge/skills (as well as providing systematic consultation skills that address students’ academic and behavioral difficulties) will be assessed through newly revised evaluation tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix E, pages 22, 35-36, 68, 81-82, and 112)) beginning this fall (2008). Further, candidates’ skills and knowledge also will be assessed through reevaluating their portfolio after their 2008-2009 internship year or at Benchmark IV (see Appendix E, page 101). 2.2 Attained: Program candidates attain strong consultation and collaboration skills. All program graduates since 1996 have obtained satisfactory grades (B or better) in the nine courses that address the development of consultation and collaboration skills (see blinded transcripts, pages 206-210, in Appendix C). 26 Disposition reviews during the previous year indicate all candidates earned 3 points or above (Satisfactory to Proficient) on the 5-point ratings in the areas of: a) Thoughtful & Responsive Listener (mean = 3.88), b) Cooperative & Collaborative (mean = 3.75), and c) Respectful (mean = 3.75) (see Appendix E, page11). Similarly, portfolio reviews from the past 4 cohorts (2004 to 2007) demonstrate candidates make developmental progress in the area of Consultation and Collaboration from Benchmark II (average rating = 2.8 or near Novice) to Benchmark III (average rating = 4.9 or near Practitioner) over the past two years. Further, portfolio work samples (or artifacts) demonstrate our candidate’s skills in applying their consultation skills in practice (please refer to consultation and behavioral intervention work samples in Appendix C, pages 1-97, pages 108-123, and pages 157164). Practicum and internship supervisors from the past three years gave candidates high ratings (means ranging from 4.15 to 4.74) on the 5-point consultation, communication, and collaboration items from the Evaluation Guide (see Appendix E, pages 19-22). Further, internship supervisor ratings on the Developmental Appraisal form from the past three years indicate candidate knowledge and skills were in the Novice stage when assessed in Practicum I (means of 2.40 and 2.23) to near Practitioner when assessed at Practicum II (means of 2.90 and 2.66) to Practitioner when assessed as interns (means of 3.25 and 3.14) (refer to Appendix E, page 10). Further, alumni and employer surveys indicate graduates are very skilled in this area of consultation and collaboration. Means from school psychology alumni survey in 2006 (the last available alumni survey dataset) indicate graduates agreed (3 = agree) to strongly agreed (4 = strongly agree) their graduate education at UW-Stout prepared them to: a) speak or present ideas effectively (means ranging from 3.3 to 4.0), listen effectively (means ranging from 3.0 to 3.6), and work in teams (means ranging from 3.5 to 4.0). Employer surveys also indicate they agreed (4 = agree) to strongly agreed (5 = strongly agree) the school psychology graduates were able to: a) speak or present ideas effectively (means ranging from 4.0 to 5.0), b) work in teams (means ranging from 4.0 to 5.0), and c) demonstrate good interpersonal skills (means ranging from 4.0 to 5.0). 2.2 Attainment data sources: • • • • • • • Candidate transcript reviews by Program Committee (Appendix C, pages 206210 for blinded transcripts) Assessment in the Major Report (Appendix E, pages 1-7) Portfolio Review Data (Appendix E, page 112) Portfolio Work Samples (Appendix C, pages 1-97, pages 108-123, and pages 157-164) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) Graduate and Employer Survey Data (Appendix E, pages 13-18) 27 2.3 Effective Instruction and Development of Cognitive/Academic Skills: School psychologists have knowledge of human learning processes, techniques to assess these processes, and direct and indirect services applicable to the development of cognitive and academic skills. School psychologists, in collaboration with others, develop appropriate cognitive and academic goals for candidates with different abilities, disabilities, strengths, and needs; implement interventions to achieve those goals; and evaluate the effectiveness of interventions and consultation. 2.3 Addressed: Effective Instruction and Development of Cognitive/Academic Skills is a program goal (see pages 6-7 in Appendix A) and required curriculum/field experiences address this domain (see course syllabi and rubrics in Appendix B). Knowledge and skills in Effective Instruction and Development of Cognitive/Academic Skills are addressed in 10 courses (see also Course Alignment Grid in Appendix E, pages 8-9): • • • • SPSY 753 SPSY 753b SPSY 778 SPSY 743 • SPSY 768/768A • • • • • SPSY 710 SPSY 775 SPSY 701 PSYC 850 PSYC 730 Psychometric Theory and Application School Psychology Lab Psychoeducational Disabilities Cognitive Assessment (formerly SPSY-743 Advanced Mental Testing) Diagnosis and Remediation of Learning Disabilities & LD Lab (now SPSY-768 LD: Assessment and Intervention) Psychoeducational Assessment of Young Children School Consultation Seminar in School Psychology Psychology of Development Advanced Psychology of Learning All candidates are required to take a graduate level course in learning/cognition (PSYC730 Advanced Psychology of Learning), as well as other courses addressing influences on learning (PSYC-850 Psychology of Development or PSYC-552 Adolescent Psychology and SPSY-778 Psychoeducational Disabilities). In these courses, knowledge about how children learn is addressed through readings, lectures, group presentations, and research. The development of appropriate academic goals and interventions for candidates is addressed throughout the assessment sequence and applied in practica and internship settings (SPSY-753 Psychometric Theory, SPSY-753a School Psychology Lab, SPSY743 Advance Mental Testing or Cognitive Assessment, SPSY-768/768A Diagnosis and Remediation of Learning Disabilities & LD Lab, SPSY-710 Assessment of the Young Child, SCOUN-798 or SPSY-788 Field Experience in School Counseling, SPSY781/782 Field Practica, SPSY-784/785 Clinical Practica, and SPSY-792 Internship in School Psychology). 28 Skills in implementing evidenced-based academic interventions and applying single case design principles to evaluate intervention outcomes are learned in SPSY-768 LD: Assessment and Intervention (previously SPSY-768/768A Diagnosis and Remediation of Learning Disabilities & LD Lab). In SPSY-768, candidates learn curriculum-based measurement (CBM) techniques and apply their intervention skills while progressmonitoring one child’s learning through the course of one semester through the use of weekly CBM probes (refer to Appendix B, pages 89-99, and Appendix D, pages 139149). Additionally, practicum candidates are now required to implement and monitor the effects of an academic intervention through CBM (see Appendix D, pages 8, 34, 54, and 80). Through field based experiences, candidates learn to develop their skills through such activities such as generating recommendations and evaluating the effectiveness of academic skills in their school-based and clinical settings (see Appendices A & B for Handbooks and Appendix B for syllabi and rubrics). 2.3 Assessed: In addition to tests, graded papers, and graded psycho-educational reports required in the learning and assessment classes (refer to rubrics in Appendix B), our candidate’s ability to evaluate the effectiveness of academic interventions is assessed through faculty evaluations of graphed curriculum-based measurement profiles (CBM) developed in SPSY-768 LD: Assessment and Intervention (previously SPSY-768/768A Diagnosis and Remediation of Learning Disabilities/LD Lab) (see pages 89-99 in Appendix B and pages 139-149 in Appendix D) and through a required case study in SPSY 781/2 (see pages 34 and 80 in Appendix D). The assessment of candidate knowledge and skills in the area of learning and academic interventions are also addressed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, pages 35-36.). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (now Mentor) to assess candidate progress in Effective Instruction and Development of Cognitive/Academic Skills through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) The assessment of candidates’ knowledge and skills in this area was further evaluated through two separate practicum and internship evaluation forms completed by field supervisors. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ ability to make meaningful recommendations and effectively administer/score intelligence tests, achievement tests, cognitive processing assessments, curriculum-based measures, and diagnostic reading tasks on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. 29 Another evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ knowledge and skills in Effective Instruction and Development of Cognitive/Academic Skills along a 4-stage developmental continuum ranging from Observer to Mastery. (The Developmental Performance Appraisal form was developed in 2002 to monitor the progress of candidates through the course of the program.) Finally, some indication of candidates’ knowledge related to Effective Instruction and Development of Cognitive/Academic Skills are assessed through the PRAXIS II School Psychology exam’s Applied Educational Foundations and Applied Psychological Foundations area scores. In response to the NASP Program Review of 2006, candidate’s Effective Instruction and Development of Cognitive/Academic Skills will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidate’s skills in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.3 Attained: Program graduates attain competencies in Effective Instruction and Development of Cognitive/Academic Skills. Since 1996, all program graduates have achieved satisfactory grades (B or better) in all coursework or practica relevant to the domain (refer to Appendix B for syllabi and rubrics). Portfolio evaluations over the past two years indicate candidates make developmental progress in the area of Effective Instruction and Development of Cognitive/Academic Skills from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 2.9 (near Novice), whereas an average of 5.2 (Practitioner) was found at Benchmark III (refer to Portfolio Review Data in Appendix E, page 112). Further, portfolio work samples (or artifacts) demonstrate our candidate’s skills in applying their academic intervention and evaluation skills in practice (please refer to work samples in Appendix C, pages 98-107 and pages 124-156). Evaluations of practicum and interning candidates indicate candidates are rated highly on items relevant to the area of Effective Instruction and Development of Cognitive/Academic Skills, as well (refer to Evaluation Guide Data in Appendix E, pages 19-22). Over the past two years, practicum and internship means for the relevant 5-point assessment items ranged from 3.81 to 4.81. Further, means ranged from 4.15 to 4.41 on the item assessing how well candidates “make meaningful recommendations.” On the Developmental Appraisal form, practicum and internship ratings (refer to Appendix E, page 10) indicate candidates’ knowledge and skills in Effective Instruction 30 and Development of Cognitive/Academic Skills ranged from Novice to Practitioner when assessed at Practicum I (means of 3.31 and 2.01) and to Practitioner when assessed as interns (means of 3.06 and 2.95). Over the past three years, results from the PRAXIS School Psychology Exam indicate most program candidates achieved average or above average area scores in Applied Psychological Foundations and Applied Educational Foundations (see PRAXIS II scores in Appendix E, page 113). Thirty out of 31 (97%) achieved average or above average area scores in Applied Psychological Foundations. Further, 28 out of 31 (90%) achieved average or above average area scores in Applied Educational Foundations. 2.3 Attainment data sources: • • • • • • Candidate transcript reviews by Program Faculty Committee (see blinded transcripts in Appendix C, page 206-210) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) Candidate Work Samples (Appendix C, pages 98-107 and pages 124-156) Praxis II Data Report (Appendix E, page 113) 2.4 Socialization and Development of Life Skills: School psychologists have knowledge of human developmental processes, techniques to assess these processes, and direct and indirect services applicable to the development of behavioral, affective, adaptive, and social skills. School psychologists, in collaboration with others, develop appropriate behavioral, affective, adaptive, and social goals for candidates of varying abilities, disabilities, strengths, and needs; implement interventions to achieve those goals; and evaluate the effectiveness of interventions. Such interventions include, but are not limited to, consultation, behavioral assessment/intervention, and counseling. 2.4 Addressed: Domain 4, Socialization and Development of Life Skills is one of the four core elements of UW-Stout’s School Psychology Program. As such program goals (see pages 7-9 of the School Psychology Program Handbook in Appendix A) and required curriculum/field experiences address this domain (see course syllabi and rubrics in Appendix B). Candidates take 10 courses, five practica, and a two-semester internship in which content relevant to Socialization and Development of Life Skills is infused into the curriculum (see Course Alignment Grid on pages 8-9 in Appendix E and course syllabi/rubrics in Appendix B). The following ten courses explicitly address knowledge and skills in this area through lectures, discussions, applied practice, and readings: 31 • • • • • • • • • • SPSY 750 SPSY 788 SPSY 753 SPSY 788 SPSY 760 SPSY 745 SPSY 775 SPSY 701 SCOUN 705 PSYC 850 Counseling Theory Counseling Process Lab Psychometric Theory and Application Psychoeducational Disabilities Behavioral Interventions in the Schools (previously SCOUN 690) Assessment of Personality School Consultation Seminar in School Psychology Play Therapy Psychology of Development Although content relevant to Socialization and Development of Life Skills is infused into all aspects of the program, required program coursework, laboratory experiences, and practicum experiences specifically target the development of counseling skills. In COUN-750 Counseling Theory and COUN-788 Counseling Process Lab, candidates learn counseling theories and develop foundational counseling skills through readings, lectures, research, and applied practice in a laboratory setting. In SCOUN-705 Play Therapy, candidates are introduced to, and receive supervised practice in, the models and methods of play therapy to address the behavioral, affective, adaptive, and social skill functioning of young children. Further, in SCOUN-705, candidates track the developmental, affective, and behavioral progress of two children through the course of one semester by observing their behavior and recording their progress in a play therapy lab. In another required counseling course, COUN-752 Group Dynamics, candidates are introduced to, and receive supervision in, models of group counseling techniques used to address the behavioral, affective, adaptive and social skill functioning needs of diverse populations. In addition, in SCOUN-798 (now SCOUN-788), candidates practice their counseling skills under the supervision of a practicing school counselor one day a week for one semester in a school setting. In PSYC-850 Psychology of Development (or PSYC-552 Adolescent Psychology), the theories and models of human developmental processes related to behavior, affective functioning, adaptive functioning, and social skill functioning are addressed. In SPSY778 Psychoeducational Disabilities, candidates are introduced to the various DSM-IV and educational disabilities and the direct/indirect services applicable to the development of behavioral, affective, adaptive, and social skills for candidates with disabilities. Techniques to assess children’s social, behavioral, and adaptive skills are addressed in SPSY-745 Assessment of Personality. SPSY-775 School Consultation addresses the various consultation models and methods to indirectly address the behavioral, affective, adaptive, and social skill deficits of children. Further, in SPSY-775, candidates learn how to communicate effectively, address resistant adults/systems, and learn strategies to affect organizational change. In addition, content about formulating data-based recommendations and implementing empirically-based behavioral interventions are addressed through practice in and out of the classroom setting in SPSY-775 and SCOUN-760 (now SPSY-690 Behavioral Interventions in the Schools). Further, skills in using single case designs to appropriately evaluate the effects of interventions are learned in SCOUN-690 Behavioral Interventions in the Schools. 32 Field experiences relevant to Domain 4 Socialization and Development of Life Skills are further developed and applied in practica and internship settings (SCOUN-798 or SCOUN-788 Field Experience in School Counseling, SPSY-781/782 Field Practica, SPSY-784/785 Clinical Practica, and SPSY-792 Internship in School Psychology). Candidates learn to develop their skills through such activities as providing counseling services to children, assessing students, generating recommendations, developing behavioral interventions, and evaluating the effectiveness of behavioral interventions in these school-based and clinical settings. 2.4 Assessed: In addition to tests, graded papers, and graded psychoeducational reports required in the developmental, disabilities, counseling and assessment classes, candidate performance is assessed through faculty evaluations of live and video-taped observations of candidates’ counseling and assessment skills (refer to rubrics in Appendix B). Further, candidates’ ability to evaluate the effectiveness of behavioral interventions is assessed through faculty evaluations of candidate-designed and implemented single case design behavioral interventions in SPSY-690 (previously SCOUN-690). Candidate knowledge and skills in Socialization and Development of Life Skills are also assessed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, page 35-36). Candidate portfolios are rated by faculty on a developmental continuum ranging from Observer to Mastery (now Mentor) to evaluate candidate progress in Socialization and Developmental Life Skills through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) The assessment of candidates’ Socialization and Development of Life Skills is further assessed by the practicum and internship evaluation forms. Knowledge and skill in Domain 4 was assessed through two separate evaluation forms completed by field supervisors. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ knowledge and skills in Socialization and Development of Life Skills, including their ability to “evaluate the effectiveness of behavioral intervention plans,” on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ knowledge and skills in Socialization and Development of Life Skills along a 4-stage developmental continuum ranging from Observer to Mastery (now Mentor). In response to the NASP Program Review of 2006, candidates’ knowledge and skills in Socialization and Development of Life Skills will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY- 33 782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in Socialization and Development of Life Skills will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.4 Attained: Program graduates attain competencies in Socialization and Development of Life Skills. Since 1996, all program graduates achieved satisfactory grades (B or better) in all coursework and field experiences relevant to the domain (see blinded transcripts in Appendix C, page 206). Portfolio evaluations indicate candidates make developmental progress in the area of Socialization and Development of Life Skills from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 3.5 (Novice), whereas an average rating of 5.4 (Practitioner) was evidenced at Benchmark III. Further, portfolio work samples (or artifacts) demonstrate our candidate’s skills in applying their functional behavioral assessment and behavioral intervention skills in practice (please refer to work samples in Appendix C, pages 1-97, pages 108-124, and pages 157-164). Practicum and internship supervisors rated candidates highly in this area, as well. Over the past two years, practicum and internship means for Direct Service ranged from 3.78 to 4.50 on the 5-point assessment items on the Evaluation Guide. Further, means ranged from 4.08 to 4.57 on the item assessing how well candidates “effectively evaluate the effectiveness of behavioral intervention plans.” Means ranged from 4.32 to 4.75 on items assessing how well candidates effectively administer and score behavioral/personality assessments, effectively conduct behavioral observations and integrate behavioral data with other diagnostic data, and make meaningful recommendations. These ratings provide evidence that candidates can effectively use assessments, provide direct service, and evaluate the effectiveness of behavioral intervention plans (refer to pages 19-22 in Appendix E). On the Developmental Appraisal form, practicum and internship ratings by field supervisors indicate candidates’ knowledge and skills in Socialization and Development of Life Skills were at the Novice level (means of 2.06 and 2.04) when assessed at Practicum I to a Practitioner level (means of 3.24 to 3.01) when assessed as interns (see page 10 in Appendix E). 2.4 Attainment data sources: • • • • Program Faculty Committee reviews of candidates’ transcripts (refer to blinded transcripts in Appendix C, pages 206-210) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) 34 • Candidate Portfolio Work Samples (Appendix C, pages 1-97, pages 108-124, and pages 157-164) 2.5 Student Diversity in Development and Learning: School psychologists have knowledge of individual differences, abilities, and disabilities and of the potential influence of biological, social, cultural, ethnic, experiential, socioeconomic, genderrelated, and linguistic factors in development and learning. School psychologists demonstrate the sensitivity and skills needed to work with individuals of diverse characteristics and to implement strategies selected and/or adapted based on individual characteristics, strengths, and needs. 2.5 Addressed: Domain 5, Student Diversity in Development and Learning is a program goal of UWStout’s School Psychology Program (see the School Psychology Program Handbook, Appendix A, pages 9-10). Required curriculum/field experiences address this domain (see course syllabi and rubrics in Appendix B, see pages 22-27 in Appendix A for the Program Handbook, and pages 8-9 in Appendix E for the Course Alignment Grid). Candidates take 16 courses, 5 practica, and a two-semester internship in which content relevant to diversity has been infused into the content (see Course Alignment grid on pages 8-9 in Appendix E and Appendix B for course syllabi/rubrics). The following 16 courses explicitly address content relevant to human diversity through lectures, applied practice exercises, readings, and discussions: • • • • • • • • COUN 750 COUN 788 SPSY 753 SPSY 753b SPSY 778 SPSY 777 SPSY 690 SPSY 743 • SPSY 768/768A • • • • • • SPSY 745 SPSY 710 SPSY 775 SPSY 701 SCOUN 705 PSYC 850 • EDUC 536 Counseling Theory Counseling Process Lab Psychometric Theory and Application School Psychology Lab Psychoeducational Disabilities Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions (formerly SCOUN 690) Cognitive Assessment (formerly SPSY-743 Advanced Mental Testing) Diagnosis and Remediation of Learning Disabilities & LD Lab (now SPSY-768 LD: Assessment and Intervention) Assessment of Personality Psychoeducational Assessment of Young Children School Consultation Seminar in School Psychology Play Therapy Psychology of Development (or PSYC Adolescent Psychology) Multiculturalism 35 As seen in the syllabi and rubrics (see Appendix B) content relevant to the understanding and appreciation of human diversity is integrated into the objectives and content of most required courses. However, EDUC-536 Multiculturalism: Issues and Perspectives and EDUC-576 Multicultural Field Experience specifically address candidates’ understanding and appreciation of diversity through required readings, lectures, discussions, presentations, and applied experience in a multicultural/diverse setting. The primary objectives of EDUC-536 Multiculturalism: Issues and Perspectives are for candidates to develop an awareness of personal attitudes, develop a conceptual framework for multiculturalism, analyze the history and any socio-cultural factors that impact culturally diverse groups and their educational needs. Other course objectives address the implications of gender, religious diversity, and poverty on society and education. Further, in EDUC-576, candidates apply their knowledge and skills relevant to an appreciation of diversity through working directly with a diverse population for a minimum of 50 contact hours. As seen in the syllabi for SPSY-778 Psychoeducational Disabilities (refer to page 33 in Appendix B) and PSYC-850 Psychology of Development (refer to pages 223-232 in Appendix B) address the biological bases of development and behavior. Candidates’ knowledge and skills in the area of Student Diversity in Development and Learning are further developed and applied in practica and internship settings (SCOUN798 or 788 Field Experience in School Counseling, SPSY-781/782 Field Practica, SPSY-784/785 Clinical Practica, and SPSY-792 Internship in School Psychology). Although UW-Stout is located in a small community with a largely homogeneous population (i.e., primarily White or Caucasian), the university is located within driving distance to a large, metropolitan community (Minneapolis/St. Paul, MN) and other rural settings with significant migrant populations. As such, program faculty members work to ensure that each candidate has at least one practicum placement in an economically disadvantaged and/or culturally diverse educational setting prior to internship (refer to newly revised Practicum I and Practicum II Handbooks in Appendix E and see field experience placements for one cohort in Appendix E, page 12). 2.5 Assessed: Tests, graded papers, and graded presentations are components of required coursework (see rubrics in Appendix B). Six program courses explicitly assess (through rubrics) candidate knowledge and/or skills in the area of diversity: • • SPSY 753 SPSY 777 • SPSY 768/768A • • • SPSY 745 SPSY 701 PSYC 850 Psychometric Theory and Application Legal/Ethical Issues: School Counselors and School Psychologists Diagnosis & Remediation of Learning Disabilities/LD Lab (now SPSY-768 LD: Assessment and Intervention) Assessment of Personality Seminar in School Psychology Psychology of Development 36 The assessment of candidate knowledge and skills in Student Diversity in Development and Learning are assessed through faculty evaluations of candidates’ portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating from in Appendix A). Candidate portfolios are rated by faculty on a developmental continuum ranging from Observer to Mastery (now referred to as Mentor) to evaluate candidate progress in Student Diversity in Development and Learning through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the 11 domains.) Student Diversity in Development and Learning is further assessed by the practicum and internship evaluation forms. Knowledge and skill in Domain 5 was assessed through two separate evaluation forms completed by field supervisors through spring of 2008. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ knowledge and skills in Candidate Diversity in Development and Learning, including their ability to relate “well with diverse individuals” and appropriately select, administer, and interpret “data from ethnically diverse clients” on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another practicum and internship evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), was developed in 2002 to assesses and monitor the progress of candidates’ knowledge and skills in Candidate Diversity in Development and Learning along a developmental continuum ranging from Observer to Mastery (now Mentor) through the course of the program. Finally, program graduates’ knowledge and skills in the area of diversity are assessed through alumni 4-point Likert-style self-evaluation surveys previously sent out to alumni at 1-year and 5-years post graduation. In response to the NASP Program Review of 2006, candidates’ competencies in Student Diversity in Development and Learning will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ Student Diversity in Development and Learning will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.5 Attained: Evidence indicates program candidates attain knowledge and an appreciation of diversity through the course of the program. All program graduates in recent years have obtained satisfactory grades (B or better) on the two key courses devoted to diversity and multiculturalism, as well as earning a B or better on all courses with content relevant to multiculturalism or diversity (refer to blinded transcripts in Appendix C, page 206-210). 37 Portfolio evaluations indicate candidates make developmental progress in the area of Student Diversity in Development and Learning through the course of the program. In the past four years, candidates’ ratings averaged a 3.2 (Novice) at Benchmark II, whereas an average rating of 5.0 (Practitioner) was evidenced at Benchmark III (refer to page 112 in Appendix E). Practicum and internship field supervisors rated candidates highly in this area, as well. Over the past two years, practicum and internship means ranged from 3.89 to 4.75 on the 5-point assessment items relevant to diversity in the Evaluation Guide (refer to pages 1922 in Appendix E). On the Developmental Performance Appraisal form, practicum and internship ratings by field supervisors indicate candidates’ knowledge and skills in Student Diversity in Development and Learning were at the Novice level (means of 2.25 and 2.01) when assessed at Practicum I to a beginning Practitioner when assessed as interning candidates (means of 3.19 and 3.08) (refer to page 10 in Appendix E). Further, alumni surveys indicate graduates believe they have skills in the area of diversity. Means (on a 4-point scale) from school psychology alumni indicate graduates agreed their graduate education at UW-Stout prepared them to “appreciate and understand diversity” (means ranging from 3 to 3.3) (refer to 13-18 in Appendix E). Sources of attainment data for 2.5: • • • • • Program Faculty Committee reviews of candidates’ transcripts (refer to blinded transcripts in Appendix C, pages 206-210) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) Alumni and Employer Survey Data (Appendix E, pages 13-18) 2.6 School and Systems Organization, Policy Development, and Climate: School psychologists have knowledge of general education, special education, and other educational and related services. They understand schools and other settings as systems. School psychologists work with individuals and groups to facilitate policies and practices that create and maintain safe, supportive, and effective learning environments for children and others. 2.6 Addressed: School and Systems Organization, Policy Development, and Climate is one of the eleven goals of the program (see pages 10-11 in Appendix A). As such, eight courses and five required field practica address this domain (see program syllabi in Appendix B and Course Alignment Grid, page 8, in Appendix E): 38 • • • SPSY 778 SPSY 777 SPSY 690 • SPSY 768/768A • • • • • • • SCOUN 788 SPSY 710 SPSY 775 SPSY 701 SPSY 781/2 SPSY 784/5 EDUC 536 Psychoeducational Disabilities Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions in the Schools (formerly SCOUN-690) Diagnosis and Remediation of Learning Disabilities & LD Lab (now SPSY-768 LD: Assessment and Intervention) School Counseling Practicum (formerly SCOUN-798) Psychoeducational Assessment of Young Children School Consultation Seminar in School Psychology Field Practica in School Psychology Clinical Practica in School Psychology Multiculturalism Knowledge of general education, special education, and other relevant services are addressed in SPSY-778 Psychoeducational Disabilities, SPSY-777 Legal/Ethical Issues: School Counselors and School Psychologists, SPSY-786/786A Diagnosis and Remediation of Learning Disabilities/LD Lab (now SPSY-786), SPSY-701 Seminar in School Psychology, EDUC-536 Multiculturalism: Issues and Perspectives, SPSY-775 School Consultation, and SPSY-710 Psychoeducational Assessment of Young Children. In these courses, candidate’s knowledge of relevant educational programs or services is addressed through readings, lectures, presentations, and discussions. Through a recent course revision, additional content on schools and systems has been infused into SCOUN-788 School Counseling Practicum (previously SCOUN-789 School Counseling Field Experience). School, systems, policies, and climate competencies are further developed in other practica and during the internship year (EDUC-576 CrossCultural Field Experience, SPSY-781/782 Field Practica: Psychoeducational Services I & II, SPSY-784/785 Clinical Practicum: Psychoeducational Services I & II, and SPSY792 Internship in School Psychology). Further, in response to the NASP Program Review of 2006, candidates’ knowledge and skills in working with systems will be addressed through an additional course (SPSY800 Systems and Crisis Intervention) that will emphasize the development of candidates systems-level consultation, prevention, and intervention skills (see Appendix D, pages 132-138 ). This course will go through our university’s course approval process with the intent of adding to the program requirements beginning in fall of 2009. 2.6 Assessed: In addition to tests and other graded written work in the coursework germane to the development of candidate knowledge and skills in schools/systems/policy/climate (refer to rubrics in Appendix B), the assessment of Domain 6 is addressed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric form in Appendix C, page 43). Candidate portfolios are rated by faculty on a 39 developmental continuum from Observer to Mentor to assess candidate progress in the area of School and Systems Organization, Policy Development, and Climate through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) Candidates’ knowledge and skills in the area of School and Systems Organization, Policy Development, and Climate was primarily assessed through their performance in practica and on internship through the use of two separate forms completed by field supervisors through the spring of 2008. One evaluation tool, the Evaluation Guide for School Psychology Practicum/Internship (see pages 54-67 and pages 90-96 in Appendix A), was a summative evaluation on a 5-point Likert-style scale ranging from Needs Improvement to Highly Satisfactory. The Evaluation Guide directly assessed candidate performance relative to candidate knowledge on several items. These items assessed whether the candidate/intern : a) understood the role and function of other staff specialists, b) understood district policies, rules, and regulations, c) understood the place of school psychology in the overall school program, d) understood and appreciated classroom contingencies and daily problems faced by teachers, e) understood the educational process and what transpires in the classroom, f) understood the school codes and state rules and regulations pertaining to special education and pupil personnel services and their interface with the school psychologist’s role, g) made meaningful recommendations, h) contributed to the development of good relationships in the schools and community through favorable transactions, i) was aware of issues and problems in the field of school psychology, and j) had knowledge of and skill in the organization and content of effective pupil services program. The other practicum/internship evaluation tool, the Developmental Performance Appraisal, was developed in 2002 to directly assess the developmental stage of candidates’ knowledge and skills in this domain along a developmental continuum (see pages 68-73 and pages 97-103 in Appendix A). In response to the NASP Program Review of 2006, candidates’ competencies in School and Systems Organization, Policy Development, and Climate will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.6 Attained: The data indicate candidates attain satisfactory skills in this area. Since 1996, all program graduates have obtained satisfactory grades (B or better) on the courses addressing this domain (refer to blinded transcripts in Appendix C, pages 206-210). 40 Portfolio reviews, over the past four years, demonstrate that candidates make developmental progress in the area of School and Systems Organization, Policy Development, and Climate from Benchmark II (average rating = 2.3 or Observer) to Benchmark III (average rating = 4.9 or near Practitioner) (see page 112 in Appendix E). In addition, practicum and internship supervisors gave candidates high ratings (means ranging from 4.08 to 4.81) on the 5-point items relevant to this domain on the Evaluation Guide. Practicum and internship supervisor ratings on the Developmental Performance Appraisal form indicate candidate knowledge and skills were at the Novice level when assessed at Practicum I (means of 2.28 and 2.09) to near Practitioner when assessed as interning candidates (means of 3.01 and 2.90). Sources of attainment data for 2.6: • • • • Program Faculty Committee reviews of transcripts (Appendix C, pages 206-210 for blinded transcripts) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) 2.7 Prevention, Crisis Intervention, and Mental Health: School psychologists have knowledge of human development and psychopathology and of associated biological, cultural, and social influences on human behavior. School psychologists provide or contribute to prevention and intervention programs that promote the mental health and physical well-being of candidates. 2.7 Addressed: Our program goal, Prevention, Crisis Intervention, and Mental Health, is one of the four core elements of the program because of our program’s counseling emphasis. As such, content relevant to mental health pervades all aspects of the program. Program goals (see pages 11-12 in Appendix A), eleven courses, five practica, and the internship experience address this domain (refer to Appendix B for syllabi/rubrics and pages 8-9 in Appendix E for Course Alignment Grid). These courses and applied practica address content in this domain: • • • • • COUN 750 COUN 788 SPSY 753 SPSY 778 SPSY 690 • SPSY 768/768A Counseling Theory Counseling Process Lab Psychometric Theory and Application Psychoeducational Disabilities Behavioral Interventions in the Schools (formerly SCOUN-690 or SCOUN-760) Diagnosis and Remediation of Learning Disabilities & LD Lab (now SPSY-768 LD: Assessment and Intervention) 41 • • • • • • • • SCOUN 788 SPSY 745 SPSY 710 SPSY 701 SPSY 781/2 SPSY 784/5 SCOUN 705 COUN 752 School Counseling Practicum (formerly SCOUN-798) Assessment of Personality Psychoeducational Assessment of Young Children Seminar in School Psychology Field Practica in School Psychology Clinical Practica in School Psychology Play Therapy Group Dynamics Foundational prevention, intervention, and mental health theories and skills germane to behavior and social/emotional issues are addressed in COUN-750 Counseling Theory, COUN-788 Counseling and Process Lab, SCOUN-705 Play Therapy, COUN-752 Group Dynamics, SPSY-690 Behavioral Interventions in the Schools (formerly SCOUN-690 or SCOUN-760). In these courses, candidates’ knowledge and skills in the areas of counseling and behavioral interventions are addressed through readings, discussions, and lectures, and the practice of counseling skills during role plays, videotaped sessions, and live sessions in classrooms and laboratory settings. These skills also are applied in practica and internship settings (SCOUN-788 (formerly SCOUN-798), SPSY-781/782, SPSY 784/785, and SPSY-792) through such activities as providing individual and group counseling services to candidates and designing/implementing behavioral interventions. Knowledge of human development, psychopathology, and the associated biological, cultural, and social influences on human behavior are addressed in PSYC-850 Psychology of Development and in SPSY-778 Psychoeducational Disabilities. In these courses, candidates’ knowledge and skills are addressed through readings, lectures, journaling, discussions, and presentations. More emphasis on prevention has been addressed through recent course revisions in SPSY-690 Behavioral Interventions in the Schools, SPSY-786/786A Diagnosis & Remediation in Learning Disabilities/LD Lab (now SPSY-768), and SPSY-710 Psychoeducational Assessment of the Young Child. In SPSY-690, for example, candidates design and implement a behavior change intervention for a school-age child or adolescent. In SPSY-786/786A, candidates screen young children for academic delays in a local school district through the use of DIBELS screeners and learn how to use curriculum-based measurement to monitor the learning progress of children who are at risk for developing intractable academic delays. In SPSY-710, candidates become acquainted with screening assessments that can be used with young children to identify those at-risk for disability conditions or developmental delays. Further, in response to the NASP Program Review of 2006, candidates’ prevention and crisis intervention knowledge/skills will be addressed through an additional course (SPSY-800 Systems and Crisis Intervention) emphasizing the development of candidates’ systems-level consultation, prevention, and intervention skills (see Appendix E, pages 132-138). This course will go through our university’s course approval process with the intent of adding to the program requirements beginning in fall of 2009. 42 2.7 Assessed: In addition to tests and other graded written work in the foundational courses pertinent to the development of knowledge and skills in the areas of counseling and behavioral interventions, candidate performance is assessed through live demonstrations of individual counseling skills in COUN-788, play therapy skills in SCOUN-705, and group counseling skills in COUN-752, as well as through supervisors’ evaluations of applied counseling skills in SCOUN-798 School Counseling Field Experience, SPSY781/782 Field Practicum, and SPSY-784/785 Clinical Practicum, and SPSY-792 Internship in School Psychology (refer to Appendix B for course syllabi and rubrics). Candidates’ knowledge and skills in areas relevant to human development, psychopathology, and the associated biological, cultural, and social influences on human behavior is assessed through tests, graded papers, graded presentations, and other written work in PSYC-850 Psychology of Development (or PSYC-552 Adolescent Psychology) and SPSY-778 Psychoeducational Disabilities. Knowledge and skills in the area are also assessed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, pages 35-36). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (now Mentor) to assess candidate progress in the area of Prevention, Crisis Intervention, and Mental Health through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the 11 domains.) The assessment of Prevention, Crisis Intervention, and Mental Health skills has been a component of the practicum and internship evaluation forms. Knowledge and skill in this area was assessed through two separate evaluation forms completed by field supervisors through spring of 2008. One evaluation tool, the Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A), assessed candidates’ “Direct Service” skills on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another practicum and internship evaluation form, the Developmental Performance Appraisal form (refer to Practicum and Internship Handbooks in Appendix A), assessed candidates’ knowledge and skills in Prevention, Crisis Intervention, and Mental Health along a 4-stage developmental continuum ranging from Observer to Mastery (now Mentor). Finally, candidates’ knowledge in the area of Prevention and Intervention are assessed through by their area scores on the PRAXIS II School Psychology Exam typically taken during the second year of the program. 43 In response to the NASP Program Review of 2006, candidates’ competencies in Prevention, Crisis Intervention, and Mental Health will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.7 Attained: Program candidates attain competence in the area of Prevention, Crisis Intervention, and Mental Health. Since 1996, all program graduates but one obtained satisfactory grades (B or better) in coursework or practica relevant to the domain (refer to blinded transcripts in Appendix C, pages 206-210). Portfolio evaluations indicate candidates make developmental progress in the area of Prevention, Crisis Intervention, and Mental Health from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 2.8 (near Novice), whereas an average rating of a 5.1 (Practitioner) was found at Benchmark III. Practicum and internship supervisors rated candidates highly in this area, as well. Over the past two years, practicum and internship means for Direct Service ranged from 3.74 to 4.50 on the 5-point assessment items on the Evaluation Guide (refer to Evaluation Guide Data on pages 19-22 in Appendix E). On the Developmental Appraisal form, ratings indicate candidates’ knowledge and skills in Prevention, Crisis Intervention, and Mental Health were at the Novice level (means of 2.21 and 1.97) when assessed at Practicum I to Practitioner (means of 3.17 and 3.00) when assessed as interning candidates (refer to Developmental Appraisal Data on page 10 in Appendix E). Further, results from the PRAXIS II School Psychology Exam indicate most program candidates have acquired satisfactory knowledge in the area of Prevention and Intervention skills. Over the past three years, 25 out of 31 (81%) candidates scored in the average to above average range in this area. Sources of attainment data for 2.7: • • • • • Transcript reviews by Program Faculty Committee (Appendix C, pages 206-210, for blinded transcripts) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) School Psychology Exam or PRAXIS II Data (Appendix E, page 113) 44 2.8 Home/School/Community Collaboration: School psychologists have knowledge of family systems, including family strengths and influences on candidate development, learning, and behavior, and of methods to involve families in education and service delivery. School psychologists work effectively with families, educators, and others in the community to promote and provide comprehensive services to children and families. 2.8 Addressed: Domain 8, Home/School/Community Collaboration, is a program goal of UW-Stout’s School Psychology Program (see pages 12-13 in Appendix A). Seven required courses, five practica, and the two semester internship address this domain (refer to Course Alignment Grid on pages 8-9 in Appendix E and course syllabi/rubrics in Appendix B): • • • SPSY 788 SPSY 777 SPSY 760 • SCOUN 788 • SPSY 745 • SPSY 775 • SPSY 701 • SPSY 781/2 • SPSY 784/5 • SCOUN 705 • PSYC 850 • SPSY 792 Psychoeducational Disabilities Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions in the Schools (previously SCOUN 690 or SCOUN 760) School Counseling Practicum (previously SCOUN 798) Assessment of Personality School Consultation Seminar in School Psychology Field Practica in School Psychology Clinical Practica in School Psychology Play Therapy Psychology of Development Internship in School Psychology Although content relevant to the understanding and appreciation of the influences of family on candidate performance is integrated into the objectives and content of several required courses (SPSY-745 Assessment of Personality, SPSY-690 Behavioral Interventions in the Schools, and SPSY-710 Assessment of the Young Child), SPSY-775 School Consultation and SPSY-784/785 Clinical Practicum specifically address the importance of collaboration and methods to work effectively with families, educators, and community members to promote and provide comprehensive services to children and families (refer to syllabi and rubrics in Appendix B). Candidates’ knowledge and skills in the area of Home/School/Community Collaboration are further developed and applied in practica and internship settings (SCOUN-788 School Counseling Practicum, SPSY-781/782 Field Practica, and SPSY-792 Internship in School Psychology) (refer to syllabi/rubrics in Appendix B and Practicum/Internship Handbooks in Appendix A). Further, in response to the NASP Program Review of 2006, candidates’ knowledge/skills in home, school, and community collaboration will be addressed through an additional course emphasizing the development of candidates’ systems-level consultation, 45 prevention, and intervention skills (see Appendix E, pages 132-138). This course will go through our university’s course approval process with the intent of adding to the program requirements beginning in fall of 2009. 2.8 Assessed: Tests, graded papers, and graded presentations are used in classes to assess candidates’ knowledge and skills in Domain 8 (refer to syllabi and rubrics in Appendix B). In addition, candidates’ knowledge and skills in Home/School/Community Collaboration are assessed through faculty evaluations of candidates’ portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating from in Appendix A, pages 35-36). Candidate portfolios are rated by faculty on a developmental continuum ranging from Observer to Master (now Mentor) to evaluate candidate progress in Home/School/Community Collaboration through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the 11 domains.) The assessment of candidates’ Home/School/Community Collaboration was further addressed by the practicum and internship evaluation forms. Knowledge and skill in Domain 8 were assessed through two separate evaluation forms completed by field supervisors through spring of 2008. The Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A) assessed candidates’ competencies in Home/School/Community Collaboration, including their ability to relate effectively with teachers, parents, and professionals in the community. Further, the Evaluation Guide assesses candidates’ contributions “to the development of good relationships in the schools and community through favorable transactions.” In addition, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A) was developed in 2002 to assesses and monitor the progress of candidates’ knowledge and skills in Home/School/Community Collaboration along a developmental continuum ranging from Observer to Mastery (now Mentor) through the course of the program. In response to the NASP Program Review of 2006, candidates’ competencies in Home/School/Community Collaboration will now be evaluated through separate and newly revised assessment tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.8 Attained: Evidence indicates graduates attain competencies in Home/School/Community Collaboration. Since 1996, all program graduates have achieved satisfactory grades (B or 46 better) in all coursework or practica relevant to the domain (refer to blinded transcripts on pages 206-210 in Appendix C). Portfolio evaluations over the past two years indicate candidates make developmental progress in the area of Home/School/Community Collaboration from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 2.8 (Observer), whereas an average of 4.9 (near Practitioner) was found at Benchmark III (refer to Portfolio Assessment Data on page 112, Appendix E). Evaluations of practicum and interns indicate our candidates rate well on items relevant to the area of Home/School/Community Collaboration. Over the past three years, practicum and internship means for the 5-point items relevant to collaborating with teachers, administrators, parents, and professionals in the field ranged from 4.17 to 4.86. Further, means ranged from 4.41 to 4.85 on the item assessing how well candidates contribute “to the development of good relationships in the schools and community through favorable transactions.” (Refer to Evaluation Guide data on pages 19-22 in Appendix E). On the Developmental Appraisal form, practicum and internship ratings indicate candidates’ knowledge and skills in Home/School/Community Collaboration ranged from Novice at Practicum I (means of 2.26 and 2.06) to Practitioner (means of 3.22 and 3.1) when assessed as interns (refer to page 10 in Appendix E). Sources of attainment data for 2.8: • • • • Transcript reviews by Program Faculty Committee (Appendix C, pages 206-210, for blinded transcripts) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) 2.9 Research and Program Evaluation: School psychologists have knowledge of research, statistics, and evaluation methods. School psychologists evaluate research, translate research into practice, and understand research design and statistics in sufficient depth to plan and conduct investigations and program evaluations for improvement of services. 2.9 Addressed: Domain 9, Research and Program Evaluation is a program goal (see pages 13-14 in Appendix A), and 14 courses, 5 practica, and a two semester internship address this domain (see course syllabi/rubrics in Appendix B and Course Alignment Grid on page 8 in Appendix E): • • SPSY 753 SPSY 778 Psychometric Theory and Application Psychoeducational Disabilities 47 • • • • • • • • • • • • • • • • SPSY 777 SPSY 690 Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions in the Schools (previously SCOUN 690 or SCOUN 760) SPSY 768 Learning Disabilities: Assessment & Intervention (previously SPSY 768/768a) SCOUN 788 School Counseling Practicum (previously SCOUN 798) EDUC 740 Research Foundations SPSY 735 Problems in School Psychology SPSY 745 Assessment of Personality SPSY 775 School Consultation SPSY 701 Seminar in School Psychology SPSY 781/2 Field Practica in School Psychology SPSY 784/5 Clinical Practica in School Psychology COUN 752 Group Dynamics PSYC 850 Psychology of Development EDUC 536 Multiculturalism SPSY 870 Specialist Thesis in School Psychology SPSY 792 Internship in School Psychology All candidates are required to take graduate level courses in research, statistics, and evaluation (i.e., EDUC-740 Research Foundations and SPSY-753 Psychometric Theory and Application). In these courses, content relevant to research, statistics, and evaluation is addressed. In addition, candidates complete both a master’s and education specialist’s thesis (SPSY-735 Problems in School Psychology and SPSY-870 Specialist Thesis in School Psychology) through the course of the program. For the master’s thesis (Plan B), candidates typically choose to do a comprehensive literature review that includes a critical analysis of the literature in a chosen topic, but the master’s thesis also can take the form of a regional empirical study, pilot study, or program evaluation. The specialist thesis is a formal, comprehensive empirical study intended to contribute knowledge to the field of school psychology. Further, given the scientific-practitioner orientation of the program, content relevant to the understanding of research, statistics, and evaluation is integrated into the objectives and assignments of most required courses. Candidates also are encouraged to take an elective course in program evaluation (EDUC-742 Program Evaluation) to enhance their knowledge in this area. 2.9 Assessed: Tests, graded papers, and presentations are assessed through rubrics in required coursework (see Appendix B for syllabi and rubrics). In the research and psychometric classes, candidate’s knowledge of research, statistics, and evaluation methods is assessed through faculty evaluations of their thesis projects. For the master’s thesis, the research project and paper is evaluated by the candidate’s research advisor and the Coordinator for Graduate Studies. The education specialist’s thesis is advised, reviewed, and 48 approved by a three person Graduate Faculty committee and the Coordinator for Graduate Studies (refer to pages 28-32 in Appendix A). Candidates’ knowledge and skills in the area of Research and Program Evaluation are also assessed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix A, pages 35-36). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Master (now Mentor) to assess candidate progress in Research and Program Evaluation through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) The assessment of candidates’ knowledge and skills in this area was further evaluated through two separate practicum and internship evaluation forms completed by field supervisors through spring of 2008. The Evaluation Guide (see School Psychology Practicum and Internship Handbooks in Appendix A) evaluated whether candidates read “professional books and journal articles” and have “knowledge of, and skill in, research” on a Likert-style 5-point format ranging from Needs Improvement to Highly Satisfactory. Another practicum and internship evaluation form, the Developmental Performance Appraisal form (see School Psychology Practicum and Internship Handbooks in Appendix A), was developed in 2002 to monitor the progress of candidates through the course of the program. The Developmental Performance Appraisal assessed candidates’ knowledge and skills in Research and Program Evaluation along a developmental continuum ranging from Observer to Mastery (now Mentor). Finally, program graduates’ research, statistics, and analytical skills are assessed through alumni 4-point Likert-style self-evaluation surveys and 5-point employer surveys at 1year and 5-years post graduation. In response to the NASP Program Review of 2006, candidates’ competencies in Research and Program Evaluation will now be evaluated through separate and newly revised evaluation tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.9 Attained: The data indicate graduates become competent in Research and Program Evaluation. Since 1996, all program graduates have achieved satisfactory grades (B or better) in all coursework or practica relevant to the domain (refer to Appendix C, pages 206-210, for blinded transcripts). 49 Portfolio evaluations over the past four cohorts indicate candidates make developmental progress in the area of Research and Program Evaluation from Benchmark II to Benchmark III. At Benchmark II, candidates’ ratings averaged a 3.7 (Novice to Practitioner), whereas an average of 5.3 (Practitioner) was found at Benchmark III. Evaluations of practicum and interns indicate our candidates receive high ratings on items relevant to the area of Research and Program Evaluation. Over the past three years, practicum and internship means ranged from 4.04 to 4.62 on items assessing whether candidates read “professional books and journal articles” and have “knowledge of, and skill in, research.” (Refer to pages 19-22 in Appendix E). On the Developmental Appraisal form, practicum and internship ratings indicate candidates’ knowledge and skills in Research and Program Evaluation ranged from Novice (means of 2.23 and 1.96) when assessed at Practicum I to Practitioner (means of 3.08 and 2.92) when assessed as interns (see Developmental Appraisal data on page 10 in Appendix E). Further, alumni and employer surveys indicate graduates are very skilled in the area of statistics and evaluation. Means from school psychology alumni indicate graduates agreed (3 = agree) to strongly agreed (4 = strongly agree) their graduate education at UW-Stout prepared them to understand statistics (means ranging from 3.0 to 3.3) and critically analyze information (means ranging from 3.5 to 3.7). Employer surveys also indicate they agreed (4 = agree) to strongly agreed (5 = strongly agree) the school psychology graduates were able to use mathematics or statistics (means ranging from 4.0 to 5.0) and critically analyze information (means ranging from 4.0 to 5.0). Sources of attainment data for 2.9: • • • • • Program Committee Transcript Reviews (Appendix C, pages 206-210 for blinded transcripts) Portfolio Review Data (Appendix E, page 112) Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) Graduate and Employer Survey Data (Appendix E, pages 13-18) 2.10 School Psychology Practice and Development: School psychologists have knowledge of the history and foundations of their profession; of various service models and methods; of public policy development applicable to services to children and families; and of ethical, professional, and legal standards. School psychologists practice in ways that are consistent with applicable standards, are involved in their profession, and have the knowledge and skills needed to acquire career-long professional development. 50 2.10 Addressed: School Psychology Practice and Development is one of the eleven goals of the program (see pages 14-15 in Appendix A). As such, 15 required courses, 5 practica, and a two semester internship address this domain (refer to Course Alignment Grid in Handbook E, pages 8-9, and syllabi/rubrics in Appendix B). The following courses and field experiences address this domain: • COUN 788 • SPSY 753 • SPSY 753b • SPSY 778 • SPSY 777 • SPSY 690 • • • • • • • • • • • • • Counseling Process Lab Psychometric Theory and Application School Psychology Lab Psychoeducational Disabilities Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions in the Schools (previously SCOUN 690 or SCOUN 760) SPSY 743 Cognitive Assessment SPSY 768 Learning Disabilities: Assessment & Intervention (previously SPSY 768/768a) SCOUN 788 School Counseling Practicum (previously SCOUN 798) EDUC 740 Research Foundations SPSY 735 Problems in School Psychology SPSY 775 School Consultation SPSY 701 Seminar in School Psychology SPSY 781/2 Field Practica in School Psychology SPSY 784/5 Clinical Practica in School Psychology COUN 752 Group Dynamics PSYC 850 Psychology of Development SPSY 870 Specialist Thesis in School Psychology SPSY 792 Internship in School Psychology Knowledge of school psychology as a service delivery model is addressed in SPSY-778 Psychoeducational Disabilities, SPSY-777 Legal/Ethical Issues: School Counselors and School Psychologists, SPSY-753b School Psychology Lab, SPSY-786 LD: Assessment and Intervention, and SPSY-701 Seminar in School Psychology. In these courses, content pertinent to the profession of school psychology, relevant policies, and legal/ethical/ professional standards are addressed through required readings, lectures, presentations, and discussions. Candidates acquire beginning knowledge and skills in the foundation of school psychology in SPSY-778 Psychoeducational Disabilities and SPSY-753b School Psychology Lab. Further knowledge and skills relevant to the practice of school psychology are covered in SPSY-786 Learning Disabilities: Assessment and Invention and in SPSY-701 Seminar in School Psychology. Although content relevant to the ethical, professional, and legal standards of the profession are infused into many required courses, SPSY-777 Legal/Ethical Issues: School Counselors and School Psychologists specifically addresses ethical requirements, 51 laws, and relevant professional standards. Further, knowledge and skills relevant to School Psychology Practice and Development are acquired in several field experiences, practica, and throughout the internship year (SCOUN-788 School Counseling Practicum, SPSY-781/782 Field Practica: Psychoeducational Services I & II, SPSY-784/785 Clinical Practicum: Psychoeducational Services I & II, and SPSY-792 Internship in School Psychology). 2.10 Assessed: Tests, graded presentations, and other graded written work in the coursework germane to the development of candidate knowledge and skills in School Psychology Practice and Development are assessed through rubrics in several courses (refer to Appendix B for syllabi and rubrics). Further, the assessment of this domain area is addressed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric form in Appendix A). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (now Mentor) to assess candidate progress in the area of School Psychology Practice and Development through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the eleven domains.) Candidates’ knowledge and skills in the area of School Psychology Practice and Development was primarily assessed through their performance in practica and internship through the use of two separate forms completed by field supervisors through spring of 2008. One evaluation tool, the Evaluation Guide for School Psychology Practicum/Internship (see pages Appendix A for the School Psychology Practicum and Internship Manuals), is a summative evaluation tool that used a 5-point Likert-style scale ranging from Needs Improvement to Highly Satisfactory. The Evaluation Guide directly assessed candidate performance relative to candidate knowledge on several items relevant to this domain. These items assess whether candidates/interns : a) understand the role and function of the school psychologist, b) understand district policies, rules, and regulations, c) understand the place of school psychology in the overall school program, d) understand the educational process and what transpires in the classroom, e) understand the school codes and state rules and regulations pertaining to special education and pupil personnel services and their interface with the school psychologist’s role, g) take advantage of opportunities to learn, h) attend training sessions, seminars, and staff conferences, i) read professional books and journal articles, j) are aware of issues and problems in the field of school psychology, j) have knowledge of and skill in professional ethics and ethical behavior, and k) have knowledge of and skill in the organization and content of effective pupil services programs. The other practicum/internship evaluation tool, the Developmental Performance Appraisal, was developed in 2002 to directly assess the developmental stage of candidates’ knowledge and skills in all eleven domains, including School Psychology 52 Practice and Development, along a 4-stage continuum through the course of the program (see pages 35-36 in Appendix A). Finally, candidates’ knowledge of ethics and the legal requirements of the profession are assessed through the PRAXIS II School Psychology exam’s Ethical/Legal Considerations area scores. In response to the NASP Program Review of 2006, candidates’ competencies in School Psychology Practice and Development will now be evaluated through separate and newly revised evaluation tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.10 Attained: All sources of evidence indicate program candidates attain satisfactory skills in this area. Since 1996, all program graduates have obtained satisfactory grades (B or better) on the courses addressing this domain (refer to blinded copies of transcripts on pages 206-210 in Appendix C). Portfolio reviews, over the past four years, reveal candidates make developmental progress in the area of School Psychology Practice and Development from Benchmark II (average rating = 2.5 or Observer) to Benchmark III (average rating = 5.2 or Practitioner) (refer to Portfolio Assessment data in Appendix E, page 112). In addition, practicum and internship supervisors gave candidates high ratings (means ranging from 4.02 to 4.81) on the items relevant to this domain on the 5-point Evaluation Guide. Practicum and internship supervisor ratings on the Developmental Performance Appraisal form indicate candidate knowledge and skills ranged from Novice (means of 2.46 and 2.36) when assessed at Practicum I to Practitioner (means of 3.25 and 3.16) when assessed as interns (see 2005-2008 Developmental Appraisal summary data in Appendix C). Further, results from the PRAXIS II School Psychology Exam indicate most program candidates have acquired satisfactory knowledge regarding the ethical and legal standards of the profession. Over the past three years, 25 out of 31 (81%) candidates have scored in the average to above average range in this area (refer to School Psychology Praxis II scores in Appendix E, page 113). Sources of attainment data for 2.10: • • Program Faculty Committee transcript reviews (Appendix C, pages 206-210, for blinded transcripts) Portfolio Review Data (Appendix E, page 112) 53 • • • Evaluation Guide Data (Appendix E, pages 19-22) Developmental Appraisal Form Data (Appendix E, page 10) School Psychology Exam or PRAXIS II Data (Appendix E, page 113) 2.11 Information Technology: School psychologists have knowledge of information sources and technology relevant to their work. School psychologists access, evaluate, and utilize information sources and technology in ways that safeguard or enhance the quality of services. 2.11 Addressed: Information Technology is a program goal (see page 15 in Appendix A). Ten required courses and four practica address this domain (refer to Course Alignment Grid, pages 89, in Appendix E, and syllabi/rubrics in Appendix D). The following courses and field experiences address content in information technology: • • • • • • • • • • • • SPSY 778 SPSY 777 SPSY 690 SPSY 743 SPSY 768 SPSY 745 SPSY 775 SPSY 701 SPSY 781/2 SPSY 784/5 SCOUN 705 PSYC 850 Psychoeducational Disabilities Legal/Ethical Issues: School Counselors and Psychologists Behavioral Interventions in the Schools Cognitive Assessment LD: Assessment and Intervention Assessment of Personality School Consultation Seminar in School Psychology School Psychology Field Practica School Psychology Clinical Practica Play Therapy Psychology of Development The use of information technology is an integral component of a variety of course offerings in our program. The use of technology is infused into every course in the program. For example, candidates are required to make PowerPoint presentations in most courses, and candidates use computer technology in the assessment sequence, during practica, and throughout their internship year to score assessment instruments and develop psychoeducational reports. Further, several courses and the internship are enhanced through Learn@UW-Stout, our web-based instructional platform. Through Learn@UW-Stout, candidates post documents, interact through a web-based group bulletin board, and access information through relevant links. UW-Stout is a technology rich university, and we are the first “laptop campus” in the University of Wisconsin system. As such, school psychology graduate candidates have access to a variety of technologies to enhance their learning. These technologies include: a) mediated classrooms with LCD projection technology, wireless internet, and digital capabilities; b) access to journals and publications through the library’s internet database; c) internet collaboration with faculty, peers, and professionals in the field through Learn@UW-Stout, d) access to SPSS (statistical software program) through our 54 school psychology lab, and e) word processing systems to write reports and format theses. 2.11 Assessed: Psychoeducational reports, assessment protocols, and PowerPoint presentations are graded through course rubrics (refer to syllabi and rubrics in Appendix B). The assessment of candidate knowledge and skills in the area of information technology is also addressed through faculty evaluations of candidate portfolios at Benchmark II and Benchmark III (see Portfolio Rubric rating form in Appendix C, page 41). Candidate portfolios are rated by faculty on a developmental continuum from Observer to Mastery (now Mentor) to assess candidate progress in Information Technology through the course of the program. (At Benchmark II, candidates submit three artifacts for 7 out of the 11 domains. At Benchmark III, candidates submit three portfolio artifacts for each of the 11 domains.) The assessment of candidates’ knowledge and skills in this area was further evaluated through practicum and internship evaluation forms completed by field supervisors. Two separate evaluation tools were used to assess candidates’ competencies during practicum and internship until spring of 2008: 1) The Evaluation Guide, and 2) The Developmental Performance Appraisal form (see School Psychology Practicum and Internship Manuals in Appendix A). Regrettably, only the Developmental Appraisal Form assessed candidate’s information technology skills. On the Developmental Appraisal form, the developing competencies of candidates in the area of information technology were assessed along a developmental continuum ranging from Observer to Mastery (now Mentor). (See Developmental Appraisal Form in Appendix A). Finally, program graduates’ technology skills are assessed through alumni 4-point Likert-style self-evaluation surveys and 5-point employer surveys at 1-year and 5-years post graduation. In response to the NASP Program Review of 2006, candidates’ competencies in Information Technology will now be evaluated through separate and newly revised evaluation tools in Practicum I (SPSY-781), Practicum II (SPSY-782), and Internship (SPSY-792) (see Appendix D) beginning this fall (2008). Further, candidates’ competencies in this area will be assessed through reevaluating their portfolio after the conclusion of their internship year, or at Benchmark IV, beginning in 2008-2009 (see Appendix D, pages 126-127). 2.11 Attained: Portfolio evaluations over the past four years indicate candidates make developmental progress in the area of Information Technology from Benchmark II to Benchmark III. At 55 Benchmark II, candidates’ ratings averaged a 4.4 (Novice), whereas an average of 5.7 (Practitioner) was found at Benchmark III. On the Developmental Appraisal form, practicum and internship ratings indicate candidates’ knowledge and skills in Information Technology was at the near Practitioner level (means of 3.03 and 2.99) when assessed at Practicum I to Practitioner (means of 3.37 and 3.39) when assessed as interns. Alumni and employer surveys indicate graduates are very skilled in the area of technology. Means from the 4-point ratings by the most recent school psychology alumni survey indicate graduates agreed their graduate education at UW-Stout prepared them to “utilize computing and digital technology” (means ranging from 2.7 to 3.0). The 5-point employer ratings indicated they agreed to strongly agreed (means ranging from 4.0 to 5.0) the school psychology graduates were able to utilize technology effectively. Sources of attainment data for 2.11: • • • • Program Faculty Committee transcript reviews (Appendix C, pages 206-210, for blinded transcripts) Portfolio Review Data (Appendix E, page 112) Developmental Appraisal Form Data (Appendix E, page 10) Alumni and Employer Survey Data (Appendix E, pages 13-18) 3.2 The internship is a collaboration between the training program and field site that assures the completion of activities consistent with the goals of the program. A written plan specifies the responsibilities of the training program and internship site in providing supervision support, and both formative and summative performance-based evaluation of intern performance. 3.2 Policy: The requirements for internship are specified in detail in the previous and the newly revised School Psychology Internship Handbooks (Appendices A and D) and the SPSY792 Internship in School Psychology syllabus (Appendix B, page 190). A School Psychology Internship Agreement Form (see pages 88-89 in Appendix A and pages 108109 in Appendix D ) signed by the district/agency representatives includes a list of responsibilities and expectations for all involved parties. In addition, interns document their varied activities and responsibilities through the use of the weekly and summary logs of school psychology activities (see pages 101 to 102 in Appendix A and pages 128 to 129 in Appendix E). These forms are completed to ensure that interns engage in duties that reflect the varied, functional role of the school psychologist. To ensure that both formative and summative performance-based evaluation occurs, two separate rating forms were previously developed to evaluate the performance of the interns. The Evaluation Guide for School Psychology Internship was completed by the 56 field-based supervisor at the end of each semester as a summative evaluation tool (see pages 90 to 96 of the previous School Psychology Internship Handbook in Appendix A). The Developmental Performance Appraisal form was completed at the end of each internship semester (see pages 97-100 in Appendix A for the previous School Psychology Internship Handbook) to formatively evaluate candidate performance on a developmental continuum ranging from Observer to Mastery (now Mentor). The Developmental Performance Appraisal form was utilized to track the developmental progression of candidates from the beginning of their first practicum experience until the end of their internship year. A newly revised School Psychology Internship Handbook with a revised evaluation tool has been developed (see Appendix D, pages 110-112 and pages 130-131) in response to a recommendation by the NASP’s Program Review in 2006. The new internship handbook and evaluation tool more closely aligns with the NASP domains and will be used during the 2008-2009 academic year. Further, although the new practicum and internship evaluation tools continue to be similarly organized, the new internship evaluation tool contains items that differentiate between the depth and breadth of knowledge and skills expected for interns versus our practicum candidates. University supervision of the interns is provided through regular online discussions, assignments, and postings through an online course site (Learn@UW-Stout). In addition, candidates attend an on-campus seminar once a semester (see requirements in SPSY-792 Internship in School Psychology syllabus in Appendix B and the newly revised Internship Handbook in Appendix E, pages 99-102). 3.2 Practice: Cumulative 2005-2007 data from the Evaluation Guide and the Developmental Performance Appraisal forms can be located in Appendix E, pages 10 and pages 19-22. Further, intern activity summary data sheets and logs from a recent graduating cohort are provided in Appendix C, pages 165-204. 3.5 The internship placement agency provides appropriate support for the internship experience including: (a) a written agreement specifying the period of appointment and any terms of compensation; (b) a schedule of appointments, expense reimbursement, a safe and secure work environment, adequate office space, and support services consistent with that afforded agency school psychologists; (c) provision for participation in continuing professional development activities; (d) release time for internship supervision; and (e) commitment to the internship as a diversified training experience. 57 3.5 Policy: The previous and new internship handbooks include a School Psychology Internship Agreement Form (see School Psychology Internship Handbook, Appendix A, pages 8889 and the new School Psychology Internship Handbook, Appendix D, pages 108-109). In addition to contractual matters, the Internship Handbook and the Internship Agreement Form specifies that the internship will “provide a balanced exposure to general and special education programs,” the internship will “meet the diverse training objectives of the university,” and the internship placement agency will include “a commitment to the internship as a diversified training experience.” In addition, the School Psychology Internship Handbook (Appendix A, page 83 and Appendix D, page 103) specifies that the field supervisor will “ensure the intern’s assigned responsibilities are appropriate in scope and number for the intern’s skill development.” The School Psychology Internship Agreement Form is signed by the intern, the school district administrator, the field supervisor, and the university supervisor prior to the beginning of the internship year. The process is overseen by the School of Education Clinical Placement Coordinator and follows the guidelines of university policies related to offcampus field experiences. In addition to the School Psychology Internship Agreement Form, the intern is required to complete weekly and summary internship logs to document the provision of a broad range of school psychological services during the internship year (see Appendix A, pages 101 to 102, and Appendix E, pages 128 to 129). Interns submit the summary internship logs to their university supervisor at the end of each semester. 3.5 Practice: Weekly and summary log forms may be found in Appendix A, pages 101-102 and in Appendix D, pages 128 to 129. Further, summary data and blinded activity summaries for recently graduating cohorts are available in Appendix C, pages 165-204. 4.3 The program employs a systematic valid process to ensure that all candidates, prior to the conclusion of the internship experience, are able to integrate domains of knowledge and apply professional skills in delivering a comprehensive range of services evidenced by measurable positive impact on children, youth, families, and other consumers. 4.3 Policy: Detailed explanations of the candidate assessment process are outlined in the School Psychology Program Handbook (Appendix A, pages 1-41) and in the School of Education’s Performance Based Assessment Requirement Checklist for Pupil Service Programs (see Appendix E, page 111). At a minimum, formal candidate evaluations occur at regular benchmark periods (annually) throughout the course of the program. Candidates are assessed through the 58 use of multiple methods. These include: a) disposition reviews, b) portfolio reviews, c) course-related assessments or rubrics, d) transcript reviews, e) formative practicum/intern evaluations, f) summative practicum/intern evaluations, and f) scores on the PRAXIS II School Psychology Examination. These methods are used to ensure all candidates make satisfactory progress toward the goal of developing the requisite skills and knowledge needed to provide a comprehensive range of services in the schools. To provide evidence that all candidates are able to integrate the domains of knowledge and skills, candidates must demonstrate the ability to write psychoeducational integrative reports with meaningful recommendations in the assessment coursework sequence (see SPSY-753b, SPSY-743, SPSY-768/768A (now SPSY-768), SPSY-710, and SPSY-745 syllabi and rubrics in Appendix B), in practica (see SPSY-781/782 School Psychology Practicum Handbooks in Appendix A and Appendix D and see SPSY-784/785 syllabus and rubrics in Appendix B), and during internship (see SPSY-792 School Psychology Internship Handbook in Appendices A and D). Further, the Internship Handbook specifies that interns must be given the opportunity to “integrate the competencies that address the domains of professional preparation” (see Appendix A, page 78, and Appendix D, preface, School Psychology Internship Handbook) outlined by the program, NASP, and Wisconsin’s Department of Public Instruction. To ensure that all candidates can provide services that result in a positive impact on children, candidates must demonstrate the effects of a behavioral intervention in SCOUN-690 Behavioral Interventions in the Schools (now SPSY-690; see syllabus in Appendix B, pages 63-79) and the effects of an academic intervention in SPSY786/786a, now SPSY-768, (see syllabi and rubrics in Appendix B, pages 89-99) through graphed data. In addition, interns are required to present an evidence-based project with results (see project description in Appendix D, pages 123-124) Further, to determine whether the services delivered by interns result in a measurable positive impact on candidates, youth, families, and other consumers, The Evaluation Guide for School Psychology Internship tool (see pages 90-96 of the School Psychology Internship Manual in Appendix A) asked the field supervisor to answer whether the intern a) “makes meaningful recommendations,” b) “contributes to the development of good relationships in the schools and community through favorable transactions,” c) “makes appropriate contributions to staffings and IEP team meetings,” d) “effectively provides counseling services for individual candidates,” e) “effectively provides counseling services for groups of candidates, and f) “effectively implements behavioral intervention plans.” The internship’s Log of School Psychology activities (see Appendix A, pages 101-102 of the School Psychology Internship Handbook and Appendix D, pages 128-129, in the newly revised Internship Handbook) provides evidence that interns are expected to deliver a comprehensive range of services consistent with the goals of the program and NASP’s standards. In addition, the School Psychology Internship Handbooks (page 78 in Appendix A and in the Preface section of the Internship Handbook of Appendix D ) 59 specifies that the “internship experience provides entry-level school psychologists with the opportunity to develop the specific skills needed for effective school psychological service delivery and to integrate the competencies that address the diverse domains of professional preparation and practice” outlined by the program, NASP, and Wisconsin’s Department of Public Instruction. In response to the NASP Program Review of 2006, interns will be required to demonstrate they have delivered services that result in a measurable, positive impact on students and others through providing case study examples during their internship year. Further, interns’ knowledge and skills in this area will be more comprehensively assessed through newly revised evaluation tools that measure the intern’s impact on children, youth, families, and other consumers (see Appendix D, pages 110-122) beginning this fall (2008). Further, candidates’ skills and knowledge also will be assessed through reevaluating their portfolio at Benchmark IV after their 2008-2009 internship year (see Appendix D, page 101 and pages 126-127). 4.3 Practice: Evaluation Guide data summaries for internship are available in Appendix E, pages 1923. Internship Developmental Appraisal data are available in Appendix E, page 10. Log summaries and blinded activity logs for recently graduating cohorts are found in Appendix C, pages 165 to 204. Selected work samples from interns are available in Appendix C, pages 157-164. 60