Central Washington University Assessment of Student Learning Department and Program Report Please enter the appropriate information concerning your student learning assessment activities for this year. Academic Year of Report: 2012-2013 Department: Psychology College: COTS Program: M.S. Applied Behavior Analysis 1. What student learning outcomes were assessed this year, and why? In answering this question, please identify the specific student learning outcomes you assessed this year, reasons for assessing these outcomes, with the outcomes written in clear, measurable terms, and note how the outcomes are linked to department, college and university mission and goals. The student learning outcomes for the Applied Behavior Analysis program as well as a summary of the related department, college, and university goals are shown in Appendix 1. Because the Applied Behavior Analysis program was approved as a degree program by Faculty Senate in October 2013, there are currently no previous assessment data to report. As such, this report will focus on providing an overview of the manner in which the seven SLOs for the Applied Behavior Analysis program will be assessed in future years. There are four main areas included in the Applied Behavior Analysis SLOs: 1) Competency in academic content areas required by the Behavior Analysis Certification Board (BACB). Students will demonstrate mastery of material included in the BACB’s current coursework Task List (4 th edition). The Task List includes content related to Basic Behavior Analytic Skills (SLO Goal #1), ClientCentered Responsibilities (SLO Goal #2), and Foundational Knowledge (SLO Goal #3). 2) Mastery of clinical skills. Students will demonstrate the ability to apply the knowledge from the BACB Task List in an applied setting (SLO Goal #4). 3) Professionalism. Students will demonstrate appropriate professional skills both as a student in classes on campus as well as during internships in applied settings in the community. (SLO Goal #5). 4) Competency in Research. Students will demonstrate competency in research through formulation and execution of a thesis project (SLO Goal #6) and attendance or participation at professional behavior analytic conferences (SLO Goal #7). 2. How were they assessed? In answering these questions, please concisely describe the specific methods used in assessing student learning. Please also specify the population assessed, when the assessment took place, and the standard of mastery (criterion) against which you will compare your assessment results. If appropriate, please list survey or questionnaire response rate from total population. A) What methods were used? B) Who was assessed? C) When was it assessed? Please refer to Appendix 1 which identifies the student learning outcomes for the Applied Behavior Analysis program for a summary of the methods of assessment to be employed, which students will be assessed, and when the assessments will take place. There are currently two 1st year students enrolled in the program. The 1 st year students were assessed in relation to professionalism (SLO Goal #5) at the end of Fall quarter 2013. Professionalism was assessed using a Likert-type scale evaluating several areas related to professional development (see Appendix 2). 3. What was learned? In answering this question, please report results in specific qualitative or quantitative terms, with the results linked to the outcomes you assessed, and compared to the standard of mastery (criterion) you noted above. Please also include a concise interpretation or analysis of the results. 2/15/16 1 Student Learning Outcomes 1. Students will demonstrate mastery in the core content areas of Applied Behavior Analysis that supports the development of competence across Basic Behavior Analytic Skills Sub-area as defined by the BACB’s BCBA Task List. Basic Behavior Analytic Skills •Measurement •Experimental Design •Behavior Change Considerations •Specific Behavior Change Procedures •Behavior Change Systems Criterion of Mastery Non Academic Criteria All students will earn a score of 80% or higher on the Basic Behavior-Analytic Skills subsection of the Posttest. Assessment Results No data regarding grades for the current 1st year students are yet available, and the post-test related to the BACB Task List items has not yet been administered. Passage of the BACB’s BCBA examination within 3 years following graduation. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. 2. Students will demonstrate mastery in the core content areas of core content areas of Applied Behavior Analysis that supports the development of competence across Client Centered Responsibilities Sub-area as defined by the BCBA Task List. Non Academic Criteria Client-Centered Responsibilities •Identification of the problem •Measurement •Assessment •Intervention •Implementation, management and Supervision Passage of the BACB’s BCBA examination within 3 years following graduation. 3. Students will demonstrate mastery in the core content areas of core content areas of Applied Behavior Analysis that supports the development of competence across Foundational Knowledge Sub-area as defined by the BCBA Task List. Foundational Knowledge • Philosophical Assumptions • Verbal Operants • Measurement Concepts • Mastery of specific aspects of Foundational Knowledge All students will earn a score of score of 80% or higher on the Client Centered Responsibilities subsection of the Post-Test. No data regarding grades for the current 1st year students are yet available, and the post-test related to the BACB Task List items has not yet been administered. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. Non Academic Criteria All students will earn a score of core of 80% or higher on the Foundational Knowledge subsection of the Post-Test. No data regarding grades for the current 1st year students are yet available, and the post-test related to the BACB Task List items has not yet been administered. Passage of the BACB’s BCBA examination within 3 years following graduation. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. 4. Students will demonstrate mastery in 2/15/16 Non-Academic Criteria 2 No data regarding internship area of application of BCBA Task List principles and skills in at least one applied setting. 100% of the students will receive a positive PSY 590 evaluation. • Collaboration with other professionals • Functional Behavioral Assessments • Task analyses • Develop and implement data sampling and data collection/maintenance systems • Design and implement behavioral interventions • Ensure human protections • Assess Procedural Integrity • Develop professional communication skills • Staff/parental training • Identify ethical principles in practice 5. Students will demonstrate professional work characteristics and dispositions appropriate to the field of Applied Behavior Analysis. Successful completion of a minimum of 30% of new items on the ABA program Internship Performance Review. Non-Academic Criteria 100% of students in the ABA program will meet the individual criteria listed below by the end of the first year. Total scores above 85% on the total checklist for each student. A minimum of 3 on the individual items on the checklist. performance are available for the current 1st year students as internship is completed during the second year of study. First year students were evaluated at the end of Fall quarter 2013 using the “ABA Professional Behavior and Disposition Survey”. Both current 1st year students were rated as satisfactory or better, which is indicated by a rating of 3 or higher on the survey. The mean students rating for Fall 2013 was 3.23. 100% of students in the ABA program will meet the individual criteria listed below by the end of the second year. Scores above 90% on the total checklist for each student A minimum of 4 on the individual items on the checklist. 6. Students will demonstrate competence in basic or applied research including: •The ability to write using the appropriate language and format of the discipline •To formulate a viable research question • To summarize the literature relevant to a specific research question. •To design and implement an experimental research design •To gather and analyze data using appropriate analytical methods 2/15/16 Academic Criteria 80% of students leaving the program will have completed the following: Successful completion and approval of the thesis proposal as signified by the signing of the option approval form by the thesis committee. 3 No data regarding competence in research are available for the current 1st year students as thesis is completed during the second year of study. •To summarize and integrate research findings into the existing research literature. Completion of the HSRC training required for human research. HSRC approval to conduct data collection. Successful defense of the thesis as signified by the signing of the oral defense of thesis forms and thesis signature pages by the thesis chair and committee Successful acceptance of the written thesis by the graduate school. 7. Students will engage in professional exchange with psychology and behavior analytic professionals 100% of all ABA students will attend at least one national, regional, or local professional conference by the end of their second year. No data regarding professional activities are available for the current 1st year students. 25% of all ABA students will present their thesis research at least one national, regional, or local professional conference within one year of graduation. Based on these assessment results, we can draw the following conclusions: Both of the 1st year master’s students are performing satisfactorily in regards to professional disposition based on faculty evaluation. As a new master’s degree program, there are several academic and non-academic assessment methods that will be carried out over the course of future years to provide data on the success of the Applied Behavior Analysis program. 4. What will the department or program do as a result of that information? In answering this question, please note specific changes to your program as they affect student learning, and as they are related to results from the assessment process. If no changes are planned, please describe why no changes are needed. In addition, how will the department report the results and changes to internal and external constituents (e.g., advisory groups, newsletters, forums, etc.). Current assessment results do not indicate a need for change within the program. Results from assessments conducted in future years will be used to guide potential program changes. 5. What did the department or program do in response to last year’s assessment information? In answering this question, please describe any changes that have been made to improve student learning based on previous assessment results. Please also discuss any changes you have made to your Assessment Plan or assessment methods. The Applied Behavior Analysis specialization of the Experimental Psychology program was separated into a standalone degree program, and the SLOs and assessment plan for the new Applied Behavior Analysis degree program were approved in October 2013. Therefore, there is no opportunity for response to a previous year’s assessment information. 2/15/16 4 6. Questions or suggestions concerning Assessment of Student Learning at Central Washington University: 2/15/16 N/A 5 Appendix 1: Psychology Student Learning Outcomes (2012-2013) Program: M.S. Program in Applied Behavior Analysis Student Learning Outcomes Program Goals Related Department Goals Goal 1: Students will demonstrate mastery in the core content areas of Applied Behavior Analysis that supports the development of competence across Basic Behavior Analytic Skills Subarea as defined by the BACB’s BCBA Task List. Basic Behavior Analytic Skills •Measurement •Experimental Design •Behavior Change Considerations •Specific Behavior Change Procedures •Behavior Change Systems (See attached BCAB Task List (4th Ed)) Goal 1: Provide a foundation in the core content areas of Applied Behavior Analysis that supports the development of competence across Basic Behavior Analytic Skills Sub-area as defined by the BACB’s BCBA Task List. Goal 1. Assess and improve our curriculum and instructional methods. 2/15/16 Related College Goals Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 9. Enrich student life at CWU and the region. 6 Related University Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. Method(s) of Assessment Non-Academic Assessment Pretest in the first quarter of first year. Post-test at the end of the first year Passage of the BACB’s BCBA examination after graduation. Academic Assessments Overall GPA Class grades Who/What Assessed For first year students, a PreTest and a PostTest will be given to assess mastery of the Basic BehaviorAnalytic Skills required by the BACB. When Assessed Initial Pre-Test will be given before November 1st during the student’s first year in the program. The Post-Test will be given between May 15th and the end of the Spring quarter during the student’s first year in the program. Criteria of Achievement Non Academic Criteria All students will earn a score of 80% or higher on the Basic BehaviorAnalytic Skills subsection of the Posttest. Passage of the BACB’s BCBA examination within 3 years following graduation. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. Student Learning Outcomes Program Goals Related Department Goals Goal 2: Students will demonstrate mastery in the core content areas of core content areas of Applied Behavior Analysis that supports the development of competence across Client Centered Responsibilities Subarea as defined by the BCBA Task List. Client-Centered Responsibilities •Identification of the problem •Measurement •Assessment •Intervention •Implementation, management and Supervision (See attached BCAB Task List (4th Ed)) Goal 1: Provide a foundation in the core content areas of Applied Behavior Analysis that supports the development of competence across Basic Behavior Analytic Skills Sub-area as defined by the BACB’s BCBA Task List. Goal 1. Assess and improve our curriculum and instructional methods. 2/15/16 Related College Goals Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 9. Enrich student life at CWU and the region. 7 Related University Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. Method(s) of Assessment Non-Academic Assessment Pretest in the first quarter of first year. First post-test in the first quarter of second year. Passage of the BACB’s BCBA examination after graduation. Academic Assessments Overall GPA Class grades Who/What Assessed For first year students, a PreTest and a PostTest will be given to assess mastery of the Client Centered Responsibilities required by the BACB. When Assessed Initial Pre-Test will be given before November 1st during the students’ first year in the program. The Post-test will be giving between May 15 and the end of the spring quarter during the students’ first year in the program. Criteria of Achievement Non Academic Criteria All students will earn a score of score of 80% or higher on the Client Centered Responsibilities subsection of the PostTest. Passage of the BACB’s BCBA examination within 3 years following graduation. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. Student Learning Outcomes Program Goals Related Department Goals Goal 3: Students will demonstrate mastery in the core content areas of core content areas of Applied Behavior Analysis that supports the development of competence across Foundational Knowledge Sub-area as defined by the BCBA Task List. Foundational Knowledge • Philosophical Assumptions • Verbal Operants • Measurement Concepts • Mastery of specific aspects of Foundational Knowledge (See attached BCAB Task List (4th Ed)) Goal 1: Provide a foundation in the core content areas of Applied Behavior Analysis that supports the development of competence across Basic Behavior Analytic Skills Sub-area as defined by the BACB’s BCBA Task List. Goal 1. Assess and improve our curriculum and instructional methods. 2/15/16 Related College Goals Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 9. Enrich student life at CWU and the region. 8 Related University Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. Method(s) of Assessment Non-Academic Assessment Pretest in the first quarter of first year. First post-test in the first quarter of second year. Passage of the BACB’s BCBA examination after graduation. Academic Assessments Overall GPA Class grades Who/What Assessed When Assessed For first year students, a PreTest and a PostTest will be given to first year students to assess mastery of the Foundational Knowledge Subarea as defined by the BCBA Task List. Initial Pre-Test will be given before November 1st during the students’ first year in the program. The Post-test will be giving between May 15 and the end of the spring quarter during the students’ first year in the program. Criteria of Achievement Non Academic Criteria All students will earn a score of core of 80% or higher on the Foundational Knowledge subsection of the PostTest. Passage of the BACB’s BCBA examination within 3 years following graduation. Academic Criteria Cumulative GPA above 3.0 and no grade lower than a B in all required and elective courses. Student Learning Outcomes Program Goals Related Department Goals Goal 4. Students will demonstrate mastery in area of application of BCBA Task List principles and skills in at least one applied setting. • Collaboration with other professionals • Functional Behavioral Assessments • Task analyses • Develop and implement data sampling and data collection/maintenance systems • Design and implement behavioral interventions • Ensure human protections • Assess Procedural Integrity • Develop professional communication skills • Staff/parental training • Identify ethical principles in practice Goal 4. To provide students with relevant and meaningful opportunities to successfully practice and apply the BCBA® Task List principles and skills in applied settings during their internship. Goal 1. Assess and improve our curriculum and instructional methods. 2/15/16 Related College Goals Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 5. Support involvement of undergraduate and graduate students in psychological research Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal V: Build partnerships with private, professional, academic, government and community-based organizations. Goal 8. Serve as a center for psychological and educational services to the community Goal 9. Enrich student life at CWU and the region. 9 Related University Goals Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. Method(s) of Assessment Non-Academic Assessments Cooperative Education (PSY 590) Evaluation Academic Assessments ABA program Internship Performance Review Who/What Assessed Internship site supervisors will be asked to complete the Cooperative Education (PSY 590) Postinternship assessment and evaluation. CWU faculty supervisor will complete the ABA program Internship Performance Review outlining the actual experience with items in the BCBA Task List. When Assessed At the end of each academic term of enrollment in PSY 590 Internship. Criteria of Achievement Non-Academic Criteria 100% of the students will receive a positive PSY 590 evaluation. Successful completion of a minimum of 30% of new items on the ABA program Internship Performance Review. Student Learning Outcomes Program Goals Goal 5. Students will demonstrate professional work characteristics and dispositions appropriate to the field of Applied Behavior Analysis. See attached Professional Disposition Checklist Goal 1. Assess and improve our curriculum and instructional methods. Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 9. Enrich student life at CWU and the region. Related Department Goals Related College Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. Related University Goals Goal 1. Assess and improve our curriculum and instructional methods. Goal 2. Promote excellence in learning to prepare students for careers and advanced study. Goal 9. Enrich student life at CWU and the region. Method(s) of Assessment Non-Academic Assessments Professional Disposition Checklist Who/What Assessed When Assessed Instructional faculty will complete a 15item Likert-style checklist for each student in the program. At the end of each academic term during the first and second years • Initiative • Dependability • Time management and organization •Problem solving/Critical thinking • Respect for Diversity Goal V: Build partnerships with private, professional, academic, government and communitybased organizations. • Communication Skills • Interpersonal Relations • Teamwork •Adaptability and Flexibility • Response to Feedback • Self awareness • Professional behavior and appearance • Independence • Data-based conceptualization 2/15/16 10 Criteria of Achievement Non-Academic Criteria 100% of students in the ABA program will meet the individual criteria listed below by the end of the first year. Total scores above 85% on the total checklist for each student. A minimum of 3 on the individual items on the checklist. ********** 100% of students in the ABA program will meet the individual criteria listed below by the end of the second year. Scores above 90% on the total checklist for each student A minimum of 4 on the individual items on the checklist. Student Learning Outcomes Program Goals Related Department Goals Goal 6. Students will demonstrate competence in basic or applied research including: •The ability to write using the appropriate language and format of the discipline •To formulate a viable research question • To summarize the literature relevant to a specific research question. •To design and implement an experimental research design •To gather and analyze data using appropriate analytical methods •To summarize and integrate research findings into the existing research literature. Goal 3. To form research-based partnerships between graduate students and faculty members who share interests and offer expertise in a chosen area of study in an effort to encourage the development of the necessary skills to execute valid independent research in applied behavior analysis. Goal 1. Assess and improve our curriculum and instructional methods. . Goal 2. Promote excellence in learning to prepare students for careers and advanced study Goal 5. Support involvement of undergraduate and graduate students in psychological research Goal 6. Work with the College of the Sciences to provide support for faculty scholarship and professional development activities Related College Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry. . Goal IV: Develop a diversified funding base to support teaching, research, and service. Goal V: Build partnerships with private, professional, academic, government, and community-based organizations. Goal 8. Serve as a center for psychological and educational services to the community Goal 9. Enrich student life at CWU and the region. Goal 11. Promote collegial relations 2/15/16 Related University Goals 11 Method(s) of Assessment Academic Assessments Thesis proposal Approved HSRC application Thesis defense Thesis Approval by OGSR Who/What Assessed When Assessed Students will be assessed by the program director in collaboration with the graduate school and the HSRC Office in terms of student progress toward thesis completion at the end of each academic year. At the end of each academic year. Criteria of Achievement Academic Criteria 80% of students leaving the program will have completed the following: Successful completion and approval of the thesis proposal as signified by the signing of the option approval form by the thesis committee. Completion of the HSRC training required for human research. HSRC approval to conduct data collection. Successful defense of the thesis as signified by the signing of the oral defense of thesis forms and thesis signature pages by the thesis chair and committee Successful acceptance of the written thesis by the graduate school. Student Learning Outcomes Goal 7. Students will engage in professional exchange with psychology and behavior analytic professionals Program Goals Goal 5. To encourage our students and faculty to engage in professional exchange with external behavior analytic professionals. Related Department Goals Goal 1. Assess and improve our curriculum and instructional methods. Goal 2. Promote excellence in learning to prepare students for careers and advanced study Goal 4. Pursue diversity goals in attracting women and minority students and faculty members Goal 5. Support involvement of undergraduate and graduate students in psychological research. Related College Goals Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goals I & II: Maintain and strengthen an outstanding academic and student life at all sites. Goal III: Provide for outstanding graduate programs that meet focused regional needs and achieve academic excellence. Goal V: Achieve regional and national prominence for the university. Goal V: Build partnerships with private, professional, academic, government, and community-based organizations. Goal VI: Build inclusive and diverse campus communities that promote intellectual inquiry Goal 8. Serve as a center for psychological and educational services to the community Goal 9. Enrich student life at CWU and the region. Goal 11. Promote collegial relations. 2/15/16 Related University Goals 12 Method(s) of Assessment Non-Academic Assessments Attendance at least one national, regional, or local professional conference. Who/What Assessed When Assessed Criteria of Achievement Faculty will be surveyed to determine which ABA students have attended a conference each year. At the end of each academic year. 100% of all ABA students will attend at least one national, regional, or local professional conference by the end of their second year. Faculty will be surveyed to determine which ABA students have presented their research at a conference each year. 25% of all ABA students will present their thesis research at least one national, regional, or local professional conference within one year of graduation. Appendix 2: ABA Professional Behavior and Disposition Survey Student: Quarter & Year: Instructor: Course: Please rate the student on each item using the scale below. Data from all faculty respondents will be averaged, and generalized feedback will be provided to the student. When completed, please return this form to Dr. Sadie Lovett (MS 7575). N/A 1 2 3 4 5 Rating Scale No opportunity to observe/ Don’t know Needs substantial improvement/unsatisfactory Needs some improvement/ Somewhat lower than expected Satisfactory/Expected Very good/ Better than expected Superior/Greatly exceeds expectations Characteristic Oral Communication: Expresses self clearly & effectively Written Communication: Writes in an organized/clear fashion; effectively uses scientific writing (e.g. APA). Problem Solving/Critical Thinking: Effectively analyzes situations/problems and conceptualizes solutions. Data-oriented: Uses data to conceptualize problems, generate solutions, and evaluate outcomes. Dependability/Time Management: Reliably follows through on tasks; completes assignments in a timely manner. Independence/Adaptability: Functions with minimal supervision & initiates activities when appropriate; adapts effectively to the demands of a situation Interpersonal Relations/Teamwork: Relates effectively to colleagues/faculty; works and collaborates well with others Respect for Diversity: Respects all human differences; demonstrates the sensitivity to work with diverse groups. Self-awareness: Shows realistic awareness of strengths & weaknesses and how this impacts professional functioning. Responsiveness to Feedback: Is open to feedback and responds appropriately and in a timely manner Professional Ethics: Follows ethical guidelines; is aware of ethical issues as they relate to research/professional settings. Professional Appearance: Presents self professionally in terms of language, demeanor, dress, and disposition. 1 1 Rating (circle one) 2 3 4 5 2 3 4 5 NA NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA 1 2 3 4 5 NA Please list any concerns about this student’s performance or general comments below: 2/15/16 13