Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Version 1.0 Adopted by The Diagnostic Medical Sonography Faculty: September 6, 2011 Submitted to The Director of the College: November 15, 2011 The Office of Academic Affairs: <date> Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 1 of 20 TABLE OF CONTENTS Introduction ______________________________________________________________________ 3 Program Student Learning Outcomes __________________________________________________ 4 Assessment Tools __________________________________________________________________ 5 Assessment Implementation & Analysis for Program Improvement __________________________ 6 Appendix A: Clinical Competency Evaluations __________________________________________ 7 Appendix B: Clinical Experience Documentation _______________________________________ 10 Appendix C: Capstone Evaluation ___________________________________________________ 12 Appendix D: Graduate Survey ______________________________________________________ 14 Appendix E: National Certification Results ____________________________________________ 17 Appendix F: Employer Survey ______________________________________________________ 18 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 2 of 20 Introduction The Diagnostic Medical Sonography program goal is to prepare competent entry-level general sonographers in the cognitive, psychomotor, and affective learning domains. The students completing the program will receive an Associate of Applied Science degree, and will be eligible to apply for certification with the American Registry of Diagnostic Medical Sonographers (ARDMS) and the American Registry of Radiologic Technologists (ARRT). The program was developed following the National Education Curriculum for Sonography and information from several diagnostic medical sonography accrediting and certifying organizations. The professional organizations referenced were: the Commission on Accreditation of Allied Health Programs (CAAHEP), the Joint Review Committee on Education in Diagnostic Medical Sonography (JRC-DMS), the Society of Diagnostic Medical Sonographers (SDMS), the American Registry of Diagnostic Medical Sonographers (ARDMS), and the American Registry of Radiologic Technologists (ARRT). These organizations’ standards and guidelines address program planning, implementation, and evaluation; and the standards address curriculum requisites, learning competencies, and clinical competency requirements. This document defines the educational objectives and expected outcomes for the Diagnostic Medical Sonography program, and outlines a plan for assessing the achievement of the stated objectives and outcomes. The development of the objectives and outcomes consisted of the faculty, an advisory group, a review of educational accreditation standards, and a review of national certification requirements. The faculty met and accepted the objectives, outcomes, and assessment tools on September 6, 2011. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 3 of 20 Program Student Learning Outcomes Upon completion of this program, students are expected to: 1. Apply entry-level knowledge of physics, anatomy, physiology, and pathophysiology related to sonography. 2. Perform general sonography procedures with continuing competency. 3. Demonstrate proficiency in patient assessment and care activities related to sonography. 4. Utilize effective oral and written communication with patients, physicians, and other medical personnel. 5. Employ professional and ethical judgment in the performance of sonographic duties. Table 1 Apply entry-level knowledge of physics, anatomy, physiology, and pathophysiology related to sonography. Perform general sonography procedures with continuing competency. Demonstrate proficiency in patient assessment and care activities related to sonography. Utilize effective oral and written communication with patients, physicians, and other medical personnel. Employ professional and ethical judgment in the performance of sonographic duties. Employer Survey National Certification Results Graduate Survey Capstone Evaluation Clinical Experience Documentation Objectives and Outcomes Clinical Competency Evaluations Association of Assessment Tools to Program Objectives an Outcomes 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1 0 1 1 0 1 1 0 1 1 0 1 0 = Tool is not used to measure the associated objective. 1 = Tool is used to measure the associated objective. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 4 of 20 Assessment Tools A description of the tools used in the assessment of the program objectives and their implementation are summarized in Table 2, below. The tools and their relationships to the program objectives are listed in Table 1, above. There is a separate appendix for each tool that shows the tool itself and describes its use and the factors that affect the results. Table 2 Program Objectives Assessment Tools and Administration Tool Description Clinical Competency Evaluations Tool used to assess student performance in completion of required competencies. Clinical Experience Documentation Tool required by ARRT for certification to record student performance of procedures. Capstone Evaluation Graduate Survey Tool used to assess student cumulative knowledge, skill, and behavior in performance of sonographic duties and procedures. Tool used to assess how well the student feels the program prepared them for entry-level employment. Frequency/ Start Date Continuously / During DMS A295A, DMS A295B, and DMS A395 Continuously / During DMS A295A, DMS A295B, and DMS A395 Collection Method Administered by Submitted to faculty Clinical instructors and preceptors Completed copy submitted to faculty Clinical instructors, preceptors, and faculty Once / During last eight weeks of DMS A395 Direct observation by faculty Faculty Once / Graduation Submitted to faculty Faculty National Certification Results Examinations administered by the ARDMS and ARRT Once / PostGraduation Mailed to program ARDMS and ARRT Employer Survey Tool used to assess how the employer feels the program prepared the student for entrylevel employment. Once / 90 days after hire Mailed to program Director or supervisor Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 5 of 20 Assessment Implementation & Analysis for Program Improvement General Implementation Strategy The Diagnostic Medical Sonography program will employ the six assessment tools identified in tables 1 and 2. They will be implemented in the following manner: The clinical competency evaluations, clinical experience documentation, and capstone evaluation will be utilized by clinical trainers and faculty; the graduate survey will be administered by faculty; the national certification results are prepared by the ARDMS and ARRT; and the employer survey is mailed by faculty. The data will be gathered according to the following timelines: The clinical competency evaluations and clinical experience documentation will be collected weekly during DMS A295A, DMS A295B, and DMS A395; the capstone evaluation will be conducted once during the last eight weeks of DMS A395; the graduate surveys are collected by faculty during the two weeks proceeding graduation; the national certification results and the employer survey are received by mail. All assessment tools will be reviewed by faculty immediately upon receipt. The data will be analyzed according to program objectives and outcomes. Method of Data Analysis and Formulation of Recommendations for Program Improvement Faculty of the program will meet, at minimum, once a year to review the data collected using the assessment tools. This meeting should result in recommendations for program changes that are designed to enhance performance relative to the program’s objectives and outcomes. A plan should be formulated during this meeting to implement the recommended changes, and to notify program stakeholders of the changes. The results of the data collected and program changes are to be submitted using the prescribed format to the office of Academic Affairs by the end of May each year. The proposed changes may be any action, change in policy, and/or revision of curriculum that the faculty deems necessary to improve performance relative to programs objectives and outcomes. Recommended changes should also consider workload (faculty, staff, and students), budgetary, facilities, and other constraints. A few examples of changes made by programs at UAA include: Changes in course content, scheduling, sequencing, prerequisites, delivery methods, etc. Changes in faculty/staff assignments Changes in advising methods and program requirements Addition and/or replacement of equipment Changes to facilities Modification of the Assessment Plan The faculty, after reviewing the data collected and the methods used to collect it, may decide to alter the assessment plan. Changes may be made to any component of the plan, including the objectives, outcomes, assessment tools, or any other aspect of the plan. The changes are to be approved by the faculty of the program. The modified assessment plan is to be forwarded to the dean/director’s office and the Office of Academic Affairs. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 6 of 20 Appendix A: Clinical Competency Evaluations Tool Description: This tool provides clinical trainers a standardized format to evaluate students in specific performance areas. There are 31 general sonography procedures and activities that require competency evaluation. These competencies for general sonography are dictated by the ARDMS and ARRT. Each competency evaluation utilizes a procedure/activity specific form. It will be utilized throughout DMS A295A, DMS A295B, and DMS A395. Factors that affect the collected data: The experience of the clinical trainer. The leniency of the clinical trainer. The medical condition of the patient. The cooperation of the patient. How to interpret the data: The clinical competency evaluations indicate the minimum level of proficiency, under normal conditions, required for each procedure or task. The data on this form correlates directly to all program objectives and outcomes. The data will show trends in performance of a specific task and/or student. Trends in performance failure for a specific task can indicate where a change might need to be made, in either the curriculum or the clinical education. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 7 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 8 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 9 of 20 Appendix B: Clinical Experience Documentation Tool Description: This tool provides students a method to document completion of specific procedural requirements. The ARRT requires the documented performance of all required procedures within the 24-month period immediately preceding the student’s application for certification. It will be utilized throughout DMS A295A, DMS A295B, and DMS A395. Factors that affect the collected data: The initiative of the student. The workload of the clinical training facility. How to interpret the data: The clinical experience documentation indicates completion of the minimum number of procedures required to be eligible to apply for certification by the ARRT. The data on this form correlates directly to three program objectives and outcomes. The data will show the volume of procedures completed by a student during the clinical practicums. The volume demonstrated may help faculty determine the best clinical training facilities for specific procedures. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 10 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 11 of 20 Appendix C: Capstone Evaluation Tool Description: This tool provides faculty an opportunity and a standardized format to evaluate the sonographic knowledge and skill of each student. This evaluation will be used to determine a student’s final grade for the course. It will be utilized at some point during the last eight weeks of DMS A395. Factors that affect the collected data: None. How to interpret the data: The capstone evaluation indicates the readiness of a student to perform general sonography with entrylevel ability. The data on this form correlates directly to all program objectives and outcomes. The data will show the student’s cumulative proficiency in the performance of sonographic duties and procedures. This data will help faculty to determine the effectiveness of the program to prepare students for entry-level employment. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 12 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 13 of 20 Appendix D: Graduate Survey Tool Description: This tool provides students an opportunity to assess how well they feel the program prepared them for entry-level employment based upon their academic knowledge and clinical proficiency. This survey will evaluate specific aspects of the course. It will be administered during the last two weeks of DMS A395. Factors that affect the collected data: The individual experiences of the student. How to interpret the data: The graduate survey indicates student preparedness to perform entry-level general sonography upon graduation. The data on this form correlates directly to all program objectives and outcomes. The data will show the students’ satisfaction with the program in meeting the program objectives and outcomes. This data will help faculty make improvements to the program based on student feedback. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 14 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 15 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 16 of 20 Appendix E: National Certification Results Tool Description: This tool provides a method for the program to receive statistical information on the number of students who pass the certification examination. It evaluates sonographic knowledge in specific topical areas. This examination is administered by the ARDMS and ARRT. The student can challenge the examination post graduation, and after submitting the appropriate application/documentation. Factors that affect the collected data: The amount of preparation by the student. How to interpret the data: The certification examination results indicate the academic knowledge of the student. The data from these results correlate directly to one program objective and outcome. The data will show how well the program taught specific sonographic information. This data will help faculty make adjustments to curriculum content to match knowledge areas being tested. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 17 of 20 Appendix F: Employer Survey Tool Description: This tool provides employers/supervisors an opportunity to assess how well they feel the program prepared the student for entry-level employment based upon the sonography services offered, and the needs of the health care facility. This survey will evaluate specific aspects of the course. It will be administered no sooner than 90 days after being employed as an entry-level general sonographer. Factors that affect the collected data: Notification of being employed as a sonographer by the student. How to interpret the data: The employer survey indicates student preparedness to perform entry-level general sonography post graduation. The data on this form correlates directly to all program objectives and outcomes. The data will show the employers’ satisfaction with the program in meeting the program objectives and outcomes. This data will help faculty make improvements to the program based on employer feedback. Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 18 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 19 of 20 Diagnostic Medical Sonography Educational Effectiveness Assessment Plan Page 20 of 20