Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Version 1.0 Adopted by The Welding & Nondestructive Testing Technology faculty: April 2011. Submitted to The Dean of the Community & Technical College: June 2011 The Office of Academic Affairs: June 2011 Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 1 of 15 TABLE OF CONTENTS Introduction _________________________________________________________________ 3 Program Objectives ___________________________________________________________ 4 Program Outcomes ___________________________________________________________ 5 Assessment Tools _____________________________________________________________ 6 Assessment Implementation & Analysis for Program Improvement ____________________ 10 General Implementation Strategy _________________________________________________________ 10 Method of Data Analysis and Formulation of Recommendations for Program Improvement ___________ 10 Modification of the Assessment Plan ______________________________________________________ 10 Appendix A: Graduate Survey _______________________________________________ 11 Appendix B: Employer Survey _______________________________________________ 12 Appendix C: Qualification Tests _____________________________________________ 13 Appendix D: Advisory Committee ____________________________________________ 14 Appendix E: Program Review _______________________________________________ 15 Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 2 of 15 INTRODUCTION This document defines the educational objectives and expected outcomes for the Welding & Nondestructive Testing Technology program and outlines a plan for assessing the achievement of the stated objectives and outcomes. The development of the objectives and outcomes followed the original program assessment work in 2000/2001 and program revisions planned and started in 2001/2002. The outcomes and nature of the major revision completed in 2002/2003 were based on meetings with the advisory group during 2000/2001 and a symposium held in December 2000 to discuss broader welding and nondestructive testing issues facing the state’s workforce development efforts. The faculty met and accepted the objectives, outcomes, and assessment processes in the spring of 2002 and initiated work on a major curriculum revision, completed during the summer of 2002, reviewed by faculty during the fall of 2002, accepted by the Provosts office in the spring of 2003, and approved by the Board of Regents in June 2003. During the spring of 2003 the department received assistance with a graduate survey. Those results are not included in this version of the plan, but will be added in the fall. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 3 of 15 PROGRAM OBJECTIVES The educational objectives of the Welding and Nondestructive Testing Technology program are to produce graduates who: Are prepared for employment in welding and/or nondestructive examination as entrylevel technicians. The program offers two transcripted certificates that together with General University Requirements qualify for a AAS degree. o Industrial welding technician students develop manual skills in four main welding processes and three thermal cutting processes, as well as gain a wide range of technical knowledge in welding application, procedure/welder qualification, reading plans and specifications, and applied metallurgy. Welder qualification tests are administered as prescribed in AWS D1.1, API Standard 1104, or ASME IX welding codes. o Nondestructive Testing Technology students examine metallic components or weldments to locate and evaluate discontinuities by learning to apply liquid penetrant (PT), magnetic particle (MT), eddy current (ET), radiographic (RT) and ultrasonic (UT) test methods. Student qualification in each NDT method is based on general, specific and practical examinations administered as prescribed in the American Society for Nondestructive Testing (ASNT) Recommended Practice No. SNT-TC-1A. Provides students with technical and administrative skills required in today’s metal fabrication and inspection environments required to apply specifications and codes to complete specific job tasks. Develop human relations, communications and computational skills. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 4 of 15 PROGRAM OUTCOMES The specific educational outcomes that support the program objectives are to produce graduates who are able to: Industrial welding technician students: Pass three (3) separate all-position welder qualification tests. Nondestructive Testing technician students: Pass two (2) separate NDT method qualification tests. Demonstrate technical knowledge of the interrelationship between welding and inspection processes. Work safely by assessing hazards and using best practices to avoid exposing themselves or others to risk of injury, and avoiding damage to equipment. Work and communicate effectively with other employees, customers, and management. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 5 of 15 ASSESSMENT TOOLS A description of the tools used in the assessment of the program objectives and their implementation are summarized in Table 1. The tools and their relationships to the program objectives are listed in Table 2. A description of the tools used in the assessment of the program outcomes and their implementation are summarized in Table 3. The tools and their relationships to the program outcomes are listed in Table 4. There is a separate appendix for each tool that describes the factors that affect the results and give examples of the tools and how they will be implemented. Table 1 Program Objectives Assessment Tools and Administration Tool Graduate Survey Frequency/ Start Date Collection Method Administered by Annual. Initial survey tested S03 Faculty distribution to students S03. Faculty. See Annual Workload for assignment. Survey of employers of graduates within the first year of completion. Annually. Survey to be developed. Faculty distribution following student reporting of employer. Faculty. See Annual Workload for assignment. Database of AWS and ASNT qualification tests by students including attempts and passing results. Data collected as tests administered. Database & Forms are required. Faculty or Lab Tech reporting to Administrative Assistant Admin Assist maintains database, Faculty review annually. Advisory group review of program and recommendations to assessment or curriculum. Annually – F2003 Minutes of Meeting. Admin Assist Meeting Minutes. Cyclical program review by the Office of Academic Affairs (OAA) 5 year cycle Completed in 2010. OAA Procedures OAA Description Survey of students completing certificates or the AAS degree within the first year of completion. (Including ID of employer.) Employer Survey Qualification Tests Advisory Committee Feedback Program Review Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 6 of 15 Students are prepared for employment in welding and/or nondestructive examination as entry-level technicians. Provides students with technical and administrative skills required in today’s metal fabrication and inspection environments required to apply specifications and codes to complete specific job tasks. Industrial welding technician students develop manual skills in four main welding processes and three thermal cutting processes, as well as gain a wide range of technical knowledge in welding application, procedure/welder qualification, reading plans and specifications, and applied metallurgy. Nondestructive Testing Technology, students are qualified in appropriate Nondestructive Testing technician students examine metallic components or weldments to locate and evaluate discontinuities by learning to apply liquid penetrant (PT), magnetic particle (MT), eddy current (ET), radiographic (RT) and ultrasonic (UT) test methods. 1 0 1 1 1 Develop human relations, communications and computational skills. 1 1 Program Review 1 1 Advisory Committee Feedback Qualification Tests 1 Graduate Survey Employer Survey Table 2 Association of Assessment Tools to Program Objectives 1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 0 = Tool is not used to measure the associated objective. 1 = Tool is used to measure the associated objective. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 7 of 15 Table 3 Program Outcomes Assessment Tools and Administration Tool Graduate Survey Frequency/ Start Date Collection Method Annual. Initial survey tested S03 Faculty distribution to students S03. Faculty. See Annual Workload for assignment. Survey of employers of graduates within the first year of completion. Annually. Survey to be developed. Faculty distribution following student reporting of employer. Faculty. See Annual Workload for assignment. Database of AWS and ASNT qualification tests by students including attempts and passing results. Data collected as tests administered. Database & Forms are required. Faculty or Lab Tech reporting to Administrative Assistant Admin Assist maintains database, Faculty review annually. Advisory group review of program and recommendations to assessment or curriculum. Annually – F2003 Minutes of Meeting. Admin Assist Meeting Minutes. Description Survey of students completing certificates or the AAS degree within the first year of completion. (Including ID of employer.) Employer Survey Qualification Tests Advisory Committee Feedback Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Administere d by Page 8 of 15 0 1 0 0 0 1 0 Demonstrate knowledge of the interrelationship between welding and inspection processes. 1 1 0 1 Work safely by assessing hazards and using best practices to avoid exposing themselves or others to risk of injury, and avoiding damage to equipment. 1 1 0 1 Work and communicate effectively with other employees, customers, and management. 1 1 0 0 Industrial welding technician students: Pass three (3) separate all-position welder qualification tests. Nondestructive Testing technician students: Pass two (2) separate NDT method qualification tests. Advisory Committee Feedback Qualification Tests 0 Graduate Survey Employer Survey Table 4 Association of Assessment Tools to Program Outcomes 0 = Tool is not used to measure the associated objective. 1 = Tool is used to measure the associated objective. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 9 of 15 ASSESSMENT IMPLEMENTATION & ANALYSIS FOR PROGRAM IMPROVEMENT General Implementation Strategy Faculty has primary responsibility for development and administration of the assessment plan. Annually faculty will be selected and assigned responsibility as a component of the faculty Workload Agreement as a Service component of the annual activities. Administrative staff for the department is responsible for distributing and collecting surveys. Faculty and division director are responsible for analysis of data. Method of Data Analysis and Formulation of Recommendations for Program Improvement The program faculty meets at least once a year to review the data collected using the assessment tools. This meeting should result in recommendations for program changes that are designed to enhance performance relative to the program’s objectives and outcomes. The results of the data collection, an interpretation of the results, and the recommended programmatic changes are to be forwarded to the office of Academic Affairs (in the required format) by the end of May each year. A plan for implementing the recommended changes, including of advertising the changes to all the program’s stakeholders, is also to be completed at this meeting. The proposed programmatic changes may be any action or change in policy that the faculty deems as being necessary to improve performance relative to programs objectives and outcomes. Recommended changes should also consider workload (faculty, staff, and students), budgetary, facilities, and other relevant constraints. A few examples of changes made by programs at UAA include: o changes in course content, scheduling, sequencing, prerequisites, delivery methods, etc. o changes in faculty/staff assignments o changes in advising methods and requirements o addition and/or replacement of equipment o changes to facilities Modification of the Assessment Plan The faculty, after reviewing the collected data and the processes used to collect it, may decide to alter the assessment plan. Changes may be made to any component of the plan, including the objectives, outcomes, assessment tools, or any other aspect of the plan. The changes are to be approved by the faculty of the program. The modified assessment plan is to be forwarded to the dean/director’s office and the Office of Academic Affairs. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 10 of 15 APPENDIX A: GRADUATE SURVEY Tool Description: Written survey completed by students that have completed a certificate program or the AAS degree. Factors that affect the collected data: Because the assessment tool relies on voluntary compliance, the data will not include information from all completers. How to interpret the data: See above. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 11 of 15 APPENDIX B: EMPLOYER SURVEY Tool Description: Written survey completed by employers with direct knowledge of a previous student’s workplace skills. Factors that affect the collected data: Primary limitations of this instrument will be the low numbers of completers. It will be critical that students that complete the program provide employer information and that the department as some method to collect and maintain this data. It will also be critical that employers are only asked to provide feedback on students that have completed a full program of study. Many students have left the program after a couple initial courses have prepared them initial employment – the department should be careful to insure that employers are only rating those students that have completed a certification or AAS degree. The survey should provide some explicit instructions and clarity related to the student’s certificate or degree to insure accurate and relevant assessment. How to interpret the data: See above. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 12 of 15 APPENDIX C: QUALIFICATION TESTS Tool Description: Database of student attempts and results from AWS and ANST qualification tests for review by faculty to assess student success and common causes of failures during testing. Factors that affect the collected data: Testing should be administered only by qualified faculty or lab technicians. Testing and testing results should not be punitive in nature in a way that would bias the testing results or reduced the normal attempts to pass the test. (Although if students become more self aware of the testing standards and know their work will not pass the tests, then the lack of testing in that case will be an indication that students have demonstrated awareness of the quality of their work.) A method will have to be determined to separate student practice work, from a formal attempt to pass the qualification test. A form will be needed to document the testing and a process to enter the data in to a database. A standard set of failure modes and descriptions will be required to assure consistency in the characterization of results – particularly failures. How to interpret the data: The AWS and ASNT guidelines provide clear testing procedures. Interpretation by the faculty should be based on assessing the reasons student do not pass the test and the use of this information to modify instruction or lab activities. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 13 of 15 APPENDIX D: ADVISORY COMMITTEE Tool Description: Annual advisory committee meetings with review of program changes, lab facilities and equipment, tests, and community feedback on students work. Factors that affect the collected data: Selection and critical objectivity of advisory committee members. How to interpret the data: Results from annual meetings will be largely anecdotal, but should provide important indicators to the faculty about changes in industry expectations, student skills, and shifts in industry practices or certifications. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 14 of 15 APPENDIX E: PROGRAM REVIEW Tool Description: Cyclical Program Review administered by the Office of Academic Affairs. Factors that affect the collected data: See OAA procedures and BOR policy for reviews. The principle factor affecting the reviews is the make up of the advisory committee and the preparation of a self review. The selection of impartial members that can be objective and critical of the programs successes and short comings assure a valuable result that can guide future program improvements. How to interpret the data: See OAA procedures and BOR policy for reviews. Welding & Nondestructive Testing Technology Educational Effectiveness Assessment Plan Page 15 of 15