ABET-ASAC Accreditation Workshop ABET Criteria and Outcomes Assessment Copyright © 2013 by ABET Criteria for Accrediting Applied Science Programs The document can be found on the ABET website (www.abet.org) The General Criteria applies to all programs seeking accreditation under ASAC In addition to the General Criteria, if there is program specific criteria for the type of program being considered, then the program will also be required to meet program specific criteria Applicable Program Specific Criteria is determined by the program title Currently there is no construction management program criteria in force. CMAA has submitted the draft program criteria to ABET for review and approval. Once it is approved, programs will then be required, where applicable, to meet the program criteria as well. Copyright © 2013 by ABET 2 3 General Criteria for Accrediting Applied Science Programs Copyright © 2013 by ABET 4 General Criteria Sections Criterion 1: Students Criterion 2: Program Educational Objectives Criterion 3: Student Outcomes Criterion 4: Continuous Improvement Criterion 5: Curriculum Criterion 6: Faculty Criterion 7: Facilities Criterion 8: Institutional Support Copyright © 2013 by ABET Criterion 1: Students The program must: Evaluate student performance, advise students, and monitor students’ progress. Have and enforce policies for acceptance of transfer students and validation of courses taken elsewhere. Have and enforce procedures to assure that all students meet all program requirements. Copyright © 2013 by ABET Criterion 2: Program Educational Objectives (PEOs) The program must have in place: Published PEOs consistent with mission and these criteria. Process that periodically documents and demonstrates that the PEOs are based on the needs of the program’s constituencies. Copyright © 2013 by ABET 7 Criterion 2: PEO Characteristics Clearly defined Broadly applicable Consider constituents and mission Achievable/realistic Relevant to the Profession Do not contradict one another Align with Professional Standards Feasible to measure Copyright © 2013 by ABET Criterion 3: Student Outcomes The program must demonstrate that (a) – (k) are attained. Note: For Associate degree programs, (a) – (i) listed under “Associate Degree Programs.” Student outcomes are defined as (a) – (k) plus any additional ones articulated by the program. Student outcomes must foster attainment of the PEOs. There must be an assessment and evaluation process that periodically documents and demonstrates the degree to which outcomes are attained, which is addressed under Criterion 4 – Continuous Improvement. Copyright © 2013 by ABET Criterion 3: Student Outcomes Describe what students are expected to know and be able to do at the time of graduation. Relate to skills, knowledge, and behaviors that students acquire as they progress through the program. Copyright © 2013 by ABET 9 Criterion 3: Student Outcomes Baccalaureate degree programs must demonstrate that graduates have: a) an ability to apply knowledge of mathematics, science, and applied sciences b) an ability to design and conduct experiments, as well as to analyze and interpret data c) an ability to formulate or design a system, process, or program to meet desired needs d) an ability to function on multidisciplinary teams e) an ability to identify and solve applied science problems Copyright © 2013 by ABET 10 Criterion 3: Student Outcomes f) an understanding of professional and ethical responsibility g) an ability to communicate effectively h) the broad education necessary to understand the impact of solutions in a global and societal context i) a recognition of the need for and an ability to engage in life-long learning j) a knowledge of contemporary issues k) an ability to use the techniques, skills, and modern scientific and technical tools necessary for professional practice. Copyright © 2013 by ABET 11 Program Educational Objectives Vs. Student Outcomes Program Educational Objectives specify desired future attainment (Expectations a few years after graduation). Student Outcomes specify what students will know or be able to do as a result of completing a program’s curriculum (At the time of graduation). Differences Degree of specificity Role of constituents Copyright © 2013 by ABET 12 Criterion 4: Continuous Improvement Programs shall use appropriate, documented processes for assessing and evaluating the extent to which the student outcomes are being attained. Each program must show evidence of actions taken to improve the program. These actions should be based on available information. The improvements can be based on any available information! Copyright © 2013 by ABET Criterion 5: Curriculum The curriculum requirements specify subject areas appropriate to applied science programs but do not prescribe specific courses. How does the curriculum align with the program educational objectives? How does the curriculum and its associated prerequisite structure support the attainment of the student outcomes? Copyright © 2013 by ABET Criterion 6: Faculty Sufficient number to maintain continuity, stability, oversight, student interaction, and advising Competence of faculty members must be demonstrated by such factors as education, professional credentials and certifications, etc. Responsibility and authority to improve the program Some program criteria have additional requirements. Copyright © 2013 by ABET Criterion 7: Facilities Classrooms, offices, laboratories, and associated equipment must be adequate to support attainment of the student outcomes and to provide an atmosphere conducive to learning. The team will tour the facilities as part of the visit process. Examples of facilities may include teaching labs, computer labs, libraries, advising centers, etc. Copyright © 2013 by ABET Criterion 8 : Institutional Support Institutional support and leadership must be adequate to ensure the quality and continuity of the program. Resources include institutional services, financial support, and staff (both administrative and technical). Copyright © 2013 by ABET Program- and Degree-Specific Criteria In addition to the eight General Criteria, the team will assess the program’s compliance with applicable program and degree specific criteria unless the program is being reviewed under the General Criteria only. Applicable program specific criteria are determined by the program title. Additional degree criteria apply to master’s degree programs. Copyright © 2013 by ABET Accreditation Policy and Procedure (APPM) Items There sections of the ABET Accreditation Policy and Procedure Manual (APPM) that programs should pay particular attention to: Number of graduates II.E.5. To be eligible for an initial accreditation review, a program must have at least one graduate within the academic year prior to the academic year of the on-site review. Copyright © 2013 by ABET APPM Items II.E.4. Program names must meet ABET requirements. The program name must be descriptive of the content of the program. The program name must be shown consistently on transcripts of its graduates, in the institution’s electronic and print publications, and on the ABET Request for Evaluation (RFE). If a program name implies specialization(s) for which Program Criteria have been developed, the program must satisfy all applicable Program Criteria. A program may choose to have an option, or similar designation implying specialization within the program, reviewed as a separate program. If a program name invokes review by more than one commission, then the program will be jointly reviewed by all applicable commissions. Copyright © 2013 by ABET 21 Outcomes-Based Assessment Outcomes-based Assessment Institutions and programs define mission and objectives to meet needs of their constituents Provides for program differentiation Outcomes: preparation for professional practice Programs demonstrate how student outcomes are being met Copyright © 2013 by ABET Remember… The institution must provide evidence that they have a working and effective system of assessment of student outcomes in place. The institution must describe a clear relationship between program educational objectives, student outcomes, and measurable indicators of success with required levels of achievement. The evaluation team is assessing programs based on the criteria and the strength of the evidence provided by the institution, not your own personal references. Copyright © 2013 by ABET 23 Assessment Methods DIRECT INDIRECT Student assignments Surveys and questionnaires National exams Focus groups Student portfolios Advisory board recommendation polls, exercises Exit interviews Locally developed Copyright © 2013 by ABET 24 Assessment Methods Valid assessment methods generate useful data. RELEVANT – Measures the educational outcome as directly as possible. ACCURATE – Measures the educational outcome as correctly as possible. USEFUL – Measures provide formative and summative results with clear implications for educational program evaluation and improvement. Copyright © 2013 by ABET 25 Emphasis Under Outcomes-Criteria Practice of continuous improvement Input of constituencies, with emphasis on involving an industrial advisory committee Process focus Outcomes & assessment linked to objectives Knowledge required for entry into “the profession” Copyright © 2013 by ABET 26 Assessing Student Outcomes Describe what students are expected to know at the time of graduation. Relate to skills, knowledge, and behaviors that students acquire as they progress through the program. Copyright © 2013 by ABET 27 Assessing Student Outcomes Performance Indicators Articulate key characteristics of an outcome. Make it possible for faculty to choose “good fit” methods. 2 critical components Subject content that is the focus of instruction Direct students to a specific performance Copyright © 2013 by ABET 28 Assessing Student Outcomes Performance Indicators Provide faculty with clear direction for implementation in the classroom. Make expectations explicit to students. Focus on data collection. Copyright © 2013 by ABET 29 Assessment Methods There will always be more than one way to measure any student outcome. No single method is enough for measuring a variety of different student abilities. There is generally an inverse relationship between the quality of measurement methods and their expediency. It is critical to pilot test a method to see if it is appropriate for an outcome and a program. Copyright © 2013 by ABET 30 Assessment Misconceptions School must demonstrate that EVERY student has attained the student outcomes. School must have a separate rubric for each student outcome. School must collect data from EVERY course that significantly covers the outcome. School must aggregate or average ALL data to demonstrate attainment. Copyright © 2013 by ABET 31 Reporting Student Outcomes Necessary elements in data reporting of student outcomes Performance Indicator (What is being evaluated?) Courses (Where indicator is being evaluated?) Method(s) of Assessment (How evaluated?) Summative Data Collection (Target courses?) Length of Assessment Cycle Year(s)/semester of data collection Target for Performance Copyright © 2013 by ABET 32 Reporting Misconceptions Demonstrate that EVERY student has attained outcomes. Separate rubrics for each student outcome. Include assessment of outcomes in assessment of objectives. Collect data from EVERY course that covers outcome. Aggregate or average ALL data to demonstrate attainment. Copyright © 2013 by ABET Questions? Copyright © 2013 by ABET