University of Massachusetts Amherst 2014-2015 Academic Quality Assessment & Development (AQAD) Outcomes Engineering A college offering the B.S., M.S.C.H. and Ph.D. in Chemical Engineering; the B.S., M.S.C.E., M.S. and Ph.D. in Civil and Environmental Engineering; the B.S. in Computer Systems Engineering; the M.S.E.C.E. and Ph.D. in Electrical and Computer Engineering; the B.S. in Electrical Engineering; the M.S.in Engineering Management; the B.S. in Industrial Engineering; the M.S.I.E.O.R. and Ph.D. in Industrial Engineering and Operations Research; and the B.S., M.S.M.E. and Ph.D. in Mechanical Engineering. The Review Process The College of Engineering is accredited by the Accreditation Board for Engineering and Technology (ABET), which is a federation of 32 professional and technical member societies representing the fields of applied science, computing, engineering and technology. ABET is the recognized accreditor for U.S. and international university programs in engineering and holds its evaluations on a six-year cycle that coincides with the College’s AQAD process. The results of the 2014-15 ABET review of the College as a whole are reported here. ABET reviewers were: W. Vance McCullough, Raytheon, Team Lead John Weidner, University of South Carolina Manoochehr Zoghi, California State University-Fresno Mark Petzold, St. Cloud State University Cass Kuhl, National Aeronautical and Space Administration Bala Ram, North Carolina A&T State University Eugene Brown, Virginia Polytechnic Institute and State University Main Issues The four departments within the College of Engineering (CoE) had their accreditation renewed for a full six-year term, through 2019-20. Continuation of the College’s accreditation reflects compliance with an array of standards including educational objectives, student outcomes and research innovation. The accreditation report describes a College with a glowing record whose individual subunits’ strengths include curriculum and program quality, placements in both academia and the labor force, and excellent parity in teaching and research quality among the four departments. The report especially praised the College’s institutional participation in twelve national research centers, the awarding of National Science Foundation grants to thirty faculty members and the continued pursuit of a “strong commitment to diversity for faculty and students.” The assessment found no deficiencies at the College, and therefore no issues of noncompliance. The reviewers listed a total of seven criteria that remained unresolved after the accreditation evaluation, all of which were either termed as a weakness or a concern. A weakness stipulates remedial action to strengthen that subunit’s compliance with a criterion, policy or procedure; a concern indicates the potential for a situation to change in the future, but no recommendation for follow-up action. University of Massachusetts Amherst • Office of Academic Planning and Assessment The ABET review committee’s findings (both resolved and unresolved issues) included: Concerns in two of six academic programs (computer science engineering and electrical engineering) related to Criterion 1: Students. In both cases, the issue was “inadequate advising.” In one program, the reviewers found evidence that “the majority of students received appropriate advising” but that some students “did not get adequate advising, including course selection and career matters.” In the other program, all students received advising but “the quality of that advising varied greatly.” Concerns in two of six academic programs (industrial engineering and mechanical engineering) and weakness in another (civil engineering) related to Criterion 2: Program Educational Objectives. In the two programs with a concern, the committee cited a failure to adequately document the minutes of internal meetings related to ongoing revisions to program objectives. In the program with a weakness, the reviewers found that the stated objectives for graduates “within a few years of graduation” were lacking. Concern in one of six programs (civil engineering) and weaknesses in two others (industrial engineering and mechanical engineering) related to Criterion 4: Continuous Improvement. In the one program with a concern, the committee noted that “the present processes for the collection, analysis, and evaluation of data are very cumbersome and may not be sustainable.” In the two programs with a weakness, the reviewers suggested that overreliance on student surveys in lieu of student work product could jeopardize assessment efforts. Weaknesses in two programs (industrial engineering and mechanical engineering) related to Criterion 5: Curriculum. In both programs, the committee invoked a slight misalignment of learning objectives in the major design experience (capstone) project. Concerns in four of six programs (chemical engineering, civil engineering, computer systems engineering and electrical engineering) related to Criterion 7: Facilities. In two of these cases, the problem was described as both an overall space and an individual workstation shortage. Increasing enrollment leading to stressed laboratory space was the issue in the other two programs, with one program’s evaluation additionally indicating a concern about inadequate staffing levels. Results of the Review The College of Engineering discussed the report prepared by the ABET accreditors, offered a number of responses to the stated concerns and weaknesses, and resolved most issues raised by the ABET review committee. Criterion 1: Students. Resolved. The faculty provided documentation describing six actions taken to address concerns pertaining to course selection and career advising. Criterion 2: Program Educational Objectives. Resolved. The faculty provided meeting minutes documenting the program’s review and reaffirmation of its program educational objectives in Fall 2013, prior to the accreditation visit. Criterion 4: Continuous Improvement. Weaknesses resolved; one concern unresolved. The faculty provided additional documentation demonstrating the collection, analysis and use of direct measurement data from the spring semester to improve both programs. For the program with a concern, the faculty provided documentation describing a more streamlined process that, “when fully implemented ... should help address this concern.” Criterion 5: Curriculum. Resolved. The faculty provided documentation showing that learning objectives were appropriately covered in the pre-capstone course and have now been added to the final semester course syllabus. Criterion 7: Facilities. Four concerns unresolved. While the College was able to provide information demonstrating that “substantial funds have been committed” to address space University of Massachusetts Amherst • Office of Academic Planning and Assessment constraints and to upgrade laboratories, the ABET review committee nonetheless deemed these submissions inadequate to address the larger concerns. Outcomes Assessment The four departments and six undergraduate programs within the College of Engineering each have an established set of student learning objectives. These include developing a knowledge base of mathematics, science and engineering within their major; designing and conducting experiments, as well as analyzing and interpreting data; designing systems, components or processes to meet real-world needs; and many other items. Each department incorporates systematic assessment measures to determine if these learning outcomes are being met including freshman- and senior-year surveys; course performance indices including both faculty assessment and student self-assessment; and an undergraduate curriculum committee that routinely reviews assessment results and makes recommendations to department faculty for curricular and programmatic revisions. Student Retention and Graduation Rates The College analyzed both freshman retention and graduation rate data provided by the University, focusing on the period since its previous ABET review in academic year 2007-08. The freshman retention data show that first-year persistence in the College has remained very near 80% over time, and first-year persistence has increased slightly over the past few years to over 94%. Increases of approximately 10% and 5% were seen in the four- and six-year graduation rates, respectively, within the College across the most recent cohorts for whom data are available. In the six-year cohort starting in 2008, 61.8% of the students graduated within the College and 80.5% of the students graduated in some major at UMass Amherst. From these data the department concluded that the College (1) attracts capable students who graduate at high rates within UMass Amherst; (2) fails to retain a remarkably constant 20% of students after freshman year; and (3) is increasingly successful at graduating the students who persist after freshman year. University of Massachusetts Amherst • Office of Academic Planning and Assessment