The Office of Academic Affairs External Review of Colleges and Schools Purpose Chancellor Cigarroa’s Framework for Advancing Excellence throughout The University of Texas System includes nine major areas of focus. One area addresses Faculty, Administrators, and Staff Excellence. As part of this Framework focus area, the U. T. System and institutions are required to conduct college/school external evaluations, specifically, “to review and strengthen plans to provide for holistic external reviews of each institution’s college or schools, similar to organized research unit reviews as provided in Regents’ Rule 40602.” A work group of provosts and deans convened to assist U. T. System staff in the development of a process to meet the requirements of the Chancellor’s Framework for external college reviews. The general charge to the group was to develop a review process that ensures that colleges and schools are organized to maximize student success. Review Process Institutional reports submitted to specialized accrediting bodies that include college- or schoollevel accreditation reviews will satisfy the U. T. System external review reporting requirement. Specialized accrediting bodies that include college-level review are listed in Appendix A. Each college or school under review shall do a self-study that examines key success metrics within the college/school and the factors that may affect a student’s ability to be successful in college and ultimately earn a degree. A template is included in Appendix B as a guide for the self-study. Two reviewers external to U. T. System shall review the self-study, conduct a campus visit, and write a report outlining their findings. The university should select individuals who can bring critical judgment and objectivity to the process and are able to compare the college to similar colleges at other institutions. One reviewer should be from a discipline within the college being reviewed; the other should have expertise in student success initiatives. The report shall be forwarded to the president and provost of the institution. The president and provost, in consultation with the dean of the college or school, shall determine how or whether to implement any recommendations in the report. The external reviewers’ report and an institutional response to the report shall be submitted to the Executive Vice Chancellor for Academic Affairs for review and acceptance. Self-Study Guidelines Key Student Success Metrics 1 The self-study must include a careful analysis of key student success metrics1. The Office of Strategic Initiatives (OSI) is building departmental and college-level dashboards to assist institutions with obtaining the data associated with certain metrics. Institutions may also use their own data to obtain program-, department-, and college-level metrics that they would like to include in a college or school’s self-study. The OSI dashboards are scheduled to be launched in November 2013. The following student success metrics are required to be included in a college or school’s self-study. Appendix C includes the definitions used at U. T. System for each student success metric listed below. Other metrics, such as data on student engagement from the National Survey of Student Engagement (NSSE), data on critical thinking skills from the Collegiate Learning Assessment (CLA), etc., may also be included. Appendix D includes a sample analysis of one of the student success metrics. Metric Persistence Rates of Juniors Completion Rates of Juniors2 Things to Consider: Characteristics shared among non-persisting students Effects of financial aid Registration bars on persistence Administrative barriers to completion, including availability of required courses Quality of advising Number or percentage of students switching majors or dropping out Average Number of Semester Credit Hours to Graduation Administrative barriers, including availability of required courses Quality of advising Full-time versus part-time status Quality of advising Financial obstacles, including lack of institutional support Number of students who switched majors Number of Degrees Awarded Number of degrees awarded adequate to meet the institution-level goals for degree production Number of Majors Number of majors adequate by program to produce the target number of degrees awarded Average Number of Semesters to Graduation Factors That May Affect Student Success Many factors contribute to a student’s ability to earn a baccalaureate degree. When conducting the self-study, institutions should include an analysis of relevant factors to determine areas where improvements can be made or appropriate interventions can be developed that have the potential to positively impact student success. The following is a brief list of factors that have been found to The number of metrics available to be aggregated to the college level is limited. The metrics selected are seen as “student success-oriented”, or indicators of movement towards completion (i.e., persistence rate) or through completion (i.e., number of degrees awarded) of a baccalaureate degree. 2 The metric Completion Rate of Juniors was selected in lieu of Graduation Rate because the junior cohort includes all students classified as juniors, including transfer students, while Graduation Rate only includes first-time, full-time students. 1 2 affect student success. The self-study report is intended to allow colleges/schools to investigate the issues that are important and unique to them. Colleges/schools may add or remove factors, as appropriate. Targets for each factor must be identified, along with the college/school’s plans and timelines to meet those targets. Colleges/Schools might also consider analyzing the characteristics of successful students to identify common factors that may foster successful completion of a degree. Factor Admissions Criteria Gateway Courses Guiding Questions Are the college/school’s admissions criteria in line with its mission and goals for student success? How is the college addressing the D/F/W rates within its gateway courses? Things to Consider: The degree to which the college/school maintains consistency with its mission as it revises admissions criteria Quality of advising Availability and quality of academic support services Faculty awareness and inclusion in addressing the issue Quality of advising Orientation instructions and expectations Math Course Sequence Are students adequately planning their math course sequence to avoid semester overload or graduation delays? Program Course Sequence Are students adequately planning their prerequisites to avoid semester overload or graduation delays? Quality of advising Quality of degree plan templates Presence of course rotations Financial Aid Does the college/school adequately support students financially? Financial assistance availability Methods used to determine aid packages Availability of Academic Maps Does each degree program provide students with an academic map? Student Engagement How engaged are students in the college/school? Are all required courses offered regularly enough so that both new and continuing students may progress in a timely fashion? Are the degree program requirements too rigid? Too flexible? How does the degree program handle requests for course substitutions? Is the current process flexible enough so as not to hinder student progress? How do you know that students are engaged? What opportunities do student have to do internships, service learning, study abroad, etc.? Are courses delivered in ways that promote active student engagement? 3 College/School Policies Do the policies and procedures that are in place support effective and efficient processes towards improving student success? Other Factors? Whether the governance structure of the college/school is optimally conducive to efficient operations and continuous improvement. The degree to which the people responsible for continuous improvement processes are empowered to implement recommended changes. At what levels and how often the continuous improvement process occurs. Timeline Each college or school shall be reviewed at least every ten years. The U. T. System Office of Academic Affairs has developed a 10-year schedule for the review of all colleges/schools at the U. T. System general academic institutions. (See attached.) The self-study, external review, and institutional response to the external review must be completed during the academic year identified in the 10-year review schedule. 4 Appendix A Accrediting Bodies that Include College-Level Reviews U. T. System will accept reports prepared for the following accrediting bodies in satisfaction of the external college review:3 Accrediting Body for Engineering and Technology (ABET) Accreditation Commission for Education in Nursing Accreditation Council for Pharmacy Education (ACPE) Commission on Collegiate Nursing Education (CCNE)4 Council on Social Work Education (CSWE)5 American Bar Association (ABA) Association to Advance Collegiate Schools of Business (AACSB) National Council for Accreditation of Teacher Education (NCATE) National Office for Arts Accreditation (NOAA):6 o National Association of Schools of Art & Design (NASAD) o National Association of Schools of Music (NASM) o National Association of Schools of Dance (NASD) o National Association of Schools of Theater (NAST) Others may be added if we can confirm that they accredit above the level of the degree program up to and including the administrative unit commonly known as a college or school. Program-centered accreditation processes do not satisfy the goals of this external review initiative. The external college review is not intended to duplicate program-centered accreditation processes, but rather focuses on student success indicators aggregated to the college/school level. This perspective ensures that activities beyond the scope of the program (and the department in which the program is housed) are assessed as they too play a role in student success. 4 Satisfies the external college review only if nursing programs are housed under a College or School of Nursing. 5 For universities with Schools of Social Work with only social work programs (UT Arlington and UT Austin). 6 All four accreditation reviews combined (NASAD, NASM, NASD, and NAST) satisfies the external college review requirement for the College of Fine Arts at The University of Texas at Austin. 3 5 Appendix B Self-Study Template Prior to recruiting individuals to perform an external review, the college/school must conduct a selfstudy. This template is intended as a guide, and there may be additional information that the college/school will want to note as it develops its plans to address any areas in need of improvement. Part I: Overview Please provide a brief overview of the college/school mission, organizational and leadership structure, and student demographics. Part II: Goal Alignment Has the college/school identified a process to align college-level student success goals with institutional goals? How well are the goals aligned? Part III: Student Success Metrics Each college/school is asked to assess its performance on a number of key student success metrics. How well is the college/school doing with respect to the U. T. System metrics, as well as other metrics identified at the institution level and/or the college/school level?7 Part IV: Factors Affecting Student Success What factors are affecting student success in your college/school? And what measures are you taking to address them? Part V: Findings and Plans for Improvement Note that the System has no benchmarks or targets for the student success metrics required in this review. Colleges/Schools may identify their own targets, as appropriate. 7 6 Appendix C Student Success Metric Definitions The following student success metrics are required in a college or school’s self-study. The U. T. System Office of Strategic Initiatives will provide the data for each institution at the college or school level and at the department level. OSI uses the following definitions. Persistence Rate of Juniors – Percent of junior-level, degree-seeking students of declared majors by 6digit CIP code enrolled in at least 12 SCH in their junior year and still enrolled at the same or another institution, in the same or other program, one academic year later. Completion Rate of Juniors – Junior-level, degree-seeking students who enrolled in a minimum of 12 SCH in the Fall of their junior year who graduated with a Baccalaureate degree from the same institution or another U. T. System institution after four academic years, reported by whether the student graduated in the same program or graduated in a different program (identified by a 6-digit CIP code). Flex entry students are excluded. Average Number of Semesters to Graduation – Every student who earned a baccalaureate degree in specific majors (identified by a 6-digit CIP code) at a public general academic institution was traced back to when the student first enrolled at a U. T. System institution. For each of these students, the number of fall and spring semesters attended were totaled. Average Number of Semester Credit Hours to Graduation – Every student who earned a baccalaureate degree in specific majors (identified by a 6-digit CIP code) at a public general academic institution was traced back to when the student first enrolled at a U. T. System institution. For each of these students, the number of college-level semester credit hours attempted (excluding developmental education credits) were totaled for fall, spring, and summer semesters. Number of Majors – Number of undergraduate declared majors in the program by 6-digit CIP code for Fall and Spring semesters. Flex entry and dual credit students are excluded. Number of Degrees Awarded – Number of baccalaureate degrees awarded by 6-digit CIP code. 7 Appendix D Sample Student Success Metric Analysis Persistence Rate of Juniors Step 1: Assess the most recent persistence rate of juniors in the College. Discussion Question: What might be affecting persistence? Step 2: Analyze the persistence rate for the College and for the departments or programs in the College. Discussion Questions: A. Are some degree programs exhibiting lower rates of persistence than others? B. Which programs exhibit persistence rates lower than the College average and the institutional average? Step 3: Analyze the different types of students who did not reenroll from the previous semester. Analyze the breakdown of students who did not reenroll from the previous semester. Analyze the breakdown of academically eligible students from the previous semester. Discussion Questions: A. What trends do you observe? B. What can be done to increase persistence rates in the next year? C. What next steps should be taken? Step 4: Identify Potential Enrollees from Academically Eligible Group Contact eligible students who have no registration holds and students who require major advising. Forward list of students with financial, admissions, and financial aid holds to Enrollment Management for further action to determine potential pool. Forward list of students with holds to Student Affairs to determine potential pool. 8