Step 2. Identify the 4­6 most important student learning outcomes As a department or program you stand for something. When students decide to enter your major they have certain expectations about your program and the promises it holds for them. By publishing clearly written learning outcomes (a.k.a. learning goals), you present the students with your promise, a promise about what they can expect to achieve when they complete your program successfully. As a faculty, this is your commitment to your students. The assessment process presents evidence to them that you honor that commitment. During Step 2 you should again invite input from all stakeholders in your program, including staff, students, alumni and professionals in the community who will employ your graduates to create the most relevant learning outcomes possible. In the past many faculty thought that SACS wanted to know every learning objective covered in every course in the curriculum. That is no longer the case. SACS does not want to see how every student performed on every learning objective for every course. It wants to see evidence of how your students performed on the program learning outcomes, and what you did to improve the situation. As a unit you need to agree on 4-6 key program learning outcomes that answer the question: “What will students who graduate from our program be able to DO intellectually, physically, and emotionally?” What minimal skill set should these “new players” on our team have acquired by the time they leave our undergraduate training program? While SACS is mostly interested in undergraduate students at this time, you should strive to develop the undergraduate curriculum such that the transition to graduate studies is virtually seamless for the students. Drafting student learning outcomes can be difficult. It is challenging to reach the level of specificity required in a relevant, measurable student learning outcome. Therefore, faculty members might want to plan a mini-retreat or set aside specific meetings that center solely on formulating the learning outcomes so that undivided attention can be focused on this task for a period of time. To begin drafting the program’s student learning outcomes, it may be helpful to start with a very general phrasing of the outcome, and then get more specific with each revision. It may take several iterations to progress from lofty, idealistic student learning goals to one or more specific, measurable outcomes. Your learning outcomes should distinguish the program’s graduates from other University students. You can achieve this by clearly answering the question: “What knowledge, skills, or attitudes distinguish the graduates of our program from other students on campus?” To arrive at answers to this question, it is recommended that the faculty identify what constitutes the “ideal student.” The worksheet on the next page can be used as a basis for discussions and work sessions on the various questions about skills, knowledge, values and attitudes that you believe this student has acquired or strengthened as a result of your program. Individual participants should be given some time to think through these items first before continuing the discussion in either small groups or the entire body present. Having small groups discuss the items first and then report them on large easel pads usually leads to richer results. Here are some examples of learning outcomes from A Program Guide for Outcomes Assessment at Geneva College (April 2000) as presented in the UCF Handbook1: “Poor Learning Outcome Statement: Students should know the historically important systems of psychology. This is poor because it says neither what systems nor what information about each system students should know. Are they supposed to know everything about them or just names? Should students be able recognize the names, recite the central ideas, or criticize the assumptions?” “Better Learning Outcome Statement: Students should understand the psychoanalytic, Gestalt, behaviorist, humanistic, and cognitive approaches to psychology. This is better because it says what theories students should know, but it still does not detail exactly what they should know about each theory, or how deeply they should understand whatever it is they should understand.” “Best Learning Outcome Statement: Students should be able to recognize and articulate the foundational assumptions, central ideas, and dominant criticisms of the psychoanalytic, Gestalt, behaviorist, humanistic, and cognitive approaches to psychology. This is the clearest and most specific statement of the three examples. It provides even beginning students an understandable and very specific target to aim for. It provides faculty with a reasonable standard against which they can compare actual student performance.” A simple guide to writing learning outcomes can be found at the TX A&M University assessment website at http://assessment.tamu.edu/asmt_help/writing_learning_outcomes.pdf Examples of Program Learning Outcomes at UTEP Department of Civil Engineering 1. Graduates will be educated in the fundamental concepts of engineering and science to create intellectual curiosity in order to provide for a successful career and life-long learning. 2. Graduates will be able to design effective civil engineering systems. 3. Graduates will have the ability to function on multidisciplinary teams. 4. Graduates will serve as productive members of society and the profession by recognizing the social, ethical, environmental, and political implications of engineering decisions. 1 University of Central Florida Academic Program Assessment Handbook, June 2008 Edition 5. Graduates will be able to communicate effectively to technical and non-technical audiences. 6. Graduates will have exposure to real-life problems including hands-on experience Department of Computer Sciences (includes instruments and process of assessment) The overarching educational objective of the B.S. in Computer Science program at UTEP is to produce graduates who will be in a profession or in a graduate program that utilizes their technical expertise, the foundation of which was obtained at UTEP. Specifically, graduates of UTEP's undergraduate program in Computer Science will be able to: 1. Use the theoretical and technical computer science knowledge to specify requirements, develop a design, and implement and verify a solution for computing systems of different levels of complexity. (Objective 1) a. Apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems. b. Estimate the feasibility and effort required to build a particular computing system. c. Identify and specify requirements for computing systems by selecting appropriate modeling techniques and tools. d. Design, implement, and verify computing systems of varying complexity by using appropriate techniques and tools and by selecting appropriate design patterns, architectures, languages, and testing approaches. e. Evaluate a system with respect to criteria such as performance, complexity, correctness, and usability. f. Determine the impact of an architecture or platform on software design and implementation alternatives. g. Apply problem-solving techniques to solve real-world problems. 2. Convey technical information in both oral and written formats. a. Present technical information orally. b. Write a professional technical report. c. Formulate and pose incisive, technical questions. 3. Work in teams. a. Participate as a productive member of a team. b. Solve common problems in team dynamics. 4. Apply a professional code of ethics in the daily practice of their profession. a. Project the potential impacts of technical decisions on the individuals, organizations and external constituencies involved, and identify ethical and legal implications. b. Apply the insights embodied in professional codes of ethics. 5. Stay current in their profession. a. Describe the importance of and options available for continuing education. b. Describe the role of professional societies c. Articulate the benefits of graduate studies Assessment Instruments used in Computer Science The instruments used to assess the program include: Graduating Senior Survey; Alumni Survey; College of Engineering Employer’s Survey; Senior Exit Interviews; Departmental Advisory Board; Teaching Evaluations; Course Assessment; Industry Feedback; Advising Survey. The long version of this document includes description, frequency and timing of instrument, data that is collected, means of data collection, sources of the data, and use of each instrument. Process used in Computer Science The Department’s process for improvement and evaluation of attainment of Program Educational Objectives (PEOs) and Program Outcomes (POs) is a cyclical, continuousimprovement model consisting of three phases. Fig. 1 shows the information flow into and out of each phase. The open rectangles represent information stores, the sources of information are denoted by a square, and the rounded boxes represent the processes associated with a phase. Phase 1: Short-term Program and Curriculum Review Each semester, faculty review their course assessments and teaching evaluations. There are four main sub-committees (each representing a grouping of courses) that review whether course outcomes are met. The courses associated with each of the subcommittees are scheduled for formal assessment every other year, or as deemed necessary. Results from the review are discussed at a faculty meeting. Phase 2: Long-term Program and Curriculum Review Approximately every two to three years, the Department reviews the scheduled recommendations from Phase 1 and assessment results. Changes made at this level are significant and have broad impact on the program and, thus, require deeper analysis and discussion with the Department faculty at large. Phase 3: Educational Objectives and Outcomes Review Once every five years, the Department meets to evaluate the POs for relevance. During this evaluation, recommendations, assessment results, and national trends are reviewed and discussed by the faculty. If warranted, the POs and the PEOs are revised to meet the current needs of our constituencies. There are many resources available on websites of other universities. It is recommended that you visit the sites of programs in your discipline to avoid duplicating work. Some professional organization such as the American Psychological Association have published learning outcomes on their websites for various programs (see http://www.apa.org/ed/pcue/taskforcereport2.pdf) . Each department or program should look for and examine these resources carefully in light of the department or program’s unique nature and the mission and vision of UTEP. Clearly, simply copying the learning outcome statements from a program at another university would be a mistake.