Computer Science Department’s Student Outcomes Assessment Plan The original SOA plan adopted in 1990 was portfolio­based. Collection of information for the portfolios was sporadic and assessment of the portfolios provided little useful information for program improvement. In 1997 the current SOA plan was designed and adopted. It is aimed at providing the departmental curricular review process useful information without overburdening the faculty with data collection. The remaining of this section discuss the redesigned SOA plan. A. Benchmarks for Student Outcomes Assessment. We classify the expected outcomes and competencies into two categories: computer science content (skills and knowledge) and general skills. The general skills we deem necessary for computer science majors are related to oral­communication skills, written­ communication skills, and the ability to work in teams. The specific outcomes and competencies for these general skills are: Outcome 1. Students shall possess the capability to communicate orally and in written form according to the standards of the discipline. Competency 1.1. Be able to make an oral presentation to a group of peers that successfully communicates about computer science or the application of computer science. Competency 1.2. Be able to create a written document that effectively communicates about computer science or the application of computer science. Outcome 2. Students shall possess the capability to work effectively in a team. Competency 2.1. Be able to meet deadlines determined by the team. Competency 2.2. Be able to make a substantial contribution to the teams effort. We assess a student's computer science skills and knowledge learned at a relatively detailed level. The goal is to perform a per­course evaluation for all required courses in each of the undergraduate programs. To avoid overburdening faculty, per­course assessments utilize anonymous, end­of­semester surveys about student understanding of skills and knowledge related to the course. Appendix A contains a sample survey for the Introduction to Computing (810:051) which was the first course in the major in 1997. A web­based assessment tool, called PAT (Program Assessment Tool), was developed to help manage these end­of­semester, per­course surveys. PAT was never fulling utilized. Faculty found the effort to populate the list of topics needed to administer the surveys not worth the little information that the self­assessments provided. Currently, we are discussing alternative to PAT, such as senior exit examinations.
B. Procedures. The current SOA plan uses five difference assessment instruments: i) Declaration Survey The first assessment occurs when a student declares a major. At declaration time students are required to complete a questionnaire (see Appendix A) addressing their reasons for selecting the major, career goals, and previous computer science background. Until this questionnaire is completed, their "Declaration of Major Form" is not forwarded to the Registrar. Additional materials such as ACT scores, high school GPA, high school rank, and a current transcript are collected at this time. The purpose of this stage of the assessment is to aid the computer science faculty in understanding the background of our students and perceptions of the field. Since computer science has a high attrition rate, this information might enable us to find some success indicators for our programs. ii) PAT (Program Assessment Tool) The primary mechanism for conducting assessment of the computer science knowledge and skills is the software package PAT (Program Assessment Tool). PAT assesses the effectiveness of individual classes in conveying the computer science topics (knowledge and skills) to the level of understanding expected by the department. Each semester during the last week of classes, students in each program take a blind (anonymous) survey that rates their perceived understanding of each topic using a low, medium­low, medium, meduim­high, and high scale. This is done for each required class for which a student is enrolled. PAT administers the surveys, tabulates the results, compares the results against the departmental expectations for each topic, and generates necessary reports. Currently, PAT is not fully implemented, but previously prototype versions of PAT was tried and found useful. iii) "Senior Sequence" Each computer science student is required to take a two semester sequence of courses usually in their senior year. A major focus of these courses is a large, team programming project that involves written documentation and culminates with a final oral presentation. Instructors of these senior sequences are responsible for assessing the written­communication, oral­presentation, and teamwork skills of individual students. These instructors complete summary reports (see Appendix A) indicating for each student their skill levels. iv) Exit Surveys Each graduating senior is asked to complete an exit survey (see Appendix A). Since most graduating seniors already have had several interviews, they are able to supply valuable feedback about employer expectations and interests. Additionally, this allows the department to track its graduating seniors better since some do not utilize the Placement center. v) Alumni Surveys
Every two years (starting Summer 1999) a survey of recent graduates is conducted. The questionnaire (see Appendix A) is designed to assess the strengthens and weaknesses in the preparation received during the program.
Appendix A Student Outcome and Assessment Forms Computer Science Department
Computer Science Major Declaration Survey Name ______________________________________ Date _____________________________ Degree: BS in Computer Science ____ BA in Computer Science ____ BA in Computer Information Systems____ Background: What computer science or computer related courses did you take before coming to UNI.? Where did you take the courses? What computer science cources did you take at UNI? Did any of these courses influence your decision to become a computer science major? Which courses? Objectives Essay: Write a short essay on your career objectives on the back of this page. Address the following questions and include anything else you consider appropriate. 1. Why did you choose computer science? 2. What do you expect to do directly after earning your degree? 3. What do you expect to be doing five years after earning your degree? Note: Your major declaration will not be forwared to the registrar's office until you return this completed form.
Sample PAT Course Survey OUTCOMES ASSESSMENT SURVEY INTRODUCTION TO COMPUTING 810:051 SPRING 1999 The goal of the outcomes assessment survey is to judge the effectiveness of the course and NOT to judge the students. This is a blind survey because the students taking the survey cannot be identified. General Information: Yes No ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ Did you have a programming course before this course? Do you have a computer at home? If you have a computer at home, did you find it helpful in taking this course? Do you plan to take Data Structures 810:052? Do you plan on repeating this course? Are you a traditional student? (Went directly from high school to your current degree program at UNI) Are you a non­traditional student? (Have worked or received another degree before entering this program at UNI) Do you have any suggestions to future students who take this course? Do you have any suggestions that would improve your learning experience in this course? The section on the back of this survey is a list of topics relating to this course. It is not expected that you have in­ depth knowledge of all of the topics. Rank your understanding of the topics according to the descriptions of None, Low, Medium, Med/High, and High at the top of the page on the back of this survey.
None: Low: Medium: Med/High: High Topic Abstractions Variable declaration Type declaration and use Subtype declarations and use Enumerated data types Array declaration and use Array index declaration and use Records declaration and use Hierarchical record declaration and use Arrays of records declaration and use Exception handlers Block Structure Boolean expressions Conditional Statements ­ if family Looping statements Declaration and use of procedures Declaration and use of functions Parameter passing Scope Input from the keyboard Output and formatting to the screen Input and output with text files Input and output with binary files Use of existing packages
Have no knowledge of the topic. Have heard of the topic and have some idea about the general area. Understand the topic in theory but have not used it in programming or problem solving. Have a theoretical knowledge of the topic and have had use it in programming or problem solving. Have mastered the topic. Can easily use the topic in programming and problem solving. None Low Med Med/High High Creation of packages Precedence rules for operations Languages Ada EBNF Skills Time management skills Writing problem solutions as Multi­Level Diagrams Writing problem solutions as Module Structure Charts Writing problem solutions as algorithms Writing program comments as a design approach Compiling and running programs Tools NT operating system Internet e­mail Programming environment ­ pcGrasp Compiler ­ GNAT
Written, Oral, and Team Skills Report Form Outcomes Assessment Written, Oral, and Team Skills Individual Evaluation Form Course _______________ Date ________________ Student's Name Written Oral Team Acceptable written work is defined as grammatically correct work that describes a technical problem so that it can be understood by the target audience. If a technical format or length limit is assigned, the written product should adhere to the guidelines. Acceptable oral presentation in defined as a presentation that is technically correct, understandable by the target audience, and within allotted presentation and time parameters. Acceptable team work is defined as doing a fair share of the work, completing tasks on time, and approaching the team with a positive attitude. Ratings: L = Less than acceptable A = Acceptable G = Good ­­ better than acceptable E = Excellent
Department of Computer Science Outcomes Assessment Written, Oral, and Team Skills Summary Form Project Management course Date ________________ Low Acceptable Good Excellent Acceptable Good Excellent Acceptable Good Excellent Written Oral Team Distributed Computing course Low Written Oral Team Intelligent Systems course Low Written Oral Team Acceptable written work is defined as grammatically correct work that describes a technical problem so that it can be understood by the target audience. If a technical format or length limit is assigned, the written product should adhere to the guidelines. Acceptable oral presentation in defined as a presentation that is technically correct, understandable by the target audience, and within allotted presentation and time parameters. Acceptable team work is defined as doing a fair share of the work, completing tasks on time, and approaching the team with a positive attitude.
Computer Science Senior Exit Survey This survey is part of the Computer Science Department's Student­Outcome­Assessment procedure and your responses are intended to be used to improve the Computer Science programs at UNI. Please return to: Student Outcome Assessment Committee, Computer Science Department, University of Northern Iowa, Cedar Falls, Iowa 50614­0507. Additionally, please complete the Placement & Career Survey at http://www.uni.edu/placement/followup/followup.html. Name ______________________________________ Graduation Date ____________________ Degree Program: ____ BA in Computer Science; Emphasis ______________________ ____ BS in Computer Science; Emphasis ______________________ ____ BA in Computer Information Systems Job Placement: Do you have a position? ______ If so, name of employer_____________________________ Describe the position you have or are looking for. When you were interviewing, what were the skills that you learned in your program that were appreciated? When you were interviewing, what skills were you missing?
Program Review: What skills or knowledge do you believe are missing from your curriculum? Are there areas of the curriculum that need more or less emphasis? Cooperative Learning or Experience: Did you participate in cooperative learning, internship, or computer related work while a student at UNI? ______________ If so, describe the position. What were the good points or bad points of the position?
Computer Science Alumni Survey As one mechanism to evaluate the Computer Science program at UNI and to gather information to use in curriculum revision, please complete and return the following short survey. Please feel free to add additional comments on the back of this sheet. Rate the following on a scale from 1 (poor) to 5 (excellent). How well did your UNI Computer Science education prepare you: ____ to learn to work in your current hardware/operating system environment? ____ to learn to work in your current networking environment? ____ to use or learn to use the software tools that you are currently using? ____ to work in a team? ____ to give oral presentations? ____ to give written presentations? ____ for your first job (or graduate school)? What do you suggest be added to the UNI Computer Science curriculum to improve our students' transition into the work place? Name of your current employer or graduate school. (Optional) Your name, current address, e­mail address, and telephone number. (Optional)