Assessment Plan for the Computer Science Major (Revised 6/30/10) Mission Statement The mission of the Computer Science program is to prepare students as competent computer scientists in industry and academia. We expect our students to have a solid grasp of the theory of computing and the skills to produce and maintain high-quality software and computer systems. We are committed to fostering creativity and problem solving skills in all areas and applications of computing. Our goal is to make our students excel in leadership roles and as productive team members. Overview Computer Science is one of the most dynamic fields: the rate at which computing is evolving requires a constant appraisal of what is taught and how it is taught. The assessment plan is intended to gauge how well we are doing in getting our students ready for working in the industry and for advanced studies. The plan should provide constant feedback for improvement, yet it must not be a burden on the pedagogy. This assessment plan is primarily based on per-student achievement of the program goals, which are themselves a means towards satisfying our mission statement. Although, the collection of data is an ongoing process and will take place at the end of every semester, the assessment of goals will be periodic, based on when all components of a goal are available next. Besides goal assessment, evaluations for the current courses, the portfolio, and the senior and project surveys will also be assessed (qualitatively) each year. Finally, an evaluation of the applicability of our program goals and course outcomes will be included in the yearly assessment. Course Evaluation All courses taught in an academic year will be evaluated, at the end of the year, based on: 1. the course learning outcomes detailed below, 2. a short qualitative assessment that the instructor will write for each course at the end of the semester, 3. student surveys for each course, separate from the official student evaluations. These anonymous surveys will be topical and unique to each course, and will serve as a qualitative assessment tool for the program. Course Learning Outcomes Only those outcomes will be tracked (for now) that contribute to program objectives. For each outcome the student will be given a grade value between 1 and 4: 1 is unsatisfactory, 2 is minimally acceptable, 3 is satisfactory, and 4 is excellent. CPTR 105: Computer Programming I 1. Understand the problem solving process using algorithms 2. Able to implement basic programs in a programming language 3. Understanding of fundamental programming constructs: Primitive Types, Strings, Console I/O, Control Flow 4. Basic understanding of classes and objects Methods for assessment: assignments, exams questions CPTR 205: Computer Programming II 1. Object-oriented programming details: polymorphism, inheritance, packages 2. Experience writing multi-object programs for modeling real-world problems 3. Using arrays Methods for assessment: assignments, exams questions CPTR 221: Software Development 1. Familiarity with the fundamentals and the effectiveness of common software development processes; 2. Developed and delivered a software product for a real customer; 3. Communicated with a customer throughout the development process 4. Conversant with common development tools including frameworks, IDEs, debugging tools, and version control; 5. Produced effective documentation of the development process and the software product 6. Worked as a team member effectively fulfilling their role towards product development Methods for assessment: assignments, exams questions, project, presentations CPTR 308: Computer Architecture 1. 2. 3. 4. Understand the general hardware composition of a modern computer system Understand the relationship between the computer components Familiarity with a machine language Understand the transformation of information from the highest (text, images, etc.) to the lowest level (binary representation of numbers and text, memory storage) 5. Be able to describe the transformation of processing from the highest (programming language statements and programs) to the lowest level (instruction set design, hardware arithmetic) 6. Understand the tradeoffs in modern computer systems such as speed, power, and reliability Methods for assessment: assignments, exams questions CPTR 225: Database Programming 1. 2. 3. 4. Understanding of the status and applicability of databases as a data management solution Use of SQL as a programming interface for database management The design and optimization of databases with focus on application integration Use of a database management system in a functional web or desktop application Methods for assessment: assignments, exams questions, project CPTR 310: Algorithms and Data Structures 1. 2. 3. 4. 5. Demonstrate knowledge of basic data structures such as stacks, queues, and lists Develop custom data structures based on data modeling requirements Implement algorithms for searching and sorting the basic data structures Understand time complexity Understand the issues involved in optimizing algorithms Methods for assessment: assignments, exams questions CPTR 314: Operating Systems and Networks 1. Understanding of core concepts in operating systems, including memory management, processes, concurrency, and scheduling 2. Understanding of core concepts in networking: (a) the network layers and their abstractions, (b) the integration of networking modules in modern OS, (c) and the layout of small to large scale networks 3. Programmatic interaction with the lower layers of an OS kernel 4. Understanding of networked application models, such as client-server and cloud computing 5. Building networked applications such as chat clients and distributed software Methods for assessment: assignments, exams questions Program Goals 1. 2. 3. 4. Experience in designing and developing software Understanding of the architecture and organization of computer systems Understanding of the layers of information and processing abstractions in computing Understanding of theoretical foundations of CS, and the mathematics necessary for scientific computing 5. Ability to read and write programs in multiple languages, and be proficient in one language 6. Experience a liberal education in humanities and social sciences 7. Ability to communicate ideas and work as a team member Periodic evaluation of each goal will be determined considering: the (required) course outcomes that help achieve the goal the assessment methods used for course outcomes Goal 1: Experience in designing and developing software Description: A major expectation of any CS graduate is their ability to produce effective software programs both individually and as a team member. The graduate must be comfortable with the prevalent tools and methodologies in the software industry and understand the complexities of the production and management lifecycles of a mature software product. be familiar with the major development processes; developed at least one complete software product with a prevalent process; conversant with common development tools including frameworks, IDEs, debugging tools, and version control; familiar with integrating an application’s data requirements using a database. Course Outcomes: 222.1, 222.2, 222.4, 222.5; 312.4 Goal 2: Understanding of the organization and architecture of computer systems Description: Computer Systems have been aggregating incremental changes in their architecture and organization in the last four decades, while keeping the basic fundamentals more or less the same. This requires a CS graduate to (a) have a thorough understanding of the foundation topics, such as von-Neumann architecture and Ethernet communication, and (b) be informed of the current and upcoming technologies, such as multi-core CPUs, and cloud computing. Such knowledge requires a grounding in the hardware as it supports the systems software, as well as the software that supports the expected operational expectations from a computer system. Course Outcomes: 308.1, 308.4, 308.5, 312.1, 314.1, 314.2, 314.5 Goal 3: Understanding of the layers of information and processing abstractions in computing Description: A recurring theme in computer science is that of abstractions. The use of abstractions as the basis of our current model of computing is central to an understanding of the overall field at all levels. In programming, for example, the use of increasingly complex abstractions such as statements, functions, classes, libraries, and frameworks not only makes the practitioner a good software developer, but also able to detect when and how these abstractions fail in practice. Other examples of abstractions are information, processing, communication, and design pattern abstractions. Course Outcomes: 205.1, 308.4, 308.5, 312.1, 310.1, 314.3 Goal 4: Understanding of the theoretical foundations of CS, and the mathematics necessary for scientific computing Description: Historically and at heart, computer science is founded on theoretical ideas traditionally expressed by using some mathematical language. This demands that the CS graduate gains a solid understanding of the mathematics that is either related directly to computing, or for gaining a maturity expected in higher-level computing courses. Course Outcomes: all required math courses (MATH 121, MATH 130, MATH 251), 205.1, 221.1, 312.3, 310.4 Goal 5: Ability to read and write program in multiple languages, and be proficient in one language. Description: A programming language serves as the main tool for practical exploration of CS topics as well as for producing software artifacts. As such, it is de rigueur for the student to have a thorough knowledge and experience of at least one language. One the other hand, exposure to solely one language can prove severely limiting to the graduate’s expressivity and marketability. Also, learning the syntax and semantics of a language does not equate to being a master of its usage. Therefore, the student must gain an understanding to the fundamentals of programming language features. Sustained exposure to the programming tool ecosystem, especially debuggers and profiling tools is becoming increasingly essential in the field. Course outcomes: 105.1, 105.2, 105.3, 205.1, 205.2, 221.2, 310.3, and relevant outcomes from electives (CPTR 222, 324, 410, 415) Goal 6: Experience a liberal education in humanities and social sciences Description: a CS graduate is expected to have a well-rounded liberal arts background in order to keep up with the current issues and events. The student must be able to evaluate the importance and effect of technology in the society. An exposure to a diverse education gives the student a perspective in the ethical and moral issues that they may face in their career. Course outcomes: this goal will be achieved by the liberal arts courses that the student is required to take at Manchester College. Goal 7: Ability to communicate ideas and work as a team member Description: Communication is crucial for a CS graduate: the expression of ideas goes beyond writing source code for software programs. The ability to write documentation, present ideas to an audience, and describe efforts and processes in clear and concise language is necessary for working in a professional environment as well as in higher studies. Course Outcomes: 221.3, 221.5, 221.6, 475/499 presentation and paper. Portfolio The intention is more towards providing the student a collection of achievements that can serve as an example of their capabilities and experience, than as a direct assessment tool. Still, such an artifact can be useful in providing an overall view of the student’s achievements for assessment. It will best serve as a tool for qualitative assessment. One major program assignment from 105 One major program assignment from 205 Project Documentation & Output from 221 Project from 312 One major program assignment from 310 Internship/Senior Project paper + Product Output Internship/Senior Project presentation + grade (see rubrics in Appendices B and C) Any other research or project output Portfolio can be saved on user accounts in the Linux lab. Faculty will review portfolio every year/semester. Student Surveys Course Survey: as discussed in course evaluation above. Internship survey: although students experience close mentorship with faculty in senior projects, internships can be difficult to evaluate specially for assessment purposes. A simple survey’s completion will be a requirement towards SCE. (See Appendix A.) Senior/Exit Survey: every major student will be expected to complete an exit survey in their last semester. This will be in the form of a questionnaire with the intent to gather student’s impression about the program and their experience. Its completion will be a requirement towards SCE. (See Appendix A.) Appendix A: Internship and Exit Surveys Internship Survey 1. Resources used to obtain internship? 2. Employer details. 3. Skills learnt. 4. CS courses that helped prepare. 5. Rate the following for the quality of experience: a. interaction with mentor b. interaction with employer c. team work d. match between interests and work e. overall experience 6. Comments. Senior Exit Survey Personal 1. Currently employed? 2. Job prospects? 3. Graduate school? Specific Program Goal Impressions (graded 1 - 4) 1. I am experienced in designing and developing software. 2. I have an understanding of the architecture and organization of computer systems. 3. I have an understanding of the layers of information and processing abstractions in computing. 4. I have an understanding of theoretical foundations of CS, and the mathematics necessary for scientific computing. 5. I am able to read and write programs in multiple languages, and be proficient in one language. 6. I am able to communicate ideas and work as a team member. General Program Impressions (graded 1 - 4) 1. I found the courses in the program intellectually stimulating. 2. The teaching was adequate for me to succeed in the courses. 3. Which was the most useful course? 4. Which was the least useful course? 5. The theoretical knowledge was adequate. 6. The practical experience was adequate. 7. Suggestions for improving the CS program. Appendix B: Evaluation Rubric for Internship Presentation Written and Oral Communication 3 Understands key Applies computer Can effectively work Can effectively explain computer science concepts to within a team to set computer science science design solutions using goals, divide concepts, algorithms and concepts; can appropriate steps in a responsibilities for implementation issues in explain their logical sequence; various portions of oral and written form, at a significance in constructs clear, work product, level appropriate to the the broad correct and efficient communicate and audience. context of algorithms. coordinate between computing and team members as can provide project progresses and relevant resolve any examples. disagreements. Conceptual Understanding 2 Knows basic computer science concepts; can explain how they relate to specific examples. Technical Skills Teamwork Applies computer Is able to work science concepts to effectively once design solutions using assignment is given, appropriate steps in a communicates with reasonably logical other team members sequence; constructs when needed, correct algorithms contributes to although may be decision-making. lacking in clarity or efficiency. Can effectively explain computer science concepts, simple algorithms and basic implementation issues in both oral and written form. 1 Knows basic Has difficulty computer applying concepts to science concepts arrive at a solution; but has algorithms illustrate difficulty basic principles but relating them to may be confusing or specific occasionally produce examples. incorrect results. Can complete assignment when given but does not always effectively coordinate work process and product with others. Can explain basic computer science concepts and simple algorithms; has little ability to explain basic implementation issues. 0 Does not know Cannot construct basic computer solutions using basic science concepts building blocks of required to solve computer science. common types of problems. Has significant difficulty working with others, coordinating efforts or communicating with a project team. Cannot effectively communicate basic computer science concepts or issues. 3 = High Pass; 2 = Pass; 1= Low Pass; 0 = Fail; NA = Not Applicable/Observed Evaluator: ________________________________ Date: ________________ Material Evaluated: _________________________________________ Comments: Appendix C: Evaluation Rubric for Senior Project Presentation Program Teamwork Implementation Understands key Applies computer Implements algorithms Can effectively work computer science concepts to in an effective and within a team to set science design solutions using maintainable fashion, goals, divide concepts; can appropriate steps in a with clear user responsibilities for explain their logical sequence; interface and program various portions of significance in constructs clear, documentation; work product, the broad context correct and efficient displays a high communicate and of computing algorithms. command of coordinate between and can provide programming team members as relevant languages and project progresses and examples. software development resolve any tools (e.g., editors, disagreements. compilers and debuggers). Knows basic Applies computer Displays sufficient Is able to work computer science concepts to command of effectively once science design solutions using programming assignment is given, concepts; can appropriate steps in a languages and communicates with explain how they reasonably logical software development other team members relate to specific sequence; constructs tools (e.g., editors, when needed, examples. correct algorithms compilers and contributes to although may be debuggers) to produce decision-making. lacking in clarity or working programs; efficiency. programs lack some clarity in logic, user interface or documentation. Knows basic Has difficulty applying Knows basic syntax of Can complete computer concepts to arrive at a languages and assignment when science concepts solution; algorithms rudiments of software given but does not but has difficulty illustrate basic development tools but always effectively relating them to principles but may be has difficulty coordinate work specific confusing or identifying logical process and product examples. occasionally produce errors or using tools to with others. incorrect results. detect errors; programs are sometimes incomplete or contain non-trivial bugs. Does not know Cannot construct Has not mastered basic Has significant basic computer solutions using basic syntax and use of difficulty working science concepts building blocks of software tools; with others, required to solve computer science. programs often do not coordinating efforts or common types of provide desired result; communicating with a problems. is not able to address project team. program deficiencies. Conceptual Understanding 3 2 1 0 3 = High Pass; Technical Skills 2 = Pass; 1= Low Pass; Can effectively explain computer science concepts, simple algorithms and basic implementation issues in both oral and written form. Can explain basic computer science concepts and simple algorithms; has little ability to explain basic implementation issues. Cannot effectively communicate basic computer science concepts or issues. 0 = Fail; NA = Not Applicable/Observed Evaluator: ________________________________ Date: ________________ Material Evaluated: _________________________________________ Comments: Written and Oral Communication Can effectively explain computer science concepts, algorithms and implementation issues in oral and written form, at a level appropriate to the audience.