SJSU Annual Program Assessment Form
Academic Year 2014-2015
Department: Graduate Extended Studies
Program: Off Campus Program (MS in Software Engineering with Emphasis in Cloud Computing and
Virtualization)
College: Engineering
Website: http://engr-extendedstudies.sjsu.edu/
x Check here if your website addresses the University Learning Goals.
http://engr-extendedstudies.sjsu.edu/assessment/
Program Accreditation (if any): NA
Contact Person and Email: Lee Chang, lee.chang@sjsu.edu
Date of Report: June 1, 2015
Part A
1. List of Program Learning Outcomes (PLOs)
List of PLOs
(a) Be able to demonstrate an understanding of advanced knowledge of the practice of software
engineering, from vision to analysis, design, validation and deployment.
(b) Be able to tackle complex engineering problems and tasks, using contemporary engineering
principles, methodologies and tools.
(c) Be able to demonstrate leadership and the ability to participate in teamwork in an environment
with different disciplines of engineering, science and business.
(d) Be aware of ethical, economic and environmental implications of their work, as appropriate.
(e) Be able to advance successfully in the engineering profession, and sustain a process of life-long
learning in engineer or other professional areas.
(f) Be able to communicate effectively, in both oral and written forms.
Decision on PLO content
The PLOs were proposed and approved by the Computer Engineering Department faculty members
and the Department Advisory Council.
Criteria for assessing levels of mastery
Each course is designed to achieve a set of Course Learning Objectives (CLOs). For each course, the
assessment of these CLOs is described in a Course Assessment Journal (CAJ). By aggregating the
levels of mastery of the CLOs in relationship to the corresponding PLOs of all assessed courses, the
Program Assessment Coordinator decides the overall assessing levels of the mastery of PLOs.
2. Map of PLOs to University Learning Goals (ULGs)
Specialized
Broad
Intellectual
PLO/ULG Knowledge
Integrative
Skills
Applied
Knowledge
Social and
Global
1
Knowledge
(a)
X
(b)
(c)
(d)
X
X
X
(e)
X
(f)
X
Responsibilities
X
X
X
X
X
X
X
X
X
X
X
X
3. Alignment – Matrix of PLOs to Courses
PLO
200W
(a)
(b)
(c)
(d)
(e)
(f)
202
X
X
203/
277
X
X
235
272
X
X
X
X
Course
282
281
X
X
284/
298
X
X
283
X
X
X
X
295A
295B
X
X
X
X
X
X
X
X
X
X
X
X
4. Planning – Assessment Schedule
This is a cohort-style program. The students in the same cohort should take courses together. The
instructor of each course is required to generate a CAJ. At the end of each academic year, the
Program Assessment Coordinator generates a Program Assessment Report based on the CAJs and
other assessment activities, e.g., faculty and student surveys, informal discussion, etc. CMPE 203
will not be assessed after Spring 2015 since it will not be a degree core course effective Fall 2014.
Semester
Course
200W
202
235
272
281
282
283
203/277
284/297
295A
295B
S13
3
F13
S14
4
F14
3
3
4
4
4
4
3
S15
5
5
4
2
3
(Cells contain cohort numbers.)
S16
6
5
5
5
4
3
3
3
F15
6
5
7
7
7
6
6
5
5
5
4
S17
7
6
6
6
5
4
4
4
F16
7
6
6
6
5
6
2
5. Student Experience
PLOs and ULGs are posted at the website (http://msse.sjsu.edu/). For the master project classes
(CMPE 295A/B), PLOs are included in the syllabi and students provide feedback to PLOs. Students
provide their feedback to PLOs through the discussions with the program coordinator. The feedback
will be considered during the next discussion session about the enhancement of PLOs.
Part C
6. Closing the Loop/Recommended Actions
Action 1:
Status 1:
Action 2:
Status 2:
Action 3:
Status 3:
Action 4:
Status 4:
Action 5:
Status 5:
Include the course delivery of CMPE 272 by identifying emerging software design
paradigms and increasing the coverage of various software architecture models and
technologies to design enterprise software systems.
Continue to include emerging software design paradigms and architecture models
Stop formal assessment of CMPE 203 after it is not a degree core course anymore after
Fall 2014.
Completed.
Propose to improve the lab sessions of CMPE 283 by ensuring the equipment setup with
proper configuration parameters the machines, increasing security of accessibility,
installing latest software, and software fixes.
Continue to ensure the updates and operations of the required lab environments
Propose the instructor(s) of CMPE 283 to try the project tasks first before assigning the
tasks to students in order to reduce uncertainties.
It is the current practice.
Propose the instructor(s) of CMPE 284 to devote more lectures on networking
protocols, give students more assignments to reinforce the concepts, and adjust the
difficulty level of exam questions
CMPE 284 will be not offered for this program
7. Assessment Data
Each course in this program is required to generate a Course Assessment Journal (CAJ). PLOs are
assessed by at least one of these courses. A set of Course Learning Objectives (CLOs) has been
assigned to each course. The course instructor(s) generates the CAJ. The data presented in CARs is
the main contributor to address the achievement of PLOs. The CAJs collectively address
achievement of PLOs and also recommend actions. A sample CAJ is attached as appendix.
8. Analysis
Based on the assessment data and summary of the CAJs, all 6 PLOs have been achieved. As shown
in Section 6, all actions recommended during last assessment cycle have been implemented. The
3
changes proposed during this assessment cycle are listed in Section 9. These proposed changes will
be evaluated and prioritized for the implementation in the next assessment cycle.
9. Proposed changes and goals (if any)
Proposal 1: Combine two emphases (Cloud Computing & Virtualization Emphasis and Mobile
Application Emphasis) into one (Cloud & Mobile Computing).
Proposal 2: Replace CMPE 203 by CMPE 277 (Smartphone Application Development) and replace
CMPE 284 by CMPE 297 (Special Topics in Computer Engineering – Emerging Mobile
Technologies)
Proposal 3: Schedule CMPE 272 as the first class. Strengthen the coverage of operating systems,
and database design in CMPE 272.
Proposal 4: Add more homework assignments to help students to design and develop mobile-based
software systems and commerce applications. [CMPE 235, Fall 2014]
Proposal 5: At the end of the course, the students were able to build solutions with software
virtualization components, using their own laptops. The WiFi network in the facilities
was adequate for general web browsing, but collapsed under the load during lab day.
Proposed action: Increase WiFi capacity in the classroom (at least temporarily during lab
day). [CMPE 283, Fall 2014]
Proposal 6: The students were able to perform hands-on experiments using desktop based
products, but would have preferred more enterprise class virtualization products on
university servers. Proposed change: Upgrade lab computers. [CMPE 283, Fall 2014]
4
Appendix CMPE 235 Course Assessment Journal of Spring 2014
MS in Software Engineering Program Outcomes (PO)
Description
PO 1
Be able to demonstrate an understanding of advanced knowledge of the practice of software
engineering, from vision to analysis, design, validation and deployment.
PO 2
Be able to tackle complex engineering problems and tasks, using contemporary engineering
principles, methodologies and tools.
PO 3
Be able to demonstrate leadership and the ability to participate in teamwork in an environment
with different disciplines of engineering, science and business.
PO 4
Be aware of ethical, economic and environmental implications of their work, as appropriate.
PO 5
Be able to advance successfully in the engineering profession, and sustain a process of life-long
learning in engineer or other professional areas.
PO 6
Be able to communicate effectively, in both oral and written forms.
Course Learning Objectives (CLO)
Description
CLO 1
Understand mobile computing and mobile cloud concepts, including objectives, markets and
business perspectives, advantages, features, and evolution history.
CLO 2
Understand how to design and develop mobile-based software systems and commerce
applications.
CLO 3
Understand the issues and solutions in engineering mobile-based software and commerce
applications as well as mobile clouds.
CLO 4
Utilize and apply modern technologies and solutions to design and develop mobile-based
computing applications and mobile cloud components and services.
CLO 5
Work effectively in a team environment for a selected project
Course Learning Objectives Support Program Outcomes
PO 1
PO 2
PO 3
PO 4
PO 5
PO 6
5
CLO 1
X
CLO 2
X
CLO 3
X
CLO 4
X
CLO 5
X
Two different approaches are used to assess the student performance based on course learning
objectives. They are: a) direct measurement, and b) indirect measurement.
(a) Direct Measurement: For each CLO, please set the performance criteria. This is the percentage
at which a student demonstrates satisfactory proficiency. The performance criteria should not
be less than 75%. The direct assessment (performance) data was collected from homework
assignments, lab assignments, exam questions, etc.
CLO 1
CLO 2
Performance criteria set at 75%, minimum percentage of students who should reach
performance criteria: 75%
- Homework #1: 90% of students scored above 75%
- Final Exam: 100% students scored above 75%
Performance criteria set at 75%, minimum percentage of students who should reach
performance criteria: 75%
-
CLO 3
CLO 4
CLO 5
Class Group Project: 100% of students scored above 75%
- Final Exam: 95% of students scored above 75%
Performance criteria set at 75%, minimum percentage of students who should reach
performance criteria: 75%
- Class Group Project: 100% of students scored above 75%
- Lab #1: 100% of students scored above 75%
- Lab #2: 100% of students scored above 75%
- Final Exam: 95% of students scored at or above 75%
Performance criteria set at 75%, minimum percentage of students who should reach
performance criteria: 75%
- Class Group Project: 100% of students scored above 75%
- Homework #1: 100% of students scored above 75%
- Final Exam: 100% of students scored above 75%
Performance criteria set at 75%, minimum percentage of students who should reach
performance criteria: 75%
6
-
Class Group Project: 100% of students scored above 75%
Project presentation: 100% of students scored above 75%
2. Indirect Measurement: This section contains two parts: a) CLO survey results, b) qualitative
feedback (e.g. student focus groups).
a) CLO Results
In the survey, students were asked to give a letter (A to E) to measure how important each CLO was
and whether they felt they had learned and achieved the CLO. Please convert the A to E
measurement into numeric scores of 5 to 1 (respectively). If there is a difference of over 1 pt,
between Part I and Part II, please address in Section 3 (Findings and Analysis)
Part I – importance of course topic
Part II – portion learned and achieved
CLO 1
4.5
4.1875
CLO 2
4.625
4.125
CLO 3
4.75
4.125
CLO 4
4.5
4.125
CLO 5
4.625
4.5625
b) Qualitative Feedback
Informal group discussion was held with students four times during the 8-week course. Topics
discussed included: course material, pace of course, teaching effectiveness, etc. Students generally
felt that 8 weeks was fast and that workload was very large. Majority of students found material to
be challenging and relevant to current job or could use knowledge for future projects. A good
portion of students appeared to have inadequate preparation in math (more specifically,
trigonometry, calculus, vector algebra and complex number manipulations). As a result, the course
objective has been scaled down so students would not fall behind in the short 8-week course.
7
3. Findings and Analysis: Provide an analysis of the data collected for Direct and Indirect Measurement.
a) Direct Measurement Findings and Analysis
CLO 1
No significant findings for this CLO.
CLO 2
In Test 2, Questions 3 & 7 indicated that only 60% of students showed proficiency. Many
students also commented that they felt this topic was rushed. They did not feel comfortable
with the area prior to the exam and that the HW assignments were easier that the test
questions.
b) Indirect Measurement Findings and Analysis
CLO 1
This CLO was rated as Very Important to students (4.8), but the score of 3.7 indicates that they
did not learn or achieve this topic adequately. This may be due to the rushed nature of the
course.
CLO 2
No significant findings for this CLO.
c) Other Findings and Analysis (e.g. Discrepancies between a) & b) above)
CLO 1
The direct measure found that students performed well in this area, but the Indirect findings
suggest that students did not feel they learned/achieved in this area. We did not go into very
much depth in this topic. Students may have learned what was taught very well, but wished to
learn more, but we did not have enough time.
CLO 2
Students only felt this area was moderately important, and so perhaps they did not put as
much effort into testing well in this area, explaining the failure to achieve proficiency for some.
The CLO survey results do show that they are satisfied with the amount that they
learned/achieved.
4. Actions: What actions were planned and/or implemented to address the findings from collected
data? Actions taken can include revising course content, test revision, projects
CLO 2 Planned
Extra homework problems assigned for this area and post solutions onto website. Offer
extra office hours for to help students master topic
8
CLO 2
Will be in effect for the next time this course is offered.
Implementation
9