SJSU Annual Program Assessment Form
Academic Year 2014-2015
Department: Industrial & Systems Engineering
Program: M. S. Industrial & systems Engineering
College: Engineering
Website: http://ise.sjsu.edu/
X- Check here if your website addresses the University Learning Goals.
Program Accreditation (if any): None
Contact Person and Email: Minnie H. Patel, minnie.patel@sjsu.edu
Date of Report: May 31, 2015
Part A
1. List of Program Learning Outcomes (PLOs)
The student outcomes of the master’s degree in ISE are:
1.
2.
3.
4.
5.
6.
7.
8.
9.
Student will be able to function effectively and provide leadership within an organization.
Student will be able to facilitate and participate in teams.
Student will be able to understand organizational processes and behaviors.
Student will have knowledge of methodological and computational skills with which to operate effectively
Student will be able to collect, analyze, and interpret data
Student will be able to approach unstructured problems and synthesize and design solutions for this problem
Student will be able to evaluate the impact of these solutions in the broader context of the organization and society
Student will be able to effectively present and sell solutions in the form of written, oral and electronic media
Student will be able to accomplish life-long growth within the field of profession of ISE
6. Able to approach and solve
unstructured problems
7. Evaluate the impact of
solutions in broader context
8. Effectively present and sell
solutions
9. Life-long growth within the
ISE field
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Social and Global Responsibilities
Applied Knowledge
X
Intellectual Skills
Broad Integrative knowledge
PLO/ULG
1. Function effectively and
provide leadership
2. Facilitate and participate in
teams
3. Understand organizational
processes and behaviors
4. Knowledge of methodological
and computational skills
5. Collect, analyze, and interpret
data
Specialized knowledge
2. Map of PLOs to University Learning Goals (ULGs)
X
X
X
X
X
X
X
3. Alignment – Matrix of PLOs to Courses
Matrix mapping of course topics to Program Learning Outcomes
Table 3.2 – ISE Program – Outcome Mapping Matrix
Outcome Mapping Matrix –
Program
Outcome:
1
2
3
4
5
6
X
X
X
X
7
8
9
Required Courses (Engineering Core)
ISE 130
ISE 140
X
X
X
X
X
X
X
ISE 167
X
X
X
X
X
X
X
X
X
X
ISE 200
X
ISE 230
X
X
ISE 235
X
X
X
X
X
X
X
X
X
X
Specialty Area 1 : Production and Quality Assurance (Four out of Six Courses)
ISE 202
X
X
ISE 241
X
X
X
X
X
X
X
X
X
X
X
ISE 245
X
X
X
X
ISE 250
X
X
X
X
X
X
X
X
X
ISE 251
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
ISE 265
Specialty Area 2: Supply Chain Engineering (Four out of Seven Courses)
ISE 245
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
1
2
3
4
5
6
7
8
9
X
X
X
X
X
X
X
X
X
REQUIRED
ISE 241
X
REQUIRED
ISE 247
Program
Outcome:
ISE 250
ISE 251
X
X
X
X
X
X
ISE 265
X
X
X
X
X
ISE 270
X
X
X
X
X
X
X
X
X
X
X
Specialty Area 3: System and Information and Modeling (Four out of Six Courses)
ISE 222
X
ISE 241
X
ISE 242
X
X
X
X
X
ISE 245
X
X
X
ISE 265
X
X
X
ISE 270
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Specialty Area 4: Human Factors (Four out of Six Courses)
ISE 210
X
X
X
X
X
X
X
REQUIRED
ISE 202
X
X
X
ISE 212
X
X
X
X
X
X
ISE 215
X
X
X
X
X
X
ISE 217
X
X
X
X
X
X
ISE 219
X
X
X
X
X
X
Specialty Area 5: Service Systems Engineering (Four out of Six Courses)
ISE 242
X
X
X
X
X
X
X
X
X
X
X
X
1
2
3
4
5
6
7
8
9
X
X
X
X
X
X
X
X
X
X
X
X
X
X
REQUIRED
BUS 297D
REQUIRED
Program
Outcome:
ISE 265
ISE 250
X
ISE 222
ISE 270
X
X
X
X
X
X
X
X
X
X
X
Capstone Courses
ISE 298
X
X
X
X
X
X
X
X
X
ISE 299
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
ISE 245
X
X
X
X
X
X
X
ISE 247
X
X
X
X
X
X
X
Elective Courses
ISE 202
ISE 251
X
+ Skill level 1 or 2 in Bloom’s
 Skills relevant but not presently assessed
Taxonomy
++
X
Skill level 3 or 4 in Bloom’s Taxonomy
+++
Skill level 5 or 6 in Bloom’s Taxonomy
The Outcome Mapping Matrix in Table 3.2 above indicates across the ISE curriculum, each outcome is addressed many
times at all levels of Bloom’s Taxonomy. The table also points out the contributions of the Engineering Core and Technical
Writing course to the achievement of Student Outcomes
4. Planning – Assessment Schedule
We assess all the program learning outcomes every two years. The first time these outcomes were assessed was in Fall
2012 and then in Spring 2013. Our assessment cycle for program learning outcomes is two-year long, with the first year
consisting of collection and analysis of data and the second year of the cycle consisting of implementation of the
recommendations based on the analysis results obtained from the previous year of the cycle. See assessment schedule
below.
Data
F 12
S13
F13
S14
F 14
S 15
F15
S16
F16
S17
Collection
X
X
X
X
X
X
Analysis
X
X
X
X
X
X
Recommendations
X
X
X
X
X
X
Implementation
X
X
X
X
Performance measure: 80% of the students score 80% or above.
Interpretation of the performance measure is as follows: The instructors use the holistic rubric included in the Appendix of this
document to grade students’ performance. One or more criteria of the rubric may be used, depending on the type of assignment
and the requirement of the performance criterion that is being evaluated. Finally, students’ performance is scaled and converted
to a percentage. A score of 80% reflects meeting expectations. A description of ‘meets expectation’ for each criterion is given
in the corresponding row under the ‘meets expectations’ column of the rubric.
Table 4.1 Program Learning Outcome Assessment Schedule
MS-ISE Outcome
Performance
Course
Criteria
1
Function
effectively and
provide
Develop a lean solution for
an organization to improve
productivity
ISE251
Fall
X
Spring
leadership within
organization.
2
3
4
5
6
Facilitate and
participate in
teams.
Understand
organizational
processes and
behaviors.
Collect, analyze,
and interpret
data.
Approach
unstructured
problems and
synthesize and
design solutions
for these
problems.
Evaluate the
impact of these
solutions in the
broader context
of organization
and society.
Develop a DMAIC solution
ISE 250
X
Assessment from team
members of student
participation on final project
ISE250
X
Assessment from team
members of student
participation on final project
ISE 202
X
Perform a DMAIC study for
an organization
ISE 250
X
Supply chain analysis
business operations
ISE 245
Collect necessary financial
data and analyze them to
assess profitability, financial
position, and cash flow
generation
ISE 200
Acquire statistical models
and techniques developed
for assuring quality of
enterprise products and
operations.
ISE235
X
Formulate a quantitative
problem in existing
frameworks.
ISE230
X
Formulate and analyze a
problem using a fault tree
diagram
ISE235
X
Evaluation of investment
alternatives using financial
and non-financial factors
ISE200
X
X
X
Evaluation of impact of
supply chain on society and
environment
ISE 245
X
7
Effectively
present and sell
solutions in the
form of written,
oral and
electronic data.
Develop a solution for a
complex ISE problem
ISE 298
X
8
Operate the
organization
effectively and
efficiently by
applying
knowledge and
computational
skills acquired in
the program.
Acquire mathematical
models and techniques
developed for optimizing
efficiency of enterprise
operations.
ISE230
X
Acquire statistical models
and techniques developed
for assuring quality of
enterprise products and
operations.
ISE235
X
Explain why a particular
methodology works.
ISE230
X
Explain how statistical
process control works
ISE235
X
Solve Complex ISE
problems
ISE 298
X
9
Accomplish lifelong growth
within the
field/profession
of ISE.
X
X
5. Student Experience
The PLOs are included in the green sheets of each graduate course and are mapped to course learning outcomes beginning
spring 2015. The PLOs are posted on the ISE webpage. Here is the link http://ise.sjsu.edu/content/bs-ise-student-outcomes.
The students’ feedback is considered in defining and improving program objectives via alumni survey. The program
learning outcomes are then revised accordingly since they map to program objectives. Thus students’ feedback is
considered indirectly.
Part B
6. Graduation Rates for Total, Non URM and URM students (per program and degree)
st-Time Freshmen
Undergraduate Transfer
New Credential
First-Time Graduate
Fall 2008 Cohort: 6-Year
Graduation Rate
Fall 2011 Cohort: 3-Year
Graduation Rate
Fall 2011 Cohort: 3-Year
Graduation Rate
Fall 2011 Cohort: 3-Year
Graduation Rate
Progra
m
Cohort
Size
Progra
m
Grad
Rate
Colle
ge
Grad
Rate
Universi
ty Grad
Rate
Progra
m
Cohort
Size
Progra
m
Grad
Rate
Colle
ge
Grad
Rate
Universi
ty Grad
Rate
Progra
m
Cohort
Size
Progra
m
Grad
Rate
Colle
ge
Grad
Rate
Universi
ty Grad
Rate
Progra
m
Cohort
Size
Progra
m
Grad
Rate
Colle
ge
Grad
Rate
Universi
ty Grad
Rate
Total
4
50.0
%
40.5%
49.7%
12
58.3
%
37.5%
55.3%
0
/0
/0
8.3%
50
60.0
%
64.7%
60.8
%
URM
2
0.0%
22.8%
40.9%
1
100.0
%
36.0%
55.2%
0
/0
/0
12.2%
5
80.0
%
60.0%
65.2
%
NonURM
2
100.0
%
47.8%
53.3%
8
62.5
%
38.2%
54.9%
0
/0
/0
8.0%
25
56.0
%
46.2%
54.2
%
All
others
0
/0
39.3%
52.9%
3
33.3
%
36.4%
56.9%
0
/0
/0
4.9%
20
60.0
%
77.1%
69.4
%
Note: Cohort size too small to make any meaningful interpretation.
7. Headcounts of program majors and new students (per program and degree)
Fall 2014
Total
New Students
Continuing Students
FT Admit
New Transf
Continuing
Returning
Trnst-Ugrd
Total
78
20
243
1
1
343
BS
10
20
127
1
1
159
MS
68
184
116
8. SFR and average section size (per program)
Fall 2014
Subject SFR
College
SFR
Lower Division
University
SFR
26.4
31.0
Upper Division
50.6
27.0
25.5
Graduate Division
34.5
40.9
20.8
Fall 2014
Subject
Headcount per
Section
College
Headcount
per Section
Lower Division
Upper Division
Graduate Division
62.5
27.5
University
Headcount
per Section
48.2
35.6
37.2
28.0
31.6
15.8
Note: The department is meeting the SFR targets as outlined by COE for our department.
9. Percentage of tenured/tenure-track instructional faculty (per department)
Fall 2014
Department FTEF #
Department
FTEF %
College
FTEF %
University
FTEF %
Tenured/Tenure-track
3.0
66%
42.7%
42.8%
Not tenure-track
1.5
34%
57.3%
57.2%
Total
4.5
100%
100.0%
100.0%
Part C
10. Closing the Loop/Recommended Actions
The following actions have been implemented
Note: Each recommended action from the previous year is stated first followed by the actions taken.
1.
For assessing program learning outcome 2, a performance criterion was added in spring 2014 and it will be assessed
using ISE 202 course. This course is a very popular elective course of the program. The department faculty felt that
this course should be included in the assessment matrix.
This new perfomance criterion was assessed in fall 2104 using ISE 202 course.
2.
For assessment of program learning outcome 3, we added ISE 245 course in spring 2014 since a number of students
gets jobs in this course subject matter area and is important and popular course in the program
This new performance criterion was assessed in fall 2014 using ISE 245 couse.
3.
For assessment of program learning outcome 6, we added ISE 245 course in spring 2014 since a number of students
gets jobs in this course subject matter area and is important and popular course in the program.
Outcome 6 was assessed using ISE 245 course in fall 2014.
11. Assessment Data
Performance measure: 80% of the students scored 80% or above.
Interpretation of the performance measure is as follows: The instructors use the holistic rubric included in the Appendix of
this document to grade students’ performance. One or more criteria of the rubric may be used, depending on the type of
assignment and the requirement of the performance criterion that is being evaluated. Finally, students’ performance is scaled
and converted to a percentage. A score of 80% reflects meeting expectations. A description of ‘meets expectation’ for each
criterion is given in the corresponding row under the ‘meets expectations’ column of the rubric in the appendix.
Table 11.1 Data collected and Assessment Methods Used (Fall 2012 and Spring 2013)
Data Collected in Fall 2014 and Spring 2015
MS-ISE Outcome
1
2
3
4
Function
effectively and
provide
leadership within
organization.
Form, facilitate,
lead, and
coordinate and
participate in
teams.
Understand
organizational
processes and
behaviors.
Collect, analyze,
and interpret
data.
Performance
Criteria
Course
Assessment
Method
Instructor
Develop a lean solution for
an organization to improve
productivity
ISE251
Final Project
100% of the
students scored
80% or above
on Final Project
Develop a DMAIC solution
ISE 250
Final Project
100% of the
students scored
92% or abov
Assessment from team
members
of
student
participation on final project
ISE250
Final Project
92% of the
ratings were 4
or above on a
scale of 1 to 5
(1 poor to 5
excellent)
on
participation
ISE 202
Final Project
95.5 % of the
students rated 3
or above (on a
1 to 4 scale)
Perform a DMAIC study for
an organization
ISE 250
Green
Belt
Certification
100% of the
students scored
80% or above
Supply chain processes and
their interactions
ISE 245
Homework #1
100% of the
students scored
80% or above
Collect necessary financial
data and analyze them to
assess profitability, financial
position, and cash flow
generation
ISE 200
Case Study 1
82% of the
students scored
80% or above
Acquire statistical models
and techniques developed
for assuring quality of
enterprise products and
operations.
ISE235
Test
#1
Understanding
of Statistical
Process
Control (SPC)
21% of the
students scored
80% or above
ISE235
Test #2 on
Acceptance
35% of the
students scored
80% or above
5
6
Approach
unstructured
problems and
synthesize and
design solutions
for these
problems.
Evaluate the
impact of these
solutions in the
broader context
of organization
and society.
sampling plan
on Q2, Test 2
ISE235
Final
exam:
Reliability
theory,
data
collection and
analysis
72% of the
students scored
80% or above
on Q5, Test 3
ISE230
Homework 1
(LP
formulation in
different
frameworks
including
finance,
production
planning, shift
scheduling)
83.4 % of the
students scored
80% or above
ISE235
Test #3 Q2 on
formulation of
a
reliability
problem as a
fault tree
54% of the
students scored
80% or above
ISE235
Test #3 Q4
analysis of a
fault tree
90%% of the
students scored
80% or above
on Q4, Test 3
Evaluation of investment
alternatives using financial
and non-financial factors
ISE200
Final
exam,
Q1: Coparison
of investment
alternatives
87% of the
students scored
80% or above
on a case study
Evaluation of Supply Chain
alternatives
ISE 245
Homework #2
100% of the
students scored
80% or above
Formulate a quantitative
problem
in
existing
frameworks.
7
Effectively
present and sell
solutions in the
form of written,
oral and
electronic data.
Develop a solution for a
complex ISE problem
ISE 298
Final Project
100% of the
students scored
a passing grade
8
Operate the
organization
effectively and
efficiently by
Acquire
mathematical
models and techniques
developed for optimizing
efficiency of enterprise
ISE230
Homework 5
97% of the
students scored
80% or above
(LP
formulation,
Shortest Path,
applying
knowledge and
computational
skills acquired in
the program.
operations.
Acquire statistical models
and techniques developed
for assuring quality of
enterprise products and
operations.
9
Accomplish lifelong growth
within the
field/profession
of ISE.
Assignment
problem
fomulations
and solutions
using different
optimization
methods)
ISE235
Test
#2
Statistical
Process
Control (SPC)
Over 90% of the
students scored
80% or above
ISE235
Test #3 Q2 on
formulation of
a
reliability
problem as a
fault tree
54% of the
students scored
80% or above
ISE235
Test #3 Q4
analysis of a
fault tree
90%% of the
students scored
80% or above
on Q4, Test 3
Explain why a particular
methodology works.
ISE230
Homework 2
(About how
Simplex
method moves
at every
iteration to
demonstrate
how it works)
92% of the
students scored
80% or above.
Explain why statistical
process control tools work
ISE235
Test
Statistical
Process
Control
Over 90% of the
students scored
80% or above
Develop a solution for a
complex ISE problem
ISE 298
Final Project
#2
100% of the
students scored
a passing grade
12. Analysis
Referring to Table 11.1, it is clear that the student learning outcomes 4, 5, and 8 are partially achieved in terms of some of
their performance criteria were not achieved at the desired level. All other outcomes were achieved at the desired level. In
ISE 298 course, the students work on new systems and learn new areas of the industrial and systems engineering
discipline. The students do get passing grade in the final projects where they solve complex problems often involving new
scenarios and adapting methodologies learned in the classes. In this sense, students are successful in life-long growth
within the industrial and systems engineering field. Most of the projects are undertaken by the students during their college
practical training while working with their employers in analyzing and solving problems they currently face. Often
students work on lean six sigma and supply chain analysis projects.
13. Proposed changes and goals (if any)
Deficiencies in achievement of outcomes 4, 5, and 8 arise from ISE 235 course.
In Fall 2014, a little more theory was covered, due to the necessity for life-long learning, in the sense of teaching students to
learn how to learn, despite the fact that the MS-ISE Program at SJSU is practice oriented.
Currently, half of the ISE 235 course materials (more precisely, lectures covered every other week) have been recorded as
video clips. In the next year, attempt will be made to record more lectures on SPC as video clips, for students to review or
preview. In addition, attempts will be made to create optional video recordings aimed to review key concepts and methods of
probability and statistics. The homework exercises will also be expanded with a little more theory, so as to better prepare the
student for learning the little more theory.
While the students were able to analyze a fault tree quite well, they did perform well enough on formulation of a fault tree. In
the next offering of this course, the students will be given more opportunities to formulate problems as a fault tree.
Because ISE 235 course is a graduate course, “know-why” is emphasized. For virtually each and every exam. question, one or
more parts of the question asks the student to explain WHY the methods or techniques used in other parts of the same question
are valid or appropriate, often with a limit of up to three short sentences. Explaining how Statistical Process Control (SPC)
works is an integral part of mid-term 1 and mid-term 2. It has been observed that a majority of the students who could use the
methods or techniques correctly could explain why. In fall 2015, an attempt will be made to put in place a mechanism to track
the percentage of students who could use the methods or techniques correctly could also explain why in exam questions.
Appendix : Holistic Rubric
Criteria
1. Reading and
Interpretation
(The student accurately and
appropriately reads and
interprets data found in various
quantitative formats.)
2. Representation
(The student accurately
represents the quantitative
analysis to be accomplished.)
3. Evaluation of the
Data
(The student considers
quantitative information
critically.
E.g., the student evaluates the
efficacy of the data using
criteria such as limitations,
source of the data, potential
bias, timeliness, credibility,
relevance, usefulness, etc.)
4. Assumptions and
Data Limitations
Well Below
Expectations
1
Does not read and
interpret the meaning
of data found in
written, symbolic,
tabular, and/or
graphic form.
Translates words
into numbers.
Below Expectations
2
Meets Expectations
3
Attempts to read and
interpret the meaning
of data found in
written symbolic,
tabular, and/or graphic
form but makes
significant errors.
Uses proper notation,
conventions, etc.
Usually read and
interpret the meaning
of data found in
written symbolic,
tabular, and/or graphic
form but might make
minor errors.
Accurately converts
words into symbolic
frameworks or
equations.
Asks useful questions
about the data and
attempts to answer
them.
Does not question
the data (assumes
the data are valid).
Identifies some
questions about the
data but does not
answer them.
Does not mention
any assumptions.
Identifies assumptions.
Evaluates
assumptions.
Provides rationale for
why each assumption
is appropriate.
Determine if/when
computations are
necessary.
Set up the necessary
computations.
Design a strategy for
completing a
quantitative task.
Does not attempt to
manipulate the data
to meet given
purposes.
Attempts to
manipulate data into
alternate formats for
given purposes but
makes significant
errors.
Attempt to perform
computations but
exhibit many errors.
Select the appropriate
mathematical model to
use in given
computational
situations.
Accurately
manipulates data into
alternate formats for a
given purpose.
Perform calculations
(arithmetic, algebraic,
geometric, etc.) with
minor errors.
Accurately perform
calculations
(arithmetic,
algebraic, geometric,
etc.)
Consistently and
accurately makes
decisions that are
consistent with the
data and situation.
(The student evaluates
assumptions in given
quantitative situations.)
5. Process Modeling
(The student utilizes the
appropriate model for
completing a quantitative task.)
6. Data Manipulation
(The student manipulates data
into alternate formats for given
purposes.)
7. Raw Computation
(The student accurately
performs arithmetic, algebraic,
geometric, etc. calculations.)
8. Decision Making
(The student makes
decision/conclusions that are
consistent with the data and
situation.)
Exceeds
Expectations
4
Consistently and
accurately read and
interpret the meaning
of data found in
written symbolic,
tabular, and/or
graphic form.
Accurately represents
necessary work in
symbolic, tabular,
and/or graphic form.
Asks insightful
questions about the
data and uses
quantitative
reasoning to discuss
the strengths and
weaknesses in the
data.
Do not attempt to
perform
computations.
Does not make
decisions that are
consistent with the
data and situation.
Attempts to make
decisions that are
consistent with the
data and situation but
makes significant
errors.
Usually makes
decisions that are
consistent with the
data and situation but
might make minor
errors.
Selects the best
method for
manipulating data to
address a given
purpose.
Score
0=N/A
9. Validation
(The student judges the
soundness of conclusions
or decisions.)
10. Results
Representation
(The student organizes
and represents
information in
quantitative formats.)
Does not judge the
soundness of
conclusions or
decisions.
Does not organize
and represents
information in
quantitative formats
11. Process Description Does not provide
(The student provides a written written description
description of the
of the quantitative
quantitative process
process used.
employed.)
12. Make Meaning
(The student makes meaning
out of quantitative
information (e.g.,
computations, results,
graphs, etc.)
Lists the numeric
results or provides a
graphic, but does
not describe the
meaning of the data.
Attempts to judge the
soundness of
conclusions or
decisions but
makes significant
errors. to organize
Attempts
and represents
information in
quantitative formats
but makes
significant errors.
Attempts to provide
written description
of the quantitative
process used but
makes significant
errors.
Provides a written
description of the
quantitative
information but
provides limited
explanation of
the meaning.
Usually judges the
soundness of
conclusions or
decisions but might
make minor errors.
Usually organizes and
represents information
in quantitative formats
but might make minor
errors.
Provides an
understandable
narrative description
of the quantitative
process used.
Provides meaningful
descriptions of the
meaning of the
quantitative
information.
Consistently and
accurately judges
the soundness of
conclusions or
decisions.
Consistently and
accurately
organizes and
represents
information in
quantitative
formats.
Provides a detailed
narrative
description of the
quantitative process
used that fully
explains the process
employed. the
Organizes
material and
narrative to make a
point, resolve an
issue, or provide
evidence.