Name of Program: MS – Business Analytics 2014 - 2015
Program Director: Deborah Kellogg (Acting Program Director)
Person Completing Assessment Plan: Deborah Kellogg
Date: May 25, 2015
Program's educational goals or objectives:
The Master of Science in Business Analytics is designed to:

acquire a comprehensive understanding of data, technology, and mathematical models
with the goal to produce evidence-based information for business and government
decision makers;

become proficient users of the latest software and modeling techniques and tools.
Student learning objectives:
Graduates of the program will be able to:
1.
recognize settings or situations that would benefit from the application of Business
Analytic techniques;
2.
propose an appropriate model or solution method;
3.
design solution options using current modeling techniques and tools;
4.
evaluate solution options in light of organizational goals;
5.
communicate solution options in both technical and non-technical ways.
Outcomes Assessment MS-BANA
2014-2015
Page 1
Data Collection and Analysis:
The following courses were selected for outcomes assessment this academic year:

BUSN 6630 – Management of Operations
Two assessments were used in this course. The first was used for all students,
which included MBA and MS-BANA students. The second was used just for the
MS-BANA students.
1.
2.

A case study requiring a spreadsheet model and a memo was used for
evaluation. All students were evaluated and students worked
independently on this assignment. Assignments were assessed by the
faculty member with a rubric.
A project, self-defined by the student was used for evaluation. Projects
were assessed by the faculty member with a rubric. Nineteen students
were used for this evaluation.
BANA 6620 – Computing for Business Analytics
Final projects from all students were evaluated by the instructor.

BANA 6630 – Business Forecasting
Student-designed course projects were evaluated. All student projects were
evaluated

BANA 6640 – Decision Analysis
Exam scores from all students. The exam requires students to evaluate real world
scenarios using current modeling techniques and tools to help answer the
questions of an organization. In many cases, questions are worded in nontechnical ways. Students must then move these to their tools, and they must
finally arrive at a non-technical interpretation to provide to organization decision
makers.

BANA 6910 – Business Analytics Practicum:
The entire semester is focused on defining and solving an industry supported
problem. Outcomes evaluation is based on multiple deliverables throughout the
semester that include documents, models, and presentations. One deliverable
was submitted that represented the work of all students.
Outcomes Assessment MS-BANA
2014-2015
Page 2
Objective 1: Recognize settings or situations that would benefit from the application of
Business Analytic techniques.
BUSN 6630:
Category
Exceeds Expectations
Percentage
Rubric
Situation was appropriate for the
application of Linear Programming
and was particularly complex and/or
creative.
Situation was appropriate for the
application of Linear Programming
Situation was not appropriate for the
application of Linear Programming or
the problem was trivial.
21%
Meets Expectations
74%
Below Expectations
5%
BANA 6630 Assessment of Objective 1:
Category
Exceeds Expectations
Percentage
60%
Meets Expectations
40%
Below Expectations
0%
Rubric
Topic chosen by the student was
appropriate for a forecasting project
and was notably creative or complex
Topic chosen was appropriate for
forecasting but of limited creativity
and complexity
Topic chosen was not appropriate for
forecasting techniques
BANA 6910 Assessment of Objective 1:
Category
Exceeds Expectations
Percentage
100%
Meets Expectations
0%
Below Expectations
0%
Outcomes Assessment MS-BANA
Rubric
Topic chosen by the student was
appropriate for a forecasting project
and was notably creative or complex
Topic chosen was appropriate for
forecasting but of limited creativity
and complexity
Topic chosen was not appropriate for
forecasting techniques
2014-2015
Page 3
Objective 2: Propose an appropriate model or solution method
BANA 6910 assessment of Objective 2:
Category
Exceeds Expectations
Count
100%
Meets Expectations
0%
Below Expectations
Outcomes Assessment MS-BANA
0%
Criteria
Excellent approach. Simulation model
well presented. Significant,
meaningful analysis of results.
Good approach. Simulation model
clearly presented. Acceptable job with
the analysis.
Problems with the model. Inadequate
analysis.
2014-2015
Page 4
Objective 3: Design solution options using current modeling techniques and tools
BUSN 6630 assessment of Objective 3:
Category
Exceeds
Expectations
Percentage
40%
Meets Expectations
45%
Below Expectations
15%
Criteria
Excellent approach. Spreadsheet well
presented. Significant, meaningful
sensitivity analysis.
Good approach. Spreadsheet clearly
presented. Some sensitivity analysis.
Significant errors. Lack of understanding in
relationships.
BANA 6910 assessment of Objective 3:
Category
Exceeds
Expectations
Percentage
Meets Expectations
Below Expectations
Outcomes Assessment MS-BANA
100%
0%
0%
Criteria
Underlying model is complex and wellapplied. Application goes beyond
‘textbook’ example.
Underlying model is robust, but not
complex. More than textbook example.
Underlying model is forced and lack
realism. Similar to textbook example.
2014-2015
Page 5
Objective 4: Evaluate solution options in light of organizational goals.
BUSN 6630 assessment of Objective 4:
Category
Exceeds
Expectations
Percentage
31.5%
Meets Expectations
31.5%
Below Expectations
37%
Criteria
Organizational goals correctly identified
and pros and cons of options linked to
goals and final recommendation. Includes
discussion on what is left unsolved.
Missing one aspect from “Exceeds”
category.
Missing more than one aspect from
“Exceeds” category.
BANA 6620 assessment of Objective 4:
Category
Exceeds
Expectations
Percentage
50%
Meets Expectations
50%
Below Expectations
0%
Outcomes Assessment MS-BANA
Criteria
Full database created and explained in
both technical and non-technical terms.
Excellent use of visuals to understand the
database, and how it meets the business
needs.
Full database created and explained in
both technical and non-technical terms.
Excellent use of visuals to understand the
database, and how it meets the business
needs. Some uncertainty about the
technical aspects that could use some
revisiting to produce a better solution.
Database not created to meet business
needs. Explanations are not clear for
either technical or non-technical user.
2014-2015
Page 6
BANA 6640 assessment of Objective 4:
Category
Exceeds
Expectations
Meets Expectations
Percentage
Below Expectations
Outcomes Assessment MS-BANA
32%
Criteria
Better than 90% on exam
38%
Better than 80% but less than 90% on
exam.
30%
Below 80% on exam.
2014-2015
Page 7
Objective 5: Communicate solution options in both technical and non-technical ways.
BANA 6630 assessment of Objective 5:
Category
Exceeds
Expectations
Percentage
13%
Meets Expectations
73%
Below Expectations
14%
Criteria
Analysis well developed and presented,
numerical support, clear managerial
implications to consider. Implementation
considered.
Clear recommendation, supported by
numerical findings
Recommendations unclear or missing. No
support. Poorly presented.
BANA 6630 assessment of Objective 5:
Category
Exceeds
Expectations
Percentage
80%
Meets Expectations
20%
Below Expectations
0%
Outcomes Assessment MS-BANA
Criteria
Executive summary contained appropriate
nontechnical summaries of the project and
outcomes: non-technical language
appropriate for upper-level management,
recommendations for action and specific
forecasting results. Mostly free of
grammatical or other presentation issues.
Executive summary contained only at most
two of these three: non-technical
language, recommendations, and
forecasts. Most free of
grammar/presentation problems.
Executive summary contained extensive
technical information (i.e., not writing to a
nontechnical audience). Focus for upperlevel management reader was
inappropriate. Contained either multiple
grammar/presentation errors or failed to
specify the numerical values of the
forecasts. Recommendations were absent
or inappropriate.
2014-2015
Page 8
BANA 6910 assessment of Objective 5:
Category
Exceeds
Expectations
Percentage
100%
Meets Expectations
0%
Below Expectations
0%
Outcomes Assessment MS-BANA
Criteria
All communication (written and
presentation) contained both technical
and nontechnical summaries of the project
and outcomes
All communication (written and
presentation) contained only nontechnical
“executive summaries” of the project and
outcomes and mostly free of presentation
errors.
All communication (written and
presentation) contained only nontechnical
“executive summaries” and contained
presentation errors (spelling, grammar,
failure to meet “executive summary”
guidelines provided by the instructor)
2014-2015
Page 9
Aggregated Data over all assessed courses:
Objective
1
2
3
4
5
“Exceeds”
60%
100%
70%
38%
64%
“Meets”
38%
0%
23%
39%
31%
“Below”
2%
0%
8%
22%
5%
Interpretation of the Results:
Both individual course and aggregate data meet the goal of having 75% or better meeting or
exceeding learning goals.
BUSN 6910, the practicum course does present an assessment dilemma. Only one “deliverable”
is produced by all students. To date, the industry contacts that provided the business problem
and the instructor believe that the work exceeded expectations. It was, however, a group
effort. The area has decided not to eliminate that work from the outcomes evaluation process
as it does provide valuable information. It may, however, skew the results. Percentages were
computed without that course and the goal of 75% or better meeting or exceeding learning
goals was still achieved.
Objective 4 was newly added this academic year. There is no previous year’s data for
comparison, although the percentages are lower than for other objectives. Note: The old
Objective 4 became Objective 5. The communication skills in the Objective 5 are assessed but
not really taught in this program. Thus, we wished it to be the last of the objectives. However, it
is important to assess these skill as we need feedback on whether we need to add learning
material for these skills.
Feedback and Use of Information:
No major changes are indicated.
Based on outcomes data, this year we added Objective 4 to better evaluate students at the high
performing level of learning (metacognition.) This year two classes began to implement
assessments in this area. This year’s data suggests that we need better differentiation between
Objective 4 and 5. We also need to continue to develop assessments and rubrics for Objective
4.
Last year’s goal of updating program goals and learning objectives to better reflect the new
focus of the “Business Analytics” program name was complete. Over the next year we need to
carefully map the learning objectives to the existing and new courses and include additional
courses in the assessment effort.
A goal for next year is to include more of the new or re-vamped Business Analytic course in the
assessment process.
Outcomes Assessment MS-BANA
2014-2015
Page 10