Considerations for
Assessing Program Objectives
OCTOBER 2011
Malcolm LeMay
Director of Operations
College of Business
Assessment of Learning – Lessons Learned
 Considerations and approaches
 Focus on accreditation requirements and expectations
 Minimize burden on faculty while ensuring participation
 Close the loop from assessment to improvement
 Ensure value-added activities and focus on
tracking/reporting relevant info
 Use technology to reduce confusion, ensure consistency and
improve process
Process for Assessment of Student Learning
1. Define goals and objectives
2. Align curricula with goals
3. Identify instruments and measures to assess
learning
4. Collect, analyze and disseminate assessment
information
5. Use assessment information for continuous
improvement
In other words…
1. What will our students learn in our program?
What are our expectations?
2. How and when will they learn it?
3. How will we know they have learned it or not?
4. What will we do if they have not learned it?
Concept of Accreditation Maturity Model
Capability Maturity Model:
1. Initial - ad hoc
2. Repeatable practices
3. Practices defined
4. Managed and measured
5. Optimizing
Accreditation Maturity Model
 Developed interpretation of AMM levels for all
AACSB standards
 Training during faculty retreat 18 months prior to
Accreditation visit
 Conducted self-assessment on AMM levels
 Results



Increased faculty awareness and buy-in
Process gaps identified
Improved understanding of expectations
Applying AMM for Assessment
Standard 16 – Learning Goals
 Adapting expectations to the school’s mission and cultural circumstances, the
school specifies learning goals and demonstrates achievement of learning
goals.
 Learning goals address:
 General knowledge and skills (i.e., communications, problem-solving,
ethical reasoning)
 Management or discipline specific knowledge and skills (entrepreneurial
perspective, professional behavior)
Basis for judgment:
 Demonstrate that students meet the learning goals
 If learning goals are not being met, demonstrate efforts to eliminate the
discrepancy
Applying AMM for Assessment
 Level 1 – General objectives exist and are measured through
traditional course evaluation (initial)
 Level 2 – Program objectives exist, assessment of learning relies on
indirect measures (repeatable)
 Level 3 – Program objectives integrated with curriculum
development; relies on direct assessment (defined)
 Level 4 – Assessment process generates quantitative and qualitative
data, improvements are planned and implemented (managed)
 Level 5 – Assessment is an ongoing process aimed at continuous
improvement of programs, curricula and course delivery. Processes
are periodically reviewed and improved (optimizing)
OSU Vision/Mission
COB Vision/Mission
Assessment Approaches:
1. Direct assessment of course
outcomes mapped to
program objectives
2. Direct assessment of
program objectives through
course embedded measures
(e.g., exams, projects,
papers)
3. Direct assessment through
stand-alone performance
(MBA oral exams)
4. Indirect assessment of
program objectives (student
surveys)
Program
Goal #1
Program
Goal #2
PO 2.1
PO 2.2
Program
Objective
1.1
Program
Objective
1.2
Courses
Results:
Exceeded – xx%
Met – xx%
Below – xx%
Courses
Impact on program offerings:
New courses, changes to
sequencing, adjustments to
program objectives and course
outcomes
Responses/Actions:
Changes to curriculum,
pedagogy, coverage,
prerequisites
Faculty Involvement in Assurance of Learning Process
Mapping Core Courses to Program
Objectives
Mapping Course Outcomes to Objectives
Example (Business Administration)
 Program Goal #1: Students will identify, assess
and shape entrepreneurial opportunities in a
variety of contexts.

Program Objective 1.1: Recognize entrepreneurial
opportunities for new business ventures and evaluate their
potential for business success.

Course Outcome (BA-260.6): Apply venture opportunity
screening techniques to an actual start-up idea, and
subsequently develop and prepare a feasibility plan.
 Course Outcome Assessment: Measured by evaluating team
projects requiring a presentation analyzing the feasibility of a
new business concept.
Measurement methods
 Direct and Course Embedded:
 Individual/team presentations
 Oral exams
 Mock interview reviews
 Group collaboration assignments
 Simulation results
 Essay exam questions
 Course entrance exams
 MBA business plan competition
 Indirect
 Exit surveys
 Recruiter feedback
Use of Technology
 Simplify collection, analysis and




reporting
Ensure consistency
Improve overall awareness of
learning and student success
Ease burden on faculty
Reduce reliance on Excel
spreadsheets
To the Cloud  COB Force.com course information
Goal 2 - Professional
communication and
behavioral skills


2.1 Communicate effectively
and professionally in
business situations through

physical or virtual presence,
writing, speaking, listening,

and electronic media.

2.2 Demonstrate the ability to
lead by using team building

skills and facilitating
collaborative behaviors in
the accomplishment of
group goals and objectives.
2.3 Demonstrate good work
habits, time management
and self discipline.
Assessment Findings
Written communication skills
assessed below desired level
Oral communication skills for
professional situations assessed
below desired level
Limited opportunities to expose
students to virtual collaboration.
Students lack professional
behavioral skills
Students’ leadership skills
assessed as less than strong
Desire greater participation in
experiential learning





Actions Taken
Added BA 353 Professional Development
course to the business core as a “Writing
Intensive Course” to provide students with
the opportunity to learn business
communications earlier in the curriculum.
BA 353 added assessment of oral and
presentation skills through mock interview
sessions.
BA 370 added course projects using
SharePoint to emphasize virtual collaboration
skills.
BA 352 revised to emphasize practical skills
in leadership and team building. The course
is now taught exclusively to business
students.
Implemented a professional development
program which has improved the awareness
levels of internship opportunities through
greater publicity efforts, created a system for
monitoring the number of internships
undertaken by students during the year, and
provided a year round program of
professional development .
Moving Forward with Program Assessment
 Target level 4 for all programs
 Ongoing
Involve faculty in program mapping and analysis
 Use technology to reduce support requirements,
encourage consistency and ease demands on faculty
 Improve reporting and follow-up to “close the loop”
 Keep in mind “big picture” - continuous improvement
 Aspire for “accreditation-ready”

Back up slides
Bloom’s Taxonomy of Learning Domains
1. Knowledge: Recall data or information
2. Comprehension: Understand the meaning,
translation, interpolation, and interpretation of
instructions and problems.
3. Application: Use a concept in a new situation or
unprompted use of an abstraction.
4. Analysis: Separates material or concepts into
component parts so that its organizational structure
may be understood.
5. Synthesis: Builds a structure or pattern from diverse
elements. Put parts together to form a whole, with
emphasis on creating a new meaning or structure.
6. Evaluation: Make judgments about the value of
ideas or materials