LEARNING-CENTERED
ASSESSMENT:
MODELS & STRATEGIES FOR
ASSESSING INSTITUTIONAL
LEARNING OUTCOMES
Ashley Finley, Ph.D
Senior Director of Assessment & Research, AAC&U
National Evaluator, Bringing Theory to Practice
Kutztown University
January 19, 2012
WHY ARE WE HERE?  What does it mean to be

 What
are you
doing?
 Why are you
doing it?
 How do you
know you’re
doing it well?




learning-centered at KU?
What practices promote and
advance student learning?
How pervasive are these?
What is the change you want
to see in your students when
they leave the institution?
What are the foundational
skills and competencies
students should acquire
throughout their learning
experiences?
What are the effective
approaches for implementing
a cycle of inquiry and
improvement?
THE NATIONAL WORK ON LEARNING-CENTERED ASSESSMENT,
LIBERAL EDUCATION OUTCOMES, & GENERAL EDUCATION

Liberal Education: A philosophy of learning that
empowers & prepares individuals to deal with
complexity, diversity, & change.
= Broad knowledge combined w/ in-depth study
  To help students develop a sense of social responsibility,
strong & transferable intellectual and practical skills & a
demonstrated ability to apply knowledge.

Vast majority of institutions have articulated learning
outcomes
 ANY institution can achieve the goals of a liberal
education – 3 Major National Initiatives

Compass Project (State University Systems – CA, OR, WI)
 Quality Collaboratives
 VALUE Rubric Project

WHAT EMPLOYERS SAY ABOUT LIBERAL EDUCATION SKILLS:
% of Employers who agree with each statement
• Our company is asking employees to take on more responsibilities
and to use a broader set of skills than in the past
91%
• Employees are expected to work harder to coordinate with other
departments than in the past
90%
• The challenges employees face within our company are more
complex today than they were in the past
88%
• To succeed in our company, employees need higher levels of
learning and knowledge today than they did in the past
88%
Source: Raising the Bar (AAC&U, 2010)
% of Employers who agree that two- and four-year colleges
should place MORE emphasis on helping students develop
the following:
Effective oral/written
communication
Critical thinking/
analytical reasoning
Knowledge/skills
applied to real world
settings
Analyze/solve complex
problems
Connect choices and
actions to ethical
decisions
Teamwork skills/ ability
to collaborate
Ability to innovate and
be creative
Concepts/developments
in science/technology
89%
81%
79%
75%
75%
71%
70%
70%
Source: “Raising the Bar: Employers’ Views on College Learning in the Wake of the
Economic Downturn” (AAC&U and Hart Research Assoc. 2010)
AND IN THESE AREAS…
Locate/organize/evaluate information
68%
Understand global context of
situations/ decisions
Global issues’ implications for future
67%
Understand & work with
numbers/statistics
65%
63%
Understand role of U.S. in the world
57%
Knowledge of cultural diversity in
US/world
57%
Civic knowledge, community engagement
52%
“Raising the Bar: Employers’ Views on College Learning in the Wake of the Economic
Downturn” (AAC&U and Hart Research Assoc. 2010)
HOW MUCH DOES A LIBERAL EDUCATION PAY?:
“IMAGINATION IS MORE IMPORTANT THAN KNOWLEDGE”
-ALBERT EINSTEIN
Mean Earnings: Use of Speaking Skills
earnings
60,000
30,000
Earni…
0
q1(low)
q2
q3
q4
q5(high)
quintiles
Source: Georgetown University, Center for Education and the Workforce
(Anthony Carnavale)
Mean Earnings: Use of Originality
earnings
60,000
Earnings
30,000
0
q1(low)
q2
q3
quintiles
q4
q5(high)
Source: Georgetown University, Center for Education and the Workforce
Mean Earnings: Use of Judgement & Decision-Making
Skills
earnings
70,000
35,000
Earnings
0
q1(low)
q2
q3
q4
q5(high)
quintiles
Source: Georgetown University, Center for Education and the Workforce
earnings
60,000
Mean Earnings: Use of Social Skills
30,000
Earnings
0
q1(low)
q2
q3
q4
q5(high)
quintiles
Source: Georgetown University, Center for Education and the Workforce
Articulating Student Learning and Development
Content Outcomes
•Theoretical
traditions
•Methodology
•Topical areas
How do we assess
these?
• Course Exams
•Course Papers
•Individual/ Group
course projects
•In-Class work/
participation
Learning Outcomes
•????
•????
•????
Kutztown Mission:
“provide a high quality
education at the
undergraduate and
graduate levels in order to
prepare students to meet
lifelong intellectual,
ethical, social, and career
challenges.”
How do we
assess these?
•Goal 1: Cultivate intell. & pract. skills
•Knowledge of Human
•Inquiry & Analysis
Cultures & the Physical &
•Decision making
Natural World
•Written & Oral Communication
•Content areas
•Quantitative
Literacy
Institutional
•Intellectual & Practical
•Information Literacy
Skills
•Teamwork & problem solving
•Inquiry & Analysis
•Wellness
•Practiced throughout curr, w/ progressively •Critical & Creative Thinking
more challenging probs, projects & standards
•Written & Oral Comm.
•Goal 2: Develop understanding of human
•Reading
cultures & physical and natural world
•Quantitative Literacy
•Sciences, mathematics, social sciences,
•Information Literacy
humanities, histories, languages, arts
•Teamwork & Problem-solving
•Engage w/ big questions, contemp. & enduring •Personal & Social Resp.
•Goal 3: Sense of personal & social resp.
•Civic Knowledge
•Local/Global civic knowledge
•Intercultural Knowledge &
•Intercultural knowledge & competence
Competence
•Ethical reasoning & action
•Ethical Reasoning
•Personal qualities & attitudes (passion,
•Lifelong Learning
curiosity, self-confidence, imagination,
•Integrative & Applied
cooperation, commitment, support)
•Active inv. w/diverse comm. & real world chall.
Learning
Outcomes
Gen.Ed./
Major
Course
What is the story you want to tell
about student learning?
Assess
Learning
Outcomes
Learning
Outcomes
Curriculum
Assessment
Outcomes
What is the story you want to tell
about student learning?
Assess
Learning
Outcomes
Learning
Outcomes
Basic Logic Model
INPUTS
ACTIVITIES
Resources
are needed
Actions or
for activities
processes
– What is
necessary to
needed to
produce
start or keep
outputs –
something
What
going?
activities
•Human
need to occur
• Financial
to produce
•Technological
evidence?
•Curricular/
Program
OUTPUTS
OUTCOMES
Products
used to
assess
outcomes What can be
counted as
evidence of
change?
Expected
Changes and
Benefits –
What are the
goals you
want to reach?
Assumptions
• Short-term
• Intermediate
• Long-term
Sample Logic Model
INPUTS
•GE
•Depts.
•Advising
•Writing
Center
•Stud. Aff.
•Health &
Wellness
Center
•Alumni
Engagement
•Multicult.
Services
•Athletics
•Distance Ed
•Career
Services
•Inst. Res.
ACTIVITIES
•Experiential
learning
•Civic learning
•Interaction
•Engagement in
life’s big
questions
•Practiced
extensively &
progressively
•Active involve.
In diverse comm.
& real-world
challenges
OUTCOMES
OUTPUTS
• Goals &
•Signature
Domains of
assignments
Learning
•Reflection
• Understanding
papers
of human
•Group
cultures…
projects
• Intellectual
•Communityand practice
based
skills
projects
• Critical
thinking
• Personal &
Social
Responsibility
• Civic
engagement
VALUE PROJECT (WWW.AACU.ORG/VALUE)


Project Goals
 1) Create dialogue and develop shared understanding of
common learning outcomes
 2) Create template for direct assessment of student work (in
text and non-text formats)
 3) Create student-friendly format to engage students in selfevaluation
Rubric Development
 Teams of faculty/scholars nationwide
 Reviewed existing rubrics to identify commonalities, clarify
language, and develop broad agreement on outcomes
criteria (openedpractices.org)
 To date accessed by over 3000 institutions/organizations,
11,000 individuals


Domestic & international, K-12, state systems
Reliability study
LIST OF VALUE RUBRICS

Knowledge of Human
Cultures & the
Physical & Natural
Worlds



Civic Knowledge &
Engagement
 Intercultural
Knowledge &
Competence
 Ethical Reasoning
 Foundations & Skills
for Lifelong Learning

Content Areas No
Rubrics
Intellectual and
Practical Skills










Inquiry & Analysis
Critical Thinking
Creative Thinking
Written Communication
Oral Communication
Reading
Quantitative Literacy
Information Literacy
Teamwork
Problem-solving
Personal & Social
Responsibility

Integrative &
Applied Learning

Integrative & Applied
Learning
Located at:
http://www.aacu.org/value/index.cfm
The Anatomy of a VALUE Rubric
Criteria
Levels
Performance
Descriptors
THE CALIBRATION TRAINING PROCESS

Scoring Steps:
Review rubric to familiarize yourself with structure,
language, performance levels
 Ask questions about the rubric for clarification or to get
input from others regarding interpretation
 Read student work sample
 Connect specific points of evidence in work sample with
each criterion at the appropriate performance level (if
applicable)


Calibration Steps:





Review scores
Determine common score(s)
Hear from outliers
Discuss
Determine final score
THE GROUND RULES
This is not grading.
 We are not changing the rubric (today).
 Our work is time sensitive. Go with your instinct.
 Think globally about student work and about the
learning skill. Think beyond specific disciplinary
lenses or content.
 Start with 4 and work backwards. Pick one
performance benchmark per criterion. Avoid “.5”.
 Zero does exist. Assign “0” if work does not meet
benchmark (cell one) performance level. N/A exists.
Assign “not applicable” if the student work is not
intended to meet a particular criterion.

SIGNATURE ASSIGNMENTS

Assignment should enable attainment of criteria
 Break
down criteria to determine key components for
assignment

What should students do with content to meet
criteria?
 E.g.
What are the pieces to be analyzed, compared, integrated?
Will the assignment be used for more than one
outcome?
 What are the types of assignments that will be
most helpful for allowing students to demonstrate
competency?
