Patricia M. King
Center for the Study of Higher and
Postsecondary Education
University of Michigan
Tensions underlying Learning Outcomes
“We seek understanding for the pleasure and
confidence it brings, and we seek puzzlement for selfconscious ignorance for the mental itching and
scratching it engenders. We want students who will
leave our institutions deeply committed to values and
civic and moral responsibility; yet we must never forget
that they must also be committed to skepticism and
doubt. We foster the transformation of thought into
action, but we also strive to educate for delay, selfcriticism, and reflection.” (Lee Shulman, 2002)
Concerns about Student Learning
Popular press: Many reports of “underachieving”
or “failing colleges” or of students being
“academically adrift”
Employer reports of widespread dissatisfaction
with students’ skills in critical thinking, ability to
solve complex problems, and collaborate with
others in diverse group settings (e.g., AACU, 2010).
More Concerns about Student Learning
Most studies of student development have
shown that very few college students (and
college educated adults) achieve highest level
of development consistent with college missions.
In light of these concerns, our work is cut out for
us. Luckily, we have resources to tap here.
Intentionally Promoting Learning
Outcomes: Factors to Consider Today
Four Interrelated Factors
 1. Goals for Student Learning
 2. Student Characteristics
 3. Principles of Student Development
 4. Creative Implementation (class/program,
department/functional area, campus wide)
Three Interrelated Levels of Focus
or Course
1 on 1
Helping Students Achieve Learning Outcomes
1) Be clear about the learning goals of your program.
2) Know your students; what are their current skills
(cognitive, emotional, relational)?
These two ideas anchor the two ends
of the bridge of your journey together.
3) Add knowledge of student development
4) Map out this journey.
5) Hang on for the ride!
Learning goals-3 levels
Institutional: mission, specific campus-wide initiatives
Programmatic: general education, by major, by
focus (leadership, intergroup relations, career, etc.)
Individual: student-specific, often embedded within
programmatic or institutional goals
1) Identify Your Learning Goals
What aspect of development do you wish to
What skills are particularly important at this time?
Are there key features (e.g., opportunities,
constraints) that may affect these goals?
Example: promote reflective thinking
 Why
is this important at this time?
 Are there specific concerns that you wish to address?
21st Century Essential Outcomes
Knowledge of Human Cultures & the Physical and Natural
Intellectual and Practical Skills: Inquiry & analysis, critical &
creative thinking, communication, information literacy,
teamwork, & problem solving
Personal and Social Responsibility: civic knowledge &
engagement, intercultural competence, ethical reasoning &
action, skills for lifelong learning
Integrative Learning: synthesis, application of knowledge, skills,
and responsibilities to new settings and complex problems
American Associations of Colleges and Universities LEAP Project,
Capacities Underlying Learning Outcomes
See knowledge as contextual and constructed; apply
complex reasoning skills to judgment making
Have an internally generated identity and sense of
self that regulates experiences and choices (e.g., to
apply one’s values in deciding how to act)
Have the capacity to engage in authentic,
interdependent relations with diverse others
Mature capacities of self-authorship and other
lifespan & college student development theories
Without Mature Capacities, …
report feeling overwhelmed in
situations that require
 Making an independent judgment, taking a
stand, voicing an unpopular opinion;
 Taking responsibility despite extenuating
 Navigating life’s complexities (e.g., disciplinary,
social) that can’t be reduced to dichotomies.
2) Know Your Students
Gifted in math, geography, dance, science, languages
Remarkably aware and worldly
Remarkably naïve and self-absorbed
Remarkably compassionate
Remarkably mean spirited
--And everything in between…
Regardless, it is important to know where they are
coming from, and accept them “where they are.”
Be aware of their developmental capacities
What are their cognitive capacities (critical thinking,
complex reasoning, seeing subtleties and nuances) ?
What are their interpersonal skills (empathy,
understanding of others, balancing individual and
community goals)?
How do they define themselves (clarity of values,
social identities [gender, ethnicity, religion, sexual
orientation], sense of purpose)?
Learn How Students Make Judgments
How would you characterizes their problem solving?
How well do they already engage in reflective
On what do they base their judgments currently?
How do they reconcile differing opinions/conclusions?
What are common frameworks for interpreting data
used by students with whom you work?
Development and Schemas
Schemas are known as frames of reference,
meaning perspectives, habits of mind, mind-sets.
They are powerful because
They work implicitly, in the background, so are
not typically scrutinized or evaluated
They generate beliefs and guide actions
At the heart of development is examining and
changing one’s schemas about the world & oneself
Schemas Develop
The process by which we transform our taken-forgranted frames of reference…to make them
More inclusive,
Emotionally capable of change, and
So that they may generate beliefs and opinions that will
prove more true or justified to guide actions.
Mezirow (2000), Learning as transformation: Critical perspectives on a theory in progress (pp. 7-8 ). San Francisco:
Well- and Ill-Structured Problems
Can be described with a
high degree of
Can be solved with a high
degree of certainty
Cannot be described with a high
degree of completeness
Cannot be solved with a high
degree of certainty
Experts usually agree on the Experts often disagree about the
correct solution
best solution, even when the
problem can be considered solved
Goal: Learn to reason to
correct solutions
Learn to construct and defend
reasonable solutions
RJ Levels-Views of Knowledge
Early-Prereflective: Knowledge exists
absolutely and concretely
Middle-Quasi-Reflective: Knowledge is
uncertain and idiosyncratic to the knower
Advanced-Reflective: Knowledge is the
outcome of a process of reasonable
RJ Levels-How to Justify Beliefs
 Early:
Observe phenomenon yourself or
ask an authority figure
 Middle: Gather evidence and arguments;
choice is idiosyncratic to knower (e.g.,
choose evidence that fits a belief)
 Advanced: What is most compelling based
on evidence from variety of considerations
(e.g., most complete, reasonable, plausible)
Basis for Beliefs-Early Levels
Point of View #1
Point of View #2
How I was raised
You were raised differently
How I was taught
How you were taught
What I want to believe
What you want to believe
What I take as evidence
What you take as evidence
 Assumes one is right and one is wrong.
 No explicit criteria for judging this
Patricia M. King, U of Michigan
Basis for Beliefs-Middle Levels
Point of View #1 ------------------ Point of View #2
 Take perspectives into account, allow them to be
 See that both perspectives reflect assumptions, are
based on evidence
 See both as valid, begin to examine strength of
Patricia M. King, U of Michigan
Basis for Beliefs–Advanced Levels
Think of each box as clusters of evidence; Sort
strong and weak evidence/arguments; see how they
fit together to construct an argument.
Patricia M. King, U of Michigan
“You’ve got to want to connect the dots, Mr. Michaelson.”
Assessing Student Characteristics
Formal: Institutional Studies (CIRP, NSSE, Wabash
National Study); systematic records of drop out and
transfer, disciplinary infractions, academic integrity
queries, basis for roommate change requests,
participation rates in service learning & other high
impact practices, etc.
Informal: Listen to students, ask about their views, how
their views have changed over time, decisions they
are facing; reasoning used in course assignments
Assessing RJ Concepts
Do students acknowledge the basis for
multiple points of view about
controversial issues? Rely on authorities,
peers, evidence?
 Can they articulate more nuanced
differences (e.g., racist vs. race-conscious)
 See themselves as learners who construct
their beliefs?
Listening for Reasoning, Meaning
People convey making changes in their meaning
making through comments such as,
“I don’t think of it that way anymore” and
“I never thought I’d be saying this, but…”
“I was really uncomfortable and knew I had to do
When these changes are developmentally
adaptive, they result in “transformative learning.”
Linking Goals and Student Characteristics:
Building Developmental Bridges
3) Apply Knowledge of Student Devt
In light of both goals and student characteristics:
 Identify consistencies and discrepancies between
learning goals and learner capacities
 Balance level of challenge with level of support
 Use developmental sequencing to outline the
“curriculum” of the journey
AACU’s VALUE Rubrics
Promote authentic assessment of student work
and campus-wide shared understanding of
student learning outcomes
VALUE: Valid Assessment of Learning in
Undergraduate Education (
15 rubrics created by teams of faculty and
academic professionals on campuses
Problem Solving Rubric
Identify strategies
 Propose solutions/hypotheses
 Evaluate potential solutions
 Implement solution
 Evaluate outcomes
Each component is defined by 4 levels:
benchmark  2 milestones  capstone
Proposes a solution/hypothesis that…
Benchmark: is difficult to evaluate because it is
vague or only indirectly addresses the problem
Milestone: 1) is “off the shelf” rather than
individually designed to address specific contextual
factors; 2) indicates comprehension; is sensitive to
one other dimension (ethical, logical, OR cultural)
Capstone: indicates a deep comprehension of the
problem; is sensitive to contextual factors; is
sensitive to ethical, logical, AND cultural dimensions
of the problem
Teamwork Rubric
Contributes to team meetings
Facilitates the contributions of team members
Individual contributions outside of team meetings
Fosters constructive team climate
Responds to conflict
Facilitates the Contributions of Team
Members: Engages team members by...
Benchmark: …taking turns and listening to others
without interruption
Milestones: 1)…restating views of others;
2)...building upon, synthesizing contributions of
Capstone: …constructively building upon or
synthesizing the contributions of others, noticing
when someone is not participating, inviting them to
Encouraging Reflective Thinking
Be explicit about expectations for
achieving learning outcomes
 Provide many opportunities for students to
examine the basis for own & others’ views
 Help them discern their underlying
assumptions and schemas
 Give feedback to help them see how close
they are to a “capstone” level
Encourage Reflective Thinking
Create climates where it is safe for students
to explore options and take risks as they
explore and construct responses
 Explore the basis for different beliefs
 Encourage them to articulate their ideas
and conclusions, however tentatively
 Give options that encourage
Encourage Complex Reflections
1) Simple Prompt: Which do you prefer? (That’s all.)
2) More Complex, Multidimensional Prompts
What evidence does each person/approach offer?
Which facts are central to the argument? Which are
supplemental? Which are irrelevant?
Where are the “holes” in the each argument?
Taking each approach as a whole, how strong is it?
Compare the weight of the evidence for each;
which is stronger? [What are good criteria for
Encourage Complex Reflections
3) Integrative Prompts
How did your analysis affect your conclusion?
How could you use this kind of analysis when
making decisions in other contexts?
What will you take away from your involvement
in this activity that you could apply elsewhere?
How will you apply what you learned this year to
your plans for next year?
Encourage Reflective Thinking
 Offer many opportunities to talk about
meaning and interpretation (“What do you
think?” “What did you make of that?”)
 Validate that their ideas are important by
giving space and time for this.
 Ask how their experiences affected them
(how they think and react).
 Offer progressively challenging tasks, with
appropriate support to succeed at them.
Encourage Reflective Thinking
Promote a campus culture of reflection
 Where do opportunities for reflection build upon
and reinforce each other?
 How
does what you learned in that class contribute to
your understanding of your major?
 How will your experiences this year affect your goals
for next year?
Scaffold deeper reflection through e-portfolios,
rubrics, course projects, volunteering, internships, etc.
Baxter Magolda, M. B. & King, P. M. (2008). Toward reflective conversations: Promoting selfauthorship through advising. Peer Review, 10 (1) 8-11.
Promote Development!
“Comfort the disturbed;
disturb the comfortable.”
“We do not make transformative changes in the way
we learn as long as what we learn fits comfortably in
our existing frames of reference.”
Mezirow (1997). Transformative learning: Theory to practice. In New Directions for Adult and
Continuing Education, 74, 5-12.