Robert Mundhenk and Jennifer Fager
Assessment of Academic Achievement Conference
University of Wisconsin-Superior
18 January 2011

Assessment is a process in which those
responsible for student learning on campus
collect, analyze, and use information about
student learning to improve it.

“Outcomes-assessment practices in higher
education are grotesque, unintentional
parodies of both social science and
‘accountability.’ No matter how much they
purport to be about ‘standards’ or student
‘needs,’ they are in fact scams run by
bloodless bureaucrats who . . . do not
understand the holistic nature of a good
college education.”
--Laurie Fendrich, “A Pedagogical
Straitjacket,” Chronicle of Higher
Education, June 8, 2007, B6.

“The real culprit is the education establishment’s
continuing reluctance to give the public a clear
and accurate glimpse at how colleges spend their
money. What’s missing, by and large, is
‘sunshine.’”
--John A. Boehner, Speaker of the
House of Representatives, though said as Chair,
House Committee on Education and the
Workforce



Assessment to Improve Learning
Assessment as Meaningless Bureaucratic
Imposition that Violates Academic Freedom
Assessment as an Element of Institutional
Accountablity

[Colleges constantly tell us] “Everything is
fine, we are doing a great job, send us more
money, and leave us alone.”
--Rep. Buck McKeon, House
Committee on Education and
the Workforce

SPELLINGS COMMISSION (8/3/06):
◦ “ . . . The quality of student learning at U.S. colleges and
universities is inadequate and, in some cases, declining.”
(7)
◦ “Accreditation reviews are typically kept private, and
those that are made public still focus more on process
reviews than bottom-line results for learning or costs.”
(13)


Duncan Department of Education attacks the
laxity of accreditors, including HLC, in
assuring institutional and program quality
“Gainful employment” and “program integrity”
assurance pushed by Duncan Department of
Education
 Student debt levels and employment rates

“But can I assume that my graduate is going
to be literate, know how to write, be exposed
to math, be taught by senior faculty?”
--Anne D. Neal, National Advisory
Council on Institutional Quality and Integrity,
12/20/07

“Institutions are asked, ‘Are you measuring
student learning?’ And they check yes or no.
That must change. Whether students are
learning is not a yes-or-no question. It’s
How? How much? And to what effect?”
--Margaret Spellings Sept 06

"We think -- I certainly think -- that it would
be helpful when students head off to college
that they have some basic information that
they often don't have now: what's this going
to cost, what's the [loan] repayment at that
school, the expected graduation rate, the
placement rate if I go into certain fields. . . .

“One fallout from the gainful employment
discussion is that a lot of the things people
are pushing the for-profits for would be
useful for all students to have, whether
they're going to a land-grant or a private
college, to a for-profit or a nonprofit. I think
we'll be likely to look at getting students that
kind of information.
John Kline (R-MN), Chair, House
Committee on Education and Labor
(IHE 12/23/10)



“Robust culture of accountability” with a “consumerfriendly database that provides access to institutional
performance and aggregate student outcomes” (21)
“Accreditation agencies should make performance
outcomes, including completion rates and student
learning, the core of their assessment . . . to . . . allow
comparisons among institutions regarding learning
outcomes and other performance measures” (23)
“Develop interoperable outcomes-focused accountability
systems designed to be useful to students, policymakers,
and the public. . . .”(22)

Educational Testing Service:
 “What is needed is a systematic, data-driven,
comprehensive approach to understanding the quality of
two-year and four-year postsecondary education, with direct,
valid, and reliable measures of student learning.”




Workplace readiness and general skills
Domain-specific knowledge and skills
Soft skills . . . .
Student engagement with learning

Standard IPEDS data
◦ Enrollment, persistence, graduation, etc.

Student “experiences and perceptions”
◦ satisfaction and engagement data

Student learning outcomes:
◦
◦
◦
◦
Critical thinking
Analytic reasoning
Written communication
“. . . Using one of three tests”


Ranking colleges based on graduates’
performance evaluations
“It’s really about improving the dialogue on
curriculum, performance, and how we can
build a stronger relationship between the
colleges, universities, and us, because,
ultimately, their students become our
employees.” --Richard D. Stephens, Senior Vice President for
Human Resources (Chronicle of Higher Education, 9/19/08)

“Accountability in higher education begins
with the establishment of public definitions of
degrees and criterion-referenced statements
of academic performance so that, when an
institution awards a credential it can assert,
with confidence, ‘This is what a degree
represents, this is what the student did to
earn the degree, and a warrantee has been
issued on behalf of both institution and
student.’
--Clifford Adelman, Learning Accountability from Bologna (IHEP,
July 2008)





Government and employers are seeking
greater accountability from higher education
Government is demanding greater
accountability and rigor of accrediting
commissions
Higher education associations are responding
with outcome-based models
Some approaches are clearly inadequate
Institutions must deal with internal blowback
while dealing with external pressures

How do we avoid being reductive (or worse)
while accommodating the great triad of
accountability, accreditation, and
assessment?

Refocus on student learning within Five Criteria:






Mission and Integrity
Engagement and Service
Acquisition, Discovery, Application of Knowledge
Student Learning and Effective Teaching
Preparation for Future
The new Criteria see learning as infused
throughout all Criteria, so responsibility for
supporting, enhancing, and improving learning
becomes an institutional responsibility







Shift from inputs to outcomes
Shift from capacity to student learning
Shift from processes to results
Concern with learning and its improvement infuses
all criteria
Focus on shared responsibility
Focus on accountability, but not on reporting for its
own sake
Concentration on continuous improvement
Assessment “should be informed by the
institution’s mission and include explicit public
statements regarding the knowledge, skills, and
competencies students should possess as a
result of completing course and program
requirements. . . . ”


“The institution defines quality by how well it fulfills its
declared learning mission.”
The institution documents student learning by:






Setting clear learning goals
Collecting evidence of attainment
Applying collective judgment as to the meaning and utility of evidence
Using evidence to effect improvement
It uses multiple sources and kinds of evidence
It shares results with stakeholders
Assessment’s “effectiveness in improving student
learning relies on its integration into the
organization’s processes for program review,
departmental and organizational planning, and
unit and organizational budgeting.”





Emphasis on student learning
Focus shifts from content and inputs to
outcomes: what students know and value—
and can do with what they know and value
General emphasis on improvement over
reporting
Consideration of stakeholders
Focus on use of data and integration into
institutional processes



How are your stated student learning outcomes
appropriate to your mission, programs, and
degrees?
What evidence do you have that students achieve
your stated learning outcomes?
In what ways do you analyze and use evidence of
student learning?



How do you ensure shared responsibility for
student learning?
How do you evaluate and improve the
effectiveness of your efforts to assess and
improve student learning?
In what ways do you inform the public about what
students learn—and how they learn it?




Effective assessment becomes a matter of
commitment to the improvement of learning
by improving the processes that produce it
Effective assessment focuses on how data can
be used to improve, not on how it can be
reported
Effective assessment focuses on inquiry
rather than evidence
In that sense, effective assessment is at the
heart of what we do as academic institutions
Meaningful
Useful
Workable
Reasonable



Outcomes, plans, strategies, and approaches
need to be focused on student learning in the
context of institutional mission and values
A wide range of data-collecting strategies are
possible, so long as they serve institutional
needs in improving learning
Attention must be given to assessing learning
in non-classroom settings—and to shared
responsibility for that learning


Assessment strategies must be appropriate
and credible, not uniform across the
institution—or the state
Assessment strategies must produce
information that can:
--credibly be used to improve
learning
--adequately inform stakeholders
of the degree to which outcomes
have been achieved



Institutions must become comfortable with
risk—and with the probability of failure—in
the service of improved quality of learning
Assessment data must be seen as the basis
for institutional conversations about learning,
not as a process of self-justification
The quality, credibility, and integrity of an
institution will be assumed to be reflected in
its effectiveness in producing student
learning

Assessment is a transparent process in which
those responsible for student learning on
campus collect, analyze, and use information
about student learning to improve it, while
sharing information about the achievement
and improvement of learning with interested
stakeholders.

A process based in inquiry:





What do our students learn?
Is what they learn what we say they will learn?
How do we know they have learned?
What do we do with our knowledge?
How can we improve student learning?

Predicated on clear delineation of student learning
outcomes:
 What students can do with what they know
 Aligned at several levels:
 Institutional mission
 Program/major purpose
 Course’s contribution to both

A form of action research that can include:




Observations
Judgments
Validated data
Grades that correlate to outcomes




A way to improve learning
A way to improve the systems that produce
learning
A way to demonstrate effectiveness to various
stakeholders
A basis for conversations with stakeholders



A formalization of what many of us already
do intuitively and individually
An attempt to judge achievement of
essential outcomes rather than a
compilation of grades or test scores
A process focused on change for
improvement, not on institutional selfjustification