Association of Research Libraries
Assessment Conference, Baltimore,
Maryland: October 27, 2010
PAPER BY: Rachel A. Fleming-May, Assistant Professor, School of Information
Sciences, The University of Tennessee, and
Crystal Sherline, M.L.I.S., Doctoral Student College of Communication and
Information, The University of Tennessee (presenter)
Current Academic Library
Assessment Climate (in a nutshell):
 Darkening financial horizon
 Increasing costs of resources and
services
 Increased emphasis on
institutional-level evaluation
for
Foundations, so far…
 Significant contributions to understanding library’s contribution
to research:
 Study of library return on investment (ROI) in library resources in
development of successful grant applications:
 Phase I (U.S.) and
 Phase II (International)
 Ongoing efforts to shift academic library assessment
conversation from input and output measurement to a focus on
outcomes…
 ARL New Measures Initiative
 …and to improve understanding of the “state of the art” of
academic library assessment
 Megan Oakleaf and ACRL’s Value of Academic Libraries: A
Comprehensive Research Review and Report
Significant progress has
been made in
establishing models for
demonstrating Return
on Investment for
Research.
…but what of the Academic
Library’s Contribution to
Teaching and Learning ?
 More difficult to assess
and demonstrate
CAUSATION. Most studies
demonstrate either
 Correlation between
student success and
library use, or
 Short-term effects of
information literacy
instruction.
We’re also not always entirely sure what we should be
trying to demonstrate.
Our Project
Part of IMLS-Funded Project:
Carol Tenopir, The University of Tennessee: Lead Principal Investigator
Paula Kaufman, The University of Illinois: Co-Principal Investigator
http://libvalue.cci.utk.edu/
Our Goals and Approach:
 Helping to create structure for future assessment
efforts:
 Shift from initial conceptualization presented in
abstract for the LAC:
 Focus on Six Regional Accrediting Agencies’
Standards
 Several LIS-related projects have looked at Regional
Standards, but most have either:
• Focused on one agency’s standards, or
• Focused on standards specifically related to THE LIBRARY or
LIBRARY-RELATED ENDEAVORS, such as INFORMATION LITERACY
SO WHAT! IL is important!
, IT IS…
But we believe (and we’re
not alone) academic
libraries need to expand
evidence of their
influence to reflect
institution-level standards
in order to maintain
relevance.
QUICK REVIEW: Institution-Level
Accreditation in the U.S.
 No common standards at the national level;
 Six regional agencies
 Similar themes in standards, but lack of uniformity:
 Language
 Structure
 Frequency of updates
 All Six Agencies’ Standards address
 Library Collections with varying degrees of specificity;
 Information Literacy in direct or “Equivalent Language”
Other Standards
for Academic
Libraries:
 Program-level
accreditation; varies by
institution and accredited
degree
 Association of College and
Research Libraries (ACRL)
Standards for Libraries in
Higher Education (2004)
 Written to “provide a
comprehensive outline to
methodically examine and
analyze all library
operations, services, and
outcomes in the context of
accreditation.”
 Reviewed the most recent
Standards documents of the Six
Regional Agencies
 Collected statements related to
Teaching and/or Learning
 …and library’s support of same
 Identified common themes
 Reviewed ACRL’s Standards
and matched statements to
themes from the Agencies’
Standards
…then,
 We reviewed the LIS literature
for existing models appropriate
for assessing library services re:
the themes we identified.
What we did…
Common
Themes:
While the six
Agencies’
Standards do
differ
significantly, it
was possible
for us to
identify several
shared themes
and objectives:
I. The Institution is Focused on Student Achievement
Subtheme a.
Institution supports learners
Subtheme b.
Institution maintains uniform educational offerings
for all students
Subtheme c.
Institution promotes a life of learning/development of
lifelong learners
Subtheme d.
Faculty are qualified to facilitate and accountable for
student success
II. Teaching and Learning is a Clear Institutional Priority
Subtheme a.
Institution provides adequate support and resources
for teaching and learning
Subtheme b.
Institution Recognizes and Promotes the Scholarship
of Teaching and Learning (SoTL)*
III. The Institution Promotes a “Culture of Assessment”
Subtheme a.
Assessment findings are employed to improve
student success/evidence-informed pedagogy
Subtheme b.
Assessment efforts are consistent, thorough, and
well-documented; standards, tools, and findings are
shared
Common
Themes:
While the six
Agencies’
Standards do differ
significantly, it was
possible for us to
identify several
shared themes and
objectives:
I. The Institution is
Focused on Student
Achievement
II. Teaching and
Learning is a Clear
Institutional
Priority
III. The Institution
Promotes a
“Culture of
Assessment”
I. The Institution is Focused on
Student Achievement
Institution supports learners
Accreditation
Standards: e.g.
“The organization provides an environment that supports all learners
and respects the diversity they bring.”
ACRL Standard? Instruction: “As an academic or instructional unit within the
institution, the library should facilitate student success”
Institution maintains uniform educational offerings for all students
Accreditation
Standards:
e.g.
“The institution provides programs, wherever offered and however
delivered, with appropriate content and rigor that are consistent with its
mission; culminate in achievement of clearly identified student learning
outcomes; and lead to collegiate-level degrees or certificates with
designators consistent with program content in recognized fields of
study.”
ACRL
Standard?
Resources: “Furthermore, distance learning programs should be
supported by equivalent means such as remote electronic access to
collections, the provision of reliable network connections, and electronic
transmission or courier delivery of library materials to remote users.
I. The Institution is Focused on Student Achievement (cont.)
Institution promotes a life of learning/development of lifelong learners
Accreditation
Standards: e.g.
“The organization promotes a life of learning for its faculty, administration,
staff, and students by fostering and supporting inquiry, creativity, practice, and
social responsibility in ways consistent with its mission.”
ACRL Standard?
Instruction: “As an academic or instructional unit within the institution, the
library should facilitate student success, as well as encourage lifelong
learning.”
Faculty are qualified to facilitate and accountable for student success
Accreditation
Standards: e.g.
“Responsibilities of teaching faculty include instruction and the systematic
understanding of effective teaching/learning processes and outcomes in
courses and programs for which they share responsibility”
ACRL Standard?
Instruction: “In addition, librarians should collaborate frequently with
classroom faculty; they should participate in curriculum planning and
information literacy instruction as well as educational outcomes assessment.”
Staff: “Librarians should have a graduate degree from an ALA-accredited
program.
All library professionals should be responsible for and participate in
professional activities.”
Resources: “The library should provide varied, authoritative and up-to-date
resources that support its mission and the needs of its users.”
II. Teaching and Learning is a Clear
Institutional Priority
Institution provides adequate support and resources for teaching and learning
Accreditation
Standards: e.g.
“Budgeting priorities reflect that improvement in teaching and
learning is a core value of the organization.”
ACRL Standard?
Services: “Hours of access to the library should be reasonable
and convenient for its users.”
Staff: “The staff should be sufficient in size and quality to meet
the programmatic and service needs of its primary users.
Facilities: “The library facility and its branches should be well
planned; it should provide secure and adequate space, conducive
to study and research with suitable environmental conditions for
its services, personnel, resources and collections.
The library's equipment should be adequate and functional.”
Budget: “The budget should meet the reasonable expectations
of library users when balanced against other institutional needs.”
II. Teaching and Learning is a Clear
Institutional Priority (cont.)
Institution Recognizes and Promotes the Scholarship of Teaching and Learning
(SoTL)*
Accreditation
Standards: e.g.
“The institution, with significant faculty involvement, engages in
ongoing inquiry into the processes of teaching and learning, as
well as the conditions and practices that promote the kinds and
levels of learning intended by the institution.”
ACRL Standard?
Instruction: “By combining new techniques and technologies
with the best of traditional sources, librarians should assist
primary users and others…”
Resources (indirect): “The library should provide varied,
authoritative and up-to-date resources that support its mission
and the needs of its users.”
III.The Institution Promotes a
“Culture of Assessment”
Assessment findings are employed to improve student success/evidenceinformed pedagogy
Accreditation
Standards: e.g.
“The faculty takes responsibility for evaluating the effectiveness of the teaching and learning process and uses the
results for improvement.”
ACRL Standard?
Outcomes Assessment: “…can be an active mechanism for
improving current library practices.”
Accrediting Commission for Senior Colleges and Universities,
Western Association of Schools and Colleges , Standard 4.6
Creating an Organization Committed to Learning and
Improvement.
III.The Institution Promotes a “Culture of Assessment” (cont.)
Assessment efforts are consistent, thorough, and well-documented; standards, tools,
and findings are shared
Accreditatio
n Standards:
e.g.
“Leadership at all levels is committed to improvement based on the results of the
inquiry, evaluation and assessment that is used throughout the institution.”
“An accredited institution is expected to possess or demonstrate the following
attributes or activities:
…a documented, organized, and sustained assessment process to evaluate and
improve student learning… evidence that student learning assessment information is
shared and discussed with appropriate constituents”
ACRL
Standard?
Outcomes Assessment: “Outcomes assessment will increasingly measure and affect
how library goals and objectives are achieved. It will address the accountability of
institutions of higher education for student achievement and cost effectiveness.”
Instruction: “Librarians should participate in… educational outcomes assessment”
Resources: “Policies regarding access should be appropriately disseminated to library
users.”
Outcomes Assessment (indirect): “The instrument should be valid, and the way it is
used should be appropriate for the task. Colleagues at peer institutions may render
invaluable assistance by suggesting assessment questions and sample sizes, by
sharing lessons learned, and suggesting alternative methods for measuring
outcomes.”
*Focus: Library Support for Teaching and the
Scholarship of Teaching and Learning (SoTL)
 Concept introduced by Ernest Boyer in Scholarship
Reconsidered: Priorities of the Professoriate (1990)
 Boyer pointed out that faculty are rarely afforded the same
respect for excellence in teaching as they are for successful
research.
 Boyer called for abandoning “the old `teaching versus research’
debate” in favor of acknowledging the scholarly aspects of
teaching by giving “the familiar and honourable term
`scholarship’ a broader, more capacious meaning, one that brings
legitimacy to the full scope of academic work.”
 Lee Shulman credited with expanding and clarifying the
dimensions of the concept.
Tenets of SoTL:
 Application of principles and methods of scholarship to the
questions, problems and challenges of teaching
 Serious study of teaching within disciplinary context
 Empirical investigation
 Dissemination of findings in same venues appropriate for other
discipline-specific research (i.e., not confined to teaching-related
publications and conferences)
 Reflective teaching practice
 Peer review of teaching effectiveness
 Continuous assessment and improvement
 Application of findings from original research, peer assessment,
and other modes of study to improving teaching
“The SoTL movement
provides excellent
opportunities for librarians,
both in respect to developing
their own projects and for
supporting, and developing
relationships with, faculty
working on SoTL projects.
Despite this potential, there
has been very little written
about SoTL in the
professional library
literature.”
SoTL presents a
natural fit for
academic libraries…
…so why have
librarians been so
slow to pick up on it?
Specifically…
 Application of principles and methods of scholarship to the questions,
problems and challenges of teaching
 Creation of collections to support scholarly inquiry into teaching and
learning
 Hosting events related to SoTL, including workshops and conferences
 Engaging in SoTL-related research into librarians’ own instruction
 Reflective teaching practice
 Providing support and facilities for faculty and other instructors to
engage in observation and review of teaching
 Librarians engaging in peer review of own instruction
 Continuous assessment and improvement
 Application of findings from original research, peer assessment, and
other modes of study to improving teaching (in classroom and library
instructional settings)
Next Steps?
 For Libraries (our suggestions):
 Conduct in-depth review of
relevant regional accreditation
standards
 Identify standards (beyond
those that address the library
specifically) related to teaching
and learning
 Develop an action plan for
efforts to address and support
those priorities
 Communicate efforts to
stakeholders in the institution
and beyond.
 For us?
 Themes discusses in this paper
will provide a framework for a
series of studies at UTK related
to library support for teaching
and learning
 Instructor surveys
 Instructor interviews/focus
groups
 Experimental study of
impact of library instruction;
multiple sections of
undergraduate required
classes
Thank you for
your time!
Questions?
Rf-m@utk.edu
Note: full references for works cited in presentation are available in our paper.