LOEX 2007
Applying Student
Learning
Data to Enhance
Instruction
An Anteater Initiative
Cathy Palmer
Head, Education and Outreach
UC Irvine Libraries
LOEX 2007
Outline
 Background
Program Overview
Incentives/Challenges
Methodology
Results
Lessons Learned
 What next?





LOEX 2007
Writing Program/Library Collaboration
 Lower-division writing requirement
 Fulfilled in part by successful completion
of Writing 39C: Argument and Research
 Course integrated library research
assignment since 1986
 Library research assignment evolved as
course curriculum evolved
Library Incentives for Evaluation
 Major investment of library resources
• Personnel
• Space
 Availability of numeric data
• Number of students reached
• Number of sessions taught
• Number of librarian hours spent
 Were our resources being put to the best
use?
Yearly Statistics
1996/1997
2005/2006
• 1727 Students
• 2504 Students
• 76 Sessions
• 126 Sessions
• 82 Hours
• 150 Hours
Session Outline
 Libraries Homepage (Lecture/Demo)
• Ask a Librarian
• Connect from off-campus
 ANTPAC catalog (Lecture/Demo)
 Expanded Academic (Lecture/Demo)
 Hands-on exploration of assigned databases
• Sociological Abstracts
• L/N Congressional Universe
• United States Newspapers
 Review of each database (Discussion)
 Evaluations
Evaluation 1.0
Overall Value of Session to Student
Very Useful
Useful
Somewhat Useful
Not Useful
No Response
Total
591
44%
621
115
10
0
1338
46%
9%
1%
0%
100
Student rating of Librarian Presentation
Understand Clearly
616 50%
Understand
Understand
Somewhat
Don't Understand
No Response
525 43%
2 0%
0
84
0%
7
Choose Databases—student self rating
Understand Clearly
529 40%
Understand
Understand Somewhat
Don't Understand
No Response
702 52%
69 5%
3 0%
35 3%
Select Search Terms-student self rating
Understand Clearly
523 39%
Understand
Understand Somewhat
Don't Understand
No Response
618 46%
69 5%
2 0%
126 9%
Evaluate Results of Search-self rating
Understand Clearly
593
44%
Understand
Understand
Somewhat
Don't Understand
No Response
530
79
40%
6%
6
130
0%
10%
Interpret Citations-Find
Material-self-rating
Understand Clearly
457
34%
Understand
Understand
Somewhat
Don't Understand
No Response
591
145
44%
11%
11
134
1%
10%
But...
What were the
students REALLY
learning in the
session?
The "Eureka!"
moment
Esther Grassian workshop
Ask the students
questions about the
content of the session
as part of the
evaluation.
The BIG IDEA
Explicit Objective:
Students will understand that they have a
variety of information resources available to
them and that each resource has strengths
and weaknesses. They will be challenged to
answer the question "Why use this resource
rather than that resource?"
The Invisible BIG IDEAs
Implicit Objectives:
Students will leave knowing that the
librarians are there to help them.
Involve students in their own learning by
making the session as hands-on as possible.
Learning Outcomes
At the end of the 50 minute session,
students will:
 Recognize the characteristics of generalized and
specialized information resources and be able to
differentiate between them.
 Learn and apply basic search strategies that will
help them locate information
 Select, explore and use 3-5 library databases to
find information on their topic.
Session Outline
 Libraries Homepage (Lecture/Demo)
• Ask a Librarian
• Connect from off-campus
 ANTPAC catalog (Lecture/Demo)
 Expanded Academic (Lecture/Demo)
 Hands-on exploration of assigned databases
• Sociological Abstracts
• L/N Congressional Universe
• United States Newspapers
 Review of each database (Discussion)
 Evaluations
Early Results
Read each statement and indicate the best resource
to use by placing a check in the appropriate column.
(Fall, 2003—383 responses)
ANTPAC
Find books
Expanded
Academic
ASAP
UCelinks
38%
53%
Get help
Find
electronic
copy
Ask a
Librarian
62%
Find
scholarly
articles
Find
scholarly
and popular
articles
Sociological
Abstracts
77%
62%
Another
Eureka Moment
Let's use Sociological
Abstracts as the
demonstration
database instead of
Expanded Academic
ASAP.
Read each statement and indicate the best resource
to use by placing a check in the appropriate column.
(Spring 2004, 809 evaluations)
ANTPAC
Find books
Find scholarly
articles
Find scholarly and
popular articles
Expanded
Academic
ASAP
Sociological
Abstracts
UCelinks
73.9%
48.3%
52.7%
55.8%
Get help
Find electronic
copy
Ask a
Librarian
74.7%
64.4%
Student Performance Comparison
Fall, 2003
Spring, 2004
Use ANTPAC to find books
61.8
73.9
Use Soc Abs to find
scholarly articles
Use Expanded Academic to
find popular and scholarly
Ask a Librarian for help
38
52.7
53
54.8
76.6
74.7
62.1
64.4
Use UCeLinks to find
electronic copies
Bonuses
Ability to look for themes in student
comments
What parts of the presentation helped you
the most?
 Following along step by step (71)
 Visual aid/demonstration (73)
 Group work (83)
Student comments cont.
What helped you the least?
 Group work (59)
 Not enough time (38)
 Nothing/everything helped (202)
What could improve the session?
 Longer session (184)
 Slower and more detail about resources (58)
 Speak louder/microphone
Still more student comments
I still need to learn more about:
 What each database is used for and when to use
it for certain searches (78)
• The other databases we didn't get to explore
(59)
• How to find the actual copies of books and
articles in the library (34)
Problems with Paper
ANTPAC
Expanded
Academic
ASAP
Sociological
Abstracts
UCelinks
Ask a
Librarian
Find books
X
X
X
X
X
Find
scholarly
articles
X
X
X
X
X
Find
scholarly
and popular
articles
X
X
X
X
X
Get help
X
X
X
X
X
Find
electronic
copy
X
X
X
X
X
Problems with Paper
ANTPAC
Find books
X
Find
scholarly
articles
X
Find
scholarly
and popular
articles
X
Get help
Find
electronic
copy
Expanded
Academic
ASAP
X
Sociological
Abstracts
X
UCelinks
X
X
X
X
X
X
X
X
X
X
Ask a
Librarian
X
X
Problems with Paper
ANTPAC
Find books
Expanded
Academic
ASAP
X
X
Find
scholarly
and popular
articles
X
Find
electronic
copy
X
UCelinks
X
Find
scholarly
articles
Get help
Sociological
Abstracts
X
X
X
Ask a
Librarian
X
X
X
X
X
X
X
X
Benefits of Automating
Evaluation Form




Could only select one response
More comments/longer comments
Results available immediately
Results could be sorted in a variety of
ways
Numeric Results
 Individual Librarian/Individual Session
 Individual Librarian/All Sessions
 Overall evaluations by quarter
 Overall evaluation by academic year
Individual Performance Improvements
Assessment/Feedback methodology
 Use of Observation Rubric
– Presentation style
• Use of projected image
• Can be heard
– Session Content
• Outlines session
• Pacing
– Classroom management
• Interaction with students
• Execution of learning techniques
 Comparison between Observation and
Evaluations
Results
Clear, concise presentation
+
Hands-on
+
Review of experience
Best performance by students on
selection of best resource
Example
Another Example
Librarians: All
Academic year: 2006 - 2007
Total evaluations: 1733
Q1. Session Content
The Library
Research Skills
session was useful.
I learned new skills.
Strongly agree:
873
(50%)
Agree:
778
(45%)
Neutral:
64
(4%)
Disagree:
9
(1%)
No opinion:
1
(0%)
Strongly agree:
907
(52%)
Agree:
713
(41%)
Neutral:
78
(5%)
Disagree:
18
(1%)
3
(0%)
No opinion:
Librarians: All
Academic year: 2006 - 2007
Total evaluations: 1733
Q2. Instructors
The librarian
Strongly agree:
presented
information in a way Agree:
that I could
Neutral:
understand.
Disagree:
970
(56%)
689
(40%)
42
(2%)
15
(1%)
4
(0%)
Strongly agree:
960
(55%)
Agree:
636
(37%)
Neutral:
105
(6%)
No opinion:
8
(0%)
Disagree:
8
(0%)
No opinion:
The librarian
encouraged and
responded to
questions.
Q3. Choose the Best Resource
A. I can use this resource to
find out if the UCI Libraries
own the book Reckoning
with Homelessness.
D. I can get help with my
topic by using this resource.
Antpac:
1315
(76%)
134
(8%)
Ask a Librarian:
74
(4%)
Sociological Abstracts:
84
(5%)
UC-eLinks:
73
(4%)
Antpac:
165
(10%)
ASAP:
176
(10%)
1160
(67%)
Sociological Abstracts:
76
(4%)
UC-eLinks:
99
(6%)
Antpac:
334
(19%)
ASAP:
197
(11%)
Ask a Librarian:
46
(3%)
Sociological Abstracts:
50
(3%)
1042
(60%)
ASAP:
Ask a Librarian:
E. I can use this online
resource to see if there is an
electronic copy of an article
available.
UC-eLinks:
Q3. Choose the Best Resource
B. I can use this online
resource to locate scholarly
peer-reviewed journal
articles on the topic of
homelessness.
Antpac:
160
(9%)
ASAP:
660
(38%)
21
(1%)
779
(45%)
52
(3%)
211
(12%)
1078
(62%)
25
(1%)
287
(17%)
77
(4%)
Ask a Librarian:
Sociological Abstracts:
UC-eLinks:
C. This is a good resource to
use when I am beginning my
research because it covers
both scholarly journals and
popular magazines.
Antpac:
ASAP:
Ask a Librarian:
Sociological Abstracts:
UC-eLinks:
Future Wishlist
 Pre and post testing
 Ability to share results of evaluation with
students
 More nuanced testing of ability to apply
concepts learned
 Consistent evaluation of learning
outcomes from other library research
skills sessions
Questions?
Contact Me
Cathy Palmer
cpalmer@uci.edu
949-824-4972