Virtual Windows:
Observing Chat Reference Encounters
through Transcript Analysis
Lynn Silipigni Connaway, Ph.D.,
Marie L. Radford, Ph.D.
Lawrence Olszewski, Ph.D.
19 IAPS International Conference
September 11-16, 2006
Alexandria, Egypt
Morphing Idea of Library
• Proliferation of digital libraries
– Virtual Reference Services (VRS)
– Electronic collections
• User Preferences
– Sources
• Internet
– Electronic sources
• Humans
– Parents
– Colleagues/Friends
– Professors
– Interface design
• Google-like
• Amazon
Privacy and Confidentiality
• Traditional reference
(FtF and Telephone)
– Anonymity and privacy assumed
• VRS
– Verbatim transcripts allow unobtrusive
research opportunities
– Transcripts provide physical evidence of
session
Privacy and Confidentiality
• Known identity of user
– Authenticate and improve service
– Identify repeat user
– Send follow-up information
• Nature and subject of query
– Sensitive questions
• Medical
• Legal
• Personal situations
– Confidentiality of all queries should be
respected
Evaluation of VRS
• Sustainability of VRS
– Factors that influence selection and use of
VRS
– Behavior of users and librarians in VRS
sessions
– User and librarian perceptions of satisfaction
Seeking Synchronicity:
Evaluating Virtual Reference Services from
User, Non-User, and Librarian Perspectives
• $1,103,572 project funded by:
– Institute of Museum and Library Services
$684,996 grant
– Rutgers, The State University of New Jersey
and OCLC Online Computer Library Center
$405,076 in kind contributions
Seeking Synchronicity:
Evaluating Virtual Reference Services from
User, Non-User, and Librarian Perspectives
Project duration
10/1/2005-9/30/2007
Four phases:
I. Focus group interviews*
II. Analysis of 1,000+ QuestionPoint
transcripts
III. 600 online surveys*
IV. 300 telephone interviews*
*Interviews & surveys with VRS users, non-users, & librarians
Seeking Synchronicity:
Evaluating Virtual Reference Services from
User, Non-User, and Librarian Perspectives
•
Identify what individuals say they do
Focus group interviews
– Online surveys
– Telephone interviews
–
•
Identify what individuals actually do
–
Transcript analysis
Phase II:
24/7 Transcript Analysis
•
Generated random sample
July 7, 2004 through June 27, 2005
– 263,673 sessions
– 25 transcripts/month = 300 total
–
•
256 usable transcripts
–
Excluding system tests and technical
problems
6 Analyses
– Geographical Distribution
•
•
–
–
Type of Library
Type of Questions
•
–
Katz/Kaske Classification
Subject of Questions
•
–
–
Library receiving query
Library answering query
Dewey Decimal Classification
Session Duration
Interpersonal Communication
•
Radford Classification
Librarian Location - Question Received
California = 77
Maryland = 47
Australia = 36
Massachusetts = 21
North Carolina = 14
Utah = 8
Washington = 8
New York = 7
Canada = 7
Delaware = 6
Kansas = 5
Arizona = 4
Pennsylvania = 4
United Kingdom = 1
Other States = 10
0
10
20
30
40
50
Number of Questions
n=255
60
70
80
90
Librarian Location - Question Referred/Answered
California = 88
Australia = 36
Maryland = 35
Massachusetts = 10
Connecticut = 9
New York = 8
North Carolina = 7
Washington = 7
Canada = 7
Hawaii = 6
Colorado = 4
Michigan= 4
Pennsylvania = 4
Germany = 1
Other states = 12
0
n=238
10
20
30
40
50
60
Number of questions
70
80
90
100
Type of Library Receiving Question
90
80
Number of Questions
70
60
50
40
30
20
10
0
Consortium
n=256
Public
University
National
Law
State
Not
Available
K-12
Type of Question Asked
Holdings
6%
Inappropriate
0%
Subject Search
37%
Procedural
25%
Research
2%
Ready Reference
30%
n=273
Procedure and Subject
Compuer science,
information & general
works
Philosophy &
4%
psychology
1%
Library procedure &
miscellaneous
25%
Religion
0%
Social sciences
31%
History & geography
16%
Language
1%
Literature
5%
Arts & recreation
4%
n=273
Technology
5%
Science
8%
Dewey Decimal Classification
Compuer science,
information & general
works
5% Philosophy &
psychology
1%
History & geography
21%
Religion
0%
Literature
6%
Arts & recreation
5%
Technology
7%
n=273
Science
11%
Language
2%
Social sciences
42%
Service Duration
• Mean Service Duration:
13:53
• Median Service Duration:
10:37
Positive Transcript Example
U Where can I find the leading drug companies
in boston doing diabetes treatment / prevention
R&D?
L I can probably give you a few sources to get
started, but I may wind up referring you to a
business and/or medial librarian specialist.
L Let's start witht eNortheastern library web
page...
U ok great thanks
Positive Transcript Example
L OK. I'm going to try the "co-browse" option -- that might
let us see the same information at once...(if it's
working!)
U wonderful
L since what you want to find are drug companies, I'll try
to get you into a busienss database...
U perfect thank you
L Sorry, I thought there was a way you could search by
sic code and get a ranked list of companies in a certian
code.
U thats alright, seemed liek you were on the right
track
Negative Transcript Example
U
Which way is ur car accelerating when you’re thrown
forward after hitting another bumper car?
L
Is this a homework question.
L
U
I'm not an expert on driving so I really can't
answer that
can u find a website or something
L
I'm not sure what you are asking.
Negative Transcript Example
U
L
U
L
…hello?
I really don't understand how I can answer
that for you.
can i hav another librarian
The information you gave you me does not
help me find any resources to help you.
Focus Group Interviews
Reasons for Using VRS
•
•
•
•
•
•
Convenient
Efficient
More reliable than search engines & free
Allows multi-tasking
Email follow-up & provision of transcript
Pleasant interpersonal experience
– Librarian on first name basis – more personalized
• Less intimidating than physical reference desk
– Feel comfortable abruptly ending session
Focus Group Interviews
Reasons for not using VRS
• Graduate students
– Fear of
• Bothering librarian
• Looking stupid & advisors finding out
– Questions may not be taken seriously
– Potential technical problems
– Bad experiences in FtF influence expectations of VRS
• Screenagers
– Virtual stalkers (“psycho killers”)
– Not finding a trusted librarian
– Unsure of what to expect
Focus Group Interviews
Challenges for Users & Non-Users
• Speed and technical problems
• Delayed response time
• Librarians are not in users’ libraries
– Fear of no subject expertise
• Fear of overwhelming librarian
Focus Group Interviews
Suggestions from Users & Non-Users
• Inclusion of multiple languages
• Access to subject specialists
• Better marketing and publicity
– Information on how to connect and use VRS
– Reassurance that users will not bother librarians – the
library wants the service to be used
• Faster technology
• Improved interface design
– More color
– More attractive
Service Implications
• Sustainability of VRS
– Encourage repeat use
• Protect privacy and anonymity
–
–
–
–
Encryption programs
“Anonymity Button”
Opt-out after registering
Opt-in only for necessary information
– Sales/Homework Help models
• Build interpersonal relationships
– Disclose first name - trusted librarian
– Positive relational communication
– Trade-offs in service
• Personal service vs. personal disclosure
– Follow-up capability vs. anonymity
– Amazon-like services vs. protection of personal information
Conclusions
• Current service models do not address
privacy issues
– Millennial generation wary of virtual
environments
• Remote communication poses less
interpersonal risk than FtF
• Positive interpersonal communication
imperative for VRS success
• Many users appreciate convenience
and immediacy of VRS
Next Steps
• Conduct
– Two additional focus group interviews – VRS users
– Online survey & telephone interviews with VRS
• Users
• Non-users
• Librarians
• Analyses
– Gender
– User Type
• Child/Young adult
• Adult
• Unknown
End Notes
This is one of the outcomes from the project
Seeking Synchronicity: Evaluating Virtual
Reference Services from User, Non-User, and
Librarian Perspectives, Marie L. Radford and
Lynn Silipigni Connaway, Co-Principal
Investigators.
Funded by IMLS, Rutgers University and OCLC,
Online Computer Library Center.
Project web site:
http://www.oclc.org/research/projects/synchr
onicity/
Questions
Marie L. Radford, Ph.D.
Email: mradford@scils.rutgers.edu
www.scils.rutgers.edu/~mradford
Lynn Silipigni Connaway, Ph.D.
Email: connawal@oclc.org
www.oclc.org/research/staff/connaway.htm
Lawrence Olszewski, Ph.D.
Email: olszewsl@oclc.org