Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Lynn Silipigni Connaway and Marie L. Radford QuestionPoint Users Group Meeting June 25, 2006 New Orleans, Louisiana Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives • $1,103,572 project funded by: – Institute of Museum and Library Services $684,996 grant – Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Project duration 10/1/2005-9/30/2007 Four phases: I. Focus group interviews* II. Analysis of 1,000+ QuestionPoint transcripts III. 600 online surveys* IV. 300 telephone interviews* *Interviews & surveys with VRS users, non-users, & librarians Phase II: 24/7 Transcript Analysis • Generated random sample – – – • July 7, 2004 through June 27, 2005 263,673 sessions 25 transcripts/month = 300 total 256 usable transcripts – Excluding system tests and technical problems 6 Analyses • Geographical Distribution • Library receiving query • • • Library answering query Type of Library Type of Questions • Katz/Kaske Classification • Subject of Questions • Dewy Decimal Classification Session Duration •Interpersonal Communication • • Radford Classification Librarian Location - Question Received California = 77 Maryland = 47 Australia = 36 Massachusetts = 21 North Carolina = 14 Utah = 8 Washington = 8 New York = 7 Canada = 7 Delaware = 6 Kansas = 5 Arizona = 4 Pennsylvania = 4 United Kingdom = 1 Other States = 10 0 n=255 10 20 30 40 50 Number of Questions 60 70 80 90 Librarian Location - Question Referred/Answered California = 88 Australia = 36 Maryland = 35 Massachusetts = 10 Connecticut = 9 New York = 8 North Carolina = 7 Washington = 7 Canada = 7 Hawaii = 6 Colorado = 4 Michigan= 4 Pennsylvania = 4 Germany = 1 Other states = 12 0 n=238 10 20 30 40 50 60 Number of questions 70 80 90 100 Type of Library Receiving Question 90 80 Number of Questions 70 60 50 40 30 20 10 0 Consortium n=256 Public University National Law State Not Available K-12 Type of Question Asked Holdings 6% Inappropriate 0% Subject Search 37% Procedural 25% Research 2% Ready Reference 30% n=273 Procedure and Subject Compuer science, information & general works Philosophy & 4% psychology 1% Library procedure & miscellaneous 25% Religion 0% Social sciences 31% History & geography 16% Language 1% Literature 5% Arts & recreation 4% n=273 Technology 5% Science 8% Dewey Decimal Classification Compuer science, information & general works 5% Philosophy & psychology 1% History & geography 21% Religion 0% Literature 6% Arts & recreation 5% Technology 7% n=273 Science 11% Language 2% Social sciences 42% Service Duration • Mean Service Duration: 13:53 • Median Service Duration: 10:37 Focus Group Interviews Reasons for Using VRS • • • • • • Convenient Efficient More reliable than search engines & free Allows multi-tasking Email follow-up & provision of transcript Pleasant interpersonal experience – Librarian on first name basis – more personalized • Less intimidating than physical reference desk – Feel comfortable abruptly ending session Focus Group Interviews Reasons for not using VRS • Graduate students – Fear of • Bothering librarian • Looking stupid & advisors finding out – Questions may not be taken seriously – Potential technical problems – Bad experiences in FtF influence expectations of VRS • Screenagers – Virtual stalkers (“psycho killers”) – Not finding a trusted librarian – Unsure of what to expect Focus Group Interviews Challenges for Users & Non-Users • Speed and technical problems • Delayed response time • Librarians are not in users’ libraries – Fear of no subject expertise • Fear of overwhelming librarian Focus Group Interviews Suggestions from Users & Non-Users • Inclusion of multiple languages • Access to subject specialists • Better marketing and publicity – Information on how to connect and use VRS – Reassurance that users will not bother librarians – the library wants the service to be used • Faster technology • Improved interface design – More color – More attractive Next Steps • Conduct – Three focus group interviews – VRS users – Online survey & telephone interviews with VRS • Users • Non-users • Librarians • Analyses – Gender – User Type • Child/Young adult • Adult • Unknown End Notes • • • This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators. Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center. Project web site: http://www.oclc.org/research/projects/synchr onicity/ Questions Marie L. Radford, Ph.D. Email: mradford@scils.rutgers.edu www.scils.rutgers.edu/~mradford Lynn Silipigni Connaway, Ph.D. Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm