LibQUAL+ Survey Choice September 2009 Annie Bélanger LibQUAL+ now has multiple survey versions: the standard LibQUAL+, which was used in 2001 and 2007 LibQUAL+ Lite, a new abbreviated version of the standard survey 50/50 Standard and LibQUAL+ Lite In December, members of the LibQUAL+ Canada Consortium will need to decide which survey version they will be using. In preparation for this and our project work, I am submitting an overview of the surveys and my recommendation. Much of the information below was pulled from the LibQUAL+ Canada Consortium site. What is LibQUAL+ Lite? The Lite protocol uses item sampling methods to (a) gather data on all 22 LibQUAL+® core items, while (b) only requiring the user to respond to a subset of the 22 core questions. Sampling LibQUAL+™ Lite uses matrix sampling. Matrix sampling is a survey method that can be used to collect data on all survey items without requiring every participant to react to every survey question. With this approach, all of the LibQUAL+™ questions will still be asked but not of every respondent. For more information: The mechanics of item sampling strategy and results from pilot testing are described in a recent refereed journal article: http://www.libqual.org/documents/admin/pmm10-1_LQlite.pdf. 1 Questions & Respondents No LibQUAL+™ Lite respondent will have to answer more than 19 questions (not counting demographic items) as opposed to 34 questions for the full LibQUAL+™ survey. 1. Basic Questions 3 questions will be common to all the LibQUAL+™ Lite surveys (one from each of the three service dimensions). 8 questions will be randomly selected by the LibQUAL+™ system, for each respondent in the following manner: o 2 questions randomly selected from the remaining 8 Affect of Service questions o 2 questions randomly selected from the remaining 7 Information Control questions o 1 question randomly selected from the remaining 4 Library as Place questions 2. Optional Questions 1 question randomly selected from the library's 5 optional questions (if the library chooses to include 5 optional questions) 3. General Satisfaction questions 2 questions randomly selected from the General Satisfaction questions 4. Information Literacy Outcomes questions 2 questions randomly selected from the Information Literacy Outcomes questions 5. Library Use questions All 3 Library Use questions 6. Comments The comments box Preliminary Findings With LibQUAL+™ Lite ARL's findings to date indicate a large increase in the percentage of respondents who complete the survey with LibQUAL+™ Lite. However, ARL's analysis of the test libraries' results indicate that there is some difference in the mean scores between LibQUAL+™ Lite respondents and full LibQUAL+™ respondents. Since there are significantly more respondents for LibQUAL+™ Lite, ARL's researchers (Bruce Thompson, Martha Kyrillidou and Colleen Cook) concluded that the aggregate mean scores for the LibQUAL+™ Lite results may, in fact, be more accurate: (Thompson, B., Kyrillidou, M., & Cook, C. (2009). Item sampling in service quality assessment surveys to improve response rates and reduce respondent burden: The "LibQUAL+ Lite" example. Performance Measurement & Metrics, 10(1), 6-16. ). 2 LibQUAL+ vs. LibQUAL+ Lite Pros Cons LibQUAL+ All respondents answer all questions May have the best correlation with our 2007 survey results Lower response rate Lower completion rate – uncompleted surveys are not included in our results Longer time commitment for respondents – estimated at 15 minutes o this has received significant negative feedback in 2001 and 2007 LibQUAL+ Lite Shorter time commitment for respondents – estimated at less than 10 minutes Higher response rate Optional questions are still an option Research shows a more accurate mean due to response rate Not all respondents answer all questions Risk of less direct correlation with our 2007 survey results 50/50 Some respondents have a shorter time commitment Some respondents have to answer all questions May have the better correlation with our 2007 survey results than the Lite version only Respondents answering the full survey may be frustrated by the length of the survey Respondents answering the full survey may not complete the full survey – uncompleted surveys are not included in our results CARL Recommendation To minimize variations in comparing results among participating libraries, CARL asks that consortial participants choose either 50/50 LibQUAL+™ Lite and full LibQUAL+™ option or a 100% LibQUAL+™ Lite option. ARL Recommendation For libraries who have done the survey before, ARL recommend a 50% distribution between Lite and full surveys. This would allow those libraries to compare their 2010 results with their past results all of the survey questions. Libraries new to LibQUAL+™ might as well select 100% LibQUAL+™ Lite. My Recommendation In light of this information, I would like to recommend that we use the 100% LibQUAL+ Lite option. 3