Assessing the Information Literacy Skills of New Freshmen at SIUE
Submitted by
Julia Hansen and Lydia Jackson
1
Introduction and Significance
Information literacy is defined as a set of abilities requiring individuals to “recognize
when information is needed and have the ability to locate, evaluate, and use effectively
the needed information.” Information literacy is inextricably linked to higher education’s
commitment to educating students for lifelong learning. Indeed, the phrase, lifelong
learning has become standard in many higher education mission statements. At SIUE it
serves as one of the guiding principles of our institution and can be found as one of the
supporting ideas under the SIUE Value – Wisdom, adopted by our University in 2002.
Lifelong learning is also identified as one of the pursuits of our engaged and capable
graduates, in the opening paragraph of SIUE’s “Statement of Objectives for the
Baccalaureate Degree.” In order to achieve such purposes, the “Objectives” describes
the essential abilities and knowledge we seek to impart to our students. Information
literacy is recognized as one of the critical “Analytic, Problem-solving and Decisionmaking Skills.” Furthermore, the SIUE faculty have determined that “these skills are to
be developed throughout all undergraduate programs in all courses.” Clearly, the SIUE
faculty support the contention of contemporary educators, that “within today’s
information society, the most important learning outcome for all students is their being
able to function as independent lifelong learners. The essential enabler to reaching that
goal is information literacy.” (Brevik, 2001)
The Boyer Commission Report, Reinventing Undergraduate Education: A Blueprint for
America’s Research University, recommends strategies that require the student to engage
actively in the “framing of a significant question or set of questions, research or creative
exploration to find answers, and the communications skills to convey the results.”
Courses including these skills create student-centered learning environments where
inquiry is the norm, problem-solving becomes the focus, and thinking critically is part of
the process. Proactive learning environments require information literacy competence.
Working in collaboration with the classroom faculty, Lovejoy Library’s faculty and staff
are uniquely prepared and positioned to help advance the literacy competency of the
SIUE community.
Lovejoy Library has a very vigorous and dynamic instruction program. However we
have observed freshmen students who are often ill-prepared to apply the necessary
research skills required to succeed in their courses. Awareness among classroom faculty
of students’ academic preparedness in the area of information literacy skills is often low.
Assessing the information literacy skills of new freshmen, will provide us with the
foundation we need to design better learning experiences for our students as they
undertake college level work. We plan to administer an assessment survey to students
enrolled in New Freshman Seminar (NFS) courses. One of the stated goals of the NFS is
to “assist new freshmen in making the transition to college level work and expectations.”
(SIUE, New Freshman Seminar) The results of the survey will help us to determine if the
participants demonstrate the information literacy skills necessary to successfully meet the
expectations of the institution and the faculty.
2
There are many published articles on information literacy, lists of competencies, and
descriptions of information literacy programs and courses. However, there are limited
assessment tools that measure student competencies. Most focus on evaluating library
instruction. For example, pre-test/post-test measures assess the desired effect of particular
library instruction sessions but they do not test general information competence.
Furthermore, user surveys rating satisfaction and describing library use do not provide
objective skills assessment. Thus, one of the problems to be addressed in this study is the
creation of an assessment tool that will adequately assess the information literacy abilities
of SIUE freshmen. To obtain an objective measure of their information literacy skills, we
propose conducting an assessment project. We will create a list of competencies based on
standards established by the Association of College and Research Libraries (ACRL)
(Appendix A) and design a survey instrument which will be administered to a random
sample of new freshmen in the fall of 2007.
The main goal of the project will be to identify a baseline or benchmark of information
literacy skills new freshmen students demonstrate prior to receiving formal library
instruction in the SIUE environment. A practical objective of the study is to obtain data
to use in discussions with faculty about students’ information and research skills, the
impact of those abilities on students’ coursework, and the potential of library instruction
to improve them. We believe that the data obtained on the information competencies of
our freshmen, will extend the discussion of information literacy goals for all
undergraduates that has been stimulated by the recent BRIDGE project.
This research builds on a study funded by a Library Services and Technology Act
(LSTA) grant, Building a Framework for Partnership: Collaboration Among the
Libraries of Madison County (2005). This grant allowed us to begin to partner with local
school librarians to strengthen and enhance information literacy education at the precollege level. We believe that assessing the information literacy skills of new freshmen
enrolled at SIUE will dovetail nicely with the collaborative efforts begun with local area
school librarians. We will be able to provide those educators with a better sense of the
range of information literacy skills students demonstrate as they begin their college
experience at SIUE. If we can identify the information literacy competencies necessary
for successful college level work, middle and high school area librarians can work within
their schools to better prepare students for the challenges ahead.
Literature Review
Jeremy J. Shapiro and Shelley K. Hughes provide good insight into current attempts to
define information literacy in their article Information Literacy as a Liberal Art.
According to the authors: in its narrowest sense, information literacy includes the
practical skills involved in effective use of information technology and print or electronic
information resources. Information literacy is a new liberal art which extends beyond
technical skills and is conceived as one's critical reflection on the nature of information
itself, its technical infrastructure and its social, cultural, and even philosophical context
and impact.
The information literacy curriculum includes:
3





Tool literacy - The ability to use print and electronic resources including
software.
Resource literacy - The ability to understand the form, format, location and
access methods of information resources.
Social-structural literacy - Knowledge of how information is socially situated
and produced. It includes understanding the scholarly publishing process.
Research literacy - The ability to understand and use information technology
tools to carry out research, including discipline-related software.
Publishing literacy - The ability to produce a text or multimedia report of
research results.
Computer literacy – The ability to use applications rather than programs.

.
The lists of information literacy competencies in print and on academic library Web sites
are too numerous to cite; however, several Web sites from the California State University
(CSU) libraries are worth noting. The CSU libraries are recognized for the substantial
work they have accomplished in the area of information literacy. (The California State
University Information Competence Initiative) In her article, Educating the Cut and
Paste Generation, Lorie Roth describes information competence assessment initiatives at
several universities in California, including a study at CSU, San Marcos, to measure
student attainment in information literacy over a four-year period. (pp. 42-44) Susan
Carol Curzon’s work, Developing a Program of Information Literacy, covers the CSU
Information Competence Skills Assessment, a telephone survey in which 3,000 students
responded to hypothetical research and information-seeking scenarios. (p. 483)
Other academic libraries have undertaken related assessments and have developed their
own instruments. An effort to compare student self-assessment of information skills
with objective evaluation was described by Greer, Weston, and Alm. (pp. 549-57) Two
surveys at Johns Hopkins University compared freshmen and upperclassmen at that
institution, focusing first on evaluating basic library skills and then adding some
advanced skills. (pp. 188-201) The Indiana University Bloomington Libraries
Assessment Plan for Information Literacy, while not offering a specific assessment tool,
describes a plan for assessing information literacy and articulates specific objectives and
possible measurement techniques for each information literacy goal. (Indiana University
Bloomington Libraries Assessment Planning Committee) Kent State librarians reported
on a pretest used to assess basic library skills primarily of freshmen enrolled in an
English II course. (Kunkel, pp. 431-433) The Teaching Library at the University of
California, Berkeley, first surveyed graduating seniors in the political science and
sociology departments in March-May 1994. They have repeated the survey with students
in these and other disciplines. (Maughan, 2000)
4
Objectives of the Proposal





To gain an understanding of information competencies of new freshmen at SIUE.
To work with colleagues across the University to establish information
competency standards for SIUE. What skills do students need to be successful?
To improve the effectiveness of library instruction at SIUE by modifying current
instructional practices and strategies
To promote campus wide discussions on information literacy needs and goals
To build on the efforts of the Information Literacy Across the Curriculum (ILAC)
grant funded by EUE, 2006.
Assumptions
The overall goal of our project is to assess a baseline of information literacy skills
when freshmen enter the college environment. We make the following assumptions
regarding the information literacy needs of freshmen:
1. Students in all disciplines are expected to employ sophisticated information
gathering techniques for their coursework.
2. Freshmen must learn critical thinking and research skills in their disciplines as
preparation for a lifetime of changing information needs.
3. Their most effective learning experience in the library is usually connected to a
specific information need and is often discipline specific. Examples include the
preparation of a research paper or gathering documentation for a persuasive
speech.
4. The information environment is too complex and is changing too rapidly to expect
freshmen to acquire information literacy without a planned, systematic, and
cumulative instructional program. The hit-or-miss strategy that worked for
students and scholars in the past is not efficient or effective.
We plan to include indicators of exposure to library experiences in our assessment tool.
We anticipate the following correlations:



Students who were required to do research in high school will score higher than
students who were not.
Students who have visited the library frequently (at least once a week) to use its
resources will score higher than students who did not visit the library frequently.
The more library instruction students have had, the higher they will score.
While we have hypothesized in the broader sense about student information literacy
competencies, we have not yet determined if we will test for specific competencies such
as the ability to apply Boolean operators in online searching, or knowing when it is
5
appropriate to make a footnote, etc. We have attached in Appendix A an example of
information competencies that we might expect new SIUE freshmen to possess.
Materials, Procedures and Time Line
Design of the Study
Type of Study
This study will produce information on specific group of freshmen and the phenomena of
information seeking skills that already exist among these students. We will have created
no control or comparison groups. The study will have a descriptive or observational
design. Specifically, it is a cross section study, which provides descriptive data at one
fixed point in time. Participants will not be retested at a later date as no coding for
tracking them will be built in, as would happen in a cohort study.
Our survey will be based on the ACRL Information Literacy Competency Standards for
Higher Education, (Appendix A) and the list of information literacy competencies that
we might expect new freshmen at SIUE to possess. We will seek the input of New
Freshman Seminar instructors at their summer workshop, our colleagues in the Library,
as well as the Director of Expository Writing, Director of the Writing Center, and other
faculty and staff who work closely with new freshmen at SIUE.
We will seek the advice of faculty experienced in survey design, as we adapt and modify
the assessment surveys we have identified. (Appendix B)
Prior to finalizing the survey instrument, a draft of the questionnaire and the
competencies list will be shared with User Services librarians. We will seek the advice of
our colleagues in aligning the questions on the survey to the proposed information
literacy competencies for SIUE freshmen (Appendix A). We believe that this will give us
a better sense of both the content validity and face validity of the instrument. While there
are published surveys in existence, we believe it best to develop an instrument specific to
our needs.
The Questionnaire Instrument
We plan to use a self-administered questionnaire consisting of approximately 25
questions. We will include some demographic questions designed primarily to ascertain
students prior experience, including: school affiliation; the amount of information literacy
skill instruction students received; who instructed them (i.e. classroom teacher(s) or
librarians; when the instruction occurred, i.e. grade level; and the amount and extent to
which the students use library resources (including online) or the Internet. Other
questions will test incoming freshmen competencies related to the information
competencies stated above.
Our instrument will not test the effectiveness of particular library instruction sessions or
whole programs, nor is it a survey of opinion. It is a test to discover how skillful or
knowledgeable students are of library resources, online searching, and informationseeking concepts. For this study, the test will be administered to twenty classes using
6
scantron questionnaires in a classroom environment. We anticipate the survey will take
no more than 15-20 minutes to complete.
Procedures
1. Target population and sampling methods (Blalock, 155-166)
Our sampling is based upon the following assumptions:

Freshmen are distributed throughout the New Freshman Seminars without
regard to age, race, gender or geographic origin of high school
It must be noted that in two of the forms of New Freshman Seminar courses,
University 112 and CIV’s, Modified Skills and Introductory Courses,
placement/inclusion based on academic rank is not a factor

Distribution
If the assumptions are correct, it is not probable that the outcomes of the
survey will vary in each of the varieties of New Freshman Seminar classes

Significance
We have chosen a significance level of 95% at plus or minus 5

Statistical tests
We intend to use frequency distributions, measures of tendency and
proportions

Decision
We have decided to draw 317 cases by administering our survey to 20 classes
of New Freshman Seminar in the following proportions, based on 80 course
offerings for Fall 2007:
Honors 120– 1 class (6.25%)
CIV’s, Modified Skills and Introductory Courses – 14 classes (67.5%)
University Experience 112 – 5 classes (26.25%)
We will work with the coordinators of the New Freshman Seminar workshop to be held
during the summer of 2007, and ask to be included on the agenda in order to explain our
research proposal to the NFS instructors. We hope to be able to identify the NFS courses
that will participate in our survey as a result of dialogue established through the
workshop
2. Test Administration
Before administering the test, we will file the IRB through the Graduate School,
Office of Research and Projects to ensure our compliance with University
guidelines of research projects using human subjects.
7
The surveys will be administered during scheduled class sessions. They should
take 15 to 20 minutes to complete. We would like to administer the surveys
within the first four weeks of the semester and/or before formal Library
instruction sessions occur.
3. Procedures for data analysis





SIUE Testing Services will read the scantron forms and provide reports for us.
Additional student worker hours will be needed to compile written answers to
questions on the survey.
The data will provide a baseline of information literacy competency skills
demonstrated by new freshmen at SIUE.
Strengths as well as weaknesses, when compared to proposed standards for
information competence will be identified.
Hypotheses related to pre-college library/research experiences will be tested.
8
Timeline
Spring 2007
 Develop assessment instrument
 Work with Pam Bryan-Williams in Testing Services to design and print
questionnaire on scantron forms
Summer 2007- Spring 2008









Attend New Freshman Seminar instructor workshop
Identify NFS courses that will participate in assessment project
Recruit student workers to assist in administering test and compiling data
Hold training workshop for student assistants
Schedule test administration dates with New Freshman Seminar instructors
August through September
Compile data
October- November
Begin analysis of data
December
Continue analysis of data and begin to report findings
January – April 2008
Prepare report for presentation to the Committee on Assessment
May 2008
9
Materials required:
Spring 2007
2000 scantron sheets @$25.00/500 sheets
$100.00
4 flash drives to store and share data,
2 for researchers, 1 for survey design expert, 1 for Testing Services staff
member
@$40.00/ea
$160.00
3 gross number 2 pencils
Spring 2007 Total:
$70.00
$ 330.00
Summer 2007 - Spring 2008
80 dozen 1 oz. cookies (4 dozen per class) @ $5.50/doz
$440.00
Student workers to administer tests 20 hours @ $7.50/hr
$150.00
Student worker to input data from written answers on survey,
40 hours @ $7.50/hr
$300.00
Workshop for test administrators
5 students x 2 hrs @ $7.50/hr $75.00
Snacks - beverages, cookies, etc = $35.00
Summer 2007/Spring 2008 Total:
$110.00
$1000.00
10
Appendix A
We have identified information competencies that all freshmen should possess. The
numbered items on the list are the competencies; the bulleted items are the specific
behaviors and skills a student needs to employ effectively in order to achieve that
competence.
Information competence of freshmen students at SIUE is defined by the ability to:
1. Define the research topic and the information need
 state a research question, problem, or issue
 understand the need to identify and define relevant terminology and
keywords and the concept of controlled vocabulary
 understand what types of materials exist (including books, journals,
Internet, government documents, fieldwork, datasets, media, primary vs.
secondary sources, popular vs. scholarly, etc.) and which are needed for
the research
 determine who would be the producers and providers of the information
required for the research
 understand the limitations of information availability
2. Develop and implement an effective search strategy/process appropriate for
an information need
 understand what types of reference sources exist (specialized
encyclopedias, indexes, abstracts, databases, bibliographies, library
catalogs, search engines, etc.) and the purpose of each
 identify appropriate reference sources for a given research question or
information need
 select appropriate subject headings, keywords, and Boolean search
strategies
 determine how to access and use the reference sources
3. Locate and retrieve information
 access and effectively use the library’s online catalogs and indexes and an
Internet browser
 accurately read, interpret, and write citations
 download or email citations from computer-based systems
 understand key elements of call numbers and URLs and be able to use
them to locate library materials and Web sites
11
4. Evaluate the information and the search strategy
 check the reliability, authority, level, accuracy, and timeliness of
information sources, including reference sources, books, articles, Web
sites, mass media, etc.
 determine whether the information retrieved is relevant and sufficient for
the research question and if further sources are needed
5. Organize and synthesize information
 critically use and integrate information from a variety of sources
appropriate to the research question
 understand the need to cite the source of information
 compile a bibliography and create footnotes
(ACRL. Information Literacy Competency
Standards for Higher Education)
12
Appendix B
Examples of information literacy competency surveys
13
References
Association of College and Research Libraries. Information Literacy Competency
Standards for Higher Education. Retrieved at
http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm,
accessed April 6, 2007.
Blalock, Hubert M. Social Statistics, 2nd ed. New York: McGraw-Hill, 1972.
Boyer Commission on Education Undergraduates in the Research University.
Reinventing Undergraduate Education: A Blueprint for America’s Research
University. 1998. Modified August 7, 2001.
Retrieved at http: //naples.cc.sunysb.edu/Pres/boyer.nsf/.>, accessed June 9,
2004.
Breivik, Patricia. Information Literacy and Lifelong Learning: The Magical Partnership.
2001. Retrieved at
http://bivir.uacj.mx/dhi/DoctosNacioInter/INFORMATION%20LITERACY%20
AND%20LIFELONG%20LEARNING.htm, accessed June 23, 2004.
The California State University Information Competence Initiative. Retrieved at
http://www.calstate.edu/LS/infocomp.shtml, accessed on April 4, 2006.
Coupe, Jill. “Undergraduate Library Skills: Two Surveys at John Hopkins University,”
Research Strategies 11 (Fall 1993): 188-201.
Curzon, Susan Carol. “Developing a Program for Information Literacy,” College &
Research Libraries News 61 (June 2000): 483+.
Greer, Arlene, Lee Weston, and Mary Alm. “Assessment of Learning Outcomes: A
Measure of Progress in Library Literacy,” College and Research Libraries 52
(November 1991): 549-57.
Kunkel, Lilith R. Susan M. Weaver, and Kim N. Cook. “What Do They Know? An
Assessment of Undergraduate Library Skills,” Journal of Academic
Librarianship, 22(6) (November 1996): 430-34.
Jackson, Lydia and Julia Hansen. Building a Framework for Partnership: Collaboration
Among the Libraries of Madison County. (2005). LSTA Grant # 05-3069
Johnson, Charlotte and Lydia Jackson. Information Literacy Across the Curriculum
(ILAC). EUE Grant, 2006.
Maughan, Pat Davitt. “Information Literacy Survey.” University of California, Berkeley
Library, 1995. Retrieved at http://lib.berkeley.edu/TeachingLib/Survey.html,
accessed on April 9, 2006.
14
Roth, Lorie. “Educating the Cut and Paste Generation,” Library Journal 124 (November
1, 1999): 42-44.
Shapiro, Jeremy J. and Shelley K. Hughes. “Information Literacy as a Liberal Art
Enlightenment Proposals for a New Curriculum,” Educom Review (March/April
1996): 31:2. Retrieved at
http://www.educause.edu/pub/er/review/reviewarticles/31231.html, accessed on
April 3, 2006.
Southern Illinois University Edwardsville. New Freshman Seminar: Call for Proposals.
http://www.siue.edu/UGOV/FACULTY/curr_NFSproposals.htm, accessed
April 15, 2007.
---.
Statement of Objectives for the Baccalaureate Degree.
http://www.siue.edu/UGOV/FACULTY/bridgestatementobjbachdeg,
accessed April 15, 2007.
15