Methodologies
Setting up a Study
Review
 Problem Statement




Problem
Originality
Direction
Significance
 Literature Review & Theoretical Framework
 Logical Structure
 Research Questions
 Population Sampling
Methodologies
 Specific systems/ tools/ approaches to
gathering and analyzing data.
 Structure of the research
 Builds on and draws from previous sections:
problem statement, research questions,
literature review, theoretical framework
 Influences analysis
 How to choose an appropriate method?
Methodologies: Options
 Quantitative
 Numbers, statistics
 Qualitative
 Words, narrative
 Mixed Methods
 Both Quantitative and
Qualitative
 Triangulation
 More than one
method, may be two of
the same type
Methods: Options
Quantitative
Qualitative
Mixed
Predetermined
Emerging
Predetermined and
Instrument-based
Open-ended
questions
Performance data,
attitude data,
observational data,
and census data
Statistical analysis
questions
Interview data,
observation data,
document data, and
audiovisual data
Text and image
analysis
emerging
Open- and closeended questions
Multiple forms of
data
Statistical and
textual analysis
-Creswell, J.W.
Methodologies: Options
 Quantitative





Surveys
Structured Interviews
Questionnaires
Bibliometrics
Transaction Log Analysis
 Qualitative


Case Studies
No Contact




One-to-one





Examining Documents
Citation Analysis
Diaries
Unstructured Interviews
Think-Aloud
Ethnographic methods
Observations
Group Interaction



Focus Groups
Delphi Method
Concept Mapping
Quantitative: Bibliometrics
 Quantitative study of literature
 Patterns of publishing within a field or body of
literature
 Quantitative study of information associated
with published works: authorship, publishers,
citations
Quantitative: Transaction Log Analysis
 Quantitative study of user behavior as
exhibited through computer logs






Domains (.gov, .edu, .com)
“Hits”
Paths followed
Time spent (searching, viewing, downloading)
Errors
Applications used
Qualitative: Case Study
 In-depth review of a single situation: program,
process, phenomenon
Qualitative: Examining Documents
 Systematic review of text/ images: content
analysis
 Classifies textual or visual material
 Uses analytical constructs or rules to draw
inferences about recurring aspects of text.
Types of Documents?
 Policy Manuals
 Digital reference transcripts
 Comment/ complaint cards
 Job ads
 Published Literature
 Blogs, listserv postings, etc.
 Open-ended responses
Evaluation Research
 Published









Books
Journal articles
Conference proceedings
Theses/dissertations
Web
ALA. ACRL. CIP Notes
CLIR.
www.clir.org/pubs/pubs.html
ARL
Other

Peer Review
 Unpublished


Internal reports
Local use only reports
Qualitative Analysis: Examining
Documents
 Focus on:





Word choice
Word frequency
Word sequence
Intensity of feeling/ expression (how to
measure?)
Key word in context
 Typology of concepts/ categories?
Example
 Regional Accreditation Organizations’
Treatment of Information Literacy
Qualitative Study: Citation Analysis
 Systematic review of bibliographies/
references within published literature.
 Focus on






Authorship
Form of publication (periodical, monograph,
etc.)
Class of material (primary, secondary, etc.)
Language
Subject
Currency
Citation Analysis
 What can citations tell us?
 How scholarly is the cited literature?
 How current are the citations?
 How research-oriented is it?
 How interdisciplinary is it?
 How writes the literature? How collaborative?
 Where does the literature appear?
Citation Analysis
 Growth of literature on a subject
 Dispersion of writings on a subject across
form and journals
 Obsolescence of literature
 Scholarly networks: who cites whom?
 Publishing productivity
Citation Analysis
 Advantages
 Shows what is cited
 Does not involve
interaction with
subjects
 Profiles a literature
 Shows changes in a
field over time?
 Disadvantages
 How complete is the
work from which
citations are drawn?
 How accurate are
citations?
 Are all materials cited?
 Choosing easy to find/
retrieve items over
better quality?
 How easily retrievable
are works?
Citation Analysis: Issues
 From where are citations drawn? How far can you




generalize findings?
Does citation reflect use?
Self-citations and/ or gratuitous citations
Half-life
Impact factor (to what extent are recent articles in
journals cited?)

Calculated annually: divide the number of current
citations a journal receives to articles published in the
previous two years by the number of articles published
in those same years
Web Citations
 How “prestigious” are different online
sources- ejournals, open access, etc.
 References to and from a Web site
 Retrievability
Qualitative Study: Diaries
 Participants record activities, thoughts, reactions, etc.
daily (weekly, etc.) over a set period of time.
 Blogs- equivalent?
 When to use?
 Issues



How faithful are participants? How can you help
ensure full participation?
How much guidance to give?
Too much/ too little detail.
 Advantages and disadvantages?
Students’ World
 Nicole Henning/ Photo Diary Study (MIT)
 16 students tracked their information seeking
behavior for one week.
 Used diaries and screen shots to record their
thoughts and actions
Qualitative Study: Interviews
 Attempt to gain in-depth knowledge on a topic
 In a less structured format, interviewer may
act more as facilitator, asking open ended
questions and drawing the participant out.
 Structured, semi-structured, or unstructured?
 Format- in person, telephone, email?
Interviews
 Types of questions:






Experience/ behavior
Feeling
Opinion/ value
Knowledge
Sensory
Demographic
Interviews
 Issues




Logistics
Building trust
Delicate questions/ situations- ensuring
honesty
Accuracy of transcripts
 Advantages and disadvantages?
 Appropriate uses?
Qualitative Study: Think-Aloud
 Asks respondent to verbalize their thoughts
while performing an assigned activity or task
 Attempt to gain more insight into thought
processes
 Correct for mistakes/ assumptions of
observer
Think Aloud
 Requires participants that are highly verbal in
nature
 Adding this layer of feedback may affect their
cognitive processes and behaviors- more
“self-concious.”
 How to transcribe while observing?
Accuracy?
 Appropriate use of clarifying questions?
 Appropriate uses?
Qualitative Study: Ethnographic
 Tools developed in the field of cultural
anthropology
 Attempt to better understand people/
behavior by observing it within natural setting
 Can study observable material items,
individual behaviors and performances, or
ideas
 Requires careful attention to detail within
cultural context
Ethnographic
 Possible Tools:






Observation in context
Key informant interviewing
Drawing pictures
Taking photos
Using maps to track activities
Videotaping
Ethnographic: Examples
 Susan Gibbons and Nancy Fried Foster of
University of Rochester: Understanding
Users to Develop Better Library Services
(ACRL/NEC 2006)



User-centered studies of library use…
Mapping Diaries
Photo Elicitation Interviews
Photo Elicitation
Mapping Diaries
Sweeping Studies
 A type of spatial data analysis
 Useful for mapping out the physical spaces of
a library and investigating how people use
those spaces
Qualitative Study: Observation
 Attempt to understand activities/ behaviors
 Obtrusive or Unobtrusive
 Role of the observer:



Complete observer
Observer and participant
Participant with individual being observed
Qualitative Study: Focus Groups
 Attempt to learn about the attitudes/ beliefs/
feelings of groups and how those influence
behavior
 Why groups? One individual’s comments can
trigger important responses from others.
 Can explore large ranges of topics
Focus Groups
 Issues






How structured?
Importance of facilitator’s role- how well
trained?
Building trust
Finding reliable representative volunteers
Ensuring accuracy of transcription- audio/
video tape? Outside observation?
Facilities
Qualitative Study: Delphi Method
 A “systematic interactive forecasting method.”
 Involves interviewing/ surveying of experts
within a field- generally 9 to 99
 Experts are provided with hypotheses, trigger
statements, scenarios, etc. and asked to
respond.
 First round responses are shared with the
group anonymously, so participants can
revise earlier statements, react to responses.
Delphi Method
 After several rounds, hypotheses may be
refined, group may reach “consensus”
 Facilitator reviews responses to each roundcan filter out irrelevant content, choose
presentation of information, ask questions.
 Advantages and disadvantages?
Qualitative Study: Concept Mapping
 “Any process that represents ideas in pictures
of maps.”
 A method of organizing the ideas and
thoughts of a group to form a common
framework
 Can be used to integrate ideas from less
structured activities such as brainstorming
Concept Mapping
 Typical steps:
 Focus: determine desired outcomes/ questions to be
addressed
 Generating ideas: Brainstorm, use trigger statements,
ask questions
 Analysis: Sort ideas into large sets
 Unstructured idea sorting: ask individuals to sort ideas
into groups and label
 Sorting by stakeholders: organize ideas by group that
generated those ideas
 Ratings: assign values to ideas (importance,
feasibility, etc.)
Concept Mapping
 Map analysis: generate map based on idea
sorting (could use statistical software, or do
by hand)
 Interpret map: Share with other groups to
obtain understanding
Example: IR and IL