Master of System Engineering
Faculty of Engineering
Gadjah Mada University
Research Methodology
and
Project Proposal Preparation
Adhy Kurniawan
Faculty of Engineering
Gadjah Mada University
Adhy Kurniawan
► 1987-1990 SMA 3 Semarang
► 1990-1991 Fac. Of Economy, Diponegoro
University, Semarang
► S1(1991-1996) Civil Engineering Dept.
Gadjah Mada Univ.
► S3(1998-2003) Swiss Federal Institute of
Technology Lausanne (EPFL), Swiss
► Post Doct (nov.2005-sept.2006) Kyoto
University, Japan
My Goals for Course
►That each of you develop an
intuition for the fundamental
principles of research methodology
►That we have an enjoyable
semester learning together
Lecture and Homework
Lecture
• Presentation and discussion
Homework
►Your chance to practice using the
concepts presented in class
►Teamwork vs. Individual work?
References
►All of literature concerning: Res Met
► Marczyk, DeMatteo, Festinger. 2005,
Essentials of Research Design and
Methodology, John Wiley and Sons.
►Day and Gastel, 2006, How to write and
Publish a Scientific Report, Greenwood
Press
►Metodologi Riset, Etc.
List of students
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
►
Alif Ardy Saputra, Geodesi UGM
Anik FR, TL, ITB
Ashri Uswatun, TFisika,UGM
Ayi Fajarwati, TL, ITB
Corry Agustina, Perenc Wil, TA, UGM
Dwi Astuti, TKimia, UGM
Elva Nur , TF, UGM
Erika Kezia, TL, ITB
Fitri Wijayanti, Fisika, UNS
I Nyoman Kusuma, TF, UGM
Ihsan Hasan, T Industri, UII,
Ihwan Ghazali, T Industri, UAD
Iin Lestari, TL, ITB
M Sony Abertiawan, TL, ITB
Maria Auliana, T Sipil, UGM
Norma Pradipta, TArsitektur, UGM
Satrya Alrizki, TGeofisik, ITB
Tatag Lindu Bhakti, TFisika, UGM
Contents
The aims of research,
the research topic,
title and research problem,
literature review,
research design: population and sampling types, types
of quantitative research designs, validity of
conclusions, data-collecting methods and
measuring instruments in quantitative research,
qualitative research designs,
data analysis and interpretation of results,
report writing and the research proposal,
ethical consideration on research.
OVERVIEW OF SCIENCE AND THE
SCIENTIFIC METHOD
science can be defined as a methodological and
systematic approach to the acquisition of new
knowledge.
This definition of science highlights some of the
key differences between how scientists and
nonscientists go about acquiring new
knowledge.
Specifically, rather than relying on mere casual
observations and an informal approach to learn
about the world, scientists attempt to gain new
knowledge by making careful observations and
using systematic, controlled, and
methodical approaches (Shaughnessy &
Zechmeister, 1997).
Shaughnessy, J. J., & Zechmeister, E. B. (1997). Research methods in
psychology (4th ed.). Boston: McGraw Hill.
►In addition, scientific knowledge is not
based on the opinions, feelings, or
intuition of the scientist.
►Instead, scientific knowledge is based on
objective data that were reliably obtained
in the context of a carefully designed
research study.
►In short, scientific knowledge is based
on the accumulation of empirical
evidence (Kazdin, 2003a)
Kazdin, A. E. (2003a). Methodology: What it is and why it is so important. In A.
E. Kazdin ( Ed.), Methodological issues and strategies in clinical research (3rd
ed., pp. 5–22). Washington, DC: American Psychological Association.
►The defining characteristic of scientific
research is the scientific method .
►First described by the English
philosopher and scientist Roger Bacon in
the 13th century, it is still generally
agreed that the scientific method is
the basis for all scientific
investigation.
►The scientific method is best thought of
as an approach to the acquisition of new
knowledge, and this approach effectively
distinguishes science from nonscience.
The Scientific Method
The development of the scientific method is
usually credited to Roger Bacon, a philosopher
and scientist from 13th-century England,
although some argue that the Italian scientist
Galileo Galilei played an important role in
formulating the scientific method.
Later contributions to the scientific method were
made by the philosophers Francis Bacon and
René Descartes.
►Although some disagreement exists
regarding the exact characteristics of the
scientific method, most agree that it is
characterized by the following elements:
●
●
●
●
●
●
●
●
Empirical approach
Observations
Questions
Hypotheses
Experiments
Analyses
Conclusions
Replication
Empirical Approach
► The scientific method is firmly based on the empirical
approach. The empirical approach is an evidence-based
approach that relies on direct observation and
experimentation in the acquisition of new knowledge (see
Kazdin, 2003a).
► In the empirical approach, scientific decisions are made based
on the data derived from direct observation and
experimentation.
► Contrast this approach to decision making with the way that
most nonscientific decisions are made in our daily lives.
► For example, we have all made decisions based on feelings,
hunches, or “gut” instinct. Additionally, we may often reach
conclusions or make decisions that are not necessarily based
on data, but rather on opinions, speculation, and a hope for
the best.
► The empirical approach, with its emphasis on direct,
systematic, and careful observation, is best thought of as the
guiding principle behind all research conducted in accordance
with the scientific method.
Observations
► An important component in any scientific investigation is
observation. In this sense, observation refers to two
distinct concepts—being aware of the world
around us and making careful measurements.
► Observations of the world around us often give rise to the
questions that are addressed through scientific research.
► For example, the Newtonian observation that apples fall
from trees stimulated much research into the effects of
gravity. Therefore, a keen eye to your
surroundings can often provide you with many
ideas for research studies.
Questions
► After getting a research idea, perhaps from making
observations of the world around us, the next step in the
research process involves translating that research idea
into an answerable question.
► The term “answerable” is particularly important in this
respect, and it should not be overlooked.
► It would obviously be a frustrating and ultimately
unrewarding endeavor to attempt to answer an
unanswerable research question through scientific
investigation.
► It is therefore important to formulate a research
question that can be answered through available
scientific methods and procedures.
Hypotheses
► The next step in the scientific method is coming up with a
hypothesis, which is simply an educated—and
testable—guess about the answer to your research
question.
► A hypothesis is often described as an attempt by the
researcher to explain the phenomenon of interest.
► Hypotheses can take various forms, depending on the
question being asked and the type of study being
conducted.
► A key feature of all hypotheses is that each must make a
prediction.
► Remember that hypotheses are the researcher’s attempt
to explain the phenomenon being studied, and that
explanation should involve a prediction about the
variables being studied.
► These predictions are then tested by gathering and
analyzing data, and the hypotheses can either be
supported or refuted on the basis of the data.
► Two types of hypotheses with which you should be
familiar are
● the null hypothesis
● and the alternate (or experimental) hypothesis.
► The null hypothesis always predicts that there will be no
differences between the groups being studied.
► By contrast, the alternate hypothesis predicts that there
will be a difference between the groups.
● For example,
● the null hypothesis would predict that the
exercise group and the no-exercise group will not
differ significantly on levels of cholesterol.
● The alternate hypothesis would predict that
the two groups will differ significantly on
cholesterol levels.
● Homework: Individual
● Please try to find one example. About the null
Experiments
► After articulating the hypothesis, the next step involves
actually conducting the experiment (or research
study).
► For example, if the study involves investigating the effects
of exercise on levels of cholesterol, the researcher would
design and conduct a study that would attempt to
address that question.
► As previously mentioned, a key aspect of conducting a
research study is measuring the phenomenon of interest
in an accurate and reliable manner.
► In this example, the researcher would collect data on the
cholesterol levels of the study participants by using an
accurate and reliable measurement device.
► Then, the researcher would compare the cholesterol
levels of the two groups to see if exercise had any
effects.
Accuracy vs. Reliability
► When talking about measurement in the context of research,
there is an important distinction between being accurate and being
reliable.
► Accuracy refers to whether the measurement is correct, whereas
reliability refers to whether the measurement is consistent.
► An example may help to clarify the distinction.
● When throwing darts at a dart board, “accuracy” refers to
whether the darts are hitting the bull’s eye (an accurate dart
thrower will throw darts that hit the bull’s eye).
● “Reliability,” on the other hand, refers to whether the darts
are hitting the same spot (a reliable dart thrower will throw
darts that hit the same spot).
● Therefore, an accurate and reliable dart thrower will
consistently throw the darts in the bull’s eye. As may be
evident, however, it is possible for the dart thrower to be
reliable, but not accurate.
● For example, the dart thrower may throw all of the darts in
the same spot (which demonstrates high reliability), but that
spot may not be the bull’s eye (which demonstrates low
accuracy).
Analyses
► After conducting the study and gathering the
data, the next step involves analyzing the data,
which generally calls for the use of statistical
techniques.
► The type of statistical techniques used by a
researcher depends on the design of the study,
the type of data being gathered, and the
questions being asked.
► It is important to be aware of the role of
statistics in conducting a research study.
► In short, statistics help researchers minimize
the likelihood of reaching an erroneous
conclusion about the relationship between the
variables being studied.
Conclusions
► After analyzing the data and determining
whether to reject the null hypothesis, the
researcher is now in a position to draw some
conclusions about the results of the study.
► For example, if the researcher rejected the null
hypothesis, the researcher can conclude that
the phenomenon being studied had an effect—
a statistically significant effect, to be more
precise.
● If the researcher rejects the null hypothesis in
our exercise-cholesterol example, the researcher
is concluding that exercise had an effect on levels
of cholesterol.
►It is important that researchers
make only those conclusions that
can be supported by the data
analyses.
►Going beyond the data is a cardinal
sin that researchers must be careful
to avoid.
Replication
►One of the most important elements of
the scientific method is replication.
►Replication essentially means
conducting the same research study a
second time with another group of
participants to see whether the same
results are obtained.
►The same researcher may attempt to
replicate previously obtained results, or
perhaps other researchers may
undertake that task.
► Replication illustrates an important point about
scientific research—namely, that researchers
should avoid drawing broad conclusions based
on the results of a single research study
because it is always possible that the results of
that particular study were an aberration.
► In other words, it is possible that the results of
the research study were obtained by chance or
error and, therefore, that the results may not
accurately represent the actual state of things.
► However, if the results of a research study are
obtained a second time (i.e., replicated), the
likelihood that the original study’s findings were
obtained by chance or error is greatly reduced.
►What are the three general
goals of scientific research?
Answer:
►description,
►prediction,
►and understanding/explaining
What Exactly is Research?
►we will focus on two of the most
common types of research—
● correlational research
● and experimental research
Correlational research:
► In correlational research, the goal is to determine whether
two or more variables are related. (By the way, variables”
is a term with which you should be familiar.
► A variable is anything that can take on different values,
such as weight, time, and height.)
► For example, a researcher may be interested in
determining whether age is related to weight. In this
example, a researcher may discover that age is indeed
related to weight because as age increases, weight also
increases. If a correlation between two variables is strong
enough, knowing about one variable allows a researcher
to make a prediction about the other variable.
It is important to point out, however, that a correlation—
or relationship—between two things does not necessarily
mean that one thing caused the other.To draw a causeand-effect conclusion,
researchers must use experimental research.
.
Experimental research:
► In its simplest form, experimental research involves
comparing two groups on one outcome measure to test
some hypothesis regarding causation.
► For example, if a researcher is interested in the effects of
a new medication on headaches, the researcher would
randomly divide a group of people with headaches into
two groups.
► One of the groups, the experimental group, would receive
the new medication being tested.
► The other group, the control group, would receive a
placebo medication (i.e., a medication containing a
harmless substance, such as sugar, that has no
physiological effects).
Experimental research:
Besides receiving the different medications, the groups
would be treated exactly the same so that the research
could isolate the effects of the medications. After
receiving the medications, both groups would be
compared to see whether people in the experimental
group had fewer headaches than people in the control
group.
Assuming this study was properly designed (and properly
designed studies will be discussed in detail in later
chapters), if people in the experimental group had fewer
headaches than people in the control group, the
researcher could conclude that the new medication
reduces headaches.
Task
►Compose your own brief research
proposal.
►Try to determine your research
topic for MST final project
►Format:
● 1. In MS Word
● 2. In Power point
Task/assignment  next week
►Review 1 International Publication
(Journal, Conference paper,etc)
related to Renewable energy
►Compose the summarize of your
review
►Format:
● 1. In MS Word
● 2. In Power point
Purpose of the research proposal
►1. To inform the reader of nature of your
proposed research.
● What is the problem?
● What is its extent?
►2. To convince the reader, especially
supervisors and reviewers, of the value
of your proposed research.
● Is this project worth the time
and money?
● Will it make a difference to the
world?
Purpose of the research proposal
►3. To demonstrate your expertise and
competency in a particular area of study.
● Do you have the qualifications to conduct
this research?
● Have you informed yourself of the
existing theory and data relevant to your
topic?
● Do you have the
necessary skills to
conduct the research?
Purpose of the research proposal
► 4. To plan the research project and provide a
step-by-step guide to the tasks necessary
for its completion.
● What are the key stages of the work?
● What are the priorities?
● How do the various components fit together?
► 5. To request support from individuals and
agencies who provide supervision,
oversight or funding for the research project.
● What kinds of support does the project need?
● Are all participants properly protected?
Purpose of the research proposal
►6. To contract with the agencies
and individuals involved, including
supervisors, foundations and
participants in the research team.
● How will tasks be assigned and
resources expended?
● What does each contribute
to the collective endeavor?
First things first
1.
2.
3.
4.
5.
Basics
Topic ideas
Typical methodologies
Common pitfalls
Getting started and putting it all
together
6. Questions/discussion
Basic steps of a research
project
►Find a topicWhat, When
►Formulate questionsWhat, Why
►Define populationWho, When
►Select design & measurementHow
►Gather evidenceHow
►Interpret evidenceWhy
►Tell about what you did and found out
Selecting a Research Topic
►What are some considerations
when selecting a research topic?
Considerations in Selecting a
Topic
● Personal interest / Passion
● Importance / Contribution to the field
● Newness / Relevance
● Feasibility






Tradeoff between rigor and practicality
Time constraints
Ethical constraints
Organizational support
Economic factors
Availability of Subjects
Sources of Research Topics
● Peer-reviewed journals in your field
● Personal experiences
● Work setting experiences
● Existing literature
 “Recommendations for future research…”
Refining Your Topic
►Refinement needed for effective and
efficient research
● Narrow your topic
● Identify a theoretical framework
● Specifically and unambiguously define
terms
● State research questions and
hypotheses
Refining Your Topic (cont’d)
►A literature review will help you
● See if your idea has been tried
● Include all relevant constructs
● Select instruments
● Anticipate common problems
Components of a Concept
Paper
● Title page
● Introduction
● Nature of the Problem
● Background and Significance of the
Problem
● Preliminary Literature Review
● Initial Research Question or
Questions
Components of a Concept
Paper (cont’d)
● Brief Description of Methodology
and Research Design
● Anticipated Outcomes
● Timeline
● References
The Literature Review
What is a Literature
Review?
• According to Creswell (2005), a
review of the literature “is a written
summary of journal articles, books
and other documents that describes
the past and current state of
information, organizes the literature
into topics and documents a need
for a proposed study.” (pp. 79)
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research
Focusing on Empirical Research
►What does Empirical Mean?
►Primary Sources
● Original Research Article
►Secondary Sources
●
●
●
●
●
Newspapers
Book chapters
Television/Radio
Magazines
Wikepedia
Empirical Research
► All empirical research is inherently
flawed
● Limitations
1. Sampling
● Generalizability
● Representative
2. Measurement
● Measurement Error
● Social Desirability
3. Problem Identification
● Grasping the “Whole” Problem
Literature Reviews
►Well-written analytical narrative that
brings a reader up-to-date on what
is known on a given topic, but also
provide fresh insights that advance
knowledge
● Resolve conflicts between studies
● Identify new ways to interpret
research results
● Creating a path for future research
Anecdotal Reports
►A description of an event or
experience that happened to be
noticed
● No control
● No comparison
Review of Key Elements of
Previous Definition
• The LR is a summary of research:
• It is not a “list” of found research
but a coherent and articulate
account of past and current
research findings
• Suggestion: read 2 or 3 LRs in order
to become familiar with summary
styles
Review of Key Elements of
Previous Definition (cont’d)
• The sources typically are journal articles, books
and other documents that describe past and
present status of research in a given field:
• The LR should be exhaustive and as
current as possible.
• How many articles?
• There is no set number. As long as the search
is exhaustive and focused on the research
topic, the review will be acceptable.
Review of Key Elements of
Previous Definition (cont’d)
• How far back should one search?
• A reasonable and widely accepted
timeframe includes research conducted
during the past 10 years. Important
studies (i.e., studies that had a
significant impact on the field of study)
should also be mentioned even if these
go beyond the mentioned timeframe.
Review of Key Elements of
Previous Definition (cont’d)
• The LR should be organized:
• The review should not only be coherent, but
should organize the studies reviewed under
themes or topics.
• The reviewer is a guide and should be able
to provide readers with an in-depth and
current status of research in a given area.
• This aspect is essential for readers to
understand what the reviewer found during
the search.
Review of Key Elements of
Previous Definition (cont’d)
• The LR should document the need for a
proposed study:
• Studies should not duplicate research that
has been already done.
• Even in cases when research is duplicated
(replicated is the appropriate term), one is
responsible for documenting the need for
replication, e.g., need to explore the same
methodology with a different group or
population, or need to change methodology
with the same group.
Creswell’s 5 steps to Conduct a
Literature Review
• Step 1: Identify Key Terms or
“Descriptors”
• Extract key words from your
title (remember, you may decide
to change the title later)
• Use some of the words other
authors reported in the
literature
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Step 1: Identify Key Terms or
“Descriptors” (cont’d)
• Use the “Thesaurus of ERIC
Descriptors to look for terms
that match your topic: go to
www.eric.ed.gov and in “Search”
select “Descriptors (from
Thesaurus)”
• Scan both electronic and
library journals from the past
10 years and look for key
terms in the articles
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Creswell’s 5 steps to Conduct a
Literature Review (cont’d)
• Step 2: Locate Literature
• Use academic libraries, do not limit
your search to an electronic search
of articles
• Use primary and secondary sources.
A “primary source” is research
reported by the researcher that
conducted the study. A “secondary
source” is research that summarizes
or reports findings that come from
primary sources
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Step 2: Locate Literature
(cont’d)
• It is “best to report mostly
primary sources” (p. 82)
• Search different types of
literature: summaries,
encyclopedias, dictionaries and
glossaries of terms, handbooks,
statistical indexes, reviews
and syntheses, books, journals,
indexed publications,
electronic sources, abstract
series, and databases
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Creswell’s 5 steps to Conduct a
Literature Review (cont’d)
• Step 3: Critically Evaluate and
Select Literature
• Rely on journal articles
published in national journals
• Prioritize your search: first
look for refereed journal
articles, then, non-refereed
articles, then books, then
conference papers,
dissertations and theses and
then papers posted to websites
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Step 3: Critically Evaluate and
Select Literature (cont’d)
• Look for research articles and
avoid as much as possible
“opinion” pieces
• Blend qualitative and
quantitative research in your
review
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Creswell’s 5 steps to Conduct a
Literature Review (cont’d)
• Step 4: Organize the Literature
• Create a “file” or “abstract” system
to keep track of what you read.
Each article you read should be
summarized in one page containing
Title (use APA to type the title so that
you can later copy-paste this into the
References section of your paper)
Source: journal article, book, glossary,
etc.
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Step 4: Organize the Literature
(cont’d)
Research problem: one or two lines will suffice
Research Questions or Hypotheses
Data collection procedure (a description of
sample characteristics can be very handy as
well)
Results or findings of the study
• Sort these abstracts into groups of related
topics or areas which can then become the
different sections of your review
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Creswell’s 5 steps to Conduct a
Literature Review (cont’d)
• Step 5: Write a Literature Review
• Types of Reviews:
Thematic Review: a theme is identified and
studies found under this theme are described.
Major ideas and findings are reported rather
than details.
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Step 5: Write a Literature
Review
(cont’d)
Study-by-study Review: a detailed summary of
each study under a broad theme is provided.
Link summaries (or abstracts) using
transitional sentences. Must be organized and
flow coherently under various subheadings.
Avoid string quotations (i.e., lengthy chunks of
text directly quoted from a source)
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Preliminary Literature
Review
►This succinct review of current
literature should:
● Provide further contextual
background
● Reveal issues related to your study
● Describe similar problems in other
organizations
● Provide significance to your
approach to the study
Guidelines on Style, Mechanics, and
Language Usage
►Does your draft follow the logic or
idea that is presented in your intro
and title?
►Avoid overusing direct quotations,
especially long ones
►Check style manual for correct use
of citations
● (Doe, 2005); Doe (2005); (Doe & Smith,
2005); Doe and Smith (2005); (Black,
2005; Brown, 2006; Yellow, 2007)
Guidelines on Style, Mechanics,
and Language Usage
►Avoid using synonyms for recurring
words
● This is not creative writing and stay
consistent with terminology
 Group I, Phoenix Cohort, Experimental Group
►Spell out all acronyms when first using
them
● Traditional - American Psychological
Association (APA)
● Non-traditional - Collective Efficacy (CE)
►Yes - Do NOT use contractions; No –
Don’t use contractions
►Coined terms should be set off by quotes
Guidelines on Style, Mechanics,
and Language Usage
►Avoid the following:
● Slang – “cool”
● Colloquialisms – “thing” >> “item” or
“feature”
● Idioms – “rise to the pinnacle” >> “to
become prominent”
►Use great care to avoid Plagiarism
What needs to be included in
the Literature review.
►Provides contextual background
►Reveals related issues
►Reviews similar problems elsewhere
►Provides significance to your approach
to the study
►Includes major/seminar research articles
pertaining to study
►Written in an integrated manner
►Uses peer-reviewed research
►Includes a Reference section
Writing Your Research
Question(s)
►Reflect the problem that the researcher
wants to investigate
►Can be formulated based on theories,
past research, previous experience, or the
practical need to make data-driven
decisions in a work environment
Writing Your Research
Question(s) (cont’d)
►Are vitally important because they, in large
part, dictate what type of statistical
analysis is needed, as well as what type of
research design may be employed
►A research question should address only
1 concept
►Question must be measurable
Types of Questions Asked
►Once you have identified the topic of study, you
will need to consider the type of question you
want answered and how it will be answered
►Two paradigms
● Quantitative Paradigm
 Generally attempt to quantify variables
of interest. Questions frequently
address “how well or how much.”
Types of Questions Asked
● Qualitative Paradigm
 “there are times when we wish to know
not how many or how well, but simply
how.” (Shulman, 1988, pg. 7)
Class Exercise
►Now you’re ready to formulate your own
research question(s)
►Sample questions:
● Is there a relationship between
participation in an Elluminate chat
session and course grade?
● How do 5th grade students
experience the anticipation of
standardized testing?
Research Questions
► From Topic to Research Question
A good research topic asks a clear, concise question.
Asking a research question helps you keep a tight
focus on your topic.
► Tweaking Your Research Question
A good research topic is broad enough to allow you
to find plenty of material, but narrow enough to fit
within the size and time constraints of your paper.
● If your topic is either too broad or too narrow, consider
adding or eliminating the following elements:
Time Period, century, decade, future, Population
Type, age, gender, nationality, species, Geographic
Location country, state, region, Point of View
economic, social, cultural, biological
Assignment 2 Components
(see syllabus for details)
►Title Page
►Nature of the Problem
►Background and Significance of the
Problem
►Literature Review
►Research Questions
►References
Topic ideas
►Online chat reference
● Types of questions
 Subject? Type?
 # of turnaways*
● Difference in discourse
 In-person vs. chat
● Partnership studies
 Similar libraries with same software
Topic Ideas
►E-book usage
►Usability studies of
● Online tutorial(s)
● ‘My Library” portals
►Analysis of library web sites or
library instruction sites or
pathfinders by best practices
►Student learning outcomes in LI
programs
Types of methodologies
►QuaLitative Measures
● Descriptive
● Numbers not the primary focus
● Interpretive, ethnographic,
naturalistic
►QuaNtitative Measures
● N for numbers
● Statistical
● Quantifiable
QuaLitative measures
►Content Analysis
● Analyzed course syllabi of
library use through discipline
and level (Rambler)
● Studied online tutorials,
applying best practices
recommendations (Tancheva)
QuaLitative Measures
►Discourse Analysis
● Analyzed student responses in writing
and discussions to a short film &
compared findings to parallel study
with LIS grad Ss (Vandergrift)
►Focus Groups
● Discussed how participants
experience & use the library (Von
Seggern & Young)
● Studied why students use the
Internet and how much time they use
it (Wilson)
QuaLitative Measures
►Interviews
● Studied 25 HS students’ web use for
research assignments (Lorenzen)
● Looked at what type of information first
year students need and how they go about
acquiring it (Seamans)
►Observation (obtrusive)
● Observed students as they conducted
online research & noted their activities
(Dunn)
►Observation (Unobtrusive)
● Retrieval of discarded cheat sheets to
analyze academic misconduct (Pullen et. al.)
QuaLitative Measures
►Think Aloud Protocols
● Studied how users navigate a library
web site (Cockrell & Jayne)
►Usability testing
● Examined students’ mental models
of online tutorials (Veldof & Beavers)
QuaNtitative measures
►CompareThings
►Count Things
►Survey People About Things
QuaNtitative measures
►Comparison studies
● Experimental and control groups
● Instructional methodologies (Colaric;
Cudiner & Harmon)
● Program assessment using before/after
analysis of research papers(Emmons &
Martin)
QuaNtitative measures
►Pre & Post Tests (Van Scoyoc)
►Measures & Scales
 Bostick’s Library Anxiety Scale (Onwuegbuzie
& Jiao; Van Scoyoc)
 Procrastination Assessment Scale
(Onwuegbuzie & Jiao)
QuaNtitative measures
►Numeric Studies
● Citation AnalysisBibliometrics
(Dellavalle)
● Webometrics (Bar-Ilian)
Ready Made Data Sets
►National Survey of Student
Engagement (Whitmire)
►College Student Experiences
Questionnaire (Kuh and Gonyea)
►The Web
● Internet Archive (Ryan, Field &
Olfman)
● Electronic journals (Dellavalle)
►Library server logs
Common Pitfalls
►Problems with population
● Sampling?
 Representativeness?
 Self-selection?
Research Problem #1
A study assessing student learning
outcomes in 2 broad categories
(concepts, techniques) by examining
student research journals in 1 section
of an elective information literacy
course in fall semester.
Research Problem #4
A 2004 article on a library use and
services satisfaction study that used
as its measurement tool a survey
given to every nth person entering
the library building on 40 randomly
selected days throughout the school
year.
Research Problem #5
An outcomes assessment research
project of a 5 year old IL program in
which all incoming freshmen must
participate. Total student population on
campus is divided between 32%
freshmen to senior (or 4 year) and
68% transfer students.
Common Pitfalls
►Problems with operationalization
● Defining of what is measured
Research Problem #2
An experimental study that proposes a
fund allocation formula for academic
library collections based on the
following:
average of overall book price + average of
overall serial prices * degree level (10 for
undergraduate to 30 for doctorate) / the number of
students enrolled in degree program as majors +
the total number of faculty in the department * three
* total number of students in program.
(OAB + OAS) * D/(Sn +(Fn*3))*Sn
N.B. Not a standard formula
Research Problem #3
A newspaper article you read just the
other day stated that in a recently
published study done at a major U.S.
university, researchers found that
domestic violence affects 1 in every 4
women.
Research Problem #4
A 2004 article on a library use and
services satisfaction study that used
as its measurement tool a survey
given to every nth person entering the
library building on 40 randomly
selected days throughout the school
year.
Research Problem #5
Over a one year period, researchers
studied the occurrence of turn-aways
in a virtual reference service and noted
that the significantly high occurrence of
turn-aways indicates increased need
for virtual reference service.
Common Pitfalls
►Problems with generalizability
● False conclusions
● Transformations
Research Problem #1
A study assessing student learning
outcomes in 2 broad categories
(concepts, techniques) by examining
student research journals in 1 section
of an elective information literacy
course in fall semester.
Research Problem #7
A survey of faculty found that the
majority of those interviewed interacted
most with librarians at the reference
desk. The researchers concluded that
most faculty view librarians in a
servile role.
Keep In Mind That
►No study is perfect
►“All data is dirty is some way or
another; research is what you do
with that dirty data” (Manuel)
►Measurement involves making
choices
Be Critical About Numbers
(Best 2001)
►“Every statistic is a way of summarizing
complex information into relatively
simple numbers.” (Best)
►How did the researchers arrive at these
numbers?
►Who produced the numbers and what is
their bias?
►How can key terms be defined & in how
many different ways?
Be Critical About Numbers
►How was the choice for the
measurement made?
►What type of sample was gathered
& how does that affect result?
►Is the statistical result interpreted
correctly?
►If comparisons are made, are they
appropriate?
►Are there competing statistics?
Getting Started
► Read to learn; read to analyze
● About research methodology
● Studies on similar topics
● Interesting studies
● Non-library studies
Getting Started
►Finding a topic needn’t be
traumatic
● Work projects Research studies
 P&T overhaul
 Library GO Bond Proposal Project
 Library workshop trends
 User repair strategies
Getting Started
►Data collection involves agreement
& consent
►Forge partnerships
►At some point you will need to
leave the comfort zone of reading
and literature gathering and …
Just get out
and do it!
Questions?
Research methodology
Quantitative Methods
Qualitative procedures
Quantitative Methods
►A definition
● A survey or experiment that
provides as output a quantitative or
numeric description of some
fraction of the population, called
the sample.
Components of a survey
method
►The survey design
►The population and
sample
►The instrumentation
►Variables in the study
►Data analysis
The survey design
►Purpose of the survey
►The research question
►Type of survey
● Cross sectional
● Longitudinal
►Form of data collection
The population and sample
►Description of the population
►Sampling design
● Single stage
● Multistage
● Stratified
►Sample selection
The instrumentation
►The instrument (tool)
● Existing
● New
►Rating scale
● Likert scale: Rating the Items. 1-to-5 rating scale where:
1.
2.
3.
4.
5.
= strongly unfavorable to the concept
= somewhat unfavorable to the concept
= undecided
= somewhat favorable to the concept
= strongly favorable to the concept
►Pilot
►Administration
● Postal survey
● email
Variables and analysis
►The research question
►Variable in the research
● E.g. Number of years of academic study
►The questions in the instrument
● E.g. How many years of study in a
University
 As an undergraduate?
 As a postgraduate?
►Data analysis
● Steps
● Bias in the data
 Non-response
● Statistics, e.g. mean, standard deviation etc.
Components of an
experimental method
►Subjects
►Instruments and materials
►The experimental design
Subjects
►Selection
● Conveniently
● Random (RCT)
►Group assignment
● Random
● Matched. E.g.
● Size
►Variables
● Dependent
● Independent
Ability, Age
Randomized Controlled Trial
(RCT)
A true experiment, in which the researcher
randomly assigns some patients to at least
one maneuver (treatment) and other patients
to a placebo, or usual treatment. Key features
= the classic way to evaluate effectiveness of
drugs (or exercise, diet, counseling). Patients
are followed over time (Prospective). If
properly done, an RCT can be used to
determine cause and effect
Instrumentation and
Materials
►Description
►Validation
● Pilot
● Content validity
● Prediction validity
►Materials
The experimental design
►Type
● Pre-experimental
 No control group
● Quasi-experimental
 Control group, but not randomly
assigned
● Single subject design (over time)
● Pure experiment
● Repeated measures
 Change groups
Overview of Qualitative Research
Design
• Historical routes in anthropology
• Generates new understanding by naming and framing
concepts and themes
• Removes bias by questioning preconceived assumptions of
the social group under study
• Promotes neutrality through adoption by the researcher of
naïve stance or critical discussion, challenges pre-conceived
assumptions of both the researcher and the social group under
study
• Produces new understanding about the world, changes the
way power, culture and social interaction are understood
Data Collection in Qualitative Research
•Observation (Videoed, non-participant, semiparticipant and participant observation, field notes)
•Interviews (individual and group - known as focus
groups, tape recorded and transcribed, field notes)
•Secondary data analysis (using written material
collected for purposes other than research)
•Questionnaires (unstructured, postal, interviews)
•A mixture of all four
Questions in Qualitative Research
In qualitative research questions are open-ended.
Sometimes a check list or topic guide will be used
by the researcher to ensure all the relevant areas
are covered. This is known as semi-structured data
collection. It is used in all four methods of data
collection
Sometimes the only guide is the topic itself and
the researcher collects verbatim or naturally
occurring data. This is known as unstructured data
collection. It is used in all four methods of data
collection
Sampling in Qualitative Research
The sampling method of choice is theoretical
sampling (queuing behaviour)
However, often this is not possible and people
resort to convenience sampling (students) and
snowball sampling (mental health in black and
ethnic minority communities)
Neither of the latter two methods are considered
strong but maybe all that can be achieved.
Research must be viable.
Data Analysis in Qualitative Research
•Read and re-read data, become engrossed in it.
•Identify themes: common, conflicting, minority
•Test themes across the data set, where are they
common, under what circumstances are they found, not
found. This sets the parameters on the interpretation and
generalisation of data
•Get more than one person to analyse the data
independently then together
•Demonstrate trustworthiness in data analysis
•Examples
•Biographical continuity
•Nursing routines as a method of managing a transient
workforce
Qualitative research
►Interpretative research
►Process orientated
►Researcher(s) are the primary data
collection instrument
►Descriptive research
►Outputs are an inductive process
References
►MSc project web pages
http://www.comp.glam.ac.uk/gis/start.asp?whatfile=gis/gis
rc/msc-proj.htm
►Creswell, J. W. (1994) Research
design : qualitative and quantitative
approaches. - Thousand Oaks,
Calif.; London : Sage Publications,
ISBN 0803952546