Educational Research:
Descriptive Research
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Research...

The systematic application of a
family of methods employed to
provide trustworthy information
about problems
…an ongoing process based on many
accumulated understandings and
explanations that, when taken together,
lead to generalizations about problems
and the development of theories
Quantitative research methods...
…collect and analyze numerical data
obtained from formal instruments
Quantitative methods...
descriptive research (“survey research”)
 correlational research
 causal-comparative research
(“ex post facto research”)
 experimental research


descriptive research (“survey research”)
…collects data in order to answer
questions about the current status of
the subject or topic of study
…uses formal instruments to study
preferences, attitudes, practices,
concerns, or interests of a sample
The basic steps of descriptive
research...
An orderly scientific and disciplined
process, involving:
 recognizing and identifying a topic to
be studied
 selecting an appropriate sample of
participants
 collecting valid and reliable data
 reporting conclusions
Classifications of descriptive
research studies...
cross-sectional survey…
…involves collecting data from
selected individuals in a single
time period however long it
takes to collect data from
participants
longitudinal survey…
…involves collecting data at two or more
instances in order to measure changes
over time
self-report survey…
…requires individuals to respond to a
series of statements or questions
about themselves
observation study…
…the researcher obtains data by
watching participants in a situation
Types of longitudinal surveys...
trend survey
 cohort survey
 panel survey
 follow-up survey


trend survey
…a study where a sample is taken
from the general population in order
to collect data over time
…involves different groups and
different samples over time

cohort survey
…a study where a specific population
is examined by sampling different
groups within the population over
time
…involves the same group but
different samples from that group
over time

panel survey
…a study where the same group and
the same sample are examined over
time

follow-up survey
…a study undertaken after a panel
survey and seeks to examine
subsequent development or change
Conducting a questionnaire study…
1. state the problem
2. select participants
3. construct the questionnaire
4. prepare cover letter
5. pretest questionnaire
6. follow up activities
7. analyze/report results
1. state the problem…
…topic must be of sufficient
significance to motivate potential
respondents and justify the research
effort in the first place
…define topic in terms of specific
objectives indicating the kind of
information needed
2. select participants…
…use an appropriate sampling
technique
…participants must be able to provide
the desired information sought and
willing to provide it to the researcher
3. construct the instrument…
…should be attractive, brief, and easy
to fill out
…there are a variety of data collection
methods that can be used, including:
questionnaires, interviews,
observations, and telephone calls

types of self-report items
…scaled
…ranked
…checklist
…free response
Designing an instrument…
…include a brief statement describing
the study and its purpose at the top
of the instrument
…provide standardized directions
…organize items leaving sufficient
white space on instrument
…place similar items together
…ask general, non-threatening items
first, moving into more specific,
self-revealing items

some do’s and don’t’s…
…construct items according to a set of
predetermined guidelines
…include only items relating directly to
the purpose of the study
…avoid jargon or any term or concept
that might mean different things to
different people
…each item should deal with a single
concept and be worded as clearly as
possible
…items should indicate a point of
reference beyond the self in order to
judge the self
…avoid ambiguous terms like several
or usually
…avoid touchy matters in items which
respondents might not respond to
honestly or at all
…avoid leading questions which
suggest that one response may be
more appropriate than another
…each item must be able to stand on
its own
…don’t jam items together
…don’t put the most important
questions at the end of the
instrument
4. prepare the cover letter…
…a brief, neat, explanation of the
significance of the study and what is
being asked of the respondent and
why
…addressed to the specific, potential
respondent
…an endorsement adds credibility
…guarantee anonymity or
confidentiality
…include a specific deadline date by
which the completed instrument
(“questionnaire”) is to be returned
…individually sign each letter
…include a self-addressed, stamped,
return envelope
5. pretest the instrument…
…conduct a pilot study to gather
information about deficiencies and
suggestions for improving the
instrument
…provides greater content validity
6. follow-up activities…
…because first mailing results tend to
be low (30% - 50% response rate),
researchers need strategies to
increase the response rate

initial follow-up strategies to increase
response rate up to 20%…
…send out reminder postcard
…mail a second packet with a new,
positively worded cover letter and
another self-addressed, stamped,
return envelope

secondary follow-up strategies to
increase response rate up to 10%…
…telephone nonrespondents to
encourage them to respond

dealing with nonrespondents
…study whether nonrespondents differ
from respondents in some
systematic way by selecting a
sample from the nonrespondents
…telephone sample, aggregate data,
include in report

dealing with nonresponses
…study the items to determine the
problem with the item
…include description of the
nonresponses and the determination
of the reason in the report
7. analyze/report results…
…discriminant item analysis includes
giving the response rate for each
item as well as the total sample size
and the overall percentage of
returns, since not all respondents
will answer questions
…group items into clusters that
address the same issue and develop
total scores across an item cluster in
order to avoid “information
overload”
Conducting an interview study…
1. state the problem
2. select participants
3. construct the interview guide
4. communicate professionally and
record responses accurately
5. pretest interview procedure
6. analyze/report results
1. state the problem…
…topic must be of sufficient
significance to motivate individuals
to participate and justify the
research effort in the first place
…define topic in terms of specific
objectives indicating the kind of
information needed
2. select participants…
…use an appropriate sampling
technique
…participants must be able to provide
the desired information sought and
willing to provide it to the researcher
3. construct the interview instrument
(“instrument guide”)…
…indicates the question to be asked,
the order, and how much additional
prompting or probing is permitted
…the goal is to obtain standardized,
comparable data from each
interviewee
4. communicate professionally and
record responses accurately…
…effective interviewing requires
training and experience to avoid
interviewer bias and interviewer
error
…record responses manually on the
interview guide
…(requires interviewee permission)
use a tape recorder or VCR to verify
accuracy of responses
5. pretest interview procedure…
…use a small group from the same
population or a similar population to
the one being studied to validate the
interview instrument and the
interviewer’s communication and
recording skills
6. analyze/report results…
…item analysis includes giving the
response rate for each item, as well
as the total sample size and the
overall percentage of returns, since
not every respondent will answer all
questions
…also include in-depth data to openended interview items to portray
responses in more accurate and
honest terms
Conducting an observational study…
1. state the problem
2. select participants
3. define the observational variables
4. record observations
5. assess observer reliability
6. reduce observer bias
7. analyze/report results
1. state the problem…
…topic must be of sufficient
significance to motivate potential
respondents and justify the research
effort in the first place
…define topic in terms of specific
objectives indicating the kind of
information needed
2. select participants…
…use an appropriate sampling
technique
…participants must be able to provide
the desired information sought and
willing to provide it to the researcher
3. define the observational variables…
…operationalize the variables to be
observed in terms of specific
behaviors that can be quantified
…ensure that the observations can be
quantified in a way so that all
observers will count the behavioral
activities in exactly the same way
…simplify procedure for recording
observations by developing a
coding procedure
4. record observations…
…record behavior at the time it occurs
…alternate observation periods and
recording periods
5. assess observer reliability…
…use at least two observers who make
independent observations
…compute interobserver reliability
6. reduce observer bias…
…train observers until a satisfactory
level of agreement is achieved (at
least 80%)
…monitor observers

types of observer bias
…response set
…halo effect
…knowledge of participants

response set
…the tendency of an observer to rate
the majority of observees as above
average, average, or below average
regardless of the observees’ actual
behavior

halo effect
…the tendency of an observer’s initial
positive or negative impressions of
the observee to affect subsequent
observations

self-fulfilling prophecy
…the tendency of an observer’s
knowledge of the observees or the
purposes of the study to affect the
observation(s)
Meta-Analysis...

a statistical approach to
summarizing the results of many
studies that have investigated
basically the same problem
…provides a numerical coefficient
expressing the “average” result
of the studies
…requiring the researcher to find,
describe, classify, and code the
research studies to be included
meta-analytic review, and for
measuring and analyzing the study
findings
…each study’s results are translated
into an effect size (ES) which is a
numerical expression of the strength
or magnitude of a reported
relationship, be it causal or not
Xe – Xc
ES = 
SDc
Where Xe = the mean score of the experimental group
Xc = the mean score of the control group
SDc = the standard deviation of the control group
Strength or magnitude of an effect
size (D)…
-.80
0.00
+.80
both groups strong positive
performed
similarly
experimental
control group
group performed
performed better
better than
than experimental
control group
group
strong negative
Mini-Quiz…

True or false…
…in a descriptive research study, the
researcher predetermines what
variables will be surveyed before
selecting or observing the research
participants
True
…one of the most difficult activities on
the part of questionnaire researchers
is to write or select questions that are
clear and unambiguous
True
…longitudinal surveys are useful for
studying the dynamics of a topic or
issue over time
True
…one of the problems with longitudinal
studies is that the samples tend to
shrink as time goes by
True
…one of the problems with crosssectional studies is selecting samples
that truly represent the population at a
particular level or ability
True
…an external review of an instrument
provides the researcher greater
assurance of it its content validity
True
…if the responses from respondents
and nonrespondents are essentially
the same, the researcher may
assume that the response group is
representative of the whole sample
and that the survey results are
generalizable
True
…if the responses from respondents
and nonrespondents are different,
the generalizability across both the
respondent and nonrespondent
groups is not present and must be
discussed in the report
True
…analyzing clusters of instrument
items related to the same issue
improves the reliability of the
scores
True
…bias can affect the validity of the
interpretations in observational
studies
True
…although there are no hard and
fast rules, it is generally agreed
than an effect size in the twenties
indicates a treatment that
produces a relatively small effect,
whereas an effect size in the
eighties indicates a powerful
treatment
True

Fill in the blank…
…a study requiring individuals to
respond to a series of statements or
questions about themselves
self-report study

Fill in the blank…
…a study in which individuals are not
directly asked for information but
data is obtained as the researcher
watches participants engage in a
situation
observation study

Fill in the blank…
…items on a survey that are answered
by circling a letter, checking a list, or
numbering preferences
close-ended items

Fill in the blank…
…the researcher guarantees that no
one, including the researcher, knows
who completed each questionnaire
anonymity

Fill in the blank…
…the researcher guarantees that no
one, except the researcher, knows
who completed each questionnaire
and promises not to divulge that
information
confidentiality

Fill in the blank…
…the oral, in person administration of
an instrument to each member of a
sample
interview

Fill in the blank…
…determining the current status of a
phenomenon not through an
instrument but as the researcher
watches the participants engage in
the activity being studied
observational study

Fill in the blank…
…an observation in which the
researcher watches behavior as it
normally unfolds
naturalistic observation

Fill in the blank…
…an observation in which the
researcher creates a situation to be
observed and tells participants what
activities to engage in
simulation observation

Fill in the blank…
…a coefficient found by dividing the
total number of agreed observations
by the total number of agreed and
disagreed observations
inter-observer reliability

Fill in the blank…
…a means by which observers record
observations at the time it occurs by
simplifying the recording process
coding

Fill in the blank…
…a statistical approach that
summarizes the results of many
studies having investigate the same
problem
meta-analysis
This module has focused on...
descriptive research
…which involves collecting data in
order to test hypotheses or to
answer questions about the
opinions of people about some
topic or issue
The next module will focus on...
correlational research
...which involves collecting data in
order to determine whether, and to
what degree, a relationship exists
between two or more quantifiable
variables