Survey Research
Chapter 17: How To Design And Evaluate Research In
Education
James Blackwood
AED 615 – Fall Semester 2006
Survey Research Topics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
What Is A Survey?
Why Are Surveys Conducted?
Types of Surveys
Correlational Research
Steps In Survey Research
Nonresponse
Problems With The Instrument
Evaluating Threats To Internal Validity
Data Analysis
Journal Article Employing Survey Research
What Is A Survey
1.
2.
3.
Gathering of a sample of
data or opinions considered
to be representative of a
whole group or population.
Information is collected
primarily by asking
questions.
Information is collected
from a sample (or portion)
of a population.
Why Are Surveys Conducted?
1.
Surveys are primarily used to
describe characteristics of a
population.
2.
Researchers use surveys to
examine the distribution of the
identified characteristics within the
population.
3.
The description of the population
as a whole is inferred by the
results obtained from the sample.
Useful Surveys (Arlene Fink, 1995)
1.
2.
3.
4.
5.
6.
Specific, measurable objectives
Sound research design
Sound choice of population or sample
Reliable and valid
Appropriate analysis
Accurate reporting of results
Types of Surveys
There are two major types
of surveys..
Cross-Sectional
Surveys
&
Longitudinal
Surveys
Cross-Sectional Surveys
Information collected from a
sample of a predetermined
population
 Information is collected at
approximately one point in
time
 If an entire population is
surveyed, it is described as
being a census (Like the US
Census that is conducted
every 10 years).

Longitudinal Surveys
Information is collected at different points in time to study
changes that occur over a time period
Trend Study – samples a population whose
members may change over the course of the
study
 Cohort Study - samples a population whose
members do not change over the course of the
study
 Panel Study – survey the same sample of a
population at different times during the course
of the study

Correlational Research
Correlation of the results of two separate surveys to
determine if a relationship exists
Scores or results of surveys are examined in this method
by calculating correlation coefficients or by preparing
contingency tables
Steps In Survey Research
1.
2.
3.
4.
5.
6.
7.
Define the problem
Identify the population
Choose the type of instrument to collect the
data
Design, construct, pilot and refine the
instrument
Select a representative sample
Administer the survey
Analyze, interpret, and communicate your
findings
Nonresponse
Nonresponse refers to members of a sample
who do not respond to the instrument
Total Nonresponse – not responding to the
entire survey/instrument
Item Nonresponse – not responding to some
of the questions within a
survey/instrument
Reducing Nonresponse
Examples of methods to reduce
nonresponse…
1.
Pretesting an instrument
2.
Training interviewers
3.
Providing incentives for
participation
Contacting non-respondents using
with an alternate form of the
instrument used for testing
for nonresponse error
Problems With The Instrument

Many issues can affect the
validity of an instrument
1.
Venue/Time (location or
time that the instrument is
presented)
Question preparation
(leading questions)
External issues
Characteristics of the data
collector
2.
3.
4.
Evaluating Threats To Internal
Validity

1.
2.
3.
4.
Four main threats to internal validity
Mortailty – removing data from lost
members of a study
Location threat – data collection in
locations that would affect responses
Instrumentation – defects in the
instrument itself
Instrument decay – interviewers get
tired or are rushed
Data Analysis
1.
2.
3.
4.
Summarize responses to
draw conclusions from the
results
Size of sample &
percentage of returns
reported
Percentage of total sample
responding for each item
reported
Percentage of respondents
who chose each alternative
for each question reported
Journal Article
Example of Survey Research
Public Perception of Extension (1996) – JOE 34(4)
Paul D. Warner , James A. Christenson, Don A. Dillman, & Priscilla
Salant
This article addressed the issues of how people's
perception/awareness and use of Extension changed over a 13year period. Telephone surveys of the U.S. population in 1982
and again in 1995 were utilized as the instrument in this study.
Thanks For Listening!