Community Assessment Handout 4: Developing a Survey
Characteristics of a good Survey

Valid – accurately measuring the concept or idea that you want to measure.

Reliable – yielding consistent results over time when used the same way with similar
respondents.

Culturally appropriate – being appropriate for the target population in terms of language,
measurement approach, and questions asked.
o Minimize response options (using 2 response options vs. 4 response options:
“Agree” or “Disagree” vs. “Strongly Agree”, Agree”, “Disagree”, or “Strongly
Disagree”)
o Negative vs. positively framed item stems (“Not eating fruits is unhealthy” vs.
“Eating fruit is healthy”)

Focused – addressing your specific questions and only including questions that are essential
to know.
1
Community Assessment Handout #4
Steps to Consider When Developing a Survey
Developing a survey requires careful planning. The following are important steps to address in
survey development:
1. Purpose of the survey (Which health problem/behavior? Or a general assessment?)
2. Questions to include in a survey (Questions regarding factors associated with the health
problem/behavior) **

Example: if focusing on obesity, survey should includes questions regarding physical
activity, diet, parks in neighborhood, etc.
3. Review validated surveys to identify appropriate questions (See page 3 for websites with
standardized surveys) **
4. Adapt/Adopt questions and develop the survey **
5. Pilot test the survey (Format/length of survey, response categories, understood as
intended, etc.)
6. Also think about Survey Design: Method for Sample Selection & Survey Administration
(See pages 4-7)
** Consider sample selection when addressing these issues
2
Community Assessment Handout #4
Adopting Questions from Organizations
The following is a list of organizations that have websites with standardized questions that you
can review and adapt for your own use. Using existing validated questions can save you time
and cost, and will allow you to compare your results with findings from other surveys with
similar questions.

Centers for Disease Control and Prevention:
http://wwwn.cdc.gov/qbank/QuestionTopic.aspx

National Cancer Institute: GEM Database:
http://www.gem-beta.org/public/MeasureList.aspx?cat=2
http://healthservices.cancer.gov/surveys/screening_rp/screening_rp_breast_cervical_inst.p
df (Questions regarding physician attitude on breast and cervical cancer screening)

Behavioral Risk Factor Surveillance System (BRFSS):
http://apps.nccd.cdc.gov/BRFSSQuest/index.asp
http://www.cdc.gov/brfss/questionnaires/english.htm

California Health Interview Survey (CHIS):
http://www.chis.ucla.edu/

US Census:
http://www.census.gov/acs/www/Downloads/questionnaires/2012/Quest12.pdf

National Health Interview Survey (NHIS):
http://www.cdc.gov/nchs/nhis/quest_data_related_1997_forward.htm

LA County Health Survey:
http://publichealth.lacounty.gov/ha/hasurveyintro.htm
3
Community Assessment Handout #4
Sample Selection Methods
Types of samples: Probability and Non-probability sample
1. Probability sampling (Representative samples)
Probability samples are selected in such a way as to be representative of the population. They provide
the most valid or credible results because they reflect the characteristics of the population from which
they are selected (e.g., residents of a particular community, students at an elementary school, etc.).
Probability Sample Methods
Method
Simple Random:
Each individual in the
population of interest has
an equal likelihood of
selection.
Example
Put names of all students
in a hat, and then draw
200 students without
replacement.
Pros
-Relatively easy
-Highly
representative if all
subjects participate;
the ideal
Systematic Random:
The selection of every Nth
subject on a list or a
sequence.
Every 10th customer
entering the store may be
approached for a survey,
or every 25th number in
the phone book may be
called.
-More precise than
simple random as
more evenly spread
over population
Stratified Random:
In the survey of college
undergraduates, you may
choose to draw 50
students from each of the
freshman, sophomore,
junior, and senior strata,
instead of 200 students
from the combined
student population.
Randomly select 5 classes
of 40 students each. Each
class is called a cluster of
students.
- Can ensure that
specific groups are
represented, even
proportionally, in
the sample(s) (e.g.,
by gender), by
selecting individuals
from strata list
The population is divided
into groups (strata) and
the data is collected
from the strata by
simple random
sampling.
Cluster:
A particular segment of the
population is sampled using
existing lists (e.g., District,
Households, etc).
4
-Applicable where
no complete list of
units is available
Cons
-Not possible
without complete
list of population
members
-Time-scale may be
too long
-If list has periodic
(e.g., repeated
cycle) arrangement
then sample
collected may not
be an accurate
representation of
the entire
population
-Need a list or
mapping of
population
-More complex,
requires greater
effort than simple
random.
-Strata must be
carefully defined.
-Clusters may not be
representative of
whole population
but may be too alike
-Typically
“requires” larger
sample
Community Assessment Handout #4
2. Non-probability Samples (Non-representative samples)
As they are not truly representative, non-probability samples are less desirable than probability samples.
However, a researcher may be unable to obtain a random or stratified sample, or it may be too
expensive. A researcher may not care about generalizing to a larger population. The validity of nonprobability samples can be increased by trying to approximate random selection, and by eliminating as
many sources of bias as possible.
Non-probability Sample Methods
Method
Example
Quota sample:
To be sure of their
Deliberately set the
inclusion, a researcher
proportions of levels within could set a quota of 50%
the sample. Generally done males for the sample.
to insure the inclusion of a
particular segment of the
population. The researcher
sets a quota, independent
of population
characteristics.
Purposive Sample:
Purposefully choosing a
Non-representative subset
support group as a place to
of some larger population.
recruit participants.
Constructed to serve a very
specific need or purpose. A
researcher may have a
specific group in mind.
Researcher will attempt to
zero in on the target group,
interviewing whoever is
available.
Snowball Sample:
Find one member of a
Subset of Purposive Sample. (hard-to-reach) group;
Achieved by asking a
interview him/her then ask
participant to suggest
interviewee about other
someone else who might be individuals who may
willing or appropriate for
qualify as potential
the study.
participants.
Convenience Sample:
Conducting survey with
A matter of taking what you friends or going to the park
can get. Volunteers would
and asking anyone to take
constitute a convenience
the survey.
sample. Although selection
may be unguided, it
probably is not random.
5
Pros
-Ensures selection of
adequate numbers
of subjects with
appropriate
characteristics
Cons
-Not possible to
prove that the
sample is
representative of
designated
population
-Ensures balance of
group sizes when
multiple groups are
to be selected
-Samples are not
easily defensible as
being
representative of
populations due to
potential
subjectivity of
researcher
-Possible to include
members of groups
where no lists or
identifiable clusters
even exist (e.g.,
drug abusers,
criminals)
-Quick &
inexpensive
-No way of knowing
whether the sample
is representative of
the population
- Sample will most
likely not be
representative of
the population
Community Assessment Handout #4
Definition:
Sample: A subset of a population used to study the population as a whole.
Stratum: A layer, in this case refers to a subpopulation or level within a
population (pl.- strata).
Quota : A proportional part or share, the share assigned to each in a division
or to each member of a body.
Non-probability samples are limited with regard to generalization. Because they do not truly represent a
population, we cannot make valid inferences about the larger group from which they are drawn. Validity
can be increased by approximating random selection as much as possible, and making every attempt to
avoid introducing bias into sample selection.
 The best sampling is probability sampling, because it increases the likelihood of obtaining samples
that are representative of the population.
Sources:
http://www.stat.wmich.edu/s216/book/node28.html
http://psychology.ucdavis.edu/sommerb/sommerdemo/sampling/types.htm
http://psychology.ucdavis.edu/rainbow/html/fact_sample.html
http://staff.argyll.epsb.ca/jreed/math9/strand4/4106.htm
Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to
research design, measurement, and statistics. Thousand Oaks, CA: SAGE Publications, Inc. (p. 118)
6
Community Assessment Handout #4
Survey Methods Summary Sheet
Method
Face-to-face
Strengths
-Minimizes missing data
-Does not require high level of literacy from
respondent
-Can record reasons & characteristics on
non-consenters
-Allows much flexibility in length of survey &
style of questions
Weaknesses
-Expensive
-Relatively time-consuming
-Requires intensive interviewer
training & monitoring throughout
-Can be hard to prevent consultation
with/interference from others
Mailed Survey
-Relatively inexpensive method data
collection
-Widely dispersed populations can be
surveyed.
-No interviewer training required
-Data collection takes a long time
-Relatively low response rates
-Can result in higher amounts of
missing data
-High literacy level required
-Reminders required to increase
response rates
-Suitable only for fairly short &
straightforward surveys
-No information on non-consenters
Telephone Survey
-Widely dispersed populations can be
surveyed inexpensively & easily
-Minimizes missing data
-Relatively inexpensive data collection
method
- Relatively quick method of data collection
-Can use open-ended questions
- Can record reasons and characteristics on
non-consenters
-Does not require high level of literacy from
respondent
-Need to keep questions fairly short
and only a limited number of
response options
-Can’t ask questions requiring visual
cues
-Requires interviewer training &
monitoring throughout
-Can be hard to prevent consultation
with/interference from others
-Land phone line generally required
-Limits sample due to cell phone
prevalence
Online/Computer
Survey
-Can be relatively inexpensive once initial
costs have been overcome.
-Relatively quick method of data collection
-No interviewer training required
-Can use questions requiring visual cues
-Minimizes missing data
-Maximizes data quality
-Useful only for relatively large-scale
surveys
-High literacy level required
-Basic computer skills required from
respondents
-Limits sample due to requirement
of computer/online access
7
Community Assessment Handout #4
Adapted from Source: Newell, S. Perkins, J., Bowman, J.A. (1994). Accessing community samples: The
pros & cons of different methods. Paper presented to Hunter Centre for Health Advancement
Postgraduate Seminar, Newcastle, NSW. (Website: http://epubs.scu.edu.au/educ_pubs/203/)
8
Community Assessment Handout #4