EMR 6500:
Survey Research
Dr. Chris L. S. Coryn
Lyssa N. Wilson
Spring 2015
Agenda
•
•
•
•
•
Simple random sampling
Crafting good questions
Midterm examination
Case Study #1
Case Study #2
Simple Random Sampling
Simple Random Sample
• Recall that a simple random sample is a
sample of n elements from a population of
N in which each possible sample of size n
has the same probability of selection,
namely
1
æNö
ç ÷
ènø
• The probability of any element being
selected is equal to the ratio of the sample
size to the population size
Estimation of a Population
Mean and Total
Estimate of Population Mean
n
m̂ = y =
åy
i
i=1
n
æ n ö s2
V̂(y) = ç1- ÷
è Nø n
æ n ö s2
2 V̂ ( y ) = 2 ç1- ÷
è Nø n
Example for a Population Mean
• If n = 200 was selected from N =
1,000 and the sample mean was y =
94.22 with a sample variance of s 2 =
445.21, the bound on the error of
estimation, B, would be
æ 200 ö 445.21
æ n ö s2
2 V̂ ( y ) = 2 ç1- ÷ = ç1÷
è Nø n
è 1, 000 ø 200
= 2 1.7808 = 2.67
Estimate of Population Total
n
tˆ = Ny =
N å yi
i=1
n
2ö
æ
æ
ö
n s
2
V̂(tˆ ) = V̂ ( Ny ) = N ç1- ÷ç ÷
è N øè n ø
2ö
æ
æ
ö
n
s
2
2 V̂ ( Ny ) = 2 N ç1- ÷ç ÷
è N øè n ø
Example for a Population Total
• If n = 50 was selected from N = 750
and the sample mean was y = 10.31
(t = Ny = 750(10.31) = 7,732.5) with
2
s
a sample variance of = 2.25, the
bound on the error of estimation, B,
would be
2ö
æ
æ
ö
n s
50 öæ 2.25 ö
2æ
2
2 V̂ (tˆ ) = 2 N ç1- ÷ç ÷ = 2 ( 750) ç1÷ç
÷
è N øè n ø
è 750 øè 50 ø
= 2 23, 625 = 307.4
Selecting the Sample Size for
Estimating Population Means
and Totals
Sample Size for Estimating
Population Means
Ns 2
n=
2
N
-1
D
+
s
( )
• where
B2
D=
4
Sample Size for Estimating
Population Means
• Often, the population variance, s , is
unknown
• An approximate value of s can be
obtained by
Range
s»
4
2
Example for a Population Mean
• If N = 1,000 and the estimated
range is 100, the sample size
necessary to estimate m with B = 3
would be
Range 100
s»
=
= 25
4
4
• and
s = ( 25) = 625
2
2
Example for a Population Mean
• If
• where
Ns
n=
2
N
-1
D
+
s
( )
2
B (3)
D= =
= 2.25
4
4
2
• then
2
1, 000 ( 625)
n=
= 217.56
999 ( 2.25) + 625
Sample Size for Estimating
Population Totals
Ns 2
n=
2
N
-1
D
+
s
( )
• where
B2
D= 2
4N
Sample Size for Estimating
Population Totals
• If N = 1,000 and s 2 = 36.00, the
sample size necessary to estimate
the population total, t , with B =
1,000 would be
1, 000)
B
(
D= 2 =
= 0.25
2
4N 4 (1, 000)
2
• then
2
1, 000 (36.00)
Ns 2
n=
=
=125.98
2
( N -1) D + s 999 ( 0.25) + 36.00
Estimation of a Population
Proportion
Estimate of Population
Proportion
n
åy
i
p̂ = y =
æ n ö p̂q̂
V̂( p̂) = ç1- ÷
è N ø n -1
i=1
n
where
q̂ =1- p̂
æ n ö p̂q̂
2 V̂ ( p̂) = 2 ç1- ÷
è N ø n -1
Example for a Population
Proportion
• If n = 100 was selected from N =
300 and the sample proportion was p̂
= 0.15 the bound on the error of
estimation, B, would be
æ n ö p̂q̂
2 V̂ ( p̂) = 2 ç1- ÷
=2
è N ø n -1
= 2 ( 0.0293) = 0.059
æ 100 ö ( 0.15) ( 0.85)
ç1÷
è 300 ø
99
Selecting the Sample Size for
Estimating a Population
Proportion
Sample Size for Estimating
Population Proportions
Npq
n=
( N -1) D + pq
• where
q̂ =1- p̂
• and
B2
D=
4
Example for a Population
Proportion
• If N = 2,000 and the desired bound
on the error of estimation, B, were
0.05, and no prior information is
available to estimate p , the
necessary sample size would be
B ( 0.05)
D= =
= 0.000625
4
4
2
2
Example for a Population
Proportion
• Then
2, 000) ( 0.5) ( 0.5)
Npq
(
n=
=
( N -1) D + pq (1, 999) ( 0.000625) + ( 0.5) ( 0.5)
500
=
= 333.56
1.499
An Overview of Crafting Good
Questions
Issues to Consider
1. What survey mode(s) will be used
to ask the questions?
2. Is the question being repeated from
another survey, and/or will answers
be compared to previously collected
data?
3. Will respondents be willing and
motivated to answer accurately?
4. What type of information is the
question asking for?
Choosing Words and Forming
Question
1.
2.
3.
4.
5.
6.
7.
8.
9.
Make sure the question applies to the respondent
Make sure the question is technically accurate
Ask one question at a time
Use simple and familiar words
Use specific and concrete words to specify the
concepts clearly
Use as few words as possible to pose the question
Use complete sentences with simple sentence
structures
Make sure “yes” means yes and “no” means no
Be sure the question specifies the response task
Visual Presentation of Survey
Questions
1.
2.
3.
4.
5.
6.
7.
8.
9.
Use darker and/or larger print for the question and lighter
and/or smaller print for answer choices and answer spaces
Use spacing to create subgrouping within a question
Visually standardize all answer spaces or response options
Use visual design properties to emphasize elements that are
important to the respondent and to deemphasize those that are
not
Make sure words and visual elements that make up the question
send consistent messages
Integrate special instructions into the question where they will
be used rather than including them as freestanding entities
Separate optional or occasionally needed instructions from the
question stem by font or symbol variation
Organize each question in a way that minimizes the need to
reread portions in order to comprehend the response task
Choose line spacing, font, and text size to ensure the legibility
of the text
Additional Considerations
• Abilities/aptitudes
– Knowledge or skill
• Attitudes
– General and enduring evaluation of a
person, object, or concept
• Behaviors
– Actions and mannerisms
• Opinions/beliefs
– Subjective beliefs that occur as a result of
emotion or interpretation of facts
Closed-Ended Nominal
Questions
Closed-Ended Ordinal
Questions
Closed-Ended Ordinal
Questions
For each statement, select the response option that best indicates your degree of
agreement or disagreement with each statement.
1. People get what they deserve.
Strongly
agree
Agree
Disagree
Strongly
disagree
2. It is rare for a person to be wrongly sent to jail.
Strongly
agree
11
Strongly
disagree
22
33
44
55
3. Basically, the world is a just place.
Strongly
agree
Agree
Neither agree
nor disagree
Disagree
Strongly
disagree
4. Many people suffer through absolutely no fault of their own.
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Semantic Differentials
Visual Analogs
Guttman Scaling
• Individuals who endorse the first
response should also endorse all
others as they represent less
extreme views
Partially Closed
Open-Ended Questions
How would you describe the instructor’s teaching ability?
Filters/Skip Patterns
Do you intend to vote in the 2012 Presidential election?
Yes
No
Which political party will you vote for?
Democratic
Repulican
Libetarian
Green
Reform
Other, please describe
Midterm Examination
Midterm Examination
• The examination will consist of 50-75
multiple-choice items, scored as 0 or 1
• You will have 2½ hours to complete the
examination
• You may use one page of notes (front and
back) on 8½” X 11’’ paper
– You will be required to determine necessary
sample sizes and calculate bounds on the error
of estimation for means, totals, and
proportions
• You may use a calculator or an Excel
spreadsheet on a laptop computer
• You cannot use the internet or textbooks
Case Study #1
Case Study Activity
1. Write a focal question that addresses what
proportion of foundations formally assess the
effectiveness of their grant making activities
2. Determine the motivational features that you would
use to encourage high quantity and quality of
responses, with particular attention to:
–
–
–
How the perceived rewards for responding would be
increased
How the perceived costs of responding would be
reduced
How trust would be established so that people believe
the rewards will outweigh the costs of responding
3. Determine the necessary sample size to estimate
the population proportion, p, of foundations (N =
888) to address your focal question with a bound on
the error of estimation of B = 0.05 and B = 0.10
Case Study #2
Case Study Activity
• Consider the guidelines for
“choosing words and framing
questions” and “visual presentation
of survey questions”
1. Are there any errors associated with
the guidelines for choosing words and
framing questions?
2. Are there any errors associated with
the guidelines of the visual
presentation of survey questions?