Unit 3
Surveys, Experiments, and Simulations
Part 1
Surveys
The Purpose of Surveys
Why Survey?
We need information to guide decisions.
 Conducting a census is either too hard or impossible.

Population
(parameter)
Sample
(statistic)
Mean
μ
x̄
Standard
Deviation
σ
s
Parameters are often impossible to know, so we
conduct good surveys to find statistics.
Types of Surveys
Census: Sample Size = Population Size




Prohibitively difficult to conduct
Powerful because they report parameters
μ (mu): “true mean” or population mean
σ (sigma): “true standard deviation” or population standard deviation
Observational Studies: Sample Size < Population Size




Easier to conduct
Weaker because they report statistics
𝑥 (x-bar): “estimated mean” or sample mean
s: “estimated standard deviation” or sample standard deviation
The Power of Statistics
If we can’t know the truth, what’s the point?


Statistics have been shown to be very good predictors of parameters
under certain conditions
Having a good idea of where a parameter is, if imprecise, is still useful!
When are Statistics good predictors of Parameters?



No Bias
Large Sample Size
If both of these conditions are met, the statistics generated by an
observation study can be considered useful.
Bias
What does bias mean?

Certain outcomes are systemically favored.

Example, if I ask volunteers to come up to a platform to be measured
for height at lunch, taller people might be more inclined to participate
for various reasons. Our observed statistics will not match the
population’s parameters.
Result of Biased Survey Design:



Taller individuals were more represented
X-bar > mu
s < sigma
Bias
How do you reduce bias?

Make sure everybody from the population of interest has an equal
chance to be part of the observational study.
Types of Bias:



Under Coverage Bias (Having a lower or no chance of being selected)
Non Response Bias (Refusing to participate)
Response Bias (Lying)
Bad Design:



Convenience Sample (Leads to under coverage bias)
Voluntary Response (Leads to non response bias)
Leading Questions (Essentially the survey conductor is lying)
Bias
How do you reduce bias?

Make sure everybody from the population of interest has an equal
chance to be part of the observational study.
Good Design:




Simple Random Sample (SRS)
Stratified Random Sample
Multistage Sampling Design
Cluster Sample
Sample Size
How big should a sample be?


The “Rule of Thumb” in Statistics is 15+ individuals.
This might seem low but even with that few participants, a statistics
can be surprisingly accurate!
Is bigger always better?

No. Simply because a large sample cannot compensate for bad
design. Having a balance of good design and sample size to produce
accurate statistics is the goal of this process.