Jeremy Kees, Ph.D.

Sampling provides a means of gaining information about the population without the need to examine the population in its entirety
Sampling is far more important for descriptive/causal research than exploratory research

• A quality sample allows us to use a sample statistic to tell us something about a population parameter
• In other words, we can draw confident conclusions about how the larger population will behave without surveying everyone in that population!

• Usually not practical to collect try and collect data from the entire population of interest
– But sometimes it is!
• Workforce
• Villanova
• B2B customers

• Sampling is one “piece of the puzzle” to getting valid data
• The sample could be great, but if your measures are bad, the data will be bad
• The composition of your sample directly impacts the robustness of your learnings
– Sampling strategy relates to the internal or external validity of your findings??

• The degree to which the conclusions in our study would hold for other persons in other places and at other times

• The population you want to generalize to is different from your sample
• The setting/place you want to generalize to is different from the place you conducted your study
• The time period when you conducted your study was peculiar

How do we strengthen external validity??
• Use random selection from the population of interest
• Replicate your study with different samples, in different settings, and over time

• Includes method of selection, sample structure, and analysis plans
• The goal is to achieve a balance between
– Required precision
– Available resources

• Specify the target population as clear and complete as possible
– This is your “sampling frame”
• List of all units in your population
• Not always available
– In fact, it’s rarely available for me

Procedure for Drawing a Sample
Step 1 Define the Target Population
Step 2
Identify the Sampling Frame
Step 3 Select a Sampling Method
Step 4 Determine the Sample Size
Step 5
Step 6
Select the Sample Elements
Collect the Data from the
Designated Elements

Defining the Population (Step 1)
• Who are we interested in generalizing to?

• In some cases, we can identify all possible cases and then randomly select our sample
• However, in social science research, this is not typically the case
• We do our best to chose a sampling frame that will give us our best shot and obtaining a sample that is representative of the population of interest

• So we have decided who we’re going to sample from, now we have to decide HOW to go about doing it

Sampling Methodologies:
Probability Sampling
• Simple Random Sampling
– Systematic Random Sampling
• Stratified Random Sampling
• Cluster Random Sampling

Sampling Methodologies:
Nonprobability Sampling
• Convenience Sampling
• Modal Instance Sampling
• Expert Sampling
• Quota Sampling
• Heterogeneity Sampling
• Snowball Sampling

• Every member of the population has the same chance of being selected
• Advantages
– Simple
– Easy calculation of error
• Limitations
– Need complete and accurate population listing

• Clusters are formed and then a random sample is taken from each cluster
– Clusters are heterogeneous
– Geography
• Advantages
– Quick/easy (relative to other forms of random sampling)
– Don’t need complete population info

• Strata are formed and then a random sample is taken from each strata
– Stratas are homogenous segments
• Advantages
– Ensures underrepresented groups in the population are represented in the sample

• Stratify the population an important variable and then sample each stratum
• Advantages
– Quick/easy (relative to other forms of random sampling)
– Ensures representation from minorities in the population
• Use of quotas to get an approximate representative sample (CWL example)

• Using those who are willing to volunteer (or forced to participate)
• Advantages
– Quick/easy (relative to other forms of random sampling)
– Inexpensive
• Limitations
– Generalization problems

• Sample size is a function of…
– Effect size
– Alpha level (precision)
– Statistical power
• Sometimes difficult to estimate
– Sample size calculator

• TED Talk
• Analysis Issues to consider
– Collect as much data as you can reasonably afford
– It is often useful to look at segments

• Qualtrics
• Research Now
• CriticalMix
• Others…
• Even professional marketing research firms need accountability
• Attention screeners are highly recommended

• Cost by Incidence Rate (IR)
• Gen Pop (100% IR) = $4-5
– Mturk = $0.50 or less
• Specialized Sample (<5% IR) =
$20 - 25
– Adolescent smokers
• Physicians = $150+

• Adolescent Smokers
– Challenges??

• Veterinarians
– Challenges??

• Pet owners
– 65% female
– Age 32-49
– Urban/suburban
– $75k+ HHI
– Have visited the vet in the past year
– Pet sleeps indoors
– Must be a “conscientious” owner
• Eg, celebrate pets bday; purchase holiday gifts for pet; feel guilty about leaving the pet alone, etc.
• Challenges??

• Any research that will inform very important decisions that will have major impact
– Food/Tobacco policy decisions
(warnings/disclosures)
– Industry initiatives (FOP labeling)
• Challenges??
– As more $$ or human wellbeing is at stake, sampling becomes very important

** Don’t start on this assignment until you’ve read Fowler (CH 6-7)
• Based on your research design
1. Write a paragraph about what your measurement instrument is supposed to accomplish
2. Make a list of what should be measured to accomplish the goals of the study
3. Develop your measurement instrument

• Deliverables include:
1. A very clean, polished version that you could use to actually collect data
• This means you will need to carefully think through all of the issues we covered tonight
(e.g., set-up, ordering, length, multi-item scales, etc.)
2. Intro paragraph and variable list (see previous slide)
(Note: Don’t worry about defining your sample--you’ll have your chance to do that next week)

• Develop a sampling plan
1. Discuss your target market and define the sampling frame
2. Select a sampling method and explain why it is most appropriate for your study
3. Determine your sample size
4. Explain your data collection strategy
(e.g., how you would go about getting your sample to participate)