Truth in Human Development
& Family Studies
HD FS 503
Kevin Spacey in “Midnight in the Garden
of Good and Evil”:
? Truth, like art, is in the eye of the
beholder…
? You believe what you think
? And I’ll believe what I know.
Lecture Notes
February 4, 2002
Optimizing Criteria for
Research
1
2
Assignments
?
?
Sampling Frame
Very interesting, thoughtful analyses
Communication hints:
Avoid indefinite pronouns: this, it
Use past tense to describe all aspects of study
except theoretical conclusions
? Use while and since ONLY for time;
use although, because, for other relationships
? Distinguish between that (restrictive clause) and
, which (descriptive clause)
? Remove , which clause without altering meaning
? Avoid anthropomorphizing:
The authors found, NOT the study found
? Avoid quotes; paraphrase except for very
memorable statements
?
?
?
References: omit issue numbers
3
?
List of units from which we draw sample
? Faculty or School directory
? Telephone directory
? List of licensed child care centers in
county
? Membership directory of professional
organization
? Be sure that the sampling frame
accurately represents the population
? Not
a sampling frame of convenience!!!
phone book for 1931 election!!
? E.g.,
Challenges in sampling
Types of Sampling
? Unstable population, such that sampling
? Probability Sampling
frame quickly outdated
? Population for whom no sampling frame
exists
? Nonprobability Sampling
? Non-registered
4
family child care providers
population
? Potentially stigmatized
? Ethnic minority
? Gays
and lesbians
5
6
1
The most common probability
sampling techniques
Probability Sampling
? Requires random sampling of units from
population
? Allows generalization of findings to
population
? Selects participants or situations for
study
7
Other common probability
sampling techniques
Nonprobability sampling
Cluster sample:
? You want to study the impact of welfare
reform in Iowa on mothers’ employment
? You select several
?
Counties (rural and urban?)
Communities (large, small?)
?
Low income neighborhoods
?
?
Simple Random Sampling:
? Each unit of the population has an equal
chance of being selected
? Choice of any unit is independent of choice
of others
Proportional Stratified Random Sampling:
? For potentially moderating variables (e.g.,
gender, class), units chosen from each group
in proportion to the group size
? Caution: Strata must be accessible in
sampling frame!
8
Then do door-to-door canvassing
9
Sampling unit
? Sometimes used for convenience
? NOT a good idea if you hope to
generalize findings to a population!!!
? In qualitative research, it’s up to reader to
decide whether results can be transferred
to another context or group
? Why nonprobability sampling?
? For rarer populations
? For samples with no appropriate sampling
frame
? For VERY expensive samples
10
? e.g., clinical samples
Stratified Sampling
?
The family? child? classroom? student?
? Sampling unit is smallest unit receiving
treatment
? If teacher-taught curriculum,
?the classroom is the unit, not the
child!!
? If computer-based curriculum
?the child is the unit
? If parent-child interaction
? the
?
?
Divide members of sampling frame into
strata
? Each strata groups members who share a
characteristic
? This characteristic might affect results
? Examples of strata
gender
grade
? ethnic group
?
?
dyad is the unit
ANOVA allows testing at level of group
(first), then individual level (nested design)
?
11
May be studied as a moderating variable
12
2
Proportional Stratified
Sampling
Oversampling
?
Members selected from each strata
? According to distribution in population
? Multiple strata possible
? By sex within grade
? Sex and grade are potentially
moderating variables (“Treatments”)
? Test for interactions among independent
variables:
? for Treatment X, those who do better
may be
? boys in second grade
? girls in fifth grade
? Proportional sampling will produce too
small a sample to represent one strata
that has few members (e.g., one ethnic
group)
? Oversample strata with few members
13
14
Cluster sampling
Nonprobability sampling
? Randomly select clusters (typically,
? Judgmental sampling
defined by geographic proximity)
? Purposive sampling
? neighborhoods,
? Snowball (chain referral) sampling
? towns,
? Sequential sampling
? schools,
? classrooms
? Randomly select members within
clusters
15
16
Purposive sampling
Judgmental sampling
? NON -probability, Non-random!!!!!!!
? Special case of judgmental sampling
? Pick out certain interesting cases
? Often used in qualitative research
? Relies on your experience and
Most representative cases?
Cases likely to contradict previous results?
? Examples:
? High levels of education, low levels of income
? Hoffman-Reim : chose cases widely different
from each other
? N.B.: All sampling is purposeful; however, random
sampling is not purposive
?
?
knowledge of theory and previous
research findings
?
17
?
If I want to study elderly persons, I sample from the
population of elderly persons, not high school students!
This sampling is purposeful!!!!
18
3
Purposive sample
(e.g., Hoffman-Reim)
Typical randomly selected sample
6
40
Number of participants
30
20
10
0
5
4
3
2
1
Also known as chain referral sampling
? Ask one leader to identify other leaders in
community
? Ask those individuals to name others
? Keep going until you return to first names
? For a difficult-to-locate population (e.g.,
for study on unsafe sex practices among
homosexual males)
?
20
80
70
Quota sampling
?
?
60
Characteristic
19
Snowball sampling
90
10
0
Characteristic
50
30
0
10
10 20 30 40 50 60 70 80 90 100
40
0
0
20
Number of participants
50
? Keep knocking on doors until you fill
preset quotas
? single parent families
? DINK’s
? two parent families
? retired
couples
? single persons
More common in qualitative research
May lead to biased sample in quantitative
research
? Danger!!!
? convenience
leads to bias
21
22
What sampling strategy have
we omitted?
Sampling strategy
? Convenience sample
? Choose sampling technique
? Whoever responds
? Choose unit of analysis
? Whoever volunteers
? Choose sampling frame
? Whoever is handy
? Identify steps
(e.g., college sophomores)
? Zimbardo’s
sample?
? Danger: limits generalizability
23
24
4
Design a sampling strategy for
? The effect of premarital counseling on
couple communication skills
Complexities in the Concept of
Cause
? Which is the cause in the causal chain?
? The effect of welfare reform on family
? Causes as inferences
self-sufficiency
? The effect of home visiting infants on
parenting skills
? The relationship between gender and
wages in the human service or
hospitality industry
? Validated causal propositions
escape disconfirmation
? NEVER confirmed!
25
Causal evidence
What’s wrong with only measuring
hypothesized cause?
Requirements for cause:
? Cause precedes effect
?
What happens if alternate cause
covaries with hypothesized cause?
Implications for a cross -sectional study?
?
Presence of effect does not reflect a
chance deviation
? Result does not occur without cause
? Congruence:
larger change in IV ? larger change in DV
? Control for alternate cause
? Effect follows a detailed prediction
? Rival explanations eliminated
Be sure to review these for Exam I!
Effect
Cause 1
Alternate
cause
27
Four cells needed to separate cause
from alternate cause
Cause
Alternate
cause
High
26
28
Common confounded
variables
? Education & income
High
Low
Effect High
Effect Low
? Marital status & income
? Ethnicity & income/education
? Interest & experience
? Ability & experience
Low
Effect High
Effect Low
29
30
5
Alternative explanations?
Controls?
Chapter 9
Other Criteria for Research
? Childhood poverty leads to lower
levels of academic achievement
? Active listening leads to increased
marital satisfaction
? Enhanced parental involvement leads
to enhanced academic performance
? Earlier parent-child separation leads to
insecure adolescent attachment
31
32
Criteria to be optimized
Audience Credibility
? Audience credibility
? Validity is subjective
? Plausibility of conclusion is critical
? Was
hypnotic suggestion a credible
strategy to induce deafness in Zimbardo’s
study?
? Plausibility lies in eye of beholder
? Avoid having a bias or interest in the
? Balance of internal and external validity
outcome
? Resource allocation
33
34
Relative Weighting of
Building Audience Credibility
?
? Build on already accepted knowledge
Internal Validity (LP)
?
? Avoid weaknesses of previous studies
? Use accepted techniques
?
? Justify any use of the nonstandard
External Validity (GP)
?
? E.g., Zimbardo’s
use of hypnotically
induced deafness
?
35
Does the evidence in the study support the
existence of a relationship between or among
its variables?
Does the relationship between/among the
variables in a study generalize beyond the
specific characteristics of the study in which it
was found?
N.B.: Don’t confuse with test validity!!!
36
6
Threats to
internal validity
Balancing Internal and
External Validity
? Control strengthens internal validity;
? Artificiality threatens external validity
?
Strengthen internal validity by
narrowing scope
? How is internal validity strengthened
by conducting research in?
?
?
? Studying infant attachment
? Strange
Situation Behavior Instrument
science of strange behavior of
children in strange situations with
strange adults for the briefest possible
time.”
? “the
ISU CD Laboratory School
How is external validity threatened
by conducting research in?
?
ISU MFT Clinic
37
Resource Allocation
38
Limits and constraints
? Personnel
? Ethical standards
? Travel
? Instruments
? Institutional Constraints
? Resource Limits
? Equipment
? Office
? Telephone
? Copy machine
? Postage
39
Ethical standards
?
40
Institutional Constraints
?
First do no harm….
?
?
Access to participants
?
Questions permitted
?
IRB approval
? Participant informed consent
? Parent informed consent
?
Time and location
How much time will each person consent to be
interviewed?
? How long a survey will each respondent
complete without tossing it in the wastebasket?
? Longer survey ?
?
Confidentiality
Mandated reporter?
?
More on ethics on February 25;
complete ISU training on the web:
? http://www.research.iastate.edu/training/
Institutional Review Board
Human Subjects Committee
?
Informed consent
?
?
Review by IRB
?
Tuskegee study on syphilis
? Makes research on vulnerable subjects difficult
because of limited access
?
?
?
41
?
higher refusal (attrition) rate
less representative sample!
Conclusions permitted????
42
7
Resource Limits
? Available staff
? Equipment and/or Instruments
? Money
? Travel,
postage, phone
for administration
? Time for data entry
? Time for data analysis
? Time
43
8