Understanding the Variability of Your Data:
Dependent Variable
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
– Independent (Predictor/Explanatory) Variable(s)
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
– Independent (Predictor/Explanatory) Variable(s)
– Extraneous Variables
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
– Unsystematic
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
– Unsystematic
– Systematic
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to
extraneous variables
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to
extraneous variables
• Within conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to
extraneous variables
• Within conditions variability
• Individuals in same condition affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to
extraneous variables
• Within conditions variability
• Individuals in same condition affected differently
• Affects standard deviation, not mean, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to
extraneous variables
Common sources
individual differences
procedural variations
measurement error
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to
independent variable
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to
independent variable
• Between conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to
independent variable
• Between conditions variability
• Individuals in same condition affected similarly
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to
independent variable
• Between conditions variability
• Individuals in same condition affected similarly
• Individuals in different conditions affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to
independent variable
•
•
•
•
Between conditions variability
Individuals in same condition affected similarly
Individuals in different conditions affected differently
Affects mean, not standard deviation, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to
extraneous variable
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to
extraneous variable
• Between conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to
extraneous variable
• Between conditions variability
• Individuals in same condition affected similarly
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to
extraneous variable
• Between conditions variability
• Individuals in same condition affected similarly
• Individuals in different conditions affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to
extraneous variable
•
•
•
•
Between conditions variability
Individuals in same condition affected similarly
Individuals in different conditions affected differently
Affects mean, not standard deviation, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Roles played in the Research Situation
– Error Variability
• A nuisance – the ‘noise’ in the research situation
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability
• A nuisance – the ‘noise’ in the research situation
– Primary Variability
• The focus – the potentially meaningful effect
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability
• A nuisance – the ‘noise’ in the research situation
– Primary Variability
• The focus – the potentially meaningful effect
– Secondary Variability
• The ‘evil’ – confounds the results
Example
• Two sections of the same course
Example
• Two sections of the same course
• Individual’s score as combination of
‘sources’
Statistical decision-making
• The logic behind inferential statistics
• Deciding if there is ‘systematic variability’
– primary vs. secondary
• What do the data tell us?
• What decisions should we make?
Statistical decision-making
• A Research Example
– Research Hypothesis
– IF students chant the “Statistician’s Mantra”
before taking their Methods exam THEN they
will earn higher scores on the exam.
Statistical decision-making
• A Research Example
Your Class (M = 80, SD = 15, n = 25)
compared to a known
population Mean (M = 70) for a
standardized exam
Statistical decision-making
• A Research Example
Can estimate the Sampling Distribution
See if Population mean ‘fits’
Cause effect relationship not clear
Statistical decision-making
• A Research Example using experimental
approach
– Research Hypothesis
– IF students chant the “Statistician’s Mantra”
(vs. not chanting) before taking their Methods
exam THEN they will earn higher scores on
the exam.
Statistical decision-making
• Procedure
– Randomly divide class into two groups
• Chanters – are taught the “Statistician’s Chant”
and chant together for 5 minutes before the exam
Statistical decision-making
• Procedure
– Randomly divide class into two groups
• Chanters – are taught the “Statistician’s Chant”
and chant together for 5 minutes before the exam
• Non-chanters – sing Kumbaya together for 5
minutes before the exam
Statistical decision-making
• Results
– Compute exam scores for all students and
organize by ‘condition’ (levels of IV).
Show ‘changing’ distribution
Statistical decision-making
• Results
– Compute exam scores for all students and
organize by ‘condition’ (levels of IV).
– Compare Mean Exam Scores for two
Conditions
Statistical decision-making
• Results
– Compute exam scores for all students and
organize by ‘condition’ (levels of IV).
– Compare Means Exam Scores for two
Conditions
– What will you find?
Statistical decision-making
• Research Hypotheses generally imprecise
– Predictions are not specific
– So “testing” the Research Hypothesis, using
the available data, not reasonable
Statistical decision-making
• Null Hypothesis – a precise alternative
– Identifies outcome expected when NO
systematic variability is present
Statistical decision-making
• Null Hypothesis – a precise alternative
– Identifies outcome expected when NO
systematic variability is present
– But still must decide how close to the
predicted outcome you must be to ‘believe’ in
the Null Hypothesis
Statistical decision-making
• The Null Hypothesis Sampling Distribution
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null
Hypothesis is true
• (when there is no ‘systematic’ variability present in
the data)
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null
Hypothesis is true
– Finding all the possible outcomes?
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null
Hypothesis is true
– Finding all the possible outcomes?
– Seeing where your results fit into the Null
Hypothesis Sampling Distribution
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
•
• Reject Ho
Decision
• Not Reject Ho
•
“True” State of the World
Ho True
Ho False
Error
Correct Rejection
Correct
Nonreject
Error
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
“True” State of the World
•
Ho True
Ho False
• Reject Ho
Type 1 (p) Correct Rejection
Decision
(power = 1 – Type 2)
• Not Reject Ho Correct
Type 2
•
Nonrejection
• Deciding what confidence you want to have that you
have not made any errors
Statistical decision-making
• Trade-offs between Types of Errors
– I believe I can fly?
Statistical decision-making
• Trade-offs between Types of Errors
• Factors affecting Type 2 Errors (Power)
– “Real” systematic variability (size of effect)
– Choice of Type 1 probability
– Precision of estimates (sample size)
Statistical decision-making
• Trade-offs between Types of Errors
• Factors affecting Type 2 Errors (Power)
– “Real” systematic variability (size of effect)
• Assume .5 * SD, a moderate size effect is good
– Choice of Type 1 probability
• Use traditional .05
– Precision of estimates (sample size)
• Sample of 50 (2 groups of 25)
Statistical decision-making
• Factors affecting Type 2 Errors (Power)
– Type 2 error probability = .59
– Power = .41
Statistical decision-making
• Each ‘Decision” has an associated ‘error’
• Can only make Type 1 if “Reject”
• Can only make Type 2 if “Not Reject”
Statistical decision-making
Interpreting “Significant” Statistical Results
• Having decided to “reject” the Null
Hypothesis you can:
– State probability of Type 1 error
– State confidence interval for population value
– State percent of variability in DV ‘accounted for’
Statistical decision-making
Interpreting “Significant” Statistical Results
• For Chant vs. No Chant example
– State probability of Type 1 error
• .05
– State confidence interval for population value
• 95% CI is approximately +2 * SE
• Point estimate of 10 + 8 (Real difference between 2
and 18)
– State percent of variability in DV ‘accounted for’
• Eta2 = .20, or 20%
Statistical decision-making
Interpreting “Significant” Statistical Results
• Statistical Significance vs. Practical Significance
• How unlikely is the event in these circumstance
– versus
• How much of an effect was there
Statistical decision-making
Interpreting “Non-significant” Statistical Results
Having decide you cannot reject the Ho
State the estimated ‘power’ of your design