J. T. Stocks, Ph.D.
Group Designs and
Statistical Hypothesis Testing
Spurious Relationships
Age
Shoe
Size
Causal (Functional) Relationships
Test
Score
Intervening Variable Relationship
 Two variables are said to be functionally
related if all of the following three criteria
are met.
!Relationship: there is a relationship between the two
Seen as
threat
variables.
!Asymmetry: change in the “cause” variable results in
change in the “effect” variable, but not vice versa.
!Nonspuriousness: change in the “cause” variable
results in change in the “effect” variable regardless of the
actions of other variables.
Criticism
Aggression
Symmetry and Asymmetry
Causal (Functional) Relationships
Symmetrical Relationship
 Two variables are said to be functionally
related if all of the following three criteria
are met.
!Relationship: there is a relationship between the two
A
B
variables.
Asymmetrical Relationship
A
B
!Asymmetry:
change in the “cause” variable results in change
in the “effect” variable, but not vice versa.
!Nonspuriousness: change in the “cause” variable results
in change in the “effect” variable regardless of the actions of other
variables.
1
J. T. Stocks, Ph.D.
Dependent and Independent Variables
Threats To Validity
!Internal validity refers to the correctness
!Independent:
It is the causal variable; it is
the variable that is manipulated by the researcher.
!Dependent:
It is the effect variable; it is the
outcome variable.
of a conclusion about functional relationships
between variables. A conclusion is internally
valid to the extent to which it meets the
criteria for causality outlined previously.
!External validity refers to the correctness
of a generalization of a relationship beyond
the study sample.
Hypotheses
!Prediction About Relationship Between
Independent and Dependent Variable
!Empirical, Not Normative
!Testable
“Bad” Hypotheses
!Adults who have been sexually abused as
children may or may not remember the
abuse.
!As complexity increases, formalism
increases.
!Additional money ought to be allocated for
child care so that parents required to work
under the work provisions of the Temporary
Assistance to Needy Families Act will not
have to leave their children unsupervised.
Internal Validity - History
 An unanalyzed event (extraneous
variable) that may affect the dependent
variable. It is the “confounding”
variable that may lead to either the
conclusion that there is a relationship
between two variables when there is
none or that there is no relationship
when in fact there is one..
Internal Validity – Maturation
 The passage of time and the natural
changes that take place over time. In
absence of control, these changes may
be mistaken for the effect of an
independent variable.
2
J. T. Stocks, Ph.D.
Internal Validity - Testing
Internal Validity - Mortality
 The effects of being observed or
measured. More specifically, it refers
to the effect upon the behavior of the
individual being observed.
Internal Validity - Instrumentation
 Inaccuracy in the evaluation or
measurement process. Use of
measurement techniques with low
reliability would fall under this rubric.
Also included would be changes in the
measurement technique over the
course of a study.
Internal Validity - Regression
 This refers to the tendency of extreme
events to be followed by less extreme
events. Very high or very low scores
tend to regress, or move toward the
mean score.
 Mortality refers to the fact the some
individuals may drop out of a study and
their absence may have a significant
effect on the study’s outcome.
Internal Validity - Selection
 Pre-existing differences between
experimental and control groups may
be mistaken for the effect of an
independent variable.
Internal Validity Interactions with Selection
 This occurs when any of the previous effects
interact with selection. For example:
!A history-selection interaction would occur when one
group experiences certain events that the other does not.
!A regression-selection interaction would occur when one
group was more extreme on measures of the dependent
variable than the other.
!A mortality-selection threat would occur when there were
differences between groups with respect to dropout rates.
3
J. T. Stocks, Ph.D.
Internal Validity Diffusion of Treatment
 This refers to a situation where there is
communication between members of
the treatment and comparison groups
leading to some degree of duplication
of treatment between the groups.
Internal Validity Resentful Demoralization
 This occurs when the members of the
group receiving the perceived “less
desirable” treatment become
demoralized and simply “give up.”
Internal Validity Compensatory Equalization
Requirements for Generalization
 This threat also involves duplication of
treatments between groups by an
external agent (e.g., administrator,
therapist). This is usually based in the
belief that the experimental intervention
is more effective than the comparison
intervention.
!Consistency: There is a consistency inherent
in the situation under discussion that permits
generalization from observed events to
unobserved events
!Statistical Validity: The samples discussed in
the premises have been randomly drawn thus
allowing for exact evaluation of the inductive
probability of the argument.
Internal Validity Compensatory Rivalry
 This occurs when the members of the
group receiving the perceived “less
desirable” treatment are motivated to
exert themselves more strongly so as to
outperform the “privileged” group.
Standard Error of the Mean Population
 The Standard Error of the Mean is the
standard deviation of the distribution of sample
means.
 It is computed as
 σX = SEM = σX / √n
 where n is the sample size
4
J. T. Stocks, Ph.D.
Origins of the t Distribution
 William S. Gossett - writing
under the name of “Student” introduced the t-statistic and
its sampling distribution - the t
distribution.
 Gossett was a brewer when
engaged in his day job.
Two Distributions
 The z Distribution
X - µX

 z = ————
σX

 Only one z distribution
 The t distribution
X - µX

 t = ————
sX

 Potentially an infinite
array of t distributions.
 Different distribution
for each degrees of
freedom.
Standard Error of the Mean Estimated from Sample Data
 The Standard Error of the Mean is estimated
from sample data using the sample standard
deviation.
 It is computed as
 sX = SEM = sX / √n
 where n is the size of the sample used to
estimate the standard deviation.
Confidence Intervals
 Sets upper and lower limits for µX at a given
level of probability
!Upper Limit = X + t(αα/2tail,df) • sX
!Lower Limit = X - t(αα/2tail,df) • sX
 A 95% confidence interval = (1 - α) • 100%
 Thus, t(αα/2tail,df) would be t(.05/2tail,df)
 df = n-1
Compute Upper and Lower Limits
X
17
5
11
sX
10
10
7
n
25
9
16
Null Hypothesis and Causality
Criteria for Causality
!Relationship
!Asymmetry (Temporality)
!Nonspuriousness
5
J. T. Stocks, Ph.D.
Null Hypothesis and Sampling ErrorError-1
 H0: µ1 = µ2
 H0: P(A) = P(~A)
Assumptions for All Hypothesis Tests
!Randomness
!Independence
 The Null Hypothesis (H0) is the "no
difference" or "no relationship" hypothesis.
Null Hypothesis and Sampling ErrorError-2
 H0: µ1 - µ2 = 0
 H0: P(A) – P(~A) = 0
H0: µ1 - µ2 = 0
The Map Is Not The Territory - 1
Test: X1 - X2
Decision Rules and α and p
 p refers to the probability that a relationship or a
difference of a certain size would be seen in a
sample if the H0 were true.
 To reject H0, p must be less than or equal to α.
!Rejecting the H0: we believe that it is likely that
the relationship in the sample is generalizable to
the population.
!Not rejecting the H0: we do not believe we have
sufficient evidence to draw inferences about the
population.
6
J. T. Stocks, Ph.D.
OneOne- versus TwoTwo-Tailed Tests

Effect Size
|µ1 - µ2|
 d = ————

σ
 g = |P(A) - 0.50|
H0: µ1 - µ2 > 0
 Directional
 Nondirectional H0: µ1 - µ2 = 0
Program Comparison Change in Corporal Punishment
10
Small Effects
!Small Effect Size: d = 0.2. g = 0.05.
Behavior Management Mean = -4.36
Anger Management Mean = +3.19
8
6
Behavior
Anger
4
2
 This is the equivalent of a PVE of
approximately 0.01. It is approximately
the effect size for the average difference
in height (i.e., 0.5 inches and s = 2.1)
between 15- and 16-year-old girls
(Cohen, 1988).
0
-8 -7 -6 -5 -4 -3 -2 -1 0
1
2
3
4
5
6
7
8
The Alternative Hypothesis
(Experimental, Research)
Medium Effects
!Medium Effect Size: d = 0.5. g = 0.15.
 H1: µ1 - µ2 ≥ d
 H1: P(A) – P(~A) ≥ g
 This is the equivalent of a PVE of
approximately 0.06. It is approximately
the effect size for the average difference in
height (i.e., 1.0 inches and s = 2.0)
between 14- and 18-year-old girls (Cohen,
1988).
7
J. T. Stocks, Ph.D.
Large Effects
Type II Error and β Level
!Large Effect Size: d = 0.8. g = 0.25.
 This is the equivalent of a PVE of
approximately 0.14. This is the same
effect size as the average difference in
height for 13- and 18-year-old girls
(Cohen, 1988)
The Map Is Not the Territory - 2
Decision Rules - 1
H0: The defendant is innocent.
H1: The defendant is guilty of a specific crime.
!The defense and prosecution present evidence.
!The jury evaluates the evidence under the presumption of
innocence rule.
!The probability of the evidence under the presumption of
innocence is evaluated according to the criterion of
“reasonable doubt.”
Decision Rules - 2
H0: There is no relationship between X & Y.
H1: There is a specific relationship between X & Y.
H1: µ1 - µ2 ≥ 10
!The researcher collects sample data.
!A statistical procedure is used to determine the probability
(p) that the sample data could have occurred if the H0
were true.
!The value of p is compared with the α level. If p is less
than or equal to α , then the researcher rejects the H0.
8
J. T. Stocks, Ph.D.
Effect of Sample Size on Power
n=9
α = .05
σ = 60 minutes
d = 0.5
SEM = σ/√n
H0: µpost – µpre = µchange = 0
!Concern
"Nursing home residents participation in
rehabilitation is low
!Results of Intervention
"Increase of µchange = 30 minutes
d = µchange/SEM = _____ p = _____
Effect of Sample Size on Power
n = 25 α = .05 d = 0.5
SEM = σ/√n
σ = 60 minutes
H0: µpost – µpre = µchange = 0
!Concern
"Nursing home residents participation in
rehabilitation is low
!Results of Intervention
"Increase of µchange = 30 minutes
d = µchange/SEM = _____ p = _____
9