Psychology 242
Introduction
to Research
1
Course Overview Module
5/1/14
This module is best used as a PowerPoint
“Show”.
Go to “slide show” and click “run show”
Best way to print this:
 Click ‘File”  “Print’; In the dialogue box click
“print what?”.
 Select “Handouts (3 slides per page)”
© Dr. David J. McKirnan, 2014
The University of Illinois Chicago
McKirnanUIC@gmail.com
Do not use or reproduce without
permission
Click anywhere
Psychology 242
Introduction
to Research
Final Exam Review
2
This module is best used as a
PowerPoint “Show”.
Go to “slide show”, click “run show”
Best way to print this:
 Click ‘File”  “Print’; In the dialogue box
click “print what?”.
 Select “Handouts (3 slides per page)”
© Dr. David J. McKirnan, 2014
The University of Illinois Chicago
McKirnanUIC@gmail.com
Do not use or reproduce without
permission
Psychology 242, Dr. McKirnan
Cranach, Tree of Knowledge [of Good and Evil] (1472)
Psychology 242
Introduction
to Research
3
What is science?
What is science?
Values




Critical thought
Theory: Why? or How?
Evidence: How do you know?
Discover the natural world
Content
 Empirical findings: Facts
 Ways of classifying nature
 Well supported theories
Methods




Core empirical approach
Basic experimental design
Specific research procedures
Statistical reasoning
Psychology 242
Introduction
to Research
Irrational beliefs
 Critical thought – rational,
empirical-based analysis
– is cognitively effortful
 Our brains may be “hard
wired” for irrational
beliefs.
 Wish fulfilling, emotion-based beliefs:
• …self-satisfying; confirmatory bias
• …differentiating facts from opinions
• …emotional responses precede thought
 Cognitive biases:
• Spurious correlations
• Evaluating evidence
 Rationalism & science have
a tough row to hoe
Psychology 242, Dr. McKirnan
Week 12-13, quasi-experimental designs.
4
Psychology 242
Introduction
to Research
Four basic sources of knowledge or information:
How do we know things?

Authority:
Credible / powerful people
Social institutions
Tradition

Intuition:
Emotionality or a “hunch”
“Emotional IQ”

Empiricism:

Rationalism:
Psychology 242, Dr. McKirnan
Simple sensation / perception
Direct observation; data
Logical coherence
Articulation with other ideas
Most central to
Science
5
Psychology 242
Introduction
to Research

What does science do?
What does science do?
Describe the world
 Initial approach to scientific study: “what is it”
 Leads to hypotheses

Predict events
 Core feature of a hypothesis: if “X” then “Y”.
 Often still descriptive rather than experimental.

Test theories
 Cause and effect questions involving hypothetical constructs.
 Often controlled experiments or complex correlation designs.

Test applications of theories
 Using theory to model change
 Testing interventions or policy
Psychology 242, Dr. McKirnan Week 2: Role &
structure of science.
6
Psychology 242
Introduction
to Research

7
Basic features of a research study
Basic features of research;

Theory

Hypothetical construct

Hypothesis

Replication

Operational definition

Internal & external validity

Confound

Independent v. Dependent variables

 Click through
and be sure
you can
define each of
these.

Which is the “cause” & which is the “effect”?

Which is measured & which is manipulated?
Measurement v. experimental studies
Psychology 242, Dr. McKirnan
Weeks 1 & 2; Introduction to science.
Psychology 242
Introduction
to Research
Basic Elements of a Research Project
Phenomenon
Big picture / question
Begin with the “big question”
Core elements of a research study
Theory
Hypothetical Constructs
Causal explanation
Hypothesis
Operational definition
Specific prediction
Methods
Measurement v.
experimental
Data / Results
• Descriptive data
… articulate a clear theory
…and derive concrete
hypotheses.
Then specific methods, the
core of a scientific study.
Then actual data & results…
• Test hypothesis
Discussion
… implications for the theory
Implications for theory
Conclusions
Future research?
8
…and larger issues.
Psychology 242
Introduction
to Research
Core features of a research study:
Theory

Hypothesis

Methods

Data &
Analysis

Results

Discussion
Psychology 242, Dr. McKirnan








Hypothetical constructs
In important relationship
More specific variables
Falsifiable prediction
 Know these
key terms &
concepts.
Operational definition
Internal & external validity
Numerical representation
Normal distribution
Probability
Descriptive: Empirical question or exploration
Hypothesis: Statistical significance
Meaning of these results for the theory
Study Limitations:
 Internal validity?
 External validity?
9
Psychology 242
Introduction
to Research


Section 1 study guide
Core elements of the
research flow
Each component of the
research flow
corresponds to a later
component…
Psychology 242, Dr. McKirnan
Weeks 1 & 2; Introduction to science.
10
Psychology 242
Introduction
to Research
Research process:
The Big Picture
Phenomenon
Big picture question.
Theory 1
Possible explanation, invoking one
set of hypothetical constructs.
Hypothesis 1
A prediction that logically flows
from – and tests – the theory.
Methods 1
Operationally define the
variables & test the hypothesis.
Psychology 242, Dr. McKirnan
Theory 2
Alternate explanation, invoking
other hypothetical constructs.
Hypothesis 2
Another prediction that tests the
same theory.
Methods 2
An alternate operational definition
& way of testing the hypothesis.
Week 3; Experimental designs
11
Psychology 242
Introduction
to Research
Basics of Design: Internal Validity
12
Internal Validity:
Can we validly determine what is causing the results of
the experiment?
General Research Hypothesis: the experimental
outcome (values of the Dependent Variable) is caused only by
the experiment itself (Independent Variable).
Confound: a “3rd variable” (unmeasured variable other than the
Independent Variable) actually led to the results.
Core Design Issues:
1. Appropriate control group
2. Equivalent experimental & control groups (except for
the Independent Variable).
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
13
External validity: summary
External
Is the
sample Validity:
typical of the
Can we validly generalize
from this experiment to the
larger population?
larger world?
Is the
outcome
measure
representative,
valid &
reliable?
The
research
Sample:
The
Dependent
Variable
Is this
typical of
“real world”
The study
settings
structure & The research
Setting:
where the
context
phenomenon
The
Independent
Variable
occurs?
Does the experimental manipulation (or measured
predictor) actually create (validly assess…) the
phenomenon you are interested in?
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
14
Validity & Research approaches
Observation or Measurement
Simple Description
Qualitative
Quantitative
Explore the actual
process of a
behavior.
External
Describe a
behavioral or
social trend.
Experiments
Correlational
Studies
Quasiexperiments
“True”
experiments
Relate measured
variables to each
other to test
hypotheses.
Test hypotheses
in naturally
occurring events
or field studies.
Test specific
hypotheses via
controlled “lab”
conditions.
validity
Internal validity
Less control:
More control:

Observe / test phenomenon under
natural conditions.

Create the phenomenon in a
controlled environment

More accurate portrayal of how it
works in nature

Address specific questions or
hypotheses

Less able to interpret cause & effect

Better interpret cause & effect
 Know what these research strategies represent & how they differ.
 Understand the trade-off of internal & external validity across them.
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
15
Quasi-experiments
1. Study naturally occurring events that could not be
brought into a lab or a truedesigns
experiment.
Quasi-experimental

Measurement studies
Experimental
designs for “studies in nature”.

Retrospective designs
2. Evaluate existing groups or program(s)

Simple survey of an intervention that already occurred

Non-equivalent designs, due to

Time series designs, often with archival data
 Understand these two forms of
quasi-experiments.
 Understand these forms of nonequivalent designs.
Psychology 242, Dr. McKirnan
 Self-selection
 Non-random assignment
 Use of existing groups
 Participants not blind
Psychology 242
Introduction
to Research
True v. quasi-experimental designs, 3
True experiments:
Quasi-experiments:
Emphasize Internal Validity
 Assess cause & effect (in relatively artificial
environment)
 Test clear, a priori hypotheses
Emphasize External Validity

Describe “real” / naturally occurring events

Clear or exploratory hypotheses
Groups Equivalent at baseline
 Random Assignment (or matching).
 Participants & experimenter Blind to
assignment.
Non-equivalent groups




Control study procedures
 Create / manipulate the independent variable
 Control procedures & measures
Non-random assignment
Existing groups
Self-selection
Participants not blind.
Complete Control not Possible
 May not be able to manipulate the independent
variable
 Partial control of procedures & measures
 Know clearly how quasi-experiments differ from true experiments.
 In that light, know the core characteristics of an experiment and why those
characteristics are important.
Psychology 242, Dr. McKirnan
16
Week 12-13, quasi-experimental designs.
17
Introduction
Quasi-experiments
that do not have a control group:
to Research
Psychology 242
Group
Observe1
Intervention or event
Observe2
Observe1
Confound
Observe2
Threats to internal validity (confounds):
 History
Historical / cultural events occur between baseline &
follow-up.
 Maturation
Individual maturation or growth occurs between
baseline & follow-up.
 Reactive measures
People respond to being measured or being a
measured a second time.
 Statistical regression
Extreme scores at baseline “regress” to a more
moderate level over time.
 Mortality / drop-out
People leave the experiment non-randomly (i.e., for
reasons that may affect the results…).
 Know these!
 What is a confound? Why is that important?
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
18
Non-equivalent (quasi-experimental) designs
Two Group Pre- Post- Design
Group
Observe1
Intervention or event
Observe2
Group
Observe1
Contrast group
Observe2
Non-equivalent groups
 Self-selection
 Non-random assignment
 Use of existing groups
 Participants not blind
 Understand this slide.
Psychology 242, Dr. McKirnan
Intervention & Assessments often controlled by
researcher in these designs.
Similar to true experimental
design, except for nonequivalent groups
19
Sampling overview
Psychology 242
Introduction
to Research
Who do you want to generalize to?


Sampling
Who is the target population?

broad – external validity

narrow – internal validity
 What does this mean?
How do you decide who is a member?

demographic / behavioral criteria?

subjective / attitudinal?
 Why does this make a difference?
What do you know about the population already – what is the
“sampling frame”?
 Most externally valid & representative
Will you use a:
 Assumes: • Clear sampling frame
• Population is available
Probability or random sample?
 Less valid for hidden groups.

Non-probability or convenience  Less externally valid
sample
 Best when:
targeted / multi-frame
 snowball…

Psychology 242, Dr. McKirnan
 No clear sampling frame
 Hidden / avoidant population.
Psychology 242
Introduction
to Research
20
Ethics
Research Ethics:
The Tuskegee Study
The Common Rule
The Belmont Report
Psychology 242, Dr. McKirnan
Foundations of
REsearch

Tuskegee study begin as a potentially valuable trial of
treatment outcomes


Begun – and should have remained – a natural history of
participants’ response to treatment.
Became a wholly unethical no-treatment history.


Tuskegee Study: Overview
Based on spurious – and racist – scientific reasoning about
differences between Africans and Caucasians

Investigators took advantage of participants economic and social
vulnerability to exploit and harm them.

Note: Tuskegee participants were not actually given syphilis; they
were not given treatment.
Tuskegee led to many of our research norms and
institutional controls.
21
Psychology 242
Introduction
to Research
Ethics procedures stemming from Tuskegee
22
 Informed consent
 Non-coercive enrollment & retention
 Led to the 1979 Belmont Report
 Indirectly to core elements of the “Common Rule”.
 Ethical review & monitoring
 Led to establishment of the Federal Office for Human Research
Protections (OHRP)
 Led to laws requiring Institutional Review Boards (IRBs)
 All Federally funded research must be reviewed and monitored
by a local IRB
 Most institutions (e.g., UIC) require IRB approval of all
research, federally funded or not.
 Have a general sense of why Tuskegee was unethical, and how it influenced
our ethics decision making now
Dr. David J McKirnan, McKirnanUIC@gmail.com
Psychology 242
Introduction
to Research
The “Common Rule” criteria for Human Subjects Protection
The Common Rule
 Minimize risks
 Risks must be reasonable
 Recruit participants equitably
 Informed consent
 Understand what each of
these mean.
 Document consent
 Monitor for safety
 Protect vulnerable participants &
maintain confidentiality
Dr. David J McKirnan
23
Psychology 242
Introduction
to Research
Belmont Report
(CITI training)
1. Respect For Persons

Exercise autonomy & make informed choices.
2. Beneficence
 Minimize risk & maximize of social/individual benefit.
3. Justice
 Do not unduly involve groups who are unlikely to benefit.
 Include participants of all races & both genders
 Communicate results & develop programs/ interventions
 You know these from your
CITI training.
 Generally understand them;
be able to recognize these
key values.
Dr. David J McKirnan
24
Psychology 242
Introduction
to Research
25
Descriptive research
Quantitative
Qualitative or
Observational
Existing data
Describe an issue via
valid & reliable
numerical measures
Study behavior “in
nature” (high
ecological validity).
Use existing data for
new quantitative (or
qualitative) analyses
Simple: frequency
Qualitative
Accretion
 Interviews
 Study “remnants” of
behavior
counts of key
behavior
“Blocking” by other
variables
Correlational
research: “what
relates to what”
 Focus groups
 Textual analysis
Observational
 Direct
 Unobtrusive
 Wholly non-reactive
Archival
 Use existing data to
test new hypothesis
 Typically nonreactive
 What does it mean for research to be ‘reactive’?
Psychology 242, Dr. McKirnan
Descriptive Research.
Psychology 242
Introduction
to Research
26
Descriptive data
Testing hypothesis with Archival, Time Series data
 Archival data: Already exist, collected for another reason
 Time series:
“Snapshots” of a variable over time, sampling
different people each time
 Longitudinal: Follow the same cohort of people over time.
 Quasi-independent variable: naturally occurring event, e.g.
Magic Johnson testing positive for HIV  HIV testing rates?
 See next slide:
Psychology 242, Dr. McKirnan
Descriptive Research.
Psychology 242
Introduction
to Research
Psychology 242, Dr. McKirnan
Archival, time series data example: Magic Johnson
Descriptive Research.
27
Psychology 242
Introduction
to Research
28
Correlation designs: Drawbacks & fixes
Causality; a simple correlation may confuse cause & effect.
?
Depression
Alcohol
consumption
Confounds!; unmeasured 3rd variable problem
General optimism
Hemlines
?
Stock market
Dealing with confounds: Use complex measurements or samples to
eliminate alternate hypotheses.
 This slide illustrated the “3rd variable problem” in interpreting correlational
data.
 What does that refer to?
 Why is that important?
 Can you generate an example of that in a few words?
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
29
Descriptive Research: Overview
Basic design issues:
Reliability
Time frame
 Cross sectional
 Longitudinal
 Case study
 Test – retest
 Split – half
 Alpha (internal)
Validity
 Know what these terms mean. Go
back to the lecture notes or your
book for definitions & examples.
Psychology 242, Dr. McKirnan
Descriptive Research.





Face
Content
Predictive
Construct
Ecological
Psychology 242
Introduction
to Research
Statistics: an introduction
 Using numbers in science
 Number scales & frequency distributions
 Central Tendency: Mode, Median, Mean
 Variance: Standard Deviation
 The Z score and the normal distribution
 Using Z scores to evaluate data
 Testing hypotheses: critical ratio.
30
Psychology 242
Introduction
to Research
31
Distributions
Mode
Normal distribution: mean = mode =
Mean
Median
median at center of the distribution
 What are examples of data
that might fall into these
distributions?
Median
Mean
Bimodal distribution Mean & median
Mode
are similar, at the center.
Skewed distribution: Extreme scores in
one direction make the median, and mean
larger than the mode.
Mode
Median
Mean
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
32
Scales
Types of numerical scales
Psychology 242, Dr. McKirnan
Week 12-13, quasi-experimental designs.
Psychology 242
Introduction
to Research
33
Types of numerical scales
Ratio
zero point grounded in physical property; values
are “absolute”
continuous & equal intervals
Continuous
scales (scores
physical description: elapsed time, height
on a continuum)
Interval
no zero point; scale values relative
continuous with equal interval
behavioral research, e.g., attitude or rating scales.
 Be able to provide or
Ordinal
rank order with non-equal intervals; no ‘0’ point
Simple finish place, rank in organization...
Categorical
‘values’ = categories only
inherent categories: ethnic group, gender, zip code
Psychology 242, Dr. McKirnan
Exam #3 study guide
recognize examples
of these scale types
Psychology 242
Introduction
to Research
34
Scales and Central Tendency
Measure of Central tendency


Mode (most common score)

Median (middle of distribution) 

Mean (average score)

Typically used for:
categorical variables
often: bimodal distributions
categorical or continuous variables
highly skewed data
continuous variables only
more “normal” distributions
 use different
measures of central
tendency.
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
Two measures of variance
Measures of Dispersion or Variance
1. Range of the highest to the lowest score.

Provides simple idea of where scores fall

Very sensitive to any extreme score(s) (“outliers”).
2. Standard deviation of scores around the Mean

Similar to “average” amount each score deviates from the M.

“Standardizes” scores to a normal curve, allowing for basic statistics.

More accurate & detailed than range
 You should know these by now
Psychology 242, Dr. McKirnan
Exam #3 study guide
35
Psychology 242
Introduction
to Research
z
36
Z
You must know the Z score
 It is the core form of the critical ratio.
 It represents the:
 Strength of the experimental effect
 Adjusted by the amount of error variance
Z=
How far is your score (X) from the mean (M)
How much variance is there among all the
scores in the sample [standard deviation (S)]
Psychology 242, Dr. McKirnan
Week 12-13, quasi-experimental designs.
=
X–M
S
Psychology 242
Introduction
to Research
37
Z and the normal distribution

The normal distribution is a hypothetical distribution of cases in a sample

It is segmented into standard deviation units.

Each standard deviation unit (Z) represents a fixed % of cases

We use Z scores & associated % of the normal distribution to make statistical
decisions about whether a score might occur by chance.
 Remember
approximations of these
numbers
Psychology 242, Dr. McKirnan
 If you do not fully understand
this slide go back to the
Statistics 1 lecture notes and
figure it out!!
Exam #3 study guide
Psychology 242
Introduction
to Research
Normal distribution; Z scores
Use Z to evaluate a score
1.
Distance
from
M / (X)
“error”
variance
Calculate how
far the
score
is from
the mean (M); X–M.
2.
“Adjust” X–M by how much variance there is in the sample via
standard deviation (S).
3. Z = X–M / S
How “good” is a score of ‘6' in two groups?
Table 1, high variance
Table 2, low(er) variance
Mean (M) = 4, Score (X) = 6
Mean (M) = 4, Score (X) = 6
Standard Deviation (S) = 1.15.
(X-M = 6 - 4 = 2)
Z (X-M/S) = 2/1.15 = 1.74
Standard Deviation (S) = 2.4.
(X-M = 6 - 4 = 2)
Z (X-M/S) = 2/2.4 = 0.88
Psychology 242, Dr. McKirnan
Exam #3 study guide
38
Psychology 242
Introduction
to Research
39
Evaluating scores using Z
C. Criterion for a “significantly good” score
X = 6, M = 4, S = 2.4, Z = .88
If your criterion for a “good”
score is that it surpass 90%
of all scores…
X = 6, M = 4, S = 1.15, Z = 1.74
 With high variance a ‘6’ is
not “good”.
 With lower variance ‘6’ is
good.
70% of cases
 I need you to understand the
90% of cases
-3
-2
-1
0
+1
Z Scores
logic of this approach.
+2
(standard deviation units)
Psychology 242, Dr. McKirnan
Exam #3 study guide
+3
Psychology 242
Introduction
to Research
Core research questions
Data
40
Statistical Question
One participant’s score
Does this score differ from the M for the
group by more than chance?
Analyze with Z score
Means for 2 or more
groups
Is the difference between these Means
more than we would expect by chance?
-- more than the M difference between
any 2 randomly selected groups?
Analyze with t score
Scores on two
measured variables
Is the correlation (‘r’) between these
variables more than we would expect
by chance -- more than between any
two randomly selected variables?
Analyze with r
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
Summary
 Numbers are important for representing “reality” in
science (and other fields).
 Different measures of central tendency are useful &
accurate for different data;
 Mean is the most common.
 Median useful for skewed data
 Mode useful for simple categorical data
 Variance (around the mean) is key to characterizing a
set of numbers.
 We understand a set of scores in terms of the:
 Central tendency – the average or Mean score
 The amount of variance in the scores, typically the Standard
Deviation.
Psychology 242, Dr. McKirnan
Statistics introduction 1
41
Psychology 242
Introduction
to Research

42
Summary
Statistical decisions
follow the critical ratio:
 Z is the prototype critical ratio:
X–M
S
How far is your score (X) from the mean (M)
Z=
How much variance is there among all the scores in the
sample [standard deviation (S)]
=
 t is also a basic critical ratio used for comparing groups:
How different are the two group Means
t=
How much variance is there within each the two groups;
(“standard error of the mean”)
=
M1 – M2
Variance
n grp1
grp1

Variance
n grp2
grp2
 You must understand what a
critical ratio is.
 This slide needs to make
Psychology 242, Dr. McKirnan
Statistics introduction 1
perfect sense to you!!
Psychology 242
Introduction
to Research
Revised 4/5/0943
Dr. McKirnan, Psychology 242
Introduction to statistics # 2
 What can Plato’s Allegory of
the Cave tell us about
scientific reasoning?
 Was our hypothesis
supported? The critical ratio
and the logic of the t-test.
 The central limit theorem and
sampling distributions
 Correlations and assessing
shared variance
Statistics Introduction 2.
"The Allegory of the Cave" by Allison Leigh Cassel
Psychology 242
Introduction
to Research
44
Plato’s Cave, 6
What does Plato’s Allegory of the Cave tell us
about scientific reasoning?
We cannot observe “nature” directly, we only see its
manifestations or images:
 We are trapped in a world
of immediate sensation;
 Our senses routinely
deceive us (they have error).
 We cannot get outside
our limited sensations to
see the underlying “form”
of nature
Statistics Introduction 2.
Psychology 242
Introduction
to Research
45
Plato’s Cave, 2
We study hypothetical constructs; basic
“operating principles” of nature
 e.g., evolution, gravity, learning, motivation…



Processes that we cannot
“see” directly…
…that underlie events that
we can observe.
We test hypotheses
about what we can see and
use rational analysis –
theory – to deduce what
the “form” of these
processes must be, and
how they work.
Statistics Introduction 2.
46
Psychology 242
Introduction
to Research
Why can’t we just observe “nature” directly?
1. We can only observe the effects of hypothetical
constructs, not the processes themselves.
2. We examine only a sample of the world; no sample is
100% representative of the entire population
3. Our theory helps us develop hypotheses about
what we should observe if our theory is “correct”.
4. We test our hypotheses to infer how nature works.
5. Our inferences contain error: we must estimate the
probability that our results are due to “real” effects
versus chance.
 You must understand these
basic concepts and terms!
Statistics Introduction 2.
Psychology 242
Introduction
to Research
47
“Statistical significance”
Testing statistical significance
 We assume that a score with less than 5%
probability of occurring (i.e., higher or lower than 95% of the
other scores) is not by chance alone … p < .05)
 Z > +1.98 occurs < 95% of the time (p <.05).
 If Z > 1.98 we consider the score to be
“significantly” different from the mean
 To test if an effect is “statistically significant”…
 Compute a Z score for the effect
 Compare it to the critical value for p<.05; + 1.98
 Really important
Psychology 242, Dr. McKirnan
Statistics introduction 1
Psychology 242
Introduction
to Research
48
Statistical significance & Areas under the normal curve
95% of scores are between Z = -1.98 and Z = +1.98.
Z = -1.98
Z = +1.98
2.4% of
cases
2.4% of
cases
About 95%
of cases
-3
-2
-1
0
+1
+2
Z Scores
(standard deviation units)
Psychology 242, Dr. McKirnan
Exam #3 study guide
+3
Psychology 242
Introduction
to Research
With Z > +1.98 or < -1.98 we
reject the null hypothesis &
assume the results are not
by chance alone.
In a hypothetical
distribution:

2.4% of cases are higher
than Z = +1.98

2.4% of cases are lower
than Z = -1.98

49
Statistical significance & Areas under the normal curve
Thus, Z > +1.98 or < -1.98
will occur < 5% of the time
by chance alone.
34.13% 34.13%
of
of
cases
cases
Z = -1.98
of
cases
2.25%
of
cases
-3
-2
2.25%
of
cases
-1
0
+1
+2
Z Scores
+3
(standard deviation units)
Psychology 242, Dr. McKirnan
2.4% of
cases
95% of cases 13.59%
13.59%
of
cases
2.4% of
cases
Z = +1.98
Statistics introduction 1
50
Psychology 242
Introduction
to Research
Critical Ratio
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
51
Critical ratio
The strength of the results (our
Critical ratio =
direct observation of nature)
Amount of error variance (the odds
that our observation is due to chance)
t=
Difference between Ms for the two groups
Variability within groups (error)
Mgroup2
Mgroup1
Within-group
variance, group1
control group
Psychology 242, Dr. McKirnan
Within-group
variance, group2
experimental group
Exam #3 study guide
Psychology 242
Introduction
to Research
The Critical Ratio in action
 All three graphs have = difference
between groups.
 They differ in variance within
groups.
 The critical ratio helps us determine
which one(s) represent a statistically
significant difference.
Be able to answer these:
 How do the between group
variance & within group variance
constitute the critical ratio.
 t represents the critical ratio for
group comparisons: how does t
vary for these three examples?
 Which might reflect a statistically
significant difference?
Low variance
Medium variance
High variance
Statistics Introduction 2.
52
Psychology 242
Introduction
to Research
53
The Central Limit Theorem; small samples
Central limit theorem
True
Population M
“True” normal
distribution
 With few scores in
the sample a few
extreme or “deviant”
values have a large
effect.
The distribution
is “flat” or has
high variance.
Score
Score
Score Score
Score Score
<-- smaller
Statistics Introduction 2.
Score
Score Score Score
M
larger --->
Score
Psychology 242
Introduction
to Research
54
The Central Limit Theorem; larger samples
Central Limit Theorem
True
Population M
“True” normal
distribution
 With more scores the
effect of extreme or
“deviant” values is
offset by other values.
The distribution has
less variance & is
more normal.
Score
Score Score Score
Score
Score Score
Score Score
Score Score
Score Score Score Score Score Score
Score Score Score Score Score Score
Score Score Score Score Score Score Score
<-- smaller
Statistics Introduction 2.
M
larger --->
Psychology 242
Introduction
to Research
55
The Central Limit Theorem; large samples
Central Limit Theorem
 With many scores
“deviant” values are
completely offset by
other values.
 The distribution is
normal, with low(er)
variance.
 The sampling
distribution better
approximates the
population
distribution
Statistics Introduction 2.
True
Population M
Score Score
Score
“True” normal
distribution
Score
Score
Score
Score Score
Score
Score Score
Score
Score
Score Score
Score
Score
Score Score
Score
Score Score Score
Score
Score Score Score
Score
Score Score
Score Score
Score Score
Score Score Score Score Score Score
Score Score Score Score Score Score
Score
Score Score
<-- smaller
Score
M
Score Score
Score
larger --->
 Be able to apply the central limit theorem logic to evaluating t.
 Translate that to using the t table.
Psychology 242
Introduction
to Research
Central limit theorem & evaluating t scores
1. Smaller samples (lower df) have more variance.
2. So, t must be larger for us to consider it statistically
significant (< 5% likely to have occurred by chance alone).
3. Compare t to a sampling distribution based on df.
4. Critical value for t with p <.05 goes up or down
depending upon sample size (df)
Psychology 242, Dr. McKirnan
Exam #3 study guide
56
Psychology 242
Introduction
to Research
A t-table specifies Critical Values:
Alpha Levels
df
8
9
10
11
12
13
14
15
18
20
25
30
40
60
120

0.10
0.05
0.02
0.01
0.001
1.860
1.833
1.812
1.796
1.782
1.771
1.761
1.753
1.734
1.725
1.708
1.697
1.684
1.671
1.658
1.645
2.306
2.262
2.228
2.201
2.179
2.160
2.145
2.131
2.101
2.086
2.060
2.042
2.021
2.000
1.980
1.960
2.896
2.821
2.764
2.718
2.681
2.650
2.624
2.602
2.552
2.528
2.485
2.457
2.423
2.390
2.358
2.326
3.355
3.250
3.169
3.106
3.055
3.012
2.977
2.947
2.878
2.845
2.787
2.750
2.704
2.660
2.617
2.576
5.041
4.781
4.587
4.437
4.318
4.221
4.140
4.073
3.922
3.850
3.725
3.646
3.551
3.460
3.373
3.291
Critical values for testing
whether an effect is
Statistically Significant
Alpha = .05, df = 8
Alpha = .05, df = 18
Alpha = .05, df = 120
Alpha = .01, df = 40
Know how to use a t table.
 What is ‘Alpha’?
 What are Degrees of Freedom
(df)?
 What is a ‘Critical Value’?
57
Psychology 242
Introduction
to Research
Central Limit Theorem; variations in
sampling distributions
df = 120, t > ±1.98, p<.05
As samples sizes ( df ) go
down…
df = 18, t > ± 2.10, p<.05
the estimated sampling
distributions of t scores
based on them have more
variance,
df = 8,
 This increases
the critical value
for p<.05.
giving a more “flat”
distribution.
-2
t > ± 2.31, p<.05
-1
0
Z Score
+1
(standard deviation units)
 Get this! -- Be able to go to a t table and apply this logic.
 Give yourself the Statistics Lectures 2 notes for details.
+2
58
Psychology 242
Introduction
to Research
Taking a correlation approach
Correlations
t-test
 We create group differences
on the Independent Variable.
 …and assess how the groups
differ on the Dependent Var.
Difference between groups
standard error of M
Correlation;
 We measure individual differences
on the predictor variable…
 and see if they are associated with
differences on the outcome.
Σ (Z var1* Z var2)
Df (n-1)
Statistics Introduction 2.
59
Psychology 242
Introduction
to Research
60
Statistics summary: correlation
Pearson Correlation (r): measures how similar the
variance is between two variables (“shared variance”)
within a group of participants.

Are people a given amount above (or below) the mean of one
variable equally above (or below) the M of the 2nd variable?

We measure distance from M using Z scores.

r can range from -1.0 to +1.0

E.g., if participants who have Z = +1.5 on variable 1 also have Z
= 1.5 on variable 2, etc., r = +1.0.
r:  For each participant multiply the
Z scores for the two variables
 Sum across all participants
 Divide by df:
Psychology 242, Dr. McKirnan
Exam #3 study guide
r=
Σ (Z var1* Z var2)
Df (n-1)
61
Psychology 242
Introduction
to Research
Type I and Type II errors
Know what the Null Hypothesis is!*
*Any effect is due to chance alone
Psychology 242, Dr. McKirnan
Week 12-13, quasi-experimental designs.
Psychology 242
Introduction
to Research
62
Type I v. Type II errors
“Reality”
Accept Ho
Ho true
Ho false
[effect due to chance
alone]
[real experimental
effect]
Correct
decision
Type II error
Type I error
Correct
decision
Decision
Reject Ho
Statistics Introduction 2.
Psychology 242
Introduction
to Research

Statistical Decision Making: Errors
Type I error; Reject the null hypothesis [Ho]
when it is actually true:




Accept as ‘real’ an effect that is due to chance only
Type I error rate determined
by Alpha (.10, .05, .01…)
More “liberal” alpha (e.g., .10)
 reject Ho more often.
Worst form of error:
statistical conventions are
designed to prevent type I
errors
Statistics Introduction 2.
63
Psychology 242
Introduction
to Research
64
Type I v. Type II errors
“Reality”
Accept Ho
Ho true
Ho false
[effect due to chance
alone]
[real experimental
effect]
Correct
decision
Type II error
Type I error
Correct
decision
Decision
Reject Ho
Statistics Introduction 2.
Psychology 242
Introduction
to Research
65
Statistical Decision Making: Errors
Type II error;
Accept Ho when it is actually
false;

Assume as chance an effect
that is actually real.

Type II most strongly affected
by statistical power (df):

Central Limit Theorem:
Smaller samples
Assume more variance
More conservative
critical value*
Too conservative a critical value  Type II error
Statistics Introduction 2.
*within a given alpha…
Psychology 242
Introduction
to Research
66
Type I v. Type II errors
“Reality”
Accept Ho
Ho true
Ho false
[effect due to chance
alone]
[real experimental
effect]
Correct
decision
Type II error
Type I error
Correct
decision
Decision
Reject Ho
 Understand the logic of Type I & Type II errors.
 Be able to map these on to alpha levels and df in your study.
Statistics Introduction 2.
Psychology 242
Introduction
to Research



Inferential statistics: summary, Key terms
Plato’s cave and the estimation of “reality”

Hypothetical constructs  actual observations

Sample  population
Inferences about our observations:

Deductive v. Inductive link of theory / hypothetical constructs
& data

Generalizing results beyond the experiment
Critical ratio / Z

You will be asked to produce and describe this.

Variance, variability in different distributions

Degrees of Freedom [df]
Statistics Introduction 2.
67
Psychology 242
Introduction
to Research
Inferential statistics, cont.

t-test, between versus within –group variance

Sampling distribution, M of the sampling distribution

Alpha (α), critical value

t table, general logic of calculating a t-test

“Shared variance”, positive / negative correlation

General logic of calculating a correlation (mutual Z scores).

Null hypothesis, Type I & Type II errors.
Psychology 242, Dr. McKirnan
Week 12-13, quasi-experimental designs.
68
69
Psychology 242
Introduction
to Research
Multiple independent variables
 Testing hypotheses about > 1
independent variable
 Factorial Designs:
 Main effects,
 Additive Effects,
 Interactions
Psychology 242, Dr. McKirnan
4/14/09
Psychology 242
Introduction
to Research
> 1 independent variable
Designs with > 1 Independent Variable
Why have more than one IV?
 Include a ‘control’ variable
 Test 2 (or more) Independent variables
Psychology 242, Dr. McKirnan
Exam #3 study guide
70
Psychology 242
Introduction
to Research
> 1 independent variable
71
Include a ‘control’ variable as a second I.V.
1. Block the data by gender, age, race, attitudes, etc.
2. Test if the main Independent Variable has the same
effect within both groups
What is the effect of self-reflection on stress reduction?
EXAMPLE
 Hypothesis: training in self-reflection helps buffer the stress of
exams.
 2nd Question: is that effect the same in women and men? [old v.
young, etc…]
 Main effect: Self-reflection training  less stress
 Interaction: training  less stress worked for women, not men.
 Conclusion: Including a ‘control’ variable helped clarify the results.
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
> 1 independent variable
72
Testing more than one Independent Variable
A. Test separate, ‘main effects’ of each I.V.
(Do each of these variables significantly affect the
outcome?)
B. Test ‘additive’ effects of > 1 I.V.s
simultaneously (What is the combined effect of these
variables?)
C. Test interaction of 2 or more I.V.s (Does the
effect of one I.V. on the outcome depend upon a second
variable...?)
 Know the difference between a main effect, an
additive effect, and an interaction.
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
Interaction example: Genetics, stress and depression
Participants’ genotype and level of childhood trauma
interact in depression.
There is a
general (main)
effect whereby
more trauma
leads to greater
likelihood of
adult depression
Psychology 242, Dr. McKirnan
Exam #3 study guide
73
Psychology 242
Introduction
to Research
Interaction example: Genetics, stress and depression, 2
However … the effect of trauma interacts with genetics
 Understand clearly why/how this is an interaction,
not a main effect or additive effect.
 Also understand how the interaction tells us much
more than the simple main effect.
Childhood
trauma has no
effect in people
who have no
genetic
vulnerability.
With increasing
vulnerability,
increasing
trauma increases
the likelihood of
depression
Psychology 242, Dr. McKirnan
Exam #3 study guide
74
Psychology 242
Introduction
to Research
Example of a 3-way interaction
75
Figure 3 Mean ratings of subjective stimulation and sedation on the BAES
under 0.65 g/kg alcohol and placebo in women and men.
Alcohol (v. placebo) made
men much more stimulated.
Psychology 242, Dr. McKirnan
Alcohol made women much
more sedated
Multiple independent variables
Psychology 242
Introduction
to Research
Alternate portrayal of 3-way mood interaction
Placebo conditions do not show
much effect
The alcohol conditions show a
classic “cross-over” effect for
gender & mood;
 Why/how is this an interaction?
50
M BAES subscale scores
45
Men get aroused
40
35
Men, Alcohol
Men, Placebo
Women, Alcohol
Women, Placebo
30
25
20
15
10
Women get sedated
5
0
Stimulation
Psychology 242, Dr. McKirnan
Sedation
Multiple independent variables
76
Psychology 242
Introduction
to Research
Multiple IVs; summary 2
77
Multiple Independent Variables / Predictors:

Are critical to theory development and testing:
Stress or other environmental events can “switch on” genes that create
psychological or other problems; genetic dispositions and environment
are not separate processes.

Establish key “boundary conditions” to theory:
when and among whom does a basic
psychological process operate?
Alcohol makes it more difficult to inhibit behavior, but primarily among
men.
Psychology 242, Dr. McKirnan
Multiple independent variables
Psychology 242
Introduction
to Research

78
Summary
Key terms:

Main effect

Additive effect

Interaction

Cross-over interaction

Factorial design

Repeated measure
Psychology 242, Dr. McKirnan
Multiple independent variables
Psychology 242
Introduction
to Research
Complex experiments:
Within- subjects & blocking designs
 Own control
 Reversal designs
 Repeated measures &
Randomized block
designs
Psychology 242, Dr. McKirnan
79
Psychology 242
Introduction
to Research
Basic forms of within-subjects designs, 1
Basic forms of within subjects designs;
1. Own control

Each participant in control and experimental group.

Optimally, order is counter-balanced
2. Reversal designs
3. Repeated measures & Randomized block designs
Psychology 242, Dr. McKirnan
Exam #3 study guide
80
Psychology 242
Introduction
to Research
Basic forms of within-subjects designs, 3
Basic forms of Within subjects designs;
1. Own control
2. Reversal designs

Hypothesis: behavior controlled by clearly bounded condition

Design: “A – B – A”; impose – withdraw – impose condition
3. Repeated measures & Randomized block designs
Psychology 242, Dr. McKirnan
Exam #3 study guide
81
Psychology 242
Introduction
to Research
Basic forms of within-subjects designs, 2
Basic forms of Within subjects designs;
1. Own control
2. Reversal designs
3. Repeated measures

Multiple treatment conditions: each participant gets each
treatment.

Longitudinal / time sampling: each participant assessed over
multiple time periods

Randomized block designs: Repeated measure combined with
between-groups variable.
Psychology 242, Dr. McKirnan
Exam #3 study guide
82
Psychology 242
Introduction
to Research
83
Within subjects designs; own control, 2
1. Own Control Repeated Measures Design
Single
Group
Control
Condition
Observe1
All participants get the Control
Condition and measurement
Experimental
Condition
Observe2
All participants then get the
experimental intervention and
measurement.
 Hypothesis tested by differences between conditions (Observation1
v. Observation2) within group.
 Internal validity: eliminate possible confound of group differences
at baseline, since there is only one group.
Psychology 242, Dr. McKirnan
Exam #3 study guide
84
Psychology 242
Introduction
to Research
Reversal designs
2. “REVERSAL” DESIGNS
Test at baseline in normal state,
Test under temporary experimental condition
Test again under normal state.
Examples:
 Role of incentives in enhancing performance
 Impact of anti-depressant drug on mood
 Effect of self-awareness on following social norms
Psychology 242, Dr. McKirnan
Exam #3 study guide
Psychology 242
Introduction
to Research
Basic forms of within-subjects designs, 4
Basic forms of Within subjects designs;
1. Own control
2. Reversal designs
3. Repeated measures & Randomized block designs
 Combine a blocking variable with repeated measures.
 Common for:


Biomedical research
Behavioral intervention evaluations
Psychology 242, Dr. McKirnan
Exam #3 study guide
85
Psychology 242
Introduction
to Research
Randomized block design
Blocking Variable; between - subjects factor

Groups may be formed around a “Person” variable;

e.g., age or ethnic groups, groups based on an attitude measure…

Person variables are not “true” IVs; people not randomly assigned.
Or:

Experimental condition; drug dose, treatment, etc.

A “true” IV with random assignment
Repeated measure: within-subjects factor

Multiple treatment conditions:

Each participant is observed after each treatment condition

E.g., high v. low incentives, different instructional sets…
Or:

Longitudinal / time sampling:

Measure D.V. over multiple time periods (Cohort studies).

Here both the blocking variable and the repeated measures are
considered IVs.
Psychology 242, Dr. McKirnan
86
Psychology 242
Introduction
to Research
87
Within subjects designs; own control, 3
Repeated measures / randomized block design
Group 1
Baseline
Measure
Control
Condition
Measure2
M3
M4..
Group 2
Baseline
Measure
Experimental
Condition
Measure2
M3
M4..
Assignment
Randomly or via
natural “blocks”
Treatment vs. Placebo.
Primary Independent Variable.
Baseline assessment prior to
intervention or experimental
condition.
Psychology 242, Dr. McKirnan
Follow-up. Repeated Measures
assessment of the Dependent Variable.
Time is a 2nd Independent Variable.
Exam #3 study guide
88
Psychology 242
Introduction
to Research
There are two
Independent Variables:
Experimental treatment
(e.g., drug dose v. placebo)
Each IV may have a main
effect on the outcome
Time
(Repeated measures of the outcome variable)
If both IVs have main
effects the two together
would have an additive
effect on the outcome
Psychology 242, Dr. McKirnan
The core hypothesis would be
supported by an interaction effect
of treatment group by time.
Psychology 242
Introduction
to Research
89
Main effect example
Effect of drug treatment on systolic blood
pressure: This shows a Main Effect.
Imagine we are testing a new Statin
drug for high blood pressure.
60
The treatment group has overall lower Bp,
independent of time.
Mean systolic Blood Pressure
The study
200hypothesis is that drug
treatment will help lower Bp, with
50180
stronger
effects over time.
Blocking
variable
M = 160
Here are 160
some (made up) randomized
40
block, repeated measures data.
140
PEP users
Non users
Treatment
Placebo
30120
M = 106
100
20
80
1060
These data do not support the hypothesis that
drug treatment helps lower Bp:
40
0
Baseline
0
1
6
2
3 The treatment
4
5 group
6 was lower at baseline
treatment), and stayed lower over time.
Month of study(before
visit
12
18
24
30
36
 These data would suggest a problem with the
randomization: the groups were not equivalent
at baseline.
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
90
Main effect example
Effect of drug treatment on systolic blood
pressure: This also shows a Main Effect.
60
Both the treatment and control groups
show lower Bp over time.
200
Blocking
variable
Mean systolic Blood Pressure
50180
160
40
M = 147
140
30120
M = 105
100
PEP users
Non users
Treatment
Placebo
20
80
1060
40
0
Baseline
0
1
6
2
3
4 also5do not6support the hypothesis:
These
data
Month of study visit
12
 18
Both groups
over time.
24 got30better 36
 Drug vs. placebo treatment made no
difference.
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
91
Additive effect example
Drug treatment & systolic blood pressure:
Here is an example of an Additive Effect.
60
Both groups get better over time,
200
and the treatment group Blocking
has overall lower
variable
Bp.
Mean systolic Blood Pressure
50180
This ‘adds’ to a strong effect of treatment
at the later study visits.
160
40
140
PEP users
Non users
Treatment
Placebo
30120
100
20
80
These data also do not support the hypothesis:
1060
 Both groups did get better, and the additive effect of
group & time yielded the best outcome.
40
0
Baseline
0
1
6
2 However,
3
4 treatment
5
6

the
group
was lower at baseline,
tovisit
treatment.
Month prior
of study
12
18
24
30
36
 These data suggest that people just get better over
time, plus a problem with the randomization.
Psychology 242, Dr. McKirnan
Psychology 242
Introduction
to Research
92
Interaction effect example
Drug treatment & systolic blood pressure:
Here is an Interaction Effect.
60
The treatment group gets better over time.
200
The control group stays Blocking
stable.
Mean systolic Blood Pressure
50180
variable
40160
PEP users
Non users
Treatment
Placebo
140
30
120
20100
1080
The core hypothesis the this study is supported by this
60
0
Baseline
0
1
6
interaction effect.
2
3
4
5

The groups are equivalent at baseline.
Month of study visit
18 group
24 shows
30 an effect
36 over time, the
 12
The treatment
control group does not.
Psychology 242, Dr. McKirnan
6
Psychology 242
Introduction
to Research


Summary
93
Within – subjects designs are somewhat common in
psychological research;

Own control designs create a strong contrast for the
Independent Variable.

Since everyone gets all treatments, they eliminate problems in
creating experimental v. control groups.
Very common in biomedical or public health studies;


Most clinical studies are longitudinal; participants are
followed over time
The intervention or experimental treatment is I.V. #1 (blocking
or grouping variable).

Stability or change over time is I.V. # 2 (repeated measure).
Psychology 242, Dr. McKirnan