Name that tune.
Song title? Performer(s)?
R.G. Bias | rbias@ischool.utexas.edu |
1
Scientific Method (continued)
“Finding New Information”
3/21/2011
R.G. Bias | rbias@ischool.utexas.edu |
2
Objectives
 I want to arm you with a scientist’s skepticism, and a
scientist’s tools to conduct research and evaluate others’
research.
-
Randolph – remember to take roll.
R.G. Bias | rbias@ischool.utexas.edu |
Operational Definitions
 Explains a concept solely in terms of the
operations used to produce and measure it.
–
–
–
–
–
–
4
Bad: “Smart people.”
Good: “People with an IQ over 120.”
Bad: “People with long index fingers.”
Good: “People with index fingers at least 7.2 cm.”
Bad: Ugly guys.
Good: “Guys rated as ‘ugly’ by at least 50% of the
respondents.”
R.G. Bias | rbias@ischool.utexas.edu |
Validity and Reliability
 Validity: the “truthfulness” of a measure. Are
you really measuring what you claim to
measure? “The validity of a measure . . . the
extent that people do as well on it as they do on
independent measures that are presumed to
measure the same concept.”
 Reliability: a measure’s consistency.
 A measure can be reliable without being valid,
but not vice versa.
5
R.G. Bias | rbias@ischool.utexas.edu |
Theory and Hypothesis
 Theory: a logically organized set of propositions
(claims, statements, assertions) that serves to
define events (concepts), describe relationships
among these events, and explain their
occurrence.
– Theories organize our knowledge and guide our
research
 Hypothesis: A tentative explanation.
– A scientific hypothesis is TESTABLE.
6
R.G. Bias | rbias@ischool.utexas.edu |
A bit more about theories
 Good theories provide “precision of
prediction”
 The “rule of parsimony” is followed
– The simplest alternative explanations are
accepted
 A good scientific theory passes the most
rigorous tests
 Testing will be more informative when you
try to DISPROVE (falsify) a theory
– Start with a null hypothesis
7
R.G. Bias | rbias@ischool.utexas.edu |
If results of an experiment . . .
 . . . (a well-run experiment!) are consistent
with theory, we say we’ve supported the
theory. (NOT that it is “right.”)
 Otherwise, we modify the theory.
 Testing hypotheses and revising theories
based on the outcomes of experiments –
the long process of science.
8
R.G. Bias | rbias@ischool.utexas.edu |
Goals of Scientific Method
 Description
– Nomothetic approach – establish broad generalizations and
general laws that apply to a diverse population
– Versus idiographic approach – interested in the individual, their
uniqueness (e.g., case studies)
 Prediction
– Correlational study – when scores on one variable can be used
to predict scores on a second variable. (Doesn’t necessarily tell
you “why.”)
 Understanding – con’t. on next page
 Creating change
– Applied research
9
R.G. Bias | rbias@ischool.utexas.edu |
Understanding
 Three important conditions for making a
causal inference:
– Covariation of events. (IV changes, and the
DV changes.)
– A time-order relationship. (First the scientist
changes the IV – then there’s a change in the
DV.)
– The elimination of plausible alternative
causes.
10
R.G. Bias | rbias@ischool.utexas.edu |
Experimental Design
 Description and Prediction are crucial to the
scientific study of behavior, but they’re not
sufficient for understanding the causes. We
need to know WHY.
 Best way to answer this question is with the
experimental method.
 “The special strength of the experimental
method is that it is especially effective for
establishing cause-and-effect relationships.”
11
R.G. Bias | rbias@ischool.utexas.edu |
Confounding
 When two potentially effective IVs are allowed to covary
simultaneously.
– Poor control!
 Men, overall, did a better job of remembering the 12
“random” letters. But the men had received a different
“clue.”
 So GENDER (what type of IV? A SUBJECT variable, or
indiv. differences variable) was CONFOUNDED with
“type of clue” (an IV).
12
R.G. Bias | rbias@ischool.utexas.edu |
Populations and Samples
 Population: the set of all cases of interest
 Sample: Subset of all the population that
we choose to study.
13
Population
Sample
Parameters
Statistics
R.G. Bias | rbias@ischool.utexas.edu |
Logic of Experimental Research
 Researchers manipulate an independent
variable in an experiment to observe the
effect on behavior, as assessed by the
dependent variable.
14
R.G. Bias | rbias@ischool.utexas.edu |
Independent Groups Design
 Each group represents a different
condition as defined by the independent
variable.
15
R.G. Bias | rbias@ischool.utexas.edu |
Random . . .
 Random Selection vs. Random Assignment
– Random Selection = every member of the population
has an equal chance of being selected for the
sample.
– Random Assignment = every member of the sample
(however chosen) has an equal chance of being
placed in the experimental group or the control group.
• Random assignment allows for individual differences among
test participants to be averaged out.
16
R.G. Bias | rbias@ischool.utexas.edu |
Example
– Theory: Vitamins help humans’ immune systems fight
off disease
– Hypothesis: Taking vitamin C will lead to fewer colds
– IV – Vitamin C (some vs. none, every day)
– DV -- # of colds in a year
– H0 – Vitamin C doesn’t influence # of colds
– HA – Vitamin C reduces # of colds
– Select sample (random selection?)
– Random assignment!
R.G. Bias | rbias@ischool.utexas.edu | 17
Another example
 A researcher was interested in the affect of ambient
noise on ability to concentrate. She operationally
defined “noise” as a pneumatic jackhammer at 80
decibels, and ability to concentrate as time it took to
complete a Sudoku puzzle. Invited 20 PSY 301 students
into the lab, one at a time. First 10 completed the puzzle
in quiet environs. Next 10 completed the same puzzle
with the jackhammer sound just outside the window.
Found that people in a quiet setting completed the
puzzle on the average 2.5 minutes faster.
– What’s the IV? The DV? H0? HA? Was this a good
experiment?
R.G. Bias | rbias@ischool.utexas.edu | 18
Let’s step back a minute
 An experiment is “personkind’s way of asking
nature a question.”
 I want to know if one variable (factor, event,
thing) has an effect on another variable – does
the IV systematically influence the DV?
 I manipulate some variables (IVs), control other
variables, and count on random selection to
wash out the effects of all the rest of the
variables.
19
R.G. Bias | rbias@ischool.utexas.edu |
Challenges to Internal Validity
 Testing intact groups. (Why is the group a group? Might
be some systematic differences.)
 Extraneous variables. (Balance ‘em.) (E.g.,
experimenter).
 Subject loss
– Mechanical loss, OK.
– Select loss, not OK.
 Demand characteristics (cues and other info participants
pick up on) – use a placebo, and double-blind procedure
 Experimenter effects – use double-blind procedure
20
R.G. Bias | rbias@ischool.utexas.edu |
Notice
 Many things influence how easy or hard it
is to discover a difference.
– How big the real difference is.
– How much variability there is in the population
distribution(s).
– How much error variance there is.
– Let’s talk about variance.
21
R.G. Bias | rbias@ischool.utexas.edu |
References
 Hinton, P. R. Statistics explained.
 Shaughnessy, Zechmeister, and
Zechmeister. Experimental methods in
psychology.
R.G. Bias | rbias@ischool.utexas.edu | 22