Name that tune.
Song title? Performer(s)?
R.G. Bias | rbias@ischool.utexas.edu |
1
Scientific Method
“Finding New Information”
3/22/2010
R.G. Bias | rbias@ischool.utexas.edu |
2
First . . .
 Reaction to a mid-course class evaluation.
– Just 8 respondents.
– Mostly positive.
– One answer that troubled me:
Course objectives and assignments are clearly stated.
25% (2) 50% (4) 12.5% (1) 12.5% (1) 0% (0)
 Level-setting. Where are we and where
are we headed?
R.G. Bias | rbias@ischool.utexas.edu |
3
Objectives
After this class (or these 3 weeks of classes) you will (it is
my hope!):
 know something about how scientists (information
scientists) gather new information.
 AND you’ll be good at evaluating information others offer
you.
 I want to arm you with a scientist’s skepticism, and a
scientist’s tools to conduct research and evaluate others’
research.
-
Randolph – remember to take roll.
R.G. Bias | rbias@ischool.utexas.edu |
 There are three kinds of lies: lies, damned
lies, and statistics.
– Benjamin Disraeli (1804 – 1881), British
politician
5
R.G. Bias | rbias@ischool.utexas.edu |
 Statistics are like a bikini. What they reveal
is suggestive, but what they conceal is
vital.
– Aaron Levenstein, U.S. politician
6
R.G. Bias | rbias@ischool.utexas.edu |
 The statistics on sanity are that one out of
every four Americans is suffering from
some form of mental illness. Think of your
three best friends. If they're okay, then it's
you.
– Rita Mae Brown, U.S. author
7
R.G. Bias | rbias@ischool.utexas.edu |
First . . .
 There are two components of this and any
class: Instruction and Evaluation.
 Let’s get the evaluation out of the way,
early.
 Need one volunteer.
8
R.G. Bias | rbias@ischool.utexas.edu |
“Research shows . . .”
 Finger length is a good (and quick!)
indicator of intelligence.
 One volunteer – measure your finger
length in cm. However.
9
R.G. Bias | rbias@ischool.utexas.edu |
Hmmmm . . .
 Everyone in the class will get a grade of
“C”
 But still, we can continue with the
“instruction” part of the course.
10
R.G. Bias | rbias@ischool.utexas.edu |
Oh, so maybe . . .
 Just THIS person isn’t too smart.
 Or maybe finger length is NOT a good
indicator of intelligence.
11
R.G. Bias | rbias@ischool.utexas.edu |
Now, an experiment
 I will hand you each a slip of paper. Please read
it an do NOT let anyone else read it.
– Women receive a white slip of paper.
– Men receive a green slip of paper.
 After everyone has read his/her slip of paper
and refolded it, I’ll show some letters of the
alphabet, one at a time, for one second each.
 After the last one, I’ll say “Go,” and ask you to
write down the letters, in order.
 Any questions?
12
R.G. Bias | rbias@ischool.utexas.edu |
OK, pencils down.
13
R.G. Bias | rbias@ischool.utexas.edu |
J
F
M
A
M
J
J
A
S
O
N
D
14
R.G. Bias | rbias@ischool.utexas.edu |
Write down the letters.
 In order!
15
R.G. Bias | rbias@ischool.utexas.edu |
Answers
J
F
M
A
M
J
J
A
S
O
N
D
16
R.G. Bias | rbias@ischool.utexas.edu |
Exp. 1 -- Data
All correct
Not all
correct
Total
Men
7 (47%)
8
15
Women
3 (13%)
20
23
Total
R.G. Bias | rbias@ischool.utexas.edu |
17
Who among you . . .
 . . . is a statistical wizard?
 . . . has experience conducting research?
18
R.G. Bias | rbias@ischool.utexas.edu |
Many ways to learn new things
 Method of Authority
– trusted authority tells you something
 Method of Reason
– follow basic logical laws from philosophy




Modeling
Trial-and-error
Intuition
Scientific Method
– belief on the basis of experience
19
R.G. Bias | rbias@ischool.utexas.edu |
Three Paths to “Belief”
1 – Naïve acceptance.
2 – Cynicism.
3 – Critical skepticism.
20
R.G. Bias | rbias@ischool.utexas.edu |
Critical Skepticism!
 Rabbit pie story.
21
R.G. Bias | rbias@ischool.utexas.edu |
What you’ll learn the next 3 weeks
 Reliability. (“Oh, just measure it
however.”)
 Validity. (Finger length a good indicator of
intelligence?)
 Sampling – picking a representative
sample and then generalizing to a larger
population
 Why larger samples are better
22
R.G. Bias | rbias@ischool.utexas.edu |
What you’ll learn (cont’d.):
 How to represent a group of numbers,
meaningfully.
–
–
–
–
Frequency distributions
Measures of central tendency
Measures of dispersion (spread)
Graphs/Tables
 Operationalizing variables (“intelligence”)
 Probability
 Correlation
23
R.G. Bias | rbias@ischool.utexas.edu |
What you’ll learn (cont’d.):
 Different measurement scales
 What makes a good research question?
 Experimental design
– Independent and dependent variables
– Controls, counterbalancing, and confounds
– Hypothesis testing
– Inferential statistics (is THAT number really
bigger than THIS number?)
24
R.G. Bias | rbias@ischool.utexas.edu |
More than anything else . . .
 . . . scientists are skeptical.
 “Scientific skepticism is a gullible public’s
defense against charlatans and others
who would sell them ineffective medicines
and cures, impossible schemes to get rich,
and supernatural explanations for natural
phenomena.”
25
R.G. Bias | rbias@ischool.utexas.edu |
Research Methods
Researchers are . . .
- like detectives – gather evidence, develop a
theory.
- like judges – decide if evidence meets
scientific standards.
- like juries – decide if evidence is “beyond a
reasonable doubt.”
26
R.G. Bias | rbias@ischool.utexas.edu |
Science . . .
 . . . Is a cumulative affair. Current
research builds on previous research.
 The Scientific Method:
– is empirical (acquires new knowledge via
direct observation and experimentation)
– entails systematic, controlled observations.
– is unbiased, objective.
– entails operational definitions.
– is valid, reliable, testable, critical, skeptical.
27
R.G. Bias | rbias@ischool.utexas.edu |
CONTROL
 . . . is the essential ingredient of science,
distinguishing it from nonscientific
procedures.
 The scientist, the experimenter,
manipulates the Independent Variable (IV
– “treatment – at least two levels –
“experimental and control conditions”) and
controls other variables.
28
R.G. Bias | rbias@ischool.utexas.edu |
More control
 After manipulating the IV (because the
experimenter is independent – he/she
decides what to do) . . .
 He/she measures the effect on the
Dependent Variable (what is measured – it
depends on the IV).
29
R.G. Bias | rbias@ischool.utexas.edu |
Key Distinction
 IV vs. Individual Differences variable
 The scientist MANIPULATES an IV, but
SELECTS an Individual Differences
variable (or “subject” variable).
 Can’t manipulate a subject variable.
– “Select a sample. Have half of ‘em get a
divorce.”
 Consider an Individual Difference, or
Subject Variable, as a TYPE of IV.
30
R.G. Bias | rbias@ischool.utexas.edu |
Operational Definitions
 Explains a concept solely in terms of the
operations used to produce and measure it.
–
–
–
–
–
–
31
Bad: “Smart people.”
Good: “People with an IQ over 120.”
Bad: “People with long index fingers.”
Good: “People with index fingers at least 7.2 cm.”
Bad: Ugly guys.
Good: “Guys rated as ‘ugly’ by at least 50% of the
respondents.”
R.G. Bias | rbias@ischool.utexas.edu |
Validity and Reliability
 Validity: the “truthfulness” of a measure. Are
you really measuring what you claim to
measure? “The validity of a measure . . . the
extent that people do as well on it as they do on
independent measures that are presumed to
measure the same concept.”
 Reliability: a measure’s consistency.
 A measure can be reliable without being valid,
but not vice versa.
32
R.G. Bias | rbias@ischool.utexas.edu |
Theory and Hypothesis
 Theory: a logically organized set of propositions
(claims, statements, assertions) that serves to
define events (concepts), describe relationships
among these events, and explain their
occurrence.
– Theories organize our knowledge and guide our
research
 Hypothesis: A tentative explanation.
– A scientific hypothesis is TESTABLE.
33
R.G. Bias | rbias@ischool.utexas.edu |
Goals of Scientific Method
 Description
– Nomothetic approach – establish broad generalizations and
general laws that apply to a diverse population
– Versus idiographic approach – interested in the individual, their
uniqueness (e.g., case studies)
 Prediction
– Correlational study – when scores on one variable can be used
to predict scores on a second variable. (Doesn’t necessarily tell
you “why.”)
 Understanding – con’t. on next page
 Creating change
– Applied research
34
R.G. Bias | rbias@ischool.utexas.edu |
Understanding
 Three important conditions for making a
causal inference:
– Covariation of events. (IV changes, and the
DV changes.)
– A time-order relationship. (First the scientist
changes the IV – then there’s a change in the
DV.)
– The elimination of plausible alternative
causes.
35
R.G. Bias | rbias@ischool.utexas.edu |
Confounding
 When two potentially effective IVs are allowed to covary
simultaneously.
– Poor control!
 Men, overall, did a better job of remembering the 12
“random” letters. But the men had received a different
“clue.”
 So GENDER (what type of IV? A SUBJECT variable, or
indiv. differences variable) was CONFOUNDED with
“type of clue” (an IV).
36
R.G. Bias | rbias@ischool.utexas.edu |
References
 Hinton
R.G. Bias | rbias@ischool.utexas.edu | 37
Going forward . . .
 Annotated bibliography due 3/24 (this
Wednesday – same drill, hard-copy in
class, digital copy to your TA).
 Read those two books that are on reserve
in the library – To know a fly and How to lie
with statistics.
 See you Wednesday.
R.G. Bias | rbias@ischool.utexas.edu | 38