Name that tune.
Song title? Performer(s)?
R.G. Bias | rbias@ischool.utexas.edu |
1
Scientific Method (continued)
“Finding New Information”
3/29/2010
R.G. Bias | rbias@ischool.utexas.edu |
2
Objectives
 I want to arm you with a scientist’s skepticism, and a
scientist’s tools to conduct research and evaluate others’
research.
 Swoopin’ out of “scientific method” and “experimental
design” and into “statistics.”
-
Randolph – remember to take roll.
R.G. Bias | rbias@ischool.utexas.edu |
Critical Skepticism
 Remember the Rabbit Pie example from
earlier?
 The “critical consumer” of statistics asked
“what do you mean by ’50/50’”?
 Let’s look at some other situations and
claims.
4
R.G. Bias | rbias@ischool.utexas.edu |
Company is hurting.
 We’d like to ask you to take a 50% cut in
pay.
 But if you do, we’ll give you a 60% raise
next month. OK?
 Problem: Base rate.
5
R.G. Bias | rbias@ischool.utexas.edu |
Sale!
 “Save 100%”
 I doubt it.
6
R.G. Bias | rbias@ischool.utexas.edu |
Probabilities
 “It’s safer to drive in the fog than in the
sunshine.” (Kinda like “Most accidents occur
within 25 miles of home.” Doesn’t mean it gets
safer once you get to San Marcos.)
 Navy literature around WWI:
– “The death rate in the Navy during the SpanishAmerican war was 9/1000. For civilians in NYC
during the same period it was 16/1000. So . . . Join
the Navy. It’s safer.”
7
R.G. Bias | rbias@ischool.utexas.edu |
Are all results reported?
 “In an independent study [ooh, magic
words], people who used Doakes
toothpaste had 23% fewer cavities.”
 How many studies showed MORE cavities
for Doakes users?
8
R.G. Bias | rbias@ischool.utexas.edu |
Sampling problems
 “Average salary of 1999 UT grads –
“$41,000.”
 How did they find this? I’ll bet it was
average salary of THOSE WHO
RESPONDED to a survey.
 Who’s inclined to respond?
9
R.G. Bias | rbias@ischool.utexas.edu |
Correlation ≠ Causation
 Around the turn of the 20th century, there
were relatively MANY deaths of
tuberculosis in Arizona.
 What’s up with that?
10
R.G. Bias | rbias@ischool.utexas.edu |
Remember . . .




11
I do NOT want you to become cynical.
Not all “media bias” is intentional.
Just be sensible, critical, skeptical.
As you “consume” statistics, ask some
questions . . .
R.G. Bias | rbias@ischool.utexas.edu |
Ask yourself. . .
 Who says so? (A Zest commercial is unlikely to tell
you that Irish Spring is best.)
 How does he/she know? (That Zest is “the best
soap for you.”)
 What’s missing? (One year, 33% of female grad
students at Johns Hopkins married faculty.)
 Did somebody change the subject? (“Camrys are
bigger than Accords.” “Accords are bigger than
Camrys.”)
 Does it make sense? (“Study in NYC: Working
woman with family needed $40.13/week for adequate
support.”)
12
R.G. Bias | rbias@ischool.utexas.edu |
We run an experiment . . .
 to try to see if two (or more) levels of an IV
differentially influence a DV.
 We hope to find a difference.
 Finding NO difference can mean one of
two things:
– Truly there’s no difference.
– Our test – our experiment – just wasn’t good
enough, or sensitive enough, to detect the
difference.
R.G. Bias | rbias@ischool.utexas.edu | 13
Notice
 Many things influence how easy or hard it
is to discover a difference.
– How big the real difference is.
– How much variability there is in the population
distribution(s).
– How much error variance there is.
– Let’s talk about variance.
14
R.G. Bias | rbias@ischool.utexas.edu |
Sources of variance
 Systematic vs. Error
– Real differences
– Error variance
 What would happen to the DV if our measurement
apparatus was a little inconsistent?
 There are OTHER sources of error variance, and the
whole point of experimental design is to try to minimize
‘em.
Get this: The more error variance, the harder for real
differences to “shine through.”
15
R.G. Bias | rbias@ischool.utexas.edu |
Role of Data Analysis in Exps.
 Primary goal of data analysis is to
determine if our observations support a
claim about behavior. Is that difference
really different?
 We want to draw conclusions about
populations, not just the sample.
 Two different ways – statistics and
replication.
16
R.G. Bias | rbias@ischool.utexas.edu |
And so . . .
 . . . we make the transition to statistics!!
(Yay!)
R.G. Bias | rbias@ischool.utexas.edu | 17
How to talk about a set of numbers
 We can list ‘em.
– Can get WAY unwieldy.
– Plus hard to make any sense out of them.
 First step – put ‘em in order.
 Second step –
– Graph ‘em, and/or
– Calculate percentiles/deciles
18
R.G. Bias | rbias@ischool.utexas.edu |
Frequency Distributions Histograms
 # of pets ever owned
–
–
–
–
–
–
–
–
–
–
19
13
2
1
4
0
1
3
0
5
1
 Put ‘em in order
–
–
–
–
–
–
–
–
–
–
0
0
1
1
1
2
3
4
5
13
R.G. Bias | rbias@ischool.utexas.edu |
Freq Dist
 Raw Scores (in
order)
–
–
–
–
–
–
–
–
–
–
20
0
0
1
1
1
2
3
4
5
13
Raw Score
0
1
2
3
4
5
13
Freq
2
3
1
1
1
1
1
Cumu Freq
2
5
6
7
8
9
10
R.G. Bias | rbias@ischool.utexas.edu |
Histogram
3
2.5
2
1.5
# of pets
1
0.5
0
0
1
2
3
4
5
13
R.G. Bias | rbias@ischool.utexas.edu |
21
Percentiles
 LOCATION of 25th percentile:
– X.25 = (N+1) .25
 LOCATION of 50th percentile:
– X.50 = (N+1) .50
 LOCATION of 75th percentile:
– X.75 = (N+1) .75
 Example: If we had 10 scores,
– the 25th percentile would be the (11).25=2.75th score or part way
(half way!) between the 2nd and 3rd scores.
– The 50th percentile would be the (11).50=5.5th score, or half way
between the 5th and 6th scores.
22
R.G. Bias | rbias@ischool.utexas.edu |
Example
 # of pets ever owned
–
–
–
–
–
–
–
–
–
–
23
13
2
1
4
0
1
3
0
5
1
 Put ‘em in order
–
–
–
–
–
–
–
–
–
–
0
0
1
1
1
2
3
4
5
13
R.G. Bias | rbias@ischool.utexas.edu |
Note . . .
 With an odd number of scores, the 50th percentile will
be an actual score:
 Raw Scores (in order)
–
–
–
–
–
–
–
–
–
–
–
0
0
1
1
1
2
3
4
5
13
100
 50th percentile = (N+1).50 = (12).5 = 6th score = 2.
24
R.G. Bias | rbias@ischool.utexas.edu |
Let’s calculate some “averages”
25
R.G. Bias | rbias@ischool.utexas.edu |
A quiz about averages
1 – If one score in a distribution changes, will the mode change?
__Yes __No __Maybe
2 – How about the median?
__Yes __No __Maybe
3 – How about the mean?
__Yes __No __Maybe
4 – True or false: In a normal distribution (bell curve), the mode,
median, and mean are all the same? __True __False
26
R.G. Bias | rbias@ischool.utexas.edu |
Some rules . . .
 . . . For building graphs/tables/charts:
– Label axes.
– Divide up the axes evenly.
– Indicate when there’s a break in the rhythm!
– Keep the “aspect ratio” reasonable.
– Histogram, bar chart, line graph, pie chart,
stacked bar chart, which when?
– Keep the user in mind.
27
R.G. Bias | rbias@ischool.utexas.edu |
References
 Hinton, P. R. Statistics explained.
 Shaughnessy, Zechmeister, and
Zechmeister. Experimental methods in
psychology.
R.G. Bias | rbias@ischool.utexas.edu | 28