Research Methods
Leadership 389
Measuring the Weight of Smoke


Sometimes things that
seem immeasurable can
be measured well
It often takes creativity
and a leap of faith to
figure out how to
measure some things
Outline







How do we Know?
Scientific Method
Four Canons of Science
Hands-on Activity
Induction and Deduction
Our Bias to Prove
Three Approaches to Hypothesis Testing
Methods of Acquiring
Knowledge

Non-scientific
Tenacity
 Intuition
 Authority
 Logic
 Observation


Scientific

Science 
Tenacity


A method of acquiring
knowledge based on
superstition or habit
Mere exposure effect


Problems with tenacity



Politicians
May be inaccurate
Doesn’t provide for
corrections
Tenacity in science
Intuition

An approach to acquiring
knowledge that is not based on
reasoning or inferring

Problem with intuitive approach



No method for determining accurate
vs inaccurate knowledge
Cognitive and motivational biases
affect perception
Intuition in science

Hypotheses derived from “hunches”
Authority


A basis for acceptance of information because it
is acquired from a highly respected source
Problem with authority:


Information may be inaccurate
Authority in science:



Problem identification/hypothesis building
Study design
Data interpretation
Logic


The acquisition of
knowledge through reasoning
Problem with logic:



Often doesn’t provide accurate
information
The original assumption may
be incorrect
Logic in science:

Logic is vital to science!


Developing hypotheses
Developing method of testing
hypotheses
Observation


The acquisition of
knowledge through
experience
Problem with observation



Cognitive and motivational
biases affect perception
Memory is not infallible
Observation in science


Science based on observation

Science
"There is no institution in the modern world more
prestigious than science"

A method or logic of inquiry
The best method for acquiring knowledge
 Method can be broken down into a series of steps

Scientific Method

1) Identifying the problem and forming a
hypothesis


2) Designing the experiment


Suggestion accounts for hearing satanic messages
3 groups (given different suggestion) listen to 3
records played backwards and record responses
3) Conducting the experiment

Participants receive 1of 3 instruction sets and then
listen to 3 tapes.
Scientific Method

4) Testing the hypothesis
45
35
25
15
5
None
Listen for words
Listen for satanic
messages
-5

5) Communicating the research results

Published article in The Journal of Psychology
Four Canons of Science




Determinism
Empiricism
Parsimony
Testability
Determinism

The assumption that there is a lawfulness in
nature.


Science is “a search for order, for uniformities, for lawful
relations among the events in nature”- Skinner
Naïve determinists
Cab Problem

Cab involved in hit-and-run
1) 85% of cabs in the city are Green, 15% are Blue
 2) A witness identified cab as blue

Testing revealed the witness identified colors of
cabs correctly 80% of the time (incorrectly
20%)

What is the probability that the cab in the
accident was blue?
Cab Problem

Cab involved in hit-and-run


1) Although the 2 companies are roughly equal in size, 85%
of cab accidents in the city involve green cabs and 15%
involve blue cabs
2) A witness identified cab as blue
Testing revealed the witness identified colors of cabs
correctly 80% of the time (incorrectly 20%)

What is the probability that the cab in the accident was
blue?
Determinism

Close corollary to principle of determinism:
Theories

Theory:


Statement about causal relation between 2 or more
variables
Hypothesis

Similar to theories but more limited in scope and less
empirical support
Science is Empirical


To be empirical is to make observations
Getting it straight from the horse’s mouth

Philosophers debating the number of teeth a horse
should have
Biological philosopher: As member of family Equidae,
they should, like a zebra, have 34.
 Theological philosopher: A scripturally unclean grazer,
should have less than a cow: about 28 teeth
 Empirical philosopher: If we want to know how many
teeth it has, let’s open its mouth and count.

Science is Empirical

Relatively modern assumption

Galileo began a new science based on
observation and experimentation

Galileo’s main work showed a “passionate fight against
any kind of dogma based on authority”- Einstein
Parsimony
Parsimony

A.K.A. Occam’s Razor



Entities must not be multiplied beyond what is necessary
A problem must be stated in its basic and simplest terms
AKA: Law of parsimony




The simplest explanation should be preferred to more complex
explanations
That theory which makes the fewest number of assumptions is to be
preferred over those which require a larger number of assumptions.
Example: Crop circles
Many adoptions of Occam’s razor

Morgan’s canon

Clever Hans
Testability


Theories should be testable
Falsifiability


Karl Popper
‘all swans are white’



One can always support this
point by pointing to more white
examples
A better test would be to find a
black one
(Ironically, black swans were
later discovered in Australia! )
Hands-On Activity: Galileo’s Dice


Comparing different ways of knowing
Answer the questions gamblers once asked Galileo:

400 years ago gamblers played game with 3 standard dice. Most
believed the probability of rolling a 9 was the same as rolling a 10.
However, some had a gut intuition that 10 was a little more likely and
their casual observations supported this. Those who believed the
probability was equal said that there are six combinations of number
that total nine (126, 135 144, 225, 234, 333) and there are 6
combinations that total 10 (136, 145, 226, 235, 244, 334). So, they
approached an authority for the answer: Galileo. I would like you to
approach this problem only using one particular way of knowing.
Three ways of knowing



Group 1: The Logical Counter of Ways
Group 2: The Logical Expected Evaluators
Group 3 and 4: The Empiricists
Questions
Which roll do you think is most likely, a 9 or a 10?

1
2
3
4
5
6
7
not at all
8
9
extremely

How confident are you of this answer?

What is your best guess as to the exact probability of
rolling a 9 with a fiar set of 3 dice? What about 10?
How confident are you in this set of guesses?
Induction and Deduction


Scientists use logic to induce
and deduce valid conclusions
Induction

Moving from data to a general
theory of concept


Moving from a theory to
particular statements concerning
data

Induction
Deduction
Specific to general
Deduction

Data
General to specific
Theory
Hypothesis Testing
 Wason

Selection task
If a card has a vowel on one side, then it has an
even number on the other side
E
T
4
7
Hypothesis Testing
 Wason Selection task

If a card has a vowel on one side, then it has an
even number on the other side
E T
2 5
4 7
A U
Hypothesis Testing
 Wason Selection task

If a card has a vowel on one side, then it has an
even number on the other side
E T
2 5
4 7
A U
Our Bias to “Prove”

Pervasive confirmatory biases
Positive test bias or hypothesis confirmation bias
 EX: Snyder and Swann (1978)

• Interview person
Is he/she introvert/extrovert
 Given list of Qs to ask



some about being introverted (in what situation do you wish you
could be more outgoing?)
some about being extroverted (what would you do to liven up a
party
8
Extraversion
questions
7
6
Introversion
questions
5
4
3
Neutral questions
2
1
0
Extraversion Hypo Introversion Hypo
Behavioral Confirmation

Causing people to act the way you expect
•Word, Zanna & Cooper (1974)
–White interviewees perform “better”
–White interviewers behave differently
towards white and black interviewees
–Trained interviewers to use “White” or
“Black” style
–White interviewees who get “White”
style do better
Approaches to Hypothesis
Testing



Validation
Falsification
Qualification
Theory

“People perform tasks better when in the
presence of others than when alone”
Validation


Gather evidence to support or confirm the hypothesis
Triplett (1898)



Results:


Noticed bicyclists performed better when riding with others
Study with children performing simple task either alone or
with others.
Children performed better when in the presence of others
compared to when alone
Social facilitation
Falsification

Gather evidence to invalidate or disprove a hypothesis

Pessin (1933)


Dashiell (1930)


Presence of others inhibits performance on a maze
Learn nonsense syllables slower
Social inhibition
Qualification


Attempt to identify conditions under which a
hypothesis is true or not
Zajonc 1965

Dominant response:


Nondominant response:


Well-learned or instinctive behaviors that the organism has practiced
and is primed to perform
Novel, complicated, or untried behaviors that the organism has never
performed (or performed infrequently)
Presence of others increases our tendency to perform
dominant responses

Social facilitation/inhibition in poolroom
Players identified as above or below average
 Research team of 4 approached the table and
observed playing
 Found classic facilitation/inhibition effects

80
70
60
% shots
made
50
Alone
Observers
40
30
20
10
0
Above average
Below average