Causality, Reasoning in Research,
and Why Science is Hard
Sources:
D. Jensen. “Research Methods for Empirical Computer Science.”
William M.K. Trochim. “Research Methods Knowledgebase”
More on Causality

What is causality?
What’s Important About Causality?

Explanation
◦ Association provides prediction, but not
explanation
◦ Identifying causal mechanisms may uncover
underlying reasons for relationships

Control
◦ Understanding causality allows us to predict the
effects of actions without performing them
◦ Allows more efficient exploration of the space of
possible solutions
Conditions for Causal Inference
Problems with Association
Are Feathers Associated with Flight?
Do they have a casual relationship with the ability to fly?
Related Fallacies

Common (Questionable) Cause Fallacy
◦ This fallacy has the following general structure:
1.
2.
A and B are regularly associated (but no third, common
cause is looked for).
Therefore A is the cause of B.
◦ Called “Confusing Cause and Effect” fallacy, if in fact,
there is not common cause for A and B

Post Hoc Fallacy
◦ A Post Hoc is a fallacy with the following form:
1.
2.
A occurs before B.
Therefore A is the cause of B.
Eliminating Common Causes
Control
Randomization
Modeling
Reasoning Methodologies
in Research
Types of Reasoning in Research
Deductive vs. Inductive Methodologies

Deductive

Inductive
What is Abduction?
Examples of Abductive Reasoning

A Medical Diagnosis
◦ Given a specific set of symptoms, what is the
diagnosis that would best explain most of them?

Jury Deliberations in a Criminal Case
◦ Jurors must consider whether the prosecution or
the defense has the best explanation to cover all
of the evidence
◦ No certainty about the verdict, since there may
exist additional evidence that was not admitted in
the case
◦ Jurors make the best guess based on what they
know
 “…
when you have eliminated
the impossible, whatever
remains, however improbable,
must be the truth.”
- Sherlock Holmes
Abductive Reasoning in Science

Abduction selects from among the
hypotheses being considered, the one that
best explains the evidence
◦ Note that this requires that we consider multiple
alternative hypotheses


Abductive Reasoning is closely related to
the statistical method of Maximum Likelihood
Estimation
Possible threats to validity
◦ Small hypothesis spaces
◦ Small amounts of evidence to explain
Challenges in Abductive Reasoning

Creating hypothesis spaces likely to contain
the “true” hypothesis
◦ Approach: create large hypothesis spaces

Knowing when more valid hypotheses are
missing from the hypothesis space
◦ Approach: constantly evaluate and revise the
hypothesis space (multiple working hypotheses)

Creating good sets of evidence to explain
◦ Approach: seek diverse and independent evidence
with which to evaluate hypotheses
Why use multiple working hypotheses?
Objectivity: Helps to separate you from your
hypotheses; shift from personal investment in
hypotheses to testing the hypotheses
 Focus: Reinforces a focus in falsification rather
than confirmation
 Efficiency: Allows experiments to be designed to
distinguish among competing hypotheses rather
than evaluating a single one
 Harmony: Limits the potential for professional
conflict and rejection because all hypotheses are
considered and evaluated

“Strong Inference”



John R. Platt, Science, October 1964
◦ “Strong Inference - Certain systematic methods of
scientific thinking may produce much more rapid
progress than others.”
Not all science/research is created equal
Don’t confuse research activity with effective research
◦ Activity: building systems; proving theorems;
conducting surveys; writing and publishing articles;
giving talks; obtaining grants
◦ Research: improved predictions; better understanding
of relationships; improved control of computational
artifacts
◦ Many researchers are active; only a subset do effective
research
Initial Questions for “Strong Inference”
Arguments and Fallacies


Aside from general reasoning methodologies, one must
ensure the validity of all arguments used in any research
endeavor
An argument
◦ Consists of one or more premises and a conclusion
◦ A premise is a statement (a sentence that is either true or false)
that is offered in support of the claim being made, which is the
conclusion (also a sentence that is either true or false)
◦ Modus Ponens (and Modus Tollens)

A fallacy
◦ Generally, an error in reasoning (differs from a factual error),
◦ An "argument" in which a logically invalid inference is made
(deductive) or the premises given for the conclusion do not
provide the needed degree of support. (inductive)
Common Fallacies

Ad Hominem
Appeal to Authority
Appeal to Belief
Appeal to Common Practice
Appeal to Popularity
Begging the Question
Biased Sample
Hasty Generalization
Ignoring A Common Cause
Burden of Proof
Straw Man

See:










◦ http://en.wikipedia.org/wiki/Fallacy
◦ http://www.nizkor.org/features/fallacies/
Why is Science Hard?
[Notes]