Testing theories vs explaining
outcomes
5th seminar, reading group qualitative
methodology
Carl Henrik Knutsen 14/7-08
Two modes of doing science?
• An analogy from econometrics: Calibration vs
hypothesis testing
• How strongly do we believe in our priors, our
theory?
• Theories, verification and falsification, Popper
• Do we separate in practice between verification
and explanation?
• W.V.Quine and the critique of Kant and other
empiricists: Testing and falsification
Explanation
• Elster: must not confuse with provision of narratives…
• Mechanisms again
• Explaining by theory alone? One or many theories? The purpose of
the study: Paint the “whole picture” or focus on parts
• How do we know if our explanation is the correct one? Multiple
coherent theories
• The role of deducing several different observational implications,
and checking…but then we are involving testing
• We can provide narratives that are coherent with the outcome, but
how can we assure the plausibility of our narrative being correct?
• The two cornerstones of science: Logical coherence and empirical
evidence!
Testing theories
• “Testing” whether our theory sketched up plausible explanations in
a particular case (observable implications, strategic tests with
alternative theories, other evidence) VS testing the theory in
general
• 1) Falsificationism gone mad in the social sciences: Rejecting
theories on the basis of a case study
– Deterministic theories
– Most likely and unlikely cases, role of prior knowledge
• 2) Theoretical “believers” gone mad:
– Stretching concepts
– Monocausal explanations (awareness)
– Ignoring alternative explanations
• Example of 1) Rejecting modernization theory on the basis of single
case. Ex of 2) Rational addictions
Platt
• Why do some areas have more scientific progress than
others?
• Strong inference: Formally, regularly and explicitly:
– 1)Devise alternative hypothesis
– 2) Devise crucial experiment
– 3) Recycle the procedure, subhypothesis, sequential
hypothesis..exclude more possibilities
• Being aware of the structure..problem oriented rather
than method oriented..the important role of induction
• Aim: Interconnecting theory and empirical evidence in
most fruitful way to achieve scientific progress
• The danger of tautological theories
Chamberlin
• Creative and novel thinking in science, rather than mere
following
• Problematic: “the habit of some to hastily conjure up an
expIanation for every new phenomenon that presents itself”
• The importance of solid description and descriptive inference
before explanation
• Multiple working hypothesis: The danger of falling in love
reduced!
• “Just as the investigator armed with many working hypotheses is more
likely to see the true nature and significance of phenomena when they
present themselves, so the instructor equipped with a full panoply of
hypotheses ready for application more readily recognizes the actuality of
the situation, more accurately measures its significance, and more
appropriately applies the methods which the case calls for.”
Chamberlin
• “The explanation offered for a given phenomenon is
naturally ,under the impulse of self-consistency, offered for
like phenomena as they present themselves, and there is
soon developed a general theory explanatory of a large class
of phenomena similar to the original one. This general
theory may not be supported by any further considerations
than those which were involved in the first hasty
inspection... With this tentative spirit and measurable
candor, the mind satisfies its moral sense, and deceives
itself with the thought that it is proceeding cautiously and
impartially toward the goal of ultimate truth. It fails to
recognize that no amount of provisional holding of a theory,
so long as the view is limited and the investigation partial,
justifies an ultimate conviction.”
Geddes
• The consequence of lack of focus on basic design
issues: Relative failure of parts of comparative
politics in knowledge accumulation (development
studies, democratization etc..)
• “Steer the course between lovely, untested
theories and information unstructured by
theories”..either grand schemes or unfocussed
case-studies..
• Symptom: Sand castles
Geddes cont’d
• The main problem: Theories not checked against the facts
• Unfruitful response: “inductive fact-gathering missions
resulting in a disorganized mass of information”..Why does
this case study matter for my research and how can it be
used?
• Induction and speculative theories not tested on new
cases!!
• Focus on finer mechanisms rather than inductive search
large correlates. Big questions, little answers. Theory-based
disaggregation and process focus.
• Additional problems: Case-selection and casual attitude
towards non-quantitative measurement
Geddes cont’d
• How do we deal with big macro-structures..
• The quest for explaining everything and developing grand schemes
• Selective tests and disaggregation of grand theories..Are there
plausible elements from for example modernization and
dependency theory?
• Paradigms and Kuhn: Successful research frontiers in need of
smaller puzzles and paradoxes. Many unexplained paradoxes, from
normal science to scientific revolution..
• The grand schemes and their dependence on empirical events…
• Reason: problematic aspects of theory not taken out through
testing and checks against evidence..all or nothing approach makes
it easier to throw out the whole scheme when difficult empirical
events occurs. Theory disaggregation and theory refinement.
Geddes cont’d
• The importance of drawing out clear and testable
implications
• The importance of precise operational definitions
and classification/coding (measurement also nonquantitative)..sensitivity analysis..
• Trying to explain compound outcomes can be
hurtful, testable hyp on processes and smaller
sub-question
• More cases are better..Cases which generated
theoretical framework vs novel evidence.