GS/PPAL 6200 3.00 Section N
Research Methods and Information
Systems
February 3, 2015
Professor Brenda Spotton Visano
Office: 130 McLaughlin
Voice Mail: (416) 736-2100 ext. 20470
E-mail: spotton@yorku.ca
Agenda
• Review of last class
• Sample size in research – how big is big
enough?
• Small-N debate
• Medium-N and mixed methods
Review of Last Class
• Presentation by David Northrup and Hugh McCague from the
Institute for Social Research/York University
• “Ice is cold” - How do we “know” this? Common "ways of
knowing“ include…
– Casual empirical observation, Tradition, An authority said so, Religion,
Inductively (from specifics to the general), Deductively (from the general to
specifics)
• “Knowing”: Mill’s Methods of Agreement, Difference,
Agreement & Difference, Concomitant Variation
• Observations: What’s specific? Cases versus variables
• Observations: What’s general? How many observations are
sufficient?
• Sampling Types and Strategies for Selection
Sampling Issues
(Why Sample?)
• Sample Types: Representative (typical of
category), Prototypical (expected to become
typical), Deviant (an exception to the norm),
Crucial (tests in the least favorable conditions),
Archetypical (creates a category)
• Sample Selection: Probabilistic (random, stratified
random, systematic random) or Non-probabilistic
(convenience, purposive, expert or snowball)
• Sample Size: How many is enough?
Sample Size
• Is the Research Question privileging depth or
breadth – internal or external validity? Is the
researcher trying to describe a phenomenon in
detail (smaller sample), develop a hypothesis
about it, or to test it (larger sample)? Should the
sample be weighted to reflect population
differences?
• Link to Qualitative versus Quantitative Methods
of Data Collection and Analysis
• If the researcher uses a smaller sample, can they
make “big” conclusions…
Small-N and Big Conclusions
The Lieberson – Savolainen Debate
• Why emphasize the “probabilistic” nature of
research?
• “The value of Mill’s methods is in their
capacity to eliminate a limited set of
alternative causal statements.”
• More generally, what is the value of
conducting small-N research? What are its
limitations? (What’s the tradeoff between
large and small sample studies?)
Medium – N Analyses
…and Big Conclusions?
• Paul, Clarke, Grill, Savitsky (2013)
• Recall the Checklist of Questions:
– What is their research question?
– What is their sample and how would you
characterize it (type, selection, size)?
– How did they collect their data?
– How do they analyze it?
Bivariate Relationships
• Factor (A) present/absent – Case outcome (x:
win, y: loss) (see Table 2, p.226)
• A BC –> x =5, y = 0
• BC –> x =3, y =22
• Implicitly then by Mill’s Method of Difference,
there is a probability that the absence of A
(COIN force credibility) influences counter
insurgency “losses”
And Mill’s methods?
• “It could be that one or more of the three critical factors
are not really present in the exceptional case but they
were evaluated as present based on a superficial reading
of the history, which a detailed review exposes as
incorrect.”
• GIVEN: Superficial reading of history, conclude ABC occur
together with xyz
• AND: Exceptional case (for one factor not present): BC
occur together with xyz
• Can we conclude anything by any of Mill’s methods?
And Mill’s methods? (cont’d)
• “Or it could be that the three critical factors are very
much present, but a detailed exploration reveals…the
impact of the three factors is thwarted by the presence
of a fourth factor,…”
• GIVEN: Superficial reading of history, conclude ABC
occur together with xyz
• AND: exceptional case reveals “thwarting factor”:
ABCD does NOT occur with xyz
• Can we conclude anything by any of Mill’s methods?
Discovery and Proof
• What causes Counter-insurgency success?
• What causes major civil uprisings (revolutions)?
• “Mill’s method of agreement is…worthless as a
method of discovery and fallacious as a canon of
proof. Its true value is in its function to eliminate
alternative explanations…. No factor can explain
an outcome satisfactorily that is not common to
all occurrences of that outcome.”