Research Methods in
Politics
6
Critically Evaluating Published
Information
Research Methods in Politics: Chapter 6
1
Teaching and Learning Objectives
1.
2.
to learn how to critically evaluate published
research using research component analysis;
and,
to apply Rose’s ABCDE model (1982) for
deciphering research
Research Methods in Politics: Chapter 6
2
Research Component Analysis
1.
2.
3.
4.
5.
6.
7.
8.
when
where
who
abstract
research question and
rationale
theory and literature
review
Hypothesis
methodology
9.
10
11
12
13
14
15
data collection: what,
when, how
data presentation
data analysis,
interpretation and
discussion
conclusions
implications
style
referencing and
bibliography.
Research Methods in Politics: Chapter 6
3
Rose’s ABCDE deciphering model
A Theory: an explanatory statement about social phenomena. 


B Theoretical Proposition: specific propositions to be

investigated in the study.



C Operationalisation: decisions made on how to carry out

empirical work; techniques of data collection; sampling;

concepts and indicators; variables; units.


D Fieldwork: collecting data; practical problems of

implementing Stage C decisions.



E Results: data analysis leads to findings; interpretation leads back to C,
B, and A
Research Methods in Politics: Chapter 6
4
Three-Stage Framework for Evaluating Research
Relationship to other theory and research

External validity
…………………………………………………………………………………
A
B
C
D
E
Theory: 

Internal theoretical validity
Theoretical Proposition 
…………………………………………………………………………………

Operationalisation

 Internal empirical validity
Fieldwork


Results

Research Methods in Politics: Chapter 6
5
Rose’s Strategy for Evaluating TheoryTesting, Quantitative Research
1.
2.
3.
4.
5.
6.
summarise the research into ABCDE stages;
assess the operationalisation in C: concept-indicators, samples,
units and variables;
add further descriptive details of: the researcher’s theoretical
exposition and theory-evidence linkages; and, tables of data
presented, reliability and accuracy, and interpretation;
assess the internal empirical validity by examining the ‘fit’
between the data and its interpretation;
assess the internal theoretical validity, i.e. the relationship
between A (theory) and B (hypothesis); and,
conclude with a general assessment which consolidates the
separate assessments made and makes an assessment of the
overall contribution of the research to the discipline. (Rose, 1982:
104-05)
Research Methods in Politics: Chapter 6
6
Rose’s Strategy for Evaluating TheoryBuilding Research Utilising Qualitative Data



summarise the research report into its ABCDE stages
stating: its natural history (origins and development);
the data and methods of collection; sampling; method
of analysis; and, presentation of results;
evaluate in turn the validity of: the concept-indicator
links; theory; and, sampling and generalisation; and,
conclude by reviewing the consistency (or otherwise)
of the various validities and other factors and
assessing of the overall contribution of the new theory
to the discipline (Rose, 1982: 130-02)
Research Methods in Politics: Chapter 6
7
Questions for Discussion or Assignments
1.
Consider the concepts of: conflict; freedom; liberty; hegemony;
and, consciousness of identity. What concept-indicators would you
suggest for these? Why? What data would you collect?
2.
[For undergraduates] You are a distinguished academic. You have
been asked by the editor of a refereed journal to act as a referee
for draft research report. Select one of the two published reports:
–
–
Bevir, M., and Rhodes, R A W. (2006) “Prime Ministers,
Presidentialism and Westminster Smokescreens” IN Political Studies,
vol. 54, Number 4, pp. 671-90; OR,
Chaney, P. (2006) “Critical Mass, Deliberation and the Substantive
Representation of Women: Evidence from the UK’s Devolution
Programme” IN Political Studies, vol. 54, Number 4, pp. 691-714.
Write your assessment using the letter (below);
Research Methods in Politics: Chapter 6
8
Dr Arial Helvetica
Editor, New Politics Digest
USR
Department of Politics
University of Watersea
Dear Arial
[title of draft research paper]
Thank you for inviting me to referee this paper.
Overall, my view is that, whilst this excellent paper breaks new ground, there are a number of areas
where minor revisions and additions would be helpful.
Let me begin by summarising what I believe are the research question, theoretical framework,
hypothesis and key concepts. The research question, etc are {200-300 words]
The strengths of the paper are [300-400 words]
The weaknesses are [300-400 words]
I would also like to comment on the style of the paper [200-300 words]
The paper is, of course, suitable for publication in New Politics Digest. However, I would suggest a
number of changes. They are [200-300 words]
Best wishes, Prof [your exam number]
Research Methods in Politics: Chapter 6
9
Questions for Discussion or Assignments
[For graduates]
3. Select a research report on a subject that interests you
from a refereed journal in your university library.
Evaluate it critically using research component analysis
or Rose’s strategies. Attach a copy of the research paper
to your assignment showing what you concluded were
the various analytical components. How could the
research and the report be improved?
Research Methods in Politics: Chapter 6
10