1
Department of Political Science
San Diego State University
Spring 2014
Professor Ronald King
Nassatir Hall 108
rking@mail.sdsu.edu
POLS 615
RESEARCH DESIGN AND ANALYSIS IN POLITICAL SCIENCE
Nasatir Hall 131
W 7:00 – 9:40 pm
This course is intended to introduce graduate students to the concepts and methods of
social science research, and to show how those methods can be applied to concrete
problems. As a class on research design, it intended as the second in a three-semester
sequence on empirical methodology required of all MA students in the Political Science
Department at SDSU. Ordinarily, it is preceded by POLS 601 (Scope and Methods) and
followed by POLS 616 (Intermediate Statistics).
Most courses in the political science catalog emphasize substantive topics, divided into
such subfields as American Politics, Comparative Politics, International Relations, and
Political Theory. However, beyond matters of substance, there are certain skills and
techniques that are inherent to the discipline as a whole. These skills, which broadly can
be understood as “the methodology (methodologies) of political science,” help students to
become producers, not merely consumers of knowledge. Our focus, therefore, will be on
what political scientists do as trained and professional scholars, regardless of
specialization, and how ultimately this relates to empirical truth and normative good.
In a sense, the study of political science methods might be even more important than the
study of substance. The substance of political science changes over time; research skills
provide the means by which we evaluate existing theories and add to our stock of
knowledge. Much substantive detail can be looked up in reference books; research skills
allow one to develop substantive understanding on one's own. The substance learned in
any political science class is relatively specific and narrow in application; research skills
are general in relevance and are useful to educated individuals throughout their life.
Emphasis this semester will be on the general principles of social scientific inquiry and
the construction of a valid research design. Briefly, we will examine the sense in which
political science is a “science” -- discussing the systematic study of politics and inquiring
whether such systematic study is possible when human behavior is the focus of our
scholarly attention. Then we will attend to the main elements of social science research:
modeling underlying relationships, specifying testable hypotheses, operationalizing
dependent and independent variables, selecting the appropriate observations, collecting
data and assessing their quality, and establishing a test strategy sufficient for rigorous
inference from the data collected.
2
Grading:
1. Class Preparation/Participation (15% of grade): Students are expected to come to
every class session having completed and understood the assigned readings, ready to
discuss them. Class attendance is mandatory. Any student who must be absent should
inform me of the fact by e-mail. Students who miss more than one class (without major
excuse) or consistently come to class unprepared and/or late should expect heavy grade
penalties.
2. Short papers and exercises (50% of grade): Students will have an assignment to
complete virtually every week. These are to be submitted in class. No late papers/
exercises will be accepted (the only exception is if I give explicit approval in advance).
On occasion, I will give students the right to re-write and improve a submitted
paper/exercise.
3. Formal Research Proposal (35% of grade): Due on Wednesday, May 14 before 2:00
p.m. to the Political Science Department Office. No late papers will be accepted (the only
exception is if I give explicit approval in advance). A preliminary statement of Research
Topic is due on February 26.
I also remind you that plagiarism is a serious offense. You are responsible for learning
the university rules regarding plagiarism and for avoiding all violations. Any student
deemed guilty of plagiarism will automatically receive a failing grade for this class.
Student Learning Objectives:
-- To gain sophisticated understanding regarding the nature of political science as an
academic discipline, including the diversity of its theoretical and methodological
approaches.
-- To think, read, and write critically about the topics pertaining to some of the core
concepts of political science and the manner in which empirical research is undertaken.
-- To develop better skills in close reading of texts, note-taking, verbal and written
communication, the organization of ideas, and problem solving.
-- To learn how to specify an interesting and important research topic and prepare a
competent, formal research proposal on that topic.
-- To demonstrate the ability to produce essays and examination-exercises that impart
information, frame and support an argument, use critical thinking and analysis, make
logical assumptions, derive sound conclusions from evidence, and display graduate-level
use of language, grammar, and rhetorical structure.
Assigned Books:
Ronald F. King. 2004. “The Strategy of Research: Thirteen Lessons on the
Elements of Social Science” (San Diego: Montezuma Press, manuscript)
Gary King, Robert O. Keohane, and Sidney Verba. 1994. Designing Social
Inquiry. (Princeton: Princeton University Press).
Supplemental articles, posted on Blackboard.
3
PRELIMINARY OUTLINE OF TOPICS, READINGS, AND ASSIGNMENTS
The basic agenda for class sessions, along with assigned readings and required exercises,
is listed below. Modifications might occur, but always with sufficient in-class warning.
I. On Social Science
1.
Introduction: Research Methods and Political Science (January 22)
Methods as an Essential Aspect of Political Science Education
Problem and Hypothesis-Based Analysis (Not Proper Name-Based)
Three-Legged Political Science -- Subject, Theory, Method
Reading for this class: King Manuscript, Lesson 1
King-Keohane-Verba, Chapter 1
Assignment for next class: Count Something! Go out into the world, get curious about
social behavior, and discover something for yourself. You can do this on your
own, or in two- or three-person teams. Write a 3-4 page (typed and edited) report
describing the project, its importance, and the results obtained.
2. Philosophy of Science/Social Science (January 29)
How Scientific is Science? What Really Is Meant by “Science”?
Metaphysics, Positivism, Empiricism, and Skepticism
What is Meant by Social Science?
Nomological Propositions and Systematic Knowledge
Objections from Journalism and History: Particularity of Time and Place
Objections from Anthropology and Humanism: Culture and Reason
The Complexity of Human Subjects: Personal Choice and Action
Readings:
King Manuscript, Lesson 2
Kim Quaile Hill, (2004) “Myths About the Physical Sciences and Their
Implications for Teaching Political Science,” PS: 467-471.
(Blackboard).
King Manuscript, Lesson 3
King-Keohane-Verba, Chapter 2.1 and 2.2
Karl Hempel, (1968) “Explanation in Science and in History” in
Nedditch ed. The Philosophy of Science: 54-79 (Blackboard)
Alasdair MacIntyre, (1962) “A Mistake About Causality in Social
Science” in Laslett and Runciman eds. Philosophy, Politics, and
Society, 2nd ed: 48-70. (Blackboard)
Assignment for next class: Select an empirical article from a recent issue of the
American Political Science Review or the American Journal of Political Science.
Read the article. Briefly and clearly explain: 1) the core hypothesis, 2) the
empirical subject/data used to assess the hypothesis, (3) the underlying theory that
makes the hypothesis meaningful, and (4) the method of testing. Then briefly
assess the value that you attach to the article. Did it advance knowledge in
political science? Did you learn anything important from it? Did the article
4
written from the formal social science perspective contain visible features that
made it especially valuable? Did it, in the process, lose other virtues that might
be worthwhile?
II. On Research Models
3. Causation in Social Science (February 5)
Interpretation versus Explanation as Two Differing Approaches
The Compete Bi-Variate Linear Hypothesis
Readings:
King Manuscript, Lesson 4
King-Keohane-Verba, Chapter 3
Cross-National Poverty, Data Table (Blackboard)
Martin Gilens, (1995) “Racial Attitudes and Opposition to Welfare,”
Journal of Politics: 994-1014. (Blackboard)
Ahmet Kuru, (2007) “Passive and Assertive Secularism: Historical
Conditions, Ideological Struggles, and State Policies Toward
Religion,” World Politics: 568-594. (Blackboard)
Assignment for Next Class: Take any contemporary political issue, as would be found in
a recent news story. Conceptualize it as a particular illustration of one, simple
general bi-variate causal relationship formulated without reference to proper
names. Specify the relationship and its component variables. Provide the
anticipated sign, slope, intercept, and error.
4. More Complex Linear Models (February 12)
Readings:
King Manuscript, Lesson 5
King-Keohane-Verba, Chapter 2.3
David Cameron, (1978) “The Expansion of the Political Economy,”
American Political Science Review: 1243-61. (Blackboard)
Ronald King and Alexandra Borchardt, (1994) “Red and Green: Air
Pollution Levels and Left Party Power in OECD Countries,”
Environment and Planning C: 225-241. (Blackboard)
Richard Cole, (1973) “Toward a Model of Political Trust,” American
Journal of Political Science: 809-817. (Blackboard)
Assignment for next class: Develop, explain, and justify a full linear causal model for an
interesting research idea of your own, specifying the concepts used and the
exact linkages hypothesized. Identify the key theoretical variables, the essential
control variables, and (if any) the position of intervening variables. For each of
the linkages, specify the anticipated sign, slope, intercept, and error.
5. Analytic Modeling: (February 19)
Micro-Foundations of Behavior: Deriving Hypotheses from Logical Incentives
Equilibrium, 2-Person Games, n-Person Games – with illustrations.
Reading:
King Manuscript, Lessons 6 and 7
5
Assignment for next class: Topic for Formal Research Proposal:
Provide a title and a brief, intriguing introduction. What is the big
question/problem you wish to investigate? What is the main hypothesis/claim you
propose for examination? Specify its component variables. What is the underlying
theory from which your claim is generated? What competing claims exist in the
literature? How will discovering whether your hypothesis is correct or not
advance our understanding of politics? (In other words, you are expected to
provide first-draft text regarding items 1-4 on the syllabus description of what
constitutes a valid research proposal. Evaluation will be according to rubric
categories 1-7, 18-20).
6. Comparing Motivational Hypotheses: (February 26)
Rationality, Culture, Sociology, Psychology, and Biology
Reading:
King Manuscript, Lesson 8
Richard Nisbett, (1993) “Violence and Regional Culture,” American
Psychologist: 441-449. (Blackboard)
Angus Campbell, et al. (1964) The American Voter, Ch 11-12: 161-209.
(Blackboard)
Leonie Huddy and Anna Gunnthorsdottir, (2000) “The Persuasive Effects
of Emotive Visual Imagery,” Political Psychology: 745-778.
(Blackboard)
John Alford, Carolyn Funk, and John Hibbing, (2005) “Are Political
Orientations Genetically Transmitted?” American Political Science
Review: 153-167. (Blackboard)
Assignment for next class: Interview five people (not political scientists) about why they
vote (or don’t vote). Probe extensively for underlying motivations. Push them
regarding the rational logic of the Prisoner’s Dilemma paradox and see how they
respond. What does this tell you, in very a preliminary way, about the motivations
of people in politics?
7. Workshop on Research Proposals – Drawing Causal Models (March 5)
Bring to class: Your research proposal (with any revisions based on Professor King’s
comments). Be prepared to draw the causal model and present it rigorously to the
the group.
Assignment for next class: Revise and Resubmit the “Topic for Formal Research
Proposal,” based on Professor King’s comments and our class discussions.
III. On Measurement
8. Variable Construction, Operationalization, and Selection of Observations (March 12)
Simple and Complex Indicators
Validity and Reliability
The Right Observations and the Right Number of Observations
6
Readings:
King Manuscript, Lesson 9, pp. 1-24.
King-Keohane-Verba, Chapter 4
World Audit Democracy Rankings – (go to website)
http://www.worldaudit.org/democracy.htm
Geraldo Munch and Jay Verkuilen, (2002) “Conceptualizing and
Measuring Democracy,” Comparative Political Studies: 5-34.
(Blackboard)
Reed Wood and Mark Gibney, (2010) “The Political Terrorism Scale
(PTS): A Re-Introduction and a Comparison to CIRI,” Human
Rights Quarterly: 367-400. (Blackboard)
Steven Best and Anthony Nocella II, (2004) “Defining Terrorism’
Animal Liberation Philosophy and Policy Journal: 1-18.
(Blackboard)
Barbara Geddes, (1990) “How the Cases You Choose Affect the
Answers You Get,” Political Analysis: 131-150. (Blackboard)
Assignment for next class: Select some simple concept often found in the political
science literature (e.g. nationalism, tolerance, trust, freedom, or any other basic
concept that appeals to your research interests). Operationalize the concept
convincingly. (Do not be afraid to use a multi-dimensional operationalization.) A
successful operationalization would allow another person to fully measure and
code any randomly selected observation for the variable you choose. It therefore
must be very specific and concrete. Then, find a recent journal article that uses the
same concept empirically, and contrast the author(s)’ operationalization with
yours. Defend your operationalization relative to the one used in the article.
9. Data Collection Techniques (March 19)
The Three Sources of Data: Ask, Observe, Consult
Measurement Error as an Inherent and Unavoidable Problem
Readings
King Manuscript, Lesson 9, pp. 25-40.
King-Keohane-Verba, Chapter 5
Helene Hanff, (1961) “The Footnote-and-mouth Disease,” Harpers: 5861. (Blackboard)
Ian Lustick, (1996) “History, Historiography, and Political Science:
Multiple Historical Records and the Problem of Selection Bias,”
American Political Science Review: 606-618. (Blackboard)
Clifford Geertz, (1973) “Thick Description: Toward an Interpretive
Theory of Culture,” The Interpretation of Cultures: 3-30.
(Blackboard)
William Miles and David Rochefort, (1991) “Nationalism Versus Ethnic
Identity in Sub-Saharan Africa,” American Political Science
Review: 393-403. (Blackboard)
Mark Button and Kevin Mattson, (1999) “Deliberative Democracy in
Practice: Challenges and Prospects for Civic Deliberation,” Polity:
607-637. (Blackboard)
7
Julie Lima and Michael Siegel, (1999) “The Tobacco Settlement: An
Analysis of Newspaper Coverage of a National Policy Debate,
1997-98,” Tobacco Control: 247-253. (Blackboard
Assignment for next class: Select two empirical articles from major political science
journals. Briefly discuss the data collection strategy utilized in each, looking at
the means by which the data were collected and whether a plausible approach was
taken in data collection. Which of the three methods of data collection (asking,
observing, consulting) was used? How does each article rate regarding the three
main sources of measurement error (reliability, obtrusiveness, incomplete
perspective)? Was there any discussion of possible measurement error in the
papers? To what extent do you believe that the findings might have been affected
(either systematically or unsystematically) by the data collection strategy
adopted?
10. Univariate (Descriptive) Analysis (March 26)
Multiple Observations on One Variable, Both Qualitative and Quantitative
Three Aspects of Empirical Analysis - Central Tendency, Strength, Significance
Introduction to Inferential Testing – Association Across Two or More Variables
Experiments, Quasi-Experiments, Matched Comparisons, Case Studies
Readings:
King Manuscript, Lesson 10, pp. 18-29.
Gosta Esping-Andersen, (1989) “The Three Political Economies of the
Welfare State,” Canadian Journal of Sociology: 10-36.
(Blackboard)
David Rapoport, (2004) “Four Waves of Modern Terrorism,” in Cronin
and Ludes, Attacking Terrorism: 46-73. (Blackboard)
Robert Putnam, (1996) “Bowling Alone: America’s Declining Social
Capital,” Journal of Democracy: 65-78. (Blackboard)
Michel Foucault, (1984) “The Birth of the Asylum,” in Rabinow, The
Foucault Reader: 141-167. (Blackboard)
King Manuscript, Lesson 10, pp. 1-10.
King-Keohane-Verba, Chapter 2.4 through 2.7
Assignment for next class: A data table consists of 1) a complete list of operationalized
variables across the top; 2) a complete list of observations down the side; 3) coded
data reliably measured for each of the internal cells of the table.
Since this is a class in research design, I do not expect you to have actually
collected the data necessary to construct a full data table. Yet – for this exercise –
I want you select two observations (from among the total set of observations
relevant to your project) and complete the data-cell entry for all the main
variables of your project. Importantly – the data for the two observations can be
real – or they can be plausibly imagined.
Your task is to construct the data table for these two (actual or
hypothetical) observations. You must provide the framework for the data table
(variables across the top, observations down the side), insert data (real or
imagined) into the relevant cells, and justify their coding so that the data appear
correctly and plausibly measured.
8
The important thing is that, in order to do the assignment for the week,
you must have a reasonable list of potential observations from which you will
select two for specific measurement and coding. This means you must specify the
‘range’ of the research, with its associated population from which your
observations are chosen.
You also must have a relatively complete list of essential variables, all
fully operationalized so that effective coding of the observation can be performed.
You will also need to have modeled the empty places within which the
two variables can vary (yes/no; categorical; cardinal; ordinal), and you need to
have specified completely the coding scheme for the variables (establishing
validity) so that anyone looking at the data can precisely, reliably, and correctly
code the information the same way. Only then can you insert data into the table
and justify the coding provided.
April 2 – No Class. Spring Break
IV.
On Testing, and Inference
11. On the Forms of Testing (April 9)
On Experiments and Quasi-Experiments
Bi-Variate Analysis: Estimating Central Tendency, Strength, Significance
Readings:
King-Keohane-Verba, Chapter 6
King Manuscript, Lesson 10, pp. 10-18; 29-45.
King Manuscript, Lesson 11, pp. 1-28.
In Class: Laboratory Workshop – Bi-Variate Linear Estimation as an approach to testing.
Make sure you are very familiar with the (1) Data Code Book, (2) Data By States
spreadsheet, and (3) Notes on Excel Regression (all on Blackboard). Download
and bring a copy of the Code Book to class.
Assignment for next class: 1. Select one independent variable and one dependent
variable from the Date by States spreadsheet.. Explain each variable and its
operationalization. Specify the interesting bi-variate hypothesis you are using,
linking independent to dependent variable. What is the null hypothesis?
2. Examine the independent variable selected, find the highest state, the
lowest state, and find California. Calculate and explain the Z-score for each of
these three states.
3. Do the same for the dependent variable you have selected.
4. Examine the bivariate scatterplot. Make sure the dependent and
independent variables are entered correctly. Put in the regression line. Print it,
and describe it for me in words.
5. Generate a regression estimate using your independent and dependent
variables Supply the estimates for the central tendency -- intercept (a) and slope
(b); for the observed strength (R2); and the statistical significance (the t-statistic
and its associated probabilities). In each case, explain carefully but briefly what
they mean.
9
6. Conclude – is your proposed hypothesis most likely sustained or not,
given this specification? Do the data indicate an empirical relationship between
the independent and dependent variables? Finally, discuss with wider, theoretical
meaning of your findings.
12. Multi-Variate Analysis and the Logic of Control Variables (April 24)
Reading: King Manuscript, Lesson 11, pp. 28-43.
In Class: Laboratory Workshop -- Multivariate Estimation as an approach to testing.
Assignment for next class: Using the Data By States spreadsheet, select a dependent
variable, a main independent variable, and one or more control variables -- and
analyze the relationships using multi-variate statistics.
(If you wish, you can merely expand upon the analysis done for the
previous assignment, just adding one or more control variables to the equation.)
First, run and analyze fully the two separate bivariate relationships X1 on
Y and X2 on Y. Next run and analyze fully the multivariate relationship, X1 + X2
on Y. As before, report all estimates for the central tendency -- intercept (a) and
slope (b); for the observed strength (R2); and the statistical significance (the t
statistics and their associated probabilities). Explain carefully but briefly what
they mean.
Conclude – is your proposed hypothesis most likely sustained or not? Pay
special attention to what happens to the strength and significance of your main
independent variable once the controls were added. What does this imply for your
analysis.
13. Structured, Matched Comparisons (April 23)
Readings:
King Manuscript, Lesson 12 (pp. 1-27)
John Frendreis, (1983) “Explanation of Variation and Detection of
Covariation,” Comparative Political Studies: 255-272.
(Blackboard).
Ronald King, (1998) “Capping Entitlements: Budget Rules and the Food
Stamp Program,” Journal of Public Policy: 133-161 (Blackboard).
Diana Dumitru and Carter Johnson, (2011) “Constructing Interethnic
Conflict and Cooperation: Why Some People Harmed Jews and
Others Helped Them during the Holocaust in Romania,” World
Politics, pp. 1-42. (Blackboard)
Ann Shola Orloff and Theda Skocpol, (1984) “Why Not Equal Protection?
Explaining the Politics of Public Social Spending in Britain, 19001911 and the United States, 1980s-1920,” American Sociological
Review: 726-750 (Blackboard.).
Theda Skocpol, (1979) “State and Revolution: Old Regimes and
Revolutionary Crises in France, Russia, and China,” Theory and
Society: 7-95. (Blackboard)
Assignment for next class: Select an article from a political science journal that makes an
explicit qualitative comparison across two or more observations (something
10
from Comparative Politics or Comparative Political Studies might work well).
Briefly present and critique the methodology. Does the article utilize a most
similar or most different approach? Is the comparison constructed to provide a
plausible test of the main hypothesis? How would you construct the comparison
to improve its methodological rigor?
14. Case Studies and Their Value (April 23)
Readings:
King Manuscript, Lesson 12 (p. 27-37)
Arend Lijphart, (1971) “Comparative Politics and the Comparative
Method,” American Political Science Review: 682-693.
(Blackboard).
John Gerring, (2004) “What Is A Case Study and What Is It Good For,”
American Political Science Review: 341-354. (Blackboard)
David Coller (2011) “Understanding Process Tracing,” PS: 823-830.
(Blackboard)
Ellen Immergut, (1986), “Between State and Market: Sickness Benefits
and Social Control,” in Rein and Rainwater eds., Public/Private
Interplay in Social Protection: 57-98. (Blackboard)
Robert Bates, (1998) “The International Coffee Organization: An
International Institution,” in Bates et al. Analytic Narratives: 194230. (Blackboard)
Michael Cain and Keith Dougherty, (1999) “Suppressing Shay’s
Rebellion: Collective Action and Constitutional Design Under the
Articles of Confederation,” Journal of Theoretical Politics: 233260. (Blackboard)
Graham Allison, (1969) “Conceptual Models and the Cuban Missile
Crisis,” American Political Science Review: 689-718. (Blackboard)
Assignment for next class: Experiments, Quasi-Experiments, Matched Comparisons, and
Case Studies are four different strategies by which empirical hypotheses can be
tested. Invent and briefly describe four potential research projects, a different one
for each of the four different test methods. Specify the research subject and test
hypothesis for each and then present the test design. Defend, for each project –
why the method selected is the best for this particular research. How does the
design maximize probable leverage over the planned project?
15. Putting It All Together (May 7)
Student Reports on Research Proposals
Readings:
King Manuscript, Lesson 13
Leo Strauss, (1957) “What is Political Philosophy,” Journal of Politics:
343-368. (Blackboard)
Gregory Kasza. (2001) “Perestroika: For an Ecumenical Science of
Politics,” PS: 597-600. (Blackboard).
Stephen Bennett, (2002) "Perestroika Lost” PS: 177-179. (Blackboard)
11
Ronald King and Cosmin Marian, (2008) “Defining Political Science:
A Cross-National Survey,” European Political Science: 207219. (Blackboard)
Formal Research Proposal: Due on Wednesday, May 14 before 2:00 PM to the
Political Science Department Office. No late papers will be accepted (the only exception
is if I accept an explicit excuse submitted to me in advance of that date).
Expected length: 15-20 pages, double spaced, normal fonts and margins.
For details, see “The Research Proposal,” attached to this syllabus.
We will spend time in class discussing the proposal writing process in detail.
12
The Research Proposal
(stolen over the years, with bits and pieces chopped and adapted from many unnamed sources)
"It's too hard!" Ballplayer to Coach in A League of Their Own.
"It's supposed to be hard...otherwise everyone would be doing it." Coach (played by Tom Hanks).
In the future, when your research has to be approved by others, whether by a thesis committee or by a
funding source, you will need to write a research proposal. This is a vitally important document.
A research proposal is an academic instrument that shares attributes with both an architectural plan and a
legal contract. As a plan, it specifies exactly the problem you will research, the methods you will use to
gather data on this subject, and the analysis techniques you will use to make sense of the data. As a
contract, it specifies exactly what activities you will engage in, what valuable contribution the activities
will make, and – for sponsored research – how the money will be spent. If you get the proposal right, all
you have to do after that is execute the research program and determine whether the findings confirm or
reject the hypothesis spelled out in the proposal. Thus, the most important thinking needs to be done at
the proposal stage. Writing a good proposal should, if done properly, be the most difficult part of the
whole project.
The main purpose of the proposal is to convince others that your research is important and that
you are qualified to conduct it.
A research proposal is a statement intended to convince your professor, thesis supervisor, granting
agency, etc. that you have an interesting and significant question to research and that there is some
reasonable guarantee that you will be able to complete the research required to answer the question.
A research proposal is NOT simply a statement of interest in a general topic area. It should indicate that
you have identified an important problem connected to your topic area, thought about the range solutions
conventionally offered regarding that problem, formulated a probable solution of your own, and done
some preliminary work in determining whether the information needed to examine your proposed
solution is actually available. This is virtually impossible to do without background reading and initial
research.
Because your goal is to convince others to approve your research project, you want to demonstrate by
your proposal that your research will meet the highest standards. That means your proposal must be of
the highest standard because it is assumed that your research will be no better than your proposal. If your
proposal is vague or sloppy, it is assumed that your research will be vague or sloppy. If you begin with
“background” about how you became interested in the topic, very likely your proposal will be dismissed
because it appears that you cannot distinguish between what is relevant and what is irrelevant. If your
research question is not stated clearly or if you fail to make a connection between your question and the
data you propose to collect, it is likely that your research will also be conceptually inept. If the terms you
use in the research proposal are murky, it is assumed that your research will be as well.
A good way to ensure that your proposal is a good one is to ask as many people as possible to give it 10
minutes (same as you’ll likely get from a reviewer) and pay attention to their comments.
What goes into a proposal?
Good research is not accidental. It requires careful planning plus careful execution. The research proposal
is a necessary first step in the process.
13
A research proposal is a brief outline of the research project you are going to conduct and research report
you are going to write. A good research proposal must reflect your aim and objectives of research, your
plan of action, and the way you can accomplish your goals. It clearly specifies the test hypothesis. It
addresses the subject, theory, and methodology you are going to use in the research project. It must
contain information sufficient to demonstrate your credibility, the depth of your knowledge, and the
extent of your devotion and dedication to the project. It should convince your readers about the academic
value of the project and your capacity to perform it. The importance of the research proposal is truly
understood once it becomes a vehicle to the successful execution of your project.
The research proposal is often quite formal in construction, with several required components.
1. Title
The title should be brief and descriptive of the project.
2. Introduction – Characterization of the Research Topic
There should be a very brief introduction to the project, possibly including an anecdote or intriguing
quotation. The introduction should clearly state the core claim/hypothesis. It should quickly suggest the
potential importance of the research project and the innovations inherent to your approach to the
problem.
3. The Big Question or Problem Addressed and Its Theoretical Importance
What is the question that you intend to answer? What is the significance of the question? Why is it
important for the understanding of politics (how people live together, make decisions, construct societies,
promote or fail to promote justice, etc.)? Why is more research needed regarding this question? What
theoretical debates exist regarding the answers to this question? Is your research intended to help us
develop a useful theoretical perspective or decide across contested theoretical perspectives?
“Big Questions” should be of the sort: Why do people sometimes vote for politicians with platforms
contrary to their interests? Does ‘new media’ affect the political efficacy of younger voters? What is the
motivation of suicide bombers? Why do the industrialized countries differ widely in their treatment of
poor and destitute individuals? Why do nations sometimes go to war with enemies far more powerful
than themselves? Do participatory democratic institutions reduce the chances of ethnic violence in divided
societies? Do international courts of justice improve the chances for the implementation of effective
human rights? Has globalization reduced the sovereign power of nation-states?
Note the importance of the word “Question” in specifying the core problem to be addressed by your
research design. “I want to learn about …” is not a question. “I want to prove that…” is not a question.
The former is far too vague for specified research; the latter is an assertion about conclusions and implies
that research is unnecessary. As such, they are – truly – unacceptable.
The range of possible Big Questions is vast. You need to state the problem explicitly and justify the
choice. It is correct to say that the selection of a topic depends principally on an investigator's interest and
values. Nonetheless, there are some general criteria that can be used to evaluate the relative worth of
topics. For example, how important is the topic for development of knowledge? How would the answer to
your question affect other parts of the knowledge structure within political science? Would it develop
new knowledge? Would it replicate earlier findings in a different setting? Would it resolve present
inconsistencies in our interpretation of evidence or create inconsistencies by calling into doubt currently
accepted interpretations? You need to specify why your research is different (in data, theory, and/or
method) from existing research in the field, justifying why a new effort is necessary and why one should
not rely merely on what is available from a thorough literature review of previous findings. A project that
adds to our knowledge tends to be considered important.
While the promise of knowledge acquisition is a relevant criterion in evaluating research proposals, it is
not the only standard. It is absolutely important that the research will relate to the social, political, or
economic well-being of people. After all, our field of study is politics. There is a reason why we picked this
14
field of study. Would the answer to the question that you pose affect the achievement of social values?
Might such answers imply changes in accepted social values?
Another criterion used in evaluating the selection of a research problem is feasibility. How effectively can
the topic be studied? No one would reasonably endorse a proposal that is unlikely to produce a result.
(We are not likely ever to discover empirically the existence or non-existence of God.) No one would
reasonably endorse a proposal to expend great amounts of time, skill, and money on a project likely to add
little to our knowledge or to human welfare. (We are not likely to fund the establishment a moon colony
simply to discover whether or not the relationship between sense of political efficacy and political
participation, which we've discovered repeatedly in Western nations, persists among those living on our
permanent satellite.)
4. Clearly and explicitly specify the main hypothesis (and its underlying model) central to the
problem selected, and identify the core variables to your study.
Based on the underlying big question, what specific claim or claims do you intend to investigate? An
hypothesis is declarative statement positing a relationship within or between variables. It could be about
one variable. (For example, a follower of Plato might hypothesize that humans sort easily and exclusively
into three main types: gold, silver, brass.) It could be about two or more variables, usually entailing an
independent variable; a dependent variable (possibly with some intervening or control variables); and a
predicted relationship between them. (For example, one might hypothesize that, controlling for economic
development and party ideology, countries more racially divided will have smaller welfare states.) The
claim can be interpretive (for example, that popular elections are expressions of regime legitimacy) or
explanatory (that popular elections causally increase the degree of regime legitimacy).
It is essential that the hypothesis be specified rigorously. It is essential that the hypothesis is testable (in
some sense) and falsifiable. It must be possible to imagine empirical evidence that potentially could
demonstrate the hypothesis to be wrong. (Otherwise your argument is tautological – self-evidently true.)
It is NOT acceptable merely to state, “I intend to investigate the reasons why….” That is far too vague.
Instead, you should make a definite claim of the sort, “I intend to investigate the impact of variable X
upon variable Y.” Additional specification might well include: I expect a relationship of a given direction
(positive or negative); I expect a linear (or non-linear) relationship; I expect a relationship in which a
small change in X produces a large (or small) change in Y; I expect that X1 will have a larger effect on Y
than X2 or X3.
It is also essential to specify alternative and/or competing claims, especially those found in the literature.
It is likely that your hypothesis will be controversial. (An hypothesis in which the answer is obvious
through common sense is not an interesting one for research purposes.) Establishing the plausibility of
your claim requires not only evidence on its behalf, but also evidence that points against important
alternatives. At a minimum, you must take the null-hypothesis seriously, that there is no relationship of
the sort you believe applies. It is always useful to situate your hypothesis within the academic debates
regarding your field of study, showing what is novel or especially interesting insofar as it bears on your
big question. It is therefore important to design your research to test whether or not your proposed
hypothesis represents an improvement to our explanatory capacity, for instance supplemental to or
contrary to previous findings.
If relevant, you should situate your hypothesis in the context of a wider model. Has it been derived from a
larger logical framework? Is it part of a broader system of relationships? If so, where does it fit within
that model? Important research hypotheses most often do not sit individually by themselves, but instead
are components of more far-reaching interpretations of human social behavior and institutional
interactions. These should be carefully specified.
Moreover, you should specify what assumptions are built into the hypothesis. Are there implicit
microfoundations for the behaviors you wish to identify? What motivations are required for the expected
outcome to occur? Are these motivations tested or assumed within your hypothesis?
15
5. Clearly specify and operationalize all the main variables inherent to your study
Social science research is constructed from variables (whether one, two, or many). Regardless of how
many variables are considered, each has to be specified and carefully operationalized. It is, for example,
not enough to say that your research considers the impact of economic growth on democratic stability. It
is essential that you give empirical meaning to each component. What precise indicator(s) will you use to
operationalize each variable? What alternative operationalizations are plausible and what is the
justification for the particular operationalization selected? You must defend each variable
operationalization in terms of its validity in measuring the underlying concept. This is especially true if
your operationalization is different from those commonly found in the literature, or if the
operationalizations in the literature are essentially contested. It is always useful to refer to the existing
literature in the demonstration/defense of your operationalization.
For instance, assume the simple hypothesis that higher social class causes increased political
conservatism. Is social class validly measured by income? If so, what precisely do you mean by income
(not every economist agrees on its definition)? Do you want to consider only one year or a number of
years? If you mean more than just income, what other components of the social class indicator would you
include (e.g., wealth, status, education, family background) and how would these be combined into a
composite measure? How would you operationalize political conservatism as an individual attribute? Do
you trust a single survey question or would you rely upon a series of policy opinion survey questions?
What would these be? How are they to be scored and summarized? Is conservative behavior different
than conservative attitudes? How would this be included in the measure?
Moreover, each variable must be modeled. Is it nominal or numeric? How many “empty places” will be
established for the potential coding of data? For example, will democratic stability be considered as
dichotomous (e.g., yes/no) or categorical (e.g., highly-unstable/potentially-unstable/probablystable/highly-stable) or continuous (e.g., on a scale from 1 to 100)? What is the justification for this
modeling? What categories exist in the literature and how is your selection of categories similar or
different? Remember that categories once established can always be compressed but rarely can they be
expanded.
6. Domain of the hypothesis; the selection of observations within that domain (the right
observations; the right number of observations).
Most hypotheses in the social sciences do not refer to all individuals in all situations at all times. They are
usually restricted in domain. You need to specific the domain of your hypothesis. Does it, for example,
refer to all nation-states, to only the advanced industrial nations or to developing nations, to nations with
a colonial heritage, to post-communist nations, to nations exclusively in one part of the world (Latin
America or Asia or Africa), to those nations with names beginning or ending in a vowel? Does it refer to a
specific period of time (for example, post-World War II or after 9/11)? You need to think seriously about
the domain of your hypothesis and justify it. Arbitrary restrictions limit the field for possible data
collection and potentially will cast doubt upon the conclusions you produce.
Within the domain established for the hypothesis, what specific observations are you selecting for actual
data collection? Are you going to collect data from every relevant case? Are you selecting a random
sample? Are you intentionally selecting certain cases or possibly just one for detailed examination?
Remember that the selection and justification of observations occurs after the specification of the
research question and must be sufficient to provide convincing evidence for the evaluation of your core
hypothesis. The fact that you are personally familiar with country X or find it fun to study region Y is
NOT usually a valid empirical justification by itself.
How many observations do you need to make for a valid inference? What is the benefit and cost from each
additional observation? How difficult will it be to collect the necessary data for each observation? How
much detail is really necessary in order to code/classify the observation correctly regarding the
variable(s) in the study? How important is extensive description? How willing are you to lose some
degree of extensive description in order to improve precision in estimating the empirical character of the
relationship – its central tendency, the degree of observed variation, and its significance relative to
16
random chance. The number of observations in the research and their selection process need to be
defended explicitly.
7. Data Collection and Coding
Data collection entails the longest time commitment for most research projects. Importantly, it is not
merely a matter of hunting and pecking. A common mistake in research proposals is to carefully frame the
question and then simply say, “I'll answer this question by doing research.” Instead, it is necessary to be
specific about data sources and their utilization. At the end of the day, a large quantity of data collected
haphazardly remains nothing more than a big pile of data. Data collection must be guided by the
specifications of your research project.
First, the data must be comprehensive. For every cell in the ‘data table’ matrix – each observation by each
operationalized variable – data have to be collected. (In specified circumstances missing data can be
permissible.) You need to establish that the necessary data are available. You need to explain whether you
are collecting your own data or utilizing data previously collected for some other purpose. If you are
collecting your own data, precisely how will you do it? If you are relying on data collected by others, you
need to justify its relevance to your project.
Second, the data must be coded. It is necessary to determine, during the research process, the value
(whether quantitative or qualitative) for each observation, seen in terms of the model of empty places
established for each operationalized variable. A mechanism must thus be specified enabling one to
determine, for example, whether Country A, on the variable “Democratic Stability” is to be coded as
highly-unstable, potentially-unstable, probably-stable, or highly-stable. Or, for individual B, on the
variable “Conservative Attitudes,” a mechanism must be created to determine whether s/he to be coded as
1 (lowest conservative) or 100 (highest conservative) or anywhere in-between.
Third, the data need to be reliable. You need to give reasons for your confidence that other researchers
looking at the same data sources should agree regarding the coding of each observation upon each
operationalized variable. What process has been established to help insure reliability? It is often required
that your proposal includes a formal ‘coding guide’ to convince evaluators that data coding will not be
subjectively biased.
Finally, each data source should be cross-examined for mis-information or missing information. What are
the limitations and biases of the data source? What do you plan to do in order to correct for these
limitations and biases? A strong research proposal is explicit regarding the kinds of data necessary, the
sources that will supply the data, the strategy for collecting reliably coded observations, and your degree
of confidence that the data collected are appropriate and sufficient.
8. Method of Analysis
Whether the approach is quantitative or qualitative, the data collected have to be marshaled and
organized in order to produce a sustainable conclusion. To an extent, the method of analysis is inherent in
the selection of observations. One cannot do detailed process-tracing with many hundreds of
observations. Once cannot establish statistical significance with a single case study. Yet there are still
decisions to be made and justified. If this is a case study, how will the case description be structured in
order to help us evaluate the core hypothesis? If this is a matched-comparison, how precisely will the
comparison be constructed to enhance your ability to make valid inferences? If this is a large-n study,
what statistical techniques will be used for analysis and why are these appropriate?
Data do not speak to us by themselves. There must be an explicit strategy by which data are leveraged
and presented. Otherwise it is not possible to assert rigorously that the hypothesis (most likely) is
sustained and competing hypotheses (most likely) are minimized or dismissed. A strong research proposal
will specify precisely what kind of findings would lead you to conclude that your hypothesis is correct and
what findings would lead you to conclude that the hypothesis is wrong.
The essential principle that should govern the selection and organization of evidence is maximization of
the chances of discovering decisive negative evidence. The best design is one that would clearly and
17
quickly expose the error in a working hypothesis. It means that we go out of our way to look for negative
evidence and that we adopt those modes of analysis that make it easier to disprove our claim than to
sustain it. Otherwise, it is sort of like cheating.
9. Significance of the Research
In the conclusion of the proposal, you need to establish firmly why anyone would care about your
research. You need to address the “So What” challenge. You should begin with a concise statement of
your expected results. It is then essential to put these expected results in context.
What is the relationship of the expected results to the Big Question discussed in the first section of the
proposal? In what way will evidence regarding your hypothesis help to advance our understanding
regarding that big question. In particular, you should address:
(a) What is the relationship of the expected results to previous work in the field? What is especially new
or interesting in your anticipated findings? In what way will evidence regarding your hypothesis help to
advance our knowledge beyond what is available from the existing literature?
(b) What is the relationship of the expected results to social justice and human well-being? Why are
these results politically and socially meaningful? In what way do you hope that evidence regarding your
hypothesis will affect how humans live and behave?
You might end with a statement of what is not covered in your research project. Not all relevant
questions can be answered in a single project. Some topics (or dimensions of topics) are placed in
parentheses, awaiting further efforts. That’s perfectly fine. Yet an explicit statement of what is NOT part
of this particular project is never irrelevant.
10. Outline of Project Workload
Some research proposals require:
-- a tentative list of chapters/sections to comprise the final report
-- a list of who (if multiple investigators) will complete each section
-- a timeline for completion
-- a detailed budget and budget justification (especially if external funding is involved)
In sum, the purpose of a research proposal is to demonstrate to the intended audience that your question
is important, that you have a specific, detailed plan for how you are going go about answering it, and that
the project is “do-able.”
Be concise. If you cannot clearly spell out your question, hypothesis, research plan, and statement of
significance with a minimum number of specific sentences, the skeptical reader will doubt the quality of
your project and/or your ability to achieve it. A popular misconception is that the more you think about
something, the lengthier and more verbose your writing should be. Most often, the opposite is the case. If
you have thought carefully about each of these elements, you should be able to spell them out in very few
words -- clearly, simply and straightforwardly. Most proposals (even for very large contracts and grants)
are limited to a few pages at most. Grantees, contractors, and thesis supervisors recognize that clearly
thinking the project through should allow the proposal writer to strip away all of the extraneous material
and thus to outline the key elements of the project efficiently and effectively. It’s not easy, but it’s
essential to success.
(see below for the grading rubric that will be used in the evaluation of the research
proposals for POLS 615)
18
19
Grading Rubric
NAME _________________________________
POLS 615 – Research Proposal
10 points each category = 200 points (Divide by 2 = 100 points)
1. Title
________
2. Intriguing Brief Introduction
________
3. Specify and justify the Big Question/Problem
________
4. Specify the main hypothesis/claim
________
5. Specify the main alternative/competing claims, citing
the appropriate literature
________
6. Specify the variables relevant to the study (independent,
dependent, control, intervening, etc.) and explain
how they vary
________
7. Place all the relevant variables into a plausibly
constructed larger model
________
8. Specify the critical human microfoundations to your project
________
9. Specify and justify the domain/population for which
the study applies and from which observations are
derived
________
10. Specify and justify the selection of sample observations
in the study, including the number of observations
and how they were selected
________
Subtotal (page 1)
___________
20
11. Explain the operationalization for each of the main variables
in the study, and justify the operationalization in terms of
validity
________
12. Explain how each operationationalized variable will be coded
(i.e., yes/no, categorical, ordinal, cardinal), and justify your
choice
________
13. Explain in detail the coding instructions for each of the main
operationalized variables and justify them in terms of reliability
________
14. Specify the data sources for the main variables in the study
and justify why these sources are the right ones
________
15. Explain the possible causes of data error in your study and
how you will minimize those possible effects of error
________
16. Explain the testing strategy adopted (experiment, quasiexperiment, matched comparison, case study) and justify
why it is appropriate for evaluating your core hypothesis
________
17. Explain and justify the detailed configuration of the testing
strategy adopted, as a means of maximizing confidence
in the findings from your data.
________
18. Specify the anticipated finding(s) from your research, and
explain in detail how they should they add to the existing
literature in the field?
________
19. Organization of Proposal
________
20. Prose/Grammar/Writing Quality
________
Subtotal (page 2)
_________
Total for 20 questions
___________
Divided by 2 (scale to 100)
___________