34:833:530 Research Methods and Design
Graduate Program in Public Policy | Fall 2010
Professor Cliff Zukin zukin@rci.rutgers.edu
TAs: Jessica Godofsky jgodofsk@eden.rutgers.edu
David Drescher drescher@eden.rutgers.edu
The purpose of the social sciences is not to tell us what government should do.
Rather, it is to tell us what government is doing that is or is not working.
Course Goals
Welcome to the Public Policy program’s introduction to research methods class. Our goal is to
make you more informed consumers of (especially) quantitative information. We want you to learn to
design and interpret research so that you can solve problems and make informed decisions. While the
role of information versus values in the policy process may be hotly debated, we will take as axiomatic
the assertion that “better information leads to better policy decisions.” A central premise that follows
from this is that those who are more skilled at understanding and working with quantitative and
qualitative information will be more successful in accomplishing their aims.
We will cover three discrete areas over the course of the semester: 1) The Research Process,
including conceptualization, measurement and experimental design; 2) Survey Research, including the
use of representative samples to shed light on relationships between variables and assess programs; 3)
Qualitative Methods, including the techniques of case studies, focus groups, and one-on-one
interviewing. The second class in the methods sequence next semester focuses on the analysis of data.
There are no prerequisites, and no previous work in research methodology is assumed.
Mathematical ability matters less than logical thinking and common sense. So, there is nothing to ffeeaarr.
The reading load is quite light. This being said, it is critical that you do the reading the week it is
assigned. Lectures (and homework) will make more sense if you have done the required reading in
advance. Given that I think the most learning comes when students are called on to apply what they
have learned, the light reading load is offset with a relatively heavy amount of homework, especially in
the first part of the class. You should expect homework every week.
Evaluation Criteria
There is a midterm in-class exam on the course material up through experimental design and two short
written papers, one after the unit on qualitative methods and the other after the unit on survey research. In
terms of weighting, the midterm will be worth 35 percent of your grade, qualitative about 20 percent and survey
research 30 percent. The remaining 15 percent will be based on the weekly homework exercises and in class
participation. The two papers (and any homework of major significance, such as in week 5) will be graded on the
following scale: A = Excellent or Superior Work; B+ = Very Good; B = Good; B- = Solid Overall, but some flaws; C =
some significant problems in the work. Minor Homework assignments will get a “+” if terrific, a “√” if fine, or a “–”
if weak. Please note that most homework exercises are due by 12:00 noon on the Monday before Tuesday’s class
1
so that they can be useful in the next day’s class. Please submit homework via the Assignment Tool in Sakai as an
attachment and bring a hard copy to class. Late homework is not accepted. Late papers are downgraded.
Readings
There is one required textbook, Research Methods in Practice: Strategies for Description and
Causation by Dahlia Remler and Gregg Van Ryan, Sage Publications, 2011. This carries the load during
the first section of the class. We are using a mixture of readings to fill in the blanks for focus groups and
survey research. All of these will be available as PDF documents on the class Sakai site.
Contact Information
My main office is at the Heldrich Center for Workforce Development (2nd floor, room 205). The
entrance is at the corner of Livingston and New, different from the hotel entrance. Formal office hours
are on Mondays from 1-2 and Wednesday from 10-12. I am happy to schedule appointments with you
whenever I am in; the single best way to reach me is through email at zukin@rci.rutgers.edu. My phone
number is 732.932.4100 x6205. We will send out an announcement when Jessica and David are
available.
Course Schedule of Topics and Readings
Week Date
1
9/7
Topic
Introduction
PART 1
2
9/14
RESEARCH DESIGN
The Scientific Method, Vocabulary, The role of Applied Social Research
Read R&V Chapters 1-2 and pp 498-501
Homework 1: Write a research Problem Statement
3
9/21
Conceptualization and Measurement
Read R&V Chapter 4 and pp 242-247; Guide to the Index of Civic Engagement
Reading on School Rankings
Homework 2: Pick a statistic and deconstruct it
4
9/28
Research Design 1/Causation
Read R & V Chapter 10 and pp 395-410, 427-438
Homework 3: Find an example of research in the popular media
5
10/5
Research Design 2
Read R&V pp 355-372, 411-422, 439-454
Homework 4: Critique a research study
6
10/12 Research Design 3
Class Presentations
2
Homework 5: Prepare a Research Design
7
10/19 Midterm
No Homework.
You will be given a supplemental schedule of homework for Part 2 and 3
Part 2: Qualitative Research
8
10/26 Introduction to Qualitative Research: Case Studies and Interviewing
Read R&V pp 55-66, 80-85; Rubin & Rubin Chapters 6-7; Lapan Chapter 9
Homework
9
11/2
Focus Groups 1: Overview & Design
Read R&V pp 67-71; Krueger Chapters 1-2; Stewart Chapter 4
Homework
10
11/9
Focus Groups 2: Conduct & Analysis
Read R&V pp 75-80; Stewart Chapter 6; Edmunds Chapter 5 and Appendix C
Focus Group Paper Given; Due 11/16
Part 3 Survey Research
11
11/16 Survey Research: Overview and Sampling
Read R&V Chapter 5 and pp 250-255, 195-205; Dillman Chapter 3
Homework
12
11/23 Survey Modes, Coverage, Election Polls, Field Work
Read R&V pp 217-224; Fowler Chapter 8 Czaja & Blair Chapter 3; Zukin Primer
Homework
13
11/30 Questionnaire Design, Wording and Placement
Read R&V pp 224-239; Rea Chapter 2, 3; Colton Chapter 8
Homework
14
12/7
Ethics, Graphics, Catch up and Wrap Up
Read TBD + R & V pp 21-23, 85-87; AAPOR Code & Best Practices; Tufte
Survey Research paper given; Due 12/14
3
833:530:01
Research Methods for Public Policy
Bibliography of Readings
Colton, David and Covert, Robert W. 2007. Designing and Constructing Instruments for Social
Research and Evaluation. San Francisco, CA: Jossey-Bass, Inc.
Czaja, Ronald and Blair, Johnny. 2005. Designing Surveys: A Guide to Decisions and Procedures.
Second Edition. Thousand Oaks, CA: Pine Forge Press.
David W. Stewart, Prem N. Shamdasani and Dennis W. Rook. 2007. Focus Groups: Theory and
Practice. Second Edition. Applied Social Research Methods Series, Volume 20. Thousand Oaks,
CA: Sage Publications, Inc.
Don A. Dillman, Jolene D. Smyth and Leah Melani Christian. 2009. Internet, Mail and MixedMode Surveys: The Tailored Design Method. Third Edition. Hoboken, NJ: John Wiley & Sons, Inc.
Edmunds, Holly. 1999. The Focus Group Research Handbook. Chicago, IL: NTC Business Books.
Fowler, Jr. Floyd J. 2009. Survey Research Methods. Fourth Edition. Thousand Oaks, CA: Sage
Publications, Inc.
Krueger, Richard A. and Casey, Mary Anne. 2009. Focus Groups: A Practical Guide for Applied
Research. Fourth Edition. Thousand Oaks, CA: Sage Publications, Inc.
Lapan, Stephen D. and Armfield, Shadow W.J. “Case Study Research.” Pp. 165-180 in Research
Essentials: An Introduction to Designs and Practices. Stephen D. Lapan and Marylynn T.
Quartaroli, eds. 2009. San Francisco, CA: Jossey-Bass, Inc.
Rea, Louis M. and Parker, Richard A. 1997. Designing and Conducting Survey Research: A
Comprehensive Guide. Second Edition. San Francisco, CA: Jossey-Bass, Inc.
Remler, Dahlia and Van Ryan, Gregg. 2011. Research Methods in Practice: Strategies for
Description and Causation. Thousand Oaks, CA: Sage Publications.
Rubin, Herbert J. and Rubin, Irene S. 2005. Qualitative Interviewing: The Art of Hearing Data. Second
Edition. Thousand Oaks, CA: Sage Publications.
4