THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL
SCHOOL OF SOCIAL WORK
COURSE NUMBER: SOWO 922
COURSE TITLE, SEMESTER AND YEAR: Advanced Topics in Causal Inference:
Propensity Score Analysis Fall semester, 2015
INSTRUCTOR:
Kirsten Kainz, Ph.D.
School of Social Work
Address:
Room 421 Tate Turner Kuralt Building
CB #3550,
Chapel Hill, NC 27599-3550
Phone: (919) 962-8826
Fax: (919) 962-1486
Email: kirsten.kainz@unc.edu
CLASS MEETINGS: 9:00-11:50, Fridays
OFFICE HOURS: 2:00 – 4:00, Wednesdays (Room 421 TTK) or by appointment
COURSE DESCRIPTION: This course focuses on advanced topics in causal inference by
reviewing four recent methods developed for observational studies and evaluation of quasiexperimental programs.
COURSE OBJECTIVES: At the completion of the course, students will be able to:
1. Understand challenges posted by evaluation of quasi-experimental or observational data,
contexts under which randomized experiments are infeasible, unethical, and expensive, and the
importance of taking remedial strategies within such contexts;
2. Understand differences, debates, and similarities between statistical and econometric traditions
in developing analytical strategies to overcome challenges posted by quasi-experimental and
observational data;
3. Have a solid understanding of the Neyman-Rubin’s counterfactual framework and two
fundamental assumptions: the strongly ignorable treatment assignment, and the stable unit
treatment value. Understand Heckman’s critiques to the counterfactual framework and main
features of the Heckman’s scientific model of causality;
4. Understand the main features of Heckman’s sample selection and related models, and know
how to implement the analysis with Stata;
5. Understand the main features of propensity-score greedy matching and related models, and
know how to implement the analysis with Stata;
6. Understand the main features of propensity-score optimal matching and related models, and
know how to implement the analysis with Stata and R;
7. Understand the main features of matching estimators, and know how to implement the
analysis with Stata;
8. Understand the main features of kernel-based matching and related models, and know how to
implement the analysis with Stata;
9. Understand the main features of Rosenbaum’s sensitivity analysis to evaluate potential bias
due to hidden selection, and know how to implement the analysis with Stata;
10. Know how to read, evaluate, and criticize evaluation studies.
PRE-REQUIREMENT: Students are assumed to be familiar with descriptive and inferential
statistics. They should have statistical and statistical software background at least equivalent to
that provided by SOWO 919 “Applied Regression Analysis and Generalized Linear Models”.
Students without such prerequisites should contact the instructor to determine their eligibility to
take the course.
STATISTICAL SOFTWARE PACKAGE:
This course will use Stata, R, and SAS. Example code will be provided.
REQUIRED TEXTS/READINGS:
Guo, S. & Fraser, W.M. (2015). Propensity Score Analysis: Statistical Methods and
Applications, 2nd Edition. Thousand Oaks, CA: Sage Publications.
All required journal articles are available on the course Sakai site.
POLICIES
Grading System
The standard of School of Social Work’s interpretation of grades and numerical scores will be
used.
H = 94-100
P = 80-93
L = 70-79
F = 69 and below
HWK 1
HWK 2
HWK 3
HWK 4
Mid-Term Presentation
Mid-Term Paper
Final Presentation
Final Paper
TOTAL
5 pts.
5 pts.
5 pts.
5 pts.
5 pts.
30 pts.
5 pts.
40 pts.
100 pts.
Policy on Attendance
Class attendance will be essential for content and skill learning, and you are expected to attend
all scheduled sessions. It’s a student’s responsibility to inform the instructor via email in advance
for missing a class session. You are expected not to miss more than two sessions for the whole
semester.
Policy on Late and Incomplete Assignments
Assignments are to be turned in to the professor by the due date noted in the course outline. In
most cases, late assignments will not be accepted. In the case of an emergency extensions may
be granted by the professor, but students should contact the professor before the due date to
determine if their case is an emergency. Late assignments will be reduced 10 percent for each
day late (including weekend days). A grade of incomplete will be given only under extenuating
circumstances and in accordance with University policy.
COURSE FORMAT, OUTLINE, AND ASSIGNMENTS
Format
The course format will include a mixture of lecture, discussion/presentation, and practical work
with data. The goal of the mixed formatting is to ensure that students build conceptual
knowledge about causal inference and selection bias in observational studies, practical skill
addressing data analytic challenges, and expertise participating in authentic discussions and
creating scholarly products related to the course topics. By the end of the course students should
be able to:
1) understand, speak about, and write about major ideas from counterfactual and potential
outcomes frameworks that motivate quasi-experimental methods such as propensity score
analysis;
2) critically evaluate research that uses propensity score analysis so as to provide expert
peer reviews and research syntheses for journals;
3) conduct defensible propensity score analyses using recommended practices; and
4) understand the limitations of propensity score analysis and consider alternative quasiexperimental methods.
Outline
Date
Aug 21
Aug 28
Topic
Introduction: The context for evidence-based
policy
Counterfactual framework and observational
studies
Discussion Activity
Sep 4
Statistical and Econometric Perspectives
HWK 1 Presentations
Sep 11
Introduction to Propensity Scores
Discussion Activity
Readings
Chapter 2 Guo & Fraser 2137, 62-65
Rubin, 2007
Rubin, 2008 (Section 3 only)
Chapter 3 Guo & Fraser
Chapter 4 Guo & Fraser 95100, 127-28
Chapter 5 Guo & Fraser PT:
*Austin, 2011
*Shadish, 2013
*Shadish & Steiner, 2010
Sep 18
Propensity Scores and Matching: STATA
psmatch2– Meet in Lab 227
*Stuart & Rubin, 2008
*Thoemmes &Kim, 2011
Sep 25
Propensity Scores and Matching: STATA
teffects psmatch– Meet in Lab 227
HWK 2 Due
Propensity Scores and Matching: R Matchit Meet in Lab 227
*Stuart, 2010
Oct 2
Oct 9
Oct 16
Oct 23
Oct 30
Nov 6
Nov 13
*Stuart & Green, 2008
*Ho et al., 2007
Student Mid-Term Presentations
Mid-Term Paper Due by Noon
FALL BREAK
Propensity Scores: sub-classification,
weighting, and regression adjustment - Meet in
Lab 227
HWK 3 Due
Propensity Scores and Matching: Missing data
and clustered data
HWK 4 Due
Interrupted Time Series
Discussion Activity
Regression Discontinuity
Discussion Activity
Nov 20
Nov 30
*Reading is available on Sakai
Chapters 6, 7 Guo & Fraser
*Harder, Stuart &Anthony
*Hong & Yu, 2008
*Green & Stuart, 2014
*Wong, Cook, & Steiner,
2015
*Shadish, 2010
*Berk et al., 2010
*Imbems & Lemieux, 2008
Student Final Presentations
Final Paper Due by Noon
Assignments
Final and Mid-Term Papers
For the final assignment students will produce a systematic literature review of published, peerreviewed research entitled “Propensity Score Analysis in [insert your topic] Research: The State
of the Field”, and the Mid Term Assignment will serve as a product that gets students part-way
to the final paper. Research topics can be as broad as School Social Work, or as bounded as
Hospitalization and Subsequent Health Outcomes among Burmese Immigrants in the
Southeastern United States: 2010-2014, depending on a student’s interests and the amount of
published research using propensity scores available to review. Ideally, the systematic review
should cover all peer-reviewed propensity score analyses in the topic area, and, at a minimum,
the review should cover no fewer than 10 studies. For the mid-term assignment, students will
submit responses to items #1, #2, #3 below. For the final assignment, students will build on their
mid-term assignment (updating as needed) and respond to items #4, and #5 below. In a
successful review, students will:
1) In no more than three pages introduce the research topic and describe propensity score
methods and the problem propensity score methods aim to solve within the research
topic;
2) Thoroughly describe the method for retrieving published research using propensity scores
and selecting studies for review. Report the studies to be reviewed in a table with these
column headings: citation; sample size; ’treatment’; outcome; software used; method to
create exchangeable groups (i.e., matching, stratification, weighting, other)
3) Thoroughly describe the criteria to be used to evaluate the quality of propensity score
analyses in the research topic. These criteria must include but are not limited to
evaluation of: (a) the completeness of reporting (do the authors tell you what you need to
know to judge the quality of methods and ultimately the inferences made?); (b) the
description of the selection mechanism for ‘treatment’; (c) the software, method, and
model used to estimate propensity scores and create exchangeable groups; (d) the
analysis of ‘treatment’ effects following propensity score estimation; and (e) the ultimate
defensibility of the inferences drawn from the analyses;
4) Using the criteria outlined in item #3, evaluate the strengths and weaknesses of published
research selected for review; and,
5) Summarize your review findings and make recommendations for how to improve the
quality of inferences in your topic area with rigorous and defensible propensity score
analysis.
Final and Mid-Term Presentations
For the mid-term presentation students will present an update on their research topic and the
types/number of published studies using propensity score analysis in their research area. The
mid-term presentation should answer these questions:
•
•
•
•
•
•
What’s the research topic – what’s the ‘treatment’?
Why is it important?
How many studies with propensity score analyses are available for review?
Are the studies clustered within particular journals in the field, or spread around?
Are the studies conducted by certain kinds of researchers in the field, or spread
around?
What software and methods for creating exchangeable groups are used?
For the final presentation students will present the results from their systematic review, making
summary statements about the state of the field. The presentation should answer these questions:
•
•
•
•
What’s the research topic – what’s the ‘treatment’?
How many studies with propensity score analysis were reviewed?
What criteria were used to evaluate studies?
What are the major summary points from the evaluation?
Homework Assignments
Homework assignments can be found on Sakai
ADDITIONAL RESOURCES
Disability Services Information
The University of North Carolina – Chapel Hill facilitates the implementation of reasonable
accommodations, including resources and services, for students with disabilities, chronic medical conditions,
a temporary disability or pregnancy complications resulting in difficulties with accessing learning
opportunities. All accommodations are coordinated through the Accessibility Resources and Service Office.
In the first instance please visit their website http://accessibility.unc.edu, call 919-962-8300 or email
accessibility@unc.edu. Please contact ARS as early in the semester as possible.
Honor Code
The University of North Carolina at Chapel Hill has had a student-administered honor system and judicial
system for over 100 years. The system is the responsibility of students and is regulated and governed by
them, but faculty share the responsibility. If you have questions about your responsibility under the honor
code, please bring them to your instructor or consult with the office of the Dean of Students or the
Instrument of Student Judicial Governance. This document, adopted by the Chancellor, the Faculty Council,
and the Student Congress, contains all policies and procedures pertaining to the student honor system. Your
full participation and observance of the honor code is expected. If you require further information on the
definition of plagiarism, authorized vs. unauthorized collaboration, unauthorized materials, consequences of
violations, or additional information on the Honor Code at UNC, please visit http://honor.unc.edu.
Policy on Prohibited Harassment and Discrimination
The University’s Policy on Prohibited Harassment and Discrimination
(http://www.unc.edu/campus/policies/harassanddiscrim.pdf) prohibits discrimination or harassment on the
basis of an individual’s race, color, gender, national original, age, religion, creed, disability, veteran’s status,
sexual orientation, gender identity or gender expression. Appendix B of this Policy provides specific
information for students who believe that they have been discriminated against or harassed on the basis of
one or more of these protected classifications. Students who want additional information regarding the
University’s process for investigating allegations of discrimination or harassment should contact the Equal
Opportunity /ADA Office for assistance at 919.966.3576 or via email at equalopportunity@unc.edu or
through U.S. Mail at
Equal Opportunity/ADA Office
The University of North Carolina at Chapel Hill
100 East Franklin Street, Unit 110
Campus Box 9160
Campus Box 9160 Chapel Hill, NC 27599
References
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding
in observational studies. Multivariate behavioral research, 46(3), 399-424.
Berk, R., Barnes, G., Ahlman, L., & Kurtz, E. (2010). When second best is good enough: a comparison
between a true experiment and a regression discontinuity quasi-experiment. Journal of Experimental
Criminology, 6(2), 191-208.
Green, K. M., & Stuart, E. A. (2014). Examining moderation analyses in propensity score methods:
Application to depression and substance use. Journal of consulting and clinical psychology, 82(5), 773.
Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of
measured covariate balance to test causal associations in psychological research. Psychological
methods, 15(3), 234.
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for
reducing model dependence in parametric causal inference. Political analysis, 15(3), 199-236.
Hong, G., & Yu, B. (2008). Effects of kindergarten retention on children's social-emotional development:
an application of propensity score method to multivariate, multilevel data. Developmental Psychology,
44(2), 407.
Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of
econometrics, 142(2), 615-635.
Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: parallels
with the design of randomized trials. Statistics in medicine, 26(1), 20-36.
Rubin, D. B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied
Statistics, 808-840.
Shadish, W. R. (2010). Campbell and Rubin: A primer and comparison of their approaches to causal
inference in field settings. Psychological methods, 15(1), 3.
Shadish, W. R. (2013). Propensity score analysis: promise, reality and irrational exuberance. Journal of
Experimental Criminology, 9(2), 129-144.
Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn and Infant
Nursing Reviews, 10(1), 19-26.
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical
Science: a Review Journal of the Institute of Mathematical Statistics, 25(1), 1.
Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in nonexperimental
studies: examining the relationship between adolescent marijuana use and adult outcomes.
Developmental psychology, 44(2), 395.
Stuart, E. A., & Rubin, D. B. (2008). Best practices in quasi-experimental designs. Best practices in
Quantitative Methods, 155-176.
Wong, M., Cook, T. D., & Steiner, P. M. (2015). Adding design elements to improve interrupted time
series designs: No Child Left Behind as an example of causal pattern-matching. Journal of Research on
Educational Effectiveness, 8, 245-279.