THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL SCHOOL OF SOCIAL WORK COURSE NUMBER: SOWO 919 COURSE TITLE, SEMESTER AND YEAR: Advanced Topics in Causal Inference: Propensity Score Analysis Fall semester, 2013 INSTRUCTOR: Shenyang Guo, Ph.D. School of Social Work Address: Room 524j, Tate Turner Kuralt Building CB #3550, Chapel Hill, NC 27599-3550 Phone: (919) 843-2455 Fax: (919) 962-1486 Email: sguo@email.unc.edu CLASS MEETINGS: 9:00-11:50, Thursdays OFFICE HOURS: 8:30-10:30, Tuesdays (Room 524j TTK) COURSE DESCRIPTION: This course focuses on advanced topics in causal inference by reviewing four recent methods developed for observational studies and evaluation of quasi-experimental programs. COURSE OBJECTIVES: At the completion of the course, students will be able to: 1. Understand challenges posted by evaluation of quasi-experimental or observational data, contexts under which randomized experiments are infeasible, unethical, and expensive, and the importance of taking remedial strategies within such contexts; 2. Understand differences, debates, and similarities between statistical and econometric traditions in developing analytical strategies to overcome challenges posted by quasiexperimental and observational data; 3. Have a solid understanding of the Neyman-Rubin’s counterfactual framework and two fundamental assumptions: the strongly ignorable treatment assignment, and the stable unit treatment value. Understand Heckman’s critiques to the counterfactual framework and main features of the Heckman’s scientific model of causality; 4. Understand the main features of Heckman’s sample selection and related models, and know how to implement the analysis with Stata; 5. Understand the main features of propensity-score greedy matching and related models, and know how to implement the analysis with Stata; 6. Understand the main features of propensity-score optimal matching and related models, and know how to implement the analysis with Stata and R; 7. Understand the main features of matching estimators, and know how to implement the analysis with Stata; 8. Understand the main features of kernel-based matching and related models, and know how to implement the analysis with Stata; sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 1 9. Understand the main features of Rosenbaum’s sensitivity analysis to evaluate potential bias due to hidden selection, and know how to implement the analysis with Stata; 10. Know how to read, evaluate, and criticize evaluation studies. EXPANDED DESCRIPTION: This course will review four closely related but technically distinct propensity score models developed for intervention research when randomized clinical trials are infeasible or unethical: (a) Heckman’s sample selection model (Heckman, 1976, 1978, 1979) and its revised version estimating treatment effects (Maddala, 1983); (b) propensity score matching (Rosenbaum and Rubin, 1983) and related models; (c) matching estimators (Abadie, Drukker, Herr, & Imbens, 2004); and (d) propensity score analysis with nonparametric regression (Heckman, Ichmura, & Todd, 1997, 1998). Learning of these models will be guided by two conceptual frameworks: the Neyman-Rubin counterfactual framework and the Heckman scientific model of causality. The course also covers Rosenbaum’s approaches of sensitivity analysis to discern bias produced by hidden selections. PRE-REQUIREMENT: Students are assumed to be familiar with descriptive and inferential statistics. They should have statistical and statistical software background at least equivalent to that provided by SOWO 919 “Applied Regression Analysis and Generalized Linear Models”. Students without such prerequisites should contact the instructor to determine their eligibility to take the course. STATISTICAL SOFTWARE PACKAGE: This course will use Stata as the main software package. REQUIRED TEXTS/READINGS: Guo, S. & Fraser, W.M. (2010). Propensity Score Analysis: Statistical Methods and Applications. Thousand Oaks, CA: Sage Publications. All required journal articles are available on E-journals and the course Blackboard site. RECOMMENDED TEXTBOOKS: Rosenbaum, P. R. (2010). Design of Observational Studies. New York: Springer. Morgan, S.L, & Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. New York: Cambridge University Press. Assignments Assignment 1 Assignment 2 Assignment 3 Assignment 4 Assignment 5 Assignment 6 Final Exam (take home) Grade Percentage 10% 10% 10% 10% 10% 10% 40% GRADING SYSTEM The standard of School of Social Work’s interpretation of grades and numerical scores will be used. sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 2 H = 94-100 P = 80-93 L = 70-79 F = 69 and below POLICY ON CLASS ATTENDANCE Class attendance is an important element of class evaluation, and you are expected to attend all scheduled sessions. Each class session will cover a great deal of materials, and you will fall behind the course when you miss even one class session. It’s student’s responsibility to inform the instructor via email in advance for missing a class session. You are expected not to miss more than two sessions for the whole semester. Starting from the second missing, your course grade will be reduced by 10% for each session missed. POLICY ON INCOMPLETE AND LATE ASSIGNMENTS Assignments are to be turned in to the professor by 5pm of the due date noted in the course outline. Extensions may be granted by the professor given advance notice of at least 24 hours. Late assignments (not turned in by 5pm on the due date) will be reduced 10 percent for each day late (including weekend days). A grade of incomplete will only be given under extenuating circumstances and in accordance with University policy. POLICY ON ACADEMIC DISHONESTY Students are expected to follow the UNC Honor Code. Please include the honor code statement along with your signature on all assignments: “I have neither given nor received unauthorized aid on this assignment.” Please refer to the APA Style Guide, the SSW Manual, and the SSW Writing Guide for information on attribution of quotes, plagiarism and appropriate use of assistance in preparing assignments. If reason exists to believe that academic dishonesty has occurred, a referral will be made to the Office of the Student Attorney General for investigation and further action as required. POLICY ON ACCOMMODATIONS FOR STUDENTS WITH DISABILITIES Students with disabilities which affect their participation in the course may notify the instructor if they wish to have special accommodations in instructional format, examination format, etc., considered. COURSE OUTLINE (TOPICS, READINGS, AND ASSIGNMENTS): 08-22-13 Session 1: Introduction and Course Overview 1. Observational studies and challenges 2. History and development 3. Fisher’s randomized experiment 4. Why and when propensity score analysis is needed? 5. Course overview Readings: Guo & Fraser, chapter 1. 08-29-13 Session 2: Counterfactual Framework 1. The Neyman-Rubin’s counterfactual framework sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 3 2. The assumption about strongly ignorable treatment assignment 3. The stable unite treatment value assumption 4. Types of treatment effects Readings: Guo & Fraser, chapter 2 (pp.21-50). Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688-701. Holland, P. (1986). Statistics and causal inference (with discussion). Journal of the American Statistical Association, 81, 945-970. 09-05-13 Session 3: Intervention Research and Simple Methods for Data Balancing 1. Guest speaker Dr. Mark Fraser: Intervention research and observational studies 2. Summary of the two traditions in developing nonexperimental models 3. Review of three simple methods for data balancing 4. Key issues regarding the OLS regression Readings: Fraser, M.W., Richman, J.M., Galinsky, M.J., & Day, S.H. (2009). Intervention research. New York, NY: Oxford University Press. Chapters 1, 2, & 7. Guo & Fraser, chapter 3. 09-12-13 Session 4: Sample Selection and Related Models (Part 1) 1. Truncation, censoring, and incidental truncation 2. Key features of Heckman’s sample selection model 3. Treatment effect model 4. Illustrations Readings: Guo & Fraser, chapter 4: 4.1-4.2. Heckman, J. J. (1978). Dummy endogenous variables in a simultaneous equations system. Econometrica, 46, 931-960. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47, 153-161. 09-19-13 1. 2. 3. 4. Readings: Session 5: Sample Selection and Related Models (Part 2) and Instrumental Variables Estimator The instrumental variables estimator Review basics of running Stata Overview of Stata programs analyzing sample selection Stata lab: running treatreg Guo & Fraser, chapter 4: 4.3-4.6. Journal of the American Statistical Association (1996), Vol. 91, Number 434 Assignment 1 out (Due: 9-26-13): Exercises of running Heckit treatment effect model. 09-26-13 1. 2. 3. 4. 5. Session 6: Propensity Score Matching (Part 1) Overview of propensity score matching The Rosenbaum and Rubin’s model (1983) Strategies to seek optimal propensity scores Greedy matching Computer lab: running greedy matching with psmatch2 sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 4 Readings: Guo & Fraser, chapter 5: 5.1-5.3, 5.4.1, 5.5.1, 5.5.2. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55. D’Agostino, R. B., Jr. (1998). Tutorial in biostatistics: Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265-2281. Assignment 1 due Assignment 2 out (Due: 10-03-13): Exercise of running propensity score greedy matching with a multivariate post-matching analysis and a subclassification analysis. 10-03-13 Session 7: Propensity Score Matching (Part 2) 1. Optimal matching 2. Post-optimal-matching analysis 3. Propensity score weighting Readings: Guo & Fraser, chapter 5: 5.4.2, 5.5.3-5.5.6, 5.6-5.7. Guo & Fraser, PSA 2nd edition – to be supplied by the instructor Haviland, A., Nagin, D. S., & Rosenbaum, P. R. (2007). Combining propensity score matching and group-based trajectory analysis in an observational study. Psychological Methods, 12, 247-267. Assignment 2 due 10-10-13 Session 8: Propensity Score Matching (Part 3) 1. Introduction to R 2. Running optimal matching with R 3. Conducting post-matching analysis 4. Propensity score subclassification Readings: Guo & Fraser, chapter 5: 5.8-5.10. Guo & Fraser, draft of PSA 2nd edition – to be supplied by the instructor Assignment 3 out (Due: 10-24-13): Exercise of running optimal propensity score matching (pair, variable, and full matching) with a post-matching regression adjustment and a Hodges-Lehmann aligned rank test. 10-17-13 No class: happy fall break! 10-24-13 Session 9: Matching Estimators 1. Simple matching estimator 2. Bias-corrected matching estimator 3. Variance estimator allowing for heterosckedasticity 4. Efficacy subset analysis 5. Stata lab: running nnmatch Readings: Guo & Fraser, chapter 6. Abadie, A., Drukker, D., Herr, J. L., & Imbens, G. W. (2004). Implementing matching estimators for average treatment effects in Stata. The Stata Journal, 4, 290-311. Assignment 3 due sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 5 Assignment 4 out (Due: 10-31-13): Exercise of running matching estimators. 10-31-13 Session 10: Propensity Score Analysis with Nonparametric Regression 1. The kernel-based matching estimator 2. Review of the basic concepts of lowess 3. Stata lab: running kernel-based matching with psmatch2 Readings: Guo & Fraser, chapter 7. Heckman, J. J., Ichimura, H., & Todd, P. E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. Review of Economic Studies, 64, 605-654. Heckman, J. J., Ichimura, H., & Todd, P. E. (1998). Matching as an econometric evaluation estimator. Review of Economic Studies, 65, 261-294. Abadie, A., Imbens, G.W. (2008). On the failure of the bootstrap for matching estimators. Econometrica, 76(6):1537–1557. Assignment 4 due 11-07-13 Session 11: Selection bias and Sensitivity Analysis (Part 1) 1. Overview of selection bias 2. Rosenbaum’s sensitivity analysis Readings: Guo & Fraser, chapter 8. Rosenbaum, P.R. (2005). Sensitivity analysis in observational studies. In B.S. Everitt and D. C. Howell (Eds.) Encyclopedia of Statistics in Behavioral Science (pp.1809-1814). New York: John Wiley & Sons, Ltd. Assignment 5 out (Due: 11-14-13): Classroom presentation. Students will be divided into small groups. Each group will present study findings on the following questions: (1) What key challenges researchers may encounter in observational studies? (2) What do you learn from the debates between econometricians and statisticians? (3) What core issues may be summarized from the literature with regard to selection bias? (4) What do you learn from the critiques to the nonexperimental approaches? 11-14-13 Session 12: Selection bias, Sensitivity Analysis (Part 2), Student Presentation 1. Stata lab: running sensitivity analysis with rbounds 2. Critical review of evaluation studies 3. Criticism toward nonexperimental approaches 4. Student presentation 5. Analysis of treatment dosage Readings: Guo & Fraser, chapter 9: 9.1-9.2. Michalopoulos, C., Bloom, H. S., & Hill, C. J. (2004). Can propensity-score methods match the findings from a random assignment evaluation of mandatory welfareto-work programs? The Review of Economics and Statistics, 86, 156-179. Agodini, R., & Dynarski, M. (2004). Are experiments the only option? A look at dropout prevention programs. The Review of Economics and Statistics, 86, 180194. Guo & Fraser, draft of PSA 2nd edition – to be supplied by the instructor Assignment 5 due sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 6 Assignment 6 out (Due: 11-18-13): Exercise of running kernel-based matching and differencein-differences analysis; and exercise of running Rosenbaum’s sensitivity analysis using Wilcoxon’s signed rank test. Assignment 6 due on 11-18-13 11-21-13 Session 13: Heckman’s Model of Causality and Debates 1. Overview of Heckman’s model of causality 2. Debates between econometricians and statisticians 3. PSA and SEM 3. Other advances in modeling causality and future development Readings: Guo & Fraser, chapter 2 (pp.50-54). Guo & Fraser, chapter 9: 9.3-9.4. Guo & Fraser, draft of PSA 2nd edition – to be supplied by the instructor Heckman, J. J. (2005). The scientific model of causality. Sociological Methodology, 35, 1-97. Sobel, M. E. (2005). Discussion: “The scientific model of causality.” Sociological Methodology, 35, 99–133. Final exam out (Due: 12-06-13): In this take-home exam, you are required to write a paper comparing at least two propensity score methods. The paper should include: (1) a brief introduction to describe research questions and hypotheses; (2) a method section to describe correction methods being compared; (3) findings, and (4) discussion. The paper should meet the requirements and standards for its publication in a peer-reviewed journal. 12-06-13: Final exam due sowo 919: Advanced Topics in Causal Inference: Propensity Score and Related Models page 7