Statistics 921:
Design and Analysis of Experiments and Observational Studies
Syllabus
Professor: Dylan Small
E-mail: dsmall@wharton.upenn.edu
Office: 464 Huntsman Hall
Office Hours: Tuesday, 4:45-5:45; Wednesday, 9-10; Thursday, 4:45-5:45; by
appointment.
Class Website: www-stat.wharton.upenn.edu/~dsmall/stat921-f07
Text: The Theory of the Design of Experiments, D.R. Cox and N. Reid (recommended).
Additional readings will be handed out. A good reference for many of the topics in the
course is P. Rosenbaum (2002), Observational Studies, 2nd Edition, New York: SpringerVerlag.
Course description: This course will cover statistical methods for the design and
analysis of experiments and observational studies. The first half of the course will focus
on the design and analysis of experiments and will cover The Theory of the Design of
Experiments by Cox and Reid. The second half of the course will focus on observational
studies and will cover readings from the statistical literature. Topics for the course will
include the potential outcomes framework for causal inference; control of bias and
haphazard variation in experiments; factorial experiments; matching, propensity score
and regression methods for controlling confounding in observational studies; tests of
hidden bias; sensitivity analysis; and instrumental variables.
Course Requirements: There will be regular homework assignments and a final project.
Computing software: We will make use of the freeware statistical computing software
R. R can be downloaded from http://www.r-project.org/ .
Course Prerequisites: Stat 541.
Tentative Schedule:
1. Week 1 (1 class)
Topics: The potential outcomes framework for causal inference in statistics.
Reading: Holland, P. Statistics and causal inference (with discussion). Journal of the
American Statistical Association 1986; 81: 945-970.
2. Week 2
Topics: General concepts in designing experiments, the avoidance of bias in experiments
and methods of inference for randomized experiments.
Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapters 12.
Rosenbaum, P.R., Covariance adjustment in randomized experiments and observational
studies. Statistical Science 1984; 17: 286-304. Sections 1-2.
Small, D.S., Ten Have, T.R. and Rosenbaum, P.R, Randomization inference in a grouprandomized trial of treatments for depression: covariance adjustment, noncompliance and
quantile effects. Journal of the American Statistical Association, in press. Sections 1,
2.1, 2.3, 3.1-3.3, 4 and 5.
3. Week 3
Topics: Control of haphazard variation in experiments through blocking (randomized
block designs, balanced incomplete block designs, crossover designs).
Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapters 34.
4. Week 4
Topics: Factorial designs. Fractional factorial designs.
Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapter 5.
5. Week 5
Topics: Confounding in designs, split plot designs, response surface methods.
Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapter 6.
6. Week 6
Topics: Optimal design, overflow on design of experiments.
Reading: Cox, D.R. and Reid, N. The Theory of the Design of Experiments. Chapter 7.
7. Week 7 (1 class due to fall break)
Topics: Observational studies examples; the design of observational studies.
Reading: Rosenbaum, P.R. Observational study. In Encyclopedia of Statistics in
Behavioral Sciences, eds. B.S. Everitt and D.S. Howell, 2005.
Rubin, D.B. The design versus the analysis of observational studies for causal effects:
Parallels with the design of randomized trials. Statistics in Medicine 2007; 26: 20-36.
8. Week 8
Topics: Controlling for observed confounders in observational studies.
Readings:
(1) Rosenbaum, P.R. and Rubin, D.B. The central role of the propensity score in
observational studies for causal effects. Biometrika 1983; 70: 41-55.
(2) Rosenbaum, P.R., Covariance adjustment in randomized experiments and
observational studies. Statistical Science 1984; 17: 286-304. Sections 3-4, 7.
(3) Hansen, B.B. Full matching in an observational study of coaching for the SAT.
Journal of the American Statistical Association 2004; 467: 609-618;
(4) Joffe, M.M., Ten Have, T.R., Feldman, H.I and Kimmel, S.E. Model selection,
confounder control, and marginal structural models: review and new applications. The
American Statistician 2004; 58: 272-279.
(5) Dehejia, R.H. and Wahba, S. Causal effects in nonexperimental studies: reevaluating
the evaluation of training programs. Journal of the American Statistical Association
1994; 94: 1053-1062.
9. Week 9
Topics: Sensitivity analysis for unmeasured confounding; tests of ignorable treatment
assignment; use of multiple control groups.
(1) Rosenbaum, P.R. Sensitivity analysis. In Encyclopedia of Statistics in Behavioral
Sciences, eds. B.S. Everitt and D.S. Howell, 2005.
(2) Rosenbaum, P.R., Covariance adjustment in randomized experiments and
observational studies. Statistical Science 1984; 17: 286-304. Section 5.
(3) Rosenbaum, P.R. From association to causation in observational studies: the role of
tests of strongly ignorable treatment assignment. Journal of the American Statistical
Association 1984; 79: 41-48.
(4) Rosenbaum, P.R. Dropping out of high school in the United States: an observational
study. Journal of Educational Statistics 1986; 11: 207-224.
(5) Rosenbaum, P.R. The role of a second control group in an observational study.
Statistical Science 1987; 292-316.
10. Week 10.
Topics: Adjusting for post-treatment variables; time-varying confounding.
Readings:
(1) Rosenbaum, P.R. The consequences of adjustment for a concomitant variable that has
been affected by the treatment. Journal of the Royal Statistical Society, Series A 1984;
656-666.
(2) Hernán, M.A., Brumback, B.A. and Robins, J.M. Estimating the causal effect of
zidovudine on CD4 count with a marginal structural model for repeated measures.
Statistics in Medicine 2002; 21: 1689-1709.
11. Week 11.
Topics: Instrumental variables method for estimating causal effects.
Readings:
(1) Holland, P.W. (1988). Causal inference, path analysis and recursive structural
equations models. Sociological Methodology, 18, 449-484.
(2) Angrist, J., Imbens, G. and Rubin, D. (1996). Identification of causal effects using
instrumental variables (with discussion). Journal of the American Statistical Association
1996; 91: 444-472.
(3) Rosenbaum, P.R., Covariance adjustment in randomized experiments and
observational studies. Statistical Science 1984; 17: 286-304. Sections 5.
(4) Small, D.S., Ten Have, T.R. and Rosenbaum, P.R, Randomization inference in a
group-randomized trial of treatments for depression: covariance adjustment,
noncompliance and quantile effects. Journal of the American Statistical Association, in
press. Sections 2.2, 3.4.
(5) Small, D.S. and Rosenbaum, P.R., War and wages: the strength of instrumental
variables and their sensitivity to unobserved biases. Journal of the American Statistical
Association, in press. Sections 1-3.
12. Week 12
Topics: Design of observational studies.
(1) Rosenbaum, P.R., Choice as an alternative to control in observational studies (with
discussion). Statistical Science 1999; 14: 259-304.
(2) Rosenbaum, P.R., Design sensitivity and observational studies. Biometrika 2004; 91:
153-164.
(3) Rosenbaum, P.R., Heterogeneity and causality: Unit heterogeneity and design
sensitivity in observational studies. American Statistician 2005; 59: 147-152.
(4) Small, D.S. and Rosenbaum, P.R., War and wages: the strength of instrumental
variables and their sensitivity to unobserved biases. Journal of the American Statistical
Association, in press. Sections 4-6.
13. Week 13
Topics: Overflow; additional topics based on students’ interests.
14. Week 14
Topics: Students will make presentations on their final projects.