Searching for Statistical Causal Models: Theory and Practice Richard Scheines Carnegie Mellon University 1 Goals 1) Policy, Law, and Science: How can we use data to answer a) subjunctive questions (effects of future policy interventions), or b) counterfactual questions (what would have happened had things been done differently (law)? c) scientific questions (what mechanisms run the world) 2) Rumsfeld Problem: Do we know what we don’t know: Can we tell when there is or is not enough information in the data to answer causal questions? Early Progenitors Charles Spearman (1904) Statistical Constraints Causal Structure g rm1 * rr1 = rm2 * rr2 m1 m2 r1 r2 Early Progenitors Sewall Wright (1920s,1930s) Graphical Model Causal & Statistical Interpretation The Method of Path Coefficients (1934). Annals of Mathematical Statistics, 5, 161-215. Social Sciences: 1940s 1970s • Factor Analysis • Structural Equation Models • Instrumental Variable Estimators • Simultaneous Equation Models Economics: Sociology, Psychometrics, etc. • Cowles Commission • Franklin Fisher • Hubert Blalock • Art Goldberger • Herb Costner • Clive Granger • Otis Dudley Duncan • Herb Simon • David Heise • Haavelmo • David Kenny • R. Strotz • Ken Bollen • H. Wold Population level Counterfactuals 1970s & 1980s:Graphical Models & Independence Structures • S. Lauritzen • P. Dawid • J. Darroch • D. Cox • T. Speed • J. Robins • H. Kiiveri • J. Whittaker • N. Wermuth • D. Hausman • D. Papineau • Judea Pearl 1988 1988 1993: Axioms, Intervention, and Latent Variable Model Search • P. Spirtes, C. Glymour, and R. Scheines • Causal Markov Axiom • Full model of interventions, both surgical and non-surgical • Equivalence classes for latent variable models, with search Graphical Models Intervention & Manipulation Counterfactuals Constraints (Independence) Modern Non-Parametric Theory of Statistical Causal Models Causal Bayes Nets Semantics of SCMs Choice 1: Take direct causation as primitive, axiomatize Causal systems over V Probabilistic Independence Relations in P(V) Choice 2: Define direct causation in terms of intervention, i.e., (hypothetical) treatment) 10 Choice 1: Causal Markov Axiom If G is a causal graph, and P a probability distribution over the variables in G, then in <G,P> satisfy the Markov Axiom iff: every variable V is independent of its non-effects, conditional on its immediate causes. 11 Causal Graphs Causal Graph G = {V,E} Each edge X Y represents a direct causal claim: X is a direct cause of Y relative to V Exposure Chicken Pox Exposure Rash Infection Rash 12 Causal Graphs Not Cause Complete Exposure Omitted Causes Infection Symptoms Common Cause Complete Omitted Common Causes Exposure Infection Symptoms 13 Interventions & Causal Graphs Model an intervention by adding an “intervention” variable outside the original system as a direct cause of its target. Pre-intervention graph Education Income Taxes Income Taxes Income Taxes Intervene on Income “Soft” Intervention Education I “Hard” Intervention Education I 14 Structural Equation Models Education Causal Graph Income Statistical Causal Model Longevity 1. Structural Equations 2. Statistical Constraints 15 Structural Equation Models Education Causal Graph Income Longevity • Structural Equations: One Assignment Equation for each variable V V := f(parents(V), errorV) for SEM (linear regression) f is a linear function • Statistical Constraints: Joint Distribution over the Error terms 16 Structural Equation Models Causal Graph Education Equations: Education := ed Income := Educationincome Longevity Longevity := EducationLongevity Income SEM Graph Education (path diagram) 1 Income Income 2 Longevity Statistical Constraints: (ed, Income,Income ) ~N(0,2) 2 diagonal - no variance is zero Longevity 17 Two Routes to the Causal Markov Condition • Assumption 1: Weak Causal Markov Assumption V1,V2 causally disconnected V1 _||_ V2 • Assumption 2a: • Assumption 2b: Determinism, e.g., Causal Markov Axiom Structural Equations For each Vi V, Vi := f(parents(Vi)) 18 Choice 2: Define Direct Causation from Intervention X is a cause of Y iff x1 x2 P(Y | X set= x1) P(Y | X set= x2) X is a direct cause of Y relative to S, iff z,x1 x2 P(Y | X set= x1 , Z set= z) P(Y | X set= x2 , Z set= z) where Z = S - {X,Y} 19 Modularity of Intervention/Manipulation Causal Graph Education Income Manipulated Causal Graph Longevity Education Income M1 Longevity Structural Equations: Education := ed Longevity := f (Education)Longevity Income := f (Education)income Manipulated Structural Equations: Education := ed Longevity := f (Education)Longevity Income := f3 (M1) Manipulation --> Causal Markov • Manipulation conception of causation and Modularity --> weak version of CMA • Zhang, Jiji and Spirtes, Peter (2007) Detection of Unfaithfulness and Robust Causal Inference. In [2007] LSEPitt Conference: Confirmation, Induction and Science (London, 8 - 10 March, 2007) http://philsciarchive.pitt.edu/archive/00003188/01/Detection_of_Unfaithfu lness_and_Robust_Causal_Inference.pdf 21 SCM Search Statistical Data Causal Structure Data Equivalence Class of Causal Graphs X1 X1 X1 X2 X2 X2 X3 Statistical Inference Causal Markov Axiom (D-separation) X3 X3 Discovery Algorithm Independence Relations X1 X3 | X2 Background Knowledge - X2 before X3 22 Faithfulness Constraints on a probability distribution P generated by a causal structure G hold for all parameterizations of G. Revenues = aRate + cEconomy + Rev. Economy = bRate + Econ. Tax Rate b a c Economy Faithfulness: a ≠ -bc Tax Revenues 23 Equivalence Classes Equivalence: • Independence (M1 ╞ X _||_ Y | Z M2 ╞ X _||_ Y | Z) • Distribution (q1 q2 M1(q1) = M2(q2)) • Independence (d-separation equivalence) • DAGs : Patterns • PAGs : Latent variable models • Intervention Equivalence Classes • Measurement Model Equivalence Classes • Linear Non-Gaussian Model Equivalence Classes 24 Patterns X1 X2 Pattern X4 X3 Represents X1 X1 X2 X3 X4 25 X2 X3 X4 Patterns: What the Edges Mean X2 X1 and X2 are not adjacent in any member of the equivalence class X1 X2 X1 X2 (X1 is a cause of X2) in every member of the equivalence class. X1 X2 X1 X2 in some members of the equivalence class, and X2 X1 in others. X1 26 PAGs: Partial Ancestral Graphs X2 X1 PAG X3 Represents X2 X1 X2 X1 T1 X3 X3 etc. X1 X2 X2 X1 T1 T1 X3 27 X3 T2 PAGs: Partial Ancestral Graphs What PAG edges mean. X1 X2 X1 and X2 are not adjacent X1 X2 X2 is not an ancestor of X1 X1 X2 No set d-separates X2 and X1 X1 X2 X1 is a cause of X2 X1 X2 There is a latent common cause of X1 and X2 28 Overview of Search Methods • Constraint Based Searches • TETRAD (SGS, PC, FCI) • Very fast – max N ~ 1,000 • Pointwise Consistent • Scoring Searches • Scores: BIC, AIC, etc. • Search: Hill Climb, Genetic Alg., Simulated Annealing • Difficult to extend to latent variable models • Meek and Chickering Greedy Equivalence Class (GES) • Very slow – max N ~ 30-40 29 Tetrad Demo http://www.phil.cmu.edu/projects/tetrad_download/ 30 Case Study 1: Foreign Investment Does Foreign Investment in 3rd World Countries cause Repression? Timberlake, M. and Williams, K. (1984). Dependence, political exclusion, and government repression: Some cross-national evidence. American Sociological Review 49, 141-146. N = 72 PO degree of political exclusivity CV lack of civil liberties EN energy consumption per capita (economic development) FI level of foreign investment 31 Case Study 1: Foreign Investment Correlations fi en cv po -.175 -.480 0.868 fi en 0.330 -.391 -.430 32 Case Study 1: Foreign Investment Regression Results po = .227*fi SE t (.058) 3.941 - .176*en + .880*cv (.059) -2.99 (.060) 14.6 Interpretation: increases in foreign investment increases political exclusion 33 Case Study 1: Foreign Investment Alternatives En FI CV En FI CV En .31 -.23 FI CV .217 .88 -.176 PO -.48 PO Regression Tetrad - FCI .86 PO Fit: df=2, 2=0.12, p-value = .94 No model with testable constraint (df > 0) in which FI has a positive effect on PO Case Study 2: Welfare Reform Single Mothers’ Self-Efficacy, Parenting in the Home Environment, and Children’s Development in a Two-Wave Study (Social Work Research, 29, 1, 7-20) Aurora Jackson, Richard Scheines 35 Case Study 2: Welfare Reform Sampling Scheme • Longitudinal Data o Time 1: 1996-97 (N = 188) o Time 2: 1998-99 (N = 178) • Single black mothers in NYC • Current and former welfare recipients • With a child who was 3 – 5 at time 1, and 6 to 8 at time 2 36 Case Study 2: Welfare Reform Constructs/Scales/Measures • Employment Status • Perceived Self-efficacy • Depressive Symptoms • Quality of Mother/Father Relationship • Father/Child Contact • Quality of Home Environment • Behavior Problems • Cognitive Development 37 Case Study 2: Welfare Reform Background Knowledge Tier 1: • Employment Status Tier 2: Over 22 million path models consistent with these constraints • Depression • Self-efficacy • Mother/Father Relationship • Father/Child Contact • Mother’s Parenting/HOME Tier 3: • Negative Behaviors • Cognitive Development 38 Case Study 2: Welfare Reform Employment Status (Time 1) -.472* .215** Conceptual Model Mother’s depressive -.184* symptoms (Time 1) Mother/Father Relationship (Time 1) .407** Father/Child Contact (Time 1) -.166* .150* Mother’s self-efficacy (Time 1) Mother’s Parenting/ Home Environment (Time 1) .166* -.291** 2 = 22.3, df = 20, p = .32 Negative Behaviors (Time 2) Cognitive Development (Time 2) Mother/Father 2 P = .32 GFI = .97 AGFI = .95 Mother’s depressive * = p < .05 ** = p < .01 (20) = 22.3 Employment -.129* Relationship symptoms (Time 1) .407** Father/Child Status (Time 1) Contact (Time 1) (Time 1) .215** Tetrad Equivalence Class -.456* Mother’s self-efficacy (Time 1) 2 = 18.87, df = 19, p = .46 .162* Mother’s Parenting/ Home Environment (Time 1) .184* -.291** Negative Behaviors (Time 2) .166* Cognitive Development (Time 2) 39 Case Study 2: Welfare Reform Employment Status (Time 1) Points of Agreement: • • Mother’s Self-Efficacy mediates the effect of Employment on all other variables. HOME environment mediates the effect of all other factors on outcomes: Cog. Develop and Prob. Behaviors Mother’s depressive -.184* symptoms (Time 1) -.472* .215** Mother/Father Relationship (Time 1) .407** Father/Child Contact (Time 1) -.166* .150* Mother’s self-efficacy (Time 1) Mother’s Parenting/ Home Environment (Time 1) .166* -.291** Conceptual Model Negative Behaviors (Time 2) Cognitive Development (Time 2) Mother/Father 2 P = .32 GFI = .97 AGFI = .95 Mother’s depressive * = p < .05 ** = p < .01 (20) = 22.3 Employment -.129* Relationship symptoms (Time 1) .407** Father/Child Status (Time 1) Contact (Time 1) (Time 1) Points of Disagreement: • Depression key cause vs. only an effect .215** -.456* Mother’s self-efficacy (Time 1) .162* Mother’s Parenting/ Home Environment (Time 1) .184* -.291** Tetrad Negative Behaviors (Time 2) .166* Cognitive Development (Time 2) 40 Case Study 3: Online Courseware Online Course in Causal & Statistical Reasoning 41 Case Study 3: Online Courseware Variables Pre-test (%) Print-outs (% modules printed) Quiz Scores (avg. %) Voluntary Exercises (% completed) Final Exam (%) 9 other variables 42 Case Study 3: Online Courseware Printing and Voluntary Comprehension Checks: 2002 --> 2003 2002 2003 -.41** voluntary questions print .302* .75** pre quiz print -.16 voluntary questions -.08 pre .353* .41* .323* .25* final final 43 Case Study 4: Charitable Giving Variables Tangibility/Concreteness (Exp manipulation) Imaginability (likert 1-7) Impact (avg. of 2 likerts) Sympathy (likert) Donation ($) 44 Case Study 4: Charitable Giving Theoretical Model Impact Tangibility Imaginability Donation Sympathy study 1 (N= 94) df = 5, 2 = 52.0, p= 0.0000 study 2 (N= 115) df = 5, 2 = 62.6, p= 0.0000 45 Case Study 4: Charitable Giving Impact Tangibility Imaginability study 1: df = 5, 2 = 5.88, p= 0.32 Donation study 2: df = 5, 2 = 8.23, p= 0.14 GES Outputs Sympathy Impact Tangibility Imaginability study 1: df = 5, 2 = 3.99, p= 0.55 Donation study 2: df = 5, 2 = 7.48, p= 0.18 Sympathy 46 The Causal Theory Formation Problem for Latent Variable Models Given observations on a number of variables, identify the latent variables that underlie these variables and the causal relations among these latent concepts. Example: Spectral measurements of solar radiation intensities. Variables are intensities at each measured frequency. Example: Quality of a Child’s Home Environment, Cumulative Exposure to Lead, Cognitive Functioning 47 The Most Common Automatic Solution: Exploratory Factor Analysis • Chooses “factors” to account linearly for as much of the variance/covariance of the measured variables as possible. • Great for dimensionality reduction • Factor rotations are arbitrary • Gives no information about the statistical and thus the causal dependencies among any real underlying factors. • No general theory of the reliability of the procedure 48 Other Solutions • Independent Components, etc • Background Theory • Scales 49 Other Solutions: Background Theory Specified Model St1 12 St2 1.2 . T1 Cognitive Function Home ? St21 12 C2 . . T20 Lead C1 T2 . . C20 Key Causal Question Thus, key statistical question: Lead _||_ Cog | Home ? 50 Other Solutions: Background Theory True Model St1 12 St2 1.2 . T1 Cognitive Function Home St21 12 T2 . . T20 Lead C1 C2 . . “Impurities” C20 F Lead _||_ Cog | Home ? Yes, but statistical inference will say otherwise. 51 Purify Specified Model F1 x1 x2 x3 F3 F2 x4 y1 y2 y3 y4 z1 z2 z3 z4 52 Purify True Model F1 x1 x2 x3 F3 F2 x4 y1 y2 y3 y4 z1 z2 z3 z4 F 53 Purify True Model F1 x1 x2 x3 F3 F2 x4 y1 y2 y3 y4 z1 z2 z3 z4 F 54 Purify True Model F1 x1 x2 x3 F3 F2 x4 y1 y2 y3 y4 z1 z2 z3 z4 F 55 Purify True Model F1 x1 x2 x3 F3 F2 x4 y1 y2 y3 y4 z1 z2 z3 z4 F 56 Purify Purified Model F1 x1 x2 x3 F3 F2 y1 y2 y3 y4 z1 z3 z4 F 57 Other Solutions: Scales Scale = sum(measures of a latent) St1 12 Homescale St2 1.2 . Homescale = i=1 to 21 (Sti) Home St21 12 58 Other Solutions: Scales True Model Pseudo-Random Sample: N = 2,000 59 Scales vs. Latent variable Models True Model Regression: Cognition on Home, Lead Predictor Constant Home Lead S = 0.9940 Coef -0.02291 1.22565 -0.00575 SE Coef 0.02224 0.02895 0.02230 R-Sq = 61.1% T -1.03 42.33 -0.26 P 0.303 0.000 0.797 Insig. R-Sq(adj) = 61.0% 60 Scales vs. Latent variable Models True Model Scales homescale = (x1 + x2 + x3)/3 leadscale = (x4 + x5 + x6)/3 cogscale = (x7 + x8 + x9)/3 61 Scales vs. Latent variable Models True Model Regression: Cognition on homescale, Lead Cognition = - 0.0295 + 0.714 homescale - 0.178 Lead Predictor Constant homescal Lead Coef -0.02945 0.71399 -0.17811 SE Coef 0.02516 0.02299 0.02386 T -1.17 31.05 -7.46 P 0.242 0.000 0.000 Sig. 62 Scales vs. Latent variable Models True Model Modeling Latents Specified Model 63 Scales vs. Latent variable Models True Model Estimated Model (2 = 29.6, df = 24, p = .19) B5 = .0075, which at t=.23, is correctly insignificant 64 Scales vs. Latent variable Models True Model Mixing Latents and Scales (2 = 14.57, df = 12, p = .26) B5 = -.137, which at t=5.2, is incorrectly highly significant P < .001 65 Build Pure Clusters Output - provably reliable (pointwise consistent): Equivalence class of measurement models over a pure subset of measures Dep Stress True Model m1 m2 m3 Health m4 m5 m6 m10 m11 m7 m8 m9 m BPC L1 Output m1 m2 m3 L2 L3 m4 m5 m6 m7 m8 m9 66 Build Pure Clusters Qualitative Assumptions 1. Two types of nodes: measured (M) and latent (L) 2. M 3. Each m M measures (is a direct effect of) at least one l L 4. No cycles involving M L (measured don’t cause latents) Quantitative Assumptions: 1. Each m M is a linear function of its parents plus noise 2. P(L) has second moments, positive variances, and no deterministic relations 67 Case Study 4: Stress, Depression, and Religion MSW Students (N = 127) 61 - item survey (Likert Scale) • Stress: St1 - St21 • Depression: D1 - D20 • Religious Coping: C1 - C20 St1 12 St2 1.2 . Specified Model Stress Depression + - + St21 12 C1 C2 . . . . Dep20 12 Coping p = 0.00 Dep1 12 Dep2 12 C20 68 Case Study 4: Stress, Depression, and Religion Build Pure Clusters St3 12 St4 12 St16 12 St18 12 St20 12 Stress Depression Coping C9 C12 C14 Dep9 12 Dep13 12 Dep19 12 C15 69 Case Study 4: Stress, Depression, and Religion Assume Stress temporally prior: MIMbuild to find Latent Structure: St3 12 St4 12 St16 12 St18 12 St20 12 + Stress Depression + Coping C9 C12 C14 C15 Dep9 12 Dep13 12 Dep19 12 p = 0.28 70 Case Study 5: Test Anxiety Bartholomew and Knott (1999), Latent variable models and factor analysis 12th Grade Males in British Columbia (N = 335) 20 - item survey (Likert Scale items): X1 - X20: X2 Exploratory Factor Analysis: X3 X4 X8 X5 X9 X10 Emotionality Worry X6 X7 X15 X14 X16 X17 X18 X20 71 Case Study 5: Test Anxiety Build Pure Clusters: X3 X2 X8 Cares About Achieving X9 X10 X11 X16 X5 X7 Emotionalty SelfDefeating X6 X14 X18 72 Case Study 5: Test Anxiety Build Pure Clusters: Exploratory Factor Analysis: X3 X2 X4 X8 X3 X2 X8 Worries About Achieving X5 X5 X9 X9 X10 Emotionality Worry X6 X10 X7 Emotionalty X7 X15 X14 X16 X17 X18 X20 p-value = 0.00 X11 X16 X6 SelfDefeating X14 X18 p-value = 0.47 73 Case Study 5: Test Anxiety MIMbuild X3 X2 X8 Worries About Achieving X9 X10 X11 X16 Scales: No Independencies or Conditional Independencies X5 X7 Emotionalty SelfDefeating X6 X14 Worries About Achieving-Scale EmotionaltyScale SelfDefeating X18 p = .43 Unininformative 74 Other Cases Economics Climate Research Bessler, Pork Prices Hoover, multiple Cryder & Loewenstein, Charitable Giving Glymour, Chu, , Teleconnections Epidemiology Scheines, Lead & IQ Biology Educational Research Easterday, Bias & Recall Laski, Numerical coding Shipley, SGS, Spartina Grass Neuroscience Glymour & Ramsey, fMRI 75 References General Spirtes, P., Glymour, C., Scheines, R. (2000). Causation, Prediction, and Search, 2nd Edition, MIT Press. Pearl, J. (2000). Causation: Models of Reasoning and Inference, Cambridge University Press. Biology Chu, Tianjaio, Glymour C., Scheines, R., & Spirtes, P, (2002). A Statistical Problem for Inference to Regulatory Structure from Associations of Gene Expression Measurement with Microarrays. Bioinformatics, 19: 1147-1152. Shipley, B. Exploring hypothesis space: examples from organismal biology. Computation, Causation and Discovery. C. Glymour and G. Cooper. Cambridge, MA, MIT Press. Shipley, B. (1995). Structured interspecific determinants of specific leaf area in 34 species of herbaceous angeosperms. Functional Ecology 9. 76 References Scheines, R. (2000). Estimating Latent Causal Influences: TETRAD III Variable Selection and Bayesian Parameter Estimation: the effect of Lead on IQ, Handbook of Data Mining, Pat Hayes, editor, Oxford University Press. Jackson, A., and Scheines, R., (2005). Single Mothers' Self-Efficacy, Parenting in the Home Environment, and Children's Development in a Two-Wave Study, Social Work Research , 29, 1, pp. 7-20. Timberlake, M. and Williams, K. (1984). Dependence, political exclusion, and government repression: Some cross-national evidence. American Sociological Review 49, 141-146. 77 References Economics Akleman, Derya G., David A. Bessler, and Diana M. Burton. (1999). ‘Modeling corn exports and exchange rates with directed graphs and statistical loss functions’, in Clark Glymour and Gregory F. Cooper (eds) Computation, Causation, and Discovery, American Association for Artificial Intelligence, Menlo Park, CA and MIT Press, Cambridge, MA, pp. 497-520. Awokuse, T. O. (2005) “Export-led Growth and the Japanese Economy: Evidence from VAR and Directed Acyclical Graphs,” Applied Economics Letters 12(14), 849-858. Bessler, David A. and N. Loper. (2001) “Economic Development: Evidence from Directed Acyclical Graphs” Manchester School 69(4), 457-476. Bessler, David A. and Seongpyo Lee. (2002). ‘Money and prices: U.S. data 1869-1914 (a study with directed graphs)’, Empirical Economics, Vol. 27, pp. 427-46. Demiralp, Selva and Kevin D. Hoover. (2003) !Searching for the Causal Structure of a Vector Autoregression," Oxford Bulletin of Economics and Statistics 65(supplement), pp. 745-767. Haigh, M.S., N.K. Nomikos, and D.A. Bessler (2004) “Integration and Causality in International Freight Markets: Modeling with Error Correction and Directed Acyclical Graphs,” Southern Economic Journal 71(1), 145-162. Sheffrin, Steven M. and Robert K. Triest. (1998). ‘A new approach to causality and economic growth’, unpublished typescript, University of California, Davis. 78 References Economics Swanson, Norman R. and Clive W.J. Granger. (1997). ‘Impulse response functions based on a causal approach to residual orthogonalization in vector autoregressions’, Journal of the American Statistical Association, Vol. 92, pp. 357-67. Demiralp, S., Hoover, K., & Perez, S. A Bootstrap Method for Identifying and Evaluating a Structural Vector Autoregression Oxford Bulletin of Economics and Statistics, 2008, 70, (4), 509533 - Searching for the Causal Structure of a Vector Autoregression Oxford Bulletin of Economics and Statistics, 2003, 65, (s1), 745-767 Kevin D. Hoover, Selva Demiralp, Stephen J. Perez, Empirical Identification of the Vector Autoregression: The Causes and Effects of U.S. M2*, This paper was written to present at the Conference in Honour of David F. Hendry at Oxford University, 2325 August 2007. Selva Demiralp and Kevin D. Hoover , Searching for the Causal Structure of a Vector Autoregression, OXFORD BULLETIN OF ECONOMICS AND STATISTICS, 65, SUPPLEMENT (2003) 0305-9049 A. Moneta, and P. Spirtes “Graphical Models for the Identification of Causal Structures in Multivariate Time Series Model”, Proceedings of the 2006 Joint Conference on Information Sciences, JCIS 2006, Kaohsiung, Taiwan, ROC, October 8-11,2006, Atlantis Press, 2006. 79 References • Causation, Prediction, and Search, 2nd Edition, (2000), by P. Spirtes, C. Glymour, and R. Scheines ( MIT Press) • Causality: Models, Reasoning, and Inference (2000). By Judea Pearl, Cambridge Univ. Press • Computation, Causation, & Discovery (1999), edited by C. Glymour and G. Cooper, MIT Press • Eberhardt, F., and Scheines R., (2007).“Interventions and Causal Inference”, in PSA-2006, Proceedings of the 20th biennial meeting of the Philosophy of Science Association 2006 http://philsci.org/news/PSA06 • Silva, R., Glymour, C., Scheines, R. and Spirtes, P. (2006) “Learning the Structure of Latent Linear Structure Models,” Journal of Machine Learning Research, 7, 191-246. • TETRAD IV: www.phil.cmu.edu/projects/tetrad • Web Course on Causal and Statistical Reasoning : www.phil.cmu.edu/projects/csr/ • Causality Lab: www.phil.cmu.edu/projects/causality-lab