Nonlinear Models with Spatial Data William Greene Stern School of Business, New York University Washington D.C. July 12, 2013 Binary Outcome: Y=1[New Plant Located in County] Klier and McMillen: Clustering of Auto Supplier Plants in the United States. JBES, 2008 On Our Program School District Open Enrollment: A Spatial Multinomial Logit Approach; David Brasington, University of Cincinnati, USA, Alfonso Flores-Lagunes, State University of New York at Binghamton, USA, Ledia Guci, U.S. Bureau of Economic Analysis, USA Smoothed Spatial Maximum Score Estimation of Spatial Autoregressive Binary Choice Panel Models; Jinghua Lei, Tilburg University, The Netherlands Application of Eigenvector-based Spatial Filtering Approach to a Multinomial Logit Model for Land Use Data; Takahiro Yoshida & Morito Tsutsumi, University of Tsukuba, Japan Estimation of Urban Accessibility Indifference Curves by Generalized Ordered Models and Kriging; Abel Brasil, Office of Statistical and Criminal Analysis, Brazil, & Jose Raimundo Carvalho, Universidade Federal do Cear´a, Brazil Choice Set Formation: A Comparative Analysis, Mehran Fasihozaman Langerudi, Mahmoud Javanmardi, Kouros Mohammadian, P.S Sriraj, University of Illinois at Chicago, USA, & Behnam Amini, Imam Khomeini International University, Iran Not including semiparametric and quantile based linear specifications Also On Our Program Ecological fiscal incentives and spatial strategic interactions: the José Gustavo Féres Institute of Applied Economic Research (IPEA) Sébastien Marchand_ CERDI, University of Auvergne Alexandre Sauquet_ CERDI, University of Auvergne (Tobit) The Impact of Spatial Planning on Crime Incidence : Evidence from Koreai Hyun Joong Kimii Ph.D. Candidate, Program in Regional Information, Seoul National University & Hyung Baek Lim Professor, Dept. of Community Development SungKyul University (Spatially Autoregressive Probit) Spatial interactions in location decisions: Empirical evidence from a Bayesian Spatial Probit model Adriana Nikolic, Christoph Weiss, Department of Economics, Vienna University of Economics and Business A geographically weighted approach to measuring efficiency in panel data: The case of US saving banks Benjamin Tabak, Banco Central do Brasil, Brazil, Rogerio B. Miranda, Universidade Catolica de Brasılia, Brazil, & Dimas M Fazio, Universidade de Sao Paulo (Stochastic Frontier) Spatial Autoregression in a Linear Model y = Wy + Xβ ε. E[ε | X ] = 0, Var[ε | X ]=2 I y = [I W]1 (Xβ ε ) = [I W]1 Xβ [I W]1 ε E[y | X ] = [I W]1 Xβ Var[y | X ] = 2 [(I W)(I W)]-1 Estimators: Various forms of generalized least squares. Maximum likelihood | ε ~ Normal[0,] Spatial Autocorrelation in Regression y Xβ (I - W)ε. E[ε | X ]=0, Var[ε | X]=2 I E[y | X ]=Xβ Var[y | X] = 2 (I - W)(I - W) A Generalized Regression Model ˆ X (I - W )(I - W ) X β 2 ˆ 1 1 X (I - W )(I - W ) y 1 1 ˆ (I - W)(I - W )1 y - Xβ ˆ y - Xβ N ˆ The subject of much research Panel Data Applications (many at this meeting) E.g., N countries, T periods y it xitβ c i it ε t Wε t v t = N observations at time t. Similar assumptions Candidate for SUR or Spatial Autocorrelation model. Analytical Environment Generalized linear regression Complicated disturbance covariance matrix Estimation platform: Generalized least squares, GMM or maximum likelihood. Central problem, estimation of Practical Obstacles Problem: Maximize logL involving sparse (I-W) Inaccuracies in determinant and inverse Appropriate asymptotic covariance matrices for estimators Estimation of . There is no natural residual based estimator Potentially very large N – GIS data on agriculture plots Complicated covariance structures – no simple transformations Outcomes in Nonlinear Settings Land use intensity in Austin, Texas – Discrete Ordered Intensity = ‘1’ < ‘2’ < ‘3’ < ‘4’ Land Usage Types, 1,2,3 … – Discrete Unordered Oak Tree Regeneration in Pennsylvania – Count Number = 0,1,2,… (Excess (vs. Poisson) zeros) Teenagers in the Bay area: physically active = 1 or physically inactive = 0 – Binary Pedestrian Injury Counts in Manhattan – Count Efficiency of Farms in West-Central Brazil – Stochastic Frontier Catch by Alaska trawlers in a nonrandom sample Nonlinear Outcomes Discrete revelation of choice indicates latent underlying preferences Binary choice between two alternatives Unordered choice among multiple choices Ordered choice revealing underlying strength of preferences Counts of events Stochastic frontier and efficiency Nonrandom sample selection Modeling Discrete Outcomes “Dependent Variable” typically labels an outcome No quantitative meaning Conditional relationship to covariates No “regression” relationship in most cases. Models are often not conditional means. The “model” is usually a probability Nonlinear models – usually not estimated by any type of linear least squares Objective of estimation is usually partial effects, not coefficients. Nonlinear Spatial Modeling Discrete outcome yit = 0, 1, …, J for some finite or infinite (count case) J. i = 1,…,n t = 1,…,T Covariates xit Conditional Probability (yit = j) = a function of xit. Issues in Spatial Discete Choice A series of Issues Spatial dependence between alternatives: Nested logit Spatial dependence in the LPM: Solves some practical problems. A bad model Spatial probit and logit: Probit is generally more amenable to modeling Statistical mechanics: Social interactions – not practical Autologistic model: Spatial dependency between outcomes vs. utilities. Variants of autologistic: The model based on observed outcomes is incoherent (“self contradictory”) Endogenous spatial weights Spatial heterogeneity: Fixed and random effects. Not practical? The models discussed below Two Platforms Random Utility for Preference Models Outcome reveals underlying utility Binary: u* = ’x y = 1 if u* > 0 Ordered: u* = ’x y = j if j-1 < u* < j Unordered: u*(j) = ’xj , y = j if u*(j) > u*(k) Nonlinear Regression for Count Models Outcome is governed by a nonlinear regression E[y|x] = g(,x) Maximum Likelihood Estimation Cross Section Case: Binary Outcome Random Utility: y* = x + Observed Outcome: y = 1 if y* > 0, 0 if y* 0. Probabilities: P(y=1|x) = Prob(y* > 0|x ) = Prob( > -x ) P(y=0|x) = 1 - P(y=1|x) Likelihood for the sample = joint probability = Log Likelihood = n i=1 n i=1 Prob(y=y i|x i ) logProb(y=y i|x i ) Cross Section Case: n Observations y1=j | x1 1 or > x1 Prob(1 or > x1 ) y =j | x or > x Prob( or > x ) 2 2 2 2 Prob 2 Prob 2 = ... ... ... y =j | x or > x Prob( or > x ) n n n n n n Operate on the marginal probabilities of n observations LogL(|X, y )= Probit Logit n i=1 logF 2y i 1 x i F(t) = (t) t 1 exp( t / 2)dt 2 exp(t) F(t) = (t) = 1 exp(t) 2 t (t)dt Spatially Correlated Observations Correlation Based on Unobservables y1 x1 u1 y 2 x 2 u2 ... y n x n un 0 u1 1 u 0 2 I W 2 ~ f , I W I W ... ... ... u 0 n n W = the usual spatial weight matrix. In the cross section case, W= 0. Now, it is a full matrix. The joint probably is a single n fold integral. Spatially Correlated Observations Based on Correlated Utilities y1* y1* x1 1 x1 1 * * y 2 W y 2 x 2 2 I W 1 x 2 2 ... ... ... ... * * x n n yn y n x n n W = the usual spatial weight matrix. In the cross section case, W = 0. Now, it is a full matrix. The joint probably is a single n fold integral. LogL for an Unrestricted BC Model LogL(|X, y )= q11 1 q 2 2 q1q2 w 21 n ... ... q n n qnq1wn1 q1q2 w12 ... q1qn w1n 1 xn x1 1 ... q2qn w 2n 2 log ... d ... ... ... ... qnq2 wn2 ... 1 n qi 1 if y i = 0 and +1 if y i = 1 = 2y i 1 One huge observation - n dimensional normal integral. Not feasible for any reasonable sample size. Even if computable, provides no device for estimating sampling standard errors. Spatial Autoregression Based on Observed Outcomes y1* y1 x1 1 * x y y 2 2 2 W 2 ... ... ... * x y y n n n n y i 1[y i* 0] y1 1[(w12 y 2 w13 y 3 ...) x1 1 0] y 2 1[(w 21 y1 w 23 y 3 ...) x 2 2 0] etc. The model based on observables is more reasonable. There is no reduced form unless is W lower triangular. This model is not identified. (It is "incoherent.") See, also, Maddala (1983) From Klier and McMillen (2012) Solution Approaches for Binary Choice Approximate the marginal density and use GMM (possibly with the EM algorithm) Distinguish between private and social shocks and use pseudo-ML Parameterize the spatial correlation and use copula methods Define neighborhoods – make W a sparse matrix and use pseudo-ML Others … Spatial autocorrelation in the heterogeneity yi* x i i ji w ij j x i * yi 1 [yi 0], Prob y i 1 Var i ji w ij j or yi* x i ui x x i * i yi 1 [yi 0]Prob y i 1 Var ui i i2 1 2 ji w ij2 GMM in the Base Case with = 0 Pinske, J. and Slade, M., (1998) “Contracting in Space: An Application of Spatial Statistics to Discrete Choice Models,” Journal of Econometrics, 85, 1, 125-154. Pinkse, J. , Slade, M. and Shen, L (2006) “Dynamic Spatial Discrete Choice Using One Step GMM: An Application to Mine Operating Decisions”, Spatial Economic Analysis, 1: 1, 53 — 99. y*=X+, = Wε+u = [I-W]1u = Au Cross section case: =0 Probit Model: FOC for estimation of is based on the ˆi = y i E[i | y i ] generalized residuals u n i=1 (y (x i ))(x i ) xi i =0 ( x )[1 ( x )] i i See, also, Bertschuk, I., and M. Lechner, 1998. “Convenient Estimators for the Panel Probit Model.” Journal of Econometrics, 87, 2, pp. 329–372 GMM in the Spatial Autocorrelation Model y*=X+, = Wε+u = [I-W]1u = Au Autocorrelated Case: 0 Moment equations are still valid. Complication is computing the variance of the moment equations for the weighting matrix, which requires some approximations. Probit Model: FOC for estimation of is based on the ˆi = y i E[ | y i ] generalized residuals u n i=1 x i x i y i a () a () ii ii zi =0 xi 1 xi aii () aii () Requires at least K +1 instrumental variables. Using the GMM Approach Spatial autocorrelation induces heteroscedasticity that is a function of Moment equations include the heteroscedasticity and an additional instrumental variable for identifying . LM test of = 0 is carried out under the null hypothesis that = 0. Application: Contract type in pricing for 118 Vancouver service stations. A Spatial Logit Model y* = X+ e, e = We + = (I-W )1 2 d = 1[y 0], Var[e] = (I-W) (I-W) , ii i * 1 xi * Prob(yi 1) x i i i Algorithm Generalized residual ui di i, Instruments Z i (1 i )xi* ui / 1 1 * gi , A =( I W ) W ( I W ) x i ui / i (1 i ) 2 Aii i G [g1, g2 ,...gn ] Iterated 2SLS (GMM) q = u(,)Z(ZZ)1Zu(,) 1. Logit estimation of β | =0, G0 ˆ Z( ZZ)1ZG 2. u = (d - ˆ ), G k k ˆ G ˆ 3. k G k k k 1 k ˆ u G k k ˆ ˆ 4. k until k is sufficiently small. ˆ k 1 ˆ k An LM Type of Test? If = 0, g = 0 because Aii = 0 At the initial logit values, g = 0 If = 0, g = 0. Under the null hypothesis the entire score vector is identically zero. How to test = 0 using an LM test? Same problem shows up in RE models But, here, is in the interior of the parameter space! Pseudo Maximum Likelihood Maximize a likelihood function that approximates the true one Produces consistent estimators of parameters How to obtain standard errors? Asymptotic normality? Conditions for CLT are more difficult to establish. Pseudo MLE y*=Xθ+, = Wε+u = [I-W]1u = Au Autocorrelated Case: 0 y i* x i i ji Wij j y i 1[y i* 0]. Var[y i* ] 1 2 ji Wij2 aii () Implies a heteroscedastic probit. Pseudo MLE is based on the marginal densities. How to obtain the asymptotic covariance matrix? [See Wang, Iglesias, Wooldridge (2013)] Heteroscedastic Probit Approac h Estimation and Inference MLE: logL = n i1 (2yi -1)x i log i n ˆ MLE H1( )S( ) S=Score vector implies the algorithm, Newton's Method. EM algorithm essentially replaces H with XX during iterations. (Slightly more involved for the heteroscedasticity. LHS variable in the EM iterations is the score vector.) To compute the asymptotic covariance, we need Var[S( )] Observations are (spatially) correlated! How to compute it? Covariance Matrix for Pseudo-MLE V A(data, ˆ ) B(data, ˆ ) A(data, ˆ ) A(data, ˆ ) Negative inverse of Hessian B(data, ˆ ) Covariance matrix of scores. How to compute B(data, ˆ ) Terms are not independent in a spatial setting. ‘Pseudo’ Maximum Likelihood Smirnov, A., “Modeling Spatial Discrete Choice,” Regional Science and Urban Economics, 40, 2010. Spatial Autoregression in Utilities y * Wy * X , y 1( y* 0) for all n individuals y * (I W )1 X (I W ) 1 (I W) 1 t 0 (W) t assumed convergent = A = D + A -D where D = diagonal elements y * AX D A - D Private Social Suppose individuals ignore the social "shocks." Then nj1aij x j Prob[y i 1 or 0 | X ] F (2y i 1) , probit or logit. di Pseudo Maximum Likelihood Bases correlation on underlying utilities Assumes away the correlation in the reduced form Makes a behavioral assumption Still requires inversion of (I-W) Computation of (I-W) is part of the optimization process - is estimated with . Does not require multidimensional integration (for a logit model, requires no integration) Other Approaches Case A (1992) Neighborhood influence and technological change. Economics 22:491–508 Beron KJ, Vijverberg WPM (2004) Probit in a spatial context: a monte carlo analysis. In: Anselin L, Florax RJGM, Rey SJ (eds) Advances in spatial econometrics: methodology, tools and applications. Springer, Berlin Beron and Vijverberg (2003): Brute force integration using GHK simulator in a probit model. Impractical. Case (1992): Define “regions” or neighborhoods. No correlation across regions. Produces essentially a panel data probit model. (Also Wang et al. (2013)) LeSage: Bayesian - MCMC Copula method. Closed form. See Bhat and Sener, 2009. See also Arbia, G., “Pairwise Likelihood Inference for Spatial Regressions Estimated on Very Large Data Sets” Manuscript, Catholic University del Sacro Cuore, Rome, 2012. Partial MLE (Looks Like Case, 1992) Observation 1 y * x W 1 1 1 1j j j1 2 y 1[y * 0] Var[y * ] 1 2 W a ( ) 1 1 1 1j 11 j1 Observation 2 y * x W 2 2 2 2j j j2 2 y 1[y * 0] Var[y * ] 1 2 W a ( ) 2 2 2 2j 22 j2 Covariance of y1* and y *2 = a12 () Bivariate Probit Pseudo MLE Consistent Asymptotically normal? Resembles time series case Correlation need not fade with ‘distance’ Better than Pinske/Slade Univariate Probit? How to choose the pairings? Core Model y* = ρWSAR y * +Xβ + u, Spatial autoregression or u = ρWSEMu + ε, Spatial error model (only one at a time) ε ~ N[0,σ 2I] Censoring : Probit (0,1), Tobit (0,+) Likelihood : L(ρ,β,σ ) 2 ε (I - ρW)y * - Xβ εε exp 2 n /2 2σ 2πσ2 | I - ρW | for SAR ε (I - ρW)( y * - Xβ) for SEM LeSage Methods - MCMC • Bayesian MCMC for all unknown parameters • Data augmentation for unobserved y* • Quirks about sampler for rho. An Ordered Choice Model (OCM) y* βx , we assume x contains a constant term y 0 if y* 0 y = 1 if 0 < y* 1 y = 2 if 1 < y* 2 y = 3 if 2 < y* 3 ... y = J if J-1 < y* J In general : y = j if j-1 < y* j , j = 0,1,...,J -1 , o 0, J , j-1 j, j = 1,...,J OCM for Land Use Intensity A Dynamic Spatial Ordered Choice Model Wang, C. and Kockelman, K., (2009) Bayesian Inference for Ordered Response Data with a Dynamic Spatial Ordered Probit Model, Working Paper, Department of Civil and Environmental Engineering, Bucknell University. Core Model: Cross Section y i* βx i i , y i = j if j-1 y i* j , Var[i ] 1 Spatial Formulation: There are R regions. Within a region y ir* βx ir ui ir , y ir = j if j-1 y ir* j Spatial heteroscedasticity: Var[ir ] r2 Spatial Autocorrelation Across Regions u = Wu + v , v ~ N[0,2v I] u = (I-W)1 v ~ N[0,2v {(I-W )(I-W )} 1 ] The error distribution depends on 2 parameters, 2v and Estimation Approach: Gibbs Sampling; Markov Chain Monte Carlo Dynamics in latent utilities added as a final step: y*(t)=f[y*(t-1)]. Data Augmentation Unordered Multinomial Choice Core Random Utility Model Underlying Random Utility for Each Alternative U(i,j) = j x ij ij , i = individual, j = alternative Preference Revelation Y(i) = j if and only if U(i,j) > U(i,k) for all k j Model Frameworks Multinomial Probit: [1 ,..., J ] ~ N[0, ] Multinomial Logit: [1 ,..., J ] ~ iid type 1 extreme value Spatial Multinomial Probit Chakir, R. and Parent, O. (2009) “Determinants of land use changes: A spatial multinomial probit approach, Papers in Regional Science, 88, 2, 328-346. Utility Functions, land parcel i, usage type j, date t U(i,j,t)=jt x ijt ik ijt (In France) Spatial Correlation at Time t ij l1 willk n Modeling Framework: Normal / Multinomial Probit Estimation: MCMC - Gibbs Sampling Mixed Logit Models for Type of Residential Unit First Law of Geography: [Tobler (1970)] ‘‘Everything is related to everything else, but near things are more related than distant things”. Is there a second law of geography? Heisenberg? * http://www.openloc.eu/cms/storage/openloc/workshops/UNITN/20110324-26/Giuliani/Giuliani_slides.pdf * Arbia, G., R. Benedetti, and G. Espa. 1996. Effects of the MAUP on image classification. Geographical Systems 3:123–41. * http://urizen-geography.nsm.du.edu/~psutton/AAA_Sutton_WebPage/Sutton/Courses/ Geog_4020_Geographic_Research_Methodology/SeminalGeographyPapers/TOBLER.pdf Location Choice Model Common omitted geographic features embedded in the random utility functions Cross – nested multinomial logit model Does the model extension matter? Does the model extension matter? Canonical Model for Counts Rathbun, S and Fei, L (2006) “A Spatial Zero-Inflated Poisson Regression Model for Oak Regeneration,” Environmental Ecology Statistics, 13, 2006, 409-426 Poisson Regression y = 0,1,... exp() j Prob[y = j|x] = j! Conditional Mean = exp(x) Signature Feature: Equidispersion Usual Alternative: Negative Binomial Spatial Effect: Filtered through the mean i = exp(x i +i ) i = m1 w im m i n Zero Inflation There are two states Always zero Zero is one possible value, or 1,2,… Prob(0) = Prob(state 1) + Prob(state 2) P(0|state 2) Used here as a functional form issue – too many zeros. A Blend of Ordered Choice and Count Data Models Bicycle and pedestrian injuries in census tracts in Manhattan. (Count data and ordered outcomes) Numbers of firms locating in Texas counties: Count data (Poisson) Kriging Spatial Autocorrelation in a Sample Selection Model Flores-Lagunes, A. and Schnier, K., “Sample Selection and Spatial Dependence,” Journal of Applied Econometrics, 27, 2, 2012, pp. 173-204. Alaska Department of Fish and Game. Pacific cod fishing eastern Bering Sea – grid of locations Observation = ‘catch per unit effort’ in grid square Data reported only if 4+ similar vessels fish in the region 1997 sample = 320 observations with 207 reported full data Spatial Autocorrelation in a Sample Selection Model LHS is catch per unit effort = CPUE Site characteristics: MaxDepth, MinDepth, Biomass Fleet characteristics: Catcher vessel (CV = 0/1) Hook and line (HAL = 0/1) Nonpelagic trawl gear (NPT = 0/1) Large (at least 125 feet) (Large = 0/1) Spatial Autocorrelation in a Sample Selection Model yi*1 0 xi1 ui1 ui1 j i cij u j1 i1 yi*2 0 xi 2 ui 2 ui 2 j i cij u j 2 i 2 0 12 12 i1 , (?? 1 1??) ~ N , 2 0 12 2 i 2 Observation Mechanism yi1 1 yi*1 > 0 Probit Model yi 2 yi*2 if yi1 = 1, unobserved otherwise. Spatial Autocorrelation in a Sample Selection Model u1 Cu1 1 C = Spatial weight matrix, Cii 0. u1 [I C]1 1 = (1) 1 , likewise for u 2 () , Var[u ] ( ) Cov[u , u ] () ( ) y 0 xi1 j 1 () i1 , Var[ui1 ] * i1 N y 0 xi 2 j 1 ( ) * i2 N (1) ij (2) ij N 2 1 i2 i2 j 1 2 1 N j 1 N i1 i2 12 j 1 (1) 2 ij ( 2) 2 ij (1) ij (2) ij Spatial Weights 1 cij 2 , dij d ij Euclidean distance Band of 7 neighbors is used Row standardized. Two Step Estimation Probit estimated by Pinske/Slade GMM 0 xi1 N N 2 (1) 2 (1) (2) 1 j 1 ()ij ()ij ( )ij j 1 i N (1) 2 ( ) j 1 ij 0 xi1 N 2 (1) 2 1 j 1 ()ij Spatial regression with included IMR in second step (*) GMM procedure combines the two steps in one large estimation. Spatial Stochastic Frontier Production function model y = βx + ε y = βx + v - u v unexplained noise = N[0,1] u = inefficiency > 0; efficiency = exp(-u) Object of estimation is u, not Not a linear regression. Fit by MLE or MCMC. 247 Spanish Dairy Farms, 6 Years A True Random Effects Model* Spatial Stochastic Frontier Models; Accounting for Unobserved Local Determinants of Inefficiency. Schmidt, Moriera, Helfand, Fonseca; Journal of Productivity Analysis, 2009. yij Output of farm j in municipality i in Center-West Brazil yij αi +βxij +vij - u ij (α1 ,...,α n ) conditionally autoregressive based on neighbors αi -α k is smaller when municipalities i and k are closer together * Greene, W., (2005) "Reconsidering Heterogeneity in Panel Data Estimators of the Stochastic Frontier Model", Journal of Econometrics, 126(2), 269-303 Spatial Frontier Models Cost Model Production Model Estimation by Maximum Likelihood LeSage (2000) on Timing “The Bayesian probit and tobit spatial autoregressive models described here have been applied to samples of 506 and 3,107 observations. The time required to produce estimates was around 350 seconds for the 506 observations sample and 900 seconds for the case involving 3,107 observations. … (inexpensive Apple G3 computer running at 266 Mhz.)” Time and Space (In Your Computer) Thank you! …………………………………………….