Spatial Dynamical Modeling with TerraME Tiago Carneiro Gilberto Câmara Pedro Andrade Licence: Creative Commons By Attribution Non Commercial Share Alike http://creativecommons.org/licenses/by-nc-sa/2.5/ Modelling human-environment interactions What models are needed to describe human actions? Clouds: statistical distributions Clocks, clouds or ants? Clocks: deterministic equations Ants: emerging behaviour Dynamic Spatial Models f (It) f (It+1) F f (It+2) f ( It+n ) F .. “A dynamical spatial model is a computational representation of a real-world process where a location on the earth’s surface changes in response to variations on external and internal dynamics on the landscape” (Peter Burrough) Nature-society modelling with TerraME Nature: Physical equations Describe processes Society: Decisions on how to Use Earth´s resources Nature-society modelling with TerraME Nature: Cellular space Nature: Physical equations Describe processes Society: Agents Society: Decisions on how to Use Earth´s resources Modelling collective spatial actions Agent Agent Space Space Benenson and Torrens, “Geographic Automata Systems”, IJGIS, 2005 (but many questions remain...) Computational Modelling with Cell Spaces Cell Spaces Generalized Proximity Matriz – GPM Hybrid Automata model Nested scales Agents in space TerraME - overview Model data in cell spaces Read/write data from a database Cellular Data Base Resolution 2500 m 2.500 m e 500 m Spatial structure Large farmer (25 cells) Small farmer (2 cells) 500 m (all) Behavior is non-homogeneous in space and time Statistics and agents Express anisotropy y=a0 + a1x1 + a2x2 + ... +aixi +E Multi-scale modelling Phase transitions latency > 6 years Newly implanted Year of creation Deforesting Deforestation > 80% Slowing down Iddle Deforestation = 100% Tools for observing simulations TerraME functionality TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax TerraLib: spatio-temporal database as a basis for innovation G. Câmara et al.“TerraLib: An open-source GIS library for large-scale environmental and socio-economic applications”. In: B. Hall, M. Leahy (eds.), “Open Source Approaches to Spatial Data Handling”. Berlin, Springer, 2008. Visualization (TerraView) Modelling (TerraME) Spatio-temporal Database (TerraLib) Statistics (aRT) Data Mining(GeoDMA) TerraME: Software Architecture Model 1 Model 2 Model 3 Model 4 TerraML Language TerraMLCompiler TerraML Virtual Machine TerraME C++ Framework C++ Signal Processing librarys TerraLib C++ Mathematical librarys C++ Statistical librarys Where is Lua? Inside Brazil Petrobras, the Brazilian Oil Company Embratel (the main telecommunication TerraME Programming Language: Extension company of Lua in Brazil) Lua is the language of choice for computer games many other companies Outside Brazil Lua is used in hundreds of projects, both commercial and academic CGILua still in restricted use until recently all documentation was in Portuguese source: the Lua team Lua[Ierusalimschy and the Web et al, 1996] Lua Roberto Ierusalimschy PUC-Rio, Brazil What is Lua? Yet Another Scripting Language an “extension” language implemented as a library in ANSI C Host Program Lua Interpreter -- a Lua script color = RED b = button { label = ‘OK’, x = 10, y = 20} Why Lua? Simple and flexible “Simple things simple, complex things possible” Small, Efficient, Portable Whole library written in ANSI C, compiles the same source code in all platforms Typical uses: MS-DOS, Windows (3.1, 95, NT), Unix (Linux, Solaris, IRIX, AIX, ULTRIX), Next, OS/2, Mac How is Lua? Pascal-like Syntax. function fat (n) if n == 0 then return 1 else return n*fat(n-1) end end Interpreter executes sequence of statements. function definitions are also statements (see later) Six types: numbers, tables, functions, strings, userdata, nil Variables and Values Case sensitive semicolon may optionally follow any statement a = 1 b = a*2 print(a) print(b) Comments double hyphen (--) until the end of the line. block comments start with --[[ and run until ]] print("hello") -- my comment -- print("hello”) --[[ print(10) -- no action (comment) --]] My first Lua program C = 2 -- rain/t K = 0.4 -- flow coefficient q = 0 -- RULES for time = 0, 20, 1 do -- soil water q = q + C - K*q end print(“q = "..q) Types Type nil Different from everything else Default variable type Also acts as false (boolean) Type boolean Comparison value if (rain == true) then .... boolean false/true nil and false are false, everything else is true zero and the empty string are true operators and, or, and not print(true and false) print(true and (false or true)) print(false or (true and false) or (true and true)) number the only type for numeric values double-precision floating-point number arithmetic operators: +, –, *, / exponent (^) and modulus (%) boolean operators (<, >, <=, >=, ~=, and ==) A = 6 + a = A ^ b = A % print(a 2.2 * 4e+3 2 7 > b) print(b ~= 2) Parentheses Always optional (except in the case of function call) When in doubt, use parentheses a+-i < b/2+1 <--> (a + (-i)) < ((b/2)+1) 5+x^2*8 <--> a < y and y <= z <--> (a < y) and (y <= z) –x^y^z <--> –(x^(y^z)) 5 + ( (x^2)*8 ) Type string Immutable No size limit (read large files as strings) No termination value (‘\0’) Powerful Pattern-matching in standard library myname = “Werner Kuhn”; if statement An if statement tests condition and executes its then-part or its else-part (optional) accordingly a = 6; b = 5 if a < b then print("a < b") elseif a < b + 5 then print("b <= a < b+5") else print("a > b+5") end for statement for var = exp1, exp2, exp3 do something end Execute something for each value of var from exp1 to exp2, using exp3 as the step to increment var. This third expression is optional (default is 1). for i = 1, 10 do print(i) end for i = 1, 10, 2 do print(i) end Tables Implement associative arrays: any value (including functions and other tables) can be used both for indices and values t = {} t[1] = "hello" t.x = print t.x(t[1]) t.next = t -- creates an empty table -- t.x is sugar for t[‘x’] -- prints ‘hello’ -- circular list table Tables can be indexed not only with numbers, but also with strings or any other value of the language, except nil loc = { cover = "forest", distRoad = 0.3, distUrban = 2 } print(loc["cover"]) print(loc.cover) loc.distRoad = loc.distRoad^2 loc.distTotal = loc.distRoad + loc.distUrban loc.deforestationPot = 1/loc.distTotal Tables within tables loc = { cover = "forest", dist = {road = 0.3, urban = 2} } print(loc.dist.road) loc.dist.total = loc.dist.road + loc.dist.urban print(loc.dist.total) Constructors: Create and init tables Record style point={x=10,y=20} print(point.y) --> 20 List style days={"Sun","Mon","Tue","Wed”, Sat"} print(days[3]) --> Tue Mixed style points={{x=0,y=0}, point, n=2} print(points[points.n].y) --> 20 Constructors calls function “article” article{ author="F.P.Brooks", title="The Mythical Man-Month", year=1975, } news = { {text = "New version 2.0", date = "21/05/1997"}, {text = "New example", date = "21/05/1997"}, {text = "New version: 2.1",date = "17/06/1997"}, } Lua and the Web function A function can carry out a specific task (commonly called procedure) or compute and return values. A function is a first-class value in Lua. Functions can be stored in variables and in tables, can be passed as arguments, and can be returned by other functions, giving great flexibility to the language. myprint = print print = nil myprint(2) print = myprint Functions in Lua function fat (n) if n == 0 then return 1 else return n*fat(n-1) end end Higher-order Functions Functions can also be parameters to other functions. This kind of function is what we call a higher-order function. function foreach(tab, func) for position, value in pairs(tab) do func(value, position) end end x = {7, 3, 2, 6, 4} foreach(x, function(element) print(element) end) foreach(x, function(value, position) print(position, value) end) Functions in Lua First class values function inc (x) return x+1 end sugar inc = function (x) return x+1 end Example: cloning a table t clone = {} foreach(t, function (i,e) clone[i]=e end) Lua and the Web Functions and Tables w = { redraw = function () ... end, pick = function (x,y) ... end, } if w.pick(x,y) then w.redraw() end Tables with functions Tables may have their own functions. loc = { cover = "forest", distRoad = 0.3, distUrban = 2, deforestPot = function(myloc) return 1/(myloc.distRoad + myloc.distUrban) end } print(loc.deforestPot(loc)) print(loc:deforestPot()) Tables with functions We can declare a “class” in Lua by creating a function that takes a table constructor as argument. function MyLocation(locdata) locdata.covertype = "forest" locdata.deforPot = function(self) return 1/(self.distRoad + self.distUrban) end return locdata end loc = MyLocation({distRoad = 0.3, distUrban = 2}) loc = MyLocation{distRoad = 0.3, distUrban = 2} print(loc.covertype) print(loc:deforPot()) Tables x Objects Tables are dynamically created objects. list = {value=v, next=list} list old list ... value - v next - Objects First-class functions+ tables = almost OO Tables can have functions as fields Sugar for method definition and call Implicit parameter self function a:foo (x) ... end a:foo(x) sugar a.foo = function (self,x) ... end sugar a.foo(a,x) My second Lua program C = 2; -- rain/t K = 0.4; -- flow coefficient q = 0; -function rain (t) if (t < 10) then return 4 – 4*math.cos(math.pi*t/10); else return 4 – 4*math.cos(math.pi*(t-10)/10); end end -for time = 0, 20, 1 do -- soil water q = q + rain(time) - K*q; end -- report print(“q = "..q); Standard libraries Basic String Table Math IO OS Debug Coroutine TerraME: Vision Nature: represented by a cellular space Society: represented by agents Several interacting entities share the same spatiotemporal structure. rain rain rain Itacolomi do Itambé Peak Lobo’s Range My third Lua program Define a two-dimensional grid Make it rain on the grid Let water flow downwards N TerraME Runtime Environment TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax TerraME allows nested scales Nested scales are necessary for humanenvironment models Diverse space partitions can have different scales Cellular Space A geographical area of interest, divided into a grid. Each cell in the grid has one or more attributes. Stored and retrieved from a TerraLib database Loading Data -- Loads the TerraLib cellular space csCabecaDeBoi = CellularSpace { dbType = "ADO", host = “localhost", database = "c:\\cabecaDeBoi.mdb", user = "", password = "", layer = "cellsLobo90x90", theme = "cells", select = { “height", “soilWater", “capInf" } } csCabecaDeBoi:load(); csCabecaDeBoi:loadMooreNeighbourhood; GIS Creating temporary cellular spaces game = CellularSpace { xdim = N, ydim = N } Referencing cells A CellularSpace has a special attribute called cells. It is a one-dimensional table of references for each Cell in the CellularSpace -- c is the seventh cell in the cellular space c = csCabecaDeBoi.cells[ 7 ]; -- Updating the attribute “ infcap ” from the seventh cell c.infcap = 10; print (csCabecaDeBoi.cells[7].infCap); Database management -- loads a cellular space csAmazonia:load() csAmazonia:loadNeighbourhood("Moore") -- save (time, themeName, attrTableName) -for time = 1, 10,1 do csAmazonia:save(time, “sim", {"water"}) end TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax The Cell type A Cell value has two special attributes: latency and past. The latency attribute registers the period of time since the last change in a cell attribute value. The past attribute is a copy of all cell attribute values in the instant of the last change. if(cell.cover == "abandon" and cell.latency >= 10) then cell.cover = "secFor" end cell.water = cell.past.water + 2 Traversing a Cell Space forEachCell(cs, function()) Applies the chosen function to each cell of the cellular space. This function enables using different rules in a cellular space. forEachCell(csQ, function(cell) cell.Water = cell.past.Water + 2 return true end ) Isotropic neighbourhoods in cell spaces Von Neumann Neighborhood Moore Neighborhood Traversing a Neighbourhood csq:loadNeighbourhood(“Moore”); forEachCell(csQ, function(cell) count = 0; forEachNeighbour(cell, 0, function(cell, neigh) if (neigh.past.value == 1 and neigh ~= cell) then count = count + 1; end end; ); -- for each neighbor Synchronizing a cell space tn tn+1 rule count = 0 ; for i, cell ipairs( csValeDoAnary ) do if ( cell.past.sim_cover == 1 ) then cell.sim_cover = 0; count = count + 1 ; end end cell.synchronize( ); ? print(“Number of deforested cells: ”.. count); Synchronizing a cell space tn tn+1 rule TerraME keeps two copies of a cellular space in memory: one stores the past values of the cell attributes, and another stores the current (present) values of the cell attributes. The model equations must read (the right side of the equation rules) the past copy, and must write (the left side of the equation rules) the values to the present copy of the cellular space. At the correct moment, it will be necessary to synchronize the two copies of the cellular space, copying the current attribute values to the past copy of the cellular space Synchronization Always read from the past Always write to the present …. csQ:syncronize(); Trajectories: spatial patterns of change modeller defined functions which map indexes (atributtes) to geo-objects (cells). it = Trajectory{ myCellSpace, function(cell) return cell.cover == "forest“ end, function( c1, c2 ) return c1.dist_roads < c2.dist_roads end } Which objects are nearest to each other? Using Generalized Proximity Matrices (GPM) Consolidated area Emergent area TerraME neighborhoods are graphs Euclidean space Open network Closed network D1 D2 [Aguiar et al., 2003] Create or load neighborhoods -- Create a Moore neighborhood createMooreNeighborhood( myCellSpace, “neighName” ) -- Create a 3x3 neighborhood create3x3Neighborhood(myCellSpace, filterF() , weightF(), name ) -- Create a MxN neighborhood createMxNNeighborhood( M, N, myCellSpace,filterF(), weightF(), name ) -- Load neighborhood from TerraLib database myCellSpace: loadTerraLibGPM(“myGPM"); -- Load neighborhood from TerraLib GAL files myCellSpace:loadGALNeighborhood("c:\\myNeigh.gal") Building neighborhoods between cell spaces spatialCoupling( M, N, cs1,cs2, filterF, weightF, name ) filterF(cell, neigh) Boolean wheighF(cell, neigh) Real Example: neighborhood to simulate rain -- Creates a 3x3 Neighborhood based on the cell "slope" -- only lower neighbors are considered create3x3Neighborhood( csQ, function(cell,neigh) return neigh.altimetry < cell.altimetry end, function(cell, neigh) return (cell.altimetry - neigh.altimetry)/ (cell.altimetry + neigh.altimetry) end, "slope" ); TerraME integration with GIS (TerraView) “GPM” Plugin TerraView 3.2.0 “FillCell” Plugin TerraView 3.2.0 TerraLib Database Conversion from GIS data to cell spaces Real world Vector geospatial data Cell space The mixed pixel problem How can you transform from vectors to cell attributes? Fill the attributes of the cell spaces For each data type to be transformed, there are appropriate operations Using “FillCell” plugin to build Cell Spaces 1. Install the FillCell plugin: Copy the file "celulas.dll" to the directory "C: \ Program Files \ TerraView3.2.0 \ plugins". 2. Build the cell space with the desired resolution Filling Cells from vector data Numerical areas Categorical areas Lines (polygons, cells) (polygons, cells) points Min, max, average, sum, standard dev Majority class (by number or by area) Percentage of each class, Percentage of majority class, area of majority class Average/Sum intersectionweighted Presence, minimum distance, count and rain rain Itacolomi do Itambé Peak N rain Lobo’s Range Picture direction Itacolomi do Itambé Peak Lobo’s Range Demo: Rain Drainage Model Database: c:\\TerraME\\Database\\CabecadeBoi.mdb Model: c:\\TerraME\\Modelos\\demo4_chuva_geoBD.lua Model: c:\\TerraME\\Modelos\\demo7_chuva_geoBD.lua Simulation Result (36 min.) Demo: Fire propagation Database: c:\\TerraME\\Database\\db_emas.mdb Model: c:\\TerraME\\Modelos\\demo6_FireSpreadModel.lua INERTE QUEIMANDO CA 1 CA 2 CA 3 CA 4 CA 5 CA 1 0.100 0.250 0.261 0.273 0.285 CA 2 0.113 0.253 0.264 0.276 0.288 CA 3 0.116 0.256 0.267 0.279 0.291 CA 4 0.119 0.259 0.270 0.282 0.294 CA 5 0.122 0.262 0.273 0.285 0.297 Demo: Amazon deforestation Database: c:\\TerraME\\Database\\amazonia.mdb Model: c:\\TerraME\\Modelos\\demo3_desflorestamento_save.lua References Carneiro, T., 2006. Nested-CA: a foundation for multiscale modeling of land use and land change., in PhD Thesis in Computer Science. National Institute of Space Research: São José dos Campos, Brazil. Carneiro, T.; Câmara, G., 2007. A Gentle Introduction to TerraME. INPE Report, 2007. Ierusalimschy, R. 2006. Programming in Lua (2nd edition). Rio de Janeiro, Lua.Org.