Spatial Dynamical Modeling with TerraME Tiago Carneiro Gilberto Câmara Dynamic Spatial Models f (It) f (It+1) F f (It+2) f ( It+n ) F .. “A dynamical spatial model is a computational representation of a real-world process where a location on the earth’s surface changes in response to variations on external and internal dynamics on the landscape” (Peter Burrough) Computational Modelling with Cell Spaces Cell Spaces Representation Cell Spaces Generalized Proximity Matriz – GPM Hybrid Automata model Nested scales TerraME - overview Model data in cell spaces Read/write data from a database Cellular Data Base Resolution 2500 m 2.500 m e 500 m TerraME functionality TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax TerraME: Software Architecture Model 1 Model 2 Model 3 Model 4 TerraML Language TerraMLCompiler TerraML Virtual Machine TerraME C++ Framework C++ Signal Processing librarys TerraLib C++ Mathematical librarys C++ Statistical librarys TerraLib: the support for TerraME Open source library for GIS Data management object-relational DBMS raster + vector geometries ORACLE, Postgres, mySQL, Access Environment for customized GIS applications Web-based cooperative development http://www.terralib.org TerraME integration with GIS (TerraView) “GPM” Plugin TerraView 3.2.0 “FillCell” Plugin TerraView 3.2.0 TerraLib Databse Conversion from GIS data to cell spaces Real world Vector geospatial data Cell space The mixed pixel problem How can you transform from vectors to cell attributes? Using “FillCell” plugin to build Cell Spaces 1. Install the FillCell plugin: Copy the file "celulas.dll" to the directory "C: \ Program Files \ TerraView3.2.0 \ plugins". 2. Build the cell space with the desired resolution Fill the attributes of the cell spaces For each data type to be transformed, there are appropriate operations Filling Cells from vector data Numerical areas Categorical areas Lines (polygons, cells) (polygons, cells) points Min, max, average, sum, standard dev Majority class (by number or by area) Percentage of each class, Percentage of majority class, area of majority class Average/Sum intersectionweighted Presence, minimum distance, count and Lua Roberto Ierusalimschy PUC-Rio, Brazil Lua and the Web What is Lua? Yet Another Scripting Language an “extension” language implemented as a library in ANSI C Host Program Lua Interpreter -- a Lua script color = RED b = button { label = ‘OK’, x = 10, y = 20} Lua and the Web Why Lua? Simple and flexible “Simple things simple, complex things possible” Small, Efficient, Portable Whole library written in ANSI C, compiles the same source code in all platforms Typical uses: MS-DOS, Windows (3.1, 95, NT), Unix (Linux, Solaris, IRIX, AIX, ULTRIX), Next, OS/2, Mac Lua and the Web Where is Lua? Inside Brazil Petrobras, the Brazilian Oil Company Embratel (the main telecommunication company in Brazil) many other companies Outside Brazil Lua is used in hundreds of projects, both commercial and academic CGILua still in restricted use » until recently all documentation was in Portuguese Lua and the Web How is Lua? Pascal-like Syntax. function fat (n) if n == 0 then return 1 else return n*fat(n-1) end end Interpreter executes sequence of statements. function definitions are also statements (see later) Six types: numbers, tables, functions, strings, userdata, nil Lua and the Web My first Lua program C = 2; -- rain/t K = 0.4; -- flow coefficient q = 0; -- RULES for time = 0, 20, 1 do -- soil water q = q + C - K*q; end print(“q = "..q); Types Type nil Different from everything else Default variable type Also acts as false (boolean) Type boolean Comparison value if (rain == true) then .... Type number Unique native type for numbers double (by default) a = 3 b = 3.5 c = 4.5e-8 Type string Immutable No size limit (read large files as strings) No termination value (‘\0’) Powerful Pattern-matching in standard library myname = “Werner Kuhn”; Tables Implement associative arrays: any value (including functions and other tables) can be used both for indices and values t = {} t[1] = "hello" t.x = print t.x(t[1]) t.next = t -- creates an empty table -- t.x is sugar for t[‘x’] -- prints ‘hello’ -- circular list Lua and the Web Constructors Expressions to create and initialize tables Record style point={x=10,y=20} print(point.y) --> 20 List style days={"Sun","Mon","Tue","Wed","Thu","Fri", "Sat"} print(days[3]) --> Tue Mixed style points={{x=0,y=0}, point, n=2} Lua and the Web Table loc = { cover = "forest", distRoad = 0.3, distUrban = 2 }; loc.cover = “cerrado”; loc[“cover”] = “soja”; if (loc.distUrban > 1.5) then Tables in Lua loc = { cover = "forest", distRoad = 0.3, distUrban = 2 }; loc.desfPot = loc.distUrban; loc.distRoad + Tables em Lua : functions loc = { cover = "forest", distRoad = 0.3, distUrban = 2 }; ... loc.reset = function( self ) self.cover = ""; self.distRoad = 0.0; self.distUrban = 0.0; end Constructors calls function “article” article{ author="F.P.Brooks", title="The Mythical Man-Month", year=1975, } news = { {text = "New version 2.0", date = "21/05/1997"}, {text = "New example", date = "21/05/1997"}, {text = "New version: 2.1",date = "17/06/1997"}, } Lua and the Web Functions in Lua function fat (n) if n == 0 then return 1 else return n*fat(n-1) end end Functions in Lua First class values function inc (x) return x+1 end sugar inc = function (x) return x+1 end Example: cloning a table t clone = {} foreach(t, function (i,e) clone[i]=e end) Lua and the Web Upvalues Mechanism to allow functions to access non-local variables An upvalue is a variable expression whose value is computed when the enclosing function is instantiated (and not when the function is executed) function add (x) return function (y) return y+%x end end add1 = add(1) print(add1(10)) --> 11 upvalue Lua and the Web Functions and Tables w = { redraw = function () ... end, pick = function (x,y) ... end, } if w.pick(x,y) then w.redraw() end Tables x Objects Tables are dynamically created objects. in the sense of Hoare list = {value=v, next=list} list old list ... value - v next Lua and the Web Objects First-class functions+ tables = almost OO Tables can have functions as fields Sugar for method definition and call Implicit parameter self function a:foo (x) ... end a:foo(x) sugar a.foo = function (self,x) ... end sugar a.foo(a,x) My second Lua program C = 2; -- rain/t K = 0.4; -- flow coefficient q = 0; -function rain (t) if (t < 10) then return 4 – 4*math.cos(math.pi*t/10); else return 4 – 4*math.cos(math.pi*(t-10)/10); end end -for time = 0, 20, 1 do -- soil water q = q + rain(time) - K*q; end -- report print(“q = "..q); Standard libraries Basic String Table Math IO OS Debug Coroutine Basic Basic functions print type setmetatable pairs String String manipulation pattern matching string.find string.gsub Table Function for table manipulation table.insert table.remove table.sort rain rain rain Itacolomi do Itambé Peak Lobo’s Range My third Lua program Define a two-dimensional grid Make it rain on the grid Let water flow downwards N TerraME: Vision An Earth´s environment can be represented as a synthetic environment where analytical entities (rules) change the space properties in time. Several interacting entities share the same spatiotemporal structure. TerraME architecture & applications RondôniaModel DinamicaModel TROLLModel CLUEModel TerraME Language TerraME Compiler TerraME Virtual Machine TerraLib Enviromental Modeling Framework C++ Signal Processing librarys TerraLib C++ Mathematical librarys C++ Statistical librarys TerraME Runtime Environment TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax The Scale Concept in TerraME Scale is a generic concept that includes the spatial, temporal, or analytical dimensions used to measure any phenomenon. Extent refers to the magnitude of measurement. Resolution refers to the granularity used in the measures. (Gibson et al. 2000) TerraME allows nested scales Nested scales are necessary for humanenvironmental models Diverse space partitions can have different scales TerraME extensions to Lua To build spatial dynamic models, TerraME includes new value types in LUA using the constructor mechanism. These values are: CellularSpace, Cell, Neighbourhood Cellular Space A geographical area of interest, divided into a grid. Each cell in the grid has one or more attributes. CellularSpaces are stored and retrieved from a TerraLib database Loading Data -- Loads the TerraLib cellular space csCabecaDeBoi = CellularSpace { dbType = "ADO", host = “localhost", database = "c:\\cabecaDeBoi.mdb", user = "", password = "", layer = "cellsLobo90x90", theme = "cells", select = { “height", “soilWater", “capInf" } } csCabecaDeBoi:load(); csCabecaDeBoi:loadMooreNeighbourhood; GIS Creating temporary cellular spaces myCellSpace = CellularSpace{ database = "", theme = "“ } for i = 1, 2, 1 do for j = 1, 2, 1 do c = Cell{ soilType = “latosolo” } c.x = i; c.y = j; myCellSpace :add( c ); end end Referencing cells A CellularSpace has a special attribute called cells. It is a one-dimensional table of references for each Cell in the CellularSpace -- c is the seventh cell in the cellular space c = csCabecaDeBoi.cells[ 7 ]; -- Updating the attribute “infcap” from the seventh cell c.infcap = 10; print (csCabecaDeBoi.cells[7].infCap); Database management -- loads a cellular space csAmazonia:load(); csAmazonia:loadNeighbourhood("Moore"); -- save (time, themeName, attrTableName) -for time = 1, 10,1 do csAmazonia:save(time, “sim", {"water"}); end TerraME INTERPRETER • model syntax semantic checking • model execution TerraView • data acquisition • data visualization • data management • data analysis LUA interpreter TerraME framework data model model TerraME/LUA interface MODEL DATA Model source code TerraLib database data Eclipse & LUA plugin • model description • model highlight syntax The Cell type A Cell value has two special attributes: latency and past. The latency attribute registers the period of time since the last change in a cell attribute value. The past attribute is a copy of all cell attribute values in the instant of the last change. if(cell.cover == "abandoned" and cell.latency >= 10 ) then cell.cover = "secFor"; end cell.water = cell.past.water + 2; Traversing a Cell Space "for...end" statement: "for i, cell in pairs (csQ.cells) do ...end”. The i and cell variable in the statement are the index and the value of a cell inside the cells attribute from the cellular space csQ. for i, cell in pairs( csQ.cells ) do cell.water = cell.past.water + 2; end Traversing a Cell Space forEachCell(cs, function()) Applies the chosen function to each cell of the cellular space. This function enables using different rules in a cellular space. forEachCell(csQ, function(cell) cell.Water = cell.past.Water + 2; return true; end); Isotropic neighbourhoods in cell spaces Von Neumann Neighborhood Moore Neighborhood Traversing a Neighbourhood csq:loadNeighbourhood(“Moore”); forEachCell(csQ, function(cell) count = 0; forEachNeighbour(cell, 0, function(cell, neigh) if (neigh.past.value == 1 and neigh ~= cell) then count = count + 1; end end; ); -- for each neighbor Synchronizing a cell space tn tn+1 rule count = 0 ; for i, cell ipairs( csValeDoAnary ) do if ( cell.past.sim_cover == 1 ) then cell.sim_cover = 0; count = count + 1 ; end end cell.synchronize( ); ? print(“Number of deforested cells: ”.. count); Synchronizing a cell space TerraME keeps two copies of a cellular space in memory: one stores the past values of the cell attributes, and another stores the current (present) values of the cell attributes. The model equations must read (the right side of the equation rules) the past copy, and must write (the left side of the equation rules) the values to the present copy of the cellular space. At the correct moment, it will be necessary to synchronize the two copies of the cellular space, copying the current attribute values to the past copy of the cellular space. Synchronizing a cell space tn tn+1 rule TerraME keeps two copies of a cellular space in memory: one stores the past values of the cell attributes, and another stores the current (present) values of the cell attributes. The model equations must read (the right side of the equation rules) the past copy, and must write (the left side of the equation rules) the values to the present copy of the cellular space. At the correct moment, it will be necessary to synchronize the two copies of the cellular space, copying the current attribute values to the past copy of the cellular space Synchronization Always read from the past Always write to the present …. csQ:syncronize(); Trajectories: spatial patterns of change modeller defined functions which map indexes (atributtes) to geo-objects (cells). it = Trajectory{ myCellSpace, function(cell) return cell.cover == "forest“ end, function( c1, c2 ) return c1.dist_roads < c2.dist_roads end } Quais objetos são mais proximos? Which objects are NEAR each other? Using Generalized Proximity Matrices (GPM) Consolidated area Emergent area TerraME neighborhoods are graphs Euclidean space Open network Closed network D1 D2 [Aguiar et al., 2003] Create or load neighborhoods -- Create a Moore neighborhood createMooreNeighborhood( myCellSpace, “neighName” ) -- Create a 3x3 neighborhood create3x3Neighborhood(myCellSpace, filterF() , weightF(), name ) -- Create a MxN neighborhood createMxNNeighborhood( M, N, myCellSpace,filterF(), weightF(), name ) -- Load neighborhood from TerraLib database myCellSpace: loadTerraLibGPM(“myGPM"); -- Load neighborhood from TerraLib GAL files myCellSpace:loadGALNeighborhood("c:\\myNeigh.gal") Building neighborhoods between cell spaces spatialCoupling( M, N, cs1,cs2, filterF, weightF, name ) filterF(cell, neigh) Boolean wheighF(cell, neigh) Real Example: neighborhood to simulate rain -- Creates a 3x3 Neighborhood based on the cell "slope" -- only lower neighbors are considered create3x3Neighborhood( csQ, function(cell,neigh) return neigh.altimetry < cell.altimetry end, function(cell, neigh) return (cell.altimetry - neigh.altimetry)/ (cell.altimetry + neigh.altimetry) end, "slope" ); Saving cell spaces as images -- attribute used to generate the image attr_name = "estado" -- values that the attribute can have attr_value = {0,1,2} -- color pallete attr_color = {{0,255,0},{255,0,0},{0,0,0}} -- directory where images will be saved path = "c:\\TerraME\\Results“ -- size of the cell in pixels cellSize = 2 -- load the o espaco celular do banco de dados TerraLib ........................ for t = 1, 100 do CStoPNG(myCellSpace, attr_name,t,path,cellSize,attr_value,attr_color) end rain rain Itacolomi do Itambé Peak N rain Lobo’s Range Picture direction Itacolomi do Itambé Peak Lobo’s Range Demo: Rain Drainage Model Database: c:\\TerraME\\Database\\CabecadeBoi.mdb Model: c:\\TerraME\\Modelos\\demo4_chuva_geoBD.lua Model: c:\\TerraME\\Modelos\\demo7_chuva_geoBD.lua Simulation Result (36 min.) Demo: Fire propagation Database: c:\\TerraME\\Database\\db_emas.mdb Model: c:\\TerraME\\Modelos\\demo6_FireSpreadModel.lua INERTE QUEIMANDO CA 1 CA 2 CA 3 CA 4 CA 5 CA 1 0.100 0.250 0.261 0.273 0.285 CA 2 0.113 0.253 0.264 0.276 0.288 CA 3 0.116 0.256 0.267 0.279 0.291 CA 4 0.119 0.259 0.270 0.282 0.294 CA 5 0.122 0.262 0.273 0.285 0.297 Demo: Desflorestamento na Amazônia Banco de dados: c:\\TerraME\\Database\\amazonia.mdb Modelo: c:\\TerraME\\Modelos\\demo3_desflorestamento_save.lua References Carneiro, T., 2006. Nested-CA: a foundation for multiscale modeling of land use and land change., in PhD Thesis in Computer Science. National Institute of Space Research: São José dos Campos, Brazil. Carneiro, T.; Câmara, G., 2007. A Gentle Introduction to TerraME. INPE Report, 2007. Ierusalimschy, R. 2006. Programming in Lua (2nd edition). Rio de Janeiro, Lua.Org.