Future Directions of Map Analysis and GIS Modeling: …where we are headed and how we get there GIS Centroid Seminar — Colorado State University September 19, 2014 Basis “They who don’t know, don’t know they don’t know” There There is is a a “map-ematics” “map-ematics” that that extends extends traditional traditional math/stat math/stat concepts concepts and and procedures procedures for for the the quantitative quantitative analysis analysis of of map map variables variables (digital (digital maps) maps) Presentation Presentation Premise: Premise: …three major considerations will influence map analysis/modeling future development— Mathematical Framework, Data Structure and Educational Approach This PowerPoint with notes and online links to further reading is posted at www.innovativegis.com/basis/Present/CentroidCSU2014/ Presentation by Joseph K. Berry Adjunct Faculty in Natural Resources, Warner College of Natural Resources, Colorado State University Adjunct Faculty in Geosciences, Department of Geography, University of Denver Principal, Berry & Associates // Spatial Information Systems Email: jberry@innovativegis.com — Website: www.innovativegis.com/basis Mapping vs. Analyzing …GIS is a Technological (Processing Mapped Data) Tool involving — −Mapping that creates a spatial representation of an area −Display that generates visual renderings of a mapped area −Geo-query that searches for map locations having a specified classification, condition or characteristic …and an Analytical Tool involving — −Spatial Mathematics that applies scalar mathematical formulae to account for geometric positioning, scaling, measurement and transformations of mapped data −Spatial Analysis that investigates the contextual relationships within and among mapped data layers −Spatial Statistics that investigates the numerical relationships within and among mapped data layers “Analyze” “Map” (Descriptive Mapping) Geographic Information Systems (Prescriptive Modeling) (map and analyze) Remote Sensing Global Positioning System (locate and navigate) (Biotechnology) GPS/GIS/RS (measure and classify) (Nanotechnology) (Berry) GIS as a Technological Tool (Critical Technological Frontiers) 3D Integration -- highly detailed 3D city models provide entirely new perspectives and realism supporting greater intuitive interaction with maps and improved understanding of spatial context. Intelligent 3D Models through the combination of GIS, CAD and BIM (Building Information Management) enabled by greater interoperability of data formats will be extended to other applications, such as forestry and natural resources management thereby translating the “What and Where” information into a seamless whole. Real-Time GIS -- the ever expanding sensor web provides the foundation for real-time GIS. Emergency response and military geospatial intelligence offer “situational awareness” for fast and effective response. Real-time GIS-enabled instruments are being coupled with automated devices, such as water levee gates and earthquake warning broadcast systems, for near instantaneous response. Incorporating the multiple data streams from live video, ground sensors, tracking devices, drone aircraft and space-based instruments within a GIS framework enable both Intelligent Agents (automated devices) and decision makers to “see What is Happening Where” and respond events as they unfold. Temporal GIS -- Temporal GIS incorporates the X, Y and Z dimensions as well as time for a 4D representation that introduces the time element to geospatial data to account for changes over time (in-situ evolution as well as positioning displacement of moving elements). Inroads have been made to display time series animation of spatial data (such as the Doppler Radar Images of a moving storm front), but the move toward fully integrated and comprehensive temporal GIS is hindered by the enormous amount of data needed to bring about the vision. However, the algorithms and methods for basic Change Detection in multiple images of the same scene taken at different times has greatly matured but need to be easier to use and more intuitive. Tomorrow’s Cyber Farmer (Precision Agriculture/Conservation example) …to pull off these amazing technological feats, significant advances in Map Analysis/Modeling capabilities and acceptance are needed. However, for the most part, there’s a big disconnect between maps as interface and the rich understanding that can be had by thinking “map-ematically”. Three major considerations will influence map analysis/modeling future development— Mathematical Framework, Data Structure and Educational Approach. …based on Matt Ball's blog at http://www.sensysmag.com/spatialsustain/what-are-some-of-the-technological-frontiers-for-gis-advancement.html (Berry) A Mathematical Structure for Map Analysis/Modeling Technological Tool Mapping/Geo-Query Geotechnology (Discrete, Spatial Objects) RS – GIS – GPS (Continuous, Map Surfaces) Analytical Tool Map Analysis/Modeling Geo-registered Analysis Frame Matrix Map Stack “Map-ematics” of Numbers Maps as Data, not Pictures Vector & Raster — Aggregated & Disaggregated Qualitative & Quantitative …organized set of numbers Grid-based Spatial Analysis Operations Map Analysis Toolbox Spatial Statistics Operations A Map-ematical Framework Traditional math/stat procedures can be extended into geographic space to support Quantitative Analysis of Mapped Data “…thinking analytically with maps” Esri Spatial Analyst operations …over 170 individual “tools” www.innovativegis.com/basis/BeyondMappingSeries/, Book IV, Topic 9 for more discussion (Berry) Spatial Analysis Operations (Geographic Context) GIS as “Technical Tool” (Where is What) vs. “Analytical Tool” (Why, So What and What if) Grid Layer Map Stack Spatial Analysis extends the basic set of discrete map features (points, lines and polygons) to map surfaces that represent continuous geographic space as a set of contiguous grid cells (matrix), thereby providing a Mathematical Framework for map analysis and modeling of the Contextual Spatial Relationships within and among grid map layers Map Analysis Toolbox Unique spatial operations Mathematical Perspective: Basic GridMath & Map Algebra ( + - * / ) Advanced GridMath (Math, Trig, Logical Functions) Map Calculus (Spatial Derivative, Spatial Integral) Map Geometry (Euclidian Proximity, Effective Proximity, Narrowness) Plane Geometry Connectivity (Optimal Path, Optimal Path Density) Solid Geometry Connectivity (Viewshed, Visual Exposure) Unique Map Analytics (Contiguity, Size/Shape/Integrity, Masking, Profile) (Berry) Spatial Analysis Operations (Math Examples) Advanced Grid Math — Math, Trig, Logical Functions Map Calculus — Spatial Derivative, Spatial Integral Spatial Derivative MapSurface 2500’ …is equivalent to the slope of the tangent plane at a location Slope draped over MapSurface 500’ Surface Fitted Plane 65% SLOPE MapSurface Fitted FOR MapSurface_slope 0% Curve The derivative is the instantaneous “rate of change” of a function and is equivalent to the slope of the tangent line at a point Dzxy Elevation ʃ Districts_Average Elevation Spatial Integral Advanced Grid Math …summarizes the values on a surface for specified map areas (Total= volume under the surface) Surface Area S_Area= Fn(Slope) …increases with increasing inclination as a Trig function of the cosine of the slope angle COMPOSITE Districts WITH MapSurface Average FOR MapSurface_Davg MapSurface_Davg S_area= cellsize / cos(Dzxy Elevation) The integral calculates the area under the curve for any section of a function. Surface Curve (Berry) Spatial Analysis Operations (Distance Examples) 96.0 minutes Map Geometry — (Euclidian Proximity, Effective Proximity, Narrowness) Plane Geometry Connectivity — (Optimal Path, Optimal Path Density) Solid Geometry Connectivity — (Viewshed, Visual Exposure) Distance Euclidean Proximity …farthest away by truck, ATV and hiking Effective Proximity Off Road Relative Barriers HQ (start) On Road 26.5 minutes Off Road Absolute Barrier …farthest away by truck On + Off Road Travel-Time Surface Farthest (end) Shortest straight line between two points… …from a point to everywhere… …not necessarily straight lines (movement) Connectivity HQ Truck = 18.8 min ATV = 14.8 min Hiking = 62.4 min (start) …like a raindrop, the “steepest downhill path” identifies the optimal route Solid Geometry Connectivity Rise Run Plane Geometry Visual Exposure (Quickest Path) Tan = Rise/Run Seen if new tangent exceeds all previous tangents along the line of sight Counts # Viewers Sums Viewer Weights Splash 270/621= 43% of the entire Viewshed road network is connected Highest Weighted Exposure (Berry) Spatial Statistics Operations (Numeric Context) GIS as “Technical Tool” (Where is What) vs. “Analytical Tool” (Why, So What and What if) Grid Layer Map Stack Spatial Statistics seeks to map the variation in a data set instead of focusing on a single typical response (central tendency), thereby providing a Statistical Framework for map analysis and modeling of the Numerical Spatial Relationships within and among grid map layers Map Analysis Toolbox Unique spatial operations Statistical Perspective: Basic Descriptive Statistics (Min, Max, Median, Mean, StDev, etc.) Basic Classification (Reclassify, Contouring, Normalization) Map Comparison (Joint Coincidence, Statistical Tests) Unique Map Statistics (Roving Window and Regional Summaries) Surface Modeling (Density Analysis, Spatial Interpolation) Advanced Classification (Map Similarity, Maximum Likelihood, Clustering) Predictive Statistics (Map Correlation/Regression, Data Mining Engines) (Berry) Spatial Statistics (Linking Data Space with Geographic Space) Roving Window (weighted average) Geo-registered Sample Data Spatial Distribution Spatial Statistics Discrete Sample Map Non-Spatial Statistics Continuous Map Surface Surface Modeling techniques are used to derive a continuous map surface from discrete point data– fits a Surface to the data (maps the variation). Standard Normal Curve Average = 22.6 In Geographic Space, the typical value forms a horizontal plane implying the average is everywhere StDev = 26.2 Histogram (48.8) 10 20 30 40 50 Numeric Distribution (Berry) X + 1StDev = 22.6 + 26.2 = In Data Space, a standard normal curve can be fitted to the data to identify the “typical value” (average) 0 …lots of NE locations exceed Mean + 1Stdev 60 70 80 Unusually high values X= 22.6 +StDev Average 48.8 Spatial Statistics Operations (Data Mining Examples) Map Clustering: Elevation vs. Slope Scatterplot “data pair” of map values “data pair” plots here in… Cluster 2 Data Space …as similar as can be WITHIN a cluster …and as different as can be BETWEEN clusters Elevation Geographic Space (Feet) Slope + Slope (Percent) Slope draped on Elevation Elev X axis = Elevation (0-100 Normalized) Y axis = Slope (0-100 Normalized) Advanced Classification (Clustering) Map Correlation: + Cluster 1 Geographic Space Data Space Spatially Aggregated Correlation Scalar Value – one value represents the overall non-spatial relationship between the two map surfaces Roving Window …1 large data table Entire Map Extent Elevation (Feet) Slope (Percent) with 25rows x 25 columns = 625 map values for map wide summary r= …where x = Elevation value and y = Slope value and n = number of value pairs …625 small data tables within 5 cell reach = 81map values for localized summary Localized Correlation Predictive Statistics (Correlation) (Berry) Map Variable – continuous quantitative surface represents the localized spatial relationship between the two map surfaces r = .432 Aggregated Map of the Correlation Grid-based Map Data Structure 90 (geo-registered matrix of map values) 2.50 Latitude/Longitude Grid (140mi grid cell size) Analysis Frame 300 (grid “cells”) Grid Lines Coordinate of first grid cell is 900 N 00 E The Latitude/Longitude grid forms a continuous surface for geographic referencing where each grid cell represents a given portion of the earth’ surface. The easiest way to conceptualize a grid map is as an Excel spreadsheet with each cell in the table corresponding to a Lat/Lon grid space (location) and each value in a cell representing the characteristic or condition (information) of a mapped variable occurring at that location. …the bottom line is that… All spatial topology is inherent in the grid. #Rows= 73 #Columns= 144 = 10,512 grid cells Conceptual Spreadsheet (73 x 144) Lat/Lon …each 2.50 grid cell is about 140mi x 140mi 18,735mi2 …maximum Lat/Lon decimal degree resolution is a four-inch square anywhere in the world …from Lat/Lon “crosshairs to grid cells” that contain map values indicating characteristics or conditions at each grid location (Berry) Universal Database Key …Spatially Keyed data in the cloud are downloaded and configured to the Analysis Frame defining the Map Stack (moving Lat/Lon from crosshairs to grid cells) Lat/Lon serves as a Universal dB Key Spatially Keyed for joining data tables based on location data in the cloud Conceptual Organization RDBMS Organization Spreadsheet 30m Elevation (99 columns x 99 rows) “Where” Each of the conceptual grid map spreadsheets (matrices) can be converted to interlaced RDBMS format with a long string of numbers forming the data field (map layer) and the records (values) identifying the information at each of the individual grid cell locations. Geographic Space Once a set of mapped data is stamped with its Lat/Lon “Spatial Key,” it can be linked to any other database table with spatially tagged records without the explicit storage of a fully expanded grid layer— all of the spatial relationships are implicit in the relative Lat/Lon positioning. Universal Spatial Key (Berry) Grid Space Wyoming’s Bighorn Mts. 2D Matrix 1D Field Database Table Keystone Concept Lat/Lon as a Data Space Each column (field) represents a single map layer with the values in the rows indicating the characteristic or condition at each grid cell location (record) “What” 5-step Process for Unlocking the Universal Spatial Db Key Spatially Aware Database Step 1. User identifies the geographic extent (XY, Value) of the analysis window. Step 2. User specifies the cell size of the analysis window. …e.g., 100m 3 9 What (Value) …value indicates characteristic or condition at a location Where (XY) 3 13 6 0 10 …Lat/Lon coordinates identify earth position of a dB record 1 Longitude Latitude Step 3. Computer determines the Lat/Lon ranges defining each grid cell (cutoffs) and the centroid location. …defines the Analysis Frame Step 4. Computer determines the appropriate grid cell for each database record that falls within the analysis frame’s geographic extent based on its Lat/Lon coordinates… then repeats for all selected dB records. Map Stack …but Lat/Lon grid cells are only square at the equator— so is the entire idea a bust? Hint: spatial resolution of the analysis frame is key Shish Kebab of numbers Step 5. Computer summarizes the values if more than one value “falls” into an individual grid cell-- result is a “Grid Map Layer” for inclusion in a map stack for subsequent map analysis. (Berry) GIS Development Cycle (…where we’re heading) Future Directions GIS Evolution Revisit Analytics (2020s) Future Directions 2D Planar 3D Solid (X,Y Data) (X,Y,Z Data) Cartesian Coordinates GeoWeb (2000s) Today Today Revisit Geo-reference (2010s) Square (4 sides) Cube (6 squares) Contemporary GIS Spatial dB Mgt (1980s) Map Analysis Future (1990s) Hexagon (6 sides) …about every decade Future Pentagonal Dodecahedral (12 pentagons) The Early Years Mapping focus Data/Structure focus Analysis focus Computer Mapping (1970s) (Berry) GIS Education (shifting the current Technical focus to a Pedagogical focus) The lion’s share of the growth has been GIS’s ever expanding capabilities as a “technical tool” for corralling vast amounts of spatial data and providing near instantaneous access to remote sensing images, GPS navigation, interactive maps, asset management records, geo-queries and awesome displays. However, GIS as an “analytical tool” hasn’t experienced the same meteoric rise— in fact it can be argued that the analytic side of GIS has somewhat stalled… partly because of… …but modern digital “maps are numbers first, pictures later” and we do mathematical and statistical things to map variables that moves GIS from— “Where is What” graphical inventories, to a “Why, So What and What If” problem solving environment— “thinking analytically with maps” (Berry) The “-ists” and the “-ologists” Together the “-ists” and the “-ologists” frame and develop the Solution for an application. The “-ists” — and — The “-ologists” …understand the “tools” that can be used to display, query and analyze spatial data …understand the “science” behind spatial relationships that can be used for decision-making Data and Information focus Knowledge and Wisdom focus Application Space Geotechnology’s Core “-ists” Technology Experts Solution Space “-ologists” Domain Experts GIS Expertise Spatial Reasoning Where is What Why, So What, What If (Berry) The “-ists” and the “-ologists” (toward a much bigger tent) …Decision Makers utilize the Solution… …under Stakeholder, Policy & Public auspices “Public” “Policy Makers” “Stakeholders” “Decision Makers” Application Space Geotechnology’s Core “-ists” Technology Experts Solution Space “-ologists” Domain Experts Spatial Reasoning GIS Expertise …and complicating GIS Technology We are simultaneously trivializing… (Berry) Conclusions/Upshot (moving away from your grandfather's map) Tomorrow’s GIS arena will be radically different from the past four decades… Computer Mapping (1970 to 1980) – automated the cartographic process where points, lines and areas (spatial objects) defining geographic features on a map are represented as an organized set of X,Y coordinates. Spatial Database Management Systems (1980 to 1990)– linked computer mapping capabilities with traditional database management capabilities by assigning an ID# to each spatial object that serves as a common database key between a spatial table (Where) and an attribute table (What). Map Analysis and GIS Modeling (1990 to 2000) – developed a comprehensive theory of map analysis where spatial information is represented numerically as continuous spatial distributions (raster), rather than in graphic fashion as discrete spatial objects (vector) identified by inked lines on a map. These digital maps are frequently conceptualized as a set of "floating maps" with a common registration, allowing the computer to "look" down and across the stack of digital maps to characterize spatial relationships of the mapped data that can be summarized (database queries) or mathematically manipulated (analytic processing). GeoWeb and Mobile Devices (2000 to 2010) – the Internet has moved maps and mapping from a “down the hall and to the right” specialist’s domain, to everyone’s desktop, notebook and mobile device. While the bulk of these applications involve navigation, mapping and geo-query (technological), they have fully established the digital map beachhead that sees “maps as data, not just images.” In the future, Geotechnology will fully exploit its numerical character by extending… 1) Scientific Use of Spatial Data – SpatialSTEM education will infuse science with “analytical tools” for unlocking radically new understandings of spatial patterns and relationships in their research. Solutions to Devices – “map-ematical solutions” will be directly tied to automated devices through continued coupling of RS, GIS, GPS and robotics. …next time Ag & NR considerations 3) Spatial Reasoning and Dialog – GIS models will enable decision-makers to interactively investigate and 2) Spatial better communicate “Why, So What and What If” of the probable spatial outcomes/impacts of critical decisions. …quantitative mapped data analysis and modeling completely changes our perspective of “what a map is (and isn’t)” See http://www.innovativegis.com/basis/BeyondMappingSeries/BeyondMapping_I/Epilog/BM_I_Epilog.htm for more discussion (Berry) So Where to Head from Here? Website (www.innovativegis.com) Online Materials (www.innovativegis.com/Basis/Courses/SpatialSTEM/) For more papers and presentations on Geotechnology ) www.innovativegis.com This PowerPoint with notes and online links to further reading is posted at www.innovativegis.com/basis/Present/CentroidCSU2014/ Beyond Mapping Compilation Series …nearly 1000 pages and more than 750 figures in the Series provide a comprehensive and longitudinal perspective of the underlying concepts, considerations, issues and evolutionary development of modern geotechnology (RS, GIS, GPS). eMail Contact Joseph K. Berry jberry@innovativegis.com