LiDAR Data Products in Indiana
Data Overview and Access
ISPLS Workshop
January 18th, 2013
Christopher Morse
NRCS Indiana GIS Coordinator
Special thanks to:
Jim Sparks
Phil Worrall
R. Wilkinson
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
A Short Review
Raster data (evenly spaced, gridded data)
Cells hold values for the height of a feature or site
referenced to a common vertical datum
Resolution refers to the size of the pixels in the data
A DEM with 30 meter resolution is composed of all cells
being 30 meters x 30 meters in the x and y directions and
each cell holds a single elevation value (z)
Elevation value (z) could be stored in a variety of units
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Definitions paraphrased from Maune et al, 2nd edition DEM Users Manual
(Terms often used interchangeably)
 DEM = Digital Elevation Model; Typically Bare Earth or terrain
 DTM = Digital Terrain Model; Similar to DEM with the addition of some
elevations for significant topographic features on the land defined by mass
points or break lines
 DSM = Digital Surface Model; Similar to a DEM or DTM, but shows the tops
of all surfaces including buildings, trees, and other features above the bare
earth
Image from State of Indiana Orthophotography and LiDAR
Program, presentation by R.N. Wilkinson
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Definitions paraphrased from Maune et al, 2nd
edition DEM Users Manual:
Mass Points are irregularly spaced vector points
with an x,y, and z value.
Breaklines are linear features that describe
changes in the terrain surface (roads, streams,
building footprints, etc…)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Common Resolutions, locally (not all resolutions
are available in all areas)
USGS primarily 30 meter, 10 meter; some 3
meter
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Common Resolutions, locally (not all resolutions
are available in all areas)
Indiana has 2005 elevation data at 5 foot
resolution for all parts of the state
Indiana is generating new DTM data for 20112013 at 5 foot resolution
Local governments may hold a variety of high
resolution products
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Why generate new DEMs at the same
resolution?
Funds
Dramatically Improved Vertical Accuracy
USGS National Elevation Data (2003) = 2.44 meters
RMSE
Indiana 2005 Data = 6 feet RMSE
Indiana 2011-2013 Data = 18.5 cm RMSE
Vertical Accuracy is a critical factor when
considering best supportable contour interval
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
What does that mean?
Best supportable auto-generated contours:
USGS National Elevation Data = over 20 foot interval
Indiana 2005 Data = 20 foot interval
Indiana 2011-2013 Data = 2 foot interval
Auto generated results limited without use of
breaklines
Image source: wikipedia.org/wiki/Contour_line
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Digital Elevation Data
Indiana’s new DEMs
Derived from LiDAR
Actually a DTM due to the inclusion of breaklines for
some hydro features
Rivers greater than 100 feet wide and water bodies of 2
acres or greater digitized from accompanying imagery
Rivers digitized in direction of flow and water bodies with a
set elevation (water bodies will be “flat” and rivers will flow
“downhill”)
No LiDAR points used within 1.5 meters of a digitized
breakline
State Plane Coordinates (NAD 83 feet, NAVD 88)
Digital Elevation Data
Indiana’s new DEMs
5 foot horizontal pixel resolution
Supports 2 foot contours (thus at least 18.5 cm
vertical RMSE accuracy met)
Part of the IndianaMap
http://www.indianamap.org
DEMs Available for download from:
Indiana Spatial Data Portal: http://gis.iu.edu/
Digital Elevation Data
Resources on Digital Elevation Data
USGS National Elevation Dataset (NED) Info:
http://ned.usgs.gov/about.asp
Text:
Digital Elevation Model Technologies and
Applications: The DEM Users Manual, 2nd Edition
Edited by David F. Maune, PhD, CP
Published by ASPRS, 2007
Helping People Help the Land
USDA is an equal opportunity provider and employer.
LiDAR Data
LiDAR = Light Detection And Ranging
Uses an active sensor to emit energy (light)
and detect returned energy
Can be collected day
or night)
Image from State of Indiana Orthophotography and LiDAR
Program, presentation by R.N. Wilkinson
LiDAR Data
Airborne and Terrestrial capabilities
Combines GPS and an Inertial Measurement
device to compute x,y,z positions
Every point recorded has an x,y,z, and
intensity value
LiDAR Data
 All reflections of emitted energy are returned,
generating a point cloud of the data
 The point cloud contains data points for scan hits at
multiple heights on objects, as well as some noise due
to atmospheric conditions.
 These hits are referred to as returns and are referenced
in ascending order from highest elevation to lowest
elevation for a set of returns
 Top of a building or tree is the 1st return
 Canopy of a tree or side of a building is 2nd or 3rd return,
and so on as the returned hits descend in elevation
LiDAR Data
All returns
1st return
2nd return
3rd return
4th return
Image from Lidar Technology Overview, presentation by USGS, June 2007
LiDAR Data
The vendor uses classification algorithms on
the data
Vendor delivers a data product depending on
the customer’s specifications
LiDAR Data
Indiana’s LiDAR Data (2011-2013)
 Classified Point Data
 1.5 meter Nominal Pulse Spacing (the estimated
average spacing of irregularly-spaced points in
both the along-track and cross-track directions –
FEMA)
 LAS files in 5000’ x 5000’ tiles
 Data Delivered in appropriate State Plane
Coordinate System (NAD 83 Feet, NAVD88)
LiDAR Data
Indiana’s LiDAR Data (2011-2013)
Classification Scheme
(This is not the same as 1st return, 2nd return)
 1 = Processed but Unclassified
 2 = Bare Earth/Ground
 7 = Noise
 9 = Water
 10 = Ignored Ground (breakline proximity)
 13 = Bridges (over 100 feet in length; foot bridges
not included)
LiDAR Data
Some LiDAR Resources
USGS LiDAR Guidelines (replaces draft versions 13 and 14)
http://pubs.usgs.gov/tm/11b4/
NDEP Guidelines for Digital Elevation Data, Version 1.0 (2004)
(Currently in work for an updated release)
http://www.ndep.gov/TechSubComm.html
Education (most class materials available freely)
https://www.e-education.psu.edu/lidar/resources/l1.html
LiDAR Data
LiDAR’s Limitations (in designs)
 Site changes – snapshot on day of flight
 Grade breaks – collection pattern is “random” and not based on
changes in grade as a field survey
 Critical elevations – may not detect control elevations such as
building floor elevations, edges of concrete, property boundaries or
culvert inlet/outlet elevations (requires local benchmarking at site
and adjustment of data to benchmark)
 Vegetation – May affect readings, dependent on quality of the data,
density of vegetation. Tillage may affect surface smoothness (can
affect slope calculations)
 Water – LiDAR can penetrate water, but type of laser and water
turbidity can affect this. Standing water can invalidate a local
elevation estimate from LiDAR. If you believe a data result is due
to influence of water, don’t use it for an elevation
LiDAR Data
LiDAR’s Uses (in designs)
 Planning – Visualization of data and its derivaties
(hillshades and contours) can be very useful in
planning
 Preliminary Design – LiDAR relative accuracy is
typically very high for a site, so preliminary design
for a number of uses can be done with CAD
generated surfaces and later tied to a site’s
elevations through adjustment to benchmarks
obtained in a field survey (if the site has not
undergone major change since the collect)
Indiana Data (2011-2013)
Acquiring Indiana LiDAR Data
• IndianaMap & Indiana Spatial Data Portal
(www.IndianaMap.org) View and File-based access to point cloud and hydroflattened DEM data
• Open Topography Server (UCSD)
• Key advantages
User Defined Area of Interest
Mitigates need for local storage of unneeded data
Opens the door to Indiana data for all users
Leverage server side processing for extraction and
derivatives
Helping People Help the Land
USDA is an equal opportunity provider and employer.
OT Links
• Open Topography Home Page:
http://www.opentopography.org
Open Topography Data Page:
http://opentopo.sdsc.edu/gridsphere/gridsphere?
cid=datasets
• Indiana’s LiDAR Data Home Page:*
http://igic.org/projects/lidar/index.html
*Recommend you use this home page – Indiana news, tips, tricks,
documentation, and instructions will be poster here.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Select a Region
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Results
Helping People Help the Land
USDA is an equal opportunity provider and employer.
More
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Get Data (Top)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Get Data (Mid)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Get Data (Bottom)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Choices
That’s a lot of boxes!
How do I get what I need?
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Choices
• Understand what data you really need
What task are you trying to accomplish?
What data do you really need for that task?
Decisions will depend on uses for LiDAR in which
you are engaged
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Choices
• Understand what data you really need
Do you need LiDAR points (LAS files) for particular
areas?
Get it here
Do you need the bare earth points or the entire point
cloud?
Decision driven by your intended use
Do you need a DEM, TIN or derivatives?
You may not want to get it here (yet) unless you need a
custom DEM or cannot process TINs or generate derivatives
in your own applications
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 1 - Basics
Select Area
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 1 - Basics
Review number of points and modify extent if
necessary
Set Ground Classification
This choice drives the nature of the derived DEM/TIN later
Select Coordinate System
Note that this choice will affect your units in later steps
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 2 – Point Data Format
Preference and capabilities
LAS = Larger d/l, no decompression needed
LAZ = smaller d/l, must decompress
ASCII = largest d/l, no decompression needed
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation
Gridding Parameters
Remember: Units match projection choice from step #1c.
Resolution: At least the point spacing of the dataset (1.5 m
or about 5 feet)
Radius: At least the resolution of the dataset. The larger the
radius, the more “smoothed” the DEM. Using 2x the
resolution guarantees a “3-cell filter”
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Methods
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Zmean Grid – A basic averaging method
Can be used when making any surface to average out data
irregularities that would be emphasized by Zmin or Zmax
If you chose all points in step #1b, but you still want a DEM
of the “bare earth”, you can select Zmin instead, but it will
have different results than Zmean. The differences may or
may not be significant to your work, but only you can
determine that.
Not clear if this implements any nearest neighbor
weightings, but from the description it seems not.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
 Zidw Grid
 IDW = Inverse Distance Weighting – explicitly implements an
assumption that things that are close to one another are more
alike than those that are farther apart. Is also an averaging
method.
 Resulting surface will not pass through the sample points.
 No option to control the power factor (possibly default of 2)
 IDW in general is not recommended for gridding Terrains per
Maune et al, Digital Elevation Models Technologies and
Applications: The DEM Users Manual, 2nd Edition; page 10. It is
not clear if the particular IDW algorithm and settings at Open
Topography would contradict that.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
 The Zmean to Zidw difference (2ft contours from 5 ft derived DEM,
smoothed with PAEK, 50 ft)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Which method should I use?
It depends on your needs
Zmin can give bare earth values when using ground returns
or entire point cloud, but will always assign the lowest value
in the search radius
Zmax can give a first return surface when using all returns
Zmean is a basic average of points to simulate bare earth
when using ground returns (can be a very reasonable DEM)
or of entire surface when using all returns
Zidw is like Zmean but more specialized to weight points that
are nearer – may result in some sharp exaggerations or
newly introduced inaccuracy. Typically not the best for
gridding terrain data.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Which method should I use?
The good news
It’s multiple choice! Try them all at once.
Alternately, you can generate your own DEM from the LiDAR
points you download using a variety of tools of your own
and you do not need to generate a DEM here at all
(however, this example is for a DEM, so this is just a
reminder that this step is optional and specialized).
You could also download the final project DEMs from ISDP,
and in the future, OT, without processing here at all and you
can resample from those.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Formats – ArcASCII Grid, GeoTIFF, IMG, or All
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 3 – DEM Generation (cont’d)
Null Filling
Will fill in small blank areas in the DEM being
generated at 3, 5, or 7 pixel filter sizes
Your choice depends on your project needs, but
setting a value here will minimize tiny holes of no data
in the resulting surface
Will not fix large holes from water bodies or buildings
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 4 – Derivative Products
Hillshade will use altitude of 45 degrees and
azimuth of 315 degrees
Slope will be degrees, not percent
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Section 5 – Visualization Products
Optional
Can generate files for use in Google Earth
Not used in this example (will uncheck)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Runtime
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Results
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Results
Data can be added straight to ArcMap, however
statistics will not be calculated by default
You can use tools in ArcGIS to calculate statistics
Alternately, you can force the statistics to calculate
under the Symbology tab by switching back and
forth from None to Standard Deviations under the
Stretch method drop down box
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Results (Zmean, elevation, hillshade)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Ground Return (LAS, DEM, No TIN) Example
• Results (Zidw, elevation, hillshade)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Limitations
• Area of interest scope is limited to 50 million
points if you are not logged into OT
• You can increase this to 150 million points
when logged into OT
• Point density and natural geography will cause
the approximate area corresponding to that
number of points to fluctuate
• Ground vs. All will return different point count
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Limitations
• Working with LAS Data
Typically requires a tool that supports LAS format
Alternately can convert to a 3d x,y,z format
• Raw LiDAR data can have unclassified points
and gaps in coverage depending on the return
or classification selected
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Limitations
• Derivative products can have similar gaps, such as
the DEM or hillshade generated at OT
• Break lines are not used to generate data from OT
• This is due to the file being generated from ONLY
the LiDAR points by the OT site
• DEMs Delivered from the vendor have been post
processed to fill in such areas and to use some
breaklines (will be added to OT, currently hosted
at ISDP)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Some Uses for OT Generated DEM
• Analysis in open terrain with few to no
structures or water bodies
• Can compensate for gaps from buildings and
water bodies in the OT generated DEM using
some techniques in ArcGIS Spatial Analyst to
further fill holes in the surface
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Fill the blanks in the DEM
 Raster calculator (re-run until no NoData returns)
Arc10:
Con(IsNull(Raster), FocalStatistics(Raster, NbrCircle(10, "CELL"), "MEAN"), Raster)
Where Raster is the name of the DEM layer, Circle is the search type, 10 is the
radius in cells. These criteria can be modified.
Arc 9:
Con(IsNull(Raster), FocalMean(Raster, rectangle,10,10), Raster)
Where Raster is the name of the DEM layer, rectangle is the search type, 10x10 is
the rectangle size (in cells). These criteria can be modified.
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Fill the blanks in the DEM
Model Builder (Arc 10)
Same expression, constructed in model builder
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Fill the blanks in the DEM
Python (with Arc 10)
Add a while loop to continue processing as long as
IsNull generates any True (1) results
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Recommended Settings
• To get a “bare earth” dataset at Open Topography
Return Classification = Ground
Coordinate System = User preference
Point Format = User Preference
DEM Generation (optional)
Method = Zmean grid
Resolution = at least 1.5 m (UTM) or 5 ft (St. Plane)
Radius = at least 3x the resolution
Null Filling = 7 (smooth out small gaps)
Should produce essentially a bare earth DEM, gaps for water
and buildings will be present
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Recommended Settings
• Example of a Zmean surface (no exaggeration)
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Recommended Settings
• To get a “1st return” dataset at Open Topography
Return Classification = All
Coordinate System = User preference
Point Format = User Preference
“DEM” Generation (optional)
Method = Zmax grid
Resolution = at least 1.5 m (UTM) or 5 ft (St. Plane)
Radius = same as resolution (minimize radius to decrease
blending of vertical features)
Null Filling = 7 (smooth out small gaps)
Should produce a surface approximating a DSM, but will lack
true vertical definition of features, gaps for water may be
present, may be affected by noise in the data
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Recommended Settings
• Example of a Zmax surface
Helping People Help the Land
USDA is an equal opportunity provider and employer.
Questions?
Chris Morse
USDA-NRCS
NRCS Indiana GIS Coordinator
317-295-5849
chris.morse@in.usda.gov
Helping People Help the Land
USDA is an equal opportunity provider and employer.