Synthetic Environment (SE) Core Database Development Process
Robert Cox
PEO STRI, PM CONSIM
rob.m.cox@us.army.mil
Guillermo Flores
PEO STRI, PM CONSIM
memo.flores@us.army.mil
Mark Johnson
PEO STRI, PM CONSIM
mark.doyle.johnson@us.army.mil
Dolores Lowe
PEO STRI, PM CONSIM
dolores.g.lowe@us.army.mil
Connie Perry
PEO STRI, PM CONSIM
connie.perry@us.army.mil
Keywords:
ABSTRACT: The SE Core is part of the United States Army's overarching strategy of developing
simulation systems that help make our Warfighters the best trained in the world. The two primary
initiatives under the SE Core program are the Architecture and Integration (A&I) and the Database Virtual
Environment Development (DVED). The A&I is not the focus of this presentation and will not be discussed.
The SE Core's primary mission is to rapidly generate correlated simulation system terrain databases. The
master SE Core database is populated from a union of multiple authoritative data sources by using a suite
of commercial and government-off-the-shelf database tools. The SE Core DVED architecture and tools will
enable the generation of SE Core runtime terrain databases in days or weeks vice months or years.
The focus of this presentation is to describe the SE Core DVED production process. Key areas that will be
discussed will include the overall process and then focusing on the details to include: the initial database
request process; how source data for the requested database is gathered and prepared for use; the
refinement and enhancement of the source data; the standardization process; and the verification and
quality measures employed before delivery. Additionally, the authors will present the current production
along with the various formats and map projections that are supported. The presentation will end with
exemplars of future efforts that SE Core DVED will be pursing in the future.
1
1. Introduction
The acquisition of simulation training databases
has been a very complex, lengthy, and costly
endeavor. SE Core’s effort is developing
processes and tools to create non-proprietary,
open format, image generator (IG) independent
Environmental Representation (ER) databases1.
The SE Core Master Database (MDB) is the
central repository for the creation of correlated
databases used to train, mission plan, or mission
rehearsal in the Live, Virtual, or Constructive
(LVC) domains. Within the DVED the database
architecture is coupled with a suite of COTS
tools that enable database development.
The SE Core effort will also develop common
virtual vehicle models, common virtual sensor
simulation software, and the virtual simulation
component of the dynamic environment. The
dynamic environment will include approximate
visual effects from simulation (e.g., munitions,
mobility, engineering, rubbled buildings, etc.)2.
The SE Core process is flexible, data-driven, and
extensible. The Standard/Rapid Database
Generation Capability (STDGC) is designed to
standardize, align, and clean source data in the
Master Terrain Database Generation Toolkit
(MTDGT), which also is used to generate
synthetic environment data using the Runtime
Database Generation Toolset (RDGT). Data is
exported in various formats from a standard
application programmer interface (API)3.
Furthermore, the SE Core process addresses
other challenges such as miscorrelation between
simulations and lower database generation costs.
In this paper we will first provide some
background on the terrain database generation
process. Next, we will present the SE Core
methodology to include how a database is
requested, the procedure for gathering source
data, the standards that are used by SE Core, and
the techniques used to refine and enhance the
data. We will end with a discussion of the
verification and quality process used by SE Core.
2. Background4
Military training simulation systems have been
around for decades. Most of these systems have
used some derivation of the real environment to
represent the three-dimensional reality. In the
last three decades the price-performance benefits
of computer technology and the demand for
reconfigurable, less expensive, and more realistic
representations have driven simulation systems
towards real-time digital computing and fully
synthesized environments.
Until the early 1980's, simulators were mostly
stand-alone systems designed for a specific task
training purpose. Until the introduction of the
SIMNET program, no one had ever used a
multitude of simulators in a combined forces
training environment, interacting over a network
in real-time. SIMNET technology allowed
crewmembers in one simulator, to interact with
many other manned or unmanned simulators
located at the same or other training sites.
Environmental Representation databases are
comprised of many parts. Those parts include
but are not limited to the terrain surface, terrain
features, 3-dimensional models, textures, images,
and other data such as system specific data to
satisfy computational requirements and labels for
producing electronic and paper maps.
There are several constraints on the creation of
environmental databases. Perhaps the most
prominent of these is dictated by the real-time
computation requirements. Since processing
power is often limited, once a computing
platform is chosen for a given cost-performance
range, the software must be designed for best
real-time performance. This means both the data
and the data structures stored in a platformspecific version (runtime) of an environmental
database play an important role in optimizing the
system performance.
Given
the
fixed
computational budget of a system, the database
designer must take into account the applicationspecific requirements, the size and extents of the
database, the desired density and fidelity, as well
as the type and amount of the available raw data
elements that must be incorporated into the
database.
There are other application-specific requirements
that must be taken into account such as those
related to whether the database will be used for
air, ground, sea, near ground, or any combination
of these simulations. The detail needed by a
simulator that allows an individual to walk on
the terrain is much greater than that expected for
a helicopter simulator typically flying several
hundred feet in the air. Database density, size
2
and extents, viewing range, field of view, and
other important simulation requirements dictate
the amount and type of data that can be included
in a database without overwhelming the
performance requirements.
condition for achieving it. Since the variables
affecting interoperability are many and complex,
effective mechanisms for making the database
interchange
process
successful
become
significantly more difficult and challenging.
Often the intended simulation platform imposes
specific constraints. The specific special purpose
hardware architectures, designed for the sole
purpose of real-time image generation, impose
vastly different constraints on the database
contents. Two database designers building a
database of the same region for two different IGs
often arrive at entirely different end results. The
polygon or object processing capacity of an IG
limits the database density to levels that can be
processed in real-time. Similarly, image
rendering techniques drive whether a database
can contain textures and how many, or if the
image generator can render all the objects that
are potentially viewable in a scene within a fixed
frame time. There are other architecture-specific
features such as caching scheme, processing of
transparent objects, or image enhancement
techniques like anti-aliasing drive how a
database may be partitioned.
A critical factor in constructing and sharing
environmental data is incorporation and use of
good tools. Most tools are special purpose, given
the various criteria and techniques employed by
different suppliers for constructing and tailoring
databases. As the domain of networked
simulation expands and other information
technology sectors emerge, the need for more
common and yet more sophisticated tools will
increase.
Standard / Rapid Terrain Database Generation Capability
Supervisor
Write
API
Read
API
Configuration
Management
Source Data
Corrections
Raw
Sources
Source Data Quality
Assurance
Refinement
Raw
Source
Master
Database
(MDB)
Source Interchange
Formats
Interoperability of multifidelity systems on the same
network is highly desirable; in
fact it is often demanded. The
challenge is in determining the "right" type and
amount of environmental data that each
simulator should use to ensure interoperability.
Most successful heterogeneous networked
exercises have been conducted under restricted
conditions.
CAD
Sources
CGF
SEDRIS Plug-In
Polygon/Geometry
Engine
MDB Repository Quality
Assurance
Specialization
Run Time Database
Generation Toolset (RDGT)
Filtered Viewer
3D Models
Vectors
Attribution
Imagery
Elevation
Master Terrain Database Generation
Toolset (MTDGT)
Source Data Inporter
These types of computation
constraints are not unique to
visual
systems.
Any
information
technology
application that must achieve
specific real-time performance
measures has to reflect such
objectives in its design. This
in turn impacts the data
structures, data types, and the
information that is needed and
expected to be available from
a database.
Figure 1 is the SE Core processes with the
objective to be able to collect, generate, and
manage the Master Database (MDB) with the
highest fidelity of source data available. There
are four major parts of the process. For this
paper the focus will be on how a database is
requested, the gathering of source data and its
refinement. We will include a discussion of the
standards and quality measures that are used.
Paper Map Plug-In
CER
Elec. Map Plug-In
SEE
API
Application
Plug-Ins
Run Time
Databases
DBDD Plug-In
Visual
Maps
Mission
Run Time
Applications
Storage
Architecture
Source Data Asset Management
A common misconception is to equate success in
the interchange of data with success in
interoperability. Interchange of data does not
guarantee interoperability, but is one necessary
PVD
Sensors
Figure 1: SE Core Process
3. Database Request Process
The SE Core Database Request process is shown
in Figure 2.
3
Figure 2: SE Core Database Request Process
data is collected (discussed in the next section)
and any revisions to the request are processed
depending on the needs and availability of data.
If all is acceptable to the customer, the request is
passed to the database development team. The
current and future data formats available from
SE Core will be discussed in a later section.
The customer is provided with a database request
form. That form allows the customer to specify
the content they require. This includes but is not
limited to the features they require (e.g. bridges,
roads, agricultural areas), areas of significance to
include their location (e.g. specific buildings,
airfields), and the format of the database
transmittal. The customer can also use this form
to request any high resolution inserts that are of
importance to them.
4. Source Data
When a database request form has been
submitted and it has been approved, the process
of gathering source data begins (Figure 3).
Initially a program fills out a database request. If
the data already exists in the MDB and a means
to generate the required formats is available (e.g.
plug-ins) the request is forwarded directly to the
production team and the desired data in the
requested format is prepared and sent back to the
requesting program.
If the desired data is not already in the MDB, the
request is validated by the US Army TRADOC
Capabilities Manager (TCM) Virtual to include
the size of the database region, the level of
fidelity required, and any other requirements of
interest to the customer which have been
validated by TCM Virtual.
The database can be divided into separate zones
if the customer requires. Those zones range
from providing only the terrain skin for a reqion
in the database to high fidelity inserts of urban
environments. Currently, there are five (5)
different zones available for the customer to
request and they include levels of fidelity of the
transportation network, population density to
include buildings and building interiors, any
hydrography, significant military operational
features, and various vegetation that is required.
Once the request has been validated, it is passed
to the program manager. At this point, source
START
DATABASE
REQUEST
SOURCE
DATA
COLLECTION
REVIEW
TECHNICAL
REQUIREMENTS
DETERMINE
RESOURCES
FOR
ACQUISITION
COORDINATE
DETERMINE
COORDINATE
STORAGE AND SOURCE DATA
EXPECTATIONS
ENTERING/
SPACE
ALLOCATION EXITING FACILITY
NO
REVIEW
SOURCE
USEABLE
SOURCE?
YES
VECTOR SOURCE
DATA
STANDARDIZATION
IMAGERY
PROCESSING
ELEVATION
PROCESSING
3D MODELING
END
Figure 3: SE Core Source Data Collection
Process
There are five (5) questions that must be
answered as the database request is fulfilled.
Those questions are:
4
1.
What are the technical requirements
(database and system)?
What are the resources required for
acquisition?
What
are
the
storage/space
requirements for the data?
What coordination of acceptance of the
source data must be done and how do
we deliver the resultant data?
Are there any conversions that are
expected?
information as the user that inserted the data into
the database, notes on why the data was
populated, and the origin of the data. SE Core
will be expanding the level and type of metadata
in the coming year.
The SE Core has also developed an extensive
document that outlines the source data
investigated for use5. The reference document
lists over 160 different types of data that are
currently investigated for use in building an SE
Core database. The principle data types that SE
Core uses include:
Along with the above standards, the SE Core
program is in compliance with the US NGA
standards that are developed by ISO Technical
Committee (TC) 211.
2.
3.
4.
5.
1.
2.
3.
4.
Imagery:
CIB, Buckeye, and
Quickbird
Vector:
VMAP, Urban Tactical
Planner, NAVTEQ, DAFIF, and Shape
Files
Elevation: DTED, LIDAR
Models:
Site Photos, Building
diagrams, CAD
Once the five questions have been answered and
the source data identified and collected, SE Core
is now ready to start processing of the data to
include refining and enhancement. However,
before this discussion is presented, a brief
discussion of source data standardization will be
presented.
The SE Core data model is based on the SEDRIS
ISO/IEC 18023-1 Data Representation Model
(DRM)8 and all data concepts are defined by the
SEDRIS ISO/IEC 18025 Environmental Data
Coding Specification (EDCS).9
Regional information is stored as areal features
with the attribution that defines the region. For
example, a feature for a given area has the
attribution that defines possible textures that can
be applied to models within the spatial extent of
that region. In addition, regions can be stacked
with a priority based on user needs.
There are several tools that are used to certify the
data is ready for use. These tools check for
invalid and null geometry and attribution. Plus,
the tools check for compliance to the SE Core
standards for geometry, spatial referencing,
definition, labelling, etc.
There will be a
complete section on the SE Core quality process
and where in the SE Core process checks are
performed to make sure the user request has been
met.
6. Data Refinement and Enhancement
5. Standards
All data is spatially partitioned based on MILPRF-89041A6. The imagery is stored based on
the US NGA CIB format. For consistency, the
vectors are also partitioned based on the CIB
schema with one important difference; they are
not chopped to the tiles, but references the
groups of features to the CIB tiles they intersect.
All data is stored in latitude/longitude using the
WGS-84 reference datum. If source data uses a
different coordinate system or reference datum,
the SEDRIS developed ISO/IEC 18026 – Spatial
Reference Model7 is used for coordinate
conversion or datum transformation.
Data element also have metadata attached.
Currently, the metadata includes such
A database, while required for a specified
purpose or training event in totality, oftentimes
contains certain areas in it that are more essential
than others for that particular use. Therefore, SE
Core has developed a stratification of “zone” that
help both the user and SE Core to focus
resources in those areas that are required by the
customer to have greater attribution, while
developing the full database.
There are five separate zones that are based on
map projections (e.g. 1:250, 1:100). The zones
contain the level of fidelity that the map
projection would have, but in some cases there is
more information added based on the
requirements that must be met. The additional
information could be based on:
5
A.
B.
C.
D.
E.
SAF only padding
Ancillary areas
Navigationally significant areas
Training areas
High resolution Inset
Also, within each of these zones/map projections
there are different significant characteristics that
are organized based on the user’s need. Below is
an example of five of those characteristics.
1.
2.
3.
4.
5.
Transportation
Population
Hydrography
Military Operations
Vegetation
Thus, if an area of the database is to be
developed at the Zone D level, the following
would be way the area is prepared for each of the
five characteristic areas:
1.
2.
3.
4.
5.
Transportation:
All transportation
features extracted as linears. Parking
lots extracted as areals.
Population: All buildings represented
with either 3D buildings or areal
features that are extruded by the RDGT.
Hydrography: Hydro features validated
against the imagery for location and
width.
Military Operations:
All features
extracted from imagery. Features not
recognizable in overhead images will be
placed by hand when appropriate source
is provided.
Vegetation: Individual tree locations
will be provided in the source data.
Forests will be extracted as areal
features. Trees will be scattered in the
RDGT.
have the correct height, and the SE Core
automated tool checks are passed.
Elevation: Checks for elevation data
include checking for spikes in the data,
making sure the proper projection is
used along with the requested resolution
is present. Also, a check is made to
make certain the data is correlated to the
ground truth.
Models: The models are reviewed for
polygon attribution, footprint, LoD,
lights, etc. Plus, there is a texture
review that includes the format, sizing,
and application.
3.
4.
Once the data has been refined and enhanced, it
is ready for the quality process checks.
7. Verification and Quality
Figure 4 shows the SE Core quality steps
(marked by letters “QC”). While there are steps
throughout the SE Core process, the majority of
steps are found in the Master Terrain Database
Generation Process. This is where source data is
taken in and a sequence of steps are executed to
guarantee the data is ready for uploading into the
SE Core MDB.
Each of the four principle source data types has a
quality check. For instance, the vector data is
checked for duplicate features, self-intersecting
features, feature overlap, and valid attribution.
For imagery, the data is checked for the
projection type and orthorectification. There are
elevation data checks for spikes and gaps and the
models are checked for conformance to the SE
Core model specification.
Standard / Rapid Terrain Database Generation Process
Supervisor
As presented in the previous section, there are
four fundamental data types that SE Core uses.
The following are the general processing that SE
Core does for each of these data types.
Master Terrain Database Generation
(MTDGT) Process
QC
Sources Standardize
QC
2.
Imagery: A check is performed for
overall aesthetic quality to include
colour and cloud cover. In addition the
resolution, projection, and correlation to
ground truth data are checked.
Vector: Checks include ensuring that
vectors are aligned with the imagery,
attributions are consistent, buildings
Pre-Validated
RDGT
QC
QC
CER
QC
QC
Textures
Elevation
Imagery
1.
DBRR
Adherence
MDB Repository Quality
Assurance
Vector Cleaning
3D Models
Run Time Database
Generation (RDGT) Process
Vector Digitizing
Raw
Sources
Source Interchange
Formats
QC
DB
QC
QC
CAD
Sources
QC
QC
QC
Master
Database
(MDB)
Storage
Architecture
Figure 4:
Process
QC
Project Set Up
Free Fly
CCDS
Technical
Interchange In
Meetings Process
Reviews
Source Data Collection
QC
Measure
Performance
Run Time
Check Out
SME
Quarterly
Working
DB
Group Request
Process
Database Request from TCM-V
Quality Checks Built into the
At the runtime database generation step a series
of tests are performed to validate the database
6
being generated for a user meets their
requirements. Checks at this step include such
items as missing textures, terrain through
bridges, disconnected bridges, bad vertices,
missing blocks, and specific tests for the various
formats that SE Core can produce (e.g. DTED,
Shape Files, OTF, CTDB, and STF).
Here is a brief list of tools that have been
created.
1.
2.
One such tool that SE Core has developed to
assist in this process is called the Vector
Analyzer (VAZER). This tool was developed as
an extension to the ESRI ArcGIS and contains a
suite of tools that assist in finding issues with
vector data the production process. This tool can
find issues such as duplicate features, or
improper attribution, or improper vector
alignment.
3.
8. Current Databases and Formats
6.
The SE Core program has produced several
databases and supports over 15 different formats.
This phase of the effort has focused on the PEO
STRI virtual programs Close Combat Tactical
Trainer
(CCTT)
and
Aviation
CATT
(AVCATT). As such, the primary databases that
have been developed by SE Core have been
those to support these programs. The databases
include Fort Hood, Fort Riley, Hawaii, Fort
Stewart, and several overseas databases.
As stated above, the following list is some of the
formats that are supported by SE Core:
1.
2.
3.
4.
5.
6.
7.
8.
CADRG
CCTT PVD
CTDB
DTED
Open Flight
OTF
SEDRIS Transmittal Format
VBS2
SE Core DVED also supports these and other
image generators:
1.
2.
3.
4.
EPX 50
Radon
SAGE
S2 Focus
Along with the various databases, format, and
image generators that SE Core supports, the
program has created over 60 tools that are used
to support these and other processes in DVED.
4.
5.
7.
8.
9.
AutoSnapper:
Used to enhance
ArcMap snapping functionality.
MADV: Extends ArcMap and provides
techniques to move, add, and delete
vertices.
Google Earth Sync tool: Syncs the area
of the world loaded within ArcMap to
Google Earth.
CM2 FID SMC tool: View and edit the
feature ID (FID) and Surface Material
Code (SMC) setting of a moving model
in Multigen Creator.
The Attributor:
Applies materials
attribution to a selected polygon based
on the data model specified.
EDM Editor: Used for editing the data
model.
Validation Plug-in: Used with Terra
Vista to validate data within the terrain
database specified by user inputs.
Illuminati: Calculates and stores default
intensity values or and average
luminance of a group of polygons.
Tune Town: Allows a user to tune
terrain database data.
9. Future Efforts
The SE Core program has been focused on
supporting the virtual domain. Now the program
will begin to add other domains. The first of
which is the constructive domain. In particular,
SE Core will start the effort to provide OneSAF
with its required environmental representation
databases.
Also, with the addition of supporting
constructive simulations, the type and amount of
metadata required will increase. The SE Core
program will re-evaluate all of its current
metadata types and add other data types to not
only support constructive, but live simulation
and training programs.
As new required
metadata types are identified, they will be
compared with ISO 19115-2, the currently used
metadata standard by the US NGA. If a new
metadata type is identified and it is not currently
in this standard, the SE Core program will
approach ISO TC 211 and request that it be
added to the standard.
7
In addition to these efforts, the program has
several other required tasks that will be worked
as resources are made available when existing
efforts are completed.
(3)
Johnson M.D., J. Freeman, C.M. Perry;
SE Core DVED – An Introduction to the
Standard/Rapid Database Generation Capability
(STDGC), July 2007 IMAGE Conference.
10. Summary
(4)
The Background Section was taken in
part from www.sedris.org.
This paper provided the current status of the SE
Core program. We provided background on why
the SE Core program is essential to the US Army
simulation community and provided the top level
SE Core process. From there, we outlined the
database request process; how to get a database
form SE Core and then described the source data
and the process used to acquire source data. The
principle data types used by SE Core are
imagery, vector, elevation, and models (both
static and moving).
Next, the SE Core
standardization process was discussed. The
process includes the use of several international
standards. A key area of discussion was how SE
Core refines and enhances the source data
collected. In particular, a discussion of how SE
Core has defined five characteristic areas (or
zone) to enhance the level and fidelity of the
databases produced. By building a database
defined by these zones, enables the production
process to focus on user required areas for high
definition/details and focus resources in those
areas. The verification and quality process was
outlined, too. There are over a dozen steps in the
SE Core process that have detailed quality
checks to make sure a database is developed and
delivered to a user correctly. The SE Core
database can be delivered in multiple formats
and can support multiple image generators.
Lastly, the future efforts were outlined. It is SE
Core’s intent to begin supporting constructive
simulation in the near future along with
reviewing the current metadata for possible
extension.
(5)
SE Core SVR Management Program
Attachment 19: Baseline Source Data Providers
List,
W900KK11R0002,
October
2010,
https://www.fbo.gov/index?s=opportunity&mod
e=form&id=314cda5d9287918a09d465a30378cc
cd&tab=core&_cview=1
(6)
US National Geospatial-Intelligence
Agency, MIL-PRF-89041A. Controlled Image
base. Bethesda, Maryland, USA: NIMA, 2000.
(7)
ISO/IEC 18026:2005, Information
technology – Spatial Reference Model (SRM).
(8)
ISO/IEC 18023-1:2006, Information
technology – Data Representation Model
(DRM).
(9)
ISO/IEC 18025:2005, Information
technology – Environmental Data Coding
Specification (EDCS).
11. Reference
(1)
SE Core Home page, Database-Virtual
Environment Development (DVED). Available
from World Wide Web: https://www.se-core.org/
(2)
SE Core Program Begins helping Army
Warfighters train as they fight, Military
Simulation & Training News, Issue Number 19,
Fall 2008/Winter 2009, CAE. Available from
World
Wide
Web:
http://www.cae.com/en/military/_pdf/Newsletter
19.pdf.
8