A key aspect of this course is a project in which you will work with a team to develop a conceptual
design for a fusion system for your selected application. The point of the exercise is to help focus your
thinking about fusion in the context of a real application. In this lesson, we provide some initial
guidelines and thoughts about such a system design.
The objective of this topic is to continue the evolving design of your data fusion system. We note that
fundamentally a data fusion system is an information technology (IT) system, since it involves sensors
and sources of data, communications, computation for data processing, a database system and humancomputer interaction. As you proceed, you will first focus on design issues related to the JDL level 0
and level 1 processing. Later in the course, we will add information about the other JDL levels. Thus,
your focus will be on the following:
Selection and characterization of the sensors and input data sources
Identification of issues and concepts related to level 0 processing, e.g., data conditioning and
transformations, signal, image or text-based processing, selection of a coordinate frame of reference,
etc.
Description and initial selection of the level-1 process for your application. This includes;
Identification of what needs to be estimated, identified, tracked (the state vector)
Link of the observables to the state vector
Description of association/correlation challenges (if any)
Identification of key data base elements for level-1 processing, and
Initial selection of candidate level-0 and level-1 algorithms.
What is an information technology system? We are surrounded by networks of such systems. So,
occasionally it is helpful to step back and consider the components of an IT system. These include
issues such as:
Input sources – what provides the inputs to the IT system – sensors, external data bases or network
links, humans acting as observers or “soft sensors”, etc.
The plumbing or connectivity – the computers, networks, storage devices, interface devices,
communications links that allow the information to flow throughout the system and be processed.
Information – What is the information to be accessed, processed, stored, manipulated, retrieved, and
displayed in the fusion system?
External interfaces – What does the data fusion system connect to, and where does the output of the
system go and to whom?
Users – who are the direct and indirect users of the data fusion system?
Environment – what is the environment in which the data fusion operates? What types of platforms,
networks, daily operational conditions constrain or drive the system?
Standards – What are the policies, procedures, security constraints and other factors that both drive and
constrain the development and operation of the system?
COTS - What commercial off the shelf (COTS) software and hardware will be used in the system?
Legacy – What legacy (previously existing) sensors, hardware, software, user interfaces, and pre-existing
processes must the fusion system use and comply with?
This is not an exhaustive list of questions, but should begin to guide your thinking about the design of a
data fusion system.
Some of the key system design issues for a data fusion system are listed here. They include:
Identification of the components and functions – what are all of the “pieces” and functions that need to
be assembled or developed to make the system function?
Functional architecture – We have previously described several types of architectures that could be used
(e.g., centralized fusion, autonomous fusion, hybrid fusion, etc.). A key design choice involves when
and where the fusion will actually occur.
Control concepts – To what extent do we have control over the input data (e.g., pointing the sensors,
selection and control of algorithms, etc.). How is this control accomplished? What is the role of the
human “in” and “on” the processing loop?
External interfaces – What are the external interfaces? What external links are there from our data
fusion system to other fusion systems, sensors, users, and sources of data?
Human computer interaction (HCI) – Who are the system users and how do we intend for them to
interact with the fusion system? What are the circumstances of these interactions?
Data structures and knowledge representation – We have previously indicated the challenges of
establishing the data structures and ways of representing knowledge for the fusion system. This is
complicated by the heterogeneity of the fusion data (e.g., text, signals, scalar data, vector data, image
data, semantic knowledge, etc.)
Timing and response predictions – Finally, what are the timing and response requirements and
predictions for the fusion system. How fast does data enter the fusion system? What is the intended
response time for user queries? What are the constraints on the response times, such as computer
speeds, communication rates, data collection rates, and other factors?
This chart simply shows a notional architecture for an IT/fusion system. The system has components
and linkages to external sources such as web interfaces, local and external databases, national data
sources, and sensors and human sources. Services include database administration, data mining, and
report generation. Control processes seek to control the flow of the fusion processes as well as
potentially the external data sources and sensors. Different point of service users need to be served,
such as fusion analysts, local users, and long-term information maintenance personnel. Finally, of
course we need to perform the functions called out by the JDL data fusion process levels.
A conceptual example of a software environment for a fusion system is illustrated here. Input sensing
devices, human observers, and external computer systems or information sources provide information
to the fusion system. Major functions include; i) information conditioning and processing, system
control and monitoring, data base management, multi-source information fusion, automated reasoning
and context-based interpretation, decision support and information retrieval aids, and human computer
interfaces. Each of these components involves unique test and evaluation challenges related to
external interfaces, interactions with humans, algorithm evaluation, possible control of hardware, and
contextual/automated reasoning.
Since fusion systems tend to be designed for operation in a real environment, for use by analysts,
soldiers, crisis management personnel, physicians, etc., there can be challenging human computer
interface (HCI) design issues. We can consider two broad categories of these issues; i) the
characteristics and conditions associated with the users, and ii) the environmental conditions in which
the system must be used
The first category (user characteristics and conditions) includes issues of
Level of user training – is the user a novice, expert, or casual user?
Ancillary user actions - what else the user is doing in addition to interacting with the fusion system (e.g.,
flying a plane, operating a vehicle)?
User preferences – how does the user prefer to interact with the system?
User profile – are there different user profiles that would affect the user/system interaction (e.g., based
on physical characteristics, mental characteristics or conditions such as stress, gender differences, or
cultural differences)?
Decision styles – Different users have different styles of decision making. For example, some people
seek to systematically search for information that affirms or refutes a hypothesis, some users are more
eclectic and dynamic in their information search and decision process, and others use a hybrid style.
The human computer interaction should support the user’s style, rather than force the user to process
information based on the system’s design.
Levels of stress and coping skills – Stress affects users in different ways, and people cope with stress in a
variety of ways. The human computer interaction design should assist a user in coping with a stressful
decision-making environment, rather than exacerbating the stressful conditions. For example, alerts
that keep “popping up” on a user’s screen may induce user stress and lead to worse decisions, than a
more selective method of display or alerts.
The second category (environmental conditions) involves another set of issues related to human
computer interface design. These include:
External noise levels – many fusion systems might be operated by a user in a cockpit or industrial
environment that provides significant distracting environmental noise
Lighting – systems designed for use in an office environment may not function well in either low light
conditions or in bright sunlight
Platform conditions - Platform and working conditions may have a strong effect on the design for input
devices. Use of a mouse or keyboard would not be appropriate for a user who was occupied with other
tasks (e.g., maintenance, surgery, piloting, etc.)
Timeliness of input and response – Human computer interface design is affected by the timeliness of
inputs and response. In some cases, it may be necessary to warn a user that a database query or
requested computation may require a lengthy response. Some systems provide alterative “fast
response but reduced accuracy” versus “longer response time but improved accuracy” options.
Variability of environmental conditions –We have addressed earlier issues related to the variability of
the surrounding environment. The environment affects the sensor performance as well as the potential
design of the human computer interaction. These environmental conditions (lighting, noise,
temperature, etc.) are considerations for the HCI design.
Need for privacy and security – Finally, issues related to privacy and security affect the human computer
interface design.
Another large component of a data fusion system is the database. In many cases the single largest
amount of development for a system involves the database (even if a commercial off the shelf system
such as ORACLE is used). Data base design issues include the following:
Size and composition of the data base – clearly the size and composition of the data base affects how
the data base is structured and what software is required for ingestion, organization, retrieval, archiving,
error checking and other functions. The heterogeneity of fusion data including signals, images, video,
text and meta data confound the overall database design.
Ingestion rates and retention intervals for data – The rate at which data must be ingested into the
system and the length of time it must be retained seriously affect database design. In general there are
tradeoffs between efficient methods for rapidly ingesting data versus the need for rapid and general
data retrievals.
Queries against the data base – Both human users and algorithms will seek to make queries against the
database. The data base design affects the ability to efficiently perform Boolean queries, pre-stored
queries, ad hoc queries, etc. There may be a need for a natural language interface.
Data relationships – Various data relationships may need to be represented including; hierarchical,
relational, network, inheritance requirements, etc.
Error checking – Extensive efforts may be required to perform error checking to verify the information
sources, validation, pedigree and other issues that could affect downstream fusion processing
Distribution of data across system nodes and components – Increasingly, data is likely to be distributed
across multiple system nodes and components. It is particularly challenging for a distributed database
system to address the coordination of database updates and synchronization.
Data protection and security – Data security can be a significant issue, not only for military applications
but also for applications in which data are sensitive or proprietary (e.g., for medical applications, critical
infrastructures, or competitive issues related to product health and status).
Data initiation and start-up – Data initiation and startup is an issue for systems involving tracking of
dynamic objects or entities. How do we produce the initial estimates that drive the subsequent
sequential processes.
Data archiving – Many analysts want to ensure that collected data is archived “forever”, just in case it
would be useful to look at the data later. However, with modern data collection capabilities, such data
archives may grow out of bounds. Establishing a policy and procedures for archiving data is a design
issue.
Mining and analysis – Finally, it is anticipated that collected data will be mined and analyzed to
determine trends, establish thresholds, support analyses. The type of analyses and mining functions
desired will impact the design of the database, to allow the requisite queries and data linkages.
The following are some common pitfalls in the preliminary design of a system. This is not an exhaustive
list, but some areas to keep in mind in your system design process.
Over design – One common fault is over designing a system to include elements not traceable to
requirements or overly stringent (thereby constraining creativity in detailed design)
Under design – The complement to over design is under design. This is a failure to account for realistic
operating conditions; failure to account for variations in inputs and human users; utilization of
unrealistic assumptions about the environment
Failure to map design components to requirements – It must be remembered that components of
elements of the system should be mapped to requirements; We need to account for all external
interfaces, operational conditions, and constraints.
Conflicting design elements – When developing a design, particularly in a team environment, we must
be cautious about conflicting design of system elements or components.
Leaping to the solution – Finally, it is tempting to leap to a solution rather than to consider viable
alternatives (e.g., algorithms, concepts, COTS alternatives, etc.).
The design of a fusion system may seem nearly overwhelming and a certain failure – however, a
collection of even simple tools may significantly improve the effectiveness of a system user/analyst. The
key is not to design an overly complicated system – instead, one should focus on how to assist a user to
sort through large data sets.