NEXT GENERATION DIGITAL VIDEO SURVEILLANCE: RESEARCH
AND DESIGN ENGINEERING
IMAGERY INTELLIGENCE BY WAY OF SECURE WIRELESS TRANSPORT,
WITHOUT THE NEED FOR SHORE POWER AND CONVENTIONAL
CONNECTIVITY MEDIUMS INHERENT TO CLOSED CIRCUIT TELEVISION
Technical POC:
Gregory Perry
Phone: (813) 505-7713
Email: gperry@adv-wireless.com
Fax: (813) 741-3033
5450 Bruce B. Downs Blvd. #313
Wesley Chapel, FL 33543
Administrative POC:
Daniel C. Roy
Phone: (617) 956-0888
Email: droy@adv-wireless.com
Fax: (617) 956-0880
155 Federal Street, 5th Floor
Boston, MA 02110
Cost:
Total Base (3 years) - $2,971,812
Year 1 (base): $1,635,781
Year 2 (base): $847,416
Year 3 (base): $488,615
Option 1: $639,933
Option 2: $2,443,361
Technical Topic Areas:
 High Fidelity, Ultra Low Power Video Surveillance Hardware
 Encrypted Wireless Transport Link for Remote High Security Applications
 Small Form Factor for Covert Installations
 Distributed Back End Processing Network for Imagery Data
Cost Sharing: Cost sharing on the hardware platform design by AWA is estimated at
approximately $370,000 total for the base and options and includes labor, waiver of fee, and the
use of the AWA data collection network and design lab facilities. Includes use of hardware and
software engineering resources (as appropriate).
1
Submitted by:
ADVANCED WIRELESS AUTOMATION, INC. (AWA)
(Type of business - "Video Surveillance Hardware and Software Provider")
Table of Contents
SECTION 1: ADMINISTRATIVE INFORMATION
1
SECTION 2: TECHNICAL PROPOSAL
A.
INNOVATIVENESS OF THE PROPOSED RESEARCH
B.
TECHNICAL RATIONALE AND APPROACH
C.
DELIVERABLES
D.
STATEMENT OF WORK
E.
MILESTONES
F.
TECHNOLOGY TRANSFER
G.
COMPARISON WITH OTHER ONGOING RESEARCH
H.
KEY PERSONNEL
I.
FACILITIES/RESOURCES
J.
COST
3
3
4
10
X
X
X
X
X
X
X
SECTION 3: ADDITIONAL INFORMATION (X Related Papers) Attached - "Sample Paper 1”
2
A.
INNOVATIVENESS OF THE PROPOSED RESEARCH
This proposal from Advanced Wireless Automation, Inc. focuses on advanced video surveillance
technology intended for use within imagery intelligence (IMINT) gathering applications, with
certain unique features that allow users to perform visual and other forms of monitoring in
remote and rugged locations previously unsuitable for deployment. By integrating existing
COTS wireless transmission products, high performance CMOS and other image sensors,
recently commercialized digital signal processing components, aggressive power saving and data
compression methods, military grade public and conventional keyed cryptography, and custom
software running on a robust and efficient embedded Linux operating system platform, AWA's
IMINT platform will operate off of the power grid and without the need for hardwired network
connectivity.
The proposal concentrates on three related thrust areas: (1) high-resolution video imaging
hardware capable of instantly locking and acquiring imagery data; (2) secure wireless mesh
networking technology to relay said imagery, telemetry, and sensor data from endpoint devices
to a centralized observation network for subsequent viewing and storage; and (3) robust
compression and power management disciplines to provide a compact form factor hardware
solution that can be deployed in rugged and remote locations without the need for shore power
supply and conventional network connectivity mediums.
Our proposed effort involves a variety of COTS product offerings used in conjunction with
AWA’s proprietary video imaging technology, with the intent of creating a unified IMINT
acquisition platform capable of being deployed in a wide variety of configurations, both overt
and covert in nature.
AWA personnel will build upon their core areas of expertise in the fields of wavelet video
compression, encryption and network security, embedded hardware research and development,
and network design engineering, to create an open architecture solution to be used for near real
time acquisition of remote imagery and sensor data in harsh environmental conditions.
B.
TECHNICAL RATIONALE AND APPROACH
AWA's approach to imagery intelligence realizes that in many situations streaming real-time
video surveillance such as that delivered by legacy CCTV systems are expensive to deploy, and
overvalued with regard to intrusion detection capability, post mortem forensic recovery,
confidentiality of imagery data, and power consumption. Traditional video surveillance products
rely on a cumbersome physical infrastructure of cabling and heavy current electrical power
supply, and the data product of CCTV and other conventional video surveillance systems are
copious in volume, low resolution analog in format, and require expensive human attention to be
of value to the end consumer. Addressing these problems in tandem requires a systemic
approach to design and architecture, and yields a product that bears little resemblance to what we
are accustomed to in traditional video surveillance systems.
The first step to solving this problem set is to efficiently remove surveillance data from the
analog domain at the first opportunity, i.e. as close to the image sensor as possible. This is
3
accomplished in the human visual spectrum by use of highly integrated CMOS image sensor
arrays, whose performance are now approaching that of legacy CCD technology while offering
large price and implementation advantages.
The second barrier to entry is that of producing an imager that is ready to operate with very small
time-to-acquire latency, while providing a reasonably swift frame rate; in addition, the imager
array must be compatible with extended battery and/or fuel cell operation, and capable of
operating asynchronously to the main platform host processor. This is accomplished by the use
of a lightweight supervisory processor, interrupted awake via various sensors, external stimulus,
or internal schedules, and in control of a programmable logic device. An FPGA in turn clocks
the image sensor and provides address generation and image data transfer to a large pool of static
memory, which can later be interrogated by a host controller or DSP to further process and
transport resulting frame data.
A further advantage of employing an FPGA for image sensor drive is that multiple sensors can
be driven by one FPGA, with sensor selection easily accomplished by supervisory processor
configuration of the FPGA.
A second processor hosting the platform operating system and other peripherals is booted on
demand by the supervisory processor. Once an image sensor frame has been captured, frame
buffer memory is made available to this host for transfer to main memory. Other scheduled
platform tasks to be accomplished by the main processor can also be initiated by the supervisory
processor, improving the operational characteristics of the image capture system and reducing
overall power consumption.
This system is designed for deployment regardless of local infrastructure, while at the same time
exploiting locally available low cost communications networks. Wireless transmission methods
vary and will continue to do so depending on deployment contexts. Possible topologies include
point to multipoint wireless networks; self healing wireless peer to peer mesh networks; stand
alone single platform installations using LEO satellite links and VSAT terminals on a one-to-one
basis, etc.
Most wireless communications methods to be employed by the AWA system are of relatively
low bandwidth. High bandwidth communications require advanced modulation techniques that
consume large amounts of power relative to typically available current. Further, when highpower transmission methods are unavoidable, it then becomes imperative that data payload is
limited, especially for high-resolution imagery data. AWA uses a very aggressive waveletencoding scheme to compress image data; although this is an extremely computationally
intensive compression technique, recent plunges in DSP prices coupled with sharp reductions in
power consumption and equally dramatic increases in performance afford sufficient horsepower
to accomplish wavelet transformation of image data. The advent of dual core chipsets
incorporating general purpose microcontroller and signal-processing functionality mean that this
class of algorithm can now be deployed in previously unavailable market segments, with
compression ratios exceeding 120:1 at 24-bits per pixel image data.
Once image data has been acquired and compressed, data security is provided by first negotiating
4
a DSA certificate exchange, followed by a Diffie-Hellman public key exchange, after which an
encrypted communications tunnel is established by way of 3DES or AES block cipher. The use
of certificate-based authentication provides AWA with an absolute mechanism for granting and
revoking access to wireless network resources on a per-endpoint basis, in the event that a device
becomes inaccessible due to theft or compromise. In addition, the use of robust encryption
technology provides for (3) layers of protection for acquired data:



Confidentiality – the use of AES/3DES block ciphers provide industrial strength privacy
for communications traversing the wireless network link, preventing eavesdropping and
recovery of network traffic via signals intelligence and/or TEMPEST-related attacks
Authenticity – communications are guaranteed to have originated from the appropriate
corresponding endpoint device, an important factor to consider when preserving forensic
quality video stills
Integrity – imagery data is guaranteed integrity from endpoint to backend collection
network by way of SHA-1/MD5 one way hashing functions, which prevents attempts to
manipulate or alter image data after the acquisition process
AWA's platform consumes miniscule quantities of electricity in idle mode, and the system can be
powered by primary or secondary batteries for extended periods of time. A caveat here is that if
the platform is deployed in a high traffic area, thus excessively increasing the duty cycle of the
electronics, some means of supplying a modest amount of power must be present. This may take
the form of a small PV panel.
In operation the products are under the immediate control of a supervisory processor, which
monitors time and sensor inputs to awaken the remaining electronics to perform various
sampling tasks, which include acquiring images, measuring temperature, or running other
sensors as per customer demands. The supervisory processor also periodically opens a receive
window via whatever communications method is present in a given deployment context,
allowing remote configuration and control of the device, as well as network synchronization.
Potential applications for AWA's platform include gate monitoring, access control, area and
perimeter surveillance, environmental sampling, and numerous other uses where access to power
and network connectivity is difficult or impossible due to geographic dispersion. Further, the
underlying architecture can be mutated and rearranged to accomplish a variety of other goals.
Some features and applications of the technology employed by AWA are described below.
5
Low Latency and Low Power Image Capture System
Design objectives:
 Ability of imager to lurk with low power consumption and low latency to acquire first image
 Ability of imager to capture multiple frames, buffer until such time as a transmission channel
is open
 Ability of imager to reduce image data to minimum size commensurate with forensic quality
restoration
Implementation:
A low power MCU typically in sleep mode monitors via hardware interrupts intrusion sensor
inputs including loop contacts and PIR motion detectors. An interrupt from a sensor input
activates the MCU, causing it to generate a signal to an FPGA (field programmable gate array).
The FPGA then generates signals to operate a solid state CMOS image sensor, and provides a
path to transfer pixel data from the image sensor to SRAM (static random access memory),
generating address signals to control pixel storage location in SRAM. One or several frames
may be captured over a time period determined by configuration of the MCU program. As and
when complete image frames are available, the FPGA switches control of the SRAM address and
data bus to a host controller, which retrieves frame data for further processing leading to
compression of image data. Compressed image data is then returned to the host controller board
for intermediate storage prior to transmission.
Another embodiment has the FPGA switch the SRAM address and data bus directly to the bus of
a DSP controller board, allowing direct access of image data by a DSP controller.
Another embodiment allows connection of SRAM to a DSP controller board via a DMA
controller on a host board supporting the DSP controller and imager support card.
Description of Fixed/Remote Host Synchronization, Minimization of Power Consumption:
Design objectives:
 Maintain a transmission availability window with minimum power consumption.
Discussion and implementation:
In order to allow a back channel for control of a battery powered remote wireless device, it is
necessary to minimize receiver power consumption at the remote device while maintaining a
predictable receive window for a command to arrive at said device. The controller hardware and
protocol necessary to do this with minimum power consumption are incompatible with the
hardware and protocol needed for actual bulk data transmission between devices. Thus it is
necessary to provide a means of switching hardware and protocol controlling a radio data modem
as necessary for the immediate needs of the device using the radio data modem.
A low power controller on the remote device is equipped with a solid-state switch capable of
asserting control of electrical signals to a radio data modem. This controller periodically turns
on a radio receiver, opening a receive window for reception of data. A data concentrator and
network controller periodically transmits a beacon signal, and this signal is used to adjust a timer
6
on the remote controller so as to synchronize the data modem activation window with the beacon
signal.
At times, a command embedded in the beacon signal or local activity at the remote device may
necessitate a hardware and protocol change of control of the radio data modem. When this
occurs, the controller deasserts control of electrical connections to the radio data modem,
allowing another controller to assume control of the radio data modem and simultaneously
changing protocol imposed on the radio channel.
Secure Deployment of Unattended Devices
Design Objectives
 Provide a means of detection of device theft or unauthorized removal leading to
inactivation of said device
Discussion & Implementation
AWA's equipment is intended for deployment in various locations, mostly unattended. For
various reasons, it is desirable that if devices in the network are tampered with or removed that
they should inert themselves and wipe out any surveillance or telemetry data they may contain.
Various means are provided for this. In the context of a micro cellular deployment involving a
master host, a device that is removed from the service area of the master controller will inert
itself after detecting that the master is unavailable for a period of time and other conditions. The
remote device looks for a characteristic signature generated by the master and embedded in the
beacon signal broadcast by the master. The signature will be rendered immune to replay attacks
by being composed of a SHA-1 message digest of the content of the beacon signal extraneous to
the signature itself. The signature is created using a private key, while the signature analysis is
performed using a public key.
Telemetry/Surveillance Network Design Objectives




Provide a means of gathering image and other sensor data in locations lacking power and
data
Provide a network that is largely self-configuring and is capable of self-healing
Provide a network that is agnostic with regard to uphaul to a central data collection point
Network may consist of 1.n devices
Discussion and implementation
The field side of the system consists of a common suite of core hardware and software
supporting low level network functions, augmented by specialized peripheral and memory
configurations designed to support functions specific to each type of equipment participating in
the network.
Hardware characteristics that are likely generic and are important to the application include: the
aggressive use of micropower operation methods; combinations of high endurance/renewable
power sources; reduction of component counts to a low number consistent with demanding
physical endurance goals; highly robust and inconspicuous packaging.
7
Proprietary hardware methods include: the use of multiple image sensors operating semiautonomously in conjunction with a single host processor so as to provide high performance
wide angle scene coverage with minimum power consumption and expense; initiation of image
sensor operation and completion of scene acquisition in advance of host processor availability;
combinations of image sensors, multiport storage devices, digital signal processors and host
processors so as to afford pipelining of image acquisition, compression, analysis, and
transmission; simple arithmetic logic control of memory devices to afford high performance; low
power hardware based data reduction; combinations of the aforementioned methods.
Important software characteristics include: the use of wavelet data compression techniques to
simultaneously compress and secure image data; wavelet scene feature recognition.
Important network characteristics include: autonomous device configuration with regard to peer
mesh and uphaul route discovery; autonomous failure management; autonomous identification
and lockout of compromised or unauthorized devices.
8
Definition of Terms
3DES – Triple DES
AES – Advanced Encryption Standard
CCD – Charged Coupled Device
CCTV – Closed Circuit Television
CMOS - Complementary Metal Oxide Semiconductor
COTS - Commercial Off-the-shelf
DES – Data Encryption Standard
Diffie-Hellman - Public key cryptography based on calculating logs in modular arithmetic
DMA – Direct Memory Access
DSA – Digital Signature Algorithm
DSP – Digital Signal Processor
FPA – Focal Plane Array
FPGA – Field Programmable Gate Array
GPS – Global Positioning Satellite
IMINT – Imagery Intelligence
LEO – Low Earth Orbit Satellite
MCU – Microcontroller
MD5 – Message Digest Algorithm RFC1321
PIR – Pyroelectric Passive Infrared
PV – Photovoltaic solar panel
SHA-1 – Secure Hash Standard
SRAM – Static Random Access Memory
VSAT – Very Small Aperture Terminal satellite communications system
Wavelet - Mathematical function useful in digital signal processing and image compression
9
C.
DELIVERABLES
Key deliverables for this project are described in the sections below.
1.
Endpoint Device
A prototype of the endpoint device will be delivered during year 2 of the project. This will
include a single board version of AWA’s wireless surveillance platform, and will include:









(1) Spread spectrum, fast frequency hopping 900 MHz ISM radio section with antenna
array
(1) 1280x1024 CMOS imager
(1) 320x200 uncooled thermal imaging FPA
(1) Light sensor
(2) PIR motion sensors
(1) Integrated GPS core
(1) Temperature and humidity sensor
(1) 3-axis magnetometer
(1) Covert pinhole lens array
10