Andrew Milluzzi, Tyler Lovelly, Donavon Bryan
EEL6935 - Embedded Systems Seminar
Spring 2013
Topic: Sensor Networks
01/24/13
1 of 42
Assessing Performance Tradeoffs in
Undersea Distributed Sensor Networks
Thomas A. Wettergren, Russell Costa, John G. Baylog, and Sandie P. Grage
Naval Undersea Warfare Center
Published in OCEANS in September 2006
2 of 42
Introduction

Large scale distributed networks


Cheaper sensors prone to false alarms


Cost becomes important factor
Tradeoff between sensitivity and false positives
Detection requires data from multiple sensors


Triangulate data to ensure it comes from same
target
Ensure data is synchronized and readings are
current
3 of 42
Performance Models

Even when an object is in sensing field, there is
still a chance the network will miss it


PSS (successful search prob.)
leverages Poisson process
to model detections by nodes
PFS (false search prob.) based on false alarms from
sensors
 Not a mixture of good and
bad data, only concerned
with false cases where we do not get useful data
4 of 42
Issue of Cost
 Many
cheap sensors vs. fewer expensive
sensors
 Cost function of field is based on size of
field and number of sensors
Same factors as PSS and PFS
 Allows for system optimization

5 of 42
Pareto Optimization

Optimization based a set of parameters that
shows tradeoffs


Allows for a decision to be made without the need
to explore the full range of every parameter
Approaches
 Gradient Based
 Useful

Evolutionary
 Iterate
6 of 42
for various combinations of objectives
to create a group of better designs
GANBI
 Genetic Algorithm-based
Boundary Intersection
Uses both approaches
to combine objectives
and iterates to find
optimal design
 ‘Convex hull’ is
combination of
objective optimizations

7 of 42
Normal
Experiment and Results

Optimization Goals:




Experiment:



Max PSS
Min PFS
Min CFIELD
Run GAMBI for 200 iterations with 4 normals with
100 designs at each iteration
Small sample
Hard to get specific values at given points in
Pareto set
8 of 42
Result Graphs




Larger sensor range = fewer sensors
Large number of short sensors = high PSS and high PFS
Small number of long sensors = low PSS and low PFS
If cost is large constraint, best results come from varying number of sensors (Fig. 3)
9 of 42
Conclusion and Future Work
When working with large scale sensor
networks, cost becomes a concern
 Using a Pareto Optimal Surface, we can
balance cost, sensor quality and quantity of
sensors
 Future work would add in new parameters to
the sensing model to account for non-uniform
distribution/environments

10 of 42
Space-Based Wireless Sensor Networks:
Design Issues
Vladimirova, T.; Bridges, C.P.; Paul, J.R.; Malik, S.A.; Sweeting, M.N.; , "Space-based wireless sensor networks: Design issues,"
Aerospace Conference, 2010 IEEE , vol., no., pp.1-14, 6-13 March 2010
VLSI Design and Embedded Systems research group, Surrey Space Centre, Department of Electronic Engineering, University of Surrey
11 of 42
Introduction

Satellite sensor networks apply concepts of terrestrial
sensor networks to space

Replacing group of sensing satellites by distributed
networked satellites increases science return per dollar

Research from Surrey Space Center aimed at space
weather missions in Low Earth Orbit (LEO)


Space weather associated with anomalies detected on
spacecraft

Spacecraft in LEO vulnerable when passing poles or South
Atlantic Anomaly (SAA)

Distributed, networked small satellite missions can study
impact of space weather phenomena (e.g. solar storms) on
Earth atmosphere and spacecraft
Space-Based Wireless Sensor Networks: Design Issues
Figure 1: Iridium LEO network

Distributed satellite system constellation scenario based on Flower constellation

Space based wireless networking based on Open Systems Interconnection (OSI) stack

System-on-a-chip (SoC) platform and agent middleware for distributed processing

Configurable inter-satellite link communications module for pico-satellites

Future applications and research for space-based wireless sensor networks
12 of 42
Mission Constellation

Space-based wireless sensor networks consist of
small satellite nodes flying in close formations

Single large expensive satellite not needed

Large number of small satellite nodes used instead


Inexpensive, mass producible
Perturbations reduce lifetime of satellite clusters

Pico-satellite constellations drift in and out of intersatellite link (ISL) length

Creates dynamic and often “disconnected” environment

Ad-hoc, autonomous distributed computing
system needed for collaboration

Flower constellation used

Geometric shapes formed to produce ‘flower’s with the
‘petals’ giving angular requirements of satellite positions

Low Earth Orbit (LEO) distributed mission feasible
Figure 2: Constellation Orbital
Characteristics and Applications
13 of 42
Mission Constellation

Flower constellation





Stable orbit configurations for micro- and nano-satellites
Applications: GPS missions, reconnaissance, two-way orbits,
science missions, planetary exploration
Axis of symmetry coincides with spin axis of Earth
All satellites have same orbit shape
Satellites equally displaced along equatorial plane
Figure 4: Flower Constellation


Figure 3: Satellite and Orbital
Properties for Flower Constellation
14 of 42
Research on Flower constellation in LEO

9 pico-satellites, ranges from 100-400km between nodes

Satellites drift together along equator, staying in formation
without maintenance

Promising for pico- (mass<1kg) and nano-satellites (mass<10kg)
Simulations using AGI’s High Precision Orbital Propagator
(HPOP) in Satellite Toolkit (STK)
Network Design

Spacecraft communications affected by orbital dynamics

Causes variable inter-satellite ranges, speeds, access

Investigated using Open Systems Interconnection (OSI) networking scheme

Functionality implemented in hardware/software

Figure 5: OSI Layers and Implementation Methods
15 of 42
Physical Layer

Radiation is inherent environmental hazard

Ground communications for pico-satellites in
VHF and UHF bands

During intense solar cycles, VHF signals can
be reflected back

GPS essential for orbit determination and
navigation; solar storms cause
synchronization errors

Models used to predict ionospheric
propagation
Network Design



Power resources limited aboard pico-satellites
Adaptive techniques used to optimize power utilization
Power variation modeled for receiving antenna for inter-satellite communication in
LEO circular polar orbits

Minimum at equator, maximum at poles

Data Link Layer





Figure 6: Power Variation with Respect to
Latitude in Southern Hemisphere
16 of 42
Multiple-access schemes used for
communications bandwidth sharing
Medium Access Control (MAC) used to
manage communication links
Long propagation delays, appropriate data
rates, forward error correction needed for
reliable space communications
Terrestrial IEEE 802.11 wireless standard
adopted for inter-satellite link design
IEEE 802.11 could be scaled from few
hundred meters to few hundred kilometers
in space
Network Design

Network Layer




Proactive and reactive topology schemes, must be capable of reconfiguration
Ad-hoc inter-satellite networking capability
Adaptable and redundant ground-link communication
Middleware tolerant to extreme mobility, intermittent connectivity, node loss

Figure 5: OSI Layers and Implementation Methods
17 of 42
Application Layer

Mission and payload dependent

High data-rate: client/server model

Low data-rate: peer-2-peer model

Consider future applications for distributed
operations, autonomy and artificial
intelligence

Data transmissions should be minimized to
reduce power overhead for communications
Distributed Computing Platform

Wireless transceiver operates in mobile environment

Hybrid software/hardware approach

IEEE 802.11 MAC layer time-critical functionality in hardware
with VHDL due to timing constraints, CRC coding used

For ease of reconfiguration, communication range prediction
done in software with LEON3 processor
Figure 8: MAC Layer Interface with Physical Layer
Figure 7: Wireless Transceiver Core Architecture
18 of 42

Direct Memory Access (DMA) core used for data
transfer between memory and wireless transceiver

MAC-Physical Interface

Appends info to packets: data type, modulation type, duration

Data rate of 6Mbps

Requires minimum DMA latency of 1.6us, achievable even in
heavy-loaded processing platform

Handshake mechanism required for synchronization between
DMA and MAC layer operation
Distributed Computing Platform


Java Co-Processor enables future distributed
computing and IP based networking capabilities

Accesses external RAM via AMBA2 bus

Multiple Instruction Multiple Data (MIMD) architecture
which fetches own instructions

Operates thread-level parallelism
Java Co-Processor pipeline stages



microcode fetch, decode, execute, additional translation
stage bytecode fetch
Hardware Exceptions

Stack overflow, null pointer, array out of bounds

Caused by processor overload or corrupt software

Stall processor, hard reset needed
Software Exceptions

Network exceptions, Application-specific exceptions

Caused by poor connectivity or programming errors
Figure 9: Java Co-Processor IP Core Wrapper
19 of 42
Distributed Computing Platform

Agent-Based Middleware with Instance Management for distributed operations

Code migration, parallel behaviors and data distribution services supported

Communications use TCP for High-Priority Data and UDP for Low-Priority Data

ProGuard, open source Java tool, used for shrinking, optimization, and obfuscation

Autonomous recovery from exceptions, expected (e.g. low-power) & unexpected (e.g. Single-Event Effects)

Figure 10: Instance Manager Thread Performing Soft Resets
20 of 42
Soft Reset Analysis

Topology reconfiguration tested with
unexpected connections/disconnections

Memory consumption increased with number of
networked nodes

Upon reconfiguration, instance is destroyed and
restarted under new conditions

Method calls needed for additional instance
increase, leading to higher memory usage

Agent instance information cost of 200KB per
node, plus 600KB for original instance
Configurable Inter-satellite Comm. Module


Figure 11: Inter-satellite Communications
Module Functional Block Diagram
21 of 42
Configurable communications module

Needed due to dynamic mobility and
communications channels

Commercial-of-the-shelf (COTS) components

Industrial Scientific and Medical (ISM)
frequencies employed

Software-Defined Radio (SDR) architecture
Key Requirements

Adhere to CubeSat design specifications

Support IEEE 802.11 specifications

Provide communications at variable data rates
and configurable waveforms

Provide link for ground communications

Provide independent beacon signal generator

Gather localization information for distance and
bearing angles
Conclusions


Space-based wireless sensor networks becoming more practical and advantageous
Surrey Space Center research presents design overview


Target LEO missions to monitor space weather phenomena
Flower constellation strategic for satellite networks



Orbital and network issues based on OSI layer stack






Hardware-accelerated wireless transceiver operates in mobile environment
Java Co-Processor for future fault-tolerance capabilities
Figure 12: EDSN CubeSat Swarm - NASA
Agent-based middleware for fault-tolerant networking design


Vulnerable to radiation in space environment
Uses terrestrial IEEE 802.11 wireless standard scaled to space
Proactive and reactive topology schemes, capable of reconfiguration
Application layer mission- and payload-dependent
Distributed computing platform employed in SoC design


All satellites have same orbit, 100-400km between nodes
Drift together along equator, stay in formation without maintenance
Instance management for distributed operation, autonomous exception recovery
Configurable inter-satellite communications module


Needed due to dynamic mobility of communications channels
Meets key requirements for networking and data transmission, low cost and power
22 of 42
Further Questions & Research


Future distributed spacecraft envisioned as
autonomous, small-sized, intelligent
Concept of satellite space sensor networks can
be applied to many missions





Realizing co-orbiting assistants
Continuous Earth coverage for remote sensing
Low-cost LEO communications
Continuous communications for remote lowpowered surface vehicles
Future Research Topics





Figure 13: Cubesat Deployment From ISS - SpaceRef
Flower constellation scale to various small satellite platforms and sizes
Alternative small satellite constellation scenarios
Terrestrial network communication issues adapting to space environment
Topology reconfig. overhead for various constellation and networking scenarios
Inter-satellite comm. tradeoffs between low-cost, low-power vs. performance
23 of 42
ESPACENET: A Framework of Evolvable and Reconfigurable Sensor
Networks for Aerospace–Based Monitoring and Diagnostics
Proceedings of the First NASA/ESA Conference on Adaptive Hardware and Systems (AHS'06)
T.Arslan, N.Haridas, E.Yang, A.T.Erdogan, N.Barton, A.J.Walton, J.S.Thompson,
A.Stoica, T.Vladimirova, K.D. McDonald-Maier, W.G.J. Howells
24 of 42
What is it?



ESPACENET is a proposed
framework for a satellite
constellation
Aspires to be flexible and
intelligent, viable alternative
to larger spacecraft
Motivations


Cost- many smaller satellites
vs. a single large spacecraft
Flexibility- multiple
coordinated nodes can react
and adapt to changing missions
25 of 42
Previous Work

Pico Beacons at Berkeley



Low power wireless transceivers
Can be organized into small networks
CubeSat platform developed by Stanford and California
Polytech


Standardized small satellite platform
Hardware and software platform readily
integrated with user instruments/payload
26 of 42
3 Parts of the ESPACENET Framework

Network Architecture


Hardware Architecture


How nodes are connected and communicate with each other
and outside the network
“guts” of the satellites, sensors and processing elements
Evolvable multi-objective algorithm controlling the
network

Algorithms for optimizing the network and
adapting to changing mission parameters
27 of 42
Network Architecture

3 level hierarchy

Pico satellites




Micro satellites (Mother Satellites)




Limited to 1kg
Carry sensors and instruments for the
mission
Coordinate with the mother satellite to
accomplish mission goals
Higher performance
Coordinate actions of the pico satellites
in its sub-orbit
Process and relay received sensor data
Ground Relay Satellites


28 of 42
Reconfigured mother satellite
Relinquishes control of pico satellites to
transmit to the nearest ground station
Hardware Architecture

Main goal is to have node level reconfiguration within the network




nodes can adapt and optimize in response to power consumption, latency, sensors, ect
Pushing for System on Chip design

Higher integration, smaller chip size

Lower power

Reduce latency between subsystems
Architecture centers around reconfigurable modules connected via AMBA bus

Filters

FPGA fabric

Communication modules
Overall function determined by payload
29 of 42
Evolving Control Algorithm

Multi-objective evolutionary algorithms (MOEAs)




System will autonomously optimize the system
Balanced between sensor, cluster, and overall network
optimizations
Criterion include power, reliability, security, ect
Modeled after process of evolution
30 of 42
Conclusions/ Future Work





Fault tolerance?
Lifetime of a ESPACENET system
Determining Ideal network size
Availability of system outside of Evolutionary
algorithms
It is a proposed framework and so case
studies of missions using it will be interesting
31 of 42
Development of a Satellite Sensor
Network for Future Space Missions
Vladimirova, T.; Xiaofeng Wu; Bridges, C.P.; , "Development of a Satellite Sensor Network for Future Space
Missions," Aerospace Conference, 2008 IEEE , vol., no., pp.1-10, 1-8 March 2008
VLSI Design & Embedded Systems research group, Surrey Space Centre, Department of Electronic Engineering,
University of Surrey
32 of 42
Introduction

Principles developed from the ESPACENET framework
are applied at University of Surrey


Test missions planned


Development of a robust satellite system with many sensor
nodes
Aiming to test many new technologies for space networking and
distributed computing
Adapts CubeSat as a platform to test a novel pico satellite
architecture
33 of 42
Space Mission

Targeting one of two launch opportunities



CubeSat Program
Surrey Satellite Technology Limited
Undertaken to test technologies


Adapting IEEE 802.11 for inter satellite communication
Distributed computing via 3 satellites





Collaborative image processing
EM measurements
Running MOEA to route signals
Secure processing for nodes/ network
SoC design with FPGA implemented controller
34 of 42
Pico satellite Design

System is designed as a payload to a cubesat



Compartmentalizing the design increases reliability
Main satellite controller is a commercial off the shelf
(COTS) MSP430
Leveraging the CubeSat kit allows for a focus on pico
satellite design
CubeSat development kit and pico satellite prototype
35 of 42
Pico Satellite Payload

Includes 3 hardware modules



Camera System
MEMS Antenna & GPS system
LEON3-based FPGA System




Image compression cores
Wireless MAC/PHY
Image encryption
Payload Computer


LEON3 Processor- SPARC V8 RISC architecture
Allows for algorithmic optimizations

36 of 42
MULT/DIV results
Satellite Sensor Network

Inter-satellite Links


Based on IEEE 802.11
standard
 Modified for range of more
than 1 kilometer
Need to modify timing in
order make system work
 Current timing constraints are for 300 meter maximum
SIFS = RxRFDelay + RxPLCPDelay + MacProcessingDelay + RxTxTurnaroundTime
SlotTime = CCATime + TxTxTurnaroundTime + AirPropagationTime + MacProcessingTime
DIFS = SIFS + 2 * SlotTime
AckTimeout =frameTXtime + AirPropagationTime + SIFS + AckTXtime + AirPropagationTime
37 of 42
Distributed Computing


Limited power and
processing constraints
keep from having on
master computation
satellite
Leverage a middleware
to manage computing
and distribute computing
load


Middleware abstracts out network and process management
Leverage concept of ‘Agent’ to abstract out roles
38 of 42
Simulation Results

Round trip delay parameters






Worst-case hardware switching
delay = 1.258 ns
No. of nodes = 3
MAC access delay = 2.049 ms
Service delay = 1 ns to 1 s
Propagation through free space of
3.33x10 5s c 2.99792458x108
WiFi (IEEE 802.1 lb) Variables:



No. of transmissions = 3
Packet sizes = 1500 of 2346 bits, Channels = 14
Image Size: 7507 x 6399 pixels, File size: 50.826 to 6.386 MB
39 of 42
Simulation Results

Network Throughput




Vary Agent size from 12 KB to 2.7 KB
Black is TCP
Red is RMI*
Not UDP
transport
*RMI = Remote Method
Invocation
40 of 42
Partial Run-Time Reconfiguration on FPGA

Payload computer implemented on SRAM-based
Field-Programmable Gate Array (FPGA)

FPGAs suitable for on-board small satellite systems

Shorter time to market, lower cost, reconfigurability

Partial run-time reconfiguration makes run-time changes
to select regions on chip; supported by Xilinx devices

Radiation in space disruptive to FPGA operation

Heavy ions from cosmic rays can deposit enough charge
to cause Single-Event Upsets (SEUs)



Reconfigurable SoC architecture of
payload computer

Upsets to SRAM configuration of FPGA can cause errors in routing and functionality of design

Partial run-time reconfiguration can mitigate SEUs by repairing areas affected by soft errors
Many FPGAs use hard cores such as BRAMs and multipliers, not only soft cores
Application-specific IP cores in development to serve satellite missions
Configuration bitstream of each SoC component stored on-board in Flash mem.
41 of 42
Conclusions & Future Work



Distributed image processing is a core application of the
satellite cluster
Network performance is optimized by a multi-objective
optimization algorithm
Use of an FPGA allows high performance data processing that
can be combined with distributed computing techniques



Partial run-time reconfiguration helps mitigate SEUs
Novel adaptations to IEEE 802.11 for usage between satellites
in space
High-performance FPGA device could enable fast on-board
processing results rather than send raw data to Earth

Can provide low-cost approach with distributed computing to
implement emergency response systems for detection and
monitoring from space
42 of 42