Ensuring Reliable Networks
Advantages of Deterministic
Ethernet for Space
Applications
Space Flight Software Workshop 2013
Christian Fidi, Product Manager Space Products
Christian.Fidi@TTTech.com
December 11th, 2013
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Ethernet a Worldwide Standard
Ensuring Reliable Networks

Worldwide used, cross industry with a
strong growth in embedded systems

IEEE802 is an open and well defined
standard

Supports different speeds and topologies

Well defined network stack ISO/OSI

Low cost COTS Ethernet equipment
available

Robust physical layer with future
enhancements e.g. BroadR-Reach
(100Mbps with 2-wire twisted-pair)

Standardized interface to the physical layer

Engineers learn about Ethernet in schools
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 2
Ethernet = Unsynchronized
Communication
NIC
Ensuring Reliable Networks
NIC
NIC
SW
ITC
H
SW
ITC
H
X
X
NIC
NIC
NIC
XIC
N
NIC
SW
ITC
H
NIC
Asynchronous Communication
 Transmission points in time are not predictable
 Transmission latency and jitter accumulate
 Number of hops has a significant impact
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
NIC
NIC
Page 4
Motivation for Time-Triggered Ethernet
Statically Configured Communication
Performance guarantees:
real-time, dependability, safety
Standards:
ARINC 664, ARINC 429, TTP,
MOST, FlexRay, CAN, LIN, …
Applications:
Flight control, powertrain, chassis,
passive and active safety, ..
Validation & verification:
Certification, formal analysis, ...
High cost
Ensuring Reliable Networks
Free-Form Communication
No performance guarantees:
“best effort” plus some QoS
Standards:
Ethernet, TCP/IP, UDP, FTP,
Telnet, SSH, ...
Applications:
Multi-media, audio, video, phones,
PDAs, internet, web, …
Validation & verification:
No certification, test, simulation, ...
Low cost
Integration of functions from both worlds requires
a communication platform supporting both worlds
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 5
TTEthernet – Big Picture
Ensuring Reliable Networks
TTEthernet = combination on the same network of
IEEE802.3
SAE AS6802
ARINC664p7
• best effort Ethernet
• asynchronous
• synchronous
• no performance
guarantee
• jitter < 500 ms
• jitter < 1 ms
• latency typical 1-10 ms
• latency < 12.5 ms/switch
(1 GBit/s Ethernet)
• TTTech AFDX licensee
• very tight control loops
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 6
TTEthernet Clock Synchronization
Time Master
Ensuring Reliable Networks
E
TT
E
TT
E
TT
TT
E
IN 1
IN 1
TT
E
Eth
E
TT
IN 1
IN 1
TT
E
Time Master
TT
E
E
TT
IN 1
E
TT
Time Master
IN 1
E
TT
Eth
15
Fault-tolerant synchronization services
are needed for establishing a robust
global time base
www.tttech.com
88
15
88
Eth
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 7
Term: Permanence
Ensuring Reliable Networks
Frames in a switched network can have different transmission delays. It is possible that
receive order is different to the transmit order.
Example:
• frame F1 is transmitted by node A at 10:00
• frame F2 is transmitted by node B at 10:05
• frame F1 has a transmission delay A  C of 0:20
• frame F2 has a transmission delay B  C of 0:05
• receiver C sees: frame F2 arrives at 10:10, then F1 arrives at 10:20
In a TTEthernet network, frame F2 is said to become “permanent” when it is certain
that no frame F1, which was transmitted at an earlier point in time than F2, will be
received anymore. TTEthernet needs to know when certain frames become permanent
to run synchronization algorithms.
B
F1
F2
10:05
10:10
A
10:00
www.tttech.com
10:20
C
Comp
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 8
Permanence of PCFs
Ensuring Reliable Networks
Using the transparent_clock value, a receiver can determine the “earliest safe” point in time when a PCF
becomes permanent:
permanence_delay = max_transmission_delay – transparent_clock
permanence_point_in_time = receive_point_in_time + permanence_delay
Example:
• max_transmission_delay in this network is 0:30
• frame F1 is transmitted by node A at 10:00
• frame F2 is transmitted by node B at 10:05
• frame F1 has a transmission delay A  C of 0:20. This is visible in F1’s transparent_clock
• frame F2 has a transmission delay B  C of 0:05. This is visible in F2’s transparent_clock
• receiver C sees: F2 arrives at 10:10, becomes permanent at 10:10 + (0:30 - 0:05) = 10:35
• receiver C sees: F1 arrives at 10:20, F1 becomes permanent at 10:20 + (0:30 - 0:20) = 10:30
 F1 becomes permanent before F2
B
F1
F2
10:05
10:10
A
10:00
www.tttech.com
10:20
C
Comp
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 9
Mixed-Criticality Architecture
Ensuring Reliable Networks
Gateway
GPS
IEEE1588
Standard Ethernet
SpaceWire /
SpaceFiber
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 10
Time-triggered Traffic Timing
Ensuring Reliable Networks
Full control of timings in the system.
Defined latency and sub-microsecond jitter
I’ll expect M
between 11:05
and 11:15
I’ll accept M only between
10:40 and 10:50
I’ll accept M
only between
10:55 and
11:05
M
M
M
I’ll forward M
at 11:00
I’ll transmit M
at 10:45
M
I’ll forward
M at 11:10
…a switch
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Let’s see if I
can receive M
Page 11
Extensions & Standard Ethernet
Ensuring Reliable Networks
Time-triggered extensions for
standard switched GigabitEthernet
• Startup
• Recovery
• Robust fault-tolerant
distributed clock
Makes Ethernet viable for safety-critical distributed applications!
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 12
TTEthernet Traffic Partitioning
Ensuring Reliable Networks
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 13
Page 13
“System of Systems” Fusion
Ensuring Reliable Networks
Priority 1
DB
Server
I/O
DVR
Router or
Gateway
C
V
SBC
TT
Flight Computers
time-triggered
SBC
SBC
I/O
DB
Display
Server
SBC
SBC
I/O
SBC
I/O
DVR
Router or
Gateway
C
V
COMM
SBC
TT
COMM
Flight Computers
SBC
SBC
I/O
I/O
Display
I/O
I/O
SBC
Multi-function
I/O
SBC
C
I/O
SBC
COMM
COMM
I/O
DB
I/O
I/O
Server
Multi-function
I/O
I/O
DVR
Router or
Gateway
C
V
C
SBC
TT
Flight Computers
SBC
SBC
I/O
Display
SBC
SBC
I/O
SBC
COMM
COMM
I/O
I/O
I/O
Multi-function
I/O
C
DB
Server
I/O
DVR
Router or
Gateway
C
V
SBC
TT
Flight Computers
Priority 2
SBC
SBC
I/O
Display
SBC
SBC
I/O
SBC
COMM
COMM
I/O
I/O
I/O
Multi-function
I/O
C
SoS architecture with TTEthernet
supports reconfiguration
Several separate vehicles or
elements fuse into a new combined
network configuration
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 14
Complexity Example:
Synchronous vs. Asynchronous
Ensuring Reliable Networks
Active standby avionics system model with three components…
• Synchronous model: 185 reachable states (~2x102)
• Asynchronous model & communication with no latency: >3x106 states
• Asynchronous model with varying communication
latency: The number of
reachable states could not
be calculated with 8Gb RAM…
>108-1010
???
The number of system
states in an integrated
systems can be very
high…
And this is still a
relatively simple
system…
https://www.ideals.illinois.edu/bitstream/handle/2142/17089/pals-formalization.pdf?sequence=2
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 15
Distributed IMA: 653 OS + TTEthernet
Ensuring Reliable Networks
Expanding "time/space partitioning" into "time/space/bandwidth partitioning"
Enabler:
Networking
Technology
IMA (Module-level
Distributed IMA
(System-level partitioning)
time/space partitioning)
=
Criticality
= Mixed-Criticality Systems
Critical Systems
(DO-254/DO-178B Level A-C)
Critical Functions
Audio/Video/Voice/Multimedia
Payload Data w. Distributed processing
Internet, LAN, Non-Critical Systems
(DO-254/DO-178B Level A-C)
Non-Critical Functions
Time & Space Partitioning
Time, Space and Network Partitioning
Time + Space + Communication Partitioning
for each
Module (653
OS) & TTEthernet
(VxWorks653
+ TTEthernet)
Time+Space Partitioning
for each
Module (653 OS)
(VxWorks653)
Host Processor
P
1
P
2
t1
t2
...
Host Processor
P
n
P
1
P
2
tn
t1
t2
www.tttech.com
Distributed MixedSystems
...
P
n
tn
Distributed Embedded Computing System
P
1
P
2
t11
t21
...
P
n
tn1
Copyright © TTTech Computertechnik AG. All rights reserved.
...
P
1
P
2
t1n
t2n
...
P
n
tnn
Page 16
Synchronous Alignment:
Resource Use & Complexity Reduction
Ensuring Reliable Networks
Maximize use of network bandwidth and computing resources for
critical embedded functions
•
Ensure unambiguous design of key system interfaces
•
Reduce uncertainity, jitter and unintended system states (prevents system state
explosion)
Improve functional alignment (and separation!)
•
Simplified sensor fusion and distributed processing
•
Simplified redundancy management
•
Minimize software complexity / simplify functional alignment
Software
Application
Middleware /
Platform
Abstraction
Software
Application
Middleware /
Platform
Abstraction
Software
Application
Middleware /
Platform
Abstraction
Asynchronous Communication
www.tttech.com
Software
Application
Software
Application
Software
Application
Middleware /
Platform Abstraction
Middleware /
Platform Abstraction
Middleware /
Platform Abstraction
Synchronous Communication
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 17
Architecture Level Approach
APP
APP
APP
• Bandwith to Memory
Partitioning mapping at
E/S based on VLs
RTEMS
Linux
OS
• Redundancy
management
Ensuring Reliable Networks
Bandwith Partitioning at
Switch Level
TSP OS
Mem
Mem
Mem
Strong Partitioning (TSP + TTEthernet)
• Bandwidth partitioning at the network level
• Bandwidth partitioning supported at the switch and E/S level
• Memory partitioning at the E/S Level
• Bandwidth to memory mapping at the E/S based on virtual links
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 18
TTEthernet COTS Products
Rugged Hardware
•
TTESwitch
•
TTEPMC
Development Systems
3U VPX Rugged
Card Rugged
Development Equipment
Switches
 TTEDev
Switch 100 Mbit/s A664
 TTESwitch
 TTEPMC
 TTEXMC
1 Gbit/s Lab 24 Ports
Card, TTEPCI Card
Card,
TTEPCIe
Card
Test and Simulation Equipment
•
TTEMonitoring
•
TTEEnd
•
TTEDev
System 1 Gbit/s v2.0
•
TTEDev
System 1 Gbit/s for VxWorks 653
Configuration & Verification Tooling
Switch 1 Gbit/s 12 Ports
 TTEDev
E/S
Ensuring Reliable Networks
Switch 1 Gbit/s 12+1 Ports
•
TTEBuild
•
TTELoad
•
TTEView
•
TTEVerify
(certification RTCA DO 178B)
Embedded Software
•
TTEDriver
•
TTECOM
Layer ARINC 653
•
TTESync
Library
and TTEAPI Library
System A664 Dev & Test
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 19
Page 19
Space Product
www.tttech.com
Ensuring Reliable Networks
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 20
Chip Product Roadmap
Ensuring Reliable Networks
TTEthernet Space IP Variants
• For rad-tolerant/hard FPGA
“Pluto” Space IP
• 2x 10/100 Mbps (small footprint IP)
PT
SR
NIC Space IP
Switch/End System ASIC
• 3x 10/100/1000 Mbps MAC
PT
SR
PT
SR
Switch Space IP
• 12x 10/100/1000 Mbps
2012
2013
AVAILABLE
www.tttech.com
2014
UNDER DEVELOPMENT
ENVISAGED
• Rad-hard ASIC
• 3x 10/100/1000Mbps End System
• 10x 10/100Mbps + 6x 10/100/1000Mbps
• Management CPU
• RGMII Interface
PT
2015
PT Prototype
SR
2016
PS Preseries
Copyright © TTTech Computertechnik AG. All rights reserved.
SR Series
Time
EOL End of Life
Page 21
Flight Hardware Products
Ensuring Reliable Networks
TTEthernet RTU
• Rad-tolerant
COTS HW using Pluto IP
PT
TTEthernet PMC Card
• Space Qualified TTEthernet PMC Card
• FPGA based (designed for ASIC)
TTE-PMC/XMC Card Space
PT
PS
SR
TTEthernet Switch Assembly
• Space Qualified TTEthernet Switch
• FPGA based (designed for ASIC)
TTE-Switch 12 Port 10/100/1000 Mbps Space
PS
PT
2012
2013
AVAILABLE
www.tttech.com
2014
UNDER DEVELOPMENT
ENVISAGED
SR
2015
PT Prototype
2016
PS Preseries
Copyright © TTTech Computertechnik AG. All rights reserved.
SR Series
Time
EOL End of Life
Page 22
System States and Complexity
Ensuring Reliable Networks
Sensors/
Actuators
Sensors
Actuators
I
II
III
IV
I
II
III
IV
Module 1
Module 2
Module (N)
Comm.
Controller
Comm.
Controller
Comm.
Controller
Expected System
States and Transitions
I
I
I
II
II
III
IV
II
III
IV
III
IV
! System before integration !
Verified Module
States and
Transitions
Verified Module
States and
Transitions
Verified Module
States and
Transitions
SYSTEM INTEGRATION TECHNOLOGY
(ETHERNET NETWORK)
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 23
Complexity
Ensuring Reliable Networks
System integration technology can reduce
complexity
Sensors/
Actuators
Sensors
Program Risks, Schedule and
Cost Overruns, More errors !!
Actuators
I
I
I
II
I
II
II
II
III
III
III
IV
III
IV
IV
IV
I
I
II
II
I
III
II
IV
Module 2
Module (N)
Comm.
Controller
Comm.
Controller
Comm.
Controller
II
IV
III
Module 1
I
III
IV
III
IV
Expected System
States and Transitions
(after integration)
Unintended
System States and Transitions
I
I
I
II
II
III
IV
II
III
IV
III
IV
SYSTEM INTEGRATION TECHNOLOGY
(ETHERNET NETWORK)
! System after integration !
www.tttech.com
Verified Module
States and
Transitions
Verified Module
States and
Transitions
Copyright © TTTech Computertechnik AG. All rights reserved.
Verified Module
States and
Transitions
Page 24
Robust TDMA Partitioning
Ensuring Reliable Networks
Robust TDM network bandwidth partitioning
• Distributed fault-tolerant timebase
• Enforcement of prescribed communiciation schedule
• Defined low latency and minimal jitter enable precise "slicing" of
network bandwidth and communication resources
xing
ultiple nication)
M
l
a
ic
u
t
Statis ined comm
a
r
t
-cons
(Rate
TDM
www.tttech.com
ition
part
e
l
y
A st
ining
Copyright
Copyright
© TTTech
© TTTech
Computertechnik
Computertechnik
AG.
AG.
All All
rights
rights
reserved.
reserved.
Page 25
Page 25
System Integration
Ensuring Reliable Networks
Impacts module and sub-system design
in all lifecycle phases:
• Software design
• Testing
• Certification
• Maintenance
• Upgrades/extensions
• Reuse/redesign
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 26
Orion‘s Virtual Backplane
Ensuring Reliable Networks
"We look forward to realizing the potential of
TTEthernet technology development, which provides
a high bandwidth avionics databus capability
supporting future technology insertion.“
NASA statement
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 27
Page 27
Avionic-X Demonstrator
Ensuring Reliable Networks
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 28
Page 28
Strategic ECU Programs
with AUDI since 2011
Ensuring Reliable Networks
Advanced Chassis Control
(integrated in front axle) and
Advanced Driver Assistance
Computing Platform
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 29
Standardization: TTTech working with
partners to drive Deterministic Ethernet
standard across industries
Working with
Honeywell, NASA
and other aerospace
partners on SAE
standardization of
Deterministic
Ethernet for
Aerospace
(SAE AS6802)
Working with
Cisco and
IEEE community on
802.1 TSN
standard
Working with
Audi/Volkswagen and
other European,
American and
Asian OEMs on
Automotive
Ethernet
(Deterministic
Ethernet)
Aerospace
SAE
Standardization
IEEE
Standardization
Industrial
www.tttech.com
Ensuring Reliable Networks
Crossindustry
standard
Automotive
Ethernet
Automotive
Copyright © TTTech Computertechnik AG. All rights reserved.
Page 30
Ensuring Reliable Networks
www.tttech.com
www.tttech.com
Copyright © TTTech Computertechnik AG. All rights reserved.