Video Over DDS
DDS Information Day
Brussels, Belgium-June-26,2007
Ertan DENIZ
MilSOFT A.S, Teknokent ODTU,Ankara/Turkey
edeniz@milsoft.com.tr
Copyright © MilSOFT,Turkey
Huseyin Kutluca, hkutluca@milsoft.com.tr
UNCLASSIFIED
1
Outline
•
•
•
•
•
•
•
•
Introduction
Image Exploitation System for UAVs
Why Use DDS for Video Streaming?
MilSOFT DDS
DDS Topics
QoS choices
Future Work
Conclusion
Copyright © MilSOFT,Turkey
UNCLASSIFIED
2
Introduction
• DDS middleware
•
•
•
•
•
•
Location transparency with pub/sub mechanism
Redundancy and fault tolerance mechanisms
Multiple readers/writers
Transient and persistent services
Asynchronous, time-independent data distribution
Rich set of QoS, suitable for real-time systems
• Wide range of Application Areas
•
•
•
•
•
C4I
Industrial automation
Distributed control and simulation
Telecom equipment control
Sensor networks
• Use Case: Image Exploitation System for UAVs
Copyright © MilSOFT,Turkey
UNCLASSIFIED
3
DDS on UAV Image Exploitation System
DB Server
Raw Data
Recording
High Capacity
Storage Unit
Pre-processed &
Georecorded Data
Sensor Data
(EO/IR, SAR, GMTI)
Raw Data
Telemetry
Data Receiver
Screening
Copyright © MilSOFT,Turkey
DDS
based
Data
Streamin
g
Exploitation 1
UNCLASSIFIED
Pre-processor
Artifacts
Exploitation 2 All Data in DB
4
DDS on UAV Image Exploitation System
• Data
• Events
• User information
• Resource Management
• Radar data streaming
• SAR streaming
• GMTI streaming
• Telemetry streaming
• Video streaming
•
•
•
•
•
Live data streaming
Recorded data streaming
Data Publisher: DataReceiver
Data Subscribers: Consoles and Pre-Processor
Content of data MPEG-2 TS
Copyright © MilSOFT,Turkey
UNCLASSIFIED
5
MPEG-2 TS (Transport Stream)
• Elementary Stream
• Video, audio, telemetry data, etc...
• From each elementary stream Packetized Elementary Stream
(PES) is formed
• And each PES is broken into fixed-size Transport Packets
• TS packets are 188 bytes of which 4 bytes are for the header
and 184 bytes are for the payload
• Transport Stream can multiplex multiple elementary streams
• The TS packet header contains a Packet Identifier (PID) to identify
different elementary streams
• 8 Mbit data rate for UAV Image Exploitation System
Copyright © MilSOFT,Turkey
UNCLASSIFIED
6
Why use DDS for video streaming?
• Publish-Subscribe simplifies the development process
• The clients do not need to know where the server is, and the server
does not need to know the clients
• DDS specification implies mutlicast communication and the
interoperability wire protocol RTPS supports and recommends
multicasting
• Dramatically decreased bandwith usage with DDS implementations
which support multicast
• Quality Of Service attributes like reliability, durability, transport
priority directly can be used for video streaming channels
• Generally DDS implementations have very low latencies in
data transmission compared to other middlewares, which
makes it a good choice
Copyright © MilSOFT,Turkey
UNCLASSIFIED
7
Why MilSOFT DDS
• MilSOFT DDS is compliant with current DDS specification,
version 1.2
• Provides Minimum Profile QoS attributes, which are
neccessary for video streaming
• Completely decentralized design
• No dynamic memory allocation after initialization
• Zero-copy access to data
• Implements multicast communications
Copyright © MilSOFT,Turkey
UNCLASSIFIED
8
MilSOFT DDS Middleware
More information on http://dds.milsoft.com.tr
Copyright © MilSOFT,Turkey
UNCLASSIFIED
9
Video Over DDS Architecture
Copyright © MilSOFT,Turkey
UNCLASSIFIED
10
DDS Types and Topics Used For Video Streaming
• So how do we encapsulate MPEG2-TS packets into DDS
types?
• There are two DDS types and two corresponding Topics.
• VideoHeader  VideoHeaderTopic
• VideoPacket  VideoPacketType
• VideoHeader
• Contains the header of the video, which is needed to initialize the
decoder. (FPS, height, width, etc... )
• VideoPacket
• Contains MPEG2-TS packets
Copyright © MilSOFT,Turkey
UNCLASSIFIED
11
VideoHeader Topic
struct VideoHeader
{
long headerType; // key
sequence<char, HEADER_MAX_DATA> headerData;
};
• Published once for each Transport Stream
• Contains the necessary information
• initialize the video decoder
• information about the other multiplexed elementary streams
• Key to distinguish the video header from other header data
• Late joiners also get this topic
Copyright © MilSOFT,Turkey
UNCLASSIFIED
12
QOS for VideoHeader Topic
• VideoHeaderTopic is necessary to initialize decoding of the
video
• Header information is critical for clients
• Use Reliability QoS (RELIABLE)
• Both publishers and subscribers are Reliable
• Late joining viewers need this information
• Use Durability QoS (TRANSIENT_LOCAL)
• There are different instances of the header according to it’s
type. We need only the last update of each instance!
• History QoS is default, kind = KEEP_LAST, depth = 1
Copyright © MilSOFT,Turkey
UNCLASSIFIED
13
VideoPacket Topic
struct VideoPacketType
{
long frameNumber;
sequence<char, PACKET_MAX_DATA_SIZE> packetData;
};
• MPEG2-TS packets are too small in size (188 bytes).
• VideoPacket type sends multiple TS packets of a video frame
at each VideoPacket sample
• Decreases the overhead and increases the bandwidth utilization
• A frame can span multiple VideoPacket samples (fragmented)
• Each sample contains TS packets of only one frame
Copyright © MilSOFT,Turkey
UNCLASSIFIED
14
QOS for VideoPacket Topic
• For VideoPacketTopic we may need both Reliable and Best
Effort delivery, according to client needs
• Use Offered >= Requested semantics
• Server publishes with a Reliable writer ( Offer Reliable communication )
• Clients subscribe with either Reliable or Best Effort readers
• Data is not needed by late joiners
• Durability is Volatile
• VideoPacket Type has no key, but we can use History QoS for
buffering
• History QoS, kind = KEEP_LAST, depth value can be used as a buffer
(set it to more than 1)
Copyright © MilSOFT,Turkey
UNCLASSIFIED
15
Other Concerns And Solutions
• We don’t want out of sequence frames
• DDS already satisfies this, no out of order packets!
• We don’t want frames which are too late
• Use Lifespan QoS
• Decoding takes time
• Either read data from a DataReaderListener and immediately put into
the queue of another thread
• Use WaitSets
Copyright © MilSOFT,Turkey
UNCLASSIFIED
16
Future Work
• Implement advanced services like rewind, replay, fast-forward
etc...
• Network reservations
• Use RSVP (IntServ) to reserve bandwith
• Analyze different methods to give DDS video streams a
priority
• Use Transport_Priority QoS (not completely standard)
• More standardization is needed in this area
• Standardize Flow Control mechanisms in DDS for better
utilization of network bandwith
Copyright © MilSOFT,Turkey
UNCLASSIFIED
17
Conclusion
• Video over DDS is possible!
• DDS provides a good set of QoS attributes for video
streaming
• Performance requirements
• Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS
• 20 Mbit video streaming is possible with DDS
Copyright © MilSOFT,Turkey
UNCLASSIFIED
18
Thank You!
Copyright © MilSOFT,Turkey
UNCLASSIFIED
19