THE FUTURE OF AUDIO IN BROADCASTING,
A DTS PERSPECTIVE
ITU-R WORKSHOP
JULY 2015
-Fadi Malak
DTS, INC.
1
Information on this slide is confidential property of DTS. Any unauthorized copying is strictly prohibited. © 2015 DTS, Inc.
AGENDA
•
•
•
•
•
•
2
Future looking trends in broadcast
Key consumer value propositions of NGA
Opportunities & Challenges ahead
Solutions of today
MDA & DTS:X Workflows & Tools
Closing remarks
IMMERSIVE AUDIO – A RENEWED OPPORTUNITY
Emergence of immersive audio systems in cinema
Arrival of UltraHDTV: new experience, new premium service tier = new experience
New standardization activities: DVB, ATSC 3.0, ITU, FoBTV, SMPTE …
Service operators battle for technological leadership (1st to do PR) & battle to differentiate services
Opportunity for new standards, technology, and an equipment refresh, which includes codecs
Address audio issues that have plagued the broadcast industry
Enable new services and new experiences in broadcast sound
3
FUTURE LOOKING TRENDS IN LIVE PRODUCTION
•  Move toward IP transport for audio production
-  AVB; AES67/Ravenna; SMPTE 2022-6; SMPTE 337
-  Standardization is key for transport, encapsulation, & production formats
•  Smarter audio
-  More metadata
-  Timing info within audio ES
•  Production in-sourcing
-  Trend of more production assets moving from OB Truck to BOC
-  e.g. Commentator, graphics
-  Even mixing and director can move to NOC rather than field
•  More mic feeds
-  Player mics to become more common
-  Player tracking, and automated panning & ducking techniques being explored
-  Ongoing research on mic isolation / sound separation
4
MDA
ITU; ETSI; SMPTE
ADM
ITU; EBU
OPEN IMMERSIVE STANDARDS IN PRODUCTION-POST-DELIVERY
Emission
Codec
MDA
Capture
Play/
Record
* Architecture diagram courtesy of:
Mix
Live
Production
Edit/PostProduction
Deliver/
MCR
Quality
Check
Play Out/
Automate
Distribute
Encoding
& Mux
DVB-T
Broadcast
STB
RX
DVB-S
DVB-C
MM amp
Media Up
Live / Stream
OTT-TV
Display
File Transfer
Tablet
PC
Archive Store
MDA
Contribution
5
5
MDA
MDA
ADM
Interchange
Acquire
ADM
CDN
Publish
Emission
Codec
FUTURE LOOKING TRENDS IN LIVE PRODUCTION
•  Move toward IP transport for audio production
-  AVB; AES67; SMPTE 2022-6; SMPTE 337
-  Standardization is key for transport, encapsulation, & production formats
•  Smarter audio
-  More metadata
-  Timing info within audio ES
•  Production in-sourcing
-  Trend of more production assets moving from OB Truck to BOC
-  e.g. Commentator, graphics
-  Even mixing and director can move to NOC rather than field
•  More mic feeds
-  Player mics to become more common
-  Player tracking, and automated panning & ducking techniques being explored
-  Ongoing research on mic isolation / sound separation
6
FUTURE LOOKING TRENDS IN HOME DELIVERY
•  Next generation delivery pipes
-  Governments and standards defining the new pipelines
-  IP –based delivery at the heart of many
-  Hybrid delivery will be key for Next-Gen Audio services
•  More connected devices
-  Enabled via IP delivery
-  Easily upgradable architectures
•  Service providers blur the lines
-  More & more operators offering OTT access to services
-  OTT no longer limited to on-demand architectures, live is a reality
•  Models changing
-  Premium content offering direct access & subscriptions
-  PayTV offering services as OTT only to not subscribers
7
With hybrid delivery services, broadcasters can
afford to offer a much broader degree of
personalization features thanks to personalized ondemand access via streaming/IP
Apps on Smart phones, tablets, STBs, and connected
TVs, makes deploying service with new features &
codecs much more streamlined
More ways to get access to content.
More content accessible in easy to deploy services
More competition for eyeballs/ears = more reasons to
differentiate services
VALUE PROPOSITIONS FOR THE CONSUMER
AS WILL BE ENABLED BY NEXT-GENERATION AUDIO SYSTEMS
8
CONSUMER VALUE PROPS ENABLED BY NGA
•  Focusing on the consumer value propositions will be key for adoption of Next-Generation Audio technologies
•  The introduction and support of Object-Based audio in Next-Gen compression systems is a key enabler
Personal and completely customizable
Flexible, adaptive, scalable
Immersive over new speaker systems, sound bars, headphones
Lean forward interactive & social
9
THE CONSUMER EXPERIENCE
… as enabled by object-based audio
Audio has just become more personal
Complete personalized experience for each viewer
Customizable to individual preferences:
Ability to “change the mix”
Preferred commentator
Personalized sonic experience:
Dialog Control
Dynamics Control
Enhanced sonic experience for HI/VI
Dialog intelligibility enhancement
Improved Descriptive Video Services
Infinite choice of languages mixed via hybrid OTT
10
THE CONSUMER EXPERIENCE
… as enabled by object-based audio
Interactive and social
Applications that are truly engaging consumers
Ability to change every aspect of the broadcast mix
Access to highly desired never before offered Aux mic feeds for
sports fans
Enable sonic aspects of social TV & social networks
e.g. Skype/Twitter with your friends
A lean forward interactive experience via 2nd screen TV companion apps
Or an intuitive, familiar on-screen simplified UI via remote control
Interactive objects can arrive via different delivery paths (hybrid delivery)
and be transparently integrated into final presentation
11
THE CONSUMER EXPERIENCE
… as enabled by object-based audio
Immersion
What is immersive audio?
Elevation effects, realistic diffuse ambient sounds, and precise point
source placement
Immersive spatial audio rendering in consumer devices: e.g.
headphones*, sound-bars and TVs
* Immersive audio can already be realized in millions of already deployed next-gen
rendering technology, DTS Headphone X
Enable optimal spatial audio fidelity
Object based audio also improves fidelity
by avoiding down-mixing side effects
12
THE CONSUMER EXPERIENCE
… as enabled by object-based audio
Flexible, adaptive, scalable
Loudspeaker configuration agnostic and layout tolerant
Adapt to non-standard setups
Support legacy setups: mono, stereo, 5.1, or 7.1
Support next gen immersive home theater layouts:
5.1.4 (9.1), 7.1.4 (11.1), 10.2, 22.2, x.y.z
Support multiple sound-bars
Scalable to allow piece meal addition
(gradually grow your home theater systems)
Adapt to home environment: dynamically adapt to noisy settings
13
OPPORTUNITIES & CHALLENGES AHEAD FOR BROADCASTERS
14
OPPORTUNITY FOR BROADCASTERS
•  Produce a higher quality, next gen production
•
-  Premium experience can be monetized
-  Deliver an enveloping immersive experience
-  Deliver an enhanced experience for HI/VI
-  Enable a simplified consumer home setup
Deliver an engaging lean forward interactive experience for fans
-  Interactivity using OBA is coming via 2nd screen or iTV
-  Offer personalized listening experience like never before
-  Produce upsell assets for tiered service subscribers
-  Locked assets are an easy extension of existing OBA metadata + DRM
15
CHALLENGES
•  Co$t
-  An increase in production costs for managing extra mic feeds
-  Questions around ROI
•  Complexity
-  Managing 5.1 workflows & multiple mixes already complex enough
-  Managing immersive mix plus numerous objects = scary
-  Restricted truck space for immersive speaker monitoring
-  Loudness solutions for OBA?
•  Infrastructure requirements and Hardware upgrades
-  What has to be upgraded in order to make the move to OBA in Live?
•  Technology / Product / Solution readiness
16
BABY STEPS SOLUTIONS
•  Technology pioneers will break chicken & egg
-  Showcase the experience, wet consumer appetite, drive demand
-  The new experience everyone wants, should drive service providers to ask for it & pay for it
•  Contribution migration strategies being proposed
-  From additional SDI feeds completely separate from current
-  To replacement of SDI channels reserved for Alt mixes (e.g. Alt languages)
-  To all objects mix (production in-sourcing), rendered in BOC for Distribution needs
•  Mixing Presets & Preproduction:
-  Presets for Immersive bed mic groups & for certain static object positions
-  Similar concept to pre-authored interactive manifest (prior to live event)
-  Reserve dynamic on the fly mixing for certain high priority mic feeds/objects
17
BABY STEPS SOLUTIONS
•  Infrastructure requirements and Hardware upgrades:
-  What has to upgraded in order to make the move to OBA in Live production?
-  Mixing console replacement?
-  IP Based transport for all audio?
-  Contribution encoders/decoders?
-  Monitoring & Loudness?
-  Next-Gen emissions codecs for OBA?
•  How many A1s / A2s does it take?
-  Tests have shown it can be handled by one A1
-  Some will prefer two A1s in early days, depending on delivery strategy
-  Extra A2s for managing extra mic feeds
•  Solutions readiness?
-  Not everything has been reinvented yet
-  But solutions to deliver OBA over existing workflows available today
18
SOLUTIONS OF TODAY
19
DTS’ SOLUTIONS FOR OBJECT-BASED/IMMERSIVE AUDIO
MDA
Open, royalty-free, uncompressed, production format and standard
-
-
-
-
-
Supports immersive audio: Object-Based, Channel-Based, Scene-Based (HOA) content
Supports unrestricted number of channels and/or objects
Specification includes: Metadata + Bitstream + Renderer
DTS Stewardship: Specification, Reference Tools, Reference Renderer
Cross-industry harmonization in content creation standards (e.g. Cinema, Broadcast, film production for home delivery)
-  ITU-R
-  SMPTE
-  ETSI
DTS:X
A next generation codec solution
-  Efficiently carries immersive audio (or MDA) content to the home
-  Adaptable and scalable rendering to any loudspeaker playback configuration
-  Customizable, immersive, personal, and interactive
20
Key Features
•  Next-generation consumer delivery format & rendering technology
-  to be submitted to relevant standards dealing with emission codec such as:
-  DVB; FoBTV; BDA; SCSA; SMPTE; DLNA; SCTE; and other toolbox codec standards
-  Codec Profiles targeting different applications & devices (e.g. Blu-ray vs Broadcast)
-  Key features:
-  Support for Immersive playback and rendering
-  Support for flexible combination of channels and objects
-  Efficient compression from Low bitrates to Lossless
-  Object-based personalization and interactive feature support
-  Backwards compatibility
-  Speaker layout agnostic, offering adaptability to sub-optimal setups
-  DTS Headphone X integrated solution for mobile
•  Encoding tools already available, being integrated and deployed in content delivery chain
•  Decoders: SDK available, already deployed in AVRs
-  15 brands committed to support DTS:X in their 2015 model year AVRs representing >90% of AVR market penetration
21
DTS:X CROSS-INDUSTRY SOLUTION
MULTIPLE CODING ENGINES, ONE COMMON FLEXIBLE BITSTREAM & METADATA FRAMEWORK
22
BROADCAST ECOSYSTEM: WHERE THINGS FIT IN
MDA
TV Studio Production
& Live Contribution Feeds
, AC-4, MPEG-H
Broadcast Head End
Real-time
Encoding / playout
BOC/MCR
VOD
ingest
Studio film
post-production
23
Consumer Ecosystem
DTS’ SOLUTIONS FOR PRODUCTION-POST-DELIVERY
MDA
DTS:X
24
Production Format
Coding & Rendering
MDA & DTS:X
WORKFLOWS
&
TOOLS
25
MDA POST PRODUCTION DAW TOOL EXAMPLE
26
CINEMA TO HOME WITH MDA & DTS-X
DTS:X Tools
MDA Creator
Plug-in
DTS:X Encoder Tool
Elements
Dialog, Music & Effects
Mix
(DME)
Home or “Near field”
Mix
Encode for DTS:X
QC DTS:X audio w/
video
Post-encode bit
stream modification
MDA
D-Cinema Tools
Cinema Print Master
Digital Cinema
Package
27
DTS:X MediaPlayer
DTS:X Stream Tools
MDA Tools
Author for Blu-ray
via partner tools
DTS BROADCAST WORKFLOW
Let’s take a closer look
28
MDA TOOLS FOR BROADCAST PRODUCTION
29
MDA TOOLS FOR BROADCAST PRODUCTION
30
DTS BROADCAST WORKFLOW
Let’s take a closer look
31
LIVE PRODUCTION WORKFLOW: A CLOSER LOOK
(an existing legacy live console + side car approach)
Mic Feeds
Existing Live Console
L
R
C
LF
Ls
Rs
Console Control
protocol
(Ethernet)
Sub-mix
elements
Monitor
switching
MDA monitor in
MCR/NOC
Console Sidecar
Mono/Stereo Objects
(MADI)
5.1 Bed(s)
(MADI)
MDA translate
MDA parser
MDA/SDI
Objects & Metadata
monitoring
& syntax check
MDA Panning
&
Realtime
OBA mix
creation
Loudness
monitoring
MDA Render
•
•
•
•
•
MDA / SDI
11.1
9.1
5.1 alt language
Stereo
Mono objects
Lt/Rt Downmix
Objects & Metadata
monitoring
& syntax check
5.1 RFO
Loudness
monitoring
Stereo RFO
Live Console
32
Return Channels
Rendered 5.1/Stereo
(MADI)
Loudness
monitoring
5.1 Render From Objects (RFO)
Stereo RFO
Console Sidecar
Optional immersive
speaker setup or headphones
Receiving/Monitoring/
Rendering unit
Live Contribution link
Live Production
Consumer playback
Real-time encoding for Emission
DTS:X
Live Mic feeds
Professional
Realtime Emission
video Encoder
hardware
SAT
uplink
Contribution
Video Encoder
Video delay
path
DTS-C
SDI
Live Mixing Console
w/ support for MDA
DTS-C pass-thru
MDA à DTS-C
Standalone Contribution
audio encoder
QPSK
Modulator
4:2:2 Video+DTS-C
MPEG-TS
QPSK
De-Mod
MDA->DTS:X
IRD
HEVC video+
DTS:X
MPEG-TS
DTS-C pass-thru
DTS-C
/SDI
IP/ASI
Modulator
Broadcast
Network
MDA
/SDI
HEVC video
+DTS:X/
MPEG-TS
DTS-C à MDA
Standalone Contribution
audio decoder
Uncompre
ssed video
+ DTS:X /
HDMI
Mix audio objects in 3D space,
Monitor metadata & loudness,
Create MDA mix, output over SDI
4:2:2 Video+DTS-C
MPEG-TS
(Ericsson/Harmonic)
12 speaker feeds of audio
MDA
33
Uncompressed video/ HDMI
7.1.4 Immersive system
CONCLUSION
•  Lots of industry activity happening
•  Proponents building out solutions
•  Standards being defined
•  Broadcast products being refreshed
•  Down to earth summary:
•  While many things possible already today, some broadcasters are still hesitant to dive in
•  Realistically you can just start by adding one static dialog object, with restricted loudness
control
•  Satisfy many of today’s uncontested value props in broadcast
34
THANK YOU
FADI.MALAK@DTS.COM
Corporate Strategy & Development
DTS, INC.
35