Metadata for Formatting
with Multiple Aspect Ratios
Graham Jones
National Association of Broadcasters
Chair – SMPTE S22 Image Formatting AHG
HPA Technology Retreat
February 24, 2006
1
Metadata for Image Formatting
• Joint initiative by SMPTE, ATSC, and CEA, to
produce an end-to-end system solution for image
formatting as related to aspect ratio
• Harmonized standards for format of necessary
metadata, metadata carriage, and recommended
practices for applications
• Work in progress – estimated completion mid2006
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
2
Issues
• Optimize display of the image for the television
viewer
– Multiple production aspect ratios
– At least two display aspect ratios
• Provide guidance for the format conversion process
– In the receiver and display
– In post and pre-broadcast
• Minimize changes to images created by the original
program producer
• Maintain critical image content
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
3
Image Aspect Ratios
4:3
14:9
16:9
>16:9
1.85
2.39
etc.
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
4
Displays
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
5
Displayed Images
16:9
4:3
4:3
16:9
4:3
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
6
Undesirable Results
Produce
Broadcast
Viewer Display
4:3
16:9 using pillarbox
4:3 using letterbox
“Postage stamp”
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
7
Undesirable Results
Produce
Broadcast
Viewer Display
16:9
4:3 using letterbox
16:9 using pillarbox
“Postage stamp”
Other undesirable images – stretched, squashed
or vital picture information lost due to cropping
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
8
Undesirable Results
Produce
Broadcast
Viewer Display
16:9
4:3 using letterbox
16:9 using pillarbox
“Postage stamp”
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
9
Image Formatting Metadata
•
•
•
•
AFD – Active Format Description
Bar Data
Pan-Scan Information
Individual specifications already exist in standards
– AFD: MPEG, ETSI/DVB, ATSC
– Bar Data: ATSC
– Pan-scan: SMPTE, MPEG-2 (also AVC and VC-1)
• Currently no carriage in HD baseband or MXF
• Limited guidelines on applications and use
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
10
US Network Distribution
NTSC
Analog
NTSC
Remote HD or SD
Network
Production
Release
& Post
Center
Venues Contribution
SMPTE
DTV
ATSC
Digital
Local
Station
ATSC
MPEG
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
CEA
11
AFD, Bar Data, and Pan-Scan Carriage
in Professional Environment
Vertical ANCillary data space
Horiz. ANCillary data space
• SMPTE standards in
preparation for carriage of
AFD, Bar Data, and Pan-Scan
in VANC of SDI and HD-DSI
for the studio environment
Active Picture Area
• These standards also define
KLV encoding for carriage in
MXF, AAF, etc.
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
12
DTV System Block Diagram
Video
Source Coding
& Compression
Audio
Service Multiplex
&Transport
Audio Subsystem
Audio
Source Coding
601 &video
(525/625)
Compression
Service
Multiplex
Router/MC Switcher
Studio/Master Control
Video Subsystem
Video
Transport
SMPTE 259M:
(SDI)
SMPTE 292M: 1080i/p & 720p (HD-SDI)
RF
Transmission
System
Channel
Coding
Modulation
Ancillary Data
Control Data
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
13
Active Format Description - AFD
• Four-bit code that defines the active image area and
protected image areas for various combinations of
aspect ratios
– AFD is independent of image resolution
• Intended to guide receivers and STBs in how to
optimize the displayed image
– Also applications in the post and pre-broadcast chain
• Recommended practice for use of AFD and Bar Data
now in preparation by CEA
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
14
AFD = ‘1000’
Full frame
Image is
the same
AFD Codes in a 4:3 Coded Frame
AFD = ‘1010’
Illustration of described format
16:9 (centre)
In 4:3 coded frame
Description
Active Format
AFD = ‘0010’
Box 16:9 (top)
Image with a 16:9 aspect ratio as letterbox at
the top of a 4:3 coded frame.
AFD = ‘1011’
14:9 (centre)
AFD = ‘0011’
Box 14:9 (top)
AFD = ‘0100’
Box >16:9 (centre)
Image w
alternate
Image with aspect ratio greater than 16:9 as
a vertically centered letterbox in a 4:3 coded
frame.
AFD = ‘1110’
16:9 (with alternate
14:9 center)
Image w
alternate
letterbox
Image is full frame, with an aspect ratio that is
the same as the 4:3 coded frame.
AFD = ‘1111’
16:9 (with alternate
4:3 center)
AFD = ‘1010’
16:9 (centre)
Image w
centered
Image with a 14:9 aspect ratio as letterbox at
the top of a 4:3 coded frame.
AFD = ‘1101’
4:3 (with alternate
14:9 center)
AFD = ‘1000’
Full frame
Image w
centered
Image w
alternate
letterbox
Image with a 16:9 aspect ratio as a vertically
centered letterbox in a 4:3 coded frame.
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
15
in a
AFD Codes in a 16:9 Coded Frame
AFD = ‘1010’
16:9 (with complete
16:9 image protected)
Imag
ratio
Illustration of described format
In 16:9 coded frame
Description
AFD
‘1011’
Image
with =
aspect
ratio greater than 16:9
(centre)
as a 14:9
vertically
centered letterbox in a
Active Format
AFD = ‘0100’
Box >16:9 (centre)
Imag
horiz
in a
16:9 coded frame.
AFD = ‘1000’
Full frame
AFD
‘1101’
Image
is full=frame,
with an aspect ratio
4:3
(with
alternate
that is the same as the 16:9 coded
14:9 centre)
frame.
Imag
alter
cent
fram
AFD = ‘1001’
4:3 (centre)
AFD
‘1110’
Image
with =
a 4:3
aspect ratio as a
16:9 (with
alternate
horizontally
centered
pillarbox image
14:9coded
centre)
in a 16:9
frame.
Imag
an a
fram
AFD = ‘1010’
16:9 (with complete
16:9 image protected)
AFD
‘1111’
Image
is full=frame,
with a 16:9 aspect
(with
alternate
ratio 16:9
and with
all image
areas protected.
Imag
an a
fram
AFD = ‘1011’
14:9 (centre)
Image with a 14:9 aspect ratio as a
horizontally centered pillarbox image
in a 16:9 coded frame.
4:3 centre)
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
16
Format Conversion with AFD
AFD code 1010
Letterbox 16:9 image vertically centered in the frame
SD 4:3
HD 16:9
UP
16:9 Letterbox
Postage stamp
Without AFD
(undesirable)
Full Frame
With AFD
(desirable)
With AFD, original image information is maintained throughout
conversion processes, while optimal viewing area is reached.
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
17
Image Aspect Ratio >16:9
AFD code 0100
aspect ratio >16:9
4:3
16:9
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
18
Bar Data
• Signaled with AFD 0100 (aspect ratio >16:9) to
indicate the exact width of the top and bottom bars
• May also be signaled with AFD 0000 (undefined)
to indicate the width of the bars for any other
non-standard active image area
• Bar Data signals one pair of bars only
– Top/bottom or left/right
– Not intended to signal a postage stamp - such
transmitted images should be discouraged
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
19
Bar Data Example
AFD code 0100 (aspect ratio >16:9)
1280 Luma Samples
26
118
1280 x 720
Progressive Frame
Line Number
(SMPTE 296M)
TOP BAR
2.40:1 ACTIVE IMAGE
(1280 x 534)
IN A 16:9 FRAME
653
745
534
Lines
720
Lines
BOTTOM BAR
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
20
Pan-Scan Information
• Used to signal how to extract a different aspect
ratio viewport from a source image
• In broadcast world intended for professional use
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
21
Pan-Scan Information
Source Center
SOURCE
Offset-H
V-Size
Offset-V
Viewport Center
H-Size
VIEWPORT
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
22
Pan and Scan
© Warner Bros.
Screen shot courtesy of Letterbox and Widescreen Advocacy Page
www.widescreen.org
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
23
US Network Distribution
Pan-Scan Application
NTSC
Analog
NTSC
Remote HD or SD
Network
Production
Center
& Post
Venues Contribution
DTV
Local
Station
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
ATSC
Digital
24
US Network Distribution
Pan-Scan Application
NTSC
Analog
Remote HD or SD
Network
Production
Center
& Post
Venues Contribution
DTV
Local
Station
ATSC
Digital
For MPEG links, Pan-Scan information is carried in
display extensions of the video sequence header
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
25
US Network Distribution
Pan-Scan Application
Remote HD or SD
Network
Production
Center
& Post
Venues Contribution
DTV
Local
Station
ATSC
Digital
For MPEG links, Pan-Scan information is carried in
display extensions of the video sequence header
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
26
Output Documents
• CEA – recommended practice for use of AFD and
Bar Data by receivers
• ATSC – minor revisions to A/53 for semantics of
AFD and Bar Data for consistency with CEA RP
• SMPTE standards for carriage of AFD, Bar Data,
and Pan-Scan information in VANC of SD and
HD video and as KLV data
• SMPTE – recommended practice for operational
use of AFD, Bar Data, and Pan-Scan information
– Contributions welcome
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
27
Related papers at NAB
Broadcast Engineering Conference
Adam Goldberg:
The Importance of Video Format Metadata
Bill Miller:
Canceling the Postage Stamp
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
28
Thank you
gjones@nab.org
Metadata for Formatting with Multiple Aspect Ratios – HPA – February 24, 2006
29