Video Fundamentals
September 9, 1999
Lawrence A. Rowe
University of California, Berkeley
URL: http://www.BMRC.Berkeley.EDU/~larry
Copyright @1999, L.A. Rowe
Outline
• History
• Broadcast System Elements
• Scanning and Interlace
• Color and Gamma
• Chroma Subsampling
• Representations
Multimedia Systems and Applications
2
History
• 1839: Daguerreotype Cameras
• 1893: Telephone Audio Broadcasting (Puskas)
• 1895: Wireless Communication (Marconi,
Popov)
• 1895: Film Presentation (Lumiere Brothers)
• 1919: Radio Broadcasting (Holland, Canada)
• 1934: US establishes FCC
• 1935: TV Broadcasting (Germany, Britain)
• 1941: US B&W TV
ADD: telephone invention, founding of ATT, tv inventions,
first radio networks/stations
Multimedia Systems and Applications
3
History (cont.)
• 1951: Videotape Recorder (Bing Crosby
Enterprises)
• 1953: US Color TV (NTSC)
• 1963: Geostationary Satellites
• mid ‘70s: Fiber Optic Transmission - cable
• 1985: FCC establishes ATSC - standard by 1993?
• 1989: Analog HDTV Broadcasting (Japan)
• 1996: ATSC Standard Adopted
ADD: first ntsc tv broadcasts, 1st cable system, consumer vcr,
tivoli/replay device, direct tv, first hdtv broadcasts
Multimedia Systems and Applications
4
Standards Groups
• ITU-T – ITU Telecommunications
Formerly CCITT
• ITU-R – ITU Radiocommunications
Formerly CCIR
• FCC
• SMPTE - Society of Motion Picture and
Television Engineers
… and many more!
Multimedia Systems and Applications
5
Receiver
Transmitter
Broadcast System
Goals:
1. Efficient use of bandwidth
2. High viewer perception of quality
Multimedia Systems and Applications
6
Scanning and Interlace
• Transmission is continuous signal
Transmitter captures images and encodes
Receiver decodes for display
• Image traced out line-by-line
Left-to-right top-down scanning
vertical/horizontal blanking interval
• Signal is interlaced
Improves perception of motion (?)
Alternative is progressive scanning
Multimedia Systems and Applications
7
Video Display Scanning
Amplitude
Cathode
Time
• Video composed of luma and chroma signals
• Composite video combines luma and chroma
• Component video sends signals separately
Multimedia Systems and Applications
8
Camera Operation
Beam
Splitter
G
B
Camera
Tubes
Luma
Encoder
Zoom
Lens
Color
Filters
R
Chroma
• Camera has 1, 2, or 3 tubes for sampling
• More tubes (CCD’s) and better lens produce
better pictures
Multimedia Systems and Applications
9
Direct View CRT
• Three guns (RGB) energize phosphors
Varying energy changes perceived intensity
Different energies to different phosphors
produces different colors
Phosphors decay so you have to refresh
• Different technologies
Shadow mask (delta-gun dot mask)
PIL slot mask
Single-gun (3 beams) aperture-grille (Trinitron)
Multimedia Systems and Applications
10
Aperture-grille –vs- Shadow Mask
R
G
B
Multimedia Systems and Applications
R
B
G
11
Scanning Notation
• Lines/”frame rate” specification
NTSC
PAL
525/59.94
625/50
• ATSC – everything is variable
1080i
720p
1920x1080 interlaced scanning
1280x720 progressive scanning
• Why 59.94?
Avoids interference problem between color
and sound subcarrier
Multimedia Systems and Applications
12
Interlaced Fields
Signal Format
Line 1 ---
vertical blanking
Line 21 ---
Raster Format
Field 1
1
2
3
485
Line 263 ---
vertical blanking
Line 283 ---
...
485
2
4
484
...
485
Field 2
ITU-R Rec. 601: 720x483
Line 525 --Multimedia Systems and Applications
484
13
Scanning (525/59.94)
Total Size
square
pixel
601
4fsc
Active
Active
Active
Multimedia Systems and Applications
Active Area
780X525
644X483
858X525
720X483
910X525
757X483
14
Aspect Ratio/Refresh Rate
• Aspect ratio
Conventional TV is 4:3 (1.33)
HDTV is 16:9 (2.11)
Cinema uses 1.85:1 or 2.35:1
• Refresh Rate
NTSC is 60Hz (59.94Hz)
PAL is 50Hz
Cinema is 48Hz (but still only 24 fps)
Multimedia Systems and Applications
15
NTSC Video
(525-lines, 60-fields/sec)
• 525 scan lines repeated 29.97 times per
second (i.e.33.37 msec/frame)
• Interlaced scan lines divide frame into 2
fields each 262.5 lines (i.e.16.68 msec/field)
• 20 lines reserved for control information at
the beginning of each field
Only 483 lines of visible data
Laserdisc and S-VHS display around 420 lines (perception)
Normal broadcast TV displays around 320 lines (“)
• Line lasts 63.6 usec(10.9 usec blanked)
Multimedia Systems and Applications
16
PAL Video
(625-lines, 60-fields/sec)
• 625 scan lines repeated 25 times per
second (i.e. 40 msec/frame)
• Interlaced scan lines divide frame into 2
fields each 312.5 lines (i.e. 20 msec/field)
• Approximately 20% more lines than NTSC
• NTSC vs. PAL  roughly same bandwidth
Multimedia Systems and Applications
17
Color Perception
• Color is perceived lightwave
400nm to 700nm received at retina
Humans more sensitive to brightness than color
• Retina composed of cones and rods
Cones respond to different frequencies (RGB)
Rods measure brightness at low light levels
(i.e., nightvision)
• CIE established standards for color
CIE XYZ, CIE xyY  Linear RGB
Multimedia Systems and Applications
18
Gamma
• CRT is inherently non-linear
Display changes based on voltage driving it
• Human vision is also non-linear
Lightness sensation is a power function of
intensity (y=xw)
• Serendipity happens…
Non-linear CRT close to inverse human lightness
Coding intensity into a gamma corrected
signal maximizes perceptual image
• Y -vs- Y’
Multimedia Systems and Applications
19
Gamma in Video
camera
transmission
display
• Camera performs gamma correction
• Display imposes inverse power function
y = x 1/0.45
Multimedia Systems and Applications
20
Color Image Coding
• Image represented by 24 bit pixel (8 bpp)
Each color value between 0 and 255
• Video uses non-linear coding
Uniform distribution of colors to codes
RGB  R’G’B’ (gamma corrected RGB)
• Video uses luminance/chromance
R’G’B’  Y’CBCR
Luminance is Y (technically luma is Y’)
Chromance is CBCR
Multimedia Systems and Applications
21
Jargon
• CBCR actually color difference signals
CB is scaled version of (Y’-B’)
CR is scaled version of (Y’-R’)
• Confusing terms: YUV, YIQ, Y CBCR,…
Scaled versions of <Y’, Y’-B’, Y’-R’>
Green has highest contribution to luminance
• Luminance –vs- Luma (i.e., Y –vs- Y’)
Y is linear luminance
Y’ is gamma corrected luminance (aka luma)
Multimedia Systems and Applications
22
Chroma Subsampling
• Chroma subsampling reduces data
2 chroma/luma  16 bpp
1 chroma/luma  12 bpp
• Notation is a:b:c
a is luma samples
b is chroma samples per odd line
c is chroma samples per even line
• Ex. 4:4:4, 4:2:2, 4:1:1, 4:2:0, …
Multimedia Systems and Applications
23
Line Sampling
Y
Y
Y
Y
Y
4:4:4
CR/CB CR/CB CR/CB CR/CB CR/CB
Y
Y
Y
Y
Y
4:2:2
CR/CB
Y
Y
CR/CB
Y
Y
CR/CB
Y
4:1:1
CR/CB
4:2:2 is referred to as broadcast quality
4:1:1 is referred to as VHS quality
4:2:0 is 2:1 down sampling in horizontal and vertical direction
Multimedia Systems and Applications
24
4:2:0 Sampling
Luma sample
Chroma sample
Multimedia Systems and Applications
25
Representations
• Composite
NTSC - 6MHz (4.2MHz video), 29.97 frames/second
PAL - 6-8MHz (4.2-6MHz video), 50 frames/second
• Component
Separation video (luma, chroma) - svhs, Hi8mm
RGB, YUV, YIQ, …
YCBCR - used for most compressed representations
• Separation video called “s-video”
Multimedia Systems and Applications
26
Analog Video Representations
• NTSC
Y = 0.299R + 0.587G + 0.114B
I = 0.596R - 0.275G - 0.321B
Q = 0.212R - 0.523G + 0.311B
composite = Y + Icos(Fsc t) + Qsin(Fsc t)
• PAL
Y = 0.299R + 0.587G + 0.114B
U = 0.492(B-Y)
Q = 0.877(R-Y)
composite = Y + Usin(Fsc t) + Vcos(Fsc t)
Multimedia Systems and Applications
27
Digitizing
• Analog TV is a continuous signal
• Digital TV uses discrete numeric values
Signal is sampled
Samples are quantized
Small, discrete regions are digitized
• Image represented by pixel array
Multimedia Systems and Applications
28
Image Sizes
160 352
120
720
800
1152
1280
1920
QSIF
(19Kp)
SIF (82Kp)
240
601 (300Kp)
483
SVGA (500Kp)
600
ATV (1Mp)
720
900
Workstation (1Mp)
HDTV (2Mp)
1080
Multimedia Systems and Applications
29
Workstation Images?
• [1 Mega Pixel] 1152 x 900 = 1,036,800 pixels
• [xvga] 1024 x 768
• [?]
1280 x 1024
Multimedia Systems and Applications
= 786,432 pixels
= 1,310,720 pixels
30
Digital Video Representations
• Digital Composite Video(D2/D3,SMPTE 244M)
142 Mb/s data rate, either parallel or serial
Subsampled color signals 4:2:2
• Digital Component Video(D1/D5,SMPTE RP125)
Maintain separate signals for luma and chroma
270 Mb/s data rate, either parallel or serial
Subsampled color signals 4:2:2
• Compressed Digital Video
MPEG, MJPEG, H.26x, DV, …
Multimedia Systems and Applications
31
Digital Video Block Structure
macroblock
• 4:2:2 YCBCR
Y1
Y2
CB1
CR1
16x16 macroblock
Y3 Y4
CB2 CR2
8x8 pixel blocks
8 bits/sample = 16 bits/pixel = 4Kbits/macroblock
• 4:1:1 YCBCR
3Kbits/macroblock
12 bits/pixel
Multimedia Systems and Applications
Y1
Y2
Y3
Y4
CB
CR
32
What is Video Data Rate?
• Digital
720x483 = 347,760 pixels/frame
4:2:2 sampling gives 695,520 bytes/frame
21 MB/sec (167 Mbs)
4:4:4 sampling gives 250 Mbs
• ATV (MPEG MP@ML)
1280x720 = 921,600 pixels/frame
4:2:0 sampling gives 1,382,400 bytes/frame
41 MB/sec (328 Mbs)
(Note: MPEG coded streams are 1.5-80 Mbs)
Multimedia Systems and Applications
33
What is Video Data Rate (cont.)?
• ATSC (720P)
720x1280 = 921,600 pixels per frame
4:2:2 sampling = 1,843,200 bytes per frame
24 fps = 44,236,800 bytes per second
 44 MB/s = 354 Mbs
• ATSC (studio 1080I)
1080x1920 = 2,073,600 pixels per frame
4:4:4 sampling = 6,220,800 bytes per frame
30 fps = 186,624,000 bytes per second
187MB/s = 1.5 Gbs
Multimedia Systems and Applications
34
Serial Digital Data Rates
• Serial digital is a video transport standard
Widely used in broadcast and production studios
• Data rates
NTSC SDI approximately 200 Mbs
ATSC SDI approximately 1.5 Gbs
• Routing Switcher
Remember typical facility might have a 256x256
routing switcher
What is data rate across the switch?
Multimedia Systems and Applications
35
Human Perception
•What is smooth motion
Depends on source material
Most action is perceived as smooth at 24 fps
•Human most sensitive
Low frequencies
Changes in luminance and blue-orange axis
•Vision emphasizes edge detection
Strong bias to horizontal and verticle lines
•Visual masking by large luminance changes
Multimedia Systems and Applications
36
Producing High Quality Video
• Need high quality camera
S-Video(SVHS, Hi8mm) better than composite
3 chips better than 1 chip
Digital better than analog
• Lights, lights, lights…
Experiment with filters to change apparent colors
• Shoot scene from different angles and cut
between them to create visual stimulation
• Study film/video techniques
Let person exit the scene without moving camera
Keep orientation of images correct
Change scene/shot to reflect time change
Multimedia Systems and Applications
37
Conclusions
• NTSC/PAL were excellent standards that
lasted over 50 years
Technology has changed dramatically during
this time
• Digital revolution is changing industry
Main impact is development of ATSC standard
• Internet revolution is also chaning industry
Webcasting –vs- mass market broadcasting
Wireless –vs- cable –vs- packet transmission
Multimedia Systems and Applications
38