Digital Media
Dr. Jim Rowan
ITEC 2110
Video Part 2
Roll call
Barton, Paul H.
Bois, Lauren C.
Bonds, Allison E.
Duncan, Jarred T.
Lawson, Joseph I.
Mulongo, Julio B.
Pennison, Heather L.
Reilly, Daniel J.
Sanchez-Casas, Jon F.
Simson, Davis
Sinnock, Grant A.
Swaim, Mark S.
Tran, Dung Q.
Vyas, Anand A.
Woldeyohannes,
Tesfamichael
Roll call
Jones, Crystal L.
Marsh, Kerreen A.
Thompson, Daniel G.
Tran, Christopher V.
Digital Video Standards
• Even though digital video COULD be
much less complicated... it isn’t
because...
• Backward compatibility requirement
– new equipment must create signals that
can be handled by older equipment
• Originally... TV signals (analog) needed
to be converted to digital format
Digital Video Standards...
• Looking at NTSC and PAL are ANALOG
standards
– Each has a screen size and frame (or refresh) rate
– Each define a number of lines on the screen (that
can be easily used for one the Y dimension)
– But what about the other, the X?...
– Each line is a continuous (analog) signal which
has to be converted to digital...
• How do you do that?
– SAMPLE the analog data!
– But directly sampling for each pixel results in a
data stream of 20 Mbytes/ second... HUGE!
Sampling for the Y component
• To reduce the data stream you can consider
human vision again
• Human eyes are less sensitive to color
changes than luminance
• Fewer samples taken for color than
luminance
– Sub-sample for color
– 4:2:2 (Y’CbCr)
– 4:1:1 used for many DV formats
Color representation in video
based on the de-facto TV
standard for CRTs
• To show a color pixel you need Intensity & R, G & B
• To reduce the data stream (for video size reduction
– designers realized that Green contributes the most to
intensity, Red is next and Blue hardly contributes anything to
intensity
– decided to use a formula for luminance
• Y = 0.2125R+0.7154G+0.0721B
Color representation in video
based on the de-facto TV
standard for CRTs
Y = 0.2125R+0.7154G+0.0721B
There is a use for algebra!
– With this formula you can derive any color
(RGB) if given Y and two of the colors then
calculate the Green
– Why transmit Y and Red and Green and
Blue if you can achieve the same thing with
Y’ & R & G?
• This is the Y’CbCr model for CRTs
NTSC & PAL weirdness
• NTSC & PAL
–
–
–
–
Define different screen sizes
Define different frame rates
Both have the same aspect ratio of 4:3
BUT... they each are digitized (through sampling)
to the same screen size
• The result?
– The pixels are not square
– PAL is taller than it is wide
– NTSC is wider than it is tall
DV and MPEG
• DV and its different forms (DVCAM &
DVPRO)
– use the same compression algorithm (5:1)
– use the same data stream (25Mbits)
– use 4:2:2 sampling where DV uses 4:1:1
• MPEG-1 originally meant for Video CD
– never got very popular
– developed into a family of standards
• MPEG-4 rose from the ashes
– used for iTunes video
and now for something
completely different!
everything we have done so far results in a
smaller video stream... but it is still too big so
what do you do?
Video Compression!
Video Compression
• Two ways to do this
– 1) Individual images can be compressed
using the techniques discussed in the
bitmapped section
• Spatial compression
– 2) Sequences or subsequences of frames
can be compressed...
• Temporal compression
Computational Irony
• Digital has been touted as a way to create
exact copies while analog (VCR) cannot...
• Analog VCR suffers from generational loss
• BUT only if you use video compression that
is... LOSSLESS
• AND... you guessed it, a lossless video
compression technique is not used because
the lossless ones don’t compress enough
Lossless compression
Original
compression
routine
Exact
duplicate
Original
compressed
original
decompress
routine
Lossy compression
Original
compression
routine
Changed
Original
compressed
original
decompress
routine
Changed
Original 2
Video Compression
• Coder/Decoder - Codec
• encodes and decodes video
– can be symmetric
– can be asymmetric
• PAL DV does 4:2:0 (p210)
• NTSC DV does 4:1:1
Temporal Compression 1
• Use the Difference in two frames
– naive approach can result in good
compression
– works well for a small amount of movement
– A Tarantino film? not so much...
Temporal Compression 2
• Use Motion Compensation
• When an object moves
– compute its trajectory
– fill in the resulting exposed background
• Works well for a small amount of movement
• A Tarantino film? not so much...
• BUT... why isn’t this an easy thing to do?
Temporal Compression 2
• Use Motion Compensation
• When an object moves
– compute its trajectory
– fill in the resulting exposed background
• Works well for a small amount of movement
• A Tarantino film? not so much...
• BUT... why isn’t this an easy thing to do?
• Bitmapped images do not have defined
objects!
• What to do?
Temporal Compression 2
• Define blocks of 16 x 16 pixels
– called a macroblock
• Compute all possible movements of the block
within a short range
• Compute a vector to define that movement
• Store the movement vectors
• Compress the vectors
Temporal Compression
• Need some place to start from
• Can be forward or backward prediction
• Called KeyFrames
–
–
–
–
pick a keyframe
compute next image from that
compute next image from that
What happens when the scene completely
changes?
• Pick a new key frame...
• But HOW?
• Requires powerful AI
Why worry about size?
• When we know that computers are always
getting faster and larger... why not just wait?
• First one on the market usually wins the
market (if it succeeds!)
• Putting video on some small device with
limited space and limited speed
– iPhone
– Spacecraft? Gotta be light weight and small!
A final worry...
• We have been talking about making
video smaller
• There are a variety of techniques to do
this
• Which to choose?
– It is a tradeoff between compression
technique and its computational complexity
Questions?