Digital Media
Dr. Jim Rowan
ITEC 2110
Video
Video
• Works because of persistence of vision
• http://en.wikipedia.org/wiki/Frame_rate
• Fusion frequency
– ~ 40 frames per second
– depends on the brightness of the image relative to
the viewing environment
• Less than that
– flickering
– individual images appear losing the illusion of
motion
Video
• Video vs Animation...
– Video - capture of frames and then
playback
– Animation - create frames individually and
then playback
Video...
Computationally demanding
– Capture must be fast enough to capture sufficient
frames to produce the illusion of motion
– Transport (if across the web) must be fast enough
to carry those captured frames at a rate fast
enough to produce the illusion of motion
– Playback must be fast enough to play those
captured frames at a rate fast enough to produce
the illusion of motion
Video
• If the transport and playback is not fast
enough, frames will be dropped
• Video players (like quicktime) make
compromises differently
– Trying to “degrade gracefully”
• Some drop frames holding the last image
– effectively losing the illusion of motion but continuing the
story as a slide show
• Some play lower resolution images
• Some continue to play audio
Video Standards
• NTSC
– America and Japan
– http://en.wikipedia.org/wiki/Ntsc
– North America and Japan
– 24 frames per second
– framesize different than PAL
Video Standards
• PAL
– Western Europe and Australia
– http://en.wikipedia.org/wiki/PAL
– Western Europe and Australia
– 25 frames per second
– framesize different than NTSC
Video gets big fast
• At a 640 X 480 framesize
• Using 3 byte color (24 bits, one byte per
color) each frame ~ 2 megabytes
• One second of video (uncompresssed)
is 26 Megabytes
• One minute is 1.85 gigabytes
The effects of large size...
• Uncompressed this exceeds most home
computer interface standards
• Strains the internal speed of the home
computer
• Strains the storage capability of home
computer
• WAY exceeds what can be carried by
the net
What to do?
Apply compression!
• On the capture side
– digitization & compression needs to be
carried out by the hardware to be fast
enough
– Can be done in the camera (VTR)
– Can be done in the computer (iSight cam)
Compression in the VTR
• Within the camera
– 3 (at least) different formats internally
• Differing error correction and compatibility
– Recording on different media
• CD
• tape
• Mini DV or DV format
– Connected to computer using firewire
– All 3 formats present the same stream of bits to
the computer
• Artifacts that interfere with processing and
recompression are created
Compression in the computer
• Analog is presented to the computer
through a video capture card
• Compression is done (usually) in the
video capture card
• Allows for a really small camera
because the work is done elsewhere
Analog vs Digital
• An analog signal to the computer is
susceptible to noise corruption
• Digital signal is not
• What’s the big deal?
• Consider compressing a video of a wall
painted a solid color
– Analog noise will cause small fluctuations from
pixel to pixel
– RLE can’t compress it because each pixel is a bit
different
Consider compressing
this using RLE
analog signal
!!!NOISE!!!
computer
iSight
Camera
video capture card
compression
digital
signal
miniDV
the scene
iMovie
compression
640 x 480 = 307,200
307,200 can be represented
by < 24 bits, call it 3 bytes
RLE:
307,200 (3bytes) +
RGB (3 bytes) = 6 bytes
640 x 480 = 307,200 bytes
Noise makes each pixel
a little different
RLE:
307,200 bytes x RGB (3bytes)
= 921600 bytes
hardware vs software
compression
• Hardware conversion... user has no control
over it... it is hardwired
– It is in the camera
– It can be in the video card
• Software conversion... is computationally
expensive... it’s a slow process
– Provides for the most flexibility
– Can use different software coder-decoders
(codec), picking and choosing what fits your needs
better
Streaming Video
http://en.wikipedia.org/wiki/Streaming_media
Video is
transported across the web
played as it arrives
Similar to broadcast TV
Can be part of video conferencing
Network bandwidth is the enemy
3 Methods of Video delivery
• True Streaming video
• Embedded video
• Progressive download or HTTP
streaming
True Streaming video
Network programs are streamed
• never stored on disk
• can be open-ended
• you don’t get a copy to play later
– solves copyright issue!
• can collect viewer stats without Nielsen ratings
• advertisers know how many viewers there are
• on some, you can advance ahead of where the
download is complete and the play will start again at
the new spot
Embedded video
• entire file is transferred before play begins
• stored on disk
• may require more disk than is available
Progressive download
or
HTTP streaming
• downloads some and then starts playing
• predicts when the download will complete and starts
playing when there is enough downloaded to safely
play without interruption
• you get a copy to play later
• takes up disk storage
• quicktime does this
• on some, you can advance ahead of where the
download is complete and the download will start
again at the new spot
The TV legacy
• Raster scan - 525 lines in US and Japan
• Not enough bandwidth to transmit the
whole frame at a rate of 40+ frames per
second
• Resulted in Interlacing fields
• Play half the Frame (and half the data)
then play the other half of the Frame
525 lines total
525 lines total
525 lines total
The problem...
• The fields were played one after the
other to avoid flicker
• BUT...
• The fields were also captured one after
the other...
• there was a time difference between
when they were captured
Why is this a problem?
• When you play them on a computer, the
computer can refresh much faster and can
display the entire frame at the same time
• To play it can put both fields in a frame buffer
and displays them at the same time
• If the object is moving fast, the second frame
shows the object in a different place
• Results in a “comb effect”
Can we solve the problem?
• You can average the two frames and
construct a single frame
• You can toss out one of the fields and
interpolate between them
• Neither is very good...
Film to video?
• Problematic
– video is 30 frames per second
– film is 24 frames per second
• How do you make 30 frames from 24?
• The 3-2 pull down
– http://en.wikipedia.org/wiki/3-2_pulldown
– 3 film frames occupy the first 3 fields
– 2 film frames occupy the next 2 fields
Next
• DV and MPEG
Questions?