Digital Media
Dr. Jim Rowan
ITEC 2110
Video
Video
• Works because of persistence of vision
• http://en.wikipedia.org/wiki/Frame_rate
• Fusion frequency
– ~ 40 frames per second
– depends on the brightness of the image relative to
the viewing environment
• Less than that
– first flickering
– then individual images appear losing the illusion of
motion
Video
• Video vs Animation...
– Video
• capture of frames in the world
• play them back
– Animation
• create frames individually
• play them back
Video...
Computationally demanding
– Capture
• must be fast enough to capture sufficient frames to
produce the illusion of motion
– Transport across the web
• if there is a live feed across the web it must be fast
enough to carry those captured frames at a rate fast
enough to produce the illusion of motion
– Playback
• the processor and all its internal parts must be fast
enough to play those captured frames at a rate fast
enough to produce the illusion of motion
Video
• Transport or playback not fast enough?
– something’s got to give
• Video players (like quicktime) make
compromises differently
– Attempt to “degrade gracefully”
• Some drop frames holding the last image
– effectively losing the illusion of motion but continuing the
story as a slide show
• Some play lower resolution images
• Some continue to play audio
Video Standards
• NTSC
• PAL
• SECAM
Video Standards
• NTSC
– America and Japan
– http://en.wikipedia.org/wiki/NTSC
– 24 frames per second
– 525 scanlines, 60Hz (486 visible)
• chosen because of vacuum tube limitations
– framesize different than PAL
Video Standards
• PAL
– Western Europe and Australia
– http://en.wikipedia.org/wiki/PAL
– 25 frames per second
– 625 (576 visible)-line/50 Hz
– framesize different than NTSC
Video Standards
• SECAM
– France, Eastern Europe
– http://en.wikipedia.org/wiki/SECAM
– frequency modulated (fm)
– Difficult to edit in it’s native form
Video gets big (filesize) fast
• At a 640 X 480 framesize
• Using 3 byte color (24 bits, one byte per
color) each frame ~ 2 megabytes
• One second of video (uncompresssed)
is 26 Megabytes
• One minute is 1.85 gigabytes
The effects of large size...
• Uncompressed?
– exceeds most home computer interface
standards
– strains the internal speed of the home
computer
– strains the storage capability of home
computer
– WAY exceeds what can be carried by the
net
What to do?
Apply compression!
• On the capture side
– digitization & compression needs to be
carried out by hardware to be fast enough
– Can be done in the camera (VTR)
– Can be done in the computer (iSight cam)
Compression in the camera
• Within the camera
– 3 (at least) different formats internally
• with differing error correction and compatibility
– Recording on different media
• CD, Tape, Memory card
• Mini DV or DV format
– All 3 media present the same stream of bits to the
computer over a firewire connection
• Compression? Artifacts!
Compression in the
computer?
• Analog is presented to the computer
through a video capture card
• Compression is done (usually) in the
video capture card
• Allows for a really small camera
because the work is done elsewhere
Analog vs Digital
• An analog signal to the computer is
susceptible to noise corruption
• Digital signal is not
• What’s the big deal?
• Consider compressing a video of a wall
painted a solid color
– Analog noise will cause small fluctuations from
pixel to pixel
– RLE can’t compress it because each pixel is a bit
different
Consider compressing
this using RLE
analog signal
!!!NOISE!!!
computer
webCam
video capture card
compression
digital
signal
miniDV
the scene
iMovie
compression
640 x 480 = 307,200
307,200 can be represented
by < 24 bits, call it 3 bytes
RLE:
307,200 (3bytes) +
RGB (3 bytes) = 6 bytes
640 x 480 = 307,200 bytes
Noise makes each pixel
a little different
RLE:
307,200 bytes x RGB (3bytes)
= 921600 bytes
hardware vs software
compression
• Hardware compression... user has no control
over it... it is hardwired
– It is in the camera
– It can be in the video card
• Software conversion... is computationally
expensive... it’s a slow process
– Provides for the most flexibility
– Can use different software coder-decoders
(codec), picking and choosing what fits your needs
better
Streaming Video
Similar to Internet radio… but with images
Video is
transported across the web
played as it arrives
Video conferencing
Network bandwidth is the chokepoint
http://en.wikipedia.org/wiki/Streaming_media
Methods of Video delivery
…NOT a mutually exclusive list
…CAN be combined
• Streaming video
• Embedded video
– embedded inside something else
• like a webpage
– plays from the beginning
– has user controls
– usually does not allow download
– youTube, google video
(more) Methods
of Video delivery
• Podcast
– download now and play later
• Progressive download
• starts playing before download is completed
• Pull Technology
– user requested (requested from the client)
• Push Technology
– server initiated (like chat)
Streaming video
• not stored on disk
• can be open-ended
• you don’t get a copy to play later
– solves copyright issue!
• Advantages?
– can collect viewer stats without Nielsen ratings
– advertisers know how many viewers there are
– on some, you can advance ahead of where the download is
complete and the play will start again at the new spot
Progressive download
• downloads some and then starts playing
• quicktime does this
– predicts when the download will complete and starts playing
when there is enough downloaded to “safely” play without
interruption
• you can get a copy to play later
• on some, you can advance ahead of where the
download is complete and the download will start
again at the new spot without waiting for the skippedover part to load
The TV legacy
• Raster scan - 525 lines in US and Japan
• Vacuum tube limitations ==> couldn’t
transmit 40 frames per second
• Resulted in Interlacing fields
• Play half the Frame (and half the data)
then play the other half of the Frame
odd field
even field
The problem...
• The fields were played one after the
other to avoid flicker
• BUT...
• The fields were also captured one after
the other...
• The result? There is a time difference
between when they were captured
Why is this a problem?
• When you play them on a computer, the
computer can refresh much faster and can
display the entire frame at the same time
• To play it can put both fields in a frame buffer
and displays them at the same time
• If the object is moving fast, the second frame
shows the object in a different place
• Results in a “comb effect”
Can we solve the problem?
• You can average the two frames and
construct a single frame
• You can toss out one of the fields and
interpolate between them
• Neither is very good...
So…
• Converting TV to Video is problematic
• (comb effect)
• Converting Film to Video is problematic
Film to Video
• Problematic
– video is 30 frames per second
– film is 24 frames per second
• How do you make 30 frames from 24?
• The 3-2 pull down…
http://en.wikipedia.org/wiki/Telecine
Film to interlaced video:
Next
• The rest of chapter 6…
Questions?