Multimedia
Class 11
LBSC 690
Information Technology
Agenda
•
•
•
•
•
•
•
Questions
Images
Audio
Transmission
Virtual reality
Midterm structure
Midterm review
Basic Image Coding
• Raster of picture elements (pixels)
– Each pixel has a “color”
• Binary - black/white (1 bit)
• Grayscale (8 bits)
• Color (each pixel has three dots)
– Red, green, blue
• Screen
– A 1024x768 image requires 2.4 MB
• So a picture is worth 400,000 words!
Some Questions
• Use this to answer:
– How many images can a 2 GB hard drive store?
• Nobody would use images!
– How long does it take to send one by modem?
• Imagine how slowly web pages would load
• But real images don’t have these problems
– How do we get around these problems?
Compression
• General goal: reduce redundancy
– Send the same information using fewer bits
•
•
•
•
more telephone calls in one cable
more faxes per minute
more images stored per disk
better quality video images
• Two basic strategies:
– Lossless, Lossy
– The two strategies can be combined
Lossy Compression
• Example - Palette selection
– No picture uses all 16 mission colors
– Select a palette of 256 colors
• Can represent with 1byte instead of 3
• Then look up each color in the palette
• JPEG (.jpg)
– Standard lossy compression for images
– Eliminates detail that’s not seen by humans
– Uses frequency representation
Lossless Compression
• Run Length Encoding (RLE)
– Pixels are organized into lines
– Most pixels are the same as the one before
• That can be coded in 1 bit (1/24 the space)
– Smaller files take less time to transmit
• GIF (.gif)
– Standard lossless compression format
Moving Images
• One image frame is much like the next
– An additional source of redundancy
• MPEG-1 (.mpg) can handle small screens
– Compression requires extensive computation
• Special purpose hardware needed to run in real-time
– Pentium processors can decode it
• MPEG-2 is needed for full-screen video
– Not yet widely used
Audio
• Sample at twice the highest frequency
– One or two bytes per sample
– Voice (0-4 kHz) requires 8 kB/s
– Music (0-22kHz) requires 44 kB/s
• Compression strategies
– Lossy is pretty good for voice
• Only some of the frequencies are actually used
– Lossless is better for music
Transmission
• MIME
– Attach a standard format to message
• Formats include .gif, .jpg, .mpg, and .au
• Messages include email and web pages
– The whole file is sent first, then played
Streaming Audio and Video
• Streaming protocols
– Replay starts almost immediately
– RealVideo has emerged as the standard
• Streaming video challenges
– Sent in small packets
– Sometimes arrive out of order
– Compensate by storing some in a buffer
• Introduces a delay
• Modems carry audio better than video
– Video data requires high “bandwidth”
– Real Video compensates with lower frame rates
Virtual Reality Displays
• Visual
– Position/motion, color/contrast, symbols
• Auditory
– Position/motion, tones, speech
• Haptic
– Mechanical, thermal, electrical, kinesthethic
• Olfactory
– Smell, taste
• Vestibular
Types of Displays
• Image display
– Fixed view, movable view, projection
• Acoustic display
– Headphones, speakers, within-ear monitors
• Tactile display
– vibrotactile, pneumatic
• Force feedback
– dexterous handmaster, joystick, pen
More Display Types
• Inertial Display
– Motion-based simulators
• Olfactory Display
– Chemical (requires re-supply)
• Locomotive display
– Stationary bicycle, treadmill, ...
• Temperature Display
Wearable Visual Interfaces
• Head mounted displays
– Two liquid crystal (camcorder-like) displays
– Cables for video and position tracking
• Binocular omni-oriented monitor
– Simpler and more comfortable
– Limited range of motion
• Shuttered glasses
– Used with fixed and projection displays
Some Input Techniques
• Arms and hands
– Aimed movement, grasp, press, tap, sign/gesture,
write/draw/paint/sculpt, crawl/swing, rub
• Legs
– Walk/run/skip/hop/crawl, swim, aimed movement
• Face
– Expression, speak, sing/hum, breathe
• Body
– Posture, appearance/clothing
Input Devices
• Position tracking
– Mechanical, magnetic, acoustic, GPS, inertial
• Gloves and suits
– Fiber optic, flexible strip, exoskeleton
• Locomotive
– Stationary bicycle, treadmill, stairs, shoes, rowing
• Conventional input devices (haptic, speech)
• Image processing (position, gesture)
• Facial sensing, eye tracking, biosignal
Position Tracking
• Magnetic
– Three fixed emitters, three movable sensors
• Acoustic
– Time delay systems have variable update rates
– Phase coherent systems can jump lanes
• Differential GPS
– Requires unobstructed sight lines to satellites
• Inertial
– Large, short term, requires initial calibration
Exam Structure
• One hour and 15 minutes
• Approximately 4 questions
– Each may have multiple parts
• Open Book (Oakman only)
– You may hand write anything in your Oakman
– No extra pages of notes
• The software you may use will be specified
• You may bring a calculator
Exam Advice
• The only goal is to get points!
– Spend each minute in the best place
• Develop a strategy for each question type
– Guessing CAN hurt on multiple choice
– Don’t write a page when a sentence will do
• Study concepts, not details
– Grading rewards conceptual understanding
– Don’t expect a clone of the sample exams