Multimedia –
Digital Audio & Video
CSC1720 – Introduction to Internet
Essential Materials
Outline


Audio / Video on the Web
Basic Digital Audio Concepts
– Streaming Audio
– Web Audio Formats

Basic Digital Video Concepts
– Video compression/decompression methods.
– Video File Formats


Other Multi-media Formats
Summary
CSC1720 – Introduction to Internet
2
All copyrights reserved by C.C. Cheung 2003.
AV on the Web?

It is part of the Internet.
– Radio Stations
– Music, sound clips
– Streaming Audio / Video
– Video conferencing
– Digital Cameras
– Animation on the Web

Shockwave, Flash
CSC1720 – Introduction to Internet
3
All copyrights reserved by C.C. Cheung 2003.
Basic Terms
Term
Definition
Bit
Binary value 0 or 1
Kilobit (Kb)
1,000 bits (approx.)
Byte
8 bits
Kilobyte (KB)
1,000 bytes
Megabyte (MB)
1,000,000 bytes
Gigabyte (GB)
1,000,000,000 bytes
Kbps
Kilobits per second (1,000 bits in a second)
KB/sec
Kilobytes per second (1,000 bytes in a second)
CSC1720 – Introduction to Internet
4
All copyrights reserved by C.C. Cheung 2003.
Analog vs. Digital

Analog: A signal that has a continuous
nature rather than a pulsed or discrete
nature.
– Note: Electrical or physical analogies,
such as continuously varying voltages,
frequencies, or phases, may be used as
analog signals.

Digital: A signal in which discrete steps
are used to represent information.
CSC1720 – Introduction to Internet
5
All copyrights reserved by C.C. Cheung 2003.
Digital Signal

Use Pulse Code
Modulation
(PCM) to
represent an
audio signal by
digital data.
CSC1720 – Introduction to Internet
6
All copyrights reserved by C.C. Cheung 2003.
ADC & DAC
Figure 4.3 Conversion from Analog to Digital requires an Analog-to-Digital Converter
Figure 4.4 Conversion from Digital to Analog requires a Digital-to-Analog Converter
CSC1720 – Introduction to Internet
7
All copyrights reserved by C.C. Cheung 2003.
Basic Digital Audio
Concepts

Sampling rate
– Number of sample taken of a signal in a given
time (usually one second)

Bit depth
– Describes the accuracy of the audio data

Channels
– E.g. Stereo has two channels

Bit rate
– Measured in kilobits per second (Kbps) is a
function of the bit depth and sampling rate
CSC1720 – Introduction to Internet
8
All copyrights reserved by C.C. Cheung 2003.
Sampling rate






The more sample taken per second, the
higher the accuracy.
Typically measured in kilohertz (KHz).
CD audio has 44,100 samples per second
(44.1KHz).
8 KHz produces lower quality radio sound.
Standard sampling rates include “8KHz”,
“11.025KHz”, …
The high-end 96K is used in DVD, but is not
applicable to the Web.
CSC1720 – Introduction to Internet
9
All copyrights reserved by C.C. Cheung 2003.
Sampling Rate
demo
CSC1720 – Introduction to Internet
10
All copyrights reserved by C.C. Cheung 2003.
Bit depth




Also called “sampling resolution” or “word
length”.
The more bits, the better is the quality of
the audio (and a larger file of course).
Common bit depths are 8-bit (telephone
like), 16-bit (CD quality), and 20, 24, 32,
48-bit depths.
How many signal can a 8-bit and a 16-bit
data represent?
– 0000 0000  1111 1111
– 0000 0000 0000 0000  1111 1111 1111 1111
CSC1720 – Introduction to Internet
11
All copyrights reserved by C.C. Cheung 2003.
Quantization
demo
CSC1720 – Introduction to Internet
12
All copyrights reserved by C.C. Cheung 2003.
Channels






Audio file can support one to six channels of
audio formats.
Mono – one channel
Stereo – two channels
Some others – three, four channels.
Six channels – 5.1-channel surround sound.
More multi-channel formats announce in the
coming years.
CSC1720 – Introduction to Internet
13
All copyrights reserved by C.C. Cheung 2003.
Channel Examples
CSC1720 – Introduction to Internet
14
All copyrights reserved by C.C. Cheung 2003.
Bit rate


Audio files are measured in terms of
bit rate which is measured in kilobits
per second (Kbps).
It can be calculated by dividing the file
size by the time (in second) to play
the audio clip.
– E.g. 3Mb file play in 30 seconds
– 3000k / 30 = 100kbps.

Quality at different compression rates
CSC1720 – Introduction to Internet
15
All copyrights reserved by C.C. Cheung 2003.
Optimizing for the Web

Length of the audio clip
– Keep the audio clip as short as possible.
– E.g. just keep the most sweetest part of
your greeting.

Number of channels
– A mono audio file is halved the space of a
stereo file.
– Depends on your actual needs.
CSC1720 – Introduction to Internet
16
All copyrights reserved by C.C. Cheung 2003.
Optimizing for the Web

Bit depth
– Audio file on the Web are usually 8-bits.
– Half the size of a 16-bit file.

Sampling rate
– Half the sampling rate will also halve the space
needed.
– Voice only audio file can be reduced to 8KHz.
– 22 KHz music clips are acceptable.

Putting all things together: Mono, 8-bit,
22KHz, MP3 compression.
CSC1720 – Introduction to Internet
17
All copyrights reserved by C.C. Cheung 2003.
Calculate audio size





8-bit mono: seconds x KHz
16-bit mono: (seconds x KHz) x 2
8-bit stereo: (seconds x KHz) x 2
16-bit stereo: ([seconds x KHz] x 2)x2
E.g. the file size of 30 seconds of 16bit, 11KHz mono audio:
– (30 x 11) x 2 = 660K.
CSC1720 – Introduction to Internet
18
All copyrights reserved by C.C. Cheung 2003.
Streaming Audio

What is it?
– Play almost immediately after the request,
continues playing the transferring data.

Advantages:
– Address the problem of long download time.
– Control distribution and protect copyright,
because the user cannot get a copy of the file.

Disadvantages:
– Sound quality may be be affected by low speed
or unstable Internet connection.

Reference
CSC1720 – Introduction to Internet
19
All copyrights reserved by C.C. Cheung 2003.
What is Streaming?
CSC1720 – Introduction to Internet
20
All copyrights reserved by C.C. Cheung 2003.
How does it work?
Streaming audio
Packet are sent to a buffer on the receiving
Computer, the RealPlayer will play the sound
File when buffer full
Web browser
Web browser request
a RealAudio from
the web server
RealAudio
Server
Web server
CSC1720 – Introduction to Internet
21
All copyrights reserved by C.C. Cheung 2003.
Quality Comparison

http://www.cit.cornell.edu/atc/itsupport/streamcompare.shtml
CSC1720 – Introduction to Internet
22
All copyrights reserved by C.C. Cheung 2003.
Web Audio Formats

WAV/AIFF (.wav/.aif/.aiff)
– The Waveform Audio File format (.wav)
was developed by Microsoft, supports
arbitrary sampling rates and bit depths.
– The Audio Interchange File format (.aif,
.aiff) was developed for Macintosh
platform.
– They are less likely used on the Web,
because people use “mp3” or “streaming”.
CSC1720 – Introduction to Internet
23
All copyrights reserved by C.C. Cheung 2003.
http://www.nuance.com/prodserv/demo_vocalizer.html
WAV/AIFF

Try the Bell-lab synthesis link.

http://www.bell-labs.com/project/tts/voices.html
Select the
Audio format
CSC1720 – Introduction to Internet
24
All copyrights reserved by C.C. Cheung 2003.
MP3 (.mp3)


Able to maintain excellent sound
quality at very small file sizes.
The compression reduces an audio file
to one-tenth of its original size.
– E.g. 40MB file  3.5MB



MP3 is actually MPEG-1 Layer-III
Good for distribution of HQ audio.
Demo: www.mp3.com
CSC1720 – Introduction to Internet
25
All copyrights reserved by C.C. Cheung 2003.
What is MP3 digital
encoding?

MP3 is actually the file extension for MPEG, audio
layer 3. Layer 3 is one of three coding schemes
(layer 1, layer 2, and layer 3) for the compression
of audio signals. Layer 3 uses perceptual audio
coding and psychoacoustic compression to remove
all superfluous information. (More specifically, it
removes the redundant and irrelevant parts of a
sound signal--the stuff the human ear doesn't hear
anyway). It also adds a MDCT (Modified Discrete
Cosine Transform) that implements a filter bank,
increasing the frequency resolution 18 times higher
than that of layer 2.
CSC1720 – Introduction to Internet
26
All copyrights reserved by C.C. Cheung 2003.
MP3 Players
CSC1720 – Introduction to Internet
27
All copyrights reserved by C.C. Cheung 2003.
Apple QuickTime Audio
(.mov)


QuickTime is a well-known video format, but
it can create audio-only movies.
QuickTime is a container format, which is
able to store still images, movie formats, …
– Excellent compression, true streaming


Netscape and IE have Plug-in now.
Quicktime : demo
CSC1720 – Introduction to Internet
28
All copyrights reserved by C.C. Cheung 2003.
MIDI (.mid/.midi)





MIDI stands for “Musical Instrument Digital
Interface” which is developed for electronic
musical instruments.
MIDI files are very compact and very good
for low-bandwidth delivery.
Instruments are “piano, drums, bass,
orchestral strings, …”
It is very attractive for adding MIDI file to
your website with very little download time.
Demo: www.findmidis.com
CSC1720 – Introduction to Internet
29
All copyrights reserved by C.C. Cheung 2003.
http://www.findmidis.com/
MIDI
CSC1720 – Introduction to Internet
30
All copyrights reserved by C.C. Cheung 2003.
RealMedia/RealAudio
(.rm/.ra)





RealAudio is a server-based streaming audio
format.
The RealServer responses to the requests
and delivers the streaming packets,
including the bandwidth negotiation.
A robust RealServer can support thousands
of simultaneous listeners.
Good for continuous-playing audio and live
broadcasts to a large group of people.
Example: RTHK Radio
CSC1720 – Introduction to Internet
31
All copyrights reserved by C.C. Cheung 2003.
RealMedia/RealAudio
CSC1720 – Introduction to Internet
32
All copyrights reserved by C.C. Cheung 2003.
Windows Media
(.wma/.asf)





Windows Media is a streaming system.
It wraps all media elements into a
Active Streaming File (.asf).
Audio may be saved as non-streaming
Windows Media Audio format (.wma).
Good for continuous-play audio.
The encoder and player is Free,
shipped with Windows OS.
CSC1720 – Introduction to Internet
33
All copyrights reserved by C.C. Cheung 2003.
Windows Media
CSC1720 – Introduction to Internet
34
All copyrights reserved by C.C. Cheung 2003.
Choosing an audio format
Audio Needs
Suggested formats
Short voice greetings
WAV, AIFF, MP3
News broadcasts
Streaming solutions (RealAudio,
Windows Media)
Background music
MIDI, WAV
Music samples for some
audience
MP3 or QuickTime
Radio-style or Live broadcasting
RealMedia System
Musical E-greeting card
MIDI, WAV
CSC1720 – Introduction to Internet
35
All copyrights reserved by C.C. Cheung 2003.
Add Audio to your Web
Page

A simple link
– <A HREF=“audio/song.wav”>Play the song </A>
– <A HREF=“music.mp3”><IMG SRC=“buttons/play.gif”></A>

Background Sound
– <BGSOUND SRC=“audio/song.mid” LOOP=3>

Link to RealMedia
– <A HREF=“song.ram”>Link to the song</A>
– pnm://domainname.com/song.rm
CSC1720 – Introduction to Internet
36
All copyrights reserved by C.C. Cheung 2003.
Break Time – 15 minutes
CSC1720 – Introduction to Internet
37
All copyrights reserved by C.C. Cheung 2003.
Basic Digital Video
Concepts






Movie length
Frame size
Frame rate
Quality
Color bit depth
Data rate (bit rate)
CSC1720 – Introduction to Internet
38
All copyrights reserved by C.C. Cheung 2003.
Movie length



File size is proportional to the movie
length.
Videos longer than 1 or 2 minutes
cause long download times.
If it is a long video, consider to use
streaming video.
CSC1720 – Introduction to Internet
39
All copyrights reserved by C.C. Cheung 2003.
Frame size




“Full-screen” video is 640x480 pixels.
The most common frame size for web
video is 160x120 pixels.
Not recommend to use a frame size
larger than 320x240.
The size depends on the CPU power
and the Internet connection
bandwidth.
CSC1720 – Introduction to Internet
40
All copyrights reserved by C.C. Cheung 2003.
Image and Video?
CSC1720 – Introduction to Internet
41
All copyrights reserved by C.C. Cheung 2003.
Frame rate




Frame rate is measured in number of
frames per second (fps).
Standard TV-quality video uses 30 fps.
For the web, 15 or even 10 fps is more
appropriate and produces fair smooth
quality for the user.
Commercial Internet Broadcasts are
using 0.5, 0.25 frames per second.
CSC1720 – Introduction to Internet
42
All copyrights reserved by C.C. Cheung 2003.
Quality




Many video-editors allow you to set
the overall quality of the video.
The degree of compression controls
the target quality.
The low or medium setting results a
fairly high compression which is
appropriate for web delivery.
Frame rate and quality are usually
tradeoff in different applications.
CSC1720 – Introduction to Internet
43
All copyrights reserved by C.C. Cheung 2003.
Color bit depth



The number of pixel colors in each
frame affects the size of the video.
The file size of the video will be
greatly reduced by changing the
number of colors from 24-bit to 8-bit.
It sacrifices the image quality of the
video.
CSC1720 – Introduction to Internet
44
All copyrights reserved by C.C. Cheung 2003.
Data rate (bit rate)



This is the rate that the data must be
transferred in order to ensure the video can
play smoothly without interruption.
It is measured in kilobytes per second
(K/sec or Kbps).
It can be calculated by dividing the size of
the file (in K) by the movie length (in
seconds).
– E.g. the video file size is 1.9MB  1900K
– Play 40 seconds long, Data rate = 47.5K/sec

Consider the Internet bandwidth!
CSC1720 – Introduction to Internet
45
All copyrights reserved by C.C. Cheung 2003.
Calculate space
requirements of Video

NTSC video (640 x 480 and 29.97 fps)
– Frame size = ([Pixel width x pixel height x
bit depth]/8)/1024
– E.g. 200KB/Frame : 6.0 MB/sec
– 200KB x 30 fps = 6000KB/s, 6 MB/sec

PAL video (768 x 576 and 25 fps)
– E.g. 200KB/Frame : 5.0 MB/sec
– 200KB x 25 fps = 5000KB/s, 5 MB/sec
CSC1720 – Introduction to Internet
46
All copyrights reserved by C.C. Cheung 2003.
Video CODECs


CODEC is
“Compression/Decompression”
algorithms.
The sound and frame images of a
digital video must be compressed.
– The vast amount of data

Compressed in a number of ways
– Lossless and Lossy compression
– Spatial and Temporal compression
CSC1720 – Introduction to Internet
47
All copyrights reserved by C.C. Cheung 2003.
Video Clip Demo

CSC1720 – Introduction to Internet
48
reference
All copyrights reserved by C.C. Cheung 2003.
Lossless and Lossy
compression


Lossless compression means no
information is lost and the final file is
the same as the original.
Most compression methods are lossy.
– Sacrifices some data from the file in order
to achieve higher compression rates.
– Use complicated algorithm to toss out
some data that is not discernible to the
human eye or ear.
CSC1720 – Introduction to Internet
49
All copyrights reserved by C.C. Cheung 2003.
Spatial and Temporal
compression


Spatial (Intraframe) compression takes
place on each individual frame of the video.
Temporal (Interframe) compression applies
on a series of frames, it takes advantages of
areas of the image remain unchanged from
frame to frame.
– Relies on the key frames and delta frames.
– A key frame is placed once every second.
– E.g. 15 fps, a key frame once 15 frames.
CSC1720 – Introduction to Internet
50
All copyrights reserved by C.C. Cheung 2003.
Delta frame
CSC1720 – Introduction to Internet
51
All copyrights reserved by C.C. Cheung 2003.
Video File Formats

QuickTime Movie (.mov)
– Introduced by Apple Computer in 1991.
– First developed for Macintosh, now also
supports the PCs.
– Also supports streaming.

How to create?
– Most video editor, QuickTime Pro.

How to play?
– QuickTime plug-in or QuickTime player.
CSC1720 – Introduction to Internet
52
All copyrights reserved by C.C. Cheung 2003.
http://www.apple.com/quicktime/gallery/cubicvr/times_square.html
Quicktime
CSC1720 – Introduction to Internet
53
All copyrights reserved by C.C. Cheung 2003.
RealMedia (.rm)





Industry standard streaming format.
RealPlayer for playback.
RealServer for serving streams.
RealProducer for creating .rm files.
Good for
– Long-playing video or broadcast to many people.

How to create?
– RealSystem Producer

How to play?
– RealPlayer (Free), RealPlayer Plus (Commerical)
CSC1720 – Introduction to Internet
54
All copyrights reserved by C.C. Cheung 2003.
Windows Media
(.wmv/ .asf)




Created by Microsoft, closely integrated with
Windows OS.
Support Windows Media Video (.wmv) and
Advanced Streaming Format (.asf) and
other formats (.avi, .mpeg, …)
Also support streaming.
How to create?
– Windows Media Encoder, Windows Media Author

How to play?
– Media Player in Windows OS
CSC1720 – Introduction to Internet
55
All copyrights reserved by C.C. Cheung 2003.
AVI (.avi)




Stands for Audio/Video Interleaved.
Introduced by Microsoft in 1992.
In a AVI file, the audio and video
information are interleaved every frame.
Good for
– Short web clips, high-quality video

How to create?
– Most video editing tools.

How to play?
– Windows Media, QuickTime, etc.
CSC1720 – Introduction to Internet
56
All copyrights reserved by C.C. Cheung 2003.
Video Clip Demo
CSC1720 – Introduction to Internet
57

reference
All copyrights reserved by C.C. Cheung 2003.
MPEG (.mpg/ .mpeg)







Created by Moving Picture Experts Group
(MPEG).
Supports 1) Video, 2) Audio, 3) Streaming.
Extremely high compression rates with small
quality degradation (lossy).
MPEG-1 : VHS quality
MPEG-2 : HQ standard for TV broadcast
MPEG-4 : Very HQ for AV compression
MPEG can be compressed by using three
schemes: Layer-I, Layer-II, Layer-III.
CSC1720 – Introduction to Internet
58
All copyrights reserved by C.C. Cheung 2003.
Official MPEG page
CSC1720 – Introduction to Internet
59

reference
All copyrights reserved by C.C. Cheung 2003.
MPEG Comparison
MPEG-1
MPEG-2
MPEG-4
Standard since
1992
1995
1999
Default Video resolution (NTSC)
352 x 288
640 x 480
640 x 480
Max. Audio Frequency range
48 KHz
96 KHz
96 KHz
Max. audio Channel
2
8
8
Regular data rate
1380 kbit/s
6500 kbit/s
880 kbit/s
Frames per sec (NTSC)
30
30
30
Video Quality
Satisfactory
Very good
Very Good
Hardware requirement for
encoding/decoding
Low
Medium
High
CSC1720 – Introduction to Internet
60
All copyrights reserved by C.C. Cheung 2003.
Flash & Shockwave

Advantages:
–
–
–
–
–

File sizes are small
Image quality is high
It uses streaming technology
It uses high-quality streaming audio
It is scriptable
Disadvantages:
– A plug-in player is required
– Expensive authoring software
– Problems on printing their content
CSC1720 – Introduction to Internet
61
All copyrights reserved by C.C. Cheung 2003.
Summary




Discuss the basic digital audio/video
terminology.
Introduction to different formats: WAV, MP3,
QuickTime, RealMedia, Windows Media, AVI,
MPEG.
To deliver long-playing audio/video or live
broadcasts, you should choose one of the
streaming media.
Flash and shockwave are popular and
appropriate format for the Web.
CSC1720 – Introduction to Internet
62
All copyrights reserved by C.C. Cheung 2003.
References









Design Web Audio (J. Beggs, D. Thede), Oreilly.
E-Video (H. Peter Alesso), Addison-Wesley.
Audio Video Knowledge Center
Bible in MP3 format
MPEG
Musiq
Luke Video
The End.
Thank you for your patience!
CSC1720 – Introduction to Internet
63
All copyrights reserved by C.C. Cheung 2003.