EEE 508 - Digital Image & Video Processing and Compression
http://lina.faculty.asu.edu/eee508/
Introduction
Prof. Lina Karam
School of Electrical, Computer, & Energy Engineering
Arizona State University
karam@asu.edu
1
Why Image and Video ?
Sample image-and video-based applications
•
•
•
•
•
•
Entertainment
Communications
Medical imaging
Security
Monitoring
Visual sensing and control
2
Basic Imaging System
Imaged Scene
x
CAMERA Imaging Device
z
y
DIGITIZER
STORAGE
Sampling + Quantization
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
Colored lights from scene are captured into red, green, and blue
pixels (picture elements)
Scene viewed through “color” filters that separate the image
into 3 color components
Digital camera systems contain optics that image light onto
sensors typically a CCD array with filters
Copyright 2007 ‐2012 by Lina J. Karam
3
Basic Imaging System
Imaged Scene
x
CAMERA Imaging Device
z
y
DIGITIZER
STORAGE
Sampling + Quantization
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
z
Quality of captured image depends on imaging optics and
electronics, “color” filter characteristics, digitization, and
processing
Copyright 2007‐2012 by Lina J. Karam
4
Basic Imaging System
Imaged Scene
x
CAMERA Imaging Device
DIGITIZER
z
y
STORAGE
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
High-end digital cameras make use of dichroic filters to split
the light into, red, green, and blue components
• Beam splitter is used to split light into three beams that are directed
through filters that filter out all but one color for each chip (“dichroic”
indicates that 2 out of the 3 colors are filtered).
• Each color component is imaged separately onto an array of sensors:
one chip “sees” red (R), one “sees” green (G), and one “sees” blue (B).
• Three values (R, G, B) captured at each pixel position
Copyright 2007 ‐2012 by Lina J. Karam
5
Basic Imaging System
Imaged Scene
x
CAMERA Imaging Device
z
y
DIGITIZER
STORAGE
Sampling + Quantization
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
Common digital cameras have a single imaging element (typically
one CCD chip) and make use of tiled Color Filter Array (CFA)
Light from scene
CFA
Bayer CFA
Each captured pixel is either
Green (G), Red (R), or Blue(B).
Interpolation is used to recover
(R,G,B) values at each pixel.
Image Sensor
Copyright 2007‐2012 by Lina J. Karam
6
Digitization: Sampling and Quantization
Imaged Scene
x
t
CAMERA Imaging Device
y
I(n1,n2;n3)
DIGITIZER
I(x,y;t)
Sampling + Quantization
STORAGE
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
• Original Imaged Scene : analog (continuous in space and time)
I(x,y;t) for video and I(x,y) for still image
I: image intensity and color at position (x,y) and at time t
• Digitized sensed image/video: digital (sampled in space and time,
plus discrete amplitudes) I(n1,n2;n3) for video and I(n1,n2) for a
still image
I: image intensity and color at integer sample position (n1,n2) and
integer time index n3
Copyright 2007‐2012 by Lina J. Karam
7
Basic Imaging System
Imaged Scene
S1or x CAMERA x
t
z
Imaging Device
S3 or z
DIGITIZER
STORAGE
Sampling + Quantization
Compression
PROCESS
Display, Analysis, Enhancement, Restoration, Compression for transmission
S2 or y
•
I(s1,s2,s3,t) : ANALOG SIGNAL
I : real value or vector of real values
(s1,s2 ,s3,t) : set of real continuous space (time) variables
•
I(n1,n2, n3,n4) : DISCRETE SIGNAL (DIGITAL)
I : discrete (quantized) real or integer value
(n1,n2,n3,n4 ) : set of integer indices
Copyright 2007‐2012 by Lina J. Karam
8
Examples
•
Sampled Black & White Photograph: I(n1,n2)
I (n1,n2) scalar indicating pixel intensity at location (n1,n2)
For example: I = 0
Black
I=1
White
0<I<1
•
In-between
Sampled color video/TV signal
IR(n1, n2, n3)
2D TV:
IG(n1, n2, n3)
IB(n1, n2, n3)
EEE 508 - Lecture 1
9
IR(n1, n2, n3 , n4)
;
3D TV:
IG(n1, n2, n3 , n4)
IB(n1, n2, n3 , n4)
9#
Digitization: Sampling and Quantization
Video Sampling
• Temporal sampling affects frame (image) rate and perceived
motion quality.
– 50 to 60 frames per second produce smooth apparent motion
– 25 (PAL) or 30 (NTSC) frames per second is standard for
television pictures; interlacing can be used to improve the
appearance of motion
• Frame rate can be referred to as temporal resolution.
Copyright 2007‐2012 by Lina J. Karam
10
Digitization: Sampling and Quantization
Video Sampling
• Progressive and Interlaced Sampling
– Progressive sampling: all lines (rows) in a frame are sampled
– Interlaced sampling: alternate between sampling the odd rows
(odd field) for one frame followed by the sampling the even
rows (even field) for next frame
Odd or Top Field
Copyright 2007‐2012 by Lina J. Karam
Even or Bottom Field
11
Spatial Resolution
A digital image is represented as a rectangular array of
picture elements (pixels or pels).
503 pixels 503x365 pixels
Total pixels = 183,595
365
pixels
Spatial resolution commonly refers to the number of pixels
in the horizontal and vertical directions.
Copyright 2007‐2012 by Lina J. Karam
12
Spatial Resolution
Video Formats based on Resolution
176
144
288
480
576
720
352
720
1280
1920
QCIF
CIF, 101 Kpixels
480i SDTV, 345 Kilo pixels
SDTV, 415 Kilo pixels
High Definition (HDTV) 1 Mega pixels
1080
Copyright 2007‐2012 by Lina J. Karam
High Definition (HDTV) 2 Mega pixels
13
Spatial Resolution
4CIF: 704x576
CIF: 352x288
QCIF: 176x144
SCIF: 128x96
Copyright 2007‐2012 by Lina J. Karam
14
Spatial Resolution
Choice of frame resolution depends on application and
on available storage and transmission capacity.
Perceived resolution refers to the maximum number of
line pairs that can be resolved on the display screen or to
the smallest details that can be resolved.
• Depends on viewing distance.
• Depends on display
Display resolution is commonly expressed in pixels per
inch.
Copyright 2007‐2012 by Lina J. Karam
15
Aspect Ratio
Aspect ratio is the ratio of the image’s width to its
height.
720
480
1280
SDTV Video
4:3
1.33:1
Widescreen SDTV
HDTV
16:9
1.78:1
720
1920
Full HDTV
1080
1.78:1
Copyright 2007 ‐2012 by Lina J. Karam
16
How do we process images?
• Exploit visual perception properties
• Use/Develop image/video processing
(computer vision) algorithms
• Use DSP concepts as tools
EEE 508 - Lecture 1
17
17#
How many possible images are there?
We represent pixels as amplitude values (gray scale).
256 levels
1
0
128 levels
1
0
64 levels
1
0
32 levels
1
0
How much to sample (quantize) the gray scale?
Humans can distinguish in the order of 100 levels of gray.
EEE
18# 508 - Lecture 1
18
How many possible images are there?
An image has pixels and dimensions, say 200x200 and assume 64
pixel values (64 gray levels).
• A 1x1 image → about 64 images
• A 1x2 image → about (64)2 images
• A 200x200 image → about (64)40000 images
• A large but finite number due to human perceptive
properties.
EEE
19# 508 - Lecture 1
19