ENEE631 Spring’09
Lecture-1 (1/26/2009)
Digital Image and Video Processing –
An Introduction
Spring ’09 Instructor: Min Wu
Electrical and Computer Engineering Department
University of Maryland, College Park
 bb.eng.umd.edu (select ENEE631 S’09)
 minwu@eng.umd.edu
M. Wu: ENEE631 Digital Image Processing (Spring'09)
ENEE631 Logistics – Spring 2009

Lectures
– Monday and Wednesday 11am-12:15pm, CSI 2120

Assignments and Projects
– Matlab will be used for many assignments; C/C++ may also be involved
in some.
– Kim Lab #2107 ~ image/video related software installed for EE408G


students are encouraged to make use of them in public lab hours.
Office Hours
– Dr. Min Wu (minwu@eng.umd.edu)


Wednesday 12:30 – 2:30pm @ Kim 2142, or by appointment
Regularly check the course web page
bb.eng.umd.edu
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [2]
Scope of ENEE631

First graduate course on image/video processing

Prerequisites: ENEE620 and 630, or by permission
– Not assume you have much exposure on image processing at
undergraduate level
– Require and build on background in random process and DSP

Emphasis on fundamental concepts
– Provide theoretical foundations on multi-dimensional signal
processing built upon pre-requisites
– Coupled with assignments and projects for hands-on experience
and reinforcement of the concepts
– Follow-up courses


image analysis, computer vision, pattern recognition
multimedia communications and security
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [3]
Textbooks and References

R.C. Gonzalez and R.E. Woods: Digital Image Processing,
Prentice Hall, 3rd Edition, 2008. (yellow cover)

Related technical publications (will be announced in class)

Other related textbooks
– Y. Wang, J. Ostermann, Y-Q. Zhang: Digital Video Processing and
Communications, Prentice Hall, 2001.
– A.K. Jain: Fundamentals of Digital Image Processing, Prentice Hall, 1989.
– John W. Woods: Multidimensional Signal, Image, and Video Processing
and Coding, Academic Press, 2006.
– A.Bovik: Handbook Of Image & Video Processing, 2nd Edition, Academic
Press, 2005.
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [4]
ENEE631 Course Organization

Grading
– Assignments and class participation
– Projects
– Exams

20%
45%
35%
Assignments: theoretical problems + computer components
– Involves Matlab or C/C++ programming and tasks with image/video tools
to reinforce concepts
– Grading is based mainly on completeness; encourage further explorations
and discussions

Projects
– Put theories and principles in use and learn from doing; critical thinking

Exams
– In-class mid-term exam: on basic concepts, theories, and approaches
– Final exam: apply theories and principles to image/video proc tasks
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [5]
ENEE631 Course Policies

No late submission will be accepted
– Start early! Plan wisely and prepare for unforeseen hurdles
– Inform instructor of special circumstances with documentation

Independent work vs. discussions
– Write up your solutions INDIVIDUALLY
– Discussions with classmates on assignments and projects are encouraged
(unless otherwise noted)

Computer codes
– You should write your own codes unless otherwise stated
– DO NOT COPY other students’ codes
– Clearly state the code modules obtained elsewhere and consult instructor
for permission to use in your project

Academic integrity: cheating, plagiarism, fabrication of results, …
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [7]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Image and Video Processing:
An Introduction and Overview
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [8]
UMCP ENEE631 Slides (created by M.Wu © 2001)
A picture is worth 1000 words.
A video is worth 1000 sentences?
http://marsrovers.jpl.nasa.gov/gallery/press/opportunity/20040125a.html
JPL Mars’ Panorama captured by the Opportunity

Rich info. from visual data

Examples of images around us
natural photographic images;
artistic and engineering drawings
scientific images (satellite, medical, etc.)

“Motion pictures” => video
movie, TV program; family video; surveillance and highway/ferry camera
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [9]
(From B. Liu EE488 F’06 at Princeton)
Increasing Use of Images –
A Glimpse from Encyclopedia Britannica

First Edition (1768-1771)
“A dictionary of arts and sciences” by
“a Society of Gentlemen in Scotland”
– 3 volumes, ~ 2600 pages
– illustrated with 160 copperplates

11th Edition (1911)
– “last time to encapsulate ALL human
knowledge”
– one picture every 4 pages

1999 Edition
– 32 volumes; in CD and DVD
– 73,000 articles; 30,000 photos and
illustrations

Now online: http://www.britannica.com/
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [10]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Why Do We Process Images?

Enhancement and restoration
– Remove artifacts and scratches from an old photo/movie
– Improve contrast and correct blurred images

Composition (for magazines and movies), Display, Printing …

Transmission and storage
– images from oversea via Internet, or from a remote planet

Information analysis and automated recognition
– Providing “human vision” to machines

Medical imaging for diagnosis and exploration

Security, forensics and rights protection
– Encryption, hashing, digital watermarking, digital fingerprinting …
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [11]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Why Digital?

“Exactness”
– Perfect reproduction without degradation
– Perfect duplication of processing result

Convenient & powerful computer-aided processing
– Can perform sophisticated processing through computer hardware
or software
– Even kindergartners can do some!

Easy storage and transmission
– 1 CD can store hundreds of family photos!
– Paperless transmission of high quality photos through network
within seconds
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [12]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Examples of Digital Image & Video Processing

Compression

Manipulation and Restoration
– Restoration of blurred and damaged images
– Noise removal and reduction
– Morphing

Applications
–
–
–
–
Visual mosaicing and virtual views
Face detection
Visible and invisible watermarking
Error concealment and resilience in video transmission
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [13]
Compression
UMCP ENEE631 Slides (created by M.Wu © 2001)


Color image of 600x800 pixels
– Without compression

600*800 * 24 bits/pixel
= 11.52K bits = 1.44M bytes
– After JPEG compression (popularly
used on web)


only 89K bytes
compression ratio ~ 16:1
Movie ~ Image Sequence
– 720x480 per frame, 30 frames/sec,
24 bits/pixel
– Raw video ~ 243M bits/sec
– DVD ~ about 5M bits/sec
– Compression ratio ~ 48:1
M. Wu: ENEE631 Digital Image Processing (Spring'09)
“Library of Congress” by M.Wu (600x800)
Lec1 – Introduction [14]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Denoising
From X.Li http://www.ee.princeton.edu/~lixin/denoising.htm
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [15]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Deblurring
http://www.mathworks.com/access/helpdesk/help/toolbox/images/deblurr7.shtml
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [16]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Special Effects: Morphing
Princeton CS426 face morphing examples
http://www.cs.princeton.edu/courses/archive/fall98/cs426/assignments/morph/morph_results.html
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [17]
Visual Mosaicing
UMCP ENEE631 Slides (created by M.Wu © 2001)
– Stitch photos together without thread or scotch tape
R. Radke – Princeton thesis 5/2001
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [18]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Face Detection
Face detection in ’98 @ CMU CS, http://www.cs.cmu.edu/afs/cs/Web/People/har/faces.html
– Image enhancement, feature extractions, and statistical modeling
are often important steps in computer vision tasks

See more image understanding examples by Prof. Chellappa’s
research group (http://www.cfar.umd.edu/~rama/research.html)
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [19]
“General Illumination Correction and Its Application to Face Normalization”, J. Zhu
et al, ICASSP 2003
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [20]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Visible Digital Watermarks
from IBM Watson web page
“Vatican Digital Library”
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [22]
UMCP ENEE631 Slides (created by M.Wu © 2001; 2009)
Invisible Watermark
– Original, marked, and their amplified luminance difference
– human visual model for imperceptibility: protect smooth areas and sharp edges
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [23]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Data Hiding for Annotating Binary Line Drawings
original
marked w/
“01/01/2000”
M. Wu: ENEE631 Digital Image Processing (Spring'09)
pixel-wise
difference
Lec1 – Introduction [25]
Error Concealment
UMCP ENEE631 Slides (created by M.Wu © 2001)
(a) original lenna image
(b) corrupted lenna image
25% blocks in a checkerboard
pattern are corrupted
(c) concealed lenna image
corrupted blocks are concealed
via edge-directed interpolation
Examples were generated using the source codes provided by W.Zeng.
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [26]
2009 International Conf. on Image Processing (ICIP)
According to the Call-for-Paper
(Cairo, Egypt, Nov. 2009)

16th in the series (since 1994)

Research frontiers ranging from traditional image processing
applications to evolving multimedia and video technologies

Areas of interest include but are not limited to:
– Image/Video Coding and Transmission: Still image coding, video coding,
–
–
–
–
–
http://icip2009.org/
stereoscopic and 3-D coding, distributed source coding, . . .
Image/Video Processing: filtering, restoration, enhancement, segmentation, video
segmentation and tracking, morphological processing, stereoscopic and 3-D
processing, feature extraction and analysis, interpolation and super-resolution,
motion detection and estimation, . . .
Image Formation: Biomedical imaging, remote sensing, geophysical and seismic
imaging, optimal imaging, synthetic-natural hybrid image systems
Image Scanning, Display, and Printing: Scanning, sampling, quantization and
halftoning, color reproduction, image representation and rendering, …
Image/Video Storage, Retrieval, and Authentication: Image/video databases,
image/video indexing and retrieval, multimodality image/video indexing and
retrieval, authentication and watermarking
Applications: biomedical sciences, geosciences and remote sensing, . . .
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [27]
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
So What’s a Digital Image After All?
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [28]
What is an Image?
UMCP ENEE631 Slides (created by M.Wu © 2007)

What we perceive as a grayscale image is a pattern of light
intensity over a 2-D plane (aka “image plane”)
– Described by a nonnegative real-valued function I(x,y)
of two continuous spatial coordinates on an image plane.
– I(x,y) is the intensity of the image at the point (x,y).
– An image is usually defined on a bounded rectangle for processing
I: [0, a]  [0, b]  [0, inf )

x
Color image
– Can be represented by three functions:
R(x,y) for red, G(x,y) for green, B(x,y) for blue.
y
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [30]
Different Ways to View an Image
(More generally, to view a 2-D realvalued function)
Intensity visualization over 2-D (x,y) plane
In 3-D (x,y, z) plot with z=I(x,y);
red color for high value and blue for low
Equal value contour in (x,y) plane
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Example from B. Liu – EE488 F’06 Princeton
Lec1 – Introduction [31]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Sampling and Quantization

Computer handles “discrete” data.

Sampling
– Sample the value of the image at the nodes of a
regular grid on the image plane.
– A pixel (picture element) at (i, j) is the image intensity
value at grid point indexed by the integer coordinate
(i, j).

Quantization
255 (white)
– Is a process of transforming a real valued sampled
image to one taking only a finite number of distinct
values.
– Each sampled value in a 256-level grayscale image is
represented by 8 bits.
0 (black)
=> Stay tuned for the theories on these in future weeks.
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [32]
Recall: 1-D Sampling Theorem
UMCP ENEE631 Slides (created by M.Wu © 2002)

1-D Sampling Theorem
– A 1-D signal x(t) bandlimited within [-B,B] can be uniquely
determined by its samples x(nT) if s > 2B (i.e. sample fast enough).
– Using the samples x(nT), we can reconstruct x(t) by filtering the impulse
version of x(nT) by an ideal low pass filter

Sampling below Nyquist rate (2B) cause Aliasing
Xs() with s > 2B
 Perfect Reconstructable
-s
Xs() with s < 2B  Aliasing
B
B
s=2/T
s=2/T
=> Will extend sampling theorem to 2-D later in the course
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [33]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Examples of Sampling
256x256
64x64
16x16
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [36]

UMCP ENEE631 Slides (created by M.Wu © 2001)
Examples of Quantizaion
8 bits / pixel
4 bits / pixel
2 bits / pixel
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [37]
An Ancient Example of Digital Image

An Old “Digital”
Picture
(from a small church in
Crete Island, Greece)
=> Colored tiles as
“pixels”
Slide from B. Liu – EE488 F’06 Princeton
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [38]
UMCP ENEE631 Slides (created by M.Wu © 2001)
Summary of Today’s Lecture

Course organization and policies

Background and examples of digital image processing

Sampling and quantization concepts for digital image

Next time
– Color and Human Visual System
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [39]
UMCP ENEE631 Slides (created by M.Wu © 2001,2004,2009)
Readings and Assignment

Introductory sections in Matlab Image Processing Toolbox
– http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml

Gonzalez-Wood book, Chapter 1

Bovik’s Handbook – Section 1 Introduction (see course web)

Go over mathematical preliminaries
–
–
–
–
Linear system and basics of 1-D signal processing
FT and ZT
Matrix and linear algebra
Probability
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [40]
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [41]
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
Color of Light

Perceived color depends on spectral content
(wavelength composition)
– e.g., 700nm ~ red.
– “spectral color”

A light with very narrow bandwidth
“Spectrum” from http://www.physics.sfasu.edu/astro/color.html

A light with equal energy in all visible bands
appears white
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [42]
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
Perceptual Attributes of Color

Value of Brightness
(perceived luminance)

Chrominance
– Hue

specify color tone (redness, greenness,
etc.)

depend on peak wavelength
– Saturation




describe how pure the color is
depend on the spread (bandwidth) of
light spectrum
reflect how much white light is added
RGB  HSV Conversion ~ nonlinear
M. Wu: ENEE631 Digital Image Processing (Spring'09)
HSV circular cone is from online
documentation of Matlab image
processing toolbox
http://www.mathworks.com/access
/helpdesk/help/toolbox/images/col
or10.shtml
Lec1 – Introduction [43]
Questions for Today (QFT)

“Seeing yellow” figure is from B.Liu ELE330 S’01 lecture
notes @ Princeton; primary color figure is from Chapter
6 slides at Gonzalez/ Woods DIP book website
Why “seeing yellow
without yellow”?
– mix green and red light to
obtain the perception of
yellow, without shining
a single yellow light
570nm
520nm
630nm
=
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Lec1 – Introduction [44]