Why D.I.P.?
Reasons for compression
– Image data need to be accessed at a different time or
location
– Limited storage space and transmission bandwidth
Reasons for manipulation
– Image data might experience nonideal acquisition,
transmission or display (e.g., restoration, enhancement and
interpolation)
– Image data might contain sensitive content (e.g., fight
against piracy, conterfeit and forgery)
– To produce images with artistic effect (e.g., pointellism)
Reasons for analysis
– Image data need to be analyzed automatically in order to
reduce the burden of human operators
– To teach a computer to “see” in A.I. tasks
Beyond Image Processing
The way of thinking
– From art (heuristics) to science (principles)
– The key is mathematics (I will write a separate
blog about the role of mathematics in DIP)
The holistic view
– Everything is connected (recall the six-degree
phenomenon in social science)
The “Google”-style re-search
– Ability to search is a basic part of learning
Image
Acquisition
D.I.P.
Theme
Park
Image
Generation
Image
Compression
Image
Manipulation
Image
Display
Image
Analysis
Image
Perception
DIP is also about connecting dots – in image compression, you
will see why you need to learn matrix theory and statistics
The Art of Image Compression
Why are images compressible?
– Redundancy in images (NOT random)
How data compression works?
– Probability theory and statistics
– Shannon’s information theory
What about the future of image
compression?
– I will discuss this in my weblog and facebook
(Google “what happened to Iterated Systems Incorporated?”)
Shannon’s Picture on
Communication (1948)
source
channel
encoder
channel
channel
decoder
destination
super-channel
source
encoder
source
decoder
The goal of communication is to move information
from here to there and from now to then
Examples of source:
Human speeches, photos, text messages, computer programs …
Examples of channel:
storage media, telephone lines, wireless transmission …
Lossless vs. Lossy Compression
Lossless: zero error tolerance
– No information loss
– Shannon’s entropy formula
– For photographic images, compression ratio is
modest (about 2:1)
Lossy: the goal is to preserve the visual quality of
images
– Information loss visually acceptable
– Shannon’s rate-distortion function
– For photographic images, compression ratio is
typically around 10-100
Popular Lossless Image
Compression Techniques
 WinZip
- Based on the celebrated Lempel-Ziv algorithm
invented nearly 30 years ago
 GIF (Graphic Interchange Format)
-Based on an enhanced version of LZ algorithm
by Welch in 1983
-Was introduced by CompuServe in 1987 and made
popular until it was not royalty-free in 1994
 PNG (Portable Network Graphics)
GIF Liberation Day: June 20, 2003
Lossy Image Compression
compressed JPEG file
(20,407 bytes)
100
Q
JPEG
decoder
Q
low compression ratio
decompressed image
high quality
high compression ratio
0
low quality
original raw image
(262,144 bytes)
From JPEG to JPEG2000
discrete cosine transform based
JPEG (CR=64)
wavelet transform based
JPEG2000 (CR=64)
Image
Acquisition
D.I.P.
Theme
Park
Image
Generation
Image
Compression
Image
Manipulation
Image
Display
Image
Analysis
Image
Perception
DIP is also about connecting dots – in image manipulation, you
will see why you need to learn calculus and Fourier transform
Image Manipulation (I):
Noise Removal
Noise contamination is often inevitable during the acquisition
salt and pepper (impulse) noise additive white Gaussian noise
You will learn how to design image filter in a principled way
Image Manipulation (II):
Deblurring
License plate is barely legible due to motion blurring
You will learn the use of FT and the necessity of regularization
Image Manipulation (III):
Contrast Enhancement
under-exposed image
overly-exposed image
You will learn how to modify the histogram of an image
Image Manipulation (IV):
Aliasing Reduction
Example: aliasing artifacts in MRI image acquisition
Ideal quality,
slow scanning
nonideal quality,
fast scanning
Tradeoff between scanning time and image quality
(image reconstruction is covered by EE425)
Image Manipulation (V):
Image Interpolation
digital
zooming
small
1M pixels
large
4M pixels
Resolution enhancement can be obtained by common image
processing software such as Photoshop or Paint Shop Pro
You will learn the difference between digital and optical zooming
Image Manipulation (VI):
Image Mosaicing
Merge multiple images of the same scene into one with larger FOV
+
=
There exist several mosaicing software for automatic stitching
F.Y.I.: search “Gigapixel images” by Google
http://triton.tpd.tno.nl/gigazoom/Delft2.htm
Image Manipulation (VII):
Error Concealment
blocks contaminated by channel errors
(this problem is covered in EE565)
Image Manipulation (VIII):
Deblocking/Deringing
Block artifacts
Ringing artifacts
You will learn how to suppress those artifacts by nonlinear diffusion
Image Manipulation (IX):
Dejittering
jittering noise (you will see it in either
bonus assignment or final project)
Image Manipulation (X):
Image Inpainting
You will learn how to do image inpainting in EE565
Image Inpainting Application:
Restore Old Photos
Image Manipulation (XI):
Color Quantization
25,680 colors (24 bits)
256 colors (8 bits)
Applications: video cell-phone, gameboy, portable DVD
Image Manipulation (XII):
Image Halftoning
grayscale: 0-255
halftoned: 0/255
You will learn the famous Floyd-Steinberg diffusion in CA
Image Manipulation (XIII):
Image Scrambling/Hashing*
original
scrambled
Image Manipulation (XIV):
Image Watermarking*
Original image
Modified image
Image Manipulation (XV):
Image Stylization*
Image Manipulation (XVI):
Image Rendering*
computer
generated
Abyss
Image-based Rendering*
Image
Acquisition
D.I.P.
Theme
Park
Image
Generation
Image
Compression
Image
Manipulation
Image
Display
Image
Analysis
Image
Perception
DIP is also about connecting dots – in image analysis, you
will see why you need to know about neuroscience and psychology
Image Analysis (I): Edge Detection
You will learn basic edge detectors based on derivatives
Image Analysis (II): Face Detection
Deceivingly simple for humans but notoriously difficult for machines
Image Analysis (III): Change Detection
Change Detection in Medical Application
Image Analysis (IV): Image Matching
Antemortem dental X-ray record
Postmortem dental X-ray record
Image Matching in Biometrics
Two deceivingly similar fingerprints of two different people
Image Analysis (V):
Image Segmentation
Image Analysis (VI): Object Recognition
License number can be automatically
extracted from the image of license plate
Object Recognition in Military
Applications
Image Analysis (VII): Event Recognition
Image-based monitoring system prevents drowning
Image Analysis (VIII):
Video Summarization
Only send out “important” motion pictures such as home-runs
Image Analysis (IX):
Content-based Image Retrieval
retrieved building images
Summary
In EE465, you will learn
– Image compression: Lempel-Ziv, Huffman coding, run-length
coding and JPEG
– Image manipulation: linear/nonlinear filtering, histogram-based
processing, linear interpolation
– Image analysis: edge/corner detection, circle/ line/ellipse
detection, chain codes/shape numbers
In EE565, you will learn
– Advanced algorithms/techniques with stronger mathematical
emphasis
Not covered by the courses I offered
– CBIR (multimedia database), face/pedestrian detection
(Advanced biometrics), PDE-based image processing (medical
image analysis)
General Cooking Recipe
Motivations
Problem statement
Heuristic observations
Examples/Illustrations
Principled approaches (via mathematics)
MATLAB implementations
Computer assignment reviews
Current state-of-the-art and future
directions