Image Pattern Recognition and
Its Applications
Chaur-Chin Chen (陳朝欽)
Institute of Information Systems & Applications
(Department of Computer Science)
National Tsing Hua University
HsinChu (新竹), Taiwan (台灣)
cchen@cs.nthu.edu.tw
May 3, 2013
Outline
•
•
•
•
•
•
Fundamental Image Processing
Fingerprint and Face Verification
Supervised vs. Unsupervised Learning
Watermarking and Steganography
Microarray Image Analysis
Some Other Application
Outline (Continuation)
•
•
•
•
•
•
•
Some Other Applications
Supervised vs. Unsupervised Learning
Data Description and Representation
8OX and iris Data Sets
Dendrograms of Hierarchical Clustering
PCA vs. LDA
A Comparison of PCA and LDA
Fundamental Image Processing
♪ A Digital Image Processing System
• Image Representation and Formats
1. Sensing, Sampling, Quantization
2. Gray level and Color Images
3. Raw, RGB, Tiff, BMP, JPG, GIF, (JP2)
• Image Transform and Filtering
• Histogram, Enhancement
• Segmentation, Edge Detection, Thinning
• Image Data Compression
•
•
•
•
Fingerprint and Face Recognition
Image Pattern Recognition
Watermarking and Steganography
Microarray Image Data Analysis
[1] R.C. Gonzalez, R.E. Woods, S.L. Eddins, Digital Image Processing Using
MATLAB, Pearson Prentice Hall, 2004
[2] R.C. Gonzalez and R.E. Woods, Digital Image Processing, Prentice-Hall, 2002+
Image Processing System
• A 2D image is nothing but a mapping from a region to a matrix
• A Digital Image Processing System consists of
1. Acquisition – scanners, digital camera, ultrasound,
X-ray, MRI, PMT
2. Storage – HD (500GB, TeraBytes, PeraBytes, …), CD (700 MB),
DVD (4.7 GB), Flash memory (2~32 GB)
3. Processing Unit – PC, Workstation (Sun Microsystems), PC-cluster
4. Communication – telephone lines, cable, wireless, Wi-Fi, LTE
5. Display – LCD monitor, laser printer, smart phone, i-Pad
Illustration of Image Processing
System
Gray Level and Color Images
Pixels in a Gray Level Image
A Gray Level Image is a Matrix
f(0,0)
f(0,1)
f(0,2) ….
…. f(0,n-1)
f(1,0)
f(1,1)
f(1,2) ….
…. f(1,n-1)
.
.
.
.
.
.
.
.
.
f(m-1,0) f(m-1,1) f(m-1,2) … …. f(m-1,n-1)
An image of m rows, n columns, f(i,j) is in [0,255]
Image Representation (Gray/Color)
• A gray level image is usually represented by an
M x N matrix whose elements are all integers in
{0,1, …, 255} corresponding to brightness scales
• A color image is usually represented by 3 M x N
matrices whose elements are all integers in {0,1,
…, 255} corresponding to 3 primary primitives of
colors such as Red, Green, Blue
Gray and Color Image Data
•
0, 64, 144, 196,
225, 169, 100, 36
(R, G, B) for a color pixel
Red – (255, 0, 0)
Green – ( 0, 255, 0)
Blue – ( 0, 0, 255)
Cyan – ( 0,255, 255)
Magenta – (255, 0, 255)
Yellow – (255, 255, 0)
Gray – (128, 128, 128)
RGB Hex Triplet Color Chart
•
•
•
•
•
•
Red
= FF0000
Green = 00FF00
Blue
= 0000FF
Cyan
= 00FFFF
Magenta= FF00FF
Yellow = FFFF00
Koala and Its RGB Components
(R,G,B) Histograms of Koala
Sensing, Sampling,
Quantization
• A 2D digital image is formed by a sensor
which maps a region to a matrix
• Digitization of the spatial coordinates (x,y)
in an image function f(x,y) is called
Sampling
• Digitization of the amplitude of an image
function f(x,y) is called Quantization
Sampling and Quantization
Image File Formats (1/2)
The American National Standards Institute (ANSI) sets standards
for voluntary use in US. One of the most popular computer
standards set by ANSI is the American Standard Code for
Information Interchange (ASCII) which guarantees all computers
can exchange text in ASCII format
BMP – Bitmap format from Microsoft uses Raster-based 1~24-bit
colors (RGB) without compression or allows a run-length
compression for 1~8-bit color depths
GIF – Graphics Interchange Format from CompuServe Inc. is
Raster-based which uses 1~8-bit colors with resolutions up to
64,000*64,000 LZW (Lempel-Ziv-Welch, 1984) lossless
compression with the compression ratio up to 2:1
Some Image File Formats (2/2)
•
Raw – Raw image format uses a 8-bit unsigned character to store a pixel value of
0~255 for a Raster-scanned gray image without compression. An R by C raw image
occupies R*C bytes or 8RC bits of storage space
•
TIFF – Tagged Image File Format from Aldus and Microsoft was designed for
importing image into desktop publishing programs and quickly became accepted by a
variety of software developers as a standard. Its built-in flexibility is both a blessing
and a curse, because it can be customized in a variety of ways to fit a programmer’s
needs. However, the flexibility of the format resulted in many versions of TIFF, some
of which are so different that they are incompatible with each other
•
JPEG – Joint Photographic Experts Group format is the most popular lossy method
of compression, and the current standard whose file name ends with “.jpg” which
allows Raster-based 8-bit grayscale or 24-bit color images with the compression ratio
more than 16:1 and preserves the fidelity of the reconstructed image
•
EPS – Encapsulated PostScript language format from Adulus Systems uses Metafile
of 1~24-bit colors with compression
•
JP2 - JPEG 2000 based on 5/3 and 9/7 wavelet transforms
Image Transforms and Filtering
• Feature Extraction – find all ellipses in an
image
• Bandwidth Reduction – eliminate the low
contrast “coefficients”
• Data Reduction – eliminate insignificant
coefficients of Discrete Cosine Transform
(DCT), Wavelet Transform (WT)
• Smooth filtering can get rid of noisy
signals
Discrete Cosine Transform
Partition an image into nonoverlapping 8 by
8 blocks, and apply a 2d DCT on each
block to get DC and AC coefficients.
Most of the high frequency coefficients
become insignificant, only the DC term
and some low frequency AC coefficients
are significant.
Fundamental for JPEG Image Compression
Discrete Cosine Transform (DCT)
X: a block of 8x8 pixels
A=Q8: 8x8 DCT matrix as
shown above
Y=AXAt
Quantized DCT Coefficients on
a 8x8 Block
Lenna Image vs. Compressed
Lenna
Wavelet Transform
• Haar, Daubechies’ Four, 9/7, 5/3
transforms
• 9/7, 5/3 transforms was selected as the
lossy and lossless coding standards for
JPEG2000, respectively
• A Comparison of JPEG and JPEG2000
shows that the latter is slightly better
than the former, however, to replace
image.jpg by image.jp2 needs time
3-Scale Wavelet Transforms
Mean and Median Filtering
• X1 X2 X3
• X4 X0 X5
• X6 X7 X8
• X1 X2 X3
• X4 X0 X5
• X6 X7 X8
Replace the X0 by the
mean of X0~X8 is
called “mean filtering”
Replace the X0 by the
median of X0~X8 is
called “median filtering”
Example of Median Filtering
Image and Its Histogram
Histogram of Image Lenna
12
10
8
6
4
2
0
0
50
100
150
200
250
Enhancement and Restoration
• The goal of enhancement is to accentuate
certain features for subsequent analysis or
image display. The enhancement process
is usually done interactively
• The restoration is a process that attempts
to reconstruct or recover an image that
has been degraded by using some
unknown phenomenon
Example of Image Enhancement
• Support that A(i, j) is image gray level at
pixel (i, j), μ and s2 are the mean and
variance of gray levels of input image, and
α=150, γ=95, γ must satisfy γ>s.
The enhanced image B( i , j ) is obtained
by a contrast stretching given below
• B( i , j ) α + γ ＊ ([A ( i , j ) – μ]／s)
Result of Image Enhancement
Segmentation and Edge Detection
• Segmentation is basically a process of
pixel classification: the picture is
segmented into subsets by assigning the
individual pixels into classes
• Edge Detection is to find the pixels whose
gray values or colors being abruptly
changed
Image Lenna and Its Histogram
Image Segmentation Algorithms
•
•
•
•
•
Otsu (1979)
Fisher (1936)
Kittler and Illingworth (1986)
Vincent and Soille (1991)
Besag, Chen and Dubes (1986, 1991)
A Simple Thresholding Algorithm
(1)
ni
(1) p i 
, where n 
n
(2)  T 


G 1
i 0
ni
G 1
i 0
kpk
(3) Do for k  0 ~ G  1

 (k )  
 (k ) 
k
i 0
k
i 0
pi
ip i
2




(
k
)


(
k
)
T
 B2 
 ( k )1   k 
(4) Select k * such t hat B2 ( k ) is maximized
Image, Histogram, Thresholding
Histograms of NA.raw (Green), TA.raw (Red)
120
100
80
60
40
20
0
0
50
100
150
200
250
Binarization by Thresholding
ICM Segmentation Algorithm
1. Given an image Y, initialize a labeling X
2. For t=1:mxn
X(t)←g0 if
Pr(X(t)=g0|XN(t),Y) > Pr(X(t)=g|XN(t),Y) for g,g0
3. Repeat step 2 until “convergence” (6 runs)
4. X is the required labeling
Chaur-Chin Chen and Richard C. Dubes
Environmental Studies and ICM Segmentation Algorithm,
Journal of Information Science and Engineering,
Vol. 6, 325-337, 1990.
Image Segmentation: ICM vs. Otsu
Image Segmentation: ICM vs. Otsu
Image Segmentation: ICM vs. Otsu
Edge Detection
-1 -2 -1
0 0 0
1 2 1
X
-1 0 1
-2 0 2
Y
-1 0 1
Large (|X|+|Y|)  Edge
Thinning and Contour Tracing
• Thinning is to find the skeleton of an image
which is commonly used for Optical Character
Recognition (OCR) and Fingerprint matching
• Contour tracing is usually used to locate the
boundaries of an image which can be used in
feature extraction for shape discrimination
Image Edge, Skeleton,
Contour
Image Data Compression
• The purpose is to save storage space and to
reduce the transmission time of information.
Note that it requires 6 mega bits to store a 24-bit
color image of size 512 by 512. It takes 6
seconds to download such an image via an
ADSL (Asymmetric Digital Subscriber Line) with
the rate 1 mega bits per second and more than
12 seconds to upload the same image
• Note that 1 byte = 8 bits, 3 bytes = 24 bits
Training Images for VQ
LBG Algorithm for Codebook
Generation
Codebook and Decoded Images
Some Applications
•
•
•
•
Fingerprint and Face Recognition
Watermarking and Steganography
Image Pattern Recognition
Microarray Image Data Analysis
美國啟用出入境指紋及人臉影像辨
識系統
• 美國國土安全部基於安全考慮，自(2004)元
月五日起，啟用數位化出入境身分辨識系
統(US-VISIT)，大部分來美的14歲至79歲
旅客，包括來自台灣、大陸、香港的留學
生，於進入美國國際機場及港口時，都要
接受拍照及留下指紋掃描紀錄以便辨識查
核。(27個免簽證國公民之入境待遇略有不
同，短期來美者，將受豁免。)，亦將需接
受指紋掃描查核。
US-VISIT
• US-VISIT currently applies to all visitors
(with limited exemptions) holding nonimmigrant visas, regardless of country of
origin.
• 2004 – US$ 330 million
• 2005 – US$ 340 million
• 2006 – US$ 340 million
• 2007 – US$ 362 million
• 2009 – US$ ??? million
入境按指紋 日本2007/11/20實施
• 日本入境排隊長 指紋掃瞄會更長！ (2007
年9月27日)
• 入境日本將按指紋 日官員赴台宣導新措施
(2007年9月27日)
• 日11月20日實施外國人入境須按指紋臉部
照片 (2007年9月25日)
• 入境按指紋 日本11月將實施 (2007年9月2
日)
A Typical Fingerprint Image
Flowchart of An AFIS
(a) Original image
(c) Binarization image
(b) Enhanced image
(d) Smoothed image
Thinning [9]
• The purpose of thinning stage is to gain the skeleton
structure of a fingerprint image.
• It reduces a binary image consisting of ridges and valleys
into a ridge map of unit width.
(d) Smoothed image
(e) Thinned image
Minutiae Definition
♫ From a thinned image, we can classify each ridge pixel
into the following categories according to its 8-connected
neighbors.
♫ A ridge pixel is called :
 an isolated point if it does not contain any 8-connected
neighbor.
 an ending if it contains exactly one 8-connected
neighbor.
 an edgepoint if it has two 8-connected neighbors.
 a bifurcation if it has three 8-connected neighbors.
 a crossing if it has four 8-connected neighbors.
Example of Minutiae Extraction
Minutiae Pattern Matching
Is this Lady in your database?
Part of 5*40 Training Face Images
Missed Face Images and Their
Wrongly-Best Matched Images
Are They the Same Person?
Challenges and Opportunities
• A perfect biometric recognition system did not
exist and will never exists
• An application based on biometrics usually
requests a perfect verification/identification
• A collection of biometric data is usually time
consuming and more or less intrudes personal
privacy
• The mechanism of achieving the trade-off
between privacy and security merits studies.
Supervised Learning Problems
☺The problem of supervised learning can be
defined as to design a function which takes the
training data xi(k), i=1,2, …ni, k=1,2,…, C, as input
vectors with the output as either a single category
or a regression curve.
☺The unsupervised learning (Cluster Analysis) is
similar to that of the supervised learning problem
(Pattern Recognition) except that the categories
are unknown in the training data.
Distinguish Eggplants from
Bananas
1. Features(characteristics)
Colors
Shapes
Size
Tree leaves
Other quantitative
measurements
2. Decision rules: Classifiers
3. Performance Evaluation
4. Classification
Possum, Dingo, Fox, Wombat
Watermarking and Steganography
• Watermarking is the practice of hiding a
message about an image, audio clip, video clip,
or other work of media within that work itself.
• Steganography is the art of writing in cipher, or
in character, which are not intelligible except to
persons who have the key. In computer terms,
steganography has evolved into the practice of
hiding a message within a larger one in such a
way that others cannot discern the presence or
contents of the hidden message.
Examples of Watermarking and
Steganography
Difference between Watermarking
and Steganography
• Watermarking
• Steganography
Insert a logo, pattern, a
message, and etc. into
an image, audio, video
to claim the ownership.
Put a cover image,
audio, video, and etc.
on a secret message to
protect the secrecy
during the transmission.
An Example of Steganography
•
•
The Precious Night
by Tsui Ping
•
The southern winds lightly kiss my
face, with the heavy scent of blossms
The southern winds lightly kiss my?
face, but the stars are sparse and the
moon veiled
We lie against each other, exchanging
endless words of love
We lie against each other, meaning
everything we say
We don't care that tomorrow we may
bid each other farewell
But remember tonight, and treasure it
On the eve of parting, we rue the sun's
imminent rising
Lingering before parting, we promise
to meet in a dream
•
•
•
•
•
•
•
Microarray Image Data Analysis
Microarray Image Data Analysis
Each gene expression
is a feature which is
measured as average
spot brightness
Top: Tumor Tissues
Bottom: Normal Tissues
Bar Code and QR code
Face and Fingerprint Images
License Plate
Fort San Domingo (淡水紅毛城)
Entrance Gate
Dutch Clogs
Android APP
iGoogle APP
iPhone App
Facebook
Newsletter
RSS Feeds
LinkedIn
Twitter
Thank You For Your Attention
Questions and Comments