Research Activities at Florida State Vision Group
Xiuwen Liu
Florida State Vision Group
Department of Computer Science
Florida State University http://fsvision.cs.fsu.edu
Group members: Lei Cheng, Donghu Sun, Yunxun Wang,
Chris Waring, Qiang Zhang,

Outline
 Introduction
• What is my research all about?
 Some applications of computer vision
• How useful are the computer vision techniques?
 Samples of my research work
• What have I done?
 Some of the research projects in my group
• What is going on within my group?
 Contact information
• How to contact me?

Introduction
 An image patch represented by hexadecimals

Introduction
- continued

Introduction
- continued
 Fundamental problem in computer vision
•
Given a matrix of numbers representing an image, or a sequence of images, how to generate a perceptually meaningful description of the matrix?
– An image can be a color image, gray level image, or other format such as remote sensing images
– A two-dimensional matrix represents a signal image
– A three-dimensional matrix represents a sequence of images

A video sequence is a 3-D matrix

A movie is also a 3-D matrix

Introduction
- continued

•
It is very interesting theoretically
– It involves many disciplines to develop a computational model for the problem
•
It has many practical applications
–
Internet applications
–
Movie-making applications
–
Military applications

Computer Vision Applications
 Eye Vision
•
Developed by Carnegie Mellon
•
It captures a dynamic event using multiple cameras and it can then synthesize new views
• http://www.ri.cmu.edu/events/sb35/tksuperbowl.html

Computer Vision Applications
- continued
 No hands across America
• sponsored by Delco Electronics, AssistWare Technology, and Carnegie Mellon University
•
Navlab 5 drove from Pittsburgh, PA to San Diego, CA, using the RALPH computer program.
•
The trip was 2849 miles of which 2797 miles were driven automatically with no hands
–
Which is 98.2%

Computer Vision Applications
– continued

Computer Vision Applications
– continued

Computer Vision Applications
– continued
 Military applications
•
Automated target recognition

Computer Vision Applications
– continued

Computer Vision Applications
– continued
 Extracted hydrographic regions

Computer Vision Applications
– continued
 Medical image analysis
•
Characterize different types of tissues in medical images for automated medical image analysis

Computer Vision Applications
– continued

Computer Vision Applications
– continued
 Biometrics
•
From faces, fingerprints, iris patterns .....
•
It has many applications such as ATM withdrawal, credit card managements .....

Computer Vision Applications – cont.

http://www.cl.cam.ac.uk/users/jgd1000/iris_recognition.html
• Companies in several countries are now using these algorithms in a variety of products.
Information about them can be found on the following websites:
–
Iridian Technologies, USA
–
IrisAccess LG Corp, South Korea
–
IrisPass OKI Electric Industries, Japan
–
EyeTicket Eyeticket Corporation, USA (ticketless air travel)
–
NCR CashPoint Machines NCR Corp, UK
–
Diebold ATMs Diebold Inc., USA
–
British Telecommunications, UK
–
The Nationwide Building Society, UK

Computer Vision Applications – cont.

Computer Vision Applications
– continued
 Content-based image retrieval has been an active research area to meet the needs of searching images on the web in a meaningful way
• Color histogram has been widely used

Content-Based Image Retrieval – cont.

Content-Based Image Retrieval – cont.
Query Image
1st 2nd 3rd 4th 5th

Vision-Based Image Morphing

Vision-Based Image Morphing
- continued

My Research Work in the Last Few Years
 Image modeling and synthesis
 Low dimensional representations of images for recognition
 Analytical probabilistic models of images

Image Modeling
 Is there a common feature that characterizes all these images perceptually?

Spectral Representation
– continued
 Given a set of filters, a spectral representation of an image consists of the marginal distributions of the filtered images.
Input image Its spectral representation

Deriving Spectral Representation
 Partition of the frequency domain
Partitioning Filters in Frequency and Spatial Domain
A filter as a surface

Deriving Spectral Representation
- continued
 Learning filters from training images as independent filters
(a) (b)
(c)

Image Modeling
- continued
 Image synthesis
• Given some feature statistics, how to generate samples from the Julesz ensemble
•
The main technical difficulty is the dimension of the image space
– If the image size is 256x256 and each pixel can have 8 values, there are 8 65536 different images
•
Markov chain Monte-Carlo algorithms

Image Synthesis Through Sampling
 Given observed feature statistics {H
(a) obs
}, we associate an energy with any image I as
Ε (I)
 a
K 

1 z
| H
I
( a
)
( z )

H
( a obs
)
( z ) | p
 Then the corresponding Gibbs distribution is q (I)

1
Z
T exp(

E (I)
)
T
• The q ( I ) can be sampled using a Gibbs sampler or other
Markov chain Monte-Carlo algorithms

Texture Synthesis Through Sampling
- continued
Image Synthesis Algorithm
•
•
•
• Compute {H obs
} from an observed texture image
Initialize I syn as any image, and T as T
0
Repeat
Randomly pick a pixel v in I syn
Calculate the conditional probability q( I syn
( v )| I syn
(v ))
Choose new I syn
( v ) under q( I syn
( v )| I syn
(v ))
Reduce T gradually
Until E(I) < e

A Texture Synthesis Example
Observed image Initial synthesized image

A Texture Synthesis Example
Temperature Image patch Energy Conditional probability
 Energy and conditional probability of the marked pixel

A Texture Synthesis Example
- continued
 A white noise image was transformed to a perceptually similar texture by matching the spectral histogram
Average spectral histogram error

A Texture Synthesis Example
- continued
 Synthesized images from different initial conditions

Texture Synthesis Examples
- continued
Observed image
 A random texture image
Synthesized image

Texture Synthesis Examples
- continued
Observed image Synthesized image
 An image with periodic structures

Texture Synthesis Examples
- continued
Mud image
Synthesized image
 A mud image with some animal foot prints

Texture Synthesis Examples
- continued
Observed image
Synthesized image
 A random texture image with elements

Texture Synthesis Examples
- continued
Original cheetah skin patch
 A cheetah skin image
Synthesized image

Texture Synthesis Examples
- continued
Observed image
 An image consisting of circles
Synthesized image

Texture Synthesis Examples
- continued
Observed image Synthesized image
 An image consisting of crosses

Texture Synthesis Examples
- continued
Observed image
Synthesized image
 A pattern with long-range structures

Comparison with Texture Synthesis Method

Example from Heeger and Bergen’s algorithm
(1995)*
Observed image Heeger and Bergen’s Our result
* Implemented by T. F. El-Maraghi, available at http://www.cs.toronto.edu/~tem/2522/texture.html

Comparison with Texture Synthesis Method
- continued

Another example from Heeger and Bergen’s algorithm
Cross image Heeger and Bergen’s Our result

Low Dimensional Representations of Images for Recognition
 In recent years, as a means of dimension reduction, principal component analysis, fisher discriminant analysis, and independent component analysis are widely used in appearance-based recognition
•
Each object type is represented by a representative set of training images using a linear subspace
•
A classifier is learned based on the training set
•
A new image is classified based on its linear representation

 Under the linear representation, an observed image window I is assumed to be generated by a linear
S
 combination of K hidden factors :
1
S

K
 y
 x i

S i


S x i
K 

1
 Under the linear assumption, recovering the representation of given an input is through pseudo inverse, given by:
 x


W y

Linear Subspaces of Images
– continued
 In the linear representation framework, each pixel is associated with a random variable
• A critical assumption is that each pixel needs to correspond to a meaningful event for the subsequent analysis to be meaningful
• This assumption, however, is often not valid due to translation, scaling, and other deformations

Spectral Representation for Recognition
 To make the assumption valid under some deformations, we propose a spectral representation
•
We represent each image by the underlying probability under the linear assumption, not the vector given by the projection onto a basis
•
This is done by breaking the images into roughly independent channels, representing each by its marginal
•
We then use linear subspaces in the spectral representation space, resulting IPCA, IICA, and IFDA

Comparison of Spaces Through Synthesis
 Synthesis using eigen face representations
Original Reconstructed Typical samples with identical eigen representations

Comparison of Spaces Through Synthesis
- continued
 Spectral representations capture perceptually important characteristics of images
Original Typical samples by matching spectral representations

Comparison of Spaces Through Synthesis
- continued
 Texture synthesis

Comparison of Clustering in Subspaces
In image space
(Two principal subspaces)
In spectral space
(Two principal subspaces)

Comparison Through Recognition Experiments
 The subspaces in the original image imply a linear generalization
• That is, the representation can not differentiate images that have the identical projection
 The subspaces in the spectral representation imply a nonlinear generalization
• Images with similar local and global structures are grouped together
• Those images can be very different in the original image space

COIL Dataset

3D Object Recognition Experiment
 We divide the images into a training and test set
•
We use the nearest neighbor rule as the classifier

3D Object Recognition Experiment
- continued
 Recognition with respect to translation

ORL Face Dataset

Face Recognition Result
 Here we divide the set randomly into a training and test set
• Here we have repeated the experiment 100 times

Dataset for Texture Classification

Texture Classification Results
 Average classification rate for 100 trials on the dataset with respect to different number of filters
•
Here linear image subspace methods do not perform well because each pixel does not correspond to a meaningful event

Recognition Performance of Different Subspaces
 It is clear that the choice of bases also affects the recognition performance within a given space
•
By viewing each basis as a point on a Grassmann manifold, we generate intermediate bases by connecting different bases through geodesic and compute their performance
Intermediate bases between PCA and ICA in the image space and their performance

Recognition Performance of Different Subspaces
 Similarly, we study different bases in the spectral space
Intermediate bases between PCA and ICA in the spectral space and their performance

Optimal Linear Basis for Recognition
 Furthermore, we find the optimal linear basis by optimizing the performance through moving along the Grassmann manifold
• Here PCA basis is used as the initial one

Optimal Linear Basis for Recognition – continued
 Another example using ICA basis as the initial one

Optimal Linear Basis for Recognition – continued
 We have compared the optimal performance with PCA/ICA/FDA on ORL dataset with respect to the dimension of the subspace
Dotted line: PCA Dash-dotted line: ICA
Dashed line: FDA Solid line: Optimal

Analytical Probability Models for Spectral Representation
 Transported generator model
(Grenander and Srivastava, 2000) where
• g i
’s are selected randomly from some generator space G
• the weigths a i
’s are i.i.d. standard normal
• the scales r i
’s are i.i.d. uniform on the interval [0,L]
• the locations z i
’s as samples from a 2D homogenous
Poisson process, with a uniform intensity l
, and
• the parameters are assumed to be independent of each other

Analytical Probability Models
- continued
 Define
 Model u by a scaled

-density

Analytical Probability Models
- continued

Analytical Probability Models
- continued

Analytical Probability Models
- continued

Research Projects at Florida State Vision Group
 The long-term goal of this group is to become a world leading group in computer vision research
• Conceptual framework of computer vision and visual recognition
• Algorithms for computer vision problems

Low Dimensional Representations of Images for Recognition
 Lei Cheng, Qiang Zhang, and Xiuwen Liu,
•
We have proposed an independent spectral representation
 Optimal filters for visual recognition
•
Lei Cheng and Xiuwen Liu
 Low dimensional representations of image manifolds for recognition
•
Qiang Zhang and Xiuwen Liu

Face Recognition
 Xiuwen Liu
• Given some examples of faces, identify a person under different pose, lighting, and expression conditions

Face Recognition
– continued
 Faces of the same person under slightly different conditions

3D Model-Based Recognition

Face Detection
 Chris Waring and Xiuwen Liu
•
Find all faces in a given picture
•
Typical faces are available

Medical Image Analysis
 Yunxun Wang and Xiuwen Liu
•
Advances in medical imaging provide many new opportunities and challenges for computer vision research
•
Automated medical image analysis

Medical Image Analysis
- continued

Medical Image Analysis
- continued

Medical Image Analysis
- continued

Medical Image Analysis
- continued

Video Sequence Analysis
 Donghu Sun and Xiuwen Liu
•
Motion analysis based on correspondence
•
Video stream-based surveillance
• Video summary
Video sequence

Summary
 Florida State Vision group offers many interesting research topics/projects
• Efficient represent for generic images
•
Computational models for object recognition and image classification
•
Medical image analysis
•
Motion/video sequence analysis and modeling
•
They are challenging
• They are interesting

Contact Information
•
•
•
•
•
•
Web site at http://fsvision.fsu.edu
http://www.cs.fsu.edu/~liux
Email at liux@cs.fsu.edu
Office at LOV 166
Office hours Tuesdays and Thursdays 9:15-10:45PM
Phone 644-0050
Courses
•
Principles and Algorithms of Computer Vision – Fall 2002
•
Theoretical Foundations of Computer Vision – Spring 2003