Subspace Clustering
Ali Sekmen
Computer Science
College of Engineering
Tennessee State University
1st Annual Workshop on Data Sciences
Outline
Subspace Segmentation Problem
Motion Segmentation
Principal Component Analysis
Dimensionality Reduction
Spectral Clustering
Presenter
Dr. Ali Sekmen
Subspace Segmentation
In many engineering and mathematics
applications, data lives in a union of low
dimensional subspaces
Motion segmentation
Facial images of a person with the same
expression under different illumination
approximately lie on the same subspace
Face Recognition
Problem Statement
Problem Statement
Problem Statement
What are we trying to solve?
Example – Motion Segmentation
Motion Segmentation
Motion segmentation problem can simply
be defined as identifying independently
moving rigid objects in a video.
Motion Segmentation
We will show that all
trajectories lie in a 4-dim
subspace of
Motion Segmentation
Z
Z
p
z
x
Y
Y
X
X
y
Motion Segmentation
Z
p
z
x
Y
X
y
Motion Segmentation
Z
p
z
x
Y
X
y
Motion Segmentation
Motion Segmentation
Y
X
Motion Segmentation
Y
X
Motion Segmentation
Motion Segmentation
Principal Component Analysis
The goal is to reduce dimension of dataset with
minimal loss of information
We project a feature space onto a smaller
subspace that represent data well
Search for a subspace which maximizes the variance
of projected points
This is equivalent to linear least square fitting
Minimize the sum of squared distances between points and
subspace
We find directions (components) that maximizes
variance in dataset
PCA can be done by
Eigenvalue decomposition of a data covariance matrix
Or SVD of a data matrix
Least Square Approximation
Principal Component Analysis
Principal Component Analysis
PCA with SVD
Coordinates w.r.t. new
basis
Principal Component Analysis
inch
cm
Principal Component Analysis
inch
cm
10
28
12
19
15
40
20
47
23
56
26
69
Solution with SVD
PCA: Pre-Processing
80
inch
cm
70
10
28
60
12
19
15
40
20
47
23
56
26
69
Inch
50
40
30
20
10
0
0
10
20
cm
Zero Mean
Zero Mean, Unit
Variance
-10
20
2
10
1
0
-5
0
5
10
inch
inch
30
-2
0
-1
0
-10
-1
-20
-2
-30
cm
cm
1
2
30
PCA: Optimization
PCA: Reduce Dimensionality
PCA: Reduce Dimensionality
General PCA
Spectral Clustering
A very powerful clustering algorithm
Easy to implement
Outperforms traditional clustering algorithms
Example: k-means
It is not easy to understand why it works
Given a set of data points and some
similarity measure between all pairs of
data points, we divide data into groups
Points in the same group are similar
Points in different groups are dissimilar
Spectral Clustering
Most of subspace clustering algorithms
employ spectral clustering as the last step
Similarity
Spectral Clustering
Spectral Clustering
Spectral Clustering
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
Spectral Clustering Example
Spectral Clustering Example
Spectral Clustering Example