Subspace Clustering Ali Sekmen Computer Science College of Engineering Tennessee State University 1st Annual Workshop on Data Sciences Outline Subspace Segmentation Problem Motion Segmentation Principal Component Analysis Dimensionality Reduction Spectral Clustering Presenter Dr. Ali Sekmen Subspace Segmentation In many engineering and mathematics applications, data lives in a union of low dimensional subspaces Motion segmentation Facial images of a person with the same expression under different illumination approximately lie on the same subspace Face Recognition Problem Statement Problem Statement Problem Statement What are we trying to solve? Example – Motion Segmentation Motion Segmentation Motion segmentation problem can simply be defined as identifying independently moving rigid objects in a video. Motion Segmentation We will show that all trajectories lie in a 4-dim subspace of Motion Segmentation Z Z p z x Y Y X X y Motion Segmentation Z p z x Y X y Motion Segmentation Z p z x Y X y Motion Segmentation Motion Segmentation Y X Motion Segmentation Y X Motion Segmentation Motion Segmentation Principal Component Analysis The goal is to reduce dimension of dataset with minimal loss of information We project a feature space onto a smaller subspace that represent data well Search for a subspace which maximizes the variance of projected points This is equivalent to linear least square fitting Minimize the sum of squared distances between points and subspace We find directions (components) that maximizes variance in dataset PCA can be done by Eigenvalue decomposition of a data covariance matrix Or SVD of a data matrix Least Square Approximation Principal Component Analysis Principal Component Analysis PCA with SVD Coordinates w.r.t. new basis Principal Component Analysis inch cm Principal Component Analysis inch cm 10 28 12 19 15 40 20 47 23 56 26 69 Solution with SVD PCA: Pre-Processing 80 inch cm 70 10 28 60 12 19 15 40 20 47 23 56 26 69 Inch 50 40 30 20 10 0 0 10 20 cm Zero Mean Zero Mean, Unit Variance -10 20 2 10 1 0 -5 0 5 10 inch inch 30 -2 0 -1 0 -10 -1 -20 -2 -30 cm cm 1 2 30 PCA: Optimization PCA: Reduce Dimensionality PCA: Reduce Dimensionality General PCA Spectral Clustering A very powerful clustering algorithm Easy to implement Outperforms traditional clustering algorithms Example: k-means It is not easy to understand why it works Given a set of data points and some similarity measure between all pairs of data points, we divide data into groups Points in the same group are similar Points in different groups are dissimilar Spectral Clustering Most of subspace clustering algorithms employ spectral clustering as the last step Similarity Spectral Clustering Spectral Clustering Spectral Clustering Spectral Clustering Example From Lecture Notes of Ulrike von Luxburg Spectral Clustering Example From Lecture Notes of Ulrike von Luxburg Spectral Clustering Example From Lecture Notes of Ulrike von Luxburg Spectral Clustering Example From Lecture Notes of Ulrike von Luxburg Spectral Clustering Example From Lecture Notes of Ulrike von Luxburg Spectral Clustering Example Spectral Clustering Example Spectral Clustering Example