Image Classification Using Deep Neural Networks
Kyle Geyser
April 30, 2014
Kyle Geyser
Image Classification Using Deep Neural Networks
TOC
1. Intro
2. Background
3. Implementation
4. Results
5. Demo
6. Questions
Kyle Geyser
Image Classification Using Deep Neural Networks
Intro
I
Image Classification
Kyle Geyser
Image Classification Using Deep Neural Networks
Background
I
Neural Network
I
Single Neuron
Kyle Geyser
Image Classification Using Deep Neural Networks
Background
I
Neural Network (continued)
I
Fully Connected Network
Kyle Geyser
Image Classification Using Deep Neural Networks
Background
I
Image Pre-Processing
I
Principle Component Analysis (PCA)
I
I
Dimension reduction algorithm
Images can be somewhat redundant (neighboring pixels often
very similar in intensity)
Kyle Geyser
Image Classification Using Deep Neural Networks
Background (PCA)
Kyle Geyser
Image Classification Using Deep Neural Networks
Background (PCA)
Kyle Geyser
Image Classification Using Deep Neural Networks
Background (PCA)
Kyle Geyser
Image Classification Using Deep Neural Networks
Background (PCA)
Kyle Geyser
Image Classification Using Deep Neural Networks
Background
I
Image Pre-Processing (continued)
I
Whitening
I
I
PCA uncorrelates the data, now to scale the data in such a
way to give each input feature
unit variance
√
Rescale each xrot,i by 1/ λi .
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
I
Pre-process images
Run through trained classifier
I
I
I
I
Stacked auto-encoders
Softmax classifier
Convolution and pooling
Deep neural network
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Pre-processing
Apply PCA and whitening to the images, retaining 99%
variance.
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Auto-encoders
I
I
I
Neural network which trains in an unsupervised setting
Target values equal to input
One hidden layer
Kyle Geyser
Image Classification Using Deep Neural Networks
Auto-Encoder Examples
8x8 image patches, 25 hidden units
Kyle Geyser
Image Classification Using Deep Neural Networks
Auto-Encoder Examples
28x28 handwritten digit images (5-9), 196 hidden units
Kyle Geyser
Image Classification Using Deep Neural Networks
Auto-Encoder Examples
8x8 color image patches, 400 hidden units
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Stacked auto-encoders
Output of one auto-encoder is used as input for another
Kyle Geyser
Image Classification Using Deep Neural Networks
Stacked AE
All layers are stacked together to form one network
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Softmax classifier
I
I
I
I
Supervised learning
Extension of single neuron example from beginning (logistic
regression)
k different classes
k probabilities summing to 1
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Convolution
I
I
I
Natural images are stationary, i.e. statistics are roughly the
same everywhere in the image.
Use features learned on small patches and convolve them with
the whole image to get feature values at each location.
This allows a model to be trained on small patches, but still
allows for larger images to be run through the model.
Kyle Geyser
Image Classification Using Deep Neural Networks
Implementation
I
Pooling
I
I
I
Due to the stationary property of natural images, we can pool
features of nearby pixels
Mean pooling finding the mean of all features in some area
A way to reduce prediction calculations on large images
Kyle Geyser
Image Classification Using Deep Neural Networks
Results
I
4 image classes (car, plane, cat, dog)
I
Double stacked auto-encoders with 400 hidden units
(pre-trained on 10,000 color patches)
Trained softmax classifier on 2000 64x64 labeled color images
from STL-10 dataset
I
I
I
8x8 patches used for features → 57x57 convolved feature
matrices
19x19 mean pooling regions → 3x3 pooled features passed on
to softmax classifier
I
Tested on 3200 64x64 labeled color images from STL-10
dataset
I
≈80.2% accurate (varies slightly depending on how well the
model is trained)
Kyle Geyser
Image Classification Using Deep Neural Networks
Demo
DEMO
Kyle Geyser
Image Classification Using Deep Neural Networks
Questions
QUESTIONS?
Kyle Geyser
Image Classification Using Deep Neural Networks