Handwriting Recognition Using Neural Networks
Troy Sornson
April 22, 2014
Troy Sornson
Handwriting Recognition Using Neural Networks
Overview of Presentation
The Point of the Project
Methods for Handwriting Recognition
Artificial Neural Networks
My Results
Improvements
Troy Sornson
Handwriting Recognition Using Neural Networks
What’s the Point?
All about Digitizing written work
Machine Printed
Uniform
Constistent
Hand Written
Not Uniform
Varies widely between individuals
No ”Standard” way to do it
On-line Vs Off-line
Troy Sornson
Handwriting Recognition Using Neural Networks
Methods for Handwriting Recognition
Usually Broken into 3 Steps:
Preprocessing
Segmentation
Feature Extraction
Classification
Neural Networks
Rule Based
Troy Sornson
Handwriting Recognition Using Neural Networks
Preprocessing
Turn to Grayscale
Apply Gaussian Blur
Open the image
Binerize
Segmentation
Letters Vs Words
Cursive Vs Hand Printed
Troy Sornson
Handwriting Recognition Using Neural Networks
Feature Extraction
Really only limited by imagination
No ”Best” Feature, as long as features are unique
Some Popular Methods
Zoning
Average all pixels in zone
Average vertical/horizontal/diagonal lines through zones
Count Contours
HOG Features
Fourier Transformation Coefficients
Troy Sornson
Handwriting Recognition Using Neural Networks
Diagonal Zones
Troy Sornson
Handwriting Recognition Using Neural Networks
Classification
Match the feature vectors to a letter/word
Neural Networks
Quick to set up
Relatively easy to train
Will be covered next
Rule Based
Requires much more development time
However, no unknowns
Troy Sornson
Handwriting Recognition Using Neural Networks
Artificial Neural Networks - With Pictures!
O=A
n
X
!
Xi Wi + b
i=0
A(x) = tanh(x)
A(x) =
1
1 + e −x
A(x) = x
A(x) = threshold(x)
Troy Sornson
Handwriting Recognition Using Neural Networks
Activation function
1
1 + e −x
tanh(x)
Troy Sornson
Handwriting Recognition Using Neural Networks
Network
Troy Sornson
Handwriting Recognition Using Neural Networks
Backpropogation
After we feed our input forward, we need to adjust the
weights to get a better result
Need to calculate the error of the weights and propogate it
backwards through the network
Troy Sornson
Handwriting Recognition Using Neural Networks
Backpropogation Continued
i - Weight
j - Neuron
k - Layer
m - Number of neurons in above layer
Given p inputs, each with expected output t
E=
p
X
||Oi − ti ||2
i=0
wijk = wijk −∆wijk
output layer αj = (Oj − tj )
∆wijk = γOj δj
δj =
∂A
(x)(αj )
∂j
all other layers αj =
m
X
wi,q,k+1 δq,k+1
q=0
Troy Sornson
Handwriting Recognition Using Neural Networks
My Project
Preprocessing
Resize all letters to 90x60 pixels
Feature Extraction
Create zones of 10x10 pixels
Perform Diagonal averaging
Classification Neural Networks
26 Neural Networks (one for each letter)
54 inputs
2 Hidden layers
100 Neurons per Hidden Layer
Use Sigmoid Function Throughout
Troy Sornson
Handwriting Recognition Using Neural Networks
Input
Troy Sornson
Handwriting Recognition Using Neural Networks
Preprocessed
Troy Sornson
Handwriting Recognition Using Neural Networks
Results
35 characters total
Not all letters have been trained for yet
Missing D, E, I, J, K, U, V, W
Letters that have been trained might be over trained
Use ? in place of unknown letters
T H ? B ? ?
?
?
L O X ? ?
M ? S O H ? R T H ? L ? Z Y ? ?
G Q ? T C ?
T H E B R O W N F O X J U M P S O V E R T H E L A Z Y D O G Q U I C K
17 correct
15 unidentified
3 false positives
Just under %50 succes rate
Troy Sornson
Handwriting Recognition Using Neural Networks
Future Plans
Use different activation function
Use HOG features
Handle page rotation
Handle shadows
Troy Sornson
Handwriting Recognition Using Neural Networks
Questions
Questions?
Troy Sornson
Handwriting Recognition Using Neural Networks