Characterization Presentation
Neural Network
Implementation On
FPGA
Supervisor: Chen Koren
Maria Nemets 309326767
Maxim Zavodchik 310623772
Introduction





Neural Network is a sort of artificial intelligence
NN is an imitation of a human body
A NN acquires knowledge through learning.
NN can learn in real time
Applications:




Optical Characters Recognition
Voice Recognition
Financial Forecasting
Medical Diagnosis
Theoretical Background

Neuron Model (Perceptron):




The single neuron is am device that implements the
function Activate _ func( w j x j  w0 )
j 1
Activate function – some bounded continuously rising
function (for example, sigmoid).
X is an input vector
W is a weight vector
 Knowledge is stored
within inter-neuron
connection strengths
Learning of the neural network
 Popular implementation is a multilayer feed forward




network
Neural network learns from examples
Learning is changing weights to implement the desired
classification function
There are algorithms which change the weights in the
network
We will use Back Propagation learning algorithm


Minimizes the difference between output and training example.
Weight changes depend on the units in the layer below.
Project Objectives





Implementing Neural Network on FPGA
Creating modular design
Implementing in software (Matlab)
Creating PC Interface
Performance Analyze:
 Area on chip
 Interconnections
 Speed vs. software implementation
 Frequency
 Cost
Development Stages










Choosing application: 0-9 Digits Recognition
Implementing a single neuron in VHDL
Determining network structure
Building NN in Matlab, training and testing it
Determining weights length and resolution
Choosing the proper hardware: FPGA and Memory
Planning FSM controller
Implementing the NN in VHDL
Simulation and Synthesis
Performance analysis: hardware vs. software
System Interface

Inputs




Binary image ( 8x8 pixels )
Weights – The size will be determined
Getting inputs by blocks
Outputs

Vector size of 4 bits coding the digit or indicating a
failure
Neuron Architecture






Multipliers for calculating wj x j
m
Adder for calculating the  w j x j  w0
j 1
ROM implementing Activation function
SRAM saving the weights
SRAM saving the input image
FSM Controller
Neuron Architecture
Hardware Requirements

Memec Development Board

Virtex II Pro FPGA



44 18x18-bit Multipliers
88 KB of on-chip memory
Power PC 4.05 CPU core
32 MB of SDRAM
 100MHz & 125 MHz clocks
 2x16 LCD panel
 Serial port interface

Hardware Requirements
Load
Weights
To
SDRAM
Return
Result To
User
Load Image
From User
Load
Weights
To
BRAM
Calculate
Output
Calculate
Intermediate
Sum
Special Problems

Problem: NN interconnection complexity
Solution: Designing sub-networks for each digit

Problem: Numbers representation
Solution: Fixed-Point representation avoiding Floating
Point operations

Problem: Determining accuracy
Solution: Choosing proper resolution from Matlab simulations

Problem: Trade-off between Area and Concurrency
Solution: Searching for big FPGA which enables
maximal parallelizing
Special Problems

Problem: Modulation vs. Speed
Solution: Simulating the hardware NN and finding the trade-off

Problem: Large Memory size
Solution: Finding FPGA with big memory or using outside
memory cores
Problem: Sigmoid function is continuous and infinite
Solution: - Function quantization according
to our resolution
- Saving values for arguments in limited range

Schedule
Date
Assignment
31.10-5.11
Learning the theoretical background
Training and testing an example NN in Matlab
7.11-12.11
Implementing a single Neuron in VHDL, synthesizing
and checking its area
14.11-19.11
Finding suitable Hardware solution (Area, Memory)
17.11
Characterization Presentation
21.11-3.12
Building training set
Building our NN in Matlab, training and testing
Determining weight length and needed resolution
5.12-10.12
Determining NN structure
Proposing final architecture
Schedule
Date:
Assignment
12.12-17.12 Implementing Sigmoid function
Planning the FSM Controller
19.12-7.01
20.12
9.01-21.01
Implementing the NN in VHDL
Midterm presentation
Simulating the NN
Synthesis
23.01-28.01 Analyzing performance
Writing project book
21.02
Final presentation