Designing Custom Functional Units (CFUs) in
the CFU Playground Framework
- Sohan Sri Dutta
CFU- Playground
●
●
●
●
●
●
Soft RISCV CPU (Open source soft CPU - VexRiscv) + your CFU→ can use any
FPGA
For TinyML - inference, quantized, constrained resources
Specialize the ML library AND the CPU+CFU for your particular model
Make it easy; allow quick iterations
Fully open source not tied to any FPGA vendor
Tailored for using an actual FPGA board (Artix -7) (but there are alternatives)
First CFU - SIMD MAC
Running Inference 1 tick = 1024 clock cycles
Notice how most of the time is consumed
by the Conv_2D operation
It’s Kernel can be targeted
First CFU - SIMD MAC
Software Optimizations ●
Adding some print statements for analysis
reveals always constant variables
●
●
Loop Unrolling
Sped upto 87M cycles
Hardware Optimizations
Simple Verilog/Amaranth Gateware
to unload the calculation to hardware
About 52% Improvement in performance
with minimal effort
Acceleration of Sparse DNNs
●
●
●
●
●
DNN Models can be pruned to not be dense with non-zero values
There exists a trade-off between accuracy and sparsity with varying success
among different pruning methods
Custom RISC-V Instructions that help in efficiently processing sparse models
are a viable way of saving computational resources
The Paper Proposes 3 types of CFUs for sparsity acceleration:
SSSA, USSA and CSA
Semi-Structured Sparsity accelerators
●
●
The author proposes lookahead - encoding to easily predict block skips based
on a simple formula.
This step involves preprocessing the weights to contain the information about
subsequent blocks
Semi-Structured Sparsity Accelerator (SSSA)
●
●
●
This preprocessing involves sacrificing the last bit of the INT8 weights to
create a 4 bit skip block with information on subsequent blocks
The author claims that this sacrifice does not adversely affect DNN accuracy
A specialised CFU is written that acts as a MAC for the 7bit weights while also
calculating the increment value for skipping blocks
Semi-Structured Sparsity Accelerator (SSSA)
●
●
●
This preprocessing involves
sacrificing the last bit of the INT8
weights to create a 4 bit skip block
with information on subsequent
blocks
The author claims that this sacrifice
does not adversely affect DNN
accuracy
A specialised CFU is written that
acts as a MAC for the 7bit weights
while also calculating the increment
value for skipping blocks
Unstructured Sparsity Accelerator (USSA)
●
●
●
●
Generates 4-bit control signal to check for 0 values
Uses Muxes to pass and align input values and weights
Uses only as many cycles as non-zero weights in the block
Uses 1 cycle in case of fully 0 block
Combined Sparsity Accelerator (CSA)
●
●
Combines the capabilities of SSSA and USSA to efficiently manage both
intra-block and inter block Zeroes.
Uses both lookahead encoding and variable cycle MAC unit
Published Results
Roadblocks
●
Unfortunately, commercial libraries like tfmot: Tensorflow Model Optimization
●
●
only support m/n sparsity where m<n
The Author uses custom pruning approaches detailed in his other research work
The CFU-Playground framework has a very inactive community with less than ideal
documentation
●
Support for models, Boards and CPU other than those listed for testing in the
framework is limited
Learnings
●
●
●
Was able to learn and build some CFUs on the CFU-Playground Framework
Built and simulated/tested the SSSA and USSA instructions on CFU-P
Got to work with Tensorflow Lite and Tensorflow Model Optimisation for
editing, processing and analysis of models
Possible Future Endeavours
●
●
Complete testing and processing of CSA along with pruning and lookahead
encoding
Try to add support for other FPGA boards like Zedboard for physical
implementation
Thank
You