Programming Multi-Core
Processors based Embedded
Systems
A Hands-On Experience on Cavium
Octeon based Platforms
Lab Exercises
Lab # 4: Network Packet
Filtering
An overview
4-2
Lab Goals

Objective




Learning parallel programming using threads
Utilizing many core systems efficiently
Performance measurement
Packet capture / filter / analyze - A case
study

We will use series of labs to achieve our
objectives. Today’s lab is about packet filtering
4-3
Prerequisites

Sniffing


Capturing of network packets arriving or departing
from a network interface
Mechanism

We use raw sockets as follows
rawSock = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL))

This system call picks every packet going out or
coming in on an Ethernet interface
4-4
Prerequisites

Testing


You can use loop back device as a network
interface
Use Netperf or MPAC for traffic generation on the
network interface
4-5
Lab Setup
0
2
4
6
0
2
4
6
1
3
5
7
1
3
5
7
System 1
1 GigE
Link
Sender
Packet
Sniffer
System 2
Receiver
Core
Packet Mapping to Cores
Data Packets
4-6
Sniffing Labs Framework

Sniffing


One thread, called the dispatcher, sniffs the
packets from the interface and puts it in one of
the workers’ queues
Filtering / Analysis



Any kind of processing on a packet is the
responsibility of the workers
Each worker has its own queue or shared queue
depending on sniffer application architecture
Dispatcher assigns packets to worker queues
4-7
Lab 4 – Packet Filtering

Objective


Use different packet header information to sniff
specific type of packets
Mechanism



Use different sniffer application architectures to
compare the performance of these architectures
Dispatcher will sniff frames and will put in worker
queues in round-robin fashion
User will specify source IP, destination IP, source
port and destination port for filtering in TCP
packets
4-8
Lab 4 – Packet Filtering

Mechanism


Each worker will process packets residing in its
queues
Observations



Observer the throughput performance with
increasing number of threads
Compare the throughput with lab 3 throughput
Use core affinity and observe throughput
4-9
Sniffer Application Architecture





MPAC packet sniffer version 1
Single queue
Dispatcher can access whole queue
Each worker thread can access only dedicated locations
In-situ sniffing


No copying from dispatcher to worker space
Each location access is mutually exclusive


Controlled by a flag per location
No locking overhead
Get packet, if flag = 1
(workers)
Location Access Function =
Put packet, if flag = 0
4-10
(Dispatcher)
MPAC Packet Sniffer
(Version 1)
T0
T1
TN-1 T0
T1
TN-1T0
T1
TN-1T0
T1
TN-1
Dispatcher putting space
TN
Worker Threads
Dispatcher putting direction
Workers getting direction
4-11
Cross Compile for Target System

Cross Compile on Host System

Go to Cavium SDK directory and run the command

host$ source env-setup <OCTEON-MODEL>
(where <OCTEON-MODEL> is the model of your target board. E.g. OCTEON_CN56XX)




host$ cd /<path-to-mpac>/mpac_1.2
host$ ./configure --host=i386-redhat-linux-gnu
--target=mips64-octeon-linux-gnu export
CC=mips64-octeon-linux-gnu-gcc
host$ make clean
host$ make CC=mips64-octeon-linux-gnu-gcc
4-12
Run on target system

Copy executable “mpac_net_bm” and
“mpac_sniffer_app” from the directory
mpac_1.2/apps/sniffer/sniffer_1Q/ on target system



target$ ./mpac_net_bm –c <Receiver> –d <duration>
–l <# of threads>
target$ ./mpac_net_bm –c <Sender> –d <duration>
–l <# of threads> -i <IP of Receiver>
target$ ./mpac_sniffer_app –n<# of Threads> –d<duration>
-f<interface to sniff> -e 4
4-13
Sniffer Application Architecture






MPAC Sniffer Version 2
Queue size distributed between worker threads
Dispatcher can access whole queue
Each worker thread can access only dedicated sub-queue
In-situ sniffing
 No copying from dispatcher to worker space
Mutually exclusion is assured by

get and set indices (get chases set)
 Location access directions
 No locking overhead
Get packet,
if get < set
(workers)
Location Access Function =
Put packet,
Wait,
4-14
if get ≤ set
otherwise
(Dispatcher)
T0
T1
T2
TN-1
MPAC Packet Sniffer
(Version 2 & 3)
Dispatcher putting space
TN
Worker Threads
Dispatcher putting direction
Workers getting direction
4-15
Cross Compile for Target System

Cross Compile on Host System

Go to Cavium SDK directory and run the command

host$ source env-setup <OCTEON-MODEL>
(where <OCTEON-MODEL> is the model of your target board. E.g. OCTEON_CN56XX)




host$ cd /<path-to-mpac>/mpac_1.2
host$ ./configure --host=i386-redhat-linux-gnu
--target=mips64-octeon-linux-gnu export
CC=mips64-octeon-linux-gnu-gcc
host$ make clean
host$ make CC=mips64-octeon-linux-gnu-gcc
4-16
Run on target system

Copy executable “mpac_net_bm” and
“mpac_sniffer_app” from the directory
mpac_1.2/apps/sniffer/sniffer_MQ/ on target system



target$ ./mpac_net_bm –c <Receiver> –d <duration>
–l <# of threads>
target$ ./mpac_net_bm –c <Sender> –d <duration>
–l <# of threads> -i <IP of Receiver>
target$ ./mpac_sniffer_app –n<# of Threads> –d<duration>
-f<interface to sniff> -e 4
4-17
Sniffer Application Architecture



MPAC Packet Sniffer Version 3
Data structures and algorithm same as that of
version 2
Packet sniffing functions are optimized for
maximum throughput



No duplicate sniffing
Packet type (IP, ARP, etc.) identification removed
from these functions
Conditionally perform computations on available
packet

Main logic responsible for packet type checking
4-18
Cross Compile for Target System

Cross Compile on Host System

Go to Cavium SDK directory and run the command

host$ source env-setup <OCTEON-MODEL>
(where <OCTEON-MODEL> is the model of your target board. E.g. OCTEON_CN56XX)




host$ cd /<path-to-mpac>/mpac_1.2
host$ ./configure --host=i386-redhat-linux-gnu
--target=mips64-octeon-linux-gnu export
CC=mips64-octeon-linux-gnu-gcc
host$ make clean
host$ make CC=mips64-octeon-linux-gnu-gcc
4-19
Run on target system

Copy executable “mpac_net_bm” and
“mpac_sniffer_app” from the directory
mpac_1.2/apps/sniffer/sniffer_MQ_optimized/ on
target system



target$ ./mpac_net_bm –c <Receiver> –d <duration>
–l <# of threads>
target$ ./mpac_net_bm –c <Sender> –d <duration>
–l <# of threads> -i <IP of Receiver>
target$ ./mpac_sniffer_app –n<# of Threads> –d<duration>
-f<interface to sniff> -e 4
4-20
Lab 4 – Five Tuple comparison
(MPAC sniffer version 3)
$ ./mpac_sniffer_app -f eth0 -d 30 -e 4 -q 1000 -n <# of Threads>
Throughput (Mbps)
Lab #4: 1 GigE
Throughput (Mbps)
Lab #4: Loopback Interface
10000
8000
6000
4000
2000
0
1020
1000
980
960
940
920
900
1
2
3
4
5
No. of Threads
1
2
3
4
5
6
7
8
No. of Threads
$ ./mpac_sniffer_app -f lo -d 30 -e 4 -q 1000 -n <# of Threads>
4-21
6
7
8