PFE FAQs
Monday, March 11, 2024
10:48 AM
Q: What is PFE(Packets Forward Engine)?
A: The PFE is a hardware-based accelerator for classification and forwarding the Ethernet traffic. It has five HIF interfaces and three EMAC interface s. The HIF are used to send frames
from/to the Host cores and the EMAC interfaces are used to send frames from/to the Ethernet network. The CLASS and UTIL run t he PFE Firmware that decides about the outbound
interface for each frame entering the PFE.
Q: What are the software components of PFE?
A: Firmware: The PFE Firmware is a software component running within the PFE. Each packet reaching the PFE (ingress/egress) is processed b y the PFE Firmware that is responsible for
packet classification, routing, modifications, and supporting functions. The PFE Firmware is under control of NXP and can be modified/extended to target new use cases and add
specific features related to Ethernet traffic processing. The PFE Firmware must be loaded into the PFE after each PFE reset. Loading the PFE Firmware is a task of the PFE master driver
of the specific operating system transparent to the user. Some PFE Drivers may require to store the PFE Firmware binary into a predefined location in the file system. All cores of the PFE
must be loaded with the appropriate PFE Firmware binaries.
A customer wanted to free the space allocated for PFE firmware, it is supported in MCAL PFE driver 1.4.0 now.
Ethernet Driver: Including the Linux PFE Ethernet driver and MCAL PFE Ethernet driver.
Linux PFE Ethernet driver: The Linux PFE Ethernet driver contains three main sub-blocks:
• The Platform Driver is an OS agnostic low-level driver that owns the hardware and firmware configuration. Hardware configuration is encapsulated by separate software m odules,
each module controls access to the underlying PFE block and provides a
higher level API for its users.
• The HIF driver is the part of the Linux driver that manages the HIF channels and the packet traffic between PFE and the Lin ux networking stack.
• The FCI endpoint is the control path driver that implements the kernel space side of the FCI for runtime configuration and monitoring of PFE. Primarily it converts user space fast path
configuration commands received via LibFCI into Platform Driver function calls.
The Linux PFE Ethernet driver runs inside the Linux Kernel space and connects PFE to the Linux networking stack. It provides access to physical Ethernet interfaces by exposing Linux
network device interfaces that are configurable via standard Linux networking tools (i.e. iproute2, net -tools, iputils).
MCAL PFE Ethernet driver: For MCAL PFE driver's functions, please refer to MCAL_ETH_43_PFE_UM chapter 3.2.
LibFCI: This is Fast Control Interface available for host applications to communicate with the networking engine. The FCI is intended to provide a generic configuration and monitoring
interface for the networking acceleration HW. Provided API shall remain the same within all HW/OS-specific implementations to keep dependent applications portable across various
systems. The LibFCI is not directly touching the HW. Instead, it only passes commands to a dedicated software component (OS/H W-specific endpoint) and receives return values. The
endpoint is then responsible for HW configuration. This approach supports a kernel -user space deployment where the user space contains only API and the logic is implemented in
kernel. For more info, please refer to FCI_API reference manual.
Q: How to build PFE driver in MCAL and Linux?
MCAL: Please refer to Integration Manual for PFE MCAL driver chapter 3 and 4.
Linux: Please refer to PFE_S32G_A53_LNX_UserManual chapter 3.
Q: Where did the PFE driver load the firmware from?
Loading firmware can only be done by the PFE master driver.
MCAL: In EB treso, there is an option related to PFE firmware loading. The default option is as below.
If customer chooses PfeClassFWArrayLabel, PFE firmware should be in the form of array. The PfeClassFwArrayLabel option should be filled with the name of the array. The whole project
would compile with the firmware and place the PFE firmware somewhere in SRAM, and then the PFE master driver would load the firmware into PFE. Customer could change the
address in linker file.
If customer chooses PfeClassFwMemoryAddress, it means customer pre-load the firmware in the form of binary and the loading address should be filled.
Linux: By adding the corresponding parameters in local.conf file, the BSP image would include the PFE firmware in the file s ystem. Customer could find the firmware in file system.
PFE FAQs Page 1
Q: What is the difference between PFE master driver and PFE slave driver?
PFE driver could run a master instance within one Host, and one or more slave instances from other Host that can have access to PFE, even from other operating systems than the Host
OS running the master instance. There is a startup synchronization requirement in the initialization of master and slave. Mas ter must be up before slave. Please refer to MCAL_ETH_43
_PFE_UM chapter 3.6.8 and PFE_S32G_A53_LNX_UserManual Chapter 2.2.5. Only the master driver could load the firmware into PFE and configure the system buffers(BMU, Routing
table) for PFE. When it comes to configure the fast path mode, PFE driver uses the FCI API to finish the task. However, the s lave driver would not directly to configure it, it would sent
the message to ask master driver to configure it.
Q: How do the master and slave communicate?
PFE slave driver provides a mechanism called "IDEX"(Inter Driver EXchange) for master and slave to communicate. Currently, th e latest PFE driver is using "IDEX 2.0", which is more
robust than the older one. Please refer to the Jira ticket AAVBR-977. In MCAL, PFE slave driver provide two parameters to ensure the robustness. More info about these two
parameters, please refer to MCAL_ETH_43_PFE_UM chapter 3.6.8. In Linux, these two parameters are configured in build_env.mak file. Please refer to PFE_S32G_A53
_LNX_UserManual Chapter 2.2.4 for more info.
Q: How to configure PFE drivers into multiples instances mode?
One classical scenario (The most common used) is to set M7 as master driver and set A53 as slave. To configure the PFE master driver in MCAL, customer should enable the Multiple PFE
drivers support and disable the Slave driver. To configure the PFE Slave driver in Linux, adding "pfe-slave" into local.conf file. Please make sure PFE master driver is up before slave.
Here are some commonly used network topologies in PFE.
Aux0 in master is used for internal communication with aux0sl in slave(Orange line). Pfe2 in master and aux0sl in slave could communicate with external network through EMAC2(Red
line). In this case, PFE2 controller should enable "Accept all traffic".
In this case, PFE2 controller should disable "Accept all traffic". PFE2 could only accept management frames(Blue line). Aux0 could communicate with external network.
PFE FAQs Page 2
If configure a network interface PFE2sl in slave, aux0sl could be only used for internal communication.
If PFE2sl is used only for management frames, please adding the following line into dts file.
In this case, both PFE2 and PFE2sl are used for handling management frames.
Q: When should customer enable/disable the parameter "Accept all traffic" in MCAL?
When customer want to use the Aux controller, they should consider to enable or disable this parameters. If customer want to use Aux controller to send frames outside of the PFE, and
if other corresponding EMAC controller(PFEx) exists, they have to disable the parameter. It is because the Aux controller is not attached with any EMAC. In this case, EMAC must be a
part of bridge to send the frames from the Aux controller. At the same time, if customer enable "Accept all traffic", the fra mes, which are sent to the Aux controller from outside, would
be sent to corresponding EMAC controller(PFEx). If customer use Aux controller only for internal communication in PFE, they s hould enable "Accept all traffic". In Linux, if customer
want to disable "Accept all traffic", they need to add ’nxp,pfeng-netif-mode-mgmt-only’ at corresponding network interface device node. When disable the "Accept all traffic", the
EMAC controller could only accept "management frame"(Egress Timestamp Reports, PTP frames).
Q: Is it possible to configure PFE master in A53 side and slave in M7?
Yes, theoretically possible. However we do not suggest customer to do that. First, customer usually enable the M7 before A53, and PFE master must be up before slave, so ideally we
put PFE master in M7 and slave in A53. If we set PFE master in A53 and slave in M7, it is complicated to handle the time sequ ences and the clock figuration. No other customer try to do
that before.
Q: What messages would the slave send to master when the slave in the process of initialization?
The slave driver would send PFE_PLATFORM_RPC_PFE_PHY_IF_CREATE through RPC to master for 7 times. When master receive this message, actually master does nothing about it.
This message will be optimized in the next PFE Linux driver release. https://jira.sw.nxp.com/browse/AAVB-9040
PFE FAQs Page 3
This message will be optimized in the next PFE Linux driver release. https://jira.sw.nxp.com/browse/AAVB-9040
Then slave would send to the messages according to the following order.
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_PHY_IF_ENABLE -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_LOG_IF_CREATE -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_LOG_IF_IS_ENABLED -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_LOG_IF_ENABLE -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_LOG_IF_IS_PROMISC -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
PFE_PLATFORM_RPC_PFE_IF_LOCK -> PFE_PLATFORM_RPC_PFE_LOG_IF_PROMISC_ENABLE -> PFE_PLATFORM_RPC_PFE_IF_UNLOCK
For more requests from the slave driver, please check the function pfe_platform_idex_rpc_cbk in pfe_platform_master.c. It contains all the requests from the slave driver.
Q: Could PFE work in U-boot and how to enable PFE driver in U-boot?
Current latest BSP version provides PFE master driver and slave driver in U-boot. However, both could not configure FCI in u-boot, which means that PFE master driver in U-boot could
only work in slow path mode. Please refer to BSP40 user manual chapter 7.3.1 for more info.
Q: How U-boot load the PFE firmware?
U-boot would load PFE firmware from file system or Qspi flash depending on the boot option. If customer want to run U -boot in emmc and load the PFE firmware. Please refer the
following tickets.
https://community.nxp.com/t5/S32G-Internal-Community/S32G2-PFE-FW-not-being-able-to-load-from-eMMC/td-p/1856940
https://community.nxp.com/t5/AP-Software-Support/S32G3-Mobileye-add-PFE-firmware-in-eMMC-on-customer-board/m-p/1705452/thread-id/754
Q: How to set the buffers' size and where to set their address?
No matter MCAL or Linux, BMU and Routing table are only configured by PFE master driver.
MCAL: BMU2's size is configured by parameter VarEthBmu2BufCnt. Size[BMU2] = 2048 * VarEthBmu2BufCnt.
Routing table is separate into two parts. Size[Routing table] = 128 * ("Routing Table Hash Size" + "Routing Table Collision S ize")
As for Buffer Descriptors and Tx/Rx buffers, please refer to MCAL_ETH_43PFE_IM(Integration manual) chapter 4.4.
Those buffers' address could be configured in linker file.
Linux: PFE master driver could only configure the BMU2 buffers, Routing table, RX/TX buffer descriptors. RX/TX buffers is con figured by the kernel in the run time. PFE slave driver could
only configure the RX/TX buffer descriptors. Customer could set these buffers' address and size in device tree file. Please refer to PFE_S32G_A53_LNX_UserManual chapter 2.9. Please
make sure that BMU2 and RT address is lower than 0xC000_0000. BMU should be aligned correctly.
https://community.nxp.com/t5/S32G-Internal-Community/s32G3-Valeo-PFE-IDEX-messages-for-slave-driver-implementation/td-p/1833068
https://community.nxp.com/t5/S32G-Internal-Community/PFE-Tresos-buffer-fifo-configuration-guidelines/td-p/1819959
https://community.nxp.com/t5/S32G-Internal-Community/Valeo-S32G3-UBOOT-PFE-driver-as-Master-stuck-when-sending-large/m-p/1876148
PFE FAQs Page 4
https://community.nxp.com/t5/S32G-Internal-Community/Valeo-S32G3-UBOOT-PFE-driver-as-Master-stuck-when-sending-large/m-p/1876148
Q: System buffer(BMU) could be placed in SRAM and DDR. Do we have benchmark for both?
Actually, placing it in SRAM typically improves fast-path performance compared to DDR.
https://jira.sw.nxp.com/browse/AAVB-4509
Q: What is the difference between HIF and HIFNOCY?
The HIF performs full copy of data from PFE to the host memory or vice-versa, whereas the HIF_NOCPY based packet transfer is just transfer of packet pointers. The host software
prepares the BDs for HIF_NOCPY similar to as is done for HIF. These BDs have the flexibility to be placed in system memory re gion but the packet buffers are restricted to PFE memory.
Transmit flow of HIF:
1) Host prepare frame data and fill Tx BD
2) T1 – HIF get a valid Tx BD
3) T2 – HIF copy frame pointed by BD to BMU buffer for transmitting
4) HIF Write back empty flag to Tx BD
Receive flow:
1) Host prepare empty buffer and file Rx BD
2) R1 – HIF get a valid Rx BD
3) R2 – HIF copy frame to Host buffer pointed by BD when a received frame is ready
4) HIF Write back ready flag to Rx BD and interrupt host
Transmit flow of HIF NOCPY:
1) Host allocate buffer from BMU, prepare frame data and fill TxBD
2) T1 – HIF get a valid Tx BD, directly reference the frame pointed to by Tx BD
3) HIF Write back empty flag to Tx BD
Receive flow:
1) Host prepare Rx BD with null pointer
2) R1 – HIF get a valid Rx BD
3) R2 – HIF write pointer value to Rx BD when a received frame is ready
4) HIF Write back ready flag to Rx BD and interrupt host
5) Host directly reference the frame pointed by BD
Q: How to use PFE as L2 Bridge mode(Switch)?
L2 Bridge is a dedicated feature to offload a host from tasks related to MAC address-based forwarding of Ethernet frames. Network interfaces could be assigned to a bridge domain
using VLAN and traffic could be routed using MAC address. MAC addresses can be learned or statically assigned.
An example is shared in the link below:
https://confluence.sw.nxp.com/pages/viewpage.action?pageId=149785358
Another example is shown as below:
PFE FAQs Page 5
Another example is shown as below :
In this case, the network flow could be:
aux0 <->aux0sl, aux0<->emac0, aux0<->emac1,aux0sl<->emac1, aux0sl<->emac2, emac1<->emac2
If the network interfaces are the standard interfaces, then they could not communicate internally like aux0 interface.
Create a new VLAN bridge domain. If using the default VLAN (id = 1), this step could be skipped.
libfci_cli bd-add –vlan x
Configure the unicast/multicast hit/miss actions for the bridge domain.
libfci_cli bd-update --vlan 1 --ucast-hit FORWARD --ucast-miss FLOOD --mcast-hit FORWARD --mcast-miss FLOOD
Add the EMAC and HIF as members of the VLAN bridge domain with VLAN tagging disabled.
libfci_cli bd-insif --vlan 1 --i emac2 --tag OFF
libfci_cli bd-insif --vlan 1 --i emac0 --tag OFF
libfci_cli bd-insif --vlan 1 --i hif2 --tag OFF
libfci_cli bd-insif --vlan 1 --i hif0 --tag OFF
Configure the physical interfaces according to : "VLAN_BRIDGE" mode, Enable traffic promiscuity, blocking state mode “NORMAL” (MAC address learning is enabled).
libfci_cli phyif-update --i emac2 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
libfci_cli phyif-update --i emac0 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
libfci_cli phyif-update --i hif2 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
libfci_cli phyif-update --i hif0 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
Or customer could also statically assign the mac address.
libfci_cli bd-update --vlan 1 --ucast-hit FORWARD --ucast-miss FLOOD --mcast-hit FORWARD --mcast-miss FLOOD
libfci_cli bd-insif --vlan 1 --i emac2 --tag OFF
libfci_cli bd-insif --vlan 1 --i emac0 --tag OFF
libfci_cli bd-insif --vlan 1 --i hif2 --tag OFF
libfci_cli bd-insif --vlan 1 --i hif0 --tag OFF
libfci_cli bd-stent-add --vlan 1 --mac 00:01:BE:BE:EF:80
libfci_cli bd-stent-update --vlan 1 --mac 00:01:BE:BE:EF:80 --egress hif0
libfci_cli bd-stent-add --vlan 1 --mac 00:01:BE:BE:EF:81
libfci_cli bd-stent-update --vlan 1 --mac 00:01:BE:BE:EF:81 --egress hif2
libfci_cli bd-stent-add --vlan 1 --mac AC:91:A1:01:E5:E5
libfci_cli bd-stent-update --vlan 1 --mac AC:91:A1:01:E5:E5 --egress emac2
libfci_cli bd-stent-add --vlan 1 --mac AC:91:A1:01:FF:FF
libfci_cli bd-stent-update --vlan 1 --mac AC:91:A1:01:FF:FF --egress emac0
libfci_cli phyif-update --i emac2 -E --promisc ON --mode VLAN_BRIDGE --bs FW_ONLY
libfci_cli phyif-update --i emac0 -E --promisc ON --mode VLAN_BRIDGE --bs FW_ONLY
libfci_cli phyif-update --i hif2 -E --promisc ON --mode VLAN_BRIDGE --bs FW_ONLY
libfci_cli phyif-update --i hif0 -E --promisc ON --mode VLAN_BRIDGE --bs FW_ONLY
Q: How to use PFE as L3 Router
Host application can change operation mode of a physical interface to IP Router and specify routing rules into routing table using FCI API.
An example is shared in the link below:
https://confluence.sw.nxp.com/display/AutoETH/L3+Router
Another example is as below:
Fast-path routing between EMAC1 and EMAC2 and slow-path routing within host CPU
Once configured, the routing points
and
forward traffic received via EMAC1 or EMAC2, which matches a routing table entry, to desired destination interface(s) (EMAC2 or EMAC1)
while the rest of traffic is sent to default interface (CPU0) where the host software can perform custom, slow -path routing.
Create new routes for physical interface.
libfci_cli route-add --route 0 --ip4 --dst-mac 34-48-ED-45-EB-1D --i emac1
libfci_cli route-add --route 1 --ip4 --dst-mac 28-F1-0E-03-92-F8 --i emac2
Create new simple contrack.
libfci_cli cntk-add --proto ICMP --sip 192.1.0.10 --dip 192.2.0.20 --sport 0 --dport 0 --route 1 --no-reply
libfci_cli cntk-add --proto ICMP --sip 192.2.0.20 --dip 192.1.0.10 --sport 0 --dport 0 --route 0 --no-reply
PFE FAQs Page 6
libfci_cli cntk-add --proto ICMP --sip 192.2.0.20 --dip 192.1.0.10 --sport 0 --dport 0 --route 0 --no-reply
Configure the physical interfaces to : “ROUTER" mode, disable traffic promiscuity
libfci_cli phyif-update -i emac1 --mode=ROUTER -E --promisc OFF
libfci_cli phyif-update -i emac2 --mode=ROUTER -E --promisc OFF
Print route and contrack
Q: How to use PFE as Flexible Router?
Flexible Router is a PFE firmware-based feature which uses logical interfaces (and their match rules) to classify ingress traffic. Replicas of the accepted tra ffic can be forwarded to one or
multiple physical interfaces. Flexible Router classification introduces a performance penalty which is proportional to a coun t of used logical interfaces (and their match rules). Always
consider whether the use of this feature is really necessary. If it is necessary, then try to use as few logical interfaces a s possible.
An example is shared in the link below:
https://confluence.sw.nxp.com/display/AutoETH/Flexible+Router
if both master and slave(standard network interfaces) use the same EMACs, customer should configure the Flexible Router as be low:
Take as PFE2 and PFE2sl as example(make sure their MAC address are different):
libfci_cli phyif-update -i emac2 --mode FLEXIBLE_ROUTER -P ON
libfci_cli logif-update --i=pfe2 --match-rules=DMAC --dmac=00:04:9F:BE:EF:02 --egress=hif0 --promisc=OFF -E
libfci_cli logif-update --i=s9.pfe2sl --match-rules=DMAC --dmac=00:04:9F:BE:FF:33 --egress=hif1 --promisc=OFF -E
But if customer want to use Aux interface, configure a bridge as below:
Another Flexible Router example is shared in below link:
https://confluence.sw.nxp.com/display/AutoETH/Flexible+Router
Q: How to use PFE as Flexible Parser?
Flexible Parser is a PFE firmware-based feature which can classify ingress traffic according to a set of custom classification rules. The feature is intended t o be used as an extension of
other PFE features/classification algorithms. Flexible Parser classification introduces a performance penalty which is propor tional to a count of rules and complexity of a used table.
Always consider whether the use of this feature is really necessary. If it is necessary, then try to use FP tables with as fe w rules as possible.
An example is as below:
configure a Flexible Filter on PFE's emac2 physical interface, allowing a Ping from a certain IP address to pass for further classification. Non-compliant traffic is discarded.
Normally, rules of the table are evaluated sequentially till the traffic is either accepted, rejected, or the end of the table is reached. If the end of
the table is reached and the traffic is still not accepted nor rejected, then Flexible Parser automatically rejects it.(Actually, customer should set the filter more specific. )
libfci_cli fprule-add --rl rule1 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 28 --layer L2 --accept
libfci_cli fprule-add --rl rule2 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 12 --layer L3 --accept
Create flexible parser table
libfci_cli fptable-add --tb table1
Insert rule into the first FP table entry
libfci_cli fptable-insrule --tb table1 --rl rule1 --pos 0
libfci_cli fptable-insrule --tb table1 --rl rule2 --pos 1
Attach the FP table to emac2
libfci_cli phyif-update -i emac2 --ff table1
PFE FAQs Page 7
Q: How to mirror the ingress or egress traffic in PFE?
An example is as below:
The scenario is the emac0 <----> emac1 works as a Vlan-bridge, and all the frames received from emac0 and emac1 would mirror to emac2.
Build a Vlan-bridge between emac0 and emac1
libfci_cli bd-update --vlan 1 --ucast-hit FORWARD --ucast-miss FLOOD --mcast-hit FORWARD --mcast-miss FLOOD
libfci_cli bd-insif --vlan 1 --i emac0 --tag OFF
libfci_cli bd-insif --vlan 1 --i emac1 --tag OFF
libfci_cli phyif-update --i emac1 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
libfci_cli phyif-update --i emac0 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
Create a new mirroring rule
libfci_cli mirror-add --mirror="MirroringRule0" -i emac2
Assign a mirroring rule to a physical interface
libfci_cli phyif-update -i emac1 --rxmirr0="MirroringRule0"
libfci_cli phyif-update -i emac0 --rxmirr0="MirroringRule0"
Set emac2 as default mode and enable it
libfci_cli phyif-update -i emac2 --mode=DEFAULT -E --bs NORMAL
One fact is that, if the frame is filtered by the flexible filter in the ingress of emac, the frame could still be mirrored i f no action is set in mirror rules.
Try following example:
Allowing ping from IP 10.3.0.3 through emac1
libfci_cli fprule-add --rl rule1 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 28 --layer L2 --accept
libfci_cli fprule-add --rl rule2 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 12 --layer L3 --accept
libfci_cli fptable-add --tb table1
libfci_cli fptable-insrule --tb table1 --rl rule1 --pos 0
libfci_cli fptable-insrule --tb table1 --rl rule2 --pos 1
libfci_cli phyif-update -i emac1 --ff table1
At this scenario, only IP 10.3.0.3 could ping through emac1.
Adding mirror rules
libfci_cli mirror-add --mirror="MirroringRule0" -i emac2
libfci_cli phyif-update -i emac1 --rxmirr0="MirroringRule0" --txmirr0="MirroringRule0"
libfci_cli phyif-update -i emac2 --mode=DEFAULT -E --bs NORMAL
Using 10.3.0.2 ping and wireshark to catch the frames from emac2.
The ARP frames with IP 10.3.0.2 could still be mirrored to emac2.
Note: Frames could only be mirrored to specific physical interface rather than logical interface.
https://community.nxp.com/t5/S32G-Internal-Community/Hirain-s-PFE-questions-about-mirror-feature/td-p/1881551
Q: If customer do not want to mirror the frames which already are filtered in emac, what should they do?
We have a flexible filter option in mirror rule. Customer could select the same flexible filter table as emac uses. Then the frames would not be mirrored.
Example as below:
libfci_cli fprule-add --rl rule1 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 28 --layer L2 --accept
libfci_cli fprule-add --rl rule2 --data 0x0A030003 --mask 0xFFFFFFFF --ofs 12 --layer L3 --accept
libfci_cli fptable-add --tb table1
libfci_cli fptable-insrule --tb table1 --rl rule1 --pos 0
libfci_cli fptable-insrule --tb table1 --rl rule2 --pos 1
libfci_cli mirror-add --mirror="MirroringRule0" -i emac2 -flexible-filter=table1
libfci_cli phyif-update -i emac1 --rxmirr0="MirroringRule0" --txmirr0="MirroringRule0"
libfci_cli phyif-update -i emac2 --mode=DEFAULT -E --bs NORMAL
libfci_cli phyif-update -i emac1 --ff table1
Note: Do not add flexible filter table to emac before adding the table to mirror rule. In a word, add the table to mirror rul e first, and then add it to emac.
To sum up:
The frames could be mirrored based on flexible filter rules(that can be anything eg vlan). We could also modify the mirror fr ames, but, up to now, we could only add the vlan header.
PFE FAQs Page 8
Q: How to use egress QoS of PFE?
PFE egress QoS (per-interface) is located in TMU(Traffic Management Unit) and contains three parts:
1. Queue (8x)
2. Scheduler (2x)
3. Sharpers (4x)
QoS(Quality of Service) refers that a network uses various technologies to provide better services for network communication. QoS can not generate new bandwidth, but effectively
manage network bandwidth according to application requirements and network management settings. The egress QoS is loacted within the egress stage of the PFE packet processing
pipeline. More generally, egress QoS of PFE is about how PFE send its packets.
Queue (8x)
Queue is an input FIFO of the egress QoS block used to hold packets to be transmitted. For each emac port, there are 8 queues shared with 255 slots in total. For HIF port, there are 2
queues shared with 32 slots. Two ways for queues to handle congestion: WRED and TAIL -DROP. 8 congestion zones available to set drop probability for WRED. Tail drop occurs when
queue fill level exceeds number of queue entries.
Scheduler (2x)
Scheduler is a component of the Egress QoS which is responsible for traffic aggregation from multiple queues. Its main task i s to select the packet from a queue which will be
transmitted next. Scheduler 1 is connected to the emac port. Each scheduler has up to 8 inputs. Scheduler 1's input can be fr om any of the 8 queues and the output of scheduler 0. Input
7 has highest priority by default. The schedulers implement the following scheduling algorithms:
• Simple Round Robin (RR)
• Weighted Round Robin (WRR)
• Deficit Weighted Round Robin (DWRR)
Every queue is configured with a specific percentage of link speed as the weight. Quota and deficit parameters are used to gu arantee committed bandwidth and fair chance for each
queue.
• Priority Queuing (PQ)
Scheduler has two mode: Packet-rate and Data-rate.
NOTE: Modification of emac Egress QoS Scheduler is permitted only if the given emac interface was not yet enabled. After the give n emac interface gets enabled for the first time, do
not attempt any
further modifications of its Egress QoS scheduler till the PFE gets reset.
Sharpers (4x)
Its purpose is to space out the frames as much as possible, reduce bursting in case of data peaks, and control maximum egress data rate (in data rate or packet rate units). In general,
shaper operates with configurable clock ticks. With every tick the internal credit value is updated according to configured p arameters. When the input queue has data and the credit
value is positive, the queue is serviced, and the credit value is decreased. When credit is negative, the input queue will no t be serviced. The sharpers can be positioned on the output of
any queue or scheduler. Customer can configure position and particular parameters of shaper according application needs.
Here is the example of setting up egress QoS in Libfci_demo:
Setting up the queues. When assigning the max threshold of the queues, it is actually assigning given number of slots.
libfci_cli qos-que-update --interface=emac0 --que=2 --que-mode=DISABLED
PFE FAQs Page 9
libfci_cli qos-que-update --interface=emac0 --que=2 --que-mode=DISABLED
libfci_cli qos-que-update --interface=emac0 --que=3 --que-mode=DISABLED
libfci_cli qos-que-update --interface=emac0 --que=4 --que-mode=DISABLED
libfci_cli qos-que-update --interface=emac0 --que=5 --que-mode=DISABLED
libfci_cli qos-que-update --interface=emac0 --que=0 --que-mode=WRED --thld-min=25 --thld-max=100 --zprob=10,20,30,40,50,60,70,80
libfci_cli qos-que-update --interface=emac0 --que=1 --que-mode=TAIL_DROP --thld-max=50
libfci_cli qos-que-update --interface=emac0 --que=6 --que-mode=WRED --thld-min=10 --thld-max=50 --zprob=20,20,40,40,60,60,80,80
libfci_cli qos-que-update --interface=emac0 --que=7 --que-mode=TAIL_DROP --thld-max=50
Setting up the schedulers.
libfci_cli qos-sch-update --interface=emac0 --sch=0 --sch-mode=PACKET_RATE --sch-algo=WRR --sch-in=que0:10000,que1:20000,D,D,D,D,D,D
libfci_cli qos-sch-update --interface=emac0 --sch=1 --sch-mode=DATA_RATE --sch-algo=PQ --sch-in=sch0_out:0,D,D,D,D,D,que6:0,que7:0
Setting up the sharpers.
libfci_cli qos-shp-update --interface=emac0 --shp=0 --shp-mode=PACKET_RATE --shp-pos=sch1_in0 --isl=1000 --crmin=-5000 --crmax=10000
libfci_cli qos-shp-update --interface=emac0 --shp=1 --shp-mode=PACKET_RATE --shp-pos=sch1_in6 --isl=2000 --crmin=-4000 --crmax=8000
libfci_cli qos-shp-update --interface=emac0 --shp=2 --shp-mode=DATA_RATE --shp-pos=sch1_out --isl=30000 --crmin=-60000 --crmax=9000
Q: How to use ingress QoS of PFE?
Ingress QoS (Quality of Service) is a configurable PFE hardware feature that allows congestion avoidance and congestion contr ol at the physical ingress ports, before the surplus traffic
can clog system resources. The Ingress QoS is embedded into the EMAC-GPI block.
The Ingress QoS contains two blocks. The blocks are ING_CLASS and ING_QOS and the ingress QoS implements the following traffi c policing mechanisms, in this order:
• Classification of ingress traffic by the following priority classes (increasing from left to right): Unmanaged, Managed and Reserved;
• Weighted Random Early Drop (WRED);
• Ingress port level rate shaping.
Classification:
The Ingress Classifier classifies the frame to Reserved/Managed/Unmanaged frames. The incoming frame is parsed and based on t he parse details such as L2/L3/L4 header fields, the
entry table is searched. If search is hit, frame is classified as Managed or Reserved based on action data in entry table or Drop the packet. If search is not found, frame is classified as
Unmanaged Frame. The maximum number of flows in flow classification table is 64.
Ingress QoS:
The Ingress QoS block is the Admittance block and it has the following functions(The result of both the functions determines whether to drop the packet):
•Weighted Random Early Drop
After the traffic classification stage, the WRED policy defines four probability zones for dropping "Unmanaged" traffic at in creasing rates based on the queue fill level (between the
"min" and "Max" levels), and one zone for dropping all traffic except the flows classified as Reserved (between the "Max" and "FULL" levels). WRED queue could be located at three
different places (DMEM, LMEM, RXF).
Maximum queue depth: 8192 for DMEM ; 512 for LMEM and RXF. The WRED block will employ large pseudo random number (32 -bit) to determine the random number used for
probability calculations. The four thresholds will translate to 5 Levels of traffic acceptance. The levels determine the prob ability and the choice of packets to drop if needed.
•Shaper
Frames passing the WRED are routed to the last Ingress QoS component: The port-based rate shaper. This shaper is implemented for the complete port traffic. It implements a standard
token bucket algorithm to count the number of bytes to be admitted. The user can configure its parameters and leverage maximu m data rate (64 kbps to 2.4 Gbps) to be received by the
particular ingress interface.
The Ingress QoS also provides statistics to inform the user about the number of drops on the particular stages (/sys/kernel/d ebug/pfeng/egpi)
1. Total drop Count
2. Classifier Drop
3. LMEM queue threshold cross Drop
4. DMEM queue threshold cross Drop
5. RXF Queue threshold cross Drop
6. Shaper0 Drop
7. Shaper1 Drop
8. Managed Packet Count
PFE FAQs Page 10
8. Managed Packet Count
9. Reserved Packet Count
10. Un-Managed Count
Scenario description
Let there be two computers (PCs), both in the same network subnet.Both PCs are connected through PFE. PFE acts as a simple br idge.
PC0(192.168.2.10) <----->(EMAC2)RDB3(EMAC1)<----->PC1(192.168.2.100)
Configure emac2 port for PFE ingress QoS.
prerequisites
setup PFE to as VLAN L2 bridge to classify traffic
libfci_cli bd-update --vlan 1 --ucast-hit FORWARD --ucast-miss FLOOD --mcast-hit FORWARD --mcast-miss FLOOD
libfci_cli bd-insif --vlan 1 --i emac2 --tag OFF
libfci_cli bd-insif --vlan 1 --i emac1 --tag OFF
libfci_cli phyif-update --i emac2 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
libfci_cli phyif-update --i emac1 -E --promisc ON --mode VLAN_BRIDGE --bs NORMAL
1. Enable the Ingress QoS module:
libfci_cli qos-pol-set -i emac2 -E
2. Configure the flowtype for different purposes:
a. Ingressing ARP traffic shall be classified as Managed.
b. Ingressing IPv4 TCP traffic from PC0 IP shall be classified as Reserved.
c. Ingressing IPv4 UDP traffic (from any source) shall be dropped.
libfci_cli qos-pol-flow-add -i emac2 --flowact MANAGED --ft TYPE_ARP
libfci_cli qos-pol-flow-add -i emac2 --flowact RESERVED -s 192.168.2.10 --s-pfx 32 -p 6 --p-mask 0xff --ft TYPE_IP4,SIP,PROTOCOL
libfci_cli qos-pol-flow-add -i emac2 --flowact DROP -p 17 --p-mask 0xff --ft PROTOCOL
3. Configure WRED policy for EMAC2's DMEM buffer pool:
libfci_cli qos-pol-wred-update -i emac2 --wred-que=DMEM -E --thmin 0 --thmax 512 --thfull 1024 --zprob=20,30,40,50
4. configure Ingress QoS shapers for emac2
libfci_cli qos-pol-shp-update -i emac2 -E --shp=0 --shp-mode=PACKET_RATE --shp-type=PORT --isl=1000 --crmin=-5000 --crmax=10000
5. check the configuration of the sharper
libfci_cli qos-pol-shp-print -i emac2
6. check the WRED configuration
libfci_cli qos-pol-wred-print -i emac2
7. check the classification table
libfci_cli qos-pol-flow-print -i emac2
Note: Please refer to PFE_S32G_A53_LNX user manual for QoS known limitation.
Q: How to use gPTP in PFE?
Q: Does PFE-slave support mdio config in Linux?
PFE-slave does not support mdio cofig directly. However, slave could send RPC message to request master to do so. Please follow t he below tickets.
https://community.nxp.com/t5/S32G-Internal-Community/Sample-code-for-PFE-master-slave-sync-and-slave-mdio-config/m-p/1843198
https://jira.sw.nxp.com/browse/AAVB-6295?jql=text%20~%20%22PFE-slave%20mdio%22
https://jira.sw.nxp.com/browse/AAVB-4505
Q: How to configure XRDC when using PFE?
Q: How to
Q: What to do with PFE when A53 performs graceful reset and ungraceful reset
Q: Serdes related problem.
PFE FAQs Page 11