Why do we need VXLAN and MP-BGP EVPN
VXLAN
• VLAN Limitation: VLANs use a 12-bit VLAN ID which provides a limited range of 4096 VLANs. VXLAN
addressed this limitation with 24-bit VXLAN Network Identifier (VNI) providing 16 Million unique VXLANs.
• Flexibility: VXLAN provides flexibility by deploying workloads anywhere irrespective of the physical
location and extend Layer 2 Segments over Layer 3 network.
• VM Mobility: Virtual Machines can be dynamically deployed and migrated anywhere regardless of the
physical location.
Note: VXLAN Tunnels can be created between VXLAN Tunnel Endpoints (VTEP) manually without MPBGP EVPN.VXLAN relies on flood-and-learn mechanism to learn remote MAC addresses. This is not a
scalable solution and not suitable for large scale deployments.
MP-BGP EVPN
•
•
Scalability: MP-BGP EVPN avoids flood-and-learn mechanism and uses BGP for learning and
distributing MAC addresses which is suitable for large deployments.
Dynamic Discovery: VXLAN Tunnel Endpoints (VTEPs) can be dynamically discovered.
MP-BGP EVPN is the Control Plane and VXLAN is the Data Plane.
© Fortinet Inc. All Rights Reserved.
2
VXLAN
• Virtual Extensible LAN (VXLAN) is a network virtualization technology which is used to create a
Layer 2 overlay network over a Layer 3 network.
• Local VXLAN Tunnel Endpoint (VTEP) encapsulates Layer 2 Ethernet frames within Layer 3 IP
packets using the UDP protocol and sent to the Remote VTEP for decapsulation.
• VXLAN encapsulation adds 50 B of overhead over 14 B (Ethernet) + 20 B (IPv4) + 8 B (UDP) + 8 B
(VXLAN).
• VLAN ID in 802.1Q header is mapped to VXLAN network identifier (VNID) in the VXLAN header.
© Fortinet Inc. All Rights Reserved.
3
VXLAN Support History Information in FortiOS
• Native VXLAN is supported by FortiOS since FOS-5.6 version onward.
Refer to FortiGate NP7-VxLAN Support at https://cpr.fortilab.fortinet.com/document/category/AVID
• VLAN inside VXLAN is supported since FOS 6.2.9
https://docs.fortinet.com/document/fortigate/6.2.9/cookbook/184150/vlan-inside-vxlan
• Virtual Wire Pair with VXLAN is supported since FOS 6.2.9
https://docs.fortinet.com/document/fortigate/6.2.9/cookbook/22733/virtual-wire-pair-with-vxlan
• VXLAN over IPSec Tunnel is supported since FOS 6.2.9
https://docs.fortinet.com/document/fortigate/6.2.9/cookbook/821119/vxlan-over-ipsec-tunnel
New Feature in 7.4.0
• MP-BGP EVPN as control plane for VXLAN is supported from 7.4.0
https://docs.fortinet.com/document/fortigate/7.4.0/new-features/52499/using-mp-bgp-evpn-with-vxlan
© Fortinet Inc. All Rights Reserved.
4
MP-BGP EVPN (RFC 7348)
• MP-BGP EVPN is a standards-based control plane that supports the distribution of attached host MAC/IP.
• A multicast group is maintained with all the VTEPs associated with the same VXLAN network identifier(VNI)
• The multicast packets that encapsulate ARP broadcast frames are sent to this multicast group, and then the
destination host replies to the source host using unicast IP packet encapsulated using VXLAN.
• The source and destination FortiGates as VTEPs each maintain a mapping of MAC addresses to remote
VTEPs.
© Fortinet Inc. All Rights Reserved.
5
MP-BGP EVPN (Control Plane) and VXLAN (Data Plane)
• Leafs act as VXLAN Tunneling Endpoints (VTEP) and perform encapsulation/de-encapsulation.
• MP-BGP EVPN supports Route-Type 2 (MAC/IP advertisement route) and Route-Type 3
(Inclusive Multicast Ethernet tag) to handle BUM Traffic.
Spine-1
• Control Plane: Learning MAC/IP and
Remote VTEP IP.
• Data Plane: Layer 2 Frame forwarding
from VTEP with VXLAN header
encapsulation.
Leaf-2
Leaf-1
VTEP2: 10.255.255.2
VTEP1: 10.255.255.1
VLAN 100
VLAN 200
VLAN 100
VLAN 200
© Fortinet Inc. All Rights Reserved.
6
VXLAN and MP-BGP EVPN Configuration
VXLAN and MP-BGP EVPN Configuration
• FortiGate Leaf VXLAN and EVPN Configuration
Step 1: Configure EVPN on FortiGate Leaf
Step 3: Configure the VXLAN on FortiGate Leaf
config system evpn
edit 100
set rd "100:100"
set import-rt "100:100"
set export-rt "100:100"
set ip-local-learning enable
set arp-suppression enable
next
edit 200
set rd "200:200"
set import-rt "200:200"
set export-rt "200:200"
set ip-local-learning enable
set arp-suppression enable
next
end
config system vxlan
edit "VXLAN1"
set interface "loopback1"
set vni 1000
set evpn-id 100
set learn-from-traffic enable
next
edit "VXLAN2"
set interface "loopback1"
set vni 2000
set evpn-id 200
set learn-from-traffic enable
next
End
Step 2: Configure the loopback on FortiGate Leaf
config system switch-interface
edit "switch1"
set vdom "root"
set member "port4" "VXLAN1"
set intra-switch-policy explicit
next
edit "switch2"
set vdom "root"
set member "port5" "VXLAN2"
set intra-switch-policy explicit
next
end
config system interface
edit "loopback1"
set vdom "root"
set ip 10.255.255.1 255.255.255.255
set allowaccess ping
set type loopback
set snmp-index 10
next
end
Step 4: Configure the virtual switch
© Fortinet Inc. All Rights Reserved.
8
VXLAN and MP-BGP EVPN Configuration
• FortiGate Leaf BGP Configuration
• FortiGate Spine BGP Configuration
config router bgp
set as 65010
set keepalive-timer 3
set holdtime-timer 9
set ibgp-multipath enable
set recursive-next-hop enable
config neighbor
edit "10.255.11.1"
set next-hop-self enable
set soft-reconfiguration-evpn enable
set remote-as 65010
set connect-timer 1
next
edit "10.255.21.1"
set next-hop-self enable
set soft-reconfiguration-evpn enable
set remote-as 65010
set connect-timer 1
next
end
config redistribute "connected"
set status enable
end
config router bgp
set as 65010
set keepalive-timer 3
set holdtime-timer 9
set ibgp-multipath enable
config neighbor-group
edit "evpn"
set next-hop-self enable
set soft-reconfiguration enable
set soft-reconfiguration-evpn enable
set remote-as 65010
set route-reflector-client enable
set route-reflector-client-evpn enable
next
end
config neighbor-range
edit 1
set prefix 10.255.0.0 255.255.0.0
set neighbor-group "evpn"
next
end
config redistribute "connected"
set status enable
end
Note:
FortiGate Leaf Configuration is needed when FortiGate is used as a Leaf.
FortiGate Spine Configuration is needed when FortiGate is used as a Spine.
© Fortinet Inc. All Rights Reserved.
9
VXLAN and MP-BGP EVPN Configuration
• Arista Leaf VXLAN Configuration
• Arista Leaf BGP Configuration
vlan 200
!
interface Ethernet3
switchport access vlan 200
!
interface Loopback1
ip address 10.255.255.3/32
!
interface Vlan200
ip address 10.0.200.1/24
!
interface Vxlan1
vxlan source-interface Loopback1
vxlan udp-port 4789
vxlan vlan 200 vni 2000
router bgp 65010
no bgp default ipv4-unicast
timers bgp 3 9
maximum-paths 4 ecmp 4
neighbor EVPN peer group
neighbor EVPN remote-as 65010
neighbor EVPN next-hop-self
neighbor EVPN send-community extended
neighbor 10.255.13.1 peer group EVPN
neighbor 10.255.23.1 peer group EVPN
redistribute connected
!
vlan 200
rd 200:200
route-target both 200:200
redistribute learned
!
address-family evpn
neighbor EVPN activate
!
address-family ipv4
neighbor EVPN activate
!
end
Note:
Arista Leaf Configuration is needed when Arista Switch is used as a Leaf.
© Fortinet Inc. All Rights Reserved.
10
VXLAN and MP-BGP EVPN Configuration
• Arista Spine BGP Configuration
router bgp 65010
no bgp default ipv4-unicast
timers bgp 3 9
maximum-paths 4 ecmp 4
bgp listen range 10.255.0.0/16 peer-group SPINE_EVPN remote-as 65010
neighbor SPINE_EVPN peer group
neighbor SPINE_EVPN remote-as 65010
neighbor SPINE_EVPN route-reflector-client
neighbor SPINE_EVPN send-community extended
redistribute connected
!
address-family evpn
neighbor SPINE_EVPN activate
!
address-family ipv4
neighbor SPINE_EVPN activate
neighbor SPINE_EVPN next-hop-self
Note:
Arista Spine Configuration is needed when Arista Switch is used as a Spine.
© Fortinet Inc. All Rights Reserved.
11
MP-BGP EVPN
Control Plane Verification
VXLAN/MP-BGP EVPN Topology
•
Spine-1
Spine-2
•
24
.0/
2
.2
55
2
.
10
Leaf-2
VTEP: 10.255.255.2
Leaf-1
VTEP: 10.255.255.1
VLAN 100
Host-1
10.0.100.10
VLAN 200
Host-3
10.0.200.10
•
24
0/
3.
.2
55
.2
10
0/24
.21.
5
5
2
10.
10.2
55.1
3.0/
24
24
0/
2.
.1
55
.2
10
4
/2
.0
1
.1
55
2
.
10
MP-BGP EVPN is a standards–based
control plane which supports
multivendor interoperability.
FortiOS 7.4.0 supports Route Type 2
and Type 3.
Validated Interoperability with
FortiOS 7.4.0 and Arista EOS
4.30.0F
VLAN 100
Host-2
10.0.100.20
Leaf-3
VTEP: 10.255.255.3
VLAN 200
Host-4
10.0.200.10
Note: FortiGate can be used as either a Leaf or Spine depending on the design requirements.
© Fortinet Inc. All Rights Reserved.
13
MP-BGP EVPN Verification
Leaf-1 learnt both VLANs MAC and IPs
• Leafs are Configured as VXLAN Tunnel Endpoints
(VTEPs).
• MP-BGP EVPN is used to learn Host MAC addresses
• L2 Network is formed between Host-1 and Host-2
© Fortinet Inc. All Rights Reserved.
14
MP-BGP EVPN Verification
Leaf-1 has both VLAN L2 EVPN Tables
© Fortinet Inc. All Rights Reserved.
15
MP-BGP EVPN Verification
Leaf-2 has only VLAN 100 MAC/IP and VLAN 200 L2 EVPN Tables
© Fortinet Inc. All Rights Reserved.
16
VXLAN
Data Plane Verification
VXLAN Packet Flow
•
•
MAC and IP are learnt on the local FGT and
receives Ethernet Frame from Host 1.
VXLAN header is added and encapsulated in a
UDP packet and routed to VTEP2. Spine-1
Port2
Port2
1
1
1
1
Port3
Port2
Leaf-1
Leaf-2
VTEP: 10.255.255.1
VTEP: 10.255.255.2
Port4
Port4
Host-1
IP: 10.0.100.10
MAC: 00:11:11:11:11:11
Host-2
VLAN 100
VLAN 100
IP: 10.0.100.20
MAC: 00:00:00:22:22:22
© Fortinet Inc. All Rights Reserved.
18
VXLAN Packet Flow
•
Spine routes the packet from VTEP1 to VTEP2
2
2
Spine-1
2
Port2
Port2
2
Port3
Port2
Leaf-1
Leaf-2
VTEP: 10.255.255.2
VTEP: 10.255.255.1
Port4
Port4
Host-1
IP: 10.0.100.10
MAC: 00:11:11:11:11:11
Host-2
VLAN 100
VLAN 100
IP: 10.0.100.20
MAC: 00:00:00:22:22:22
© Fortinet Inc. All Rights Reserved.
19
VXLAN Packet Flow
•
•
MAC and IP are learnt on the local FGT and
receives Ethernet Frame from Host 1.
VXLAN header is added and encapsulated in a
UDP packet and routed to VTEP2. Spine-1
Port2
3
3
3
Port3
Port2
Port2
Leaf-1
3
Leaf-2
VTEP: 10.255.255.1
VTEP: 10.255.255.2
Port4
Port4
Host-1
IP: 10.0.100.10
MAC: 00:11:11:11:11:11
Host-2
VLAN 100
VLAN 100
IP: 10.0.100.20
MAC: 00:00:00:22:22:22
© Fortinet Inc. All Rights Reserved.
20
Validate Scenarios
Scenario-1
FortiGates acting as Leafs and Spine Switch
SW Version: FortiOS 7.4.0
Spine-1
-B
MP
GP
MP
-B G
P
Leaf-1
VLAN 100
Leaf-2
VLAN 100
© Fortinet Inc. All Rights Reserved.
22
Scenario-2
FortiGates acting as Leafs and Arista Switch as Spine
SW Version: FortiOS 7.4.0 and Arista EOS 4.30.0F
Note: Validated with Arista but will work
with any vendor platform that supports
MP-BGP EVPN.
Spine-1
-B
MP
GP
MP
-B G
P
Leaf-1
VLAN 100
Leaf-2
VLAN 100
© Fortinet Inc. All Rights Reserved.
23
Scenario-3
FortiGates acting as Local Leaf, Spine Switch and Arista Switch acting as Remote Leaf.
SW Version: FortiOS 7.4.0 and Arista EOS 4.30.0F
Note: Validated with Arista but will work
with any vendor platform that supports
MP-BGP EVPN.
Spine-1
-B
MP
GP
MP
-B G
P
Leaf-1
VLAN 100
Leaf-2
VLAN 100
© Fortinet Inc. All Rights Reserved.
24
Scenario-4
FortiGates acting as Local Leaf and Arista Switch acting as Remote Leaf, Spine.
SW Version: FortiOS 7.4.0 and Arista EOS 4.30.0F
Note: Validated with Arista but will work
with any vendor platform that supports
MP-BGP EVPN.
Spine-1
-B
MP
GP
MP
-B G
P
Leaf-1
VLAN 100
Leaf-2
VLAN 100
© Fortinet Inc. All Rights Reserved.
25
Use Cases
Use Case-1: Data Center Interconnect (DCI)
Edge VTEP
DC1
Spine-1
DC2
Spine-1
Leaf-2
Leaf-1
VLAN 100
Edge VTEP
DCI
Benefits:
1. Data Center Interconnect
between Data Centers.
2. Layer 2 Extension beyond
geographic location.
VLAN 200
Leaf-2
Leaf-1
VLAN 100
VLAN 200
© Fortinet Inc. All Rights Reserved.
27
Use Case-2: Internal Segmentation (Selective)
Benefits:
1. Business Critical Applications
can be secured
2. Dynamic Service Insertion of
Firewalls by selectively moving
the critical workloads to Critical
Applications Zone.
3. Advanced Threat Protection for
majority of East-West Traffic
4. Extend the Fabric with FortiGate
Leafs and leverage existing
network architecture
5. No need to redesign the Fabric
6. Scalable and Flexible
Spine Layer
VXLAN/EVPN Fabric
Leaf Layer
Critical Applications Zone
© Fortinet Inc. All Rights Reserved.
28
Use Case-3: Internal Segmentation (Full)
Benefits:
1. Advanced Threat Protection for
complete East-West Traffic
Spine Layer
VXLAN/EVPN Fabric
Leaf Layer
Caveats:
1. Suitable for Small– Mid Data
Center Networks.
2. Design the Fabric based on the
Firewall and Threat Protection
Throughput requirement to avoid
bottleneck at the Spines.
© Fortinet Inc. All Rights Reserved.
29
MP-BGP EVPN Current Support
• Route type 2 (MAC/IP advertisement route) and route type 3 (inclusive multicast Ethernet tag route)
• Intra-subnet communication
• Single-homing use cases
• VLAN-based service, namely, there is only one broadcast domain per EVPN instance (EVI). This is
due to the current VXLAN design that supports a single VNI for a VXLAN interface.
• EVPN running on IPv4 unicast VXLAN
• Egress replication for broadcast, unknown unicast, and multicast (BUM) traffic
• VXLAN MAC learning from traffic
• IP address local learning
• ARP suppression
© Fortinet Inc. All Rights Reserved.
30