#CiscoLive
EVPN: Migration from Legacy
to Modern Architecture
Best Practice and Configuration Examples
Alexey Babaytsev – Technical Marketing Engineer
BRKMPL-2143
#CiscoLive
Cisco Webex App
Questions?
Use Cisco Webex App to chat
with the speaker after the session
How
1
Find this session in the Cisco Live Mobile App
2
Click “Join the Discussion”
3
Install the Webex App or go directly to the Webex space
4
Enter messages/questions in the Webex space
Enter your personal notes here
Webex spaces will be moderated
by the speaker until June 9, 2023.
https://ciscolive.ciscoevents.com/ciscolivebot/#BRKMPL-2143
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
3
• Introduction
• EVPN Migration journey
• MHD Solutions: MC-LAG, ICCP-SM,
nV Cluster
• VPWS Migration
• EVPN Headend
Agenda
• MHN and EVPN:
• EVPN Single Flow Active Mode for L2
Rings aggregation
• VRRP/HSRP Migration
• Multicast
• Conclusion
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
4
Network Simplification Journey
Network Element
xVPN Services
Legacy Networks
LDP
BGP
IP Network Scaling
BGP-LU
TE, FRR
RSVP-TE
MPLS Overlay Protocol
RSVP-TE
IPv6 Transport Overlay
IP to DWDM Transition
Private Line Services
Converged SDN Transport
EVPN for L2VPN and L3VPN
LDP
None
Transponder or Muxponder
Grey Router Interface
Dedicated OTN
Dedicated Ethernet
over DWDM
#CiscoLive
BRKMPL-2143
DCO transceivers in Cisco
routers
Private Line Emulation over
Converged SDN Transport
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
5
Legacy to EVPN Migration – Key Asks
• Hitless migration or Service disruption?
• Which services? L2/L3: VPLS, VPWS, VRRP/HSRP, Multicast…
• Redundancy: Single-homing or Multi-homing?
• Access topology: Multi-Homed Device or Multi-Homed Network?
• Which technologies can be migrated? - MC-LAG, nV-Cluster,
ICCP-SM, MSTP, REP, G.8032
• Is Brownfield deployment possible? Can EVPN and non-EVPN
nodes co-exist?
• Etc…
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
6
Migration of MultiHomed Device (MHD)
Multi-Home Device (MHD) – Legacy Design
Options
Non-EVPN (legacy) design options:
• MC-LAG
• ICCP-SM (ICCP-based Service Multihoming or Pseudo MC-LAG)
• nV Edge (Cluster)
MPLS
Core
PE1
ICCP
A
VLAN_10
VLAN_20
CE
MC-LAG
MPLS
Core
MPLS
Core
PE2
PE1
S
A S
ICCP
VLAN_10 (active) CE
VLAN_20 (standby)
C-PE
PE2
A S
VLAN_10 (standby)
VLAN_20 (active)
ICCP-SM
(Pseudo MC-LAG)
#CiscoLive
ICL
PE1
A
VLAN_10 (active)
VLAN_20 (active)
PE2
A
CE
VLAN_10 (active)
VLAN_20 (active)
nV Cluster
C-PE: Cluster PE
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
8
EVPN Design Options for Multi-Homed Device
All-Active
(per flow)
Single-Active
(per VLAN)
Port-Active
(per port)
PE1
PE1
PE1
PE2
V2 V1
PE2
V2 V1
PE2
V1
V1
CE1
Single LAG at CE
VLAN goes to both PE
Traffic hashed per flow
NDF blocks incoming BUM
Benefits: Bandwidth, Convergence
VLAN 1 (active)
VLAN 2 (standby)CE2
VLAN 1 (standby)
VLAN 2 (active)
Multiple LAGs at CE
VLAN active on single PE
Traffic hashed per VLAN
NDF blocks all traffic in both
directions per VLAN
Benefits: Billing, Policing
#CiscoLive
BRKMPL-2143
V1, V2
CE3
Single LAG at CE
Port active on single PE
Traffic hashed per port
Port on NDF is blocked
Benefits: Protocol
Simplification, QoS
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
9
EVPN All-Active Interworking with VPLS Problem
S:MAC_1
S:MAC_1
MAC
flipflopping
S:MAC_1
#CiscoLive
• New EVPN-based sites still
need to interwork with legacy
VPLS-based nodes in
brownfield deployment
• EVPN A/A along with regular
VPLS VFI causes a MAC flipflopping issue
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
10
EVPN A/A Interworking with VPLS: vES AccessVFI (ASR 9000)
• vES: Extended the concept of Ethernet Segment (ES)
to virtual circuits
• vES inherits most characters of Single-Active Ethernet
Segment:
• DF election, Software MAC learning, Per-service load
balancing, Optional manual service carving, Local MAC flush
upon DF becomes NDF
• vES Access-VFI: Used for fully meshed topology
(VPLS)
• Split-horizon: traffic received from one member PW will not be
forwarded to other member PWs
• One ESI per access-vfi, regardless of the number of member
PWs
• DF switchover only if all PWs are down
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
11
EVPN P/A, S/A Interworking with VPLS as an
Alternative Solution
• Not all Cisco SP routers support vES Access-
VFI
• Port-Active or Single-Active modes can be
used as an alternative solution
• After DF/NDF elected only DF forward traffic,
NDF block traffic in both core-to-access and
access-to-core directions
• In Port-Active mode DF/NDF is elected per
port, while in Single/Active mode DF/NDF is
elected per EVPN Service Instance (EVI)
*Port-Active mode depicted
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
12
MC-LAG to EVPN
Migration
MC-LAG Migration: Overview
• Migration from MC-LAG to EVPN Port-Active mode:
• No reconfiguration on CE
evpn
...
!
interface Bundle-Ether12
ethernet-segment
load-balancing-mode port-active
!
• Keep active-backup design – only ports on Active PE forward traffic, Standby PE sets the LACP state to
Out-of-Service (OOS) instead of bringing the interface state down, this enables better convergence time
• QOS is accurate as traffic is forwarded across one link.
• Support of both L2 and L3 services
• Greenfield and Brownfield deployments: Interoperability with EVPN and VPLS neighbors
• P/A mode is supported in 64-bit XR 7.1.2 release
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
14
MC-LAG Migration: Initial state: PE1 and PE2
work in MC-LAG mode
PE1 & PE2 work in MC-LAG mode
• CE1-facing port is active on PE2 and
standby on PE1
•
PE1
VFI_113
Te0/0/0/46
PE1_PE2 UP
PE1_PE3 UP
PE1_PE4 UP
MPLS
VFI
BD113
PE3
Port 1
BE 34
CE1
ICCP
Standby
Port 2
PE2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE1 UP
PE2_PE3 UP
PE2_PE4 UP
VFI
PE4
#CiscoLive
RP/0/RP0/CPU0:PE1#sho bundle bundle-ether 34
Mon May 23 00:33:19.854 UTC
Bundle-Ether34
Status:
mLACP hot standby
Local links <active/standby/configured>: 0 / 1 / 1
Local bandwidth <effective/available>: 0 (0) kbps
MAC address (source):
70e4.2240.d72c (Peer)
<...>
mLACP:
Operational
ICCP Group:
113
Role:
Standby
<...>
Port
Device
State
Port ID
B/W, kbps
-------------------- --------------- ----------- -------------- --------Te0/0/0/46
Local
Standby
0x0001, 0xc001 10000000.
mLACP peer is active
Te0/5/0/1
10.0.0.2
Active
0x0001, 0xb001 10000000
Link is Active
RP/0/RP0/CPU0:9902#
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
15
MC-LAG Migration: Add EVPN configuration on
both PE1 and PE2 (1)
evpn
evi 113
•
advertise-mac
!
VFI_113
!
PE1_PE2 UP
interface Bundle-Ether34
PE1_PE3 UP
VFI
ethernet-segment
PE1_PE4 UP
identifier type 0 34.34.34.34.34.34.34.34.01
Te0/0/0/46
BD113
load-balancing-mode port-active
!
Standby
EVI_113
•
!
l2vpn
Port 1
bridge group VPLS
BE 34
bridge-domain 113
interface Bundle-Ether34.113
CE1
!
vfi 113
Port 2
VFI_113
neighbor 10.0.0.1 pw-id 113
VFI
!
PE2_PE1 UP
RP/0/RP0/CPU0:PE2#sho bundle bundle-ether 34
Active
neighbor 10.0.0.3 pw-id 113
PE2_PE3 UP
Bundle-Ether34
!
PE2_PE4 UP
BD113
Status:
Up
Te0/5/0/1
neighbor 10.0.0.4 pw-id 113
Local links <active/standby/configured>: 1 / 0 / 1
!
mLACP:
Operational
evi 113
EVI_113
ICCP Group:
113
!
<...>
Port
Device
State
Port ID
B/W, kbps
-------------------- ---------- ----------- -------------Te0/5/0/1
Local
Active
0x0001, 0xc001 10000000
Link is Active
Te0/0/0/46
10.0.0.2
Standby 0x0001, 0xb001 10000000
Link is marked as Standby by mLACP peer
* MLACP synchronizes a state of links (active
or standby) with the EVPN DF selection result
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
#CiscoLive
PE1
MPLS
ICCP
PE3
PE2
PE1&PE2: Add EVPN
configuration with port-active
load-balancing mode + EVI to
L2VPN section
Traffic can switch over to a
New Active PE if the selected
DF node differs from the node
chosen previously by MLACP*
PE4
16
MC-LAG Migration: Add EVPN configuration on
both PE1 and PE2 (2)
• After PE1/PE2 discover each other
PE1
VFI_113
Te0/0/0/46
PE1_PE2 DOWN
PE1_PE3 UP
PE1_PE4 UP
BD113
through EVPN routes:
MPLS
VFI
PE3
•
EVI_113
Port 1
BE 34
CE1
ICCP
Standby
Port 2
PE2
VFI_113
Active
Te0/5/0/1
BD113
PE1-PE2 VPLS Pseudowire
disabled and EVPN service
replaces VPLS service
PE2_PE1 DOWN
PE2_PE3 UP
PE2_PE4 UP
EVI_113
By default, the Active node is chosen based on the DF Modulo Election
VFI PE4
algorithm:
• Index of sorted list of PEs (increasing IP) + <3-6> bytes of ESI are used for
the election
• A specially crafted ESI can be used to define DF in a manual manner
Access-driven DF election option can be used:
• CE may define active/standby links based on LACP attributes (system
priority, port-priority, maximum active-links, etc.)
• It will force PE to bring port in active or standby state respectively
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
17
MC-LAG Migration: Remove MLACP & ICCP
• PE1 (Standby): Swap MC-LAG to EVPN MH
PE1
VFI_113
Te0/0/0/46
PE1_PE2 DOWN
PE1_PE3 UP
PE1_PE4 UP
MPLS
VFI
BD113
PE3
EVI_113
Standby
lacp configuration on the bundle interface
(save LACP attributes - system MAC,
system priority, etc.) and remove ICCP
configuration
• PE2 (Active): Repeat the step for the PE1:
BE 34
CE1
ICCP
•
Port 1
Port 2
PE2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE1 DOWN
PE2_PE3 UP
PE2_PE4 UP
VFI
PE4
To mitigate the transition to a new LACP you
can shut down the bundle interface on PE2
and switch over traffic to PE1 before
removing MLACP/ICCP. Then the bundle
port should be brought up.
redundancy
no iccp
!
interface Bundle-Ether34
no mlacp iccp-group 113
no mlacp port-priority 1
lacp system mac 3434.3434.3434
lacp system priority 1
EVI_113
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
18
MC-LAG Migration: Verifying EVPN P/A is working
PE1
NDF brings port to
Out-of-Service
(OOS) state
Te0/0/0/46
10.0.0.1
VFI_113
PE1_PE2 DOWN
PE1_PE3 UP
PE1_PE4 UP
BD113
EVI_113
Standby
Port 1
BE 34
CE1
PE2
10.0.0.2
Port 2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE1 DOWN
PE2_PE3 UP
PE2_PE4 UP
RP/0/RP0/CPU0:PE1#sho evpn ethernet-segment carv detail
<...>
Ethernet Segment Id
Interface
Nexthops
------------------------ ---------------------------------0034.3434.3434.3434.3401 BE34
10.0.0.1
10.0.0.2
VFI
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port
:
RP/0/RP0/CPU0:PE2#sho evpn ethernet-segment carv detail
Interface name : Bundle-Ether34
<...>
Interface MAC : bcd2.95d2.4983
Ethernet Segment Id
Interface
Nexthops
IfHandle
: 0x00000560
------------------------ ---------------------------------State
: Standby
0034.3434.3434.3434.3401 BE34
10.0.0.1
Redundancy : Not Defined
10.0.0.2
ESI type
:0
ES to BGP Gates : Ready
Value
: 34.3434.3434.3434.3401
ES to L2FIB Gates : Ready
ES Import RT
: 3434.3434.3434 (from ESI)
Main port
:
Source MAC
: 0000.0000.0000
(N/A)
VFI
Interface name : Bundle-Ether34
Topology
:
Interface MAC : 70e4.2240.d72c
Operational : MH
IfHandle
: 0x000001e0
Configured : Port-Active
State
: Up
<...>
Redundancy : Not Defined
MPLS
PE3
PE4
ESI type
:0
Value
: 34.3434.3434.3434.3401
ES Import RT
: 3434.3434.3434 (from ESI)
Source MAC
: 0000.0000.0000 (N/A)
Topology
:
Operational : MH
Configured : Port-Active
<...>
EVI_113
DF makes port active
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
19
MC-LAG Migration: Final State
NDF brings port to
Out-of-Service
(OOS) state
PE1
10.0.0.1
VFI_113
PE1_PE3 UP
PE1_PE4 UP
Te0/0/0/46
•
MPLS
VFI PE3
•
BD113
EVI_113
Standby
Port 1
BE 34
CE1
•
PE2
10.0.0.2
Port 2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE3 UP
PE2_PE4 UP
VFI
PE4
•
EVI_113
•
DF makes port
active
#CiscoLive
In this step PE1 & PE2 is working in EVPN PortActive redundancy mode.
Active port is chosen the same convention as the
Designated Forwarder Modulo Election algorithm,
however at the granularity of only [ESI] with the
use of Type 4 EVPN routes.
The PE that are not elected Active from DF
Election for the ESI will bring their bundle into
Out-of-Service (OOS).
OOS is sort of “warm standby” state, which
allows for standby port be rapidly resumed upon
failure of the active links.
Traffic between PE1/PE2 & PE3 & PE4 is going
via VPLS PWs
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
Seamless VPLS to
EVPN Migration
Configure EVPN on PE4: Seamless VPLS to EVPN
Use case (1)
NDF brings port to
Out-of-Service
(OOS) state
• PE4: Configure ESI, EVI, EVPN AF, etc..
PE1
10.0.0.1
VFI_113
PE1_PE3 UP
PE1_PE4 UP
Te0/0/0/46
MPLS
VFI PE3
BD113
EVI_113
Standby
Port 1
BE 34
CE1
PE2
10.0.0.2
Port 2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE3 UP
PE2_PE4 UP
VFI
BD
EVI_113
PE4
EVI
DF makes port
active
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
22
Configure EVPN on PE4: Seamless VPLS to EVPN
Use case (2)
• PE4: Configure ESI, EVI, EVPN AF, etc..
NDF brings port to
Out-of-Service
(OOS) state
PE1
10.0.0.1
VFI_113
PE1_PE3 UP
PE1_PE4 DOWN
Te0/0/0/46
• After PE1/PE2 & PE4 discover each
MPLS
VFI PE3
BD113
• Pseudowires are shut down
• EVPN service replaces VPLS service
EVI_113
Standby
other through EVPN routes:
Port 1
BE 34
CE1
PE2
10.0.0.2
Port 2
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE3 UP
PE2_PE4 DOWN
VFI
BD
EVI_113
PE4
EVI
DF makes port
active
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
23
Configure EVPN on PE4: Seamless VPLS to EVPN
Use case (3)
• PE4: Configure ESI, EVI, EVPN AF, etc..
NDF brings port to
Out-of-Service
(OOS) state
PE1
10.0.0.1
VFI_113
PE1_PE3 UP
PE1_PE4 DOWN
Te0/0/0/46
• After PE1/PE2 & PE4 discover each
MPLS
VFI PE3
BD113
• Pseudowires are shut down
• EVPN service replaces VPLS service
EVI_113
Standby
other through EVPN routes:
Port 1
BE 34
CE1
VFI on PE4 is by default in Split Horizon Group 1:
PE2
10.0.0.2
Port 2
• SHG1 protects loops in MPLS Core
VFI_113
Active
Te0/5/0/1
BD113
PE2_PE3 UP
PE2_PE4 DOWN
VFI
BD
EVI_113
PE4
EVI
• Full Mesh of VPLS pseudowires (PW) is
required for Any-to-Any forwarding
EVI is also by default in Split Horizon Group 1:
• PE4 doesn’t forward data between VFI and
EVI !
DF makes port
active
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
24
ICCP-SM Migration
ICCP-SM Migration: Overview
• Migration from ICCP-SM to EVPN Single-Active mode:
• No reconfiguration on CE
• Keep single-active design and per-VLAN load-balancing
• Per-VLAN Policing
evpn
...
!
interface Bundle-Ether12
ethernet-segment
load-balancing-mode single-active
!
• Support of both L2 and L3 services
• Greenfield and Brownfield deployments: Interoperability with EVPN and VPLS neighbors
• EVPN S/A is supported in IOS XR 6.2.2
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
26
ICCP-SM Migration: Initial state
• PE1 & PE2 work in ICCP-SM mode (Pseudo
MCLAG)
PE1
VFI_111
PE_PE2 UP
PE_PE3 UP
PE_PE4 UP
Block V111
BD111
ICCP
Allow V111
CE1
BD111
Allow V111
RP/0/RSP0/CPU0:PE2#show l2vpn iccp-sm group 1
WedVFI
Apr 20 14:29:09.788
UTC
CE3
PE3
ICCP-based Service Multi-Homing
Group ID: 1, State: Synchronized with Peer
Local Node ID: 2, Remote Node ID: 1,
ICCP State: Transport Up, Member Up
PE2
VFI_111
Allow V111
• VLAN 111 is active on PE2 and blocked on PE1
MPLS
PE_PE1 UP
PE_PE3 UP
PE_PE4 UP
Interface Name: Bundle-Ether12
MAC flushing: MVRP
Recovery Delay: 60 (Timer not running)
Local
VFIState: Operational CE4
Remote State: Operational
PE4
Local
VLAN IDs
State
---------------------------------------------------------------------Primary: 111
Forwarding (L)
Secondary: 112
Blocked (L)
Remote
VLAN IDs
State
---------------------------------------------------------------------Primary: 112
Forwarding (R)
Secondary: 111
Blocked (R)
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
27
ICCP-SM Migration: EVPN Single-Active
configuration on both PE1 and PE2
• PE1: Shutdown Access-faced bundle interface;
PE1
• PE1&PE2: Remove ICCP part;
VFI_111
PE1_PE2 UP
PE1_PE3 UP
PE1_PE4 UP
Block V111
BD111
MPLS
• PE1&PE2: Configure EVPN S/A part: ESI, EVI,
BGP, EVPN AF, load-balancing-mode singleVFI
PE3
active, etc..
CE1
ICCP
EVI_111
Allow V111
VFI_111
Allow V111
Allow V111
PE2
BD111
PE2_PE1 UP
PE2_PE3 UP
PE2_PE4 UP
EVI_111
RP/0/RSP0/CPU0:PE1#sho run l2vpn
l2vpn
!
bridge group VPLS
bridge-domain 111
interface
VFI Bundle-Ether12.111
!
vfi 111
neighbor 10.0.0.2 pw-id 111
!
neighbor 10.0.0.3 pw-id 111
!
neighbor 10.0.0.4 pw-id 111
!
evi 111
!
PE4
#CiscoLive
BRKMPL-2143
evpn
evi 111
advertise-mac
!
!
interface Bundle-Ether12
ethernet-segment
identifier type 0 12.12.12.12.12.12.12.12.01
load-balancing-mode single-active
!
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
28
ICCP-SM Migration: Step 2
Enable access-faced interface on PE2
PE1
NDF blocks all
traffic for <ES,
VLAN 111>
• After enabling access-facing interface on PE1, PE1 & PE2 is working
VFI_111
PE1_PE2 DOWN
PE1_PE3 UP
PE1_PE4 UP
Block V111
BD111
EVI_111
Allow V111
VFI
• After PE1/PE2
discover each other through EVPN routes:
PE3
• VPLS Pseudowire is shut down and EVPN service replaces
VPLS service
• Active Vlan (EVI) is chosen the same convention as the Designated
ESI
CE1
in MPLS
EVPN Single-Active redundancy mode.
PE2
VFI_111
Allow V111
BD111
PE2_PE1 DOWN
PE2_PE3 UP
PE2_PE4 UP
Allow V111
EVI_111
DF (Designated Forwarder) for <ES,
VLAN 111>
Forwarder Modulo Election algorithm:
• Static VLAN carving can be used as an option
VFI PE4
• The PE (PE1 in this case) that is not elected Active at the DF Election
for the EVI will block all traffic on this EVI in both directions.
• The interface will still up, so the CE1 has no way to know which one
is DF and which one is non-DF:
• STP-TCN / MVRP Mac Flush is used to initiate MAC address
flush on CE device
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
29
nV Cluster Migration
nV Edge/Cluster Migration Options
EVPN
• Migration Options:
• EVPN All-Active:
• No reconfiguration on CE
• Keep all-active design and per-flow loadbalancing
MPLS
Core
• vES feature is required to communicate with
VPLS
• EVPN Port-Active:
ICL
PE1
• No reconfiguration on CE (LACP is required)
A
PE2
Service
downtime
A
EVPN
• Save QoS policy on PE side
• Other benefits non-Cluster solution:
VLAN_10 (active)
VLAN_20 (active)
CE
VLAN_10 (active)
VLAN_20 (active)
nV Cluster
• Different types of chassis are supported in the
pair, non-proprietary solution, long term
roadmap for EVPN on SP products
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
31
Migrating ASR 9000 nV Cluster to EVPN AllActive
• High level procedure:
1.
De-clustering
2.
Upgrade each node to IOS XR 6.2.2 (EMR) release or later
3.
Configure EVPN All-Active using vES feature
Service downtime
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
32
EVPN VPWS
EVPN VPWS
•
Per-EVI Ethernet A-D route is used to signal
VPWS services
•
Eth.Tag ID = AC1
Label (e.g. X)
MPLS Label – (downstream assigned) used
by remote PEs to reach segment
Homing
PE discovery & signaling via a single protocol –
BGP
ESI = ES1
Eth.Tag ID – 4-bytes local AC-ID
• Support of Single-Homing and Multi-
•
RD = RD-1a
ESI – 10 bytes ESI as specify by EVPN
Ethernet segment IETF draft
• New BGP-based signaling (L3VPN-like)
• Relies on a sub-set of EVPN routes to
advertise Ethernet Segment and AC
reachability
PE 1 Eth A-D Route (per EVI)
RD – RD unique per adv. PE per EVI
RT ext. community
RT-a
RT – RT associated with a given EVI
Layer 2 ext. community
P-bit indicates primary PE (for A/A always P=1);
B-bit indicates backup PE (S/A,P/A), C-bit
indicates CW; MTU in bytes
Handles double-sided provisioning with remote
PE auto-discovery
• Seamless migration from PW to EVPN-VPWS
P-bit, B-bit, C-bit, MTU
VPWS Service Config:
EVI = 100
Local AC ID = AC2
Remote AC ID = AC1
EVI-EAD
CE1
ES1
VPWS Service Config:
EVI = 100
Local AC ID = AC1
Remote AC ID = AC2
PE1
MPLS
PE2
CE2
ES2
EVI-EAD
• Standardization is finished: RFC 8214
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
34
EVPN VPWS Access Topologies
ALL-Active == per-flow
load-balancing CE-PEs
Single bundle on CE device
PE1 Eth A-D Route (EVI)
CE1
ES NULL
AC1
Both PEs (PE1/PE2)
shows as next hop
for the remote AC
EVI-EAD
PE1
AC1
PE1
MPLS
PE2
4.4.4.4
1.1.1.1
ES NULL
CE2
CE1
AC4
PE3
ES NULL
PE2
Single-Homed (XR 6.0.1)
CE2
AC2
AC1
PE2 Eth A-D Route (EVI)
EVI-EAD
MH: All-Active (XR 6.2.1)
Only one PE (PE1) shows as
next hop for the remote AC
Single-Active == per-vlan
load-balancing CE-PEs
Two bundles on CE device
MPLS
ES1
Port-Active == per-ink
load-balancing CE-PEs
One bundle on CE device
Only one PE (PE1)
shows as next hop
for the remote AC
PE1
PE1
CE1
DF
MPLS
ESI-1
PE2
PE3
ES NULL
ESI-1
CE1
CE2
DF
BDF
AC2
MPLS
PE3
ES NULL
CE2
AC2
PE2
BDF
MH: Single-Active (XR 7.1.1)
MH: Port-Active (XR 7.1.2)
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
35
Seamless Migration to EVPN
VPWS
Seamless Migration to EVPN VPWS
Legacy PW
CE1
PE1
PE2
AC1
CE2
AC2
MPLS
Seamless
EVPN VPWS
CE1
PE1
PE2
AC1
CE2
AC2
MPLS
#CiscoLive
• Legacy PW:
• Discovery: Static/BGP-AD
• Signaling: LDP, BGP
• Data Plane: MPLS-LDP or SR
• Access Mode:
• Single-Homed
• Multi-Homed (MC-LAG to
EVPN Port-Active migration)
• Co-existence EVPN-VPWS and
Legacy PW on the same PE
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
37
Migration to Single-Homed EVPN VPWS: Initial
State
Legacy PW
“pw-id 5”
l2vpn
xconnect group P2P
p2p LEGACY_PW
interface TenGigE0/0/0/34.451
neighbor ipv4 10.0.9.0 pw-id 5
PE7
CE1
PE7
AC1 = Ten0/0/0/34.451
PE9
MPLS
CE2
AC2 = Ten0/1/0/1.451
PE7
RP/0/RP0/CPU0:PE7-9903-2T-8A04#sho l2vpn xconnect
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
------------------------ ----------------------------- ---------------------------P2P
LEGACY_PW UP
Te0/0/0/34.451
UP
10.0.9.0
5
UP
------------------------------------------------------------------------------------RP/0/RP0/CPU0:PE7-9903-2T-8A04#
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
38
Migration to Single-Homed EVPN VPWS: Adding
EVPN Migration Configuration on PE7
l2vpn
xconnect group P2P
p2p LEGACY_PW
vpws-seamless-integration
interface TenGigE0/0/0/34.451
neighbor ipv4 10.0.9.0 pw-id 5
!
p2p EVPN_VPWS
interface TenGigE0/0/0/34.451
neighbor evpn evi 5 service 5
PE7
CLI to enable
migrate to VPWS
Legacy PW
“pw-id 5”
CE1
PE7
PE9
AC1 = Ten0/0/0/34.451
PE7
MPLS
CE2
AC2 = Ten0/1/0/1.451
RP/0/RP0/CPU0:PE7-9903-2T-8A04#sho l2vpn xconnect
Wed May 31 22:43:33.438 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
•
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
------------------------ ------------------------ ----------------------------P2P
EVPN_VPWS DN Te0/0/0/34.451
UP
EVPN 5,5,10.0.9.0
DN
-------------------------------------------------------------------------------P2P
LEGACY_PW UP
Te0/0/0/34.451
UP
10.0.9.0
5
UP
--------------------------------------------------------------------------------
•
Legacy PW is UP and
forwarding data
New EVPN-VPWS
service is ready and
is signaled via BGP
EVPN AF
RP/0/RP0/CPU0:PE7-9903-2T-8A04#show bgp l2vpn evpn rd 10.0.7.0:5
Network
Next Hop
Metric LocPrf Weight Path
Route Distinguisher: 10.0.7.0:5 (default for vrf VPWS:5)
*> [1][0000.0000.0000.0000.0000][5]/120. 0.0.0.0
0i
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
Migration to Single-Homed EVPN VPWS: Adding
EVPN Migration Configuration on PE9
l2vpn
xconnect group P2P
p2p LEGACY_PW
vpws-seamless-integration
interface TenGigE0/0/0/34.451
neighbor ipv4 10.0.9.0 pw-id 5
!
p2p EVPN_VPWS
interface Ten0/0/0/34.451
neighbor evpn evi 5 service 5
PE7
l2vpn
xconnect group P2P
p2p LEGACY_PW
vpws-seamless-integration
interface TenGigE0/1/0/1.451
neighbor ipv4 10.0.7.0 pw-id 5
!
p2p EVPN_VPWS
interface TenGigE0/1/0/1.451
neighbor evpn evi 5 service 5
PE9
Legacy PW
“pw-id 5”
CE1
MPLS
PE7
AC1 = Ten0/0/0/34.451
EVPN VPWS
PE9
CE2
AC2 = Ten0/1/0/1.451
PE7
RP/0/RP0/CPU0:PE7-9903-2T-8A04#sho l2vpn xconnect
XConnect
Segment 1
Segment 2
Group
Name
ST Description
ST
Description
ST
------------------------ --------------------- ----------------------------P2P
EVPN_VPWS
UP Te0/0/0/34.451 UP
EVPN 5,5,10.0.9.0
UP
----------------------------------------------------------------------------P2P
LEGACY_PW
DN Te0/0/0/34.451 SB(SI) 10.0.9.0
5
UP
-----------------------------------------------------------------------------
•
•
•
EVPN_VPWS is UP
Legacy LDP-based PW is Down, service is in
Seamless Inactive (SI) mode
Legacy PW can be removed
When both Legacy-PW and EVPN VPWS
exist for the same AC, EVPN VPWS gets
preference over the Legacy PW
RP/0/RP0/CPU0:PE7-9903-2T-8A04#show bgp l2vpn evpn rd 10.0.7.0:5
Network
Next Hop
Metric LocPrf Weight Path
Route Distinguisher: 10.0.7.0:5 (default for vrf VPWS:5)
*> [1][0000.0000.0000.0000.0000][5]/120
0.0.0.0
0i
*i
10.0.9.0
100
0i
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
40
Migration to Multi-Homed EVPN VPWS: MC-LAG
to EVPN Port-Active
• ICCP-based MC-LAG on PE7/PE8
• CE78 is connected by EtherChannel to PE7/PE8
• Link to PE8 (Standby PoA) is not active
Active PW
MPLS
CE9
PE9
AC2
ICCP
Backup PW
AC1
PE7
PE8
CE78
AC2
EVPN VPWS (P)
ES NULL
CE9
MPLS
PE7
PE9
AC1
AC2
ES1
EVPN VPWS (B)
PE8
AC2
CE78
#1 PE9:
• EVPN-VPWS Seamless Migration
configuration
#2 PE8:
• Bundle interface shutdown
• EVPN VPWS Port-Active configuration
• Remove mLACP/ICCP configuration
#3 PE7:
• EVPN VPWS Port-Active configuration
• Remove mLACP/ICCP configuration
#4 PE8:
• Bundle interface no shutdown
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
41
EVPN Head-End
L3 EVPN Head End (EVPN-HE)
A-PE1
EVPN
VPWS
CE1
A-PE2
S-PE1
HE
VPNv4/6
HE
S-PE2
EVPN Head-End Modes (S-PE):
Access Modes (A-PE):
•
•
•
•
•
•
All-Active EVPN-VPWS
Port-Active EVPN-VPWS
Single-Active EVPN-VPWS
#CiscoLive
Single-Active (default)
Port-Active
All-Active
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
43
EVPN-HE MH Single-Active (Default) LoadBalancing Mode DF (P-bit)
vlan2
EVPN
VPWS
A-PE2
A (P-bit)
Lowest IP (default)
vlan2
vlan1
A (P-bit)
NDF (B-bit)
DF (P-bit)
A-PE2
vlan2
vlan1
CORE-FACING SIDE (L3)
Both S-PEs Up/Up and Data-plane is Forward/Forward
S-PE1
HE
ES 2
EVPN
VPWS
ES 1
CE1
A-PE1 vlan2
VPNv4/6
HE
S-PE2
vlan1
A (P-bit)
CORE-TO-ACCESS (L2)
NDF HE-node:
•
brings PW-Ether main & sub interfaces UP
•
sends B-bit to remote PEs
•
forwards all VLANs to same remote DF A-PE
DF HE-node:
•
brings PW-Ether main & sub interfaces UP
•
sends P-bit to remote PEs
•
forwards all VLANs to same remote DF PE
S-PE1
HE
ES 2
CE1
vlan1
A-PE1
ES 1
A (P-bit)
VPNv4/6
ACCESS-TO-CORE (L2)
Only DF HE-node forwards traffic from Access to Core
• Based on the P-bit
HE
S-PE2
NDF (B-bit)
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
44
EVPN-HE MH Single-Active (Default) LoadBalancing Mode - Configuration
S-PE9
EVPN
VPWS
S-PE
NDF (B-bit)
PW-ETH41
VPNv4/6
ES 2
CE1
10.0.9.0
A-PE33
ES 1
A (P-bit)
10.0.33.0
vrf VRF_HE_41
evpn-route-sync 41
address-family ipv4 unicast
import route-target
24:41
!
export route-target
24:41
PW-ETH41
A-PE34
A (P-bit)
10.0.34.0
Lowest IP (default)
S-PE10
10.0.10.0
DF (P-bit)
A-PE
interface Bundle-Ether40.1 l2transport
encapsulation dot1q 411 , 412
evpn
evi 1
transmit-l2-mtu
!
interface Bundle-Ether40
ethernet-segment
identifier type 0 33.34.00.00.00.00.00.40.00
l2vpn
xconnect group evpn_vpws
p2p evpn_vpws_sa_411_412
interface Bundle-Ether40.1
neighbor evpn evi 1 target 1 source 1
Lowest IP is
default option
(Modulo or Highest
IP options can be
used as well)
ES 1
ES 2
Allows to synchronize
ARP/ND/IGMP
interface PW-Ether41
mtu 1518
mac-address 0.910.41
attach generic-interface-list GIL1
logging events link-status
!
interface PW-Ether41.1
vrf VRF_HE_41
ipv4 address 24.40.11.1 255.255.255.0
load-interval 30
encapsulation dot1q 411
logging events link-status
!
evpn
evi 1
transmit-l2-mtu
!
interface PW-Ether41
ethernet-segment
identifier type 0 09.10.00.00.00.00.00.41.00
generic-interface-list GIL1
!
interface HundredGigE0/0/0/6
2vpn
interface HundredGigE0/2/0/1/0
ignore-mtu-mismatch
xconnect group EVPN_HE
p2p PWHE41
interface PW-Ether41
router bgp 65000
neighbor evpn evi 1 target 1 source 1
<…>
vrf VRF_HE_41
rd auto
address-family ipv4 unicast
redistribute connected
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
45
EVPN-HE MH Single-Active (Default) LoadBalancing Mode – Verifying: A-PE
A-PE33
24.40.11.18/24
A (P-bit)
NDF (B-bit)
DF S-PE10 sends P-bit,
NDF S-PE9 sends B-bit
S-PE9
A-PE_33
PW-ETH41
RP/0/RP0/CPU0:R33-5501-H11-2#show evpn internal-label
Sat May 27 17:50:07.062 PDT
VPNv4/6
ES 2
CE1
10.0.9.0
EVPN
VPWS
ES 1
A (P-bit)
10.0.33.0
VPN-ID
Encap Ethernet Segment Id
EtherTag
---------- ------ --------------------------- ---------1
MPLS 0009.1000.0000.0000.4100
1
Summary pathlist:
0x02000001 (P) 10.0.10.1
0x00000000 (B) 10.0.9.1
PW-ETH41
A-PE34
10.0.34.0
S-PE10
10.0.10.0
DF (P-bit)
A-PE_33
RP/0/RP0/CPU0:R33-5501-H11-2#sho l2vpn xconnect
Sat May 27 18:22:26.371 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
Label
-------24010
24034
24037
1
MPLS 0009.1000.0000.0000.4100
4294967295 None
1
MPLS 0033.3400.0000.0000.4100
1
1
MPLS 0033.3400.0000.0000.4100
4294967295 None
None
RP/0/RP0/CPU0:R33-5501-H11-2#
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
---------------------------------------------------- ----------------------evpn_vpws evpn_vpws_sa_411_412
UP
BE40.1
UP
EVPN 1,1,24010
UP
---------------------------------------------------------------------------------------RP/0/RP0/CPU0:R33-5501-H11-2#
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
46
EVPN-HE MH Single-Active (Default) Mode –
Verifying: S-PE NDF
S-PE_9
RP/0/RSP0/CPU0:PE9-9906TL-H9#show evpn ethernet-segment carving detail
Ethernet Segment Id
Interface
Nexthops
------------------------ ---------------------------------- ---------0009.1000.0000.0000.4100 PE41
10.0.9.0
10.0.10.0
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port
:
Interface name : PW-Ether41
Interface MAC : 0000.0910.0041
IfHandle
: 0x00000720
State
: Up
Redundancy : Not Defined
ESI type
:0
Value
: 0009.1000.0000.0000.4100
<…>
Topology
:
Operational : MH, Anycast mode
Configured : Anycast Single-active (default)
Service Carving : Auto-selection
Multicast
: Disabled
<…>
EVPN-VPWS Service Carving Results:
Primary
:0
Backup
:1
EVI:ETag B :
1:1
Non-DF
:0
RP/0/RSP0/CPU0:PE9-9906TL-H9#sho l2vpn xconnect detail
Sat May 27 15:53:33.851 PDT
S-PE_9
Group EVPN_HE, XC PWHE41, state is down; Interworking none
AC: PW-Ether41, state is up
Type PW-Ether
Interface-list: GIL1
<…>
Internal label: 24033
Statistics:
packets: received 5, sent 3
bytes: received 536, sent 354
EVPN: neighbor 10.0.33.0, PW ID: evi 1, ac-id 1, state is standby ( provisioned )
XC ID 0xa0000009
Encapsulation MPLS
Encap type Ethernet, control word enabled
Sequencing not set
Ignore MTU mismatch: Enabled
Transmit MTU zero: Disabled
LSP : Up
Down reason(s): Not primary DF
<…>
PWHE Internal Label: 24033
Statistics:
packets: received 5, sent 3
bytes: received 536, sent 354
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
47
EVPN-HE MH Single-Active (Default) Mode –
Verifying: S-PE -> A-PE Traffic Forwarding
Although both A-PEs
work in All-Active
Mode, only one will
be used as NH
A-PE with the lowest IP by
default is used for
forwarding when A/A mode
on A-PEs
RP/0/RSP0/CPU0:PE9-9906TL-H9#show evpn internal-label vpn-id 1 esi
0033.3400.0000.0000.4100 detail
VPN-ID Encap Ethernet Segment Id
EtherTag Label
---------- ------ --------------------------- ---------- -------1
MPLS 0033.3400.0000.0000.4100 1
None
Multi-paths resolved: TRUE (Remote all-active)
Multi-paths Internal label: None
EAD/ES
10.0.33.0
0
10.0.34.0
0
EAD/EVI (P) 10.0.33.0
24009
(P) 10.0.34.0
24024
Summary pathlist:
0x02000003 (P) 10.0.33.0
24009
0x02000001 (P) 10.0.34.0
24024
<…>
RP/0/RSP0/CPU0:PE9-9906TL-H9#show l2vpn ma pwhe interface pw-ether 41 private
Interface: PW-Ether41 Interface State: Up, Admin state: Up
Interface handle 0x720
MTU: 1518
BW: 10000 Kbit
Interface MAC addresses (2 addresses):
Config: 0000.0910.0041
EMA : 6c6c.d377.353f
<…>
PW-HE IDB client data
--------------------IDB handle 0x56273c48c210
Dot1q vlan: 0x81000000
Label: 24033
Remote VC label: 24009
Remote PE: 10.0.33.0
Use flow-label on tx: N
Use flow-label on rx: N
Use flow load-balancing: N
L2-overhead: 0
VC-type: 5
CW: Y
<…>
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
48
EVPN-HE MH All-Active Load-Balancing Mode
A-PE1
ES 1
A-PE2
A (P-bit)
Lowest IP (default)
CE1
vlan2
vlan1
A-PE1
A (P-bit) A-PE2
Lowest IP (default)
EVPN
VPWS
vlan1
VPNv4/6
HE
S-PE2
A (P-bit)
A (P-bit)
vlan2
ES 1
A (P-bit)
EVPN
VPWS
S-PE1
HE
S-PE1
HE
ES 2
CE1
A (P-bit)
vlan1
ES 2
vlan2
A (P-bit)
VPNv4/6
HE
S-PE2
CORE-TO-ACCESS (L2)
NDF HE-node:
•
brings PW-Ether main & sub interfaces UP
•
sends P-bit to remote PEs
•
forwards all VLANs to same remote PE using
preferred next-hop CLI when remote is MH A/A
DF HE-node:
•
brings PW-Ether main & sub interfaces UP
•
sends P-bit to remote PEs
•
forwards all VLANs to same remote PE using
preferred next-hop CLI when remote is MH A/A
CORE-FACING SIDE (L3)
Both S-PEs Up/Up and Data-plane is Forward/Forward
ACCESS-TO-CORE (L2)
Both HE-nodes are Active and forward traffic from
Access to Core
A (P-bit)
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
49
EVPN-HE MH All-Active Load-Balancing Mode
Configuration
A (P-bit)
S-PE
10.0.9.0
S-PE9
A-PE33
EVPN
VPWS
PW-ETH43
PW-ETH43
A-PE
interface Bundle-Ether40.3 l2transport
encapsulation dot1q 431 , 432
10.0.10.0
A (P-bit)
Per EVI load-balancing
* Lowest IP or Highest IP
can be used as well
evpn
evi 3
transmit-l2-mtu
!
interface Bundle-Ether40
ethernet-segment
identifier type 0 33.34.00.00.00.00.00.40.00
l2vpn
xconnect group evpn_vpws
p2p evpn_vpws_sa_431_432
interface Bundle-Ether40.3
neighbor evpn evi 3 target 3 source 3
vrf VRF_HE_43
evpn-route-sync 43
address-family ipv4 unicast
import route-target
24:43
!
export route-target
24:43
S-PE10
A-PE34
A (P-bit)
10.0.34.0
Lowest IP (default)
VPNv4/6
ES 2
CE1
10.0.33.0
ES 1
A (P-bit)
ES 1
ES 2
Allows to synchronize
ARP/ND/IGMP
interface PW-Ether43
mtu 1518
mac-address 0.910.43
attach generic-interface-list GIL1
logging events link-status
!
interface PW-Ether43.1
vrf VRF_HE_43
ipv4 address 24.40.31.1 255.255.255.0
load-interval 30
encapsulation dot1q 431
logging events link-status
!
evpn
evi 3
transmit-l2-mtu
preferred-nexthop modulo
!
interface PW-Ether43
ethernet-segment
identifier type 0 09.10.00.00.00.00.00.43.00 generic-interface-list GIL1
interface HundredGigE0/0/0/6
load-balancing-mode all-active
interface HundredGigE0/2/0/1/0
!
2vpn
ignore-mtu-mismatch
xconnect group EVPN_HE
router bgp 65000
p2p PWHE43
<…>
interface PW-Ether43
vrf VRF_HE_43
neighbor evpn evi 3 target 3 source 3
rd auto
address-family ipv4 unicast
redistribute connected
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
50
EVPN-HE MH All-Active Load-Balancing Mode –
Verifying: A-PE
A (P-bit)
A-PE33
A-PE34
S-PE9
EVPN
VPWS
A (P-bit)
10.0.34.0
Lowest IP (default)
Both S-PE9 and S-PE10
send P-bit
10.0.9.0
A-PE_33
PW-ETH43
VPNv4/6
ES 2
CE1
10.0.33.0
ES 1
A (P-bit)
RP/0/RP0/CPU0:R33-5501-H11-2#sho evpn internal-label vpn-id 3
Sat May 27 23:59:00.170 PDT
VPN-ID
Encap Ethernet Segment Id
EtherTag
---------- ------ --------------------------- ---------3
MPLS 0009.1000.0000.0000.4300
3
Summary pathlist:
0x02000002 (P) 10.0.9.1
0x02000001 (P) 10.0.10.1
PW-ETH43
S-PE10
10.0.10.0
A (P-bit)
A-PE_33
RP/0/RP0/CPU0:R33-5501-H11-2#sho l2vpn xconnect
Sun May 28 00:07:19.860 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
Label
-------24020
24045
24037
3
MPLS 0009.1000.0000.0000.4300
4294967295 None
3
MPLS 0033.3400.0000.0000.4000
3
3
MPLS 0033.3400.0000.0000.4000
4294967295 None
None
RP/0/RP0/CPU0:R33-5501-H11-2#
XConnect
Segment 1
Segment 2
Group
Name
ST Description
ST
Description
ST
------------------------ ----------------------------- ----------------------------evpn_vpws evpn_vpws_aa_431_432
UP BE40.3
UP
EVPN 3,3,24020
UP
---------------------------------------------------------------------------------------RP/0/RP0/CPU0:R33-5501-H11-2#
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
EVPN-HE MH All-Active Load-Balancing Mode –
Verifying: S-PE
S-PE_9
S-PE_9
RP/0/RSP0/CPU0:PE9-9906TL-H9#sho l2vpn xconnect
Sun May 28 00:13:32.020 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
------------------------ ------------------- ----------------------------EVPN_HE PWHE43 UP
PE43
UP
EVPN 3,3,10.0.34.0 UP
--------------------------------------------------------------------------RP/0/RSP0/CPU0:PE9-9906TL-H9#
RP/0/RSP0/CPU0:PE10-9906TL-H4#sho l2vpn xconnect
Sun May 28 00:15:37.241 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
------------------------ ------------------- ----------------------------EVPN_HE PWHE43 UP
PE43
UP
EVPN 3,3,10.0.34.0 UP
--------------------------------------------------------------------------RP/0/RSP0/CPU0:PE10-9906TL-H4#
S-PE_10
#CiscoLive
RP/0/RSP0/CPU0:PE9-9906TL-H9#show evpn ethernet-segment carving detail
Ethernet Segment Id
Interface
Nexthops
------------------------ ---------------------------- ---------0009.1000.0000.0000.4300 PE43
10.0.9.0
10.0.10.0
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port
:
Interface name : PW-Ether43
Interface MAC : 0000.0910.0043
IfHandle
: 0x00000960
State
: Up
Redundancy : Not Defined
ESI type
:0
Value
: 0009.1000.0000.0000.4300
<…>
Topology
:
Operational : MH, All-active
Configured : All-active (AApF)
Service Carving : Auto-selection
Multicast
: Disabled
<…>
EVPN-VPWS Service Carving Results:
Primary
:1
EVI:ETag P :
3:3
Backup
:0
Non-DF
:0
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
EVPN-HE MH All-Active Mode – Verifying:
Per-EVI S-PE -> A-PE Balancing
Per-EVI Load-Balancing
(Modulo)
RP/0/RSP0/CPU0:PE9-9906TL-H9#show evpn internal-label vpn-id 3 detail
Sun May 28 02:21:25.258 PDT
VPN-ID
Encap Ethernet Segment Id
EtherTag Label
---------- ------ --------------------------- ---------- -------<…>
3
MPLS 0033.3400.0000.0000.4000
3
None
Multi-paths resolved: TRUE (Remote all-active) (Preferred NH, Modulo)
Multi-paths Internal label: None
EAD/ES
10.0.33.0
0
10.0.34.0
0
EAD/EVI (P) 10.0.33.0
24019
(P) 10.0.34.0
24032
Summary pathlist:
0x02000001 (P) 10.0.34.0
24032
0xffffffff
(B) 10.0.33.0
24019
<…>
S-PE_9
NH (A-PE) used by PWHE
for pindown
RP/0/RSP0/CPU0:PE9-9906TL-H9#show l2vpn ma pwhe interface pw-ether 43 private
Interface: PW-Ether43 Interface State: Up, Admin state: Up
Interface handle 0x960
MTU: 1518
BW: 10000 Kbit
Interface MAC addresses (2 addresses):
Config: 0000.0910.0043
EMA : 6c6c.d377.353f
<…>
PW-HE IDB client data
--------------------IDB handle 0x56273c4930d0
Dot1q vlan: 0x81000000
Label: 24038
Remote VC label: 24032
Remote PE: 10.0.34.0
Use flow-label on tx: N
Use flow-label on rx: N
Use flow load-balancing: N
L2-overhead: 0
VC-type: 5
CW: Y
FSM state: 'Up'(7)
<…>
S-PE_9
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
53
EVPN-HE MH Port-Active
Load-Balancing Mode
DF (P-bit)
vlan1
ES 1
CE1
EVPN
VPWS
DF (P-bit)
A-PE2
vlan2
CORE-TO-ACCESS
NDF HE-node:
•
brings PW-Ether main & sub interfaces DOWN
•
sends B-bit to remote PEs
DF HE-node:
•
brings PW-Ether main & sub interfaces UP
•
sends P-bit to remote PEs
•
forwards all VLANs to same remote DF PE
S-PE1
HE
ES 2
NDF (B-bit)
A-PE1
VPNv4/6
HE
S-PE2
NDF (B-bit)
CORE-FACING SIDE (L3)
Only the DF HE-node is Up and Data-plane is
Forward
DF (P-bit)
EVPN
VPWS
vlan2
DF (P-bit)
A-PE2
S-PE1
HE
ES 2
CE1
A-PE1
ES 1
NDF (B-bit)
VPNv4/6
ACCESS-TO-CORE (L2)
Only DF HE-node forward traffic from Access to Core
• Based on the P-bit
HE
S-PE2
vlan1
NDF (B-bit)
QoS accuracy can be reached
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
54
EVPN-HE MH Port-Active Load-Balancing Mode
NDF (B-bit)
10.0.33.0
10.0.9.0
S-PE9
A-PE33
DF (P-bit)
PW-ETH42
PW-ETH42
10.0.10.0
NDF (B-bit)
A-PE
interface Bundle-Ether40.2 l2transport
encapsulation dot1q 421 , 422
evpn
evi 3
transmit-l2-mtu
!
interface Bundle-Ether40
ethernet-segment
identifier type 0 33.34.00.00.00.00.00.40.00
l2vpn
xconnect group evpn_vpws
p2p evpn_vpws_pa_421_422
interface Bundle-Ether40.2
neighbor evpn evi 2 target 2 source 2
vrf VRF_HE_42
evpn-route-sync 42
address-family ipv4 unicast
import route-target
24:42
!
export route-target
24:42
S-PE10
A-PE34
10.0.34.0
VPNv4/6
ES 2
EVPN
VPWS
ES 1
CE1
S-PE
DF (P-bit)
A-PE with the
highest IP as a
preferred NH
(Modulo or Lowest
IP options can be
used as well)
ES 1
ES 2
Allows to synchronize
ARP/ND/IGMP
interface PW-Ether42
mtu 1518
mac-address 0.910.42
attach generic-interface-list GIL1
logging events link-status
!
interface PW-Ether42.1
vrf VRF_HE_42
ipv4 address 24.40.21.1 255.255.255.0
load-interval 30
encapsulation dot1q 421
logging events link-status
!
evpn
evi 2
transmit-l2-mtu
preferred-nexthop highest-ip
!
interface PW-Ether42
ethernet-segment
identifier type 0 09.10.00.00.00.00.00.42.00 generic-interface-list GIL1
interface HundredGigE0/0/0/6
load-balancing-mode port-active
interface HundredGigE0/2/0/1/0
!
2vpn
ignore-mtu-mismatch
xconnect group EVPN_HE
router bgp 65000
p2p PWHE42
<…>
interface PW-Ether42
vrf VRF_HE_42
neighbor evpn evi 2 target 2 source 2
rd auto
address-family ipv4 unicast
redistribute connected
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
55
EVPN-HE MH Port-Active Load-Balancing Mode
- Verifying: A-PE
NDF (B-bit)
10.0.33.0
10.0.9.0
A-PE34
10.0.34.0
A-PE_33
PW-ETH42
RP/0/RP0/CPU0:R33-5501-H11-2#show evpn internal-label vpn-id 2
Sun May 28 12:56:38.299 PDT
VPNv4/6
ES 2
EVPN
VPWS
ES 1
DF (P-bit)
DF S-PE9 sends P-bit,
NDF S-PE10 sends B-bit
S-PE9
A-PE33
CE1
DF (P-bit)
VPN-ID
Encap Ethernet Segment Id
EtherTag
---------- ------ --------------------------- ---------2
MPLS 0009.1000.0000.0000.4200
2
Summary pathlist:
0x02000002 (P) 10.0.9.1
0x00000000 (B) 10.0.10.1
PW-ETH42
S-PE10
10.0.10.0
NDF (B-bit)
A-PE_33
RP/0/RP0/CPU0:R33-5501-H11-2#sho l2vpn xconnect
Sun May 28 12:53:24.488 PDT
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed,
LU = Local Up, RU = Remote Up, CO = Connected, (SI) = Seamless Inactive
Label
-------24018
24034
24003
2
MPLS 0009.1000.0000.0000.4200
4294967295 None
2
MPLS 0033.3400.0000.0000.4000
2
2
MPLS 0033.3400.0000.0000.4000
4294967295 None
None
RP/0/RP0/CPU0:R33-5501-H11-2#
XConnect
Segment 1
Segment 2
Group
Name
ST
Description
ST
Description
ST
---------------------------------------------------- ----------------------evpn_vpws evpn_vpws_pa_421_422
UP
BE40.2
UP
EVPN 2,2,24018
UP
---------------------------------------------------------------------------------------RP/0/RP0/CPU0:R33-5501-H11-2#
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
56
EVPN-HE MH Port-Active Mode – Verifying:
S-PE NDF keeps PW-Eth interface in Down state
S-PE_10
S-PE_10
RP/0/RSP0/CPU0:PE10-9906TL-H4#show evpn ethernet-segment esi
0009.1000.0000.0000.4200 carving detail
Ethernet Segment Id
Interface
Nexthops
------------------------ ---------------------------------- -----------0009.1000.0000.0000.4200 PE42
10.0.9.0
10.0.10.0
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port
:
Interface name : PW-Ether42
Interface MAC : 0000.0910.0042
IfHandle
: 0x00000920
State
: Standby
<…>
Topology
:
Operational : MH
Configured : Port-Active
<…>
Forwarders : 1
Elected
:0
Not Elected : 0
EVPN-VPWS Service Carving Results:
Primary
:0
Backup
:1
EVI:ETag B :
2:2
Non-DF
:0
<…>
#CiscoLive
RP/0/RSP0/CPU0:PE10-9906TL-H4#sho l2vpn xconnect group EVPN_HE xcname PWHE42 detail
Group EVPN_HE, XC PWHE42, state is down; Interworking none
AC: PW-Ether42, state is up
Type PW-Ether
Interface-list: GIL1
<…>
Internal label: 24000
Statistics:
packets: received 0, sent 0
bytes: received 0, sent 0
EVPN: neighbor 10.0.34.0, PW ID: evi 2, ac-id 2, state is standby ( provisioned )
XC ID 0xa0000007
Encapsulation MPLS
Encap type Ethernet, control word enabled
Sequencing not set
Ignore MTU mismatch: Enabled
Transmit MTU zero: Disabled
LSP : Up
Down reason(s): AC parent down
Not primary DF
<…>
PWHE Internal Label: 24000
Statistics:
packets: received 0, sent 0
bytes: received 0, sent 0
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
57
EVPN-HE MH Port-Active Mode – Verifying:
S-PE DF forwards all traffic
RP/0/RSP0/CPU0:PE9-9906TL-H9#sho int | i PW-Ether42
Sun May 28 13:41:04.455 PDT
PW-Ether42 is up, line protocol is up
PW-Ether42.1 is up, line protocol is up
PW-Ether42.2 is up, line protocol is up
RP/0/RSP0/CPU0:PE9-9906TL-H9#
S-PE_9
RP/0/RSP0/CPU0:PE9-9906TL-H9#show bgp vpnv4 unicast rd 10.0.9.0:2
advertised summary
Sun May 28 13:38:39.154 PDT
Network
Next Hop
From
Advertised to
Route Distinguisher: 10.0.9.0:2
Route Distinguisher Version: 1026
24.40.21.0/24
10.0.9.0
Local
10.0.3.0
Local
10.0.4.0
24.40.22.0/24
10.0.9.0
Local
10.0.3.0
Local
10.0.4.0
RP/0/RSP0/CPU0:PE10-9906TL-H4#sho int | i PW-Ether42
Sun May 28 13:40:07.296 PDT
PW-Ether42 is down, line protocol is down
PW-Ether42.1 is down, line protocol is down
PW-Ether42.2 is down, line protocol is down
RP/0/RSP0/CPU0:PE10-9906TL-H4#
S-PE_10
RP/0/RSP0/CPU0:PE10-9906TL-H4#show bgp vpnv4 unicast rd 10.0.10.0:2
advertised summary
Sun May 28 13:43:12.514 PDT
RP/0/RSP0/CPU0:PE10-9906TL-H4#
NDF HE-node doesn’t
participate in traffic
forwarding
Processed 2 prefixes, 4 paths
RP/0/RSP0/CPU0:PE9-9906TL-H9#
DF HE-node is Up
and attracts all
traffic
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
58
Migration of Multi-Homed
Network (MHN)
Single-Flow-Active (SFA) as a Migration
Option for Legacy MHN
PE9
PE9
VPLS
EVPN
PE8
PE7
SW1
MST/REP/G.8032
L2
SW2
PE8
PE7
FWD
SW1
SW4
No DF Election
No SHG filter
FWD
MST/REP/G.8032
SW4
L2
SW2
SW3
7.3.1
SW3
EVPN SFA (Single-Flow-Active)
draft-ietf-bess-evpn-l2gw-proto
Scale and convergence
challenges
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
60
Why All-Active Mode cannot be used with Rings?
•
PE9
EVPN
MPLS
ECMP
PE7
NDF*
PE8
DF
election
SW1
Isolated
switches
for BUM
•
SHG
Filter
SW4
L2
SW2
DF
SW3
Location of
blocked port
•
Creates L2 and L3 aliasing resulting in
remote ECMP towards “wrong” PE :
• Unicast flow from PE9 to SW3 can not
be going via PE8 due to L2GW protocol
blocks the link between SW3-SW4
L2 BUM isolated switches:
• BUM from PE9 can’t reach SW1, SW2,
SW3 due to NDF blocks BUM from core
to access network
TCN propagation blocked by ESI splithorizon label filtering:
• TCN packets from SW4 can’t reach
SW1, SW2, SW3 via MPLS core due to
SHG filtering
* NDF in AApF blocks BUM traffic from core to access
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
61
Why Single-Active Mode cannot be used with
Rings?
PE9
B
EVPN MPLS
A
SHG
Filter
PE7
NDF**
PE8
DF
election
SW1
Isolated
switches
•
SW4
L2
SW2
DF
SW3
Location of
blocked port
•
Isolates network segments
• No way to reach SW1, SW2, SW3
from the PE9 due to DF only capable
to forward traffic to/from access
network
TCN propagation is blocked by port state
or ESI split-horizon label filtering
• TCN packets from SW4 can’t reach
SW1, SW2, SW3 via MPLS core due
to SHG filtering
** NDF in AApS blocks all traffic for both directions
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
62
Single-Flow-Active mode: Key features
PE9
EVPN
BVI401
PE7
FWD
PE8
BVI401
No DF Election
No SHG filter
FWD
G.8032, MSTP,
REP, MPLS-TP
SW4
ESI
SW1
SW2
SW3
1. Same ESI on peering PEs, No DF
Election:
o L2 Loop-Prevention protocol
dictates forwarding state
2. No Split-Horizon filtering:
o L2 Loop-Prevention protocol
ensures no loop
Location of
blocked port
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
63
Single-Flow-Active mode: Key features
3. New Single-Flow Active Flag (0x02) in BGP ESI Label
Extended Community (RT-1 Per-ES EAD Route)
PE_7
PE9
EVPN
ES-EAD (RT1): RD, ESI,
EXT COMM: SFA Flag 0x02,RT
BVI401
PE8
PE7
FWD
BVI401
FWD
ESI
SW1
G.8032, MSTP,
REP, MPLS-TP
SW2
SW3
SW4
Location of
blocked port
RP/0/RP0/CPU0:PE7-9903-2T-8A04#sho bgp l2vpn evpn rd 10.0.7.0:0
[1][10.0.7.0:1][0007.0800.0000.0000.4000][4294967295]/184
Mon May 29 19:01:12.770 PDT
BGP routing table entry for [1][10.0.7.0:1][0007.0800.0000.0000.4000][4294967295]/184, Route Distinguisher:
10.0.7.0:0
Versions:
Process
bRIB/RIB SendTblVer
Speaker
5
5
Local Label: 0
Last Modified: May 28 18:55:01.664 for 1d00h
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (10.0.7.0)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 5
Extended community: EVPN ESI Label:0x02:24018 RT:65000:40
RP/0/RP0/CPU0:PE7-9903-2T-8A04#
Redundancy mode
All-Active:
0x00
Single-Active:
0x01
Single-Flow-Active: 0x02 NEW!
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
64
Single-Flow-Active mode: Key features
4. MAC/IP Sync
o
IP3,M3 -> PE7, LP 100 (best)
o
PE9
PE7 advertise SW3 MAC+IP EVPN RT2 with BGP LocalPreference 100
PE8 synchronize SW3 ARP/ND: FIB Next-Hop -> PE1
RP/0/RP0/CPU0:PE8-9903-8H-8A04#show arp vrf SFA1
RT-2: IP3,
M3, LP100
Address
4.40.1.1
4.40.1.24
EVPN
RT-2:MPLS
IP3,
M3, LP100
Age
-
PE_8
Hardware Addr State
Type Interface
0078.0040.0001 Interface ARPA BVI401
003a.9c9c.3027 EVPN_SYNC ARPA BVI401
RP/0/RP0/CPU0:PE8-9903-8H-8A04#show evpn evi mac 003a.9c9c.3027 private
BVI401
PE8
PE7
FWD
BVI401
FWD
ESI
SW1
G.8032, MSTP,
REP, MPLS-TP
SW2
SW3
SW4
Location of
blocked port
IP3: 4.40.1.24
M3: 003a.9c9c.3027
VPN-ID Encap
MAC address IP address
Nexthop
Label
SID
------- ------- ------------ ---------------------------------------- ----------------40
MPLS
003a.9c9c.3027 4.40.1.24
10.0.7.0
24008
<…>
Ext Flags
: 0x00000500 (Pref-Rib,LP-80,)
RP/0/RP0/CPU0:PE8-9903-8H-8A04#show cef vrf SFA1 4.40.1.24
4.40.1.24/32, version 90, internal 0x5000001 0x30 (ptr 0x78443b2c) [1], 0x0 (0x0), 0x208 (0x79cc6708)
<…>
via 10.0.7.0/32, 7 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0x79def760 0x0]
recursion-via-/32
next hop VRF - 'default', table - 0xe0000000
next hop 10.0.7.0/32 via 16007/0/21
next hop 10.7.8.2/32 Hu0/0/0/4 labels imposed {ImplNull 24021}
next hop 10.7.8.4/32 Hu0/0/0/5 labels imposed {ImplNull 24021}
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
65
Single-Flow-Active mode: Key features
5. Backup Path, No Aliasing
IP3,M3 -> PE7, LP 100 (best)
-> PE8, LP 80 (backup)
o
PE9
RT-2: IP3,
M3, LP100
RT-2: IP3,
M3, LP80
EVPN
RT-2:MPLS
IP3,
M3, LP100
BVI401
PE8
PE7
BVI401
FW
D
FWD
ESI
SW1
G.8032, MSTP,
REP, MPLS-TP
SW2
SW3
SW4
Location of
blocked port
IP3: 4.40.1.24
M3: 003a.9c9c.3027
PE9 operates in a single-active way (BGP LP 80 for
re-originated RT2)
PE_9
RP/0/RSP0/CPU0:PE9-9906TL-H9#sho bgp l2vpn evpn rd 10.0.9.0:40
[2][0][48][003a.9c9c.3027][32][4.40.1.24]/136 detail
<…>
Paths: (2 available, best #1)
Path #1: Received by speaker 0
Local
10.0.7.0 (metric 33554428) from 10.0.3.0 (10.0.7.0), if-handle 0x00000000
Received Label 24008, Second Label 24021
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 1, version 3378
Extended community: SoO:10.0.7.0:40 EVPN MAC Mobility:0x00:1 0x060e:0000.0000.0191 RT:4:401
RT:65000:40
…
Path #2: Received by speaker 0
Local
10.0.8.0 (metric 33554428) from 10.0.3.0 (10.0.8.0), if-handle 0x00000000
Received Label 24016, Second Label 24021
Origin IGP, localpref 80, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: SoO:10.0.7.0:40 EVPN MAC Mobility:0x00:1 0x060e:0000.0000.0191 RT:4:401
RT:65000:40
…
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
66
Single-Flow-Active mode: Key features
6. ARP Probe, Speculative RT2
IP3,M3 -> PE7, LP 100 (best) PE8, LP 100 (best)
-> PE8, LP 80 (backup) PE7, LP 80 (backup)
o
PE9
RT-2: IP3,
M3, LP80,
Seq # +1
BVI401
EVPN
MPLS
Speculative
o
Speculative
RT-2: IP3, M3,
LP100, Seq # +1
RT-2: IP3, M3,
LP100, Seq # +1
PE8
PE7
FWD
BVI401
FWD
ESI
SW1
G.8032, MSTP,
REP, MPLS-TP
SW2
SW4
Port
SW3 unblocked!
IP3: 4.40.1.24
MAC3: 003a.9c9c.3027
ARP
Probe
After MAC Move is detected (on PE8) ARP Probe is generated to
populate the ARP table, and confirm new MAC/IP location
To achieve faster convergence, while waiting for the probe reply,
Speculative RT2 MAC-IP (LP100 and Seq. # +1 in MAC Mobility
Ext. Community) is generated and advertised to the network:
PE_9
RP/0/RSP0/CPU0:PE9-9906TL-H9#sho bgp l2vpn evpn rd 10.0.9.0:40
[2][0][48][003a.9c9c.3027][32][4.40.1.24]/136 detail
<…>
Path #1: Received by speaker 0
….
Local
10.0.7.0 (metric 33554428) from 10.0.3.0 (10.0.7.0), if-handle 0x00000000
Received Label 24008, Second Label 24021
Origin IGP, localpref 80, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: SoO:10.0.8.0:40 EVPN MAC Mobility:0x00:2 0x060e:0000.0000.0191 RT:4:401
RT:65000:40
…
Path #2: Received by speaker 0
…
Local
10.0.8.0 (metric 33554428) from 10.0.3.0 (10.0.8.0), if-handle 0x00000000
Received Label 24016, Second Label 24021
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 1, version 3398
Extended community: SoO:10.0.8.0:40 EVPN MAC Mobility:0x00:2 0x060e:0000.0000.0191 RT:4:401
RT:65000:40
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
67
Single-Flow-Active mode: Key features
7. MSTI flush/flooding
IP3,M3 -> PE8, LP 100 (best)
IP4,M4 -> PE7, LP 100 (best)
PE9
RT-2: IP4, M4,
RT-2: IP3, M3, Seq # +1
flow
to
EVPN
MPLS
IP4/M4
IP4, M4
Still on PE7
RT-2: IP3, M3, Seq # +1
RT-2: IP4, M4,
PE7
EVI
PE8
IP3, M3
moved to PE8
L2
flooding
SW1
SW4
SW2
Problem: After a failure, the affected hosts start to move to
the NEW Active PE using MAC/IP-mobility procedure, that is
prefix dependent – hosts move one-by-one:
As number of Hosts or prefixes increases, convergence
can be slow
SW3
Host 3: IP3/M3
Host 4: IP4/M4
Solution: Once the first MAC-IP mobility event is detected
on PE7 (RT2 with a new sequence number received), PE7
begins L2-flooding the traffic to PE8 and to AC-interface:
o Hosts that have not moved to Newly Active PE will
receive the flooded packed over the Previous Active
PE’s local AC
o Hosts that have moved but whose control plane mobility
event to Newly Active PE has not been processed yet
are flooded to the EVI port at Previous Active PE
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
68
Single-Flow-Active Mode: Configuration
PE9
EVPN
BVI401
PE7
FWD
PE8
BVI401
No DF Election
No SHG filter
FWD
G.8032, MSTP,
REP, MPLS-TP
SW4
evpn
evi 40
!
interface HundredGigE0/0/0/7
ethernet-segment
identifier type 0 07.08.00.00.00.00.00.40.00
load-balancing-mode single-flow-active
convergence
mac-mobility
!
!
timers
mac-postpone <sec>
ESI
SW1
SW2
SW3
Location of
blocked port
Defines the time for sending Speculative
RT2 while new learning/probing is
finished. By default, 5 min
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
69
VRRP/HSRP Migration
Migration from HSRP/VRRP to EVPN Distributed
L3 Anycast GW
•
•
•
•
•
#CiscoLive
BRKMPL-2143
EVPN All-Active mode
Identical Anycast Gateway
Virtual IP and MAC address
are configured on all the PEs
All the BVIs perform active
forwarding in contrast to
active/standby like First-hop
redundancy protocol
(HSRP/VRRP)
EVPN provides the ARP/ND
sync for failover convergence
Traffic is load-balanced in
both directions
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
71
Multicast
How Multicast works with EVPN A/A?
Source
•
•
PE3
CORE
PIM Join
NDF
PIM Join
EVPN RT-7/8
PE1 IGMP/MLD Sync PE2 DF
IGMP/MLD
Join
A
A
SW
•
•
EVPN MH (A/A) on PE1 and PE2
PE1&PE2 receives IGMP/MLD Join based
on SW LAG hash, creates the state and
sync it to the other using EVPN Sync Route
(route type 7/8 = Join/Leave respectively)
Both PEs send PIM-Join to upstream and
receive multicast stream
But only EVPN MH DF sends multicast
packet to receiver, Non-DF drops the
stream
IGMP/MLD
Join
Receiver
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
73
Questions?
Fill out your session surveys!
Attendees who fill out a minimum of four session
surveys and the overall event survey will get
Cisco Live-branded socks (while supplies last)!
Attendees will also earn 100 points in the
Cisco Live Game for every survey completed.
These points help you get on the leaderboard and increase your chances of winning daily and grand prizes
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
75
• Visit the Cisco Showcase
for related demos
• Book your one-on-one
Meet the Engineer meeting
• Attend the interactive education
Continue
your education
with DevNet, Capture the Flag,
and Walk-in Labs
• Visit the On-Demand Library
for more sessions at
www.CiscoLive.com/on-demand
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
76
Thank you
#CiscoLive
Gamify your Cisco Live experience!
Get points for attending this session!
How:
1 Open the Cisco Events App.
2 Click on 'Cisco Live Challenge’ in the side menu.
3 Click on View Your Badges at the top.
4 Click the + at the bottom of the screen and scan the QR code:
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
78
Backup
Step-by-step nV cluster to EVPN
Migration
ASR 9000 nV Cluster Migration: Initial State
XR 5.3.x
Access VFI_111
BD
111
CE1
PE12_PE3 UP
PE12_PE4 UP
PE1 (cluster
node)
VFI
PE3
CE3
• Cluster nodes: PE1&PE2 running
IOS XR 5.3.x
• Service: Active-Active access
model for L2 MP ELAN service
(VPLS)
MPLS
VFI PE4
• VFI instance used for VPLS service
CE4
PE2 (cluster
node)
• Full Mesh of pseudowires (PW) for Anyto-Any connectivity
XR 5.3.x
PE12 (cluster)
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
80
ASR 9000 nV Cluster Migration: De-clustering
and SW Upgrade to 6.2.x+ on PE2
XR 5.3.x
PE1 (cluster)
VFI_111
BD
111
CE1
PE12_PE3 UP
PE12_PE4 UP
VFI
PE3
CE3
• Go through the de-clustering
procedure for PE2 which is
backup node
MPLS
VFI PE4
• Recommended IOS XR image is
6.4.1 due to core isolation
feature availability
CE4
PE2 (standalone)
XR 6.2.x+
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
81
ASR 9000 nV Cluster Migration: Configure EVPN
All-Active mode on PE2
XR 5.3.0
PE1 (cluster)
VFI_111
CE1
ESI
BD
111
PE12_PE3 UP
PE12_PE4 UP
10.0.0.1
VFI
PE3
CE3
MPLS
10.0.0.2
BD
111
VFI PE4
CE4
interface Bundle-Ether34
lacp system mac 3434.3434.3434
lacp system priority 1
!
interface Bundle-Ether34.111 l2transport
encapsulation dot1q 111
rewrite ingress tag pop 1 symmetric
evpn
evi 111
advertise-mac
!
!
interface Bundle-Ether34
ethernet-segment
identifier type 0
34.34.34.34.34.34.34.34.01
!
l2vpn
bridge group VPLS
bridge-domain 111
interface Bundle-Ether34.111
!
evi 111
!
!
!
EVI_111
PE2 (standalone)
XR 6.2.x+
#CiscoLive
BRKMPL-2143
router bgp 65001
nsr
bgp router-id 10.0.0.2
bgp graceful-restart
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
address-family l2vpn evpn
!
neighbor-group PE
remote-as 65001
update-source Loopback0
address-family vpnv4 unicast
!
address-family l2vpn evpn
!
!
neighbor 10.0.0.1
use neighbor-group PE
!
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
82
ASR 9000 nV Cluster Migration: VPLS configuration
on PE2 to support non-EVPN enabled PEs
XR 5.3.0
PE1 (cluster)
VFI_111
BD
111
PE12_PE3 UP
PE12_PE4 UP
VFI
PE3
CE3
• Otherwise, after de-clustering, with AllActive mode on the access side and regular
VPLS on the core side we can get MAC flipflopping
MPLS
CE1
BD
111
X
Access VFI_111
PE2_PE3 DOWN
PE2_PE4 DOWN
• Virtual Ethernet-Segment (vES)
needed because of a Single Active
mode nature of vES vs ActiveActive nature of a regular VPLS VFI:
VFI PE4
CE4
EVI_111
PE2 (standalone)
XR 6.2.x+
#CiscoLive
• Access-VFI (instance, neighbors,
etc.) configured to communicate
with PEs running VPLS
• VPLS Pseudowires look like
traditional attachment circuits from
vES PEs perspective
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
83
ASR 9000 nV Cluster Migration: Access-VFI
Configuration
Configuration of Access VFI on PE2
XR 5.3.0
PE1 (cluster)
VFI_111
BD
111
PE1_PE3 UP
PE1_PE4 UP
VFI
PE3
CE3
MPLS
CE1
Access VFI_111
BD
111
PE2_PE3 UP
PE2_PE4 UP
VFI PE4
CE4
EVI_111
PE2 (standalone)
XR 6.2.x+
#CiscoLive
l2vpn
bridge group VPLS
bridge-domain 111
interface Bundle-Ether34.111
!
access-vfi 111
neighbor 10.0.0.3 pw-id 111
!
neighbor 10.0.0.4 pw-id 111
!
!
evpn
Access-facing Ethernet Segment
evi 111
advertise-mac
!
!
interface Bundle-Ether34
ethernet-segment
identifier type 0 34.34.34.34.34.34.34.34.01
!
!
Core-facing Ethernet Segment
virtual vfi 111
ethernet-segment
identifier type 0 34.34.34.34.34.34.34.34.02
!
!
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
84
ASR 9000 nV Cluster Migration:Enable Corefacing interfaces on PE2
XR 5.3.0
PE1 (cluster)
VFI_111
BD
111
PE1_PE3 UP
PE1_PE4 UP
PE1_PE2 UP
VFI
PE3
CE3
VFI PE4
CE4
MPLS
CE1
Access VFI_111
BD
111
PE2_PE3 UP
PE2_PE4 UP
PE2_PE1 UP
EVI_111
CE2
PE2 (standalone)
• Initially all interfaces (access- and
core-facing) are in shutdown state
• Enable core facing interfaces
• Verify LDP, ISIS, BGP adjacencies,
routing tables
• Ensure all prefixes have been
exchanged
• New PW between PE1 and PE2
can be needed to ensure
reachability for other CEs
connected to PE2
XR 6.2.x+
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
85
ASR 9000 nV Cluster Migration: De-clustering
procedure for PE1 and SW Upgrade to 6.2.2+
XR 6.2.x+
PE1 (standalone)
Go through the de-clustering
procedure for PE1
• Upgrade SW on PE1 to 6.2.2+
•
VFI
PE3
CE3
MPLS
CE1
Service is disrupted due
to all access interfaces
being disabled
Access VFI_111
BD
111
PE2_PE3 UP
PE2_PE4 UP
PE2_PE1 Down
VFI PE4
CE4
EVI_111
PE2 (standalone)
XR 6.2.x+
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
86
ASR 9000 nV Cluster Migration: Enable access
facing interface(s) on PE2
XR 6.2.x+
PE1 (standalone)
VFI
PE3
CE3
Service is restored !!!
MPLS
CE1
Access VFI_111
BD
111
PE2_PE3 UP
PE2_PE4 UP
VFI PE4
CE4
EVI_111
PE2 (standalone)
XR 6.2.x+
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
87
ASR 9000 nV Cluster Migration: Configure EVPN
All-Active mode on PE1
• PE1 & PE2 are in EVPN All-Active
XR 6.2.x+
NDF for <ES,
VLAN_111>
redundancy mode on access side
PE1 (standalone)
• No loops between CE1/PE1/PE2 due to
BD
111
VFI
PE3
EVPN Split Horizon (SH)
CE3
• PE1 sends BUM traffic from local CE to PE2
through Ingress Multicast tunnel using SH
label
EVI_111
MPLS
CE1
• PE1 sends known unicast packets to PE2
based on EVPN RT2 routes received from
PE2
Access VFI_111
BD
111
PE2_PE3 UP
PE2_PE4 UP
EVI_111
VFI PE4
CE4
• PE2 sends traffic to PE3 & PE4 via Access
VFI and VPLS PW
DF for <ES,
VLAN_111>
• PE2 doesn’t drop BUM from PE1
since vES and Access-side ES are
different Ethernet Segments (!)
PE2 (standalone)
XR 6.2.x+
#CiscoLive
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
88
ASR 9000 nV Cluster Migration: VPLS configuration
on PE1 to support non-EVPN capable PEs
XR 6.2.x+
NDF for <ES,
VLAN_111>
PE1 (standalone)
NDF for
<vES,VFI_111>
Access VFI_111
BD
111
PE1_PE3 UP
PE1_PE4 UP
VFI
PE3
EVI_111
• PE1 and PE2 discover each other and
elect DF and NDF for <vES,VFI_111>
• DF, PE2 (i.e.), sends traffic from local CEs
to remote PEs (PE3&PE4) via Access-VFI
CE3 • NDF, PE1(i.e.), blocks traffic in both
directions (BD_111 < ≠ > Access VFI_111)
MPLS
CE1
Access VFI_111
BD
111
DF for <ES,
VLAN_111>
PE2_PE3 UP
PE2_PE4 UP
EVI_111
PE2 (standalone)
XR 6.2.x+
Access-Side:
All-Active mode
VFI PE4
CE4
DF for
<vES,VFI_111>
Core-Side:
Single-Active mode
#CiscoLive
• If CE1’s hash algorithm sends BUM traffic to
PE1, PE1 uses IR-tunnel to forward traffic to
PE2
• If CE1’s hash algorithm sends any known
unicast to PE1, PE1 uses normal EVPN
forwarding to direct traffic to PE2:
• PE2 (DF) learns all MACs from remote PEs (PE3&PE4)
• PE2 sends MACs by EVPN RT2 to all EVPN-neighbors
(including PE1)
• PE1 populates MAC-VRF information based on EVPN
updates
BRKMPL-2143
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
89
#CiscoLive