Solving the Softwire Mesh
Problem
Chris Metz, chmetz@cisco.com
IETF Softwire WG Interim Meeting
Hong Kong
February 2006
1
Contents
•
•
•
•
Some Terminology
Basic Problem to Solve
Similarities with L3VPN
Solution Overview
– Encapsulations
– BGP Protocol Elements
• Examples
• References
2
Terminology (1)
AF = Address Family
• AF(i) Transit Core – single AF IPv4 or IPv6 backbone
network
• AFBR – Address Family Border Routers, dual-stack (I,j)
• AF(j) Access Islands – single AF(j) or dual-stack (i,j)
access networks connected to AFBR
3
Terminology (2): What it looks like
with IPv4 and IPv6 Plugged In …
4
So what is the problem we need to
solve?
• Support inter-AF(j) island routing and forwarding
across a single AF(i) transit core.
5
Problem to Solve? IPv4 Islands
across an IPv6 Core and …
6
… IPv6 Islands across an IPv4
Core
7
Some quick observations of what is
needed here (1)
• Multi-AF Route Distribution
– ex. so that routers in AF(j) Access Island-1(including AFBR-1)
learn about AF(j) prefixes located in other AF(j) access islands
reachable thru AFBR-2, .. AFBR-N
8
Some quick observations of what is
needed here (2)
• AF(i) Encapsulation of AF(j) Packets
– ex. AFBR-1 encapsulates AF(j) packet in AF(i) “wrapper” so that
packet can be forwarded across AF(i) core; wrapper is then
removed at egress AFBR
– also need to figure out how AFBRs agree on what “wrappers” to
use and how to correlate this with the AFBR and AF(j)
9
reachability …
So big picture at this point ..
• We have:
– requirement to distribute multi-AF routes (IPv4 or
IPv6) between AF access islands connected to a
single AF backbone network
– requirement to use that reachability information to
forward AF packets (IPv4 or IPv6) across that
backbone network from one access island network to
another
– requirement to encapsulate AF island-sourced IPv4 or
IPv6 packets for transport across AF backbone
network
• This has similarities to the classic L3VPN
problem and solution space. Let’s take a look …
10
Classic MPLS VPN (1)
MP-BGP = multi-protocol BGP
VRF = VPN Routing/Forwarding Table
• Define a new IPv4 VPN address family (VPNv4) to identify and store
customer VPN IPv4 routes inside VPN routing tables (VRFs) on PE
nodes
• Use MP-BGP to distribute VPNv4 routes, VPN labels, Next-Hop
information, etc. between PE nodes only
11
Classic MPLS VPN (2)
• Native IPv4 customer VPN packets are encapsulated in
MPLS labels for transport across the MPLS backbone
– IGP label(s) provide the label switched path (LSP) from PE-1 to
PE-2
– VPN label identifies which destination customer site to forward
IPv4 packet to
12
Classic MPLS VPN (3)
• Defined in RFC2547/RFC4364
• Many interoperable implementations and deployments
• Can even support IPv6 VPNs
– draft-ietf-l3vpn-bgp-ipv6-07.txt
• Extended for Multicast VPN
– draft-ietf-l3vpn-2547bis-mcast-01.txt
– only IPv4 at the moment
• Scalability
– Per-PE routing table: O(# of Internet Routes + # of VPN routes
for attached customers)
– per-PE peering: O(# of remote PEs + # of attached customer
routers)
– per-local PE-to-remote PE paths: O(# of remote PEs)
• Security
– Discussed in RFC4111, “Security Framework for ProviderProvisioned Virtual Private Networks (PPVPNs)”
13
Classic MPLS VPN (4)
• What happens if the backbone IS NOT
MPLS? Can we still do MPLS VPNs?
• Yes, we can nail up inter-PE IP tunnels
(e.g. GRE) and then tunnel the VPNlabeled customer packets thru or …
14
MPLS VPN over IP (1)
• Extend MP-BGP to advertise IP tunnel info along with
VPNv4 prefixes, VPN labels, etc.
– ex. now PE-1 learns of remote VPNv4 prefixes, the VPN labels,
the next_hop and an IPv4 tunnel to use to reach that next_hop
15
MPLS VPN over IP (2)
• Native IPv4 customer VPN packets encapsulated in VPN
label and IP Tunnel Header (e.g. GRE, L2TPv3, IPsec)
for transport across IP backbone
• Current deployments include:
– static GRE tunnels between PE nodes; BGP-advertised L2TPv3
16
tunnel encaps
Building the solution with some of
these pieces …
• MP-BGP
– efficient and scalable one (egress AFBR) to
many (ingress AFBR) delivery of multi-AF
reachabililty and IP tunnel information
• Standard Encapsulation Techniques
– IP/IP, GRE, L2TPv3, MPLS-in-IP, IPsec, etc.
• Interoperable L3VPN deployments
– VPNv4 over MPLS and IP
– VPNv6 over MPLS
17
One more bit of Terminology Softwire
• Defined as a logical pt-pt (or pt-mpt) tunnel established
between participating AFBR nodes
• Purpose is to transport packets of AF(j) across the AF(i)
backbone
18
Solution Overview (1)
Basic Idea
• Leverage and reuse existing L3VPN functions
and protocols where appropriate
• Identify/develop a set of Softwire encapsulations
using standard/existing encapsulations
• Extend MP-BGP to
– enable egress AFBR(s) to advertise their softwire
tunnel capabilties, encapsulation parameters and
preferences to participating ingress AFBR nodes …
thus forming the softwire mesh
– enable egress AFBR(s) to advertise AF prefixes and
associated softwire(s) to use to reach those prefixes
19
Solution Overview (2)
A Picture
20
Solution Overview (3)
• AF Access Islands can be single or dual-stack
• AFBR may support more than one softwire type
– ex. egress AFBR-2 may support GRE and L2TPv3
encaps and will tell other AFBRs about this along with
which one AFBR-2 would prefer to be used.
• No new AF/SAF needed to define IPv4 and IPv6
addresses for transport in MP-BGP
21
Solution Overview (4)
• Establishment of inter-AFBR softwires is decoupled from
the distribution of AF reachability information
– advertise softwire tunnel encapsulation and preferences once
and then many AF prefixes and which softwire tunnel to use.
– more efficient BGP packing and processing by eliminating
advertisement of duplicate softwire tunnel info for each prefix
– enables policy control on AFBR for softwire installation and
selection
• Not mandated to store AF prefixes in VRFs on AFBR
– only needed to support overlapping address requirement or if
operator prefers this configuration
22
Note on VRF and Global Tables
•
AF Island prefixes  VRFs
– MP-BGP advertises as VPN:AF with VPN label, RT, etc.
•
AF Island prefixes  Global
– MP-BGP advertises as AF
•
In either case:
– softwire tunnels setup is separate from AF island prefix distribution
– AF island prefix distribution (VPN or Global) can include softwire tunnel ID
23
Softwire Encapsulation Possibilities
(over IPv4 Transit)
• IP
– IPv6/IPv4
– IPv6/VPN label/IPv4 -
• UDP/IP
– IPv6/UDP/IPv4
• GRE
– IPv6/GRE/IPv4
– IPv6/VPN Label/GRE/IPv4
• L2TPv3
–
–
–
–
IPv6/L2TPv3/IPv4
IPv6/VPN label/L2TPv3/IPv4
IPv6/L2TPv3/IPsec/IPv4
IPv6/VPN
label/L2TPv3/IPsec/IPv4
– IPv6/L2TPv3/UDP/IPv4
• IPsec
– IPv6/IPsec/IPv4
• MPLS
– if IPv4 transit is MPLSenabled then MPLS label may
be pushed on top or replace
outer IPv4 header
24
Softwire Encapsulation Possibilities
(over IPv6 Transit)
• IPv6 only
– IPv4/IPv6
– IPv4/VPN label/IPv6
• UDP/IP only
– IPv4/UDP/IPv6
• GRE
– IPv4/GRE/IPv6
– IPv4/VPN Label/GRE/IPv6
• L2TPv3
–
–
–
–
IPv4/L2TPv3/IPv6
IPv4/VPN label/L2TPv3/IPv6
IPv4/L2TPv3/IPsec/IPv6
IPv4/VPN
label/L2TPv3/IPsec/IPv6
– IPv4/L2TPv3/UDP/IPv6
• IPsec
– IPv4/IPsec/IPv6
• MPLS
– if IPv6 transit is MPLSenabled then MPLS label may
be pushed on top or replace
outer IPv6 header
25
Quick MP-BGP Note
MP_REACH_NLRI Attribute
IPv4=1, IPv6=2
Unicast=1
Multicast=2
..
..
Tunnel SAFI=64
MPLS VPN=128
26
http://www.iana.org/numbers.html
BGP Solution Elements
1. Distribution of Softwire Tunnel
capabilities, encapsulation(s) types,
parameters and preferences
2. Distribution of AF Island Prefixes
3. Distribution of Softwire Tunnel IDs
27
BGP Solution Elements (1a)
• How does egress AFBR tell (N number of)
candidate ingress AFBR(s) what softwire tunnel
types, parameters and preferences it can
support?
• Answer: BGP Tunnel SAFI
BGP RR = BGP Route Reflector
28
BGP Solution Elements (1b)
BGP Tunnel SAFI
• MP_REACH_NLRI attribute with a SAFI=64
indicates tunnel attributes are present
– AFI=1 and SAFI=64 point to IPv4-specific parameters
– AFI=2 and SAFI=64 point to IPv6-specific parameters
• NLRI of Tunnel SAFI contains address of tunnel
end-point on AFBR
– same address can be used by many different tunnels
thus conserving address space on the AFBR that
terminates the tunnel
• draft-nalawade-kapoor-tunnel-safi-04.txt
29
BGP Solution Elements (1c)
Tunnel Encapsulation Attribute
• Also present when Tunnel SAFI=64 are one (or more) Tunnel
Encapsulation Attributes (TLVs)
– egress AFBR-2 tells the peering ingress AFBR(s) (1-N) what
parameters and preferences of specific encap types it can support
• Defined values so far:
–
–
–
–
–
–
Type 1: L2TPv3 Tunnel information
Type 2: mGRE Tunnel information
Type 3: IPSec Tunnel information
Type 4: MPLS Tunnel information
Type 5: L2TPv3 in IPSEC Tunnel information
Type 6: mGRE in IPSEC Tunnel information
• Includes a preference field that indicates the egress AFBR’s
preferred ordering of softwire encapsulations that the ingress AFBR
should consider when selecting a softwire tunnel.
• draft-nalawade-kapoor-tunnel-safi-04.txt
30
BGP Solution Elements (1d)
Tunnel SAFI + Tunn Encapsulation Attributes
10.1.2.1
10.1.2.1
• AFBR-2 is telling the other AFBRs that
– it can terminate an L2TPv3/IPv4
softwire at 10.1.2.1
31
BGP Solution Elements (1e)
After BGP Tunnel SAFI
• Softwire established to AFBR-2
– it is possible to establish more than one softwire to an
egress AFBR
32
BGP Solution Elements (2)
• Used existing MP-BGP protocols to distribute native
or VPN-specific AF Island Prefixes between AFBR
nodes
Prefix Type
Received
Into:
AF
SAF
Reference
Island IPv4
native
Global
1
1
RFC2858
Island IPv4
native
VRF
1
128
RFC4364
Island IPv6
native
Global
2
1
RFC2858
Island IPv6
native
VRF
2
128
draft-ietf-l3vpn-bgp-ipv6-07.txt
33
BGP Solution Elements (3a)
• Final piece is for egress AFBR to advertise specific
tunnel identifier that ingress AFBR(s) should use to
reach a particular destination AF island prefix
– ingress AFBR uses this to determine which tunnel to forward
packets through to reach the advertised destination
• Use BGP Connector Attribute. Defined value are:
– Type 1 = IPv4 address (for inter-as MDT case)
– Type 2 = Tunnel ID: Tunnel End-Point Address (IPv4/6 address)
– Type 3 = Tree ID: Tunnel End-Point Address (IPv4/6 address)
(for multicast case)
• draft-nalawade-l3vpn-bgp-connector-00.txt
34
BGP Solution Elements (3b)
• BGP AF island prefix advertisement includes connector
attribute that informs ingress AFBRs which softwire
tunnel to use
35
BGP Solution Elements
1. BGP Updates contains Tunnel SAFI and tunnel encapsulation TLV to announce softwire
capabilities, encapsulation parameters and preferences
2. BGP updates include AF Island Prefix and Connector Attribute that points to softwire to use.
36
Examples
1. Native IPv6 over IPv4 Core
2. VPNv6 over L2TPv3/IPv4 Core
3. VPNv4 over GRE IPv6 Core
37
Example 1a: IPv6 over GRE/IPv4
1
64
10.1.2.1
Type 2 (GRE)
99
IPv4
10.1.2.1
GRE
IPv4
GRE
38
Example 1b: IPv6 over GRE/IPv4
3FFE:1234:1111/48
none
egress AFBR
tunn ID: 10.1.2.1 (type 2)
IPv6
glbl
glbl
IPv4
IPv6
3FFE:1234:1111/48
10.1.2.1
IPv6
IPv4
GRE
none
IPv6
IPv6
39
Example 2a: VPNv6 over
L2TPv3/IPv4
1
64
10.1.2.1
Type 1 (L2TPv3)
99
IPv4
10.1.2.1
L2TPv3
IPv4
L2TPv3
40
Example 2b: VPNv6 over
L2TPv3/IPv4
RD:3FFE:1234:1111/48
55
egress AFBR
tunn ID: 10.1.2.1 (type 2)
IPv6
VRF
VRF
IPv4
IPv6
3FFE:1234:1111/48
10.1.2.1
IPv4
L2TPv3
IPv6
55
IPv6
IPv6
41
Example 3a: IPv4 over GRE/IPv6
2
64
2002:1111::1
Type 2 (GRE)
99
IPv6
2002:1111::1
GRE
IPv6
GRE
42
Example 3b: IPv4 over GRE/IPv6
200.1/20
none
egress AFBR
tunn ID: 2002:1111::1(Type 2)
IPv4
GBL
GBL
IPv6
IPv4
200.1/20
2002:1111::1
IPv6
GRE
IPv4
none
IPv4
IPv4
43
Additional Functions
• Inter-AS
– advertise softwire tunnel attributes and AF
reachability (to egress AFBR) across AS boundaries
then …
– advertise AF prefixes and connector attributes using
MP-eBGP across AS boundaries
• Two options for Multicast:
– Traditional IPv4 or IPv6 multicast tunneled across
softwire mesh
– Extend mVPNv4 model to include multicast IPv6
reachability and forwarding over inclusive and
selective P-multicast service interfaces (PMSI)
44
Summarizing Key Aspects of this
Solution (1)
• Leverages existing and deployed L3VPN protocols (e.g. MP-BGP)
and IP encapsulation techniques (e.g. GRE, L2TPv3)
• Scalability:
– Per-AFBR routing table: O(# of Internet Routes + # of AF island
prefixes of attached islands)
– per-AFBR peering: O(# of remote AFBRs + # of attached AF
island routers)
– per-local AFBR-to-remote AFBR paths: O(# of remote AFBRs)
• Security:
– RFC4111 provides framework
– Control Plane: BGP/TCP MD5, BGP/TCPoIPsec
– Data Plane: GRE keys, L2TPv3 cookie, IPsec
• Multicast:
– traditional multicast over softwire tunnels
– mVPN extensions
45
Summarizing Key Aspects of this
Solution (2)
• OAM
– can employ existing (e.g. Netflow, interface counters per softwire)
accounting mechanisms
– feasible to run tunnel “health probes” (e.g. LSP Ping/VCCV/BFD) along
with the usual ICMP ping/trace
• Multihoming
– no problem with multihoming from AF island into multiple AFBRs
announcing same AF prefix
• Multi-Softwire Support
– AFBR can announce different softwires (e.g. GRE and L2TPv3/IPsec), a
preference for one over the other and even can have specific prefixes
use different softwires if desired
• L2VPN
– pseudowires could provide the signaling and encapsulation to transport
L2-encapsulated IPv4 or IPv6 packets between AFBRs
– but there is O(N2) provisioning to consider plus O(N) of L2 interfaces per
AFBR
46
In Conclusion
• BGP-based VPNs (IPv4 and IPv6) as
deployed and in operation today form the
foundation for a softwire mesh solution
• Modest extensions
– support global and VRF tables
– agree on set of softwire encaps and add to
BGP Tunnel SAFI
– support BGP Connector Attribute
• Done 
47
Question?
• Currently Tunnel SAFI and Connector Attribute are not
Inter-domain Routing (IDR) WG documents.
Should we do the work here in Softwires or take it to
IDR?
Quick Note on this: Review of Paris and Vancouver IDR meeting
notes implies that IDR would review and bless the encodings once
some other WG (e.g. L3VPN, now softwires) figured out what
application and solution and proposes encodings
– References: http://www3.ietf.org/proceedings/05nov/index.html,
http://www3.ietf.org/proceedings/05aug/index.html
48
References
• RFCs:
– RFC2858 - Multiprotocol Extensions for BGP-4
– RFC4364 - BGP/MPLS IP Virtual Private Networks (VPNs)
– RFC4023 - Encapsulating MPLS in IP or Generic Routing Encapsulation
(GRE)
• Internet Drafts:
– draft-ietf-l3vpn-gre-ip-2547-05, Use of PE-PE GRE or IP in BGP/MPLS
IP Virtual Private Networks
– draft-ietf-l3vpn-bgp-ipv6-07.txt, BGP-MPLS IP VPN extension for IPv6
VPN
– draft-nalawade-kapoor-tunnel-safi-04.txt, Tunnel SAFI
– draft-nalawade-l3vpn-bgp-connector-00.txt, BGP Connector Attribute
– draft-townsley-l2tpv3-mpls-02.txt, Encapsulation of MPLS over Layer 2
Tunneling Protocol Version 3 (expired)
49
Backup Notes follow …
50
Notes (1)
• Advantages of this solution
– employs well-understood, deployed BGP protocol
– more efficient BGP processing/packing as AF NLRIs DO NOT
carry softwire tunnel header information; there is a decoupling of
the softwire tunnel header distribution from AF reachability
distribution
– multiple softwires can be set up between ingress and egress
AFBR pair and egress AFBR can express a preference for one
over the other; also possible to have one set of NLRIs use one
softwire and another set of NLRIs use a different softwire
– extensible to accommodate existing and future address families,
softwire tunnel encapsulation attributes, preferences, etc.
– Enables “3rd party” operation where “tunnel broker” injects BGP
Tunnel SAFI into system identifying softwire tunnel encaps, endpoints, etc.
51
Notes (2)
• Disadvantages of this solution
– might be viewed as cumbersome by some to
associate different connector attributes for each (set
of) AF NLRIs
52
Notes (3)
• Why not just advertise AF NLRI with
different AF next_hop?
– violates BGP spec which says NLRI and
next_hop must be same address family
– can’t communicate softwire tunnel encap
parameters and preferences in next_hop
– major change to BGP implementations
53
Notes (4)
• What about the Extended Communities approach?
– idea is to advertise AF NLRI reachability along with a new Extended
Community that carries IP tunnel capabilities
– therefore each egress AFBR must advertise the same tunnel information
O(# of AF updates) times. Example: if AFBR advertises 1000 BGP updates
for prefixes in that AF, then same IP tunnel information is advertised 1000
times. This is 999 more times than is necessary.
– Extended community only defines a bit-mask indicating the type of tunnel
supported. No means exists to define a set of tunnels, the encapsulation
parameters of the tunnels in the set or, the order of preference of tunnels
in the set.
– also assumes that IP tunnel end-point is the same as the BGP next_hop.
True when using MPLS LSP but perhaps not true when using IP tunnels.
In fact IPsec will likely use an IP address that is completely different from
BGP next_hop. Therefore IPSec protection will clearly require special
tunnel capability advertisements that identify the IPSec tunnel end-points
which Extended Communities does not support
54
– References: draft-raggarwa-l3vpn-tunnel-attribute-00.txt