Cisco ACI L3Out (Layer 3 Out) Layer 3 Outside (L3out) for Routed Connectivity to External Networks In a Cisco ACI fabric, the bridge domain is not meant for the connectivity of routing devices, and this is why you cannot configure static or dynamic routes directly on a bridge domain. You need to use a specific construct for routing configurations: the L3Out. Cisco ACI Spine Nodes Localisation : Tenant > Networking > External Routed Domains Cisco ACI Leaf Nodes A L3Out policy is used to configure interfaces, protocols, and protocol parameters necessary to provide IP connectivity to external routing devices. Part of the L3Out configuration involves also defining an external network (also known as an external EPG) for the purpose of access-list filtering. L3out APIC Cluster The external network is used to define which subnets are potentially accessible through the Layer 3 routed connection. As part of the L3Out configuration, these subnets should be defined as external networks. Alternatively, an external network could be defined as 0.0.0.0/0 to cover all possible destinations, but in case of multiple L3Outs, you should use more specific subnets in the external network definition. 0.0.0.0/0 External Networks for ACI L3out objects relationships A Layer 3 external outside network (l3extOut object) includes the routing protocol options (BGP, OSPF, EIGRP, static) and the switchspecific and interface-specific configurations. The External EPG exposes the external network to tenant EPGs through a contract. Bridge domain Tenant Access BD to L3out association EPG VRF External EPG Contract Routed connectivity to external networks is enabled by associating a fabric access external routed domain with a tenant Layer 3 external instance profile (l3extInstP or external EPG) of a Layer 3 external outside network (l3extOut), in the hierarchy in the side diagram: Layer 3 External Domain Profile BGP OSPF Vlan Pool EIGRP AAEP L3out Route control Securiy control Definitions Logical node profile Conracts Logical node profile This is the leafwide VRF routing configuration, whether it is dynamic or static routing. For example, if you have two border leaf nodes, the logical node profile consists of two leaf nodes. Logical interface profile This is the configuration of Layer 3 interfaces or SVIs on the leaf defined by the logical node profile. The interface selected by the logical interface profile must have been configured with a routed domain in the fabric access policy. This routed domain may also include VLANs if the logical interface profile defines SVIs. Node OSPF interfae profile Node BGP Peer Connectivity profile Logical interface profile EIGRP Interface profile Interface Interface External network and EPG This is the configuration object that classifies traffic from the outside into a security zone. L3out Design Gateway Resiliency (static routing) Router Router Router Some design scenarios require gateway resiliency on L3Out. For L3Outs configured with static routing, Cisco ACI provides multiple options for a resilient next hop: Secondary IP HSRP L3 out VRF This option is available on routed interfaces, subinterfaces, and SVIs, but is used mostly with SVIs. L3 out L3 out VRF VRF User Tenant User Tenant This option is available on routed interfaces and on subinterfaces (not on SVIs). It is used primarily in conjunction with an external switch. ACI Fabric One L3out object per User Tenant Common Tenant VRF VRF VRF User Tenant User Tenant User Tenant ACI Fabric ACI Fabric L3 out Leaf101 SVI .252 Static route => 192.168.1.254 Leaf102 Secondary .254 192.168.1.0/24 Author: Ben oit GON CALVES – 2020 – ACI 4.2 SVI .253 .1 One L3out object inside the Common Tenant Every user Tenant are associated to it (simplify and scale the configuration). This is called « shared services ». Example of config in page 3. Cisco ACI L3Out (Layer 3 Out) Configuration Steps Tenant Tab 1 Configure Tenant & VRF SPINE Localisation : Tenants > Add Tenant Localisation : Tenants > Networking > VRF Fabric Tab SPINE 1 2 Configure the Bridge Domain Localisation : Tenant > Networking > Bridge Domain LEAF 101 Name: Standalone.BD Clic on « Advertise Host Routes » to enable adve rtise ment to all deploye d border le af switches. VRF: attac h it to the VRF created at previous step. Subnet: 10.0.0.1/24 + « Advertise Externally » 3 LEAF 102 L3out 3 Configure the AP & EPG Localisation : Tenant > Application Profiles 4 Configure the L3out Localisation : Tenant > Networking > External Routed Networks Right click and c hoose create L3out Name: WAN-L3out VRF: Networklife External Routed Domain: WAN-L3out.RoutedDomain External En dpoint - If you need dynamic routing, tick the BGP, OSPF or EIGRP. For this example, we will configure static routing. 5 Localisation : Tenant > Networking > External Routed Networks - Inside the L3out object > Po lic y > Node Profiles, Click « + » Name : ACINodeProfile - Nodes, clic k « + », select the ID of the leaf 102 and configure the Router ID IP address - Set the static ro ute 0.0.0.0/0 with the external router IP as a ne xt-hop. 6 Don’t forget the contract. 6 Configure Interface Policies Localisation : Fabric > Acces s Policies > Policies > Interface Reuse previously created objects Configure Interface Policy Groups Localisation : Fabric > Acces s Policies > Inte rface > Leaf Interface > Policy Groups > Acces s Port Configure Interface Profiles Localisation : Fabric > Acces s Policies > Inte rface > Leaf Interface > Profiles Name: Leaf102-LeafProf - Acces s Port Selector: Eth1.01 - Acces s Bloc k Port: 1/1 - Interface Policy Group: ExternalRouter.APPG Localisation : Te nant > Networking > External Routed Networks > Logical Node Profiles > ACINodeProfile > Logical Interfac e Profiles Localisation : Tenant > Networking > External Routed Networks > Networks Localisation : Fabric > Acces s Policies > Policies > Global > Attachable Access Entity Profile ss Name: ExternalRouter.APPG Link: 1G-Auto STP: STP-BPDU-Guard-on STP: STP-BPDU-Filter-on PFC: PFC-auto LACP: LACP-ac tive AAEP: ExternalRouter.AEP Configure Logical Interface Profiles Configure External Networks (EPG) Configure AEP Name: Leaf101-LeafProf - Acces s Port Selector: Eth1.01 - Acces s Bloc k Port: 1/1 - Interface Policy Group: StandaloneServe r.APPG Name: Leaf102-IntPro f - Configure the local IP in the same subnet as the external router, you can use Routed sub-interfaces, Routed interfac es or SVI. - Choose the Po rt 1/1 previously c reated and encapulation vlan-10. 7 5 Don’t forget to attach your L3out to each BD. Configure Node Profile Localisation : Fabric > Acces s Policies > Physical and External Domains > External Domains Name: ExternalRouter.AEP Domain: WAN-L3out.RoutedDomain WAN Standalone Server Configure External Routed Domain Name: WAN-L3out.RoutedDomain Vlan Pool: L3out.VLANPo ol vlan-10 External Router Name of AP: Standalone.AP Name of EPG: Standalone.EPG BD: Standalone.BD 4 Localisation : Fabric > Acces s Policies > Pools > Vlan Name L3out.VLANPo ol Vlan: 10 Tenant: ACME VRF: Networklife 2 Configure VLAN Pool Policy Universe ACME Tenant ACCESS Name: WAN-ExtNet Subnets: 0.0.0.0/0 Standalone.AP AP Standalone.EPG EPG 8 BD Subnets Networklife VRF WAN-ExtNet L3 External Instance Profile WAN-L3out L3 Ext Outside Networks ACINodeProfile L3 External Node Profile WAN-L3out.RoutedDomain Layer 3 External Domain Profile L3out.VLANPo ol Vlan Pool Attach the BD to the L3out Localisation : Tenant > Networking > Bridge Domain - Go to the Bridge domain which need to acces s the L3out - Click on Po lic y > L3 Configuration - Into L3out, clic k « + » and add the object WAN-L3out 9 Standalone.BD Contract Create Contract and attach it to the EPGs Localisation : Tenant > Contract > Standard - Create a standard contrac t, with a filter allowing IP any. - Configure the External EPG WAN-ExtNet as Provider - Configure the vZany (EPG Collec tion for VRF) as Consumer (one application for all BDs) Author: Ben oit GON CALVES – 2020 – ACI 4.2 Leaf102-IntPro f L3 External Interface Profile ExternalRouter.AEP AAEP Cisco ACI L3Out (Layer 3 Out) Configuration Steps Shared L3out with multiple Tenants 3 validated designs are possible for « shared services »: Option 1 - BD in Common Tenant Option 2 - BD in User tenant Option 3 - Inter-VRF Leaking with Shared L3out - Shared L3 out for the fabric with static/dynamic routing in Tenant Common. - All Endpoint groups (EPGs) are configured in respective user Tenant(s) - Bridge Domains (BDs), subnets, and VRFs are all configured in the Tenant common. - Shared L3 out for the fabric with static/dynamic routing in Tenant Common. - All Endpoint groups (EPGs), Bridge Domains (BDs), and subnets are configured within the customer’s respective user Tenant(s) - The VRF is configured in the Tenant common where the L3out is configured. - Shared L3out for the fabric with static/dynamic routing in Tenant Common. - All Endpoint groups (EPGs), Bridge Domains (BDs), subnets and VRFs are configured within the customer’s respective user Tenant(s) - Only L3out is configured in the common tenant. Router Router Router L3 out VRF BD + Subnet L3 out VRF L3 out VRF BD + Subnet Common Tenant Common Tenant EPG EPG Use r Tenant User Tenant Common Tenant BD + Subnet BD + Subnet EPG EPG User Tenant ACI Fabric BD + Subnet EPG VRF User Tenant BD + Subnet User Tenant ACI Fabric EPG VRF User Tenant ACI Fabric HowTo Configure Option 3 - Inter-VRF Leaking with Shared L3out Make sure the IP subnets in user tenants do not overlap, this design requires them to be shared between VRFs. In this example, we reuse the physical topology of the page 2 (L3out on leaf 102), but the logical configuration is changing. User Tenants 1 2 static 3 1 Configure the Bridge Domain Tenant1.BD EPG ExtNet P Tenant2.BD 10.2.2.1/24 NOTE – Do not assoc iate L3out listed on the BD; when we use an Inter-vrf Shared L3out, we do not need to as sociate the user Tenant BDs with the L3out in Tenant Common. NOTE – Do not assoc iate L3out listed on the BD; when we use an Inte r-vrf Shared L3out, we do not need to as sociate the user Te nant BDs with the L3out in Te nant Common. Configure the AP & EPG 3 Localisation : Tenant > Application Profiles Configure the AP & EPG Localisation : Tenant > Application Profiles Name of AP: Standalone.AP Name of EPG: Standalone.EPG BD: Tenant2.BD Common Tenant Moving into common tenant Ct Tenant.Tn Tenant2.BD 10.1.1.1/24 10.2.2.1/24 C VRF EP EP ACI Fabric Author: Ben oit GON CALVES – 2020 – ACI 4.2 EPG EP 4 5 Tenant2.Tn Tenant1.BD EP Localisation : Tenant Tenant2.Tn > Networking > Bridge Domains > YourBD > L3 Configurations On L3 configuration, enable unic ast routing and create the subnet 10.2.2.1/24 with the following options: - Advertise Externally - to advertis e these gateway subne ts out to Share d L3Out to the internet - Shared between VRFs - To le ak the subnets to the common te nant. common Te nant EP Configure the Bridge Domain On L3 configuration, enable unic ast routing and create the subnet 10.1.1.1/24 with the following options: - Advertise Externally - to advertis e these gateway subnets out to Shared L3Out to the internet - Shared between VRFs - To leak the subnets to the common tenant. VRF default EPG Configure the Tenant Tenant2.Tn Configure the VRF Tenant2.VRF Name: Tenant2.BD 10.1.1.1/24 Name of AP: Standalone.AP Name of EPG: Standalone.EPG BD: Tenant1.BD WAN_L3out VRF 2 Localisation : Tenant Tenant1.Tn > Networking > Bridge Domains > YourBD > L3 Configurations Name: Tenant1.BD Router Vlan-10 Configure the Tenant Tenant1.Tn Configure the VRF Tenant1.VRF C EP 6 8 Configure the L3out Localisation : Tenant > Networking > External Routed Networks Configure Node Profile Localisation : Tenant > Networking > External Routed Networks Configure Logical Interface Profiles Localisation : Tenant > Networking > External Routed Networks > Logical Node Profiles > ACINodeProfile > Logical Interfac e Profiles Create Contract and attach it to the EPGs Localisation : Tenant Commo n > Contract > Standard - Create a standard contrac t, with a Global scope and a filter allowing IP any. - Configure the External EPG WAN-ExtNet as Provider P - Configure the vZany as Consumer C on Tenant1.VRF and Tenant2.VRF. Configure External Networks (EPG) 7 Localisation : Tenant > Networking > External Routed Networks > Networks Name: WAN-ExtNet Subnets: 0.0.0.0/0 EPG ExtNet Tick the following options: - External Subnets for the External EPG – allow this subnet in the external EPG - Shared Route Control Subnet – if this network is learned from the outside through this VRF, it can be leaked to the othe§I - Shared S ecurity Impo rt Subnet – sets the c lassifier for the subnets in the VRF where the routes are advertised. Shared security-import subne ts are used with shared L3Out configuration, not used for routing control. This setting configures an ACL in the VRF that is c onsuming the shared L3Out.
