#CiscoLive
Mastering ACI for CCNP-DC
Certification Candidates
Joe Rinehart, Data Center Practice Lead, CCIE #14256
@jjrinehart
DGTL-BRKCRT-2000
#CiscoLive
Agenda
•
Introduction
•
ACI Fabric Infrastructure
•
ACI Packet Forwarding
•
External Network Connectivity
•
Integrations
•
ACI Management
•
ACI Anywhere
•
Conclusion
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
3
Introduction
Introduction
•
Cisco ACI can feel confusing at first, because it seems to bend or
break the rules of traditional networking
•
The goal of this session is to equip you to better understand the
principles and operation of Cisco ACI
•
Session topics are mapped directly to the DCACI 300-320 exam
blueprint
•
Not every topic can be fully covered (time constraints)
•
No exam answers will be provided
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
5
Overview of VXLAN
Review of Classic Switch MAC Learning
WS-1
wants to
transmit
data to
WS-3 on
VLAN 23
1/48
1/48
SW-1
1/1
SW-2
1/2
VLAN 14
1/3
1/4
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
6
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
MAC
Port
0011.0011.0011
e1/1
SW-1 Records the
Source MAC and
Floods the frame.
1/48
Port
0011.0011.0011
e1/48
SW-2
1/4
FLOOD
1/3
FLOOD
1/2
VLAN 14
MAC
1/48
FLOOD
SW-1
1/1
SW-2 Records the
Source MAC and
Floods the frame.
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
7
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
MAC
Port
0011.0011.0011
e1/1
1/48
As WS-3 sends traffic
back the switches
record the source
MACs of the traffic
1/2
VLAN 14
Port
0011.0011.0011
e1/48
1/48
SW-1
1/1
MAC
SW-2
1/3
Return Traffic
1/4
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
8
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
e1/48
1/48
Once complete all the
switches have a
complete Layer 2
reachability table
1/48
SW-1
1/1
Port
e1/48
0044.0044.0044
e1/4
SW-2
1/2
VLAN 14
MAC
0011.0011.0011
1/3
1/4
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
9
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
e1/48
12-Bit 802.1Q Tag
1/48
1/48
SW-1
1/1
MAC
0011.0011.0011
Port
e1/48
0044.0044.0044
e1/4
SW-2
1/2
1/3
1/4
Frames are tagged
with 802.1Q values to
differentiate traffic
VLAN 14
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
10
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
DMAC SMAC Etype
1/48
Payload
CRC
1/48
SW-1
1/1
SW-2
1/2
1/3
1/4
VLAN 14
VLAN 14
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
11
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
802.1Q
DMAC SMAC 802.1Q
Etype
1/48
Payload
CRC
1/48
SW-1
1/1
SW-2
1/2
1/3
1/4
VLAN 14
VLAN 14
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
12
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
DMAC SMAC Etype 802.1Q
1/48
SW-1
1/1
1/2
Payload
CRC
1/48
SW-2
The tag is inserted by
the switch at the
ingress port and
removed by the egress
port
1/3
1/4
VLAN 14
VLAN 14
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
13
Overview of VXLAN
Review of Classic Switch MAC Learning (Cont.)
SVI VLAN 14
23.23.23.254
SVI VLAN 23
23.23.23.254
1/48
1/48
SW-1
1/1
1/2
VLAN 14
WS-1
SW-2
WS-2
1/3
Sending traffic to
separate VLANS
requires a Layer
3 device (interVLAN routing)
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
VLAN 14
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
1/4
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
14
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions
1/48
1/48
SW-1
1/1
SW-2
1/2
VLAN 14
1/3
1/4
VLAN 14
VLAN 23
WS-1
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
15
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
VXLAN typically utilizes
a spine-leaf
architecture in place of
a traditional design
SW-3
SW-1
1/1
SW-2
1/2
VLAN 14
1/3
VLAN 23
WS-1
WS-2
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
The direct switchto-switch
trunk connection
is usually
removed
#CiscoLive
1/4
VLAN 14
VLAN 23
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
16
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
These tunnels run over an
IP-based network in much
the same way as GRE does
SW-3
VTEP Tunnel
SW-1
1/1
SW-2
1/2
VLAN 14
1/3
VLAN 23
WS-1
VXLAN uses these
tunnels to transport
native Layer 2
frames
WS-2
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
VXLAN replaces
direct trunked
connections with
a tunneled
connection
#CiscoLive
1/4
VLAN 14
VLAN 23
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
17
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
With VXLAN tunnels the
endpoints of the tunnels are
always loopback interfaces.
feature tunnel
interface tunnel 0
tunnel source loopback0
tunnel destination 1.1.1.92
ip address 10.12.12.1/24
SW-3
/24
3.0
1
.
68
192
.1
192
1/2
VLAN 14
VLAN 23
WS-1
.23
.0/2
4
VTEP Tunnel
SW-1
1/1
.16
8
WS-2
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
Using the
example of GRE
tunneling you
have to provide
a source and
destination pair
for the tunnel
endpoints
#CiscoLive
SW-2
1/3
1/4
VLAN 14
VLAN 23
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
18
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
SW-3
/24
3.0
1
.
68
192
.1
192
4
SW-2
1/2
VLAN 14
1/3
VLAN 23
WS-1
.23
.0/2
VTEP Tunnel
SW-1
1/1
.16
8
WS-2
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
In order to learn
the IP addresses
of the loopback
interfaces a
Layer 3 routing
protocol is
required.
#CiscoLive
1/4
VLAN 14
VLAN 23
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
19
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
This IP network is
referred to as the
Underlay
Supported
protocols
include BGP,
IS-IS and
OSPF
SW-3
/24
3.0
1
.
68
192
.1
192
.23
.0/2
4
VTEP Tunnel
SW-1
1/1
.16
8
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
WS-1
wants to
transmit
data to
WS-3 on
VLAN 23
SW-3
/24
3.0
1
.
68
192
.1
192
.23
.0/2
4
VTEP Tunnel
SW-1
1/1
.16
8
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
21
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
Port
0011.0011.0011
e1/1
SW-1 Records the
Source MAC and
Floods the frame.
SW-3
4
0/2
13.
68.
.1
192
.23
.0/2
4
SW-2
1/3
FLOOD
1/2
VLAN 14
1/4
UNDERLAY
VLAN 23
WS-1
.16
8
VTEP Tunnel
SW-1
1/1
192
Multicast
Group
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
22
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
Port
0011.0011.0011
e1/1
SW-3
/24
3.0
1
.
68
.1
192
.23
.0/2
4
SW-2
1/3
FLOOD
1/2
VLAN 14
1/4
UNDERLAY
VLAN 23
WS-1
.16
8
VTEP Tunnel
SW-1
1/1
192
Multicast
Group
WS-2
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
SW-1 transmits to VLAN 23
the multicast group
WS-3
which sends it to
MAC: 0033.0033.0033
other switches
IP:
#CiscoLive
DGTL-BRKCRT-2000
23.23.23.3/24
VLAN 14
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
23
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
Port
0011.0011.0011
e1/1
SW-3
/24
3.0
1
.
68
.1
192
.16
8
.23
.0/2
4
VTEP Tunnel
SW-1
SW-2
1/2
1/3
UNDERLAY
VLAN 14
VLAN 23
WS-1
1/4
FLOOD
1/1
192
Multicast
Group
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
24
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
Port
0011.0011.0011
e1/1
SW-3
/24
3.0
1
.
68
.1
192
.16
8
.23
.0/2
4
VTEP Tunnel
SW-1
SW-2
1/2
1/3
UNDERLAY
VLAN 14
VLAN 23
WS-1
1/4
FLOOD
1/1
192
Multicast
Group
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
25
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
13
68.
.1
192
Multicast
Group
.16
8
MAC
0011.0011.0011
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
VTEP Tunnel
SW-1
1/1
Once complete all the
switches have a
SW-3
complete
Layer 2
4
reachability
table 192
.0/2
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
26
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
SW-3
/24
3.0
1
.
68
.1
192
SW-1
1/1
Multicast
Group
VTEP Tunnel
1/2
192
.16
8
MAC
0011.0011.0011
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
VNID 10011
VNID 10014
4
SW-2
1/3
1/4
UNDERLAY
VLAN 14
are encapsulated inVLAN
a
VLAN Frames
23
23
VXLAN header, using Virtual
WS-1
WS-2
WS-3
Network IDs of 24 bits to
differentiate traffic
MAC: 0011.0011.0011 MAC: 0022.0022.0022
MAC: 0033.0033.0033
IP:
14.14.14.1/24
IP:
23.23.23.2/24
IP:
#CiscoLive
DGTL-BRKCRT-2000
23.23.23.3/24
VLAN 14
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
27
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
DMAC SMAC Etype
Payload
CRC
SW-3
/24
3.0
1
.
68
.1
192
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
VXLAN also starts with
an Ethernet Frame
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
28
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
Eth
Header
FCS
SW-3
/24
3.0
1
.
68
.1
192
Payload
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
VXLAN also starts with
an Ethernet Frame
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
29
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
VXLAN
Eth
Header
FCS
SW-3
/24
3.0
1
.
68
.1
192
Payload
192
Multicast
Group
.16
8
0044.0044.0044
e1/4
4
SW-2
The VXLAN 1/1
header 1/2
is
attached, carrying VNI
information
1/3
1/4
UNDERLAY
VLAN 23
WS-1
Port
nve1
.23
.0/2
SW-1
VLAN 14
MAC
0011.0011.0011
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
30
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
UDP
VXLAN
Eth
Header
FCS
SW-3
/24
3.0
1
.
68
.1
192
Payload
192
Multicast
Group
.16
8
1/2
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/3
This gets wrapped in
UDP
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
31
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
Port
Outer
e1/1 UDP
IP
0044.0044.0044
nve1
MAC
0011.0011.0011
VXLAN
Eth
```
Header
FCS
SW-3
/24
3.0
1
.
68
.1
192
Payload
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
And then wrapped inVLAN 23
IP (tunnel header)
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
32
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
Port
Outer
Outer
0011.0011.0011
e1/1 UDP
MAC
IP
0044.0044.0044
nve1
Eth
VXLAN ``` ```
Header
FCS
SW-3
/24
3.0
1
.
68
.1
192
Payload
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
UNDERLAY
And
VLAN
14 finally
encapsulated in a
Ethernet WS-1
frame
VLAN 23
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
33
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
0044.0044.0044
Outer Outer UDP
MAC
IP
nve1
Port
e1/1
SW-1
1/1
1/2
4
0/2
13.
68.
.1
192
Eth
VXLAN ``` ```
Header
SW-3
Payload
192
Multicast
Group
.16
8
FCS0011.0011.0011
MAC
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-2
The tunnel headers are
discarded upon
reaching the egress
switch
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
34
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
Outer UDP
IP
SW-1
1/1
1/2
4
0/2
13.
68.
.1
192
Eth
```
Header
SW-3
VXLAN
Payload
192
Multicast
Group
.16
8
FCS0011.0011.0011
MAC
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-2
The tunnel headers are
discarded upon
reaching the egress
switch
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
35
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
Eth
Header
SW-3
VXLAN
4
0/2
13.
68.
.1
192
SW-1
1/1
1/2
Payload
192
Multicast
Group
.16
8
FCS0011.0011.0011
MAC
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-2
The VXLAN VNID
is removed and
mapped back to a
VLAN
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
36
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
Traffic within the same
Layer 2 domain (VLAN
to VXLAN to VLAN) is
SW-3
called VXLAN
bridging
4
0/2
13.
68.
.1
192
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
37
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
Traffic between different
Layer 2 domains (the
VXLAN version of interVLAN routing)
SW-3is called
VXLAN
routing
24
0/
13.
68.
.1
192
192
Multicast
Group
.16
8
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
1/2
1/3
1/4
UNDERLAY
VLAN 14
VLAN 23
WS-1
VLAN 14
VLAN 23
WS-2
WS-3
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
38
Overview of VXLAN
How VXLAN Mirrors Classical Ethernet Functions (Cont.)
L2VPN EVPN
Address Family
MAC
0011.0011.0011
Port
e1/1
0044.0044.0044
nve1
BG
P
SW-3
4
0/2
13.
68.
192
.1
192
BG
P
.16
8
1/2
VLAN 14
WS-1
WS-2
Port
nve1
0044.0044.0044
e1/4
.23
.0/2
4
SW-1
1/1
MAC
0011.0011.0011
SW-2
MP-BGP takes host
MAC/IP information and
sends it to the spine (route
UNDERLAY
reflector)
that advertises it
VLAN 23 to the other leaf switches
VLAN 23
(reduces broadcasts)
MAC: 0011.0011.0011 MAC: 0022.0022.0022
IP:
14.14.14.1/24
IP:
23.23.23.2/24
1/3
VLAN 14
WS-3
MAC: 0033.0033.0033
IP:
23.23.23.3/24
#CiscoLive
DGTL-BRKCRT-2000
1/4
WS-4
MAC: 0044.0044.0044
IP:
14.14.14.4/24
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
ACI Fabric
Infrastructure
ACI Fabric Infrastructure
Primary Components
The various elements of Cisco ACI fall into three broad categories:
•
Fabric – Collection of Cisco Nexus 9000 series switches
composing the physical infrastructure
•
Controllers – Orchestration appliances that configure and maintain
the environment
•
Policies – A logical construct used to create or modify any
managed object* in the Cisco ACI Fabric
*An abstract representation of network resources that are managed. In Cisco ACI, an abstraction of a Cisco ACI fabric resource.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
41
ACI Fabric Infrastructure
ACI Topology and Hardware
Spine
switch
Spine-Leaf Topology
Required
Leaf
switch
• Spine connects to leaf
• Two spines required at a
minimum
• High speed connections
between spine and leaf
• Endpoints connect to leaf
• Connection between leaf
switches not permitted
• APICs connect to leaves
• NO STP inside the fabric
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
42
ACI Fabric Infrastructure
ACI Topology and Hardware
Why a Spine-Leaf Topology?
• Easier to scale out/add
capacity
• All switches are the same
number of hops away
(ECMP)
• Handles North/South and
East/West traffic flows
• Better resiliency on link
failure
• Higher bandwidth, lower
latency over switch
infrastructure
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
43
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco ACI Fabric
Configuration:
• Cisco ACI uses IS-IS as the
underlay routing protocol
UNDERLAY
• All internal fabric operation
IS-IS
is initiated and operated by
the APICs
• Links between switches use
IP unnumbered using
private loopback addresses
• Leaf switches use VTEPs
for transport like VXLAN
• Only IS-IS L1 routes are
used
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
44
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco Nexus 9000 Series
• Cisco Nexus 9500 Series
modular chassis (spine
only)
•
4, 8 and 16 slot models
•
Various Line cards available
• Cisco Nexus 9300 Series
Top of Rack switches
•
Fixed configuration
•
Most are leaf switches
(specific chip set)
• Cisco 9300 spine only
switches (specific chip set)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
45
ACI Fabric Infrastructure
ACI Topology and Hardware
Scalable 1 GE / 10 Gbps / 40 Gbps / 100
GE Performance
Cisco Nexus 9000 Series
Cisco Nexus
9300 Series
Aggregation Line Card
36 100G QSFP28
48 x 1/10/25Gbps
SFP28 + 6 x 40/100
Gbps QSFP28
Cisco Nexus
9500 Series
40G QSFP+
36-Port Line Card
48 x 100/1000 Gbps
4 x 1/10/25 Gbps
SFP28
2 x 40/100 Gbps
QSFP28
100G ACI Spine
36 Port Line card
C9500 8-Slot
Flexible form factors can enable variable data center design and scaling.
Performance
Ports
Price
#CiscoLive
Power
DGTL-BRKCRT-2000
Programmability
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
46
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco APIC Physical Architecture
• Based on the Cisco UCS-C
server platform
• Comes in a medium (APIC-M)
and large (APIC-L)
Serial
Port
• Hardware interfaces:
•
1 GB CIMC (ILO) port
•
1 GB OOB management port
•
2x10 GB fabric ports
•
1 serial port
10 GB Ports (connect
to leaf switches)
1 GB
CIMC
Port
• KVM console available for
console-like connectivity
#CiscoLive
DGTL-BRKCRT-2000
1 GB OOB
Management
Port
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
47
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco APIC Functionality
• Deployed in a cluster of at
least 3 APIC appliances (for
fault tolerance and data
replication)
• ALL configuration of the fabric
is done on the APIC
• APIC holds all of the policies
(intended state) of the fabric
• Not in the data path or control
plane like a supervisor module
• Performs management
functions
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
48
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco APIC Accessibility
• GUI: Most frequently used
configuration method (web
access)
• CLI: NX-OS like, some of the
naming conventions differ from
the GUI (console or SSH access)
• API: Used for orchestration and
automation, utilizes REST with
XML or JSON (API access)
• Each is accessible using the OOB
management address created
during initial configuration
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
49
ACI Fabric Infrastructure
ACI Topology and Hardware
Cisco APIC Clustering Provides:
• Scalability (using sharding,
covered later)
• Load-balance data and
processing
• Can expand cluster if load
increases
• Fault tolerance (using replication)
• Fully available for any process or
APIC crash or DB corruption
• Allow replacement of any APIC
APIC Cluster
Distributed, Synchronized, Replicated
• No single point of failure
• Nondisruptive upgrades
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
50
ACI Fabric Infrastructure
ACI Topology and Hardware
APIC
Node
APIC
Node
ACI Fabric
APIC
Node
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
ACI Fabric Infrastructure
ACI Topology and Hardware
Observer
Boot
APIC
Node
Topology
APIC
Node
Policy
ACI Fabric
APIC
Node
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
ACI Fabric Infrastructure
ACI Topology and Hardware
Observer
Boot
APIC
Node
Topology
APIC
Node
Policy
Each APIC node has
all APIC functions;
however,
processing is evenly
distributed.
ACI Fabric
APIC
Node
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
53
ACI Fabric Infrastructure
ACI Topology and Hardware
shard APIC
APIC
Node
Node
Boot
A shard is a unit of data
management.
•
Data is placed into shards.
•
Each shard has three replicas.
•
Shards are evenly distributed.
ACI Fabric
APIC
Node
shard
shard
shard
shard
Observer
shard
Topology
shard
Policy
shard
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
54
ACI Fabric Infrastructure
ACI Topology and Hardware
shard APIC
APIC
Node
Node
• Shard data assignments are
based on predetermined hash
function.
• Static shard layout determines
the assignment of shards to
appliances.
ACI Fabric
APIC
Node
Boot
shard
shard
shard
shard
Observer
shard
Topology
shard
Policy
shard
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
55
ACI Fabric Infrastructure
ACI Topology and Hardware
shard APIC
APIC
Node
Node
APIC
Node
Boot
shard
shard
shard
shard
Observer
shard
Topology
shard
Policy
shard
3-31 Node Cluster
Cisco APIC Cluster Sharding
#CiscoLive
• Each replica in the shard has
use preference (1..3).
• Writes happen to the highest
preference reachable (leader).
• In the case of split-brain,
automatic reconciliation is
performed.
DGTL-BRKCRT-2000
ACI Fabric
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
56
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Health Scores:
• An integer indicating the relative
health of any object in the system
The health score is a
number between 0
and 100.
Health Score
Health
Scoring Policy
• Values range from 0 (worst
possible) to 100 (best possible)
• Health scores for child objects
affect those above it (port, card,
switch, fabric, etc.)
State
∑
∑
Weighted
Fault
Alert
• All health scores aggregate to the
Fault
Alert
Drops
system health score visible on the
main dashboard shown at login
• Values can be viewed on graphs
and used to determine cause
Health Score
of Depended
Objects
Fault
Alert Remaining
Capacity
Fault
Alert
Latency
Observed Object
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
57
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Health Scores:
• An integer indicating the relative
health of any object in the system
• Values range from 0 (worst
possible) to 100 (best possible)
• Health scores for child objects
affect those above it (port, card,
switch, fabric, etc.)
• All health scores aggregate to the
system health score visible on the
main dashboard shown at login
• Values can be viewed on graphs
and used to determine cause
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
58
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Atomic Counters:
• Troubleshooting tool to count
packets and bytes between a
source and a destination
• Only packets that traverse the
fabric are counted
• Locally switched packets are not
counted
• Packets switched in the
hypervisors are not counted
• There are two types of counters:
“ongoing” and “on demand”
counters
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
59
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Atomic Counters:
• Ongoing atomic counters:
• Not user-configurable
• Packets are counted at the
infrastructure level: the source
and destination of the flow are
Tunnel End Points (TEPs)
• Paths are unidirectional
• On Demand Atomic Counters
• Configured at the tenant to
troubleshoot issues at the level of
individual applications
• Source and destination can be
EPs, EPGs, IP addresses or “Any”
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
60
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Fault Logs:
• Four types of fault triggers:
• Specific conditions
described in the model by
fault rules
• Counters crossing
thresholds specified in
user-programmable
policies (thresholdcrossing alerts)
• Task or FSM failures
• Object resolution failures
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
61
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Fault Logs:
• Faults are raised and
managed on the node
(switch or controller) where
the condition is detected.
• Faults are raised and
cleared automatically by the
system.
• A user cannot define new
faults.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
62
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Event Logs:
• An event something that
occurs at a certain point in
time (for example, “link
went from down to up”).
• Events are represented in
the system as managed
objects of class
event:Record.
• May not require user
attention.
• Events are useful for
monitoring and debugging
issues.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
63
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Event Logs:
• Events are similar to an
entry in a log file. After an
event is created, the event
is never modified.
• An event is deleted only
when the maximum number
specified in the retention
policy is reached.
• Events are triggered by
event rules, which are
defined by developers.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
64
ACI Fabric Infrastructure
ACI Fault, Event and Audit Logs
Audit Logs:
• Used to track user-initiated
configuration changes.
• When a user creates,
modifies, or deletes a
managed object, an audit
record is created that
contains the affected
managed object DN,
username, timestamp, and
change details.
• The system also creates
logs for login or logout to
controllers and nodes.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
65
ACI Fabric Infrastructure
ACI Object Model
Overview:
Root
• Everything is an object.
• Objects are hierarchically organized.
• Contains a modeled representation
Policy
Universe
Infra
Fabric
Virtual
Network
Tenants
VLANs
Nodes
Hypervisors
of:
• Application
• Network
• Services
• Virtualization
• Management
Applications
• Objects may have parents, children,
inheritance, relationships
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
66
ACI Fabric Infrastructure
ACI Object Model
Managed Object References:
Root
• Relative Name:
• Identify object related to “siblings”
• Unique within a parent objectClass
identifies object type (Card, Port,
Path, EPG, etc.)
• Distinguished Name:
Policy
Universe
Infra
Fabric
Virtual
Network
Tenants
VLANs
Nodes
Hypervisors
• Assigned to every object
• Globally unique name
• Identifies place in hierarchy
Applications
• Syntax example:
Managed Object
• class
• DN
•…
apic2://api/node/mo/uni/tn-infra.xml
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
67
ACI Fabric Infrastructure
ACI Fabric Discovery
Requirements for Cisco ACI
Fabric Initialization :
• Correctly cable the
switches
UNDERLAY
• Spine to leaf
IS-IS
• APIC to leaf
• Run through the initial setup
script on the APICs using
either:
• Console connection
• CIMC KVM console
• Ensure that the same
software version is running
on the devices
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
68
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Initialization (APIC setup script, partial parameter list):
•
Configure the APIC cluster itself (all controllers)
•
•
•
•
•
•
•
•
Fabric name (default is ACI Fabric1)
Fabric ID (default is 1)
Number of controllers [1..9] (default is 3)
Pod ID (default is 1)
TEP address pool (default is 10.0.0.1/16)
Infra VLAN ID (default is 4)
GIPO address pool (default is 225.0.0.0/15)
OOB Management Default Gateway
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
69
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Initialization (APIC setup script, partial parameter list):
•
Configure the APIC cluster itself (all controllers)
•
•
•
•
•
•
•
•
Fabric name (default is ACI Fabric1)
Fabric ID (default is 1)
Number of controllers [1..9] (default is 3)
Pod ID (default is 1)
MUST
match on
all APICs
TEP address pool (default is 10.0.0.1/16)
Infra VLAN ID (default is 4)
GIPO address pool (default is 225.0.0.0/15)
OOB Management Default Gateway
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
70
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Initialization (APIC setup script, partial parameter list):
•
Configure the APIC cluster itself (all controllers)
•
•
Controller ID
OOB Management IP Address
MUST be
unique on
each APIC
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
71
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
72
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
• First leaf switch discovers
the primary APIC using
LLDP
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
73
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
• First leaf switch discovers
the primary APIC using
LLDP
• Leaf switch sends a DHCP
request to the APIC
• APIC responds with IP
address assignment and
switch download its
configuration
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
74
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
• First leaf switch discovers
the primary APIC using
LLDP
• Leaf switch sends a DHCP
request to the APIC
• APIC responds with IP
address assignment and
switch download its
configuration
• Spine switch discovers
attached leaf switch
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
75
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
• First leaf switch discovers
the primary APIC using
LLDP
• Leaf switch sends a DHCP
request to the APIC
• APIC responds with IP
address assignment and
switch download its
configuration
• Spine switch discovers
attached leaf switch
• Process continues until
fabric is complete
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
76
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
• First leaf switch discovers
the primary APIC using
LLDP
• Leaf switch sends a DHCP
request to the APIC
• APIC responds with IP
address assignment and
switch download its
configuration
• Spine switch discovers
attached leaf switch
• Process continues until
fabric is complete
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
77
ACI Fabric Infrastructure
ACI Fabric Discovery
Fabric Discovery Process
APIC
Cluster
can now
form
• First leaf switch discovers
the primary APIC using
LLDP
• Leaf switch sends a DHCP
request to the APIC
• APIC responds with IP
address assignment and
switch download its
configuration
• Spine switch discovers
attached leaf switch
• Process continues until
fabric is complete
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
78
ACI Fabric Infrastructure
Implement ACI Access Policies
Cisco ACI Policy Fundamentals
• Generally speaking, a policy is
guideline or principle of action
• In Cisco ACI, a policy is the
desired state of the elements
in the network
• Some examples of ACI
policies:
•
Port speed/state/duplex
•
IP addressing
•
Usernames/passwords
• Certain policies in Cisco ACI
are foundational to everything
else
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
79
ACI Fabric Infrastructure
Implement ACI Access Policies
Traditional CLI Port Configuration:
• Open a terminal services
program such as PuTTY
• Manually choose the specific
device to configure
• Access the command line
interface
• Begin configuration using the
configure terminal command
• Enter commands manually to
produce a specific result
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
80
ACI Fabric Infrastructure
Implement ACI Access Policies:
Traditional CLI Port Configuration:
• Create a VLAN using the vlan
command
• Enter interface configuration
mode with the command
interface ethernet 1/1
• Set the Layer 2 configuration for
Nexus-9000-1(config)#vlan 600
Nexus-9000-1(config)#interface e1/1
Nexus-9000-1(config-if)#switchport
Nexus-9000-1(config-if)#switchport mode access
Nexus-9000-1(config-if)#switchport access vlan 600
Nexus-9000-1(config-if)#speed 1000
Nexus-9000-1(config-if)# cdp enable
Nexus-9000-1(config-if)# lldp transmit
Nexus-9000-1(config-if)# lldp receive
the port with these commands:
•
switchport
•
switchport mode access
•
switchport access vlan 600
• Set additional commands on the
ports as needed (speed, duplex,
etc.)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
81
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
• Create a VLAN Pool identifying VLANs
you might use on an interface
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
82
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
• Create a Physical Domain(specifies
that a physical port will be used)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
83
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
AAEP
• Create a Physical Domain(specifies
that a physical port will be used)
• Create and Attachable Access Entity
Profile (AAEP), joins multiple domains
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
84
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
AAEP
• Create a Physical Domain(specifies
that a physical port will be used)
Access Port
Policy Group
• Create and Attachable Access Entity
Profile (AAEP), joins multiple domains
• Create an Access Port Policy Group,
specifying physical interface settings
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
85
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
AAEP
• Create a Physical Domain(specifies
that a physical port will be used)
• Create and Attachable Access Entity
Access Port
Policy Group
• Create an Access Port Policy Group,
Interface
Profile/Selector
Profile (AAEP), joins multiple domains
specifying physical interface settings
• Create an Interface Profile and Access
Port Selector to specify the interface
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
86
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
AAEP
• Create a Physical Domain(specifies
that a physical port will be used)
• Create and Attachable Access Entity
Access Port
Policy Group
• Create an Access Port Policy Group,
Interface
Profile/Selector
Profile (AAEP), joins multiple domains
specifying physical interface settings
• Create an Interface Profile and Access
Switch
Profile
Port Selector to specify the interface
• Create or specify a switch profile to
identify which switch to use
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
87
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
• Create a Physical Domain(specifies
that a physical port will be used)
Step Shown
Earlier Under
EPG
Configuration
AAEP
• Create and Attachable Access Entity
Access Port
Policy Group
• Create an Access Port Policy Group,
Interface
Profile/Selector
Profile (AAEP), joins multiple domains
specifying physical interface settings
• Create an Interface Profile and Access
Switch
Profile
Port Selector to specify the interface
• Create or specify a switch profile to
Map Port to EPG
identify which switch to use
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
88
ACI Fabric Infrastructure
Implement ACI Access Policies:
VLAN Pool
Cisco ACI Fabric Access Configuration:
ssh
192.168.254.101
Physical
Domain
• Create a VLAN Pool identifying VLANs
you might use on an interface
config t
vlan 600
AAEP
• Create a Physical Domain(specifies
that a physical port will be used)
• Create and Attachable Access Entity
Access Port
Policy Group
• Create an Access Port Policy Group,
Interface
Profile/Selector
Profile (AAEP), joins multiple domains
specifying physical interface settings
• Create an Interface Profile and Access
Switch
Profile
Port Selector to specify the interface
• Create or specify a switch profile to
Map Port to EPG
identify which switch to use
#CiscoLive
DGTL-BRKCRT-2000
int e1/1
switchport mode access
switchport access vlan 600
speed 1000
cdp enable
lldp transmit
lldp receive
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
89
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Peer-Link
Traditional
NX-OS
vPC
Peer-Keepalive
vPC Member
Links
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
90
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Peer-Link
Cisco ACI
vPC
Peer-Keepalive
vPC Member
Links
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
91
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Cisco ACI vPC Configuration:
• Create a vPC Explicit Protection
Group
• Navigate to Fabric à Access Policies
à Policies à Switch à Virtual PortChannel Default
• Right-click and choose Create VPC
Explicit Protection Group
• Enter the following parameters:
• Name (alphanumeric)
• ID (Domain ID, 1 to 1000)
• VPC Domain Policy (default/configure one)
• Switch 1 (choose switch)
• Switch 2 (choose switch)
• Click Submit
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
92
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Cisco ACI vPC Configuration:
• Create a vPC Interface Policy Group
• Navigate to Fabric à Access Policies
à Interfaces à Leaf Interfaces à
Policy Groups
• Right-click and choose Create VPC
Interface Policy Group
• Choose or create these policies:
• Name (alphanumeric)
• Link Level Policy
• CDP Policy
• Port-Channel Policy (LACP settings)
• Attached Entity Profile
• Click Submit
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
93
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Cisco ACI vPC Configuration:
• Create a vPC Interface Profile
• Navigate to Fabric à Access Policies
à Interfaces à Leaf Interfaces à
Profiles
• Right-click and choose Create Leaf
Interface Profile
• Enter these parameters:
• Name (alphanumeric)
• Interface Selectors (access port selectors,
click the +):
• Name (alphanumeric)
• Interface IDs (slot/port)
• Interface Policy Group (see last step)
• Click Ok and then Submit
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
94
ACI Fabric Infrastructure
Implement ACI vPC Policies:
Cisco ACI vPC Configuration:
• Create or Select a Switch Profile
• Navigate to Fabric à Access Policies
à Switches à Leaf Switches à
Profiles
• Right-click and choose Create Leaf
Profile
• Enter these parameters:
• Name (alphanumeric)
• Leaf Selectors (switches in the switch pair):
• Interface Selector Profiles (see last step)
• Click Finish
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
95
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
Cisco ACI Policy Building Blocks
• Tenants:
•
Definition: A tenant is a logical
container that is isolated from all
other tenants by default
•
Usage: You can use a tenant for:
• Customers
• Business units
• Groups
•
Three default tenants:
• common: Shared services
• infra: Internal fabric operations
• mgmt: Management access
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
96
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a Tenant:
• Click Add Tenant on the Cisco
APIC GUI
• Specify the name of the tenant,
as follows:
•
Must be globally unique
•
Does not contain spaces
•
Certain special characters not
permitted
•
May use leading numbers
(including zero)
• May configure optional items
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
97
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
Cisco ACI Policy Building Blocks
• VRFs:
•
Definition: A VRF is a Layer 3
routing instance within a tenant
•
Usage: You can use a VRF for:
• Separate IP tables
• Separate IP addresses
•
You must have at least one per
tenant but can have more
•
Also called (older terms):
• Contexts
• Private Networks
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
98
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a VRF:
Toggles contract
enforcement on and
off (for testing
purposes)
• Expand Networking folder in and
the tenant
• Right click on VRF and choose
create VRF and enter name
(does not need to be globally
unique)
•
Tenant name appended to the VRF
name
•
Does not contain spaces
•
Certain special characters not
permitted
•
May use leading numbers
(including zero)
Changes the default
interval for aging out
endpoint entries
Changes default
gateway behaviors
for the whole VRF
(migration scenarios)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
99
Cisco ACI Configuration
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Bridge Domains:
•
Definition: A bridge domain is a
Layer forwarding domain
•
Usage: You can use a bridge
domain for:
• Layer 2 broadcast domains
• Functionality similar to a VLAN
• As a container for IP subnets
•
You must have at least one per
VRF but can have more
•
Actually a bridge domain is a
VXLAN segment with a VNI and
multicast group assigned
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
100
Cisco ACI Configuration
ACI Logical Constructs
Configuring a Bridge Domain:
• Expand Networking folder in and
the tenant
• Right click on Bridge Domains,
choose create Bridge Domain
and enter name (does not need
to be globally unique)
•
Does not contain spaces
•
Certain special characters not
permitted
•
May use leading numbers
(including zero)
Changes default
gateway behaviors
for the bridge
domain(migration
scenarios)
ARP traffic sent
directly to the
endpoint (no
flooding)
#CiscoLive
Check to avoid
learning
endpoints outside
of this bridge
domain
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
101
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Subnets:
•
Definition: A subnet is an
addressable IPv4/IPv4 network
•
Usage: You can use a subnet for:
SUBNET(S)
• Defining a default gateway
• Creating an SVI on leaf switches
• Layer 3 external routing
•
You must have at least one per
bridge domain but can have
more
•
Only one subnet can be primary
and additional ones will be
secondary
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
102
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a Subnet:
• Expand the Bridge Domain
folder
• Right click on Bridge Domains,
choose Create Subnet, and
enter:
•
Gateway IP: IP address of the SVI
in CIDR notation
•
Private to VRF: Used only in the
parent VRF
•
Advertised Externally: Enabled for
usage outside the fabric (L3 out)
•
Shared Between VRFs: Visible to
other VRFs or tenants
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
103
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Application Profiles:
•
•
Definition: An application profile
is a container for endpoint
groups (EPGs)
SUBNET(S)
Usage: You can use an
application profile for:
APPLICATION PROFILE
• Defining endpoints for multitier
applications
• Containing one or more
endpoint groups
•
You must have at least one per
bridge domain but can have
more
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
104
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring an Application Profile:
• Right click on Application Profile
Profile Name
and choose Create Application
Profile and enter:
•
Profile Name: Name for the
Application Profile
•
Description: Optional
•
EPGs: Create EPGs as desired
(covered later)
Profile
description
(optional)
Endpoint
group
creation
• Click on Submit
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
105
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Endpoint Groups:
•
•
Definition: An endpoint group is a
collection of endpoints with
similar policies
SUBNET(S)
Usage: You can use an endpoint
group for:
APPLICATION PROFILE
• Mapping endpoints serving an
application tier together
EPG
1
• Grouping endpoints that can
freely communicate
•
You must have at least one per
application profile but can have
more
EPG
2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
106
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Endpoints:
•
•
Definition: An endpoint is a
device that is directly or indirectly
connected to the network
SUBNET(S)
Usage: Endpoints include:
• Physical (bare-metal) servers
APPLICATION PROFILE
• Virtualized servers
• Switches and routers
EPG
1
• L4-L5 devices (such as firewalls
or load balancing appliances)
EPG
2
• IP storage appliances
• Others
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
107
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring an EPG :
• Create an Application Profile
• Create an EPG and supply
parameters:
•
Name: Alphanumeric name of the
EPG
•
BD: Bridge domain of the EPG
•
Domain:
EPG
Name
Click
Update
• Physical: Actual port on a leaf switch
• VMM: Virtual machine port
• Others
•
Static Path: Specific switch port
•
Static Path VLAN: VLAN used on a
fabric port
Bridge
Domain
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
108
ACI Fabric Infrastructure
ACI Logical Constructs
TENANT
VRF
BRIDGE DOMAIN
Cisco ACI Policy Building Blocks
• Contracts:
•
•
Definition: A contract defines
how EPGs communicate
between one another
SUBNET(S)
Usage: You can use a contract to
define:
APPLICATION PROFILE
• What EPGs are allowed to
communicate
EPG
Co
1
ntr
ac
• What ports and protocols are
permitted
•
EPGs cannot communicate with
each other by default
#CiscoLive
DGTL-BRKCRT-2000
t
EPG
2
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
109
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Details:
• Contracts specify the rules of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
communication between EPGs
• Contracts contain one or more
Subjects, which contain:
•
Contract
Filter: Match criteria, including
Layer 2 to Layer 4 such as
protocols and ports
Subject
Filter 1 | Action | Label
Filter 2 | Action | Label
Filter 3 | Action | Label
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
110
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a Contract :
Contract
Name
• Right click on the Contracts
Folder and select Create
Contract
• Enter a contract name
Contract
Scope
• Specify the contract scope:
•
Application Profile
•
VRF
•
Tenant
•
Global
Contract
Description
(Optional)
• Create contract subjects using the
+ control
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
111
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Details:
• Contracts specify the rules of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
communication between EPGs
• Contracts contain one or more
Subjects, which contain:
•
Filter: Match criteria, including
Layer 2 to Layer 4 such as
protocols and ports
•
Action: What to do with the packet,
such as permit, block, redirect, etc.
Contract
Subject
Filter 1 | Action | Label
Filter 2 | Action | Label
Filter 3 | Action | Label
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
112
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a Filter :
• Expand the Contracts Folder
• Right click on Filters and select
Create Filter and supply a name
• Create the filter entries:
•
Name: Name of the filter entry
•
EtherType: IPv4/IPv6, MPLS, etc.
•
IP Protocol: TCP/UDP,ICMP,etc.
•
Source Port/Range: Source
TCP/UDP port (can be blank)
•
Destination Port/Range:
Destination TCP/UDP Port (can be
blank)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
113
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Details:
• Contracts specify the rules of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
communication between EPGs
• Contracts contain one or more
Subjects, which contain:
•
Filter: Match criteria, including
Layer 2 to Layer 4 such as
protocols and ports
•
Action: What to do with the packet,
such as permit, block, redirect, etc.
•
Label:
Contract
Subject
• Subject labels (tags)
Filter 1 | Action | Label
Filter 2 | Action | Label
Filter 3 | Action | Label
• Consumer/provider labels
(direction)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
114
ACI Fabric Infrastructure
ACI Logical Constructs
Configuring a Subject :
• Click on using the + control to in
the Create Contract screen
• Choose the subject options:
• Apply Both Directions: Enable
the contract to be applied in
either direction between EPGs
• Reverse Filter Ports: Allow TCP
return traffic
• Create the filter entries (as
described earlier)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
115
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Details:
EPG Web
WS 1
EP
1
• EPGs have consumer/provider roles
•
One EPG is the consumer
•
One EPG is the provider
connection establishment:
• Workstation 1 chooses a random TCP
source port number for itself
• Workstation 1 sends a TCP
connection request to Server 1 using
port 80
• After the handshake sequence, the
devices can communicate using TCP
• WS 1 is the consumer (source) and
Server 1 is the provider (destination)
Consumer
(source)
Random
Port
TCP
• How this compares to traditional
EP
2
Port
80
EPG App
Server
1
#CiscoLive
EP
3
DGTL-BRKCRT-2000
EP
4
Provider
(destination)
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
116
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Taboos:
• A taboo is a special type of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
contract that acts as a blacklist
• Contracts specify the
permissible traffic between
EPGs
Contract
Permit TCP 23
Permit TCP 22
Permit TCP 80
Permit TCP 443
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
117
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Taboos:
• A taboo is a special type of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
contract that acts as a blacklist
• Contracts specify the
Taboo
permissible traffic between
EPGs
Deny TCP 23
Deny TCP 80
• What happens when a taboo is
added:
• The taboo entry is looked up
and enforced first
Contract
Permit TCP 23
Permit TCP 22
Permit TCP 80
Permit TCP 443
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
118
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Taboos:
• A taboo is a special type of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
contract that acts as a blacklist
• Contracts specify the
Taboo
permissible traffic between
EPGs
Deny TCP 23
Deny TCP 80
• What happens when a taboo is
added:
• The taboo entry is looked up
and enforced first
Contract
Permit TCP 23
Permit TCP 22
Permit TCP 80
Permit TCP 443
• Contract entries are looked up
and enforced next
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
119
ACI Fabric Infrastructure
ACI Logical Constructs
Cisco ACI Contract Taboos:
• A taboo is a special type of
EPG Web
EP
1
EPG App
EP
2
EP
3
EP
4
contract that acts as a blacklist
• Contracts specify the
Taboo
permissible traffic between
EPGs
Deny TCP 23
Deny TCP 80
• What happens when a taboo is
added:
• The taboo entry is looked up
and enforced first
Contract
Permit TCP 23
Permit TCP 22
Permit TCP 80
Permit TCP 443
• Contract entries are looked up
and enforced next
• A taboo can override entries
in a contract
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
120
ACI Packet
Forwarding
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Host Discovery
• Unlike VXLAN which uses
MP-BGP, ACI uses COOP
Council of oracle protocols
•
Leaf switches are “citizens”
•
Spine switches are “oracles”
OP
CO
COOP
•
UNDERLAY
CO
OP
IS-IS
COOP
• Leaf switches discover
attached host IP/MAC
addresses and advertise
them to the spines
• Spine switches share this
information with the other
leaf switches
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
122
ACI Packet Forwarding
Endpoint Learning
What is an Endpoint?
• Used to forward traffic
• An endpoint is a host
OP
CO
• One MAC addresses
COOP
• Endpoints can consist of:
UNDERLAY
CO
OP
IS-IS
COOP
• MAC and one or more IP
addresses
• Types of endpoints:
• Local (locally learned)
• Remote (learned through
data plane/spine proxy
lookup)
MAC: 0001.0001.0001
MAC: 0002.0002.0002
IP: 192.168.33.33
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
123
ACI Packet Forwarding
Endpoint Learning
UNDERLAY
Endpoint Lookups:
• ACI uses an endpoint table
OP
CO
OP
CO
CO
OP
CO O P
IS-IS
in place of a MAC/ARP
tables
• All IP addresses are host
addresses (/32 or /128)
MAC: 0001.0001.0001
• ARP still used in L3 Out
Traditional Network
Table
MAC: 0002.0002.0002
IP: 192.168.33.33
Cisco ACI
Role
● IPv4 addresses (/32 and non-/32)
● IPv6 addresses (/128 and non-/128)
Table
MAC address table
MAC addresses
Endpoint
MAC and IP addresses (/32 or /128 only)
ARP table
Relationship of IP to MAC
ARP
Relationship of IP to MAC (only for Layer 3 outside [L3Out]
connections)
RIB
RIB
#CiscoLive
Role
● IPv4 addresses (non-/32*)
● IPv6 addresses (non-/128*)
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
124
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• WS-1 sends an IP packet
with WS-2’s destination
address
IS-IS
IP
IP Payload
Hdr
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
125
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• WS-1 sends an IP packet
with WS-2’s destination
address
• The IP payload is wrapped
IS-IS
in an 802.1Q frame
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
126
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• WS-1 sends an IP packet
with WS-2’s destination
address
• The IP payload is wrapped
IS-IS
in an 802.1Q frame
• The frame is transmitted to
the attached leaf switch
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
127
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• WS-1 sends an IP packet
with WS-2’s destination
address
• The IP payload is wrapped
IS-IS
in an 802.1Q frame
• The frame is transmitted to
the attached leaf switch
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
128
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• WS-1 sends an IP packet
with WS-2’s destination
address
• The IP payload is wrapped
IS-IS
in an 802.1Q frame
• The frame is transmitted to
the attached leaf switch
Payload
• At ingress, the Layer 2
header is completely
removed (original IP packet
is unaltered)
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
129
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
130
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
VXLAN
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
131
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
• The leaf switch performs a
destination lookup and then
builds the tunnel header
Outer Outer
UDP
MAC
IP
VXLAN
Eth
Header
Payload
FCS
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
132
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
• The leaf switch performs a
destination lookup and then
builds the tunnel header
• Just as in VXLAN, the frame
is sent to the spine and then
to the egress switch
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
133
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
• The leaf switch performs a
destination lookup and then
builds the tunnel header
• Just as in VXLAN, the frame
is sent to the spine and then
to the egress switch
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
134
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
• The leaf switch performs a
destination lookup and then
builds the tunnel header
• Just as in VXLAN, the frame
is sent to the spine and then
to the egress switch
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
135
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Encapsulation Steps
• The ingress leaf switch
rebuilds the Layer 2 frame
and attaches a VXLAN
header
IS-IS
• The leaf switch performs a
destination lookup and then
builds the tunnel header
• Just as in VXLAN, the frame
is sent to the spine and then
to the egress switch
• The egress switch removes
the headers and builds a
new Layer 2 frame to WS-2
WS-1
WS-2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
136
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Forwarding Lookups
• Anycast/Distributed gateway
lives on multiple leaf
switches
IS-IS
Distributed Gateway
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
137
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Forwarding Lookups
• Anycast/Distributed gateway
lives on multiple leaf
switches
• Leaf switches discover
attached hosts and can me
moved anywhere in the
fabric
IS-IS
Distributed Gateway
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
138
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Forwarding Lookups
• Anycast/Distributed gateway
lives on multiple leaf
switches
• Leaf switches discover
attached hosts and can me
moved anywhere in the
fabric
IS-IS
Distributed Gateway
• Layer 2 (bridged traffic) is
by MAC address (learned in
hardware)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
139
ACI Packet Forwarding
Endpoint Learning
Cisco ACI Forwarding Lookups
• Anycast/Distributed gateway
lives on multiple leaf
switches
• Leaf switches discover
attached hosts and can me
moved anywhere in the
fabric
IS-IS
Distributed Gateway
• Layer 2 (bridged traffic) is
by MAC address (learned in
hardware)
• Layer 3 (routed traffic) is by
IP address (learned in
hardware)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
140
External Network
Connectivity
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Outside Connection Basics:
• Inside vs. Outside the Fabric
Inside
• Inside:
• Fabric switches (spine or leaf)
• APICs
• Bare Metal/virtualized servers
• L4-L7 devices
• Outside:
• Data center legacy networks
• WAN connections
• Internet connections
• Anything not covered in the list above
Outside
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
142
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Outside Connection Basics:
• Types of Outside Connections:
• Layer 3 (Routed Outside):
• Routed connection out of the fabric
• Supported routing protocols:
• Static routes
• OSPF
• EIGRP
• BGP
• Layer 2 (Bridged Outside):
• Integration into existing data center
• Extend switched network/broadcast domain
• vPC and STP support (south of the fabric)
• VLAN/VXLAN outside support
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
143
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Outside Connection Basics:
• Layer 3 Outside Route Distribution:
• MP-BGP active inside the fabric
BGP AS
65001
• MUST be configured (not automated as ISIS is)
MP-BGP
• Spine(s) acts as BGP Route Reflector
• BGP peering between leaves and route
reflectors is located in spine
• Activated by creating a Cisco ACI Fabric
Route Reflector Policy (and applied to the
Pod/Fabric)
• External Tenant and Infra-Routes
Exchanged with External Routers
• Route redistribution between internal BGP
and outside occurs on border leaves
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
144
External Network Connectivity
Cisco ACI Outside Connections
• Routed Interface
Routed
Subinterface
• Native Layer 3 Interface
• Physical port placed in Layer 2 mode
• Subinterfaces created using 802.1Q VLANs
• SVI/VLAN Interface
• Physical port placed in Layer 2 mode
20.20.10.1/24
• Subinterface
10.10.10.1/24
• IP address applied directly
10.10.10.1/24
10.10.10.1/24
• Routed port
SVI
VLAN 20
• Supported Routed Port Types:
Routed
Interface
VLAN 10
Cisco ACI Outside Connection Basics:
• Layer 3 configuration placed on VLAN
interface
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
145
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Layer 3 Outside Configuration:
• Prerequisites:
• MP-BGP must be configured and
functional
BGP AS
65001
MP-BGP
• Route reflector policy created
• Pod policies applied to the fabric
• Peering sessions up between leaves/spines
• Tenant and dependent items already
configured
• May be configured in common tenant
• Layer 3 external domain configured
• User tenants must have all items configured:
• VRF
• Bridge domain and subnets
• Application profiles and EPGs
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
146
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Layer 3 Outside Configuration:
• Navigate to <tenant> à Networking à
External Routed Networks
• Right-click and choose Create Routed
Outside and supply these parameters:
• Name (alphanumeric)
• VRF (if more than one)
• External Routed Domain
• Layer 3 Routing Protocol (check boxes):
• BGP
• EIGRP
• OSPF (used in this example)
• OSPF area ID
• OSPF area type
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
147
External Network Connectivity
Cisco ACI Outside Connections
Cisco ACI Layer 3 Outside Configuration:
• Configure the switch (node) to use:
• Node (drop-down box)
• Description (optional)
• Router ID: IP address to use
• Target DSCP (optional)
• Use Loopback as Router ID (checked by
default)
• Loopback addresses (if configured)
• Static routes (if configured):
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
148
External Network Connectivity
Implement Layer 3 Out
Cisco ACI Layer 3 Outside Configuration:
• Configure the interface to use:
• Create Interface Profile screen (step 1)
• Name (alphanumeric)
• Description (optional)
• ND Policy
• NetFlow Monitor Policies
• Config protocol policies (checked by default)
• Click Next
• Protocol Policies screen (step 2)
• OSPF Profile
• Authentication
• OSPF Interface
• BFD Interface Profile
• HSRP Interface Profile
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
149
External Network Connectivity
Implement Layer 3 Out
Cisco ACI Layer 3 Outside Configuration:
• Configure the interface to use:
• Interfaces screen (step 3)
• Interface Type
• Routed Interfaces (used in this example)
•
Path Type (port/port-channel)
•
Node (drop-down box for specifying switch)
•
Path (drop-down box for specifying port)
•
IPv4/IPv6 Preferred Address (interface address)
•
IPv4/IPv6 Additional Addresses
•
MAC Address (prepopulated)
•
MTU (bytes)
•
Target DSCP
•
Link-Local Address
• Click on OK/Submit/Finish
to complete
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
150
External Network Connectivity
Implement Layer 2 Out
Cisco ACI Layer 2 Outside Configuration:
• Prerequisites:
• Fabric access policies already
configured
• VLAN pools
• Physical domains
• Interface policies and profiles
• Tenant and dependent items already
configured
• May be configured in common tenant
• Layer 2 external domain configured
• User tenants must have all items configured:
• VRF
• Bridge domain and subnets
• Application profiles and EPGs
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
151
External Network Connectivity
Implement Layer 2 Out
Cisco ACI Layer 2 Outside Configuration:
• Configuration Options:
• Extend an EPG outside the fabric
• Same method for static binding for bare
metal hosts
• Assigns a physical port to a specific EPG
• Manually assigns a VLAN to the port
• Extend the bridge domain outside the
fabric
• Create a Layer 2 Outside Domain
• Extends the broadcast domain to entities
outside the fabric
• An EPG is created to represent the attached
entities via Layer 2
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
152
External Network Connectivity
Implement Layer 2 Out
Interaction with Spanning-Tree Protocol:
• No STP operation within the Fabric
BP
DU
• Bridge protocol data unit (BPDU)
DU
P
B
frames are flooded within the EPG,
with no additional configuration
necessary
• The ACI Fabric passes through the
BPDU’s transparently
• External switches break any potential
loop upon receiving the flooded BPDU
from the Cisco ACI Fabric (acts like a
hub)
• The Spanning-Tree domain south of
Same EPG
STP Root Switch
the fabric sees it as one switch
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
153
External Network Connectivity
Implement Layer 2 Out
Cisco ACI Layer 2 Loop Detection:
• The ACI fabric does not generate an
STP BPDU frame.
• Connection leaf ports together can
P
O
LO
create loops detectable by LLDP
!
X
• ACI utilizes a different protocol
(miscabling protocol or MCP) to detect
loops as well
• Offending ports are immediately
disabled to prevent loops
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
154
External Network Connectivity
Implement Layer 2 Out
Cisco ACI Layer 2 Outside Configuration:
• Navigate to <tenant> à Networking à
External Bridged Networks
• Right-click and choose Create Bridged
Outside and supply these parameters:
• Name (alphanumeric)
• External Bridged Domain (if extending EPG)
• Encap
• VLAN or VXLAN (drop-down box)
• VLAN/VXLAN ID (numeric)
• Nodes and Interface Protocol Profiles
• Path Type (Port/PC/VPC)
• Node (drop-down box)
• Path (drop-down box)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
155
ACI Management
Integrations
Implement VMware vCenter DVS integration
Overview:
Creating an EPG in ACI…
Web
App
DB
automatically
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
157
Integrations
Implement VMware vCenter DVS integration
Overview:
Creating an EPG in ACI…
Web
App
DB
automatically
…creates a port group in the virtualization layer.
Web
App
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
158
Integrations
Implement VMware vCenter DVS integration
Overview:
• For this process to work, two things
must happen:
1.
The APIC must communicate with
vCenter, and an “APIC controlled”
DVS is created.
2.
The leaf node must
“discover” the host by using CDP or
LLDP.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
159
Integrations
Implement VMware vCenter DVS integration
Overview:
• For this process to work, two things
must happen:
1.
The APIC must communicate with
vCenter, and an “APIC controlled”
DVS is created.
2.
The leaf node must
“discover” the host by using CDP or
LLDP.
CDP or LLDP
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
160
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration :
VLAN
Pool
vCenter
Domain
Access
Port
Policy
Group
AAEP
#CiscoLive
DGTL-BRKCRT-2000
Interface
Profile /
Selector
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
161
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration:
1.
VLAN
Pool
Create a dynamic VLAN Pool
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
162
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration:
2.
vCenter
Domain
Create the vCenter domain
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
163
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration:
3.
AAEP
Create the Attachable Access Entity
Profile
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
164
Integrations
Implement VMware vCenter DVS integration
Access
Port
Policy
Group
Configure VMware Integration:
4.
Create an Access Port Policy Group
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
165
Integrations
Implement VMware vCenter DVS integration
Interface
Profile /
Selector
Configure VMware Integration:
5.
Create an Interface Profile and Port
Selector
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
166
Integrations
Implement VMware vCenter DVS integration
Interface
Profile /
Selector
Configure VMware Integration:
6.
Associate the interface profile with
the switch profile
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
167
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration:
7.
Map the VMM domain and port to an EPG
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
168
Integrations
Implement VMware vCenter DVS integration
Configure VMware Integration:
8.
Verify the integration is working
CLVMM
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
169
ACI Management
ACI Management
Cisco ACI Management
Cisco ACI Management Networks:
• Infrastructure VRF (infra):
• Lives in the infra tenant
infra
• VRF uses the name overlay-1
• Infra contains all of the internal
operations of the Cisco ACI Fabric:
• IS-IS adjacencies and IP routes
• LLDP neighbor discovery
• Underlay routing operations
• MP-BGP operations
• DHCP address allocation operations
• TEP management
• In most situations you do not need to
alter the infra VRF/tenant
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
171
ACI Management
Cisco ACI Management
Cisco ACI Management Networks:
• Out of Band (OOB):
• Lives in the mgmt tenant
OOB
• Tied to a default EPG
• Contains all management ports on
Cisco ACI devices:
• Spine switch nodes
• Leaf switch nodes
• APICs
• Used to assign management
addresses to devices (address
pool/node management addresses)
• Contracts can be created and applied
to restrict communication on the OOB
EPG
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
172
ACI Management
Cisco ACI Management
Cisco ACI Management Networks:
• In Band VRF:
• Lives in the mgmt tenant
• Tied to a default EPG
• Used to allow higher bandwidth
interfaces for management purposes
• Used when management addresses
need to be leaked outside the fabric
• Can utilize Layer 2 or Layer 3 external
connections
In-Band
• Contracts can be created and applied
to restrict communication on the Inband EPG
• When configured, In-band is preferred
over OOB
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
173
ACI Management
Implement AAA and RBAC
Cisco ACI Security Management:
• Security Domains:
• Tag that specifies what a user can
access in the Cisco ACI environment
• Types of security domains include:
• Default security domains (system created):
• All: Access to everything (no restrictions)
• Common: Access to the fabric
All
Fabric
X
Tenant 1
X
Tenant 2
X
User A
User B
User C
X
X
X
X
• Mgmt: Access to the management constructs
• User-Created Security Domains
• Various objects in the systems can be
tied to a security such as:
• Tenants
• Domains
• Etc.
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
174
ACI Management
Implement AAA and RBAC
System Access, Authentication,
Authorization, RBAC
Cisco ACI Security Management:
• AAA and Role-Based Access Control:
• AAA functions can be:
• Local (configured/stored on the APICs only)
Universe
• Remote (TACACS+, RADIUS, LDAP, etc.)
• Role-Based Access Control Functions:
• Control read/write access to all functions
• Enforces fabric admin and per-tenant admin
separation
• Predefined roles
• Admin (default users)
• User-created roles
• Privilege levels
Tenant: AA
Tenant: BB
Fabric
App Profile
App Profile
Switch
EPGs
EPGs
Line Cards
Layer 3
Networks
Layer 3
Networks
Ports
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
175
ACI Management
Implement AAA and RBAC
Cisco ACI Security Management:
• Creating User
Accounts:
Create
User
Specify
Domain(s)
Admin creates a
new local user by
using the local
user option from
the available
authentication
options:
• Local User
• LDAP
• RADIUS
• TACACS+
Admin creates a new
security domain or
chooses from existing
security domains to
associate with the user:
• all (system default)
• mgmt (system
default)
• common (system
default)
• Solar (a tenant)
• Wind (a tenant)
Admin creates a role for the user by selecting
privileges such as:
• admin
• aaa
• access-admin
• fabric-admin
• tenant-admin
• vmm-admin
For each privilege, admin enables a mode:
• No access
• Read only
• Read write
Creates local user
“User.”
Associates User with
the domain “all.”
Assigns User “admin” privilege (includes all
privileges) and enables read-write mode.
#CiscoLive
Assign Role(s)
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
176
ACI Management
Configure an Upgrade
Cisco ACI Firmware Management :
• Software Image Types:
• Controller (version 1.x, 2.x, etc.)
All-Spines
(Nodes)
All-Leaves
(Nodes)
• Node (version 11.x, 12.x, etc.)
• Versions should match (4.1 à 14.1)
• Catalog image
• The ACI fabric uses the same global
catalog methodology as Cisco UCS
• APIC and switch node image
management is controlled by policies:
• Firmware groups (image version)
All-Leaves
(Controllers)
• Upgrade groups (which devices)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
177
ACI Anywhere
ACI Anywhere
Describe Topology Options
Single Site/Fabric (1.0)
Multisite/Multifabric (3.0+)
Stretched Fabric (1.2)
Multipod Fabric (2.0+)
Virtual Pod (4.0+)
Remote Leaf (4.0+)
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
179
ACI Anywhere
Describe Multipod
Inter-Pod
Network
Cisco ACI Multipod Description:
• Multiple ACI Pods connected by an IP
Pod ‘A’
Inter-Pod L3 network, each Pod
consists of leaf and spine nodes
Pod ‘N’
MP-BGP
- EVPN
• Managed by a single APIC Cluster
• Single Management and Policy
Domain
• Forwarding control plane (IS-IS,
50 msec RTT
COOP) fault isolation
• Data Plane VXLAN encapsulation
between Pods
IS-IS, COOP, MP-BGP
IS-IS, COOP, MP-BGP
Availability Zone
• End-to-end policy enforcement
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
180
ACI Anywhere
Describe Multisite
Inter-Site
Network
Cisco ACI Multisite Description:
• Separate ACI Fabrics with
Site 1
independent APIC clusters
Site 2
MP-BGP
- EVPN
• ACI Multi-Site Orchestrator pushes
cross-fabric configuration to multiple
APIC clusters providing scoping of all
configuration changes
• MP-BGP EVPN control plane between
sites
• Data Plane VXLAN encapsulation
across sites
• End-to-end policy definition and
50 msec RTT
IS-IS, COOP, MP-BGP
IS-IS, COOP, MP-BGP
Availability Zone A
Availability Zone B
enforcement
Region
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
181
Conclusion
Conclusion
•
The DCACI Exam (300-620) covers many fundamental ACI topics
•
The three primary components of Cisco ACI are the Fabric, the
APICs and Policies
•
Policies are used to create the desired end state of the network,
such as IP addressing, port states, traffic flow, etc.
•
Knowing how VXLAN and EVPN operate can help you better
understand Cisco ACI
•
Some of the principles of operation of Cisco ACI can run counter to
an understanding of traditional networking
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
183
Conclusion
Recommended Cisco authorized training:
#CiscoLive
DGTL-BRKCRT-2000
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
184
Thank you
#CiscoLive
#CiscoLive