VNPT NFVi Solution Walkthrough Presentation Sharad Gangwar PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 1 ETV 2019/05/22 Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Networking — NFVi HLD Structure — Integration Points PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 2 NFVi Scope The following points are considered as in scope: — Hight level Design Scope covers two sites (Hanoi and Hochiminh) of VNPT’s network. — Software defined infrastructure design is based on NFVI software release R5.1. — Designing of Ericsson Cloud execution environment (ECEE) on SDI system. — Designing of NexentaStor Software Defined Storage solution. — HW components HP DL360 Gen10 (CCM, compute, Nexenta), HP DL380 Gen10 (Storage), NRU0201 (Network) and EAS0102 (Control network) and related infrastructure have been considered in the design scope. — SR-IOV (Single Root I/O Virtualization) for accelerating VM throughput will be used for vEPC. — NFVi Tenant Networking without SDN PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 3 Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — NFVi HLD Structure — Integration Points PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 4 NFVi Hardware Site Hanoi HCM HW Components HPE DL360 Gen10 HPE DL380 Gen10 HPE D3610 EAS0102 (Leaf) EAS0102 (Spine) NRU0201 (Leaf) NRU0201 (Spine) HPE DL360 Gen10 HPE DL380 Gen10 HPE D3610 EAS0102 (Leaf) EAS0102 (Spine) NRU0201 (Leaf) NRU0201 (Spine) PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 5 Qty. 24 2 2 2 2 4 2 21 2 2 2 2 4 2 Comment SDI Manager HW, CEE controller and VNF compute NexentaStor compute NexentaStor JBOD enclosure Control Network Switch Control Network Switch Data Network Switch Data Network Switch SDI Manager HW, CEE controller and VNF compute NexentaStor compute NexentaStor JBOD enclosure Control Network Switch Control Network Switch Data Network Switch Data Network Switch NFVi Software Component Version Software defined Infrastructure 2.10 Ericsson Cloud Execution Environment (VIM) 6.6.2 NexentaStor 5.2.1 NexentaFusion (Management of NexentaStor) 1.2.1 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 6 NFVi High Level Architecture G HNI-MPE-MX960-07/08 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 7 HPE EAC_1 VM EAC_2 VM Storage vPOD vEPC VNF vEPC VNF vEPC VNF HPE vEPC VNF Nexenta Controller 1 HPE vIMS VNF Nexenta Controller 2 HPE HPE vEPC VNF Nexenta JBOD 1 Nexenta JBOD 2 SDI Manager and all managed equipment Cisco ASR 9K G CCM VM HPE vIMS VNF OAM Network X EAC_2 VM HPE HPE HPE HPE HPE HPE HPE HPE HPE HPE HPE vEPC VNF vEPC VNF EAC_1 VM Solaris vEPC VNF CCM VM Solaris HPE vIMS VNF HDS default vPOD Solaris HPE vIMS VNF NRU leaf cluster Solaris X Control Fabric vIMS VNF CEE UBU KVM vEPC VNF EAS stack X Redundancy host vEPC VNF X AZ – vIMS Compute CEE UBU KVM X vIMS VNF AZ – vEPC SR-IOV SDI Manager and all managed equipment EAS stack vIMS VNF CEE UBU KVM vEPC VNF CEE UBU KVM vEPC VNF vIMS VNF CEE UBU KVM vEPC VNF AZ – vEPC SR-IOV vEPC VNF CEE vFuel CEE UBU KVM vEPC VNF CEE UBU KVM vEPC VNF CEE vCIC3 CEE UBU KVM CEE UBU KVM vIMS VNF CEE vFuel X CEE vPOD – VIM zone for vEPC & vIMS AZ – vEPC NON-SRIOV CEE UBU KVM Solaris vIMS VNF CEE vCIC2 X Data Fabric X AZ nova (Default) AZ – vEPC NON-SRIOV CEE UBU KVM Nexenta JBOD 2 Redundancy host vIMS VNF AZ – vIMS Compute CEE UBU KVM Nexenta JBOD 1 Solaris Nexenta Controller 2 Solaris Solaris Nexenta Controller 1 HPE AZ nova (Default) vIMS VNF CEE Atlas X HPE CEE vFuel vIMS VNF CEE vCIC1 HPE HPE CEE vCIC3 vIMS VNF X AZ – CEE CTRL HPE Storage vPOD HPE HPE HPE EAC_2 VM vIMS VNF NRU spine cluster G G TTI-PE-CI-ASR9K-05/06 EAS stack X EAS stack X CEE UBU CEE UBU KVM KVM EAC_1 VM CEE vFuel NRU leaf cluster CEE vPOD – VIM zone for vEPC & vIMS vIMS VNF O CEE UBU CEE UBU CEE UBU KVM KVM KVM CCM VM CEE vCIC2 NRU leaf cluster CEE UBU CEE UBU CEE UBU KVM KVM KVM EAC_2 VM CEE Atlas O TTI-PE-CI-ASR9K-05/06 CEE UBU CEE UBU CEE UBU KVM KVM KVM EAC_1 VM CEE UBU CEE UBU KVM KVM CCM VM CEE vCIC1 CEE UBU CEE UBU CEE UBU KVM KVM KVM HDS default vPOD HPE AZ – CEE CTRL X Hochiminh/224-TTI Converter G HPE X G HPE X HPE Data Fabric HPE X NRU leaf cluster G HNI-MPE-MX960-07/08 X HPE X G O HPE O NRU spine cluster MPLS backbone HPE Converter HPE Hanoi/030-PHG Control Fabric X X Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — Integration Points — NFVi HLD Structure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 8 SDI Physical Connectivity – 030-PHG 030-PHG POD Converter 4x100G O O 2x100G X 1 x Spine Cluster X 4x100G 8x100G to Leaf per Spine switch NRUs Data fabric 4x100G to Spine per Leaf switch 2 x Leaf Clusters X X X X 2x100G CEE Compute Nexenta compute 2x100G CEE Compute CCM/RTE Control fabric 100GE 25GE 10GE 1GE X PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 9 X EAS 2x10G X X G HNI-MPEMX960-07 G HNI-MPEMX960-08 SDI Physical Connectivity – 224-TTI 224-TTI POD Converter 4x100G O O 2x100G X 1 x Spine Cluster X 4x100G 8x100G to Leaf per Spine switch NRUs Data fabric 4x100G to Spine per Leaf switch 2 x Leaf Clusters X X X X 2x100G CEE Compute Nexenta compute 2x100G CEE Compute CCM/RTE Control fabric 100GE 25GE 10GE 1GE X PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 10 X EAS 2x10G X X G TTI-PE-CIASR9K-05 G TTI-PE-CIASR9K-06 SDI POD/vPOD Architecture DC POD VPNs VPNs CEE vPOD Nexenta vPOD vPOD user VNF Nexenta Stor — CEE vPOD: VIM to deploy vEPG and vIMS VNFs — Nexenta vPOD: provide block and file storage for vIMS — HDS Management vPOD: default vPOD (used by SDI system) CIC Atlas Fuel HDS Management vPOD RTE 2x Control Data/Storage Compute for cloud users vPOD user x3 CEE control plane x1 CCM EAC X EAS 4x PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 11 xN NRU x6 X GG DC-GW SDI Networking SDI Data fabric with L2 VLAN only IPBN CORE Data fabric physical connectivity L2/L3/ MPLS G L3 Gateway Access port to DC Gateway 2x100G to DC-GW per Spine switch X 8x100G to Leaf per Spine switch X NRUs 100GE 25GE X X 2x100G VLAN O Converter Switch VLAN X X L2 Fabric XX X L2 VLAN only XX X X X X X X X X X X X X X NRU LEAF VLAN 2x100G vPOD A Access port to compute systems vPOD B PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 12 NRU SPINE VLAN 4x100G to Spine per Leaf switch 2 x Leaf Clusters L2 DC-GW R R O 2x100G 1 x Spine Cluster L2 G R X L3 Gateway L2 VLAN Switching COMPUTE DC-GW Physical Connectivity – 030-PHG Physical Perspective HNI-MPEHNI-MPEMX960-08 MX960-07 1xMLAG to DCGW (2x100GE) 1xMLAG to DC-GW (2x100GE) Converter Switches G G Logical Perspective HNI-MPE- HNI-MPEMX960-07 MX960-08 G G LAG (2x100G) LAG (2x100G) O O 1xMLAG to Converter switches (4x100GE) O Logical Converter Switch LAG (4x100G) X Spine Cluster PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 13 X X Logical Spine Switch DC-GW Physical Connectivity – 224-TTI Physical Perspective TTI-PE-CITTI-PE-CIASR9K-05 ASR9K-06 1xMLAG to DCGW (2x100GE) 1xMLAG to DC-GW (2x100GE) Converter Switches G G Logical Perspective TTI-PE-CI- TTI-PE-CIASR9K-05 ASR9K-06 G G LAG (2x100G) LAG (2x100G) O O 1xMLAG to Converter switches (4x100GE) O Logical Converter Switch LAG (4x100G) X Spine Cluster PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 14 X X Logical Spine Switch Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — Integration Points — NFVi HLD Structure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 15 CEE Architecture Data Center Gateway — CIC: Cloud Infrastructure Controller X CEE Traffic — CEE Portal (Atlas): Management Interfaces managing the virtual infrastructure. — vFuel: Software life cycle management, adding installation, update, and equipment management support for a CEE instance — Compute: Compute capabilities to virtual applications including vCPU, memory PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 16 Storage Compute Network Hypervisor Host OS Software defined Storage (NexentaStor) Switch Fabric Ericsson vSwitch Compute HW X Control X Storage Infrastructure Management CEE Management (atlas) Cloud Infrastructure Controlller (CIC) OpenStack Controller vFuel CEE Networking Domain — Control Domain: The Control switching domain is used for the OpenStack control tasks — Traffic Domain: forward the internal and external traffic of the tenant VMs, external OAM traffic of vCIC/Atlas — Storage Domain: connect to external storage system (i.e NexentaStor) PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 17 CEE Host Networking Non-SRIOV Compute VM VM vNIC vNIC Hypervisor Nstor Cinder backend Linux Bonding (active/backup) iLO OVS OVS + DPDK OVS bonding (active/backup) OVS bonding blance-slb Embedded NIC 2 Port NIC Card 1 Migration Swift Linux Bonding (active/backup) Linux Bonding (active/backup) SRIOV Compute VM vNIC 2 Port NIC Card 2 VM vNIC vNIC VF VF VF VF VF VF vNIC vNIC vNIC VF VF VF VF VF VF Hypervisor Control switching domain Traffic switching domain Storage switching domain iLO PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 18 OVS OVS + DPDK OVS bonding (active/backup) OVS bonding blance-slb Embedded NIC Control domain 2 Port NIC Card 1 Traffic domain (OVS/DPDK) 2 Port NIC Card 2 Traffic domain (SR-IOV) 2 Port NIC Card 3 Traffic domain (SR-IOV) Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — Integration Points — NFVi HLD Structure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 19 NFVi Networking Overview VNF Traffic Types VM VM VM PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 20 vNIC Access Edge vNIC WAN Edge VNF Networking SRIOV NIC WAN MGMT Edge DC Edge Tenant Traffic Domain CSS DC MGMT Edge Data Fabric VNF Networking CSS DC-GW Functions Control Fabric Tenant Traffic Domain DC-GW NIC HDS Physical Layer Storage Network Domain NIC NFVI Control Domain Storage attachment VNF Layer Tenant Traffic Flows VM to VM via CSS nodes X X NRU Spine cluster NRU Leaf cluster 1 X X X X X CSS CSS CSS vNIC vNIC vNIC vNIC VM VM Compute 1 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 21 VM Compute 2 VLAN 10 IP: 10.0.0.5 X VLAN 10 IP: 10.0.0.4 X VLAN 10 IP: 10.0.0.3 VLAN 10 IP: 10.0.0.2 CEE VM Compute 3 NRU Leaf cluster 2 Tenant Traffic Flows VM via CSS node to VM with SR-IOV X X NRU Spine cluster NRU Leaf cluster 1 X X X X CEE X VM1 VM2 Compute 1 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 22 vNIC VM3 Compute 2 VLAN 10 IP: 10.0.0.5 vNIC VLAN 10 IP: 10.0.0.4 vNIC VLAN 10 IP: 10.0.0.3 VLAN 10 IP: 10.0.0.2 CSS vNIC VM4 Compute 3 NRU Leaf cluster 2 Tenant Traffic Flows VM with SR-IOV to VM with SR-IOV X NRU Leaf cluster 1 X NRU Spine cluster X X X X VM1 VM2 Compute 1 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 23 vNIC VM3 Compute 2 VLAN 10 IP: 10.0.0.5 vNIC VLAN 10 IP: 10.0.0.4 vNIC VLAN 10 IP: 10.0.0.3 VLAN 10 IP: 10.0.0.2 CEE vNIC VM4 Compute 3 NRU Leaf cluster 2 Tenant Traffic Flows VM via CSS node/VM with SR-IOV to DC-GW L3 VRF G RX DC-GW VRFs X X NRU Spine cluster NRU Leaf cluster 1 X X X X CSS CSS vNIC vNIC VM1 VM2 Compute 1 PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 24 vNIC VM3 Compute 2 VLAN 20 IP: 10.0.20.5 X VLAN 12 IP: 10.0.2.4 X VLAN 11 IP: 10.0.1.3 VLAN 10 IP: 10.0.0.2 CEE vNIC VM4 Compute 3 NRU Leaf cluster 2 Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — Integration Points — NFVi HLD Structure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 25 NFVi Networking – Integration to VNPT DC-GW R vEPC VRFs R vIMS VRFs R Infra OAM VRF DC-GW R Storage VRFs Data Fabric L2 Networks L2 Networks CCM access network VNPT to provide: — DC-GW ports to connect to Data and Control Fabric — Hanoi/030-PHG — 4 ports 100GE to Data Fabric — 2 port 10GE (SFP+ SM or MM LC) to Control Fabric — HCM/224-TTI — 4 ports 100GE to Data Fabric — 2 ports 10GE (SFP+ SM or MM LC) to Control Fabric — IP & VLAN in below table IP & VLAN requirements vEPC VNFs vIMS VNFs CEE Infrastructure CEE vPOD NexentaStor App Nexenta vPOD PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 26 CCM HDS Compute Resources Parameters 030-PHP 224-TTI Public IPv4 1* /28 1* /28 IPv6 1* /121 1* /121 Private IPv4 1 */19 1 */19 IP UE Pools - IPv4: 5000000 IPs - IPv6: 5000000 IPs - IPv4: 5000000 IPs - IPv6: 5000000 IPs VLAN 160 VLANs 158 VLANs Integration to VNPT NTP Compute Systems — Hanoi VNPT NTP: — NTP1: 10.149.233.10 — NTP2: 10.149.233.14 — HCM VNPT NTP: — NTP1: 10.149.7.190 — NTP2: 10.149.7.194 HDS CCM EAC Fabric Nexenta vPOD CEE vPOD Nexenta Compute CEE CIC vCIC Ctrl Ctrl Ctrl Atlas Traffic Storage hds-ccm-access-nw nstor_om_sp Control Fabric cee_om_sp atlas_nbi_sp Data Fabric DC-GW R R om-stor-vr om-dco-vr SS NTP server PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 27 DC Infrastructure Service R om_vim_vr R atlas_sbi_vr Alarms/Events integration to VNPT SNMP Server VNPT SNMP server — Alarms/Events will be forwarded as SNMP Trap 6 GUI 9 — SNMPv2c will be used CCM — VNPT to provide SNMP server IP and community string GUI 3 Atlas NRU 2 EAS Networking Infrastructure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 28 NexentaFusion 8 EAC 1 GUI vCIC 11 4 Nexenta App 7 Compute node(s) CEE CEE vPOD 10 5 Nexenta Compute(s) Nexenta vPOD Integrate to VNPT Log MGMT Server Interface Log Type Protocol CCM to VNPT Log MGMT server Audit Logs Syslog over TCP/ Syslog over UDP/ RELP CEE to VNPT Log MGMT server Security and Audit Logs Syslog over TCP/ RELP — VNPT to provide Log MGMT server IP and port/protocol PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 29 Agenda — NFVi Scope — NFVi Solution Overview — SDI Architecture — CEE Architecture — NFVi Tenant Networking — Integration Points — NFVi HLD Structure PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 30 NFVi HLD Structure 1. Executive Summary 1.1 Purpose 1.2 Introduction 1.3 Scope 1.3.1 In Scope 1.3.2 Out-of-Scope 1.4 Assumptions & Considerations 1.5 Intended Audience 1.6 Revision History 1.7 Notation 2 Solution Overview 2.1 NFVi Solution Architecture 2.2 NFVi Solution Components 2.2.1 Software Defined Infrastructure (SDI) 2.2.2 Virtual Infrastructure Management (VIM) 2.2.3 Software Defined Storage (SDS) 2.3 Software Baseline 2.4 Hardware and Software BoQ 2.5 Naming Convention 3 HDS – Software Defined Infrastructure 3.1 HDS Overview 3.2 Hardware Components 3.2.1 Rack 3.2.2 Computes 3.2.3 Storage 3.2.4 Data Switch: NRU0201 3.2.5 Control Switch: EAS0102 3.3 Run-time Environment (RTE) 3.4 CCM and EAC 3.4.1 Overview 3.4.2 POD and vPODs 3.4.3 User Management and Access Control 3.5 Physical Connectivity 3.5.1 Overview 3.5.2 Host Connectivity 3.5.3 Data Fabric 3.5.4 Control Fabric 3.6 Logical Connectivity 3.6.1 Control and Management 3.6.2 CCM Control Network VLAN Allocation 3.6.3 Data Network 4 Cloud Execution Environment (CEE) 4.1 CEE Architecture 4.1.1 Architecture Overview 4.1.2 OpenStack APIs 4.1.3 VIM Zone (Region) 4.2 Control and Deployment 4.2.1 Resource Management PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 31 4.2.2 CM-HA 4.2.3 Projects (Tenants) 4.2.4 VIM LCM – vFuel 4.2.5 vCICs 4.2.6 CEE Portal (Atlas) 4.3 Compute 4.3.1 Overview 4.3.2 CEE vPOD Compute Usage 4.3.3 Redundancy Compute Usage 4.4 Networking 4.4.1 CEE Networking Domains 4.4.2 CEE Host Networking 4.4.3 CEE Logical Networking 4.5 SR-IOV Design 4.5.1 SR-IOV Concepts 4.5.2 SR-IOV for VIM Zone 4.6 Storage 4.6.1 Block Storage (Cinder) 4.6.2 Image Service (Glance) 4.6.3 Object Storage (Swift) 4.6.4 Local Storage 4.7 Dimensioning Characteristics 4.7.1 Memory Dimensioning 4.7.2 Storage Dimensioning 4.7.3 CPU Dimensioning NFVi HLD Structure 5 Software Defined Storage 5.1 NexetaStor Design Overview 5.2 NexentaStor HA Cluster Architecture 5.3 Cache Read and Write 5.4 NexentaStor Deployment Model 5.4.1 Storage Hardware Configuration 5.4.2 Storage Software 5.4.3 Storage Pool 5.4.4Data Protection 5.4.5 NFS File Systems 5.5 NexentaStor Connectivity 5.5.1 NexentaStor Physical Connectivity 5.5.2 NexentaStor vPOD 5.5.3 NexentaStor Logical Networks 6 NFVi Tenant Networking 6.1 NFVi Networking Overview 6.2 NFVI Tenant Traffic Domain Layers 6.3 VNF Networking de-activated SDN 6.3.1 Limitation 6.3.2 VNF L2/L3 Service Traffic Flows 7 NFVi Operation and Management 7.1 Fault Management 7.1.1 HDS Infrastructure 7.1.2 CEE vPOD 7.1.3 Nexenta vPOD 7.2 Performance Management 7.2.1 HDS Infrastructure 7.2.2 CEE vPOD 7.2.3 Nexenta vPOD 7.3 Log Management 7.3.1 HDS Infrastructure 7.3.2 CEE vPOD 7.3.3 Nexenta vPOD 7.4 Infrastructure Backup & Restore 7.4.1 Infrastructure Backup 7.4.2 Infrastructure Restore 8 NFVi Security 8.1 Security Management 8.2 Identity and Access Management (IdAM) PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 32 8.2.1 HDS Identity and Access Management 8.2.2 CEE Identity and Access Management 8.2.3 Nexenta Identity and Access Management 8.3 Certificate Management 8.4 Network Isolation 8.5 Data Protection 8.6 Node Hardening 9 NFVi Services Integration 9.1 NTP 9.1.1 NTP Hierarchy 9.1.2 NFVi External Time Sources 9.1.3 HDS NTP Establishment 9.1.4 CEE vPOD 9.1.5 Nexenta vPOD Q&A PA1 | 2019-05-22 | VNPT NFVi Solution | Open | Commercial in Confidence | Page 33