Palo Alto Networks SD-WAN REFERENCE ARCHITECTURE GUIDE JUNE 2020 Table of Contents Table of Contents Preface..................................................................................................................................................................... 1 Purpose of This Guide........................................................................................................................................... 3 Audience................................................................................................................................................................................................. 3 Related Documentation....................................................................................................................................................................... 3 Introduction...........................................................................................................................................................4 PAN-OS Secure SD-WAN Design Model........................................................................................................................................... 5 CloudGenix SD-WAN Design Model.................................................................................................................................................. 7 Palo Alto Networks SD-WAN Strategy.............................................................................................................................................. 8 WAN and Remote Site Concepts and Services..................................................................................................9 WAN Overview.......................................................................................................................................................................................9 Internet Access Options for Remote Sites...................................................................................................................................... 17 Secure Access Service Edge...............................................................................................................................................................22 SD-WAN Options................................................................................................................................................................................22 Palo Alto Networks Design Details...................................................................................................................27 New Constructs for PAN-OS Secure SD-WAN...............................................................................................................................28 Managing SD-WAN Deployments with Panorama....................................................................................................................... 37 Security Zones.....................................................................................................................................................................................43 AutoVPN Overlay.................................................................................................................................................................................45 SD-WAN Routing................................................................................................................................................................................48 Securing the Sites.............................................................................................................................................................................. 49 SD-WAN Design Models.....................................................................................................................................52 PAN-OS Secure SD-WAN Design Model.........................................................................................................................................54 CloudGenix with Prisma Access Design Model.............................................................................................................................58 Summary...............................................................................................................................................................62 Palo Alto Networks Preface Preface GUIDE TYPES Overview guides provide high-level introductions to technologies or concepts. Reference architecture guides provide an architectural overview for using Palo Alto Networks® technologies to provide visibility, control, and protection to applications built in a specific environment. These guides are required reading prior to using their companion deployment guides. Deployment guides provide decision criteria for deployment scenarios, as well as procedures for combining Palo Alto Networks technologies with third-party technologies in an integrated design. DOCUMENT CONVENTIONS Notes provide additional information. Cautions warn about possible data loss, hardware damage, or compromise of security. Blue text indicates a configuration variable for which you need to substitute the correct value for your environment. In the IP box, enter 10.5.0.4/24, and then click OK. Bold text denotes: • Command-line commands. # show device-group branch-offices • User-interface elements. In the Interface Type list, choose Layer 3. • Navigational paths. Navigate to Network > Virtual Routers. • A value to be entered. Enter the password admin. Palo Alto Networks 1 Preface Italic text denotes the introduction of important terminology. An external dynamic list is a file hosted on an external web server so that the firewall can import objects. Highlighted text denotes emphasis. Total valid entries: 755 ABOUT PROCEDURES These guides sometimes describe other companies’ products. Although steps and screen-shots were up-to-date at the time of publication, those companies might have since changed their user interface, processes, or requirements. GETTING THE LATEST VERSION OF GUIDES We continually update reference architecture and deployment guides. You can access the latest version of this and all guides at this location: https://www.paloaltonetworks.com/referencearchitectures WHAT’S NEW IN THIS RELEASE • This is a new guide. Palo Alto Networks 2 Purpose of This Guide Purpose of This Guide This guide describes reference architectures for Palo Alto Networks software-defined wide-area network (SD-WAN). This guide includes design guidance for connecting your remote sites to data centers or central sites via SD-WAN, as well as accessing SaaS applications. This guide: • Provides an overview of WAN and remote-site design concepts. • Introduces CloudGenix, a Palo Alto Networks company, and describes CloudGenix SD-WAN. • Describes the technical design aspects of the Palo Alto Networks SD-WAN offerings and explores several technical design models. • Describes the technical concepts for PAN-OS® Secure SD-WAN that enable per-application path selection with path health monitoring. • Provides a framework for SD-WAN architectural discussions between Palo Alto Networks and your organization. • Is required reading prior to using the Palo Alto Networks PAN-OS Secure SD-WAN: Deployment Guide. The deployment guide provides decision criteria for deployment scenarios, as well as procedures for deploying PAN-OS Secure SD-WAN. AUDIENCE This design guide is for technical readers, including system architects and design engineers, who want to deploy the Palo Alto Networks SD-WAN. It assumes the reader is familiar with the basic concepts of applications, networking, routing, security, and high availability, as well as a basic understanding of network and data center architectures. To be successful, you must have a working knowledge of networking and policy in PAN-OS. RELATED DOCUMENTATION The following documents support this architecture guide: • Palo Alto Networks PAN-OS Secure SD-WAN: Deployment Guide—Details deployment scenarios and step-by-step guidance for deploying PAN-OS Secure SD-WAN. • Prisma Access for Networks: Reference Architecture Guide—Presents a detailed discussion of the available design considerations and options for Prisma™ Access for remote networks. • Prisma Access for Networks: Deployment Guide—Details deployment scenarios and guidance for deploying Prisma Access for remote networks. Palo Alto Networks 3 Introduction Introduction This reference architecture describes how SD-WAN allows organizations to use their WAN links more efficiently and gain visibility and control over both remote-site user traffic that is bound for the internet, as well as the organization’s WAN traffic bound for another remote site, large campus, or data center. This reference architecture includes best-practice recommendations that simplify and optimize the WAN design for your organization and concurrently provides secure direct internet access for cloud-based applications. Palo Alto Networks provides organizations with two recommended design models for deploying SD-WAN: • PAN-OS Secure SD-WAN • CloudGenix SD-WAN with Prisma Access In addition to a broad WAN overview, this guide covers the technical aspects of PAN-OS Secure SD-WAN in-depth and describes the best methods for integrating CloudGenix with Prisma Access. You can use this guide to determine which option is most effective in meeting the business requirements of your organization. Palo Alto Networks 4 Introduction PAN-OS SECURE SD-WAN DESIGN MODEL Palo Alto Networks next-generation firewalls running PAN-OS natively include a rich set of security and SD-WAN features that provide the highest level of security, visibility, and control for your organization. PAN-OS version 9.1 introduces a native SD-WAN subscription in order to provide intelligent and dynamic path selection on top of the industry-leading security that PAN-OS already delivers. Beginning with this release, your organization can avoid the growing complexity of a multivendor, multifunction approach by consolidating multiple functions into a single, integrated multifunction security device that includes SD-WAN. The next-generation firewall provides both visibility into the use of applications on the network and the ability to control users’ access to those applications. Key to both visibility and control is the service’s App-ID™ capability. By inspecting the session and payload information of the traffic traversing the firewall, App-ID identifies applications and provides granular application control. This comprehensive visibility is also essential for SD-WAN to provide intelligent per-application path selection. Figure 1 PAN-OS Secure SD-WAN Central Site INET-1 INET-2 MPLS Direct Internet Access Direct Internet Access SD-WAN Remote Sites The PAN-OS Secure SD-WAN solution is orchestrated by Panorama™, which you use to configure and monitor the central-site and remote-site devices. You use templates for network and device configuration and device groups for policy and object configuration including the new PAN-OS version 9.1 SD-WAN specific features. Panorama includes a SD-WAN plugin, which you use to build the IPSec tunnel-based overlay network, configure the dynamic routing between the sites, and provide centralized visibility of the SD-WAN. Palo Alto Networks 5 Introduction In addition to SD-WAN capabilities, Palo Alto Networks next-generation firewalls provide comprehensive security that enables organizations to: • Securely enable users, content, and applications, including SaaS applications, by classifying all traffic irrespective of port. • Reduce risk of an attack by using a positive enforcement model—allowing all desired applications and blocking everything else. • Apply security policies to block known vulnerability exploits, viruses, ransomware, spyware, botnets, and other unknown malware, such as advanced persistent threats. • Apply consistent security across your on-premises and cloud environments, as well as branch locations. Palo Alto Networks 6 Introduction CLOUDGENIX SD-WAN DESIGN MODEL Palo Alto Networks acquired CloudGenix in April 2020. Its cloud-delivered branch capability and application-centric approach will accelerate the Palo Alto Networks vision for the secure access service edge. The CloudGenix Autonomous SD-WAN delivered by CloudGenix Instant-On Network (ION) devices allows you to enforce policies based on business intent, enables dynamic path selection, and provides visibility into performance and availability for applications and networks. You connect your sites using AppFabric, a secure application fabric, which you establish amongst all ION devices. AppFabric creates a VPN over every WAN link. You can define policies that are aligned with your organization’s business intent. Based on the performance, compliance, and security policies for your applications, ION devices automatically choose the best WAN path for your applications. ION devices use the business policy and real-time analysis of the application performance metrics and WAN links to determine the appropriate path. Figure 2 CloudGenix SD-WAN Central Site INET-1 INET-2 MPLS CloudGenix Portal Direct Internet Access Direct Internet Access SD-WAN Remote Sites CloudGenix Portal, fully delivered from the cloud, is the management and monitoring platform for the CloudGenix SD-WAN. The portal manages the policies for application performance, security, and compliance. The ION devices are pre-configured to authenticate to the portal and support zero-touch provisioning and deployment. Palo Alto Networks 7 Introduction PALO ALTO NETWORKS SD-WAN STRATEGY Palo Alto Networks SD-WAN strategy allows you to choose which SD-WAN architecture is right for your organization. We will continue to enhance both the existing CloudGenix SD-WAN technology and PAN-OS Secure SD-WAN technology. This two-pronged strategy for SD-WAN provides you the flexibility to implement SD-WAN on-premises or in the cloud, with your choice of platform. Prisma Access for networks provides security services and threat prevention for your remote networks, safely enabling commonly used applications and web access. You connect remote networks to Prisma Access via an industry-standard IPSec VPN-capable device. You use Panorama to manage Prisma Access for consistency and to enable the full suite of PAN-OS features. Prisma Access is ideally suited for any remote site with one or multiple internet links and provides direct internet access and connectivity to another enterprise remote site through Prisma Access. Prisma Access provides direct internet access without the requirement to backhaul traffic to a central site. Functionally, there is no need to compromise on remote-site security, because Prisma Access provides the exact same security, visibility, and control as provided by the Palo Alto Networks next-generation firewalls at the central site. CloudGenix SD-WAN, even prior to the acquisition by Palo Alto Networks, had a strong integration with Prisma Access. The combination of these two technologies allows you to have a lightweight remote-site footprint while still being able to provide comprehensive security. Figure 3 Prisma Access for networks Cortex Data Lake Data Center (Private Cloud) Prisma Access Service Connection Remote Network Connection Remote Sites Palo Alto Networks 8 Remote Network Connection Remote Network Connection WAN WANand andRemote RemoteSite Site Concepts and Services WAN and Remote Site Concepts and Services Many organizations use different approaches for securing central-site networks as compared to securing remote-site networks. The central site includes a highly visible network perimeter for the organization, which justifies a significant investment in security infrastructure to secure this perimeter. The approach used for the central site usually includes the network security “full stack,” complemented by the presence of on-site security operations staff to manage and respond to security threats. The remote-site approach typically differs in multiple ways. It is rare to have on-site security operations staff at a remote site. There may be hundreds or thousands of remote sites, making it difficult to justify the cost of even a scaled-down “full stack” solution. Most network traffic from a remote site is bound for the central site or the internet. If security concerns are paramount, organizations can mandate that the network forwards all internet traffic to the central site for inspection. To accommodate evolving business requirements, organizations have adopted SaaS applications and moved internal applications and workloads to the public cloud. It is inefficient to rely on a legacy network and security architecture that requires you to send all network traffic to a central site for inspection when users could access many of these same applications more efficiently with direct access through public networks. However, you might increase the overall risk to your organization by permitting access to public networks from a multitude of remote sites without providing security capabilities equivalent or comparable to that of the central site. Some organizations might make a business decision to accept some additional risk to reduce costs. Typically, this approach involves deploying a low-cost unified threat management (UTM) system, which often lacks features and capabilities when compared to the systems deployed at the central site. Another common approach is to enable stateful firewall features in combination with simple network address translation (NAT) on an existing WAN router. To provide the background for the design discussions that follow, this section of the guide provides an overview of commonly used WAN technologies and their evolution. It also discusses available options for securing internet traffic from WAN connected remote sites and the associated pros and cons of each option. WAN OVERVIEW This section provides a brief review of WAN technologies and introduces technology and terms that support the in-depth discussions that follow. The purpose of a WAN is to interconnect local-area networks (LANs) at an organization’s central-site locations, such as headquarters and data centers, as well as remote-site locations. WAN transports and technology have evolved, and in the recent years, multi-protocol label switching (MPLS)-based offerings and the internet have displaced most other options. Most recently, SD-WAN has further continued this displacement, by enabling the commoditization of both MPLS and the internet. Palo Alto Networks 9 WAN and Remote Site Concepts and Services Each WAN technology described has its own pros and cons. Understanding the deficiencies of each and the methods developed to overcome the deficiencies provides the background information for and key drivers of the development of SD-WAN. Figure 4 WAN design Central Site Central Site WAN Remote Site Remote Site Remote Site MPLS VPN The networking industry considers MPLS services a private WAN service, in that the service provider provides segmentation and isolation between customers even though they share a common infrastructure across the provider’s network. The MPLS service provider’s network uses a high-speed core network to interconnect provider edge (PE) devices that the service provider uses for customer access. The service provider can share PE devices across multiple customers; the service provider creates a MPLS virtual private network (VPN) for each customer, which provides logical separation. The PE devices use multiprotocol BGP to distribute routing information for each customer in a MPLS VPN. The central-site and remote-site locations use customer edge (CE) devices to connect to PE devices in the MPLS network. Whereas multiple customers share PE devices, each CE device is dedicated to a single customer. You can use either BGP or static routing for the PE-CE connections. Palo Alto Networks 10 WAN and Remote Site Concepts and Services The MPLS network supports a variety of advanced capabilities, such as QoS, traffic engineering, virtual private LAN service, pseudo wires, and multicast. Customer access to an MPLS network can use traditional time-division multiplexing services (T1, E1, and DS3), but now more typically uses Ethernet attachment at data rates ranging as high as multi-gigabit. Figure 5 MPLS VPN Central Site Central Site MPLS Provider Edge (PE) MPLS VPN Remote Site Remote Site Customer Edge (CE) Remote Site Internet VPN The internet is a public network with no explicit assurance of data privacy. This restriction alone makes the internet unsuitable for use as a private WAN transport, besides the fact that it also lacks the advanced capabilities of MPLS, most importantly QoS. However, when paired with IPSec to create encrypted VPN tunnels, the internet VPN WAN provides a relatively inexpensive and secure alternative to MPLS. Customer access to the internet might use traditional methods, but now, more typically, uses Ethernet, DSL/cable (with Ethernet handoff), or 4G/LTE. Note The term internet VPN implies that the network uses encryption to ensure data privacy and data integrity. The term MPLS VPN only implies that the MPLS service provider keeps customer data logically separate from other customers. If the customer desires encryption, the customer typically implements it on their own on-premises equipment. Palo Alto Networks 11 WAN and Remote Site Concepts and Services Figure 6 Internet VPN WAN Central Site IPSec VPN tunnel Internet Remote Site IPSec VPN tunnel Remote Site Many organizations have now adopted internet VPN for the WAN to some degree. The combined benefits of a relatively inexpensive service with data privacy provided through strong encryption are compelling. Early implementations of internet VPN were not only difficult to manage but also suffered from poor performance due to a variety of reasons, including the lack of hardware accelerated encryption and poorly understood technical characteristics. These technical characteristics include: • Overhead due to tunnel encapsulation—This overhead decreases the maximum transmission unit (MTU) size on the tunnels and often causes IP fragmentation resulting in packet loss. • Crypto-authentication using pre-shared keys—The large-scale usage of pre-shared keys is difficult to manage and the key distribution is cumbersome. Sites with dynamic addressing might require wildcard pre-shared keys, which further magnifies the impact of key rotation. • Crypto-authentication using digital certificates—This method is an industry best-practice, but until streamlined workflows and tools become available to manage the public key infrastructure, organizations often find digital certificates difficult to implement, maintain, and support. • Tunnel instability when using a common routing information base—Keep dynamic routing within the overlay (tunnel) network separate from the routing used to build the overlay, especially when using default routes. Current implementations of internet VPN perform well when you enable advanced features and follow best practices. When considering internet VPN, you should ensure that your devices are able to minimize the impact of IP fragmentation. A key option is Adjust TCP MSS, which you enable on your VPN endpoints in order to ensure that the communication endpoints negotiate the TCP maximum segment size (MSS) to Palo Alto Networks 12 WAN and Remote Site Concepts and Services a value that eliminates the need for fragmentation. The communication endpoints should take advantage of techniques, such as internet control message protocol (ICMP) path MTU discovery, to identify the smallest MTU along a path. Note If you explicitly block all ICMP message types, communication endpoints are unable to determine the most effective MTU by using ICMP path MTU discovery. If you use digital certificates, industry best-practices recommend that, when available, you enable automated processes for certificate enrollment and revocation, such as the simple certificate enrollment protocol and the online certificate status protocol. These automated processes streamline and simplify the overall certificate management workflow. If you choose to use pre-shared keys, use tools to generate strong, random keys and use unique keys for each remote site. If supported, you should also use multiple virtual router instances on your VPN endpoints. The publicfacing (or “front door”) virtual router provides the routing information that the endpoints use to build the VPN tunnels between endpoints. The internal virtual router provides the routing information for traffic passed inside the VPN tunnels. The use of multiple virtual routers separates the routing control planes and permits you to use multiple default routes, while also preventing instability when the VPN endpoint learns duplicate routes through VPN tunnels. In this approach, you assign the public-facing interface of your device to the front door virtual router, which needs only a static default route. All other interfaces on your device, including the VPN tunnels, continue to use the default virtual router. You can implement static or dynamic routing on the default virtual router. Figure 7 Front-door virtual router 40.91.121.247 (eth1) (tunne l.1) VR-Default VR-FrontDoor 10.50.1.0/24 (eth2) Zone: Private Palo Alto Networks Zone: Public IKE gateway inte rface: eth1 Zone: VP N Virtual router: VR-Default Virtual router: VR-FrontDoor Static route: 0.0.0.0/0 next hop = tunnel.1 Static route: 0.0.0.0/0 next hop 40.91.121.247 13 WAN and Remote Site Concepts and Services WAN High Availability Although MPLS and the internet are both highly reliable in most cases, when a failure occurs, the impact is significant. WAN outages and disruptions might occur within a service provider’s backbone. Though these outages are uncommon, they impact multiple customers. More commonly, WAN failures are associated with the local access link for a remote site, in which case the failure impacts only select locations for a few customers. An effective approach to improve the overall reliability of the WAN is to incorporate multiple links and link diversity. You should connect to at least two WAN transports at all of your organization’s business-critical locations. The chosen transports should not share any common infrastructure, such as power, physical media (fiber), or equipment, and they should use different service providers. The cost of improved diversity can be high, so you should compare this cost with the associated business risk of downtime for your organization’s remote sites. After you have two or more paths in your network, you must rely on dynamic routing protocols to select the optimal paths through the network. Dynamic routing protocols use a variety of parameters to calculate a routing table, including link speed, shortest path, or autonomous system. You might use static routing for sites with only a single path, but otherwise you should avoid using static routing. A proper WAN design that uses multiple WAN transports should rapidly detect network failures and dynamically reroute traffic to alternate paths. You typically configure traditional routed WAN networks by using the following methods (assuming only two paths): • Active/standby—Forwards all traffic on a primary path and reroutes traffic to a secondary path during an outage. The standby path is idle at other times. • Active/active—Shares traffic on both paths based on session information. If a path fails, traffic from the failed path is rerouted to the remaining active path. Traffic can always use both paths. Figure 8 WAN Active/standby with MPLS VPN and internet VPN Central Site Active path Standby path Internet MPLS VPN (active) (standby) Remote Site Palo Alto Networks Remote Site 14 WAN and Remote Site Concepts and Services There are pros and cons associated with both methods. You may find it difficult to justify the cost of a WAN transport used only as a standby. Similarly, if you are designing to reduce the impact of an outage, then each path used with the active/active method needs to have enough bandwidth to accommodate all traffic and should never exceed 50% utilization when both paths are active. The design and implementation of WAN diversity is further complicated when mixing and matching different transports, such as MPLS and VPN WAN over internet, that don’t support the same capabilities, such as QoS. Applications that require QoS might continue to operate during a failover scenario when the applications reroute to the internet, but without QoS, they might operate in a degraded mode. Other factors that affect implementing WAN diversity include the complexity of the configuration. Your service provider might require BGP when connecting to an MPLS network, and your VPN WAN might use a different routing protocol, such as OSPF. Effective load-sharing and planning for fault tolerance might not be possible if all paths don’t have the same bandwidth. Software-Defined WAN SD-WAN provides increased flexibility and control of WAN traffic when using multiple WAN transports. In general, SD-WAN solutions enable you to build a WAN overlay network or networking fabric using a variety of WAN transports with different bandwidth and performance characteristics. SD-WAN typically shares the traffic load across all paths in an active/active manner, based on the application. These solutions have centralized virtually all aspects of device management, including device onboarding and key-management workflows, making large-scale deployment relatively seamless. SD-WAN overcomes the limitations of MPLS VPN and internet VPN by providing ubiquitous data encryption, simple configuration, active utilization of all links, and rapid convergence while also reducing cost through the expanded usage of low-cost WAN transports. Figure 9 SD-WAN overlay Central Site SD-WAN Overlay MPLS Internet Remote Site Palo Alto Networks Remote Site 15 WAN and Remote Site Concepts and Services All SD-WAN offerings provide encryption for data privacy with the assumption that one or more transports use a public network. Some offerings might also include additional native security capabilities, but these are rarely more than basic firewalls. A common security strategy for a pure-play SD-WAN vendor is to integrate with a third-party security vendor, such as Palo Alto Networks. If you are using standalone next-generation firewalls, you can place them inline with the SD-WAN device. Similarly, you might deploy a VM-Series firewall on the same host system as the SD-WAN virtual machine. Each site requires an SD-WAN device that connects to the various WAN transports. A centralized SD-WAN management platform configures and manages the SD-WAN devices. Both on-premises and cloud-based management platforms are commonly used. The monitoring and reporting capabilities of the management platform include complete visibility of all connected sites and which transports each uses for access. In many cases, both the WAN terminating devices and management platforms can be virtualized. The SD-WAN devices collect performance data by using passive and/or active monitoring and then use this information to determine the real-time performance characteristics of the SD-WAN paths, such as delay, loss, and jitter. The SD-WAN devices use this performance data to identify faults, such as: • Link failures—The complete failure of an attached link, often detected through a line protocol change. • Blackholes—A partial failure often observed when there is complete packet loss in one direction. • Brownouts—A partial failure often observed with a low-level loss condition (< 10%) in one or both directions. This fault is highly impactful to voice and video applications and might cause retransmission for TCP applications. • Delay—A partial failure often observed when round trip latency increases significantly from the average. This fault is highly impactful to voice and video applications above a certain threshold and might lower throughput for file transfers. • Jitter—A partial failure often observed when round trip latency variance increases significantly from the average. This fault is highly impactful to voice and video applications above a certain threshold. The advanced capabilities of SD-WAN enable an organization to craft routing policies that match applications with strict performance requirements to the best performing path. The SD-WAN device uses the policies along with performance data to control the flow of network traffic through the SD-WAN. Similarly, the SD-WAN device can use the performance data to rapidly reroute traffic around failed or degraded paths. SD-WAN devices can typically build VPN tunnels from any WAN interface. Rather than requiring a front-door virtual router, the device uses interface-specific default routes. These provide the routing information that the endpoints use to build the VPN tunnels between endpoints. Palo Alto Networks 16 WAN and Remote Site Concepts and Services The SD-WAN overlay network integrates with central-site and remote-site networks using static routing or standard routing protocols, typically BGP and OSPF. However, the SD-WAN overlay network itself may use routing protocols or a centralized SD-WAN controller to distribute routing information. Proprietary methods, such as probes, are used to support application-specific routing and rapid failure detection. Traditional routing protocols, such as BGP, are typically unable to identify partial failures, such as blackholes or brownouts, within the WAN because their convergence mechanisms are dependent on the loss of multiple hello messages. Aggressive tuning of routing protocol timers might improve detection times for some outage events, but otherwise SD-WAN fault detection is superior to the routing protocol based approaches used by other WAN architectures in all cases. INTERNET ACCESS OPTIONS FOR REMOTE SITES Remote sites use the WAN to communicate with headquarters, data centers, and other remote-site locations. The networking industry typically consider WANs a private network, although only encryption provides true data privacy across a service-provider network. It is straightforward to encrypt data communication over the WAN; most approaches simply encrypt all traffic leaving the site without regard for the applications in use or communication endpoints. This approach extends the security perimeter to include the remote sites and is effective for ensuring data privacy for network traffic internal to the organization. You must take additional steps when considering network traffic that leaves your organization. External connections might include communication to business partners, cloud-delivered services, and general web access. Encryption alone is not sufficient to secure external communications. To protect your organization properly, you must have full visibility of the users, applications, and content traversing corporate networks, the cloud, and endpoints. Only then is it possible to implement security policies and take actions, such as blocking unknown traffic, identifying advanced attacks, or permitting only the applications that have a valid business purpose. Along with encryption, you should also use nextgeneration firewalls to provide inspection and visibility for traffic entering and leaving the network. When considering external communications, there are a variety of options for remote sites to access the internet for general web access, as well as SaaS and other cloud-delivered services. Palo Alto Networks 17 WAN and Remote Site Concepts and Services Centralized Internet Access All internet traffic is backhauled to a central site, such as headquarters or a data center location, across the private WAN. This location then provides secure access to the internet for the remote sites using the same security infrastructure that secures the central site. This option reduces or eliminates the need for network security infrastructure at the remote sites. You secure each remote site with the network security infrastructure at the central site. This option, however, uses the central site’s network bandwidth inefficiently, because you process internet bound traffic from remote sites twice: once as it travels across the WAN from the remote site, and then again as the traffic travels to the internet. Increased latency might degrade the user experience at your remote site due to increased latency when accessing regional resources if the central site is far from your remote site. Figure 10 Centralized internet access Central Site Central-site traffic Internet traffic Private WAN Internet Remote Site Palo Alto Networks 18 WAN and Remote Site Concepts and Services Customer-Provisioned, Cloud-Based Internet Access With this option, you transport all internet traffic across the private WAN to the closest cloud-based internet access facility. These internet access facilities contain the same stack of security infrastructure that you would use in the central site. Because you can deploy multiple cloud-based facilities in a given geographic region, you can reduce the latency of backhauling internet bound traffic. These internetaccess facilities are typically based in carrier facilities or carrier neutral facilities and have access to the service provider transport. The customer rents rack-space and power for the security infrastructure equipment. The customer typically owns the security infrastructure equipment and is responsible for the cost of administrating, configuring, maintaining, and monitoring the equipment. Figure 11 Customer-provisioned, cloud-based internet access Cloud-Based Security Stack Central Site Private WAN Internet Central-site traffic Internet traffic Remote Site Direct Internet Access All internet-based application traffic travels directly to the application’s destination without traversing the private WAN. Each remote-site location provides local secure access to the internet with its own security infrastructure. This option is costly both from the network security infrastructure perspective and the management and operational perspective, especially as the total number of remote sites increases. This option also uses the central-site’s bandwidth efficiently because only traffic destined to the central site consumes bandwidth. Latency does not affect the user experience when accessing regional resources, regardless of the distance to your central site. Primarily due to cost, when you choose to enable direct internet access (DIA), you might decide to implement a different security solution than at the central site or implement only a limited subset of functions, which might increase the risk to your organization. However, the Palo Alto Networks recommendation is to provide consistent visibility by using the comprehensive protection of a Palo Alto Networks 19 WAN and Remote Site Concepts and Services next-generation firewall. An organization is unable to protect against what it cannot see. As previously mentioned, to protect your organization properly, you must have full visibility of the users, applications, and content traversing corporate networks, the cloud, and endpoints. Figure 12 Direct internet access Central Site Central-site traffic Internet traffic Private WAN Internet Remote Site Trusted Direct Internet Access Only internet-based application traffic, for trusted locations and SaaS applications, travels directly to the application’s destination without traversing the WAN. Each organization must determine which applications to trust, such as Office 365, G Suite, or Box. Each remote-site location provides local secure access for the trusted internet traffic with its own security infrastructure. This method forwards all other internet traffic to a central site, such as headquarters or a data center location, across the private WAN, which then provides secure access to the internet for the remote sites using the same security infrastructure that secures the central site. This option is a compromise that provides low-latency access to trusted resources that often have regional points of presence and concurrently reduces the total amount of traffic that the network must forward to the central site. Each remote-site location provides local secure access to the trusted locations and applications with its own security infrastructure. This option relies upon the security infrastructure to properly identify and classify trusted locations and applications and apply any additional user-based policies. This option is costly both from the network security infrastructure perspective and the management and operational perspective, especially as the total number of remote sites increases. Palo Alto Networks 20 WAN and Remote Site Concepts and Services This option uses the central site’s bandwidth more efficiently than the centralized internet access option, because only traffic destined to the central site and untrusted traffic to the internet consumes bandwidth. Remote-site to central-site latency does not affect the user experience when accessing trusted in-region resources, regardless of the distance to your central site. Figure 13 Trusted direct internet access Central Site Central-site traffic Internet traffic Trusted internet traffic Private WAN Internet Internet Remote Site Primarily due to cost, you might decide to implement a different security solution at your remote site than at the central site or implement only a limited subset of functions such as those provided by a UTM platform. This decision might seem somewhat more acceptable due to the trusted nature of the applications you choose to permit; however, this option might still increase the risk to your organization. A potential threat exists even with trusted applications. Although rare, hackers might have compromised trusted applications and converted them into malware delivery platforms. The key principle of a Zero Trust architecture is “never trust, always verify.” The most effective way to verify is to inspect and log all traffic. As mentioned earlier in this guide, the Palo Alto Networks recommendation is to provide consistent visibility—an organization is unable to protect against what it cannot see. We suggest you provide consistent visibility by using the comprehensive protection of a next-generation firewall. Palo Alto Networks 21 WAN and Remote Site Concepts and Services SECURE ACCESS SERVICE EDGE As the enterprise evolves, and technology shifts where applications reside and how customers and employees access them, networks must also evolve to support these changes. Digital transformation improves agility and your ability to compete; however, it also challenges how you connect to the data as well as how you secure the data, whether on-premises or in the cloud. A new concept, secure access service edge (SASE), has emerged to describe this convergence of technologies required to secure the networks for organizations. SASE describes a category of services and products that offer transport and security in a cloud-native converged and managed solution. The common services include WAN edge/SD-WAN, secure web gateway, cloud access security broker, Zero Trust network access, DNS security, and firewall as a service. The elements in an SASE-enabled network are: • Converged WAN edge and network security—Provides true integration of services, not service chains, with combined services and visibility for all locations, mobile users, and cloud. • Cloud-native cloud-based delivery—Uses many points of presence to reduce latency with support of in-country or in-region resources and regulatory requirements. • Broad network edge support—Takes you beyond box-based access support with agent-based capability managed as a cloud service. • Identity and network location—Provides policy enforcement beyond IP address by using identitybased policy enforcement with real-time conditions like device-type, posture, and location. The shift to SASE forces existing networking and security models to evolve to integrated, cloud-delivered services. Prisma Access provides an SASE-enabled service with over 100 locations and the ability to fully migrate the WAN and security services to a cloud-native architecture. SD-WAN OPTIONS As organizations adopt SD-WAN technology to replace traditional WAN, they include remote-site internet access as a key design consideration. SD-WAN, like other internet VPN solutions, extends the security perimeter to include the remote sites and is effective for ensuring data privacy for network traffic internal to the organization. You must take additional steps when considering network traffic that leaves your organization. External connections might include communication to business partners, cloud-delivered services, and general web access. This section compares the design options for SD-WAN with DIA. When considering external communications, there are a variety of methods for remote sites to access the internet for general web access, as well as SaaS and other cloud-delivered services. This discussion assumes that one or more WAN transports for the SD-WAN uses a public network. Palo Alto Networks 22 WAN and Remote Site Concepts and Services SD-WAN Standalone with Direct Internet Access The remote site connects to the SD-WAN by using an SD-WAN appliance. All central-site application traffic travels directly to the application’s destination by using the SD-WAN, which provides intelligent path-selection. All internet-based application traffic travels directly to the application’s destination without traversing the SD-WAN and is instead forwarded directly to a public network through the SD-WAN appliance. Figure 14 SD-WAN standalone with DIA Central Site Central-site traffic Internet traffic SD-WAN Internet Remote Site Each remote-site location provides local secure access to the internet by using only the native security capabilities of the standalone SD-WAN appliance. No additional security resources are located at the remote site, which reduces cost and simplifies the remote-site management. This option uses the centralsite’s bandwidth efficiently because only traffic destined to the central site across the SD-WAN consumes bandwidth. Remote-site to central-site latency does not affect the user experience when accessing regional resources, regardless of the distance to your central site. The security capabilities of SD-WAN appliances are not typically equivalent to the comprehensive protection of a next-generation firewall. An SD-WAN appliance may implement only a limited subset of functions, which might increase the risk to your organization. The Palo Alto Networks recommendation is to provide consistent visibility by using the comprehensive protection of a next-generation firewall. An organization is unable to protect against what it cannot see. To protect your organization properly, you must have full visibility of the users, applications, and content traversing corporate networks, the cloud, and endpoints. Only then is it possible to implement security policies and take actions, such as blocking unknown traffic, identifying advanced attacks, or permitting only the applications that have a valid business purpose. Palo Alto Networks 23 WAN and Remote Site Concepts and Services SD-WAN with Dedicated Security DIA This option is similar to the previous option, SD-WAN Standalone with Direct Internet Access, but each remote-site location provides local secure access to the internet by using the comprehensive protection of a next-generation firewall deployed in-line with the SD-WAN appliance. Figure 15 SD-WAN with dedicated security Central Site Central-site traffic Internet traffic SD-WAN Internet Remote Site The next-generation firewall provides full visibility of the users, applications, and content traversing corporate networks, the cloud, and endpoints. With this visibility, you can implement security policies and take actions, such as blocking unknown traffic, identifying advanced attacks, or permitting only the applications that have a valid business purpose. You may also use the next-generation firewall to reduce the attack surface at your remote site through segmentation. To reduce the scope of compliance and to limit data exfiltration, use segmentation to partition the network into manageable, secure parts. As in the previous option, this option continues to use the central-site’s bandwidth efficiently because only traffic destined to the central site across the SD-WAN consumes bandwidth. Remote-site to centralsite latency does not affect the user experience when accessing regional resources, regardless of the distance to your central site. However, in this option, you have additional security resources located at the remote site. The requirement to separately manage both an SD-WAN device and a security device increases cost and complicates the remote-site management. Palo Alto Networks 24 WAN and Remote Site Concepts and Services Next-Generation Firewall SD-WAN with DIA This option is similar to the previous option, the SD-WAN Standalone with Dedicated Security DIA, but does not require the SD-WAN appliance. In this option, the next-generation firewall provides SD-WAN capabilities in addition to comprehensive security protection. The SD-WAN features on the nextgeneration firewall use existing advanced security capabilities such as App-ID to provide per-application intelligent path selection. Figure 16 Next-generation firewall with SD-WAN Central Site Central-site traffic Internet traffic SD-WAN Internet Remote Site The next-generation firewall provides full visibility of the users, applications, and content traversing corporate networks, the cloud, and endpoints. With this visibility, you can implement security policies and take actions, such as blocking unknown traffic, identifying advanced attacks, or permitting only the applications that have a valid business purpose. You can also use the next-generation firewall to reduce the attack surface at your remote site through segmentation. To reduce the scope of compliance and to limit data exfiltration, use segmentation to partition the network into manageable, secure parts. As in both of the previous options, this option continues to use the central-site’s bandwidth efficiently because only traffic destined to the central site across the SD-WAN consumes bandwidth. Latency does not affect the user experience when accessing regional resources, regardless of the distance to your central site. However, in this option, you have a single device at the remote site providing both security and SD-WAN functions, which decreases cost and simplifies the remote-site management. Palo Alto Networks 25 WAN and Remote Site Concepts and Services SD-WAN Standalone with Prisma Access DIA With this option, you connect your remote site to Prisma Access by using an IPSec tunnel from the standalone SD-WAN appliance. You use this tunnel to transport all internet traffic to Prisma Access. All central-site application traffic travels directly to the application’s destination by using the SD-WAN, which provides intelligent path selection. Figure 17 SD-WAN standalone with Prisma Access DIA Central Site Central-site traffic Internet traffic Internet SD-WAN Prisma Access Remote Site Prisma Access for networks provides security services, such as App-ID and threat prevention, for your remote networks, safely enabling commonly used applications and web access. Prisma Access provides direct internet access without the requirement to backhaul traffic to a central site. Functionally, there is no need to compromise on remote-site security because Prisma Access provides the exact same security, visibility, and control as provided by next-generation firewalls. Palo Alto Networks 26 Palo Alto Networks Design Details Palo Alto Networks Design Details PAN-OS version 9.1 introduces a number of new SD-WAN features that are included with your SD-WAN subscription. This section provides an in-depth discussion of the SD-WAN capabilities of the nextgeneration firewall and the design considerations associated with these capabilities. You can configure SD-WAN hub or branch functions on your devices after adding the SD-WAN subscription to any of the next-generation firewalls or VM-Series models listed in Table 1 and Table 2. These tables indicate a suggested role of branch or hub for each device model, but you may use any device model in either role as long as the device model meets the performance requirements for the location. For the latest platform and performance specifications, see the SD-WAN datasheet. Table 1 SD-WAN remote-site branch device specifications PA-220/ PA-220R PA-820 PA-850 PA-3220 PA-3250 PA-3260 Aggregate WAN bandwidth (recommended range) 1-150 Mbps 50-500 Mbps 50-700 Mbps 500-2000 Mbps 1000-3100 Mbps 2000-5000 Mbps Maximum IPSec tunnels 1000 1000 1000 2000 3000 3000 Maximum concurrent sessions 64,000 128,000 196,000 1,000,000 2,000,000 3,000,000 Maximum IPv4 routes 2500 10,000 10,000 28,000 88,000 88,000 Table 2 SD-WAN central-site hub device specifications PA-5220 PA-5250 PA-5260 VM-300 VM-500 Aggregate IPSec bandwidth 8 Gbps 16 Gbps 24 Gbps 1.8 Gbps 4 Gbps Maximum IPSec tunnels 3000 4000 5000 1000 1000 Maximum concurrent sessions 4,000,000 8,000,000 32,000,000 819,200 2,000,000 Maximum IPv4 routes 200,000 200,000 200,000 2000 4000 Application-aware next-generation security is naturally complemented by networking functions like SD-WAN application-aware routing. The approach of running both SD-WAN and security on an integrated branch device results in more efficient management across networking and security, with a lower chance of misconfiguration compared to a multi-device approach. The next-generation firewalls continue to use IPSec to provide a WAN overlay tunneling technology coupled with strong encryption for data privacy. The firewalls extend the use of application and protocol decoding within App-ID to rapidly and accurately identify applications running across the WAN. Other new capabilities specific to SD-WAN include: path-health monitoring for key path metrics such as latency, jitter and loss, and per-application policies for path selection and traffic distribution. The next section discusses new features and capabilities in-depth. Palo Alto Networks 27 Palo Alto Networks Design Details NEW CONSTRUCTS FOR PAN-OS SECURE SD-WAN PAN-OS 9.1 adds several new constructs that are required to enable SD-WAN on your devices: • SD-WAN interface profile—Characteristics and parameters that define the SD-WAN behavior for physical ethernet interfaces • SD-WAN virtual interfaces—Logical grouping of Ethernet interfaces or IPSec tunnel interfaces used for SD-WAN • SD-WAN path quality profiles—Metrics and acceptable performance limits required for various application classes using the SD-WAN • SD-WAN traffic distribution profiles—Path selection methods used by SD-WAN policies • SD-WAN traffic policy rules—Rules that match security zones, IP addresses and applications which are then used to apply the SD-WAN traffic distribution profiles SD-WAN Interface Profile The SD-WAN interface profile defines the link type, link tag, upload/download bandwidth speed of the link, and whether you should use aggressive or relaxed path monitoring for the link. After you have enabled SD-WAN for an interface, you must associate the interface profile with the interface. For path selection ordering, you use the link tag in the traffic distribution profile. Link tags also group several interfaces for session load-sharing or link bundling. If multiple interfaces are assigned SD-WAN interface profiles with the same link tag, those links are logically bonded together for traffic distribution. The Panorama SD-WAN plugin uses the link-type information to determine which device peers are used for a link’s overlay network. If you designate a device interface as connected to any public link type (ADSL/DSL, cablemodem, Ethernet, fiber, LTE/4G, or WiFi), then the SD-WAN plugin configures the device to connect to the overlay network for that link type. If a device has multiple interfaces with the same link class, each interface has a connection to the overlay network. The overlay network is composed of IPSec point-to-point tunnels. SD-WAN hub devices at central sites use passive configuration for IPSec tunnels and do not initiate connections. SD-WAN branch devices at remote sites use dynamic peer configuration for IPSec tunnels to central sites. Note PAN-OS version 9.1 software supports only hub-and-spoke topology for an overlay network. Multiple hubs are supported. Palo Alto Networks 28 Palo Alto Networks Design Details Table 3 SD-WAN link types Path monitoring method (default) Link type Link class Tunnel peers ADSL/DSL public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Aggressive Cablemodem public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Aggressive Ethernet public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Aggressive Fiber public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Aggressive LTE/3G/4G/5G public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Relaxed MPLS private (MPLS) MPLS Aggressive Microwave/Radio private (microwave/radio) Microwave/Radio Aggressive Satellite private (satellite) Satellite Relaxed WiFi public ADSL/DSL Cablemodem Ethernet Fiber LTE/3G/4G/5G WiFi Aggressive Other private (other) Other Aggressive Palo Alto Networks 29 Palo Alto Networks Design Details When you select the link type, a default path monitoring method is automatically selected. Path monitoring is used to evaluate path health based on delay, loss, and jitter metrics, which are calculated from data collected from traffic probes. The path monitoring assumes that you have a Palo Alto Networks SD-WAN device terminating the IPSec tunnels at both ends of a connection. The device, when configured to use aggressive mode, sends probe packets at a constant frequency of 5 probes per second. The device, when configured to use relaxed mode, sends probe packets at a frequency of 5 probes per second for a duration of 7 seconds and then pauses probing for an idle period of 60 seconds before restarting the probes. You may override the default path monitoring method. Using aggressive mode detects degraded paths quickly, but it also consumes more bandwidth for probes. Using relaxed mode consumes less bandwidth but also takes longer to detect a degraded path. The probe frequency is configurable for both modes, and the idle period is configurable for relaxed mode. Note If you modify the default probe frequencies, you have to adjust your packet loss thresholds. The packet loss calculation is based on the default probe frequencies. If you reduce the probe frequency, it may take longer to detect a loss condition. Because the application lifetimes may be relatively short lived, you should reduce the packet loss threshold in order to increase the likelihood of detecting a lost probe. Example: Probe frequency set to default 5 probes/sec. Packet loss threshold for application set to 5%. If you reduce probe frequency to 2 probes/sec, you should reduce the packet loss threshold for that application to 2%. One of the primary benefits of SD-WAN is the ability for your devices to move traffic flows from degraded paths to better-performing paths. Ideally, when a degraded path has recovered, your device reinstates flows to that path. To avoid the risk of path flapping, where traffic is continuously moved from path to another, the SD-WAN device enforces a fallback hold time. The hold timer ensures that a path has remained healthy for 120 seconds, before the SD-WAN devices reinstates flows to that path. The hold timer is configurable. Palo Alto Networks typically recommends that you use a unique SD-WAN interface profile and link tag for each physical interface. This approach gives you the greatest amount of granularity in crafting your SD-WAN policies. However, if you have a set of physical interfaces that are functionally equivalent, you may share a single link tag across the set of interfaces. This approach allows you to emulate a link bundle and simplifies the configuration required for basic load-sharing. Palo Alto Networks 30 Palo Alto Networks Design Details SD-WAN Virtual Interface You can configure the SD-WAN virtual interface (VIF) to contain either SD-WAN–enabled physical interfaces or IPSec tunnel interfaces as group members. All group members must be of the same type. Figure 18 SD-WAN virtual interfaces Ping probes (to default-gateway) Path-monitoring probes (end-to-end) INET-1 (eth1/1) VPN VIF INET-2 (eth1/2) DIA VIF IPSec tunnels VPN VIF The VIF performs several key functions on an SD-WAN device: • Provides a Layer 3 virtual interface to be used as an IP forwarding table destination interface and decouples the underlying physical or tunnel interfaces from the Layer 3 IP-forwarding decisions • Generates path-monitoring probes for each group member • Enables physical interface group members to maintain an interface-specific default gateway without requiring a specific entry in the IP forwarding table • Performs intelligent traffic distribution across group member interfaces, depending on the traffic distribution profile In most cases, the Panorama SD-WAN plugin automatically creates the SD-WAN VIFs. Table 4 VIFs created by SD-WAN plugin DIA VIF configuration VPN VIF configuration Single DIA VIF that includes all SD-WAN– enabled physical interfaces as group members One VPN VIF for each remote site. Each includes all IPSec tunnels to a particular remote site as group members Single DIA VIF that includes all SD-WAN– enabled physical interfaces as group members One VPN VIF for each central site. Each includes all IPSec tunnels to a particular central site as group members Central site Remote site Palo Alto Networks 31 Palo Alto Networks Design Details On all SD-WAN devices, the plugin creates a single VIF that includes the physical interface group members. This VIF is typically used for DIA and is commonly referred to as a DIA VIF. The plugin also creates a static default route with this VIF as a destination using a route metric of 5. The physical interface group members must each have a default gateway that is either statically configured or assigned as a DHCP option. Note The SD-WAN plugin automatically includes private interfaces, such as MPLS, as DIA VIF group members. You should exclude the private interfaces from your SD-WAN traffic distribution profile for DIA. In some cases, you may also need to manually create a VIF with physical interface group members before completing the SD-WAN configuration with the Panorama SD-WAN plugin. In these cases, when configuring a static default route with the manually created VIF as a destination interface, use a route metric greater than 5 so that this route is less preferred compared to the static default route created by the plugin. The DIA VIF performs path monitoring, like other VIFs, but does not have an explicit peer to probe. When the DIA VIF is created, it sends ping probes to the interface-specific default gateways. At least one of the default gateways must respond; otherwise the DIA VIF itself is considered down. The SD-WAN devices do not collect any performance metrics from the ping probes. Note Verify that the default gateways for your physical interfaces respond to ICMP echo requests. If an interface does not respond to an ICMP echo request, the DIA VIF marks the physical interface as DOWN and does not use it to forward traffic. On central-site SD-WAN hub devices, the plugin creates a VPN VIF for each remote site, with the IPSec tunnels that connect to that remote site as group members for the VIF. On remote-site SD-WAN devices, the plugin creates a VPN VIFs for each central site, with the IPSec tunnels that connect to the central site as group members for the VIF. Each IPSec tunnel for a VPN VIF is considered to be a potential path to a destination site. The VPN VIFs perform path monitoring by sending path-monitoring probes through the tunnel. Probes are sent across each active IPSec tunnel to the far-end peer. The SD-WAN devices collect path-monitoring probe data that is used to calculate path performance metrics used for path selection. Palo Alto Networks 32 Palo Alto Networks Design Details Note After an IPSec tunnel for a VPN VIF changes to an UP state and path monitoring is active, the DIA VIF suspends its ping path monitoring. Each physical interface group member in the VIF inherits the path monitoring state of the IPSec tunnel sourced from that interface. If multiple tunnels are sourced from that interface, for the path monitoring state, the physical interface group member uses the average of all tunnels sourced from that interface. The SD-WAN device uses VIFs to simplify IP routing. Without the VIF logical interface, if your device has multiple paths to a destination, then its virtual router contains multiple entries in the routing table. The best path in the routing table is installed in the forwarding table, or if equal-cost multipath (ECMP) is enabled, the virtual router performs load sharing across these paths, and multiple paths are installed in the forwarding table. You may choose the load-sharing method used by ECMP, but none of the methods use real-time performance metrics like you could with SD-WAN. By using a VIF, the virtual router requires only a single entry in the routing and forwarding tables. After traffic is forwarded to the VIF, it determines which path to follow based on the SD-WAN policy, traffic distribution profile and real-time conditions of each path. The VIF can add active member interfaces when their state changes to UP and remove inactive member interfaces when their state changes to DOWN. The virtual router does not experience any changes to its routing and forwarding tables as these interface transitions occur. The DIA VIF simplifies another routing issue related to the treatment of default routes with DHCP assigned default gateways. If you have two interfaces that are both using DHCP address assignment, and the DHCP servers are also configured to provide the default gateway using the router option, this results in two equal cost paths, with each path through a different interface. In this case, SD-WAN seamlessly solves this issue. If you configure your SD-WAN device so that the default route is not installed in the routing table, the DIA VIF transparently retains each interface-specific default gateway. You can then use the DIA VIF as the destination interface for a default route, without needing to specify a next-hop IP address for the static route. After the VPN VIFs are created and the tunnels are active, the SD-WAN devices can apply the SD-WAN policies and select the best paths per-application based on current real-time conditions of the SD-WAN. SD-WAN Path-Quality Profiles Each SD-WAN path-quality profile (PQP) is composed of a set of ranked metrics and thresholds that an application requires for optimal performance. You typically use a unique PQP for each set of businesscritical and latency-sensitive applications. The PQP is one of the key components of the SD-WAN path selection logic. The SD-WAN path-monitoring probes are used to calculate the PQP metrics for a path. Each IPSec tunnel in a VPN VIF measures and calculates its metrics independently. Palo Alto Networks 33 Palo Alto Networks Design Details The metrics each PQP uses are: • Latency—The round-trip delay time for a data packet sent across a SD-WAN tunnel (measured in ms). By default, the latency is calculated every 200ms by using a sliding window that includes the last three measurements. • Jitter—The variance in the delay compared to the average delay for a data packet sent across a SD-WAN tunnel (measured in ms). By default, the jitter is calculated every 200ms by using a sliding window that includes the last three measurements. • Packet loss—The percentage of packets lost compared to total packets sent across a SD-WAN tunnel. The packet loss is calculated every 200ms using a sliding window that includes the last 100 measurements. The primary function of the PQP is to determine if a path is qualified (operating normally) or is disqualified (operating in a degraded mode). The secondary function of the PQP is to determine which path to use when more than one path is qualified, and these paths are otherwise considered equivalent by the path selection algorithm. The threshold for each metric is considered a maximum upper limit for that metric. If your SD-WAN device measures the value of any PQP metric for a path and the value exceeds the configured threshold, the path is disqualified. When a link is disqualified, this triggers your SD-WAN device to select an alternate qualified path. If a set of paths are all qualified, then the SD-WAN device’s path selection algorithm considers the sensitivity of the metrics to determine the best available path. You can select from three sensitivity settings to rank the importance of a metric: low, medium, or high. If you configure a metric to have a higher sensitivity than the other metrics, then the path selection algorithm considers that metric first. If all metrics have the same value, then the path selection algorithm first considers packet loss, followed by latency, and then jitter. Palo Alto Networks 34 Palo Alto Networks Design Details SD-WAN Traffic Distribution Profiles An SD-WAN policy rule defines the criteria used by the SD-WAN path-selection algorithm. Each policy rule includes both a PQP and a traffic distribution profile (TDP), which are evaluated concurrently to determine the path for an application. Note When configuring a TDP, you use link tags instead of SD-WAN interface profile names. If you are using unique SD-WAN interface profiles and link tags for each physical interface, the link tag corresponds to all paths sourced from that physical interface. You can use the tags to differentiate between the physical interfaces in your SD-WAN policy. If you are using a shared SD-WAN link tag that is assigned to multiple physical interfaces, the link tag corresponds to all paths sourced from any of the physical interfaces. You cannot differentiate between the physical interfaces in your SD-WAN policy. As part of your SD-WAN policy, you use SD-WAN TDPs to assign which traffic distribution method the path-selection algorithm uses for an application. The path-selection algorithm uses the path metrics, thresholds, and sensitivities as defined in the PQP referenced in the SD-WAN policy rule. When you configure your TDP, you can choose from the following traffic distribution methods: • Best available path—This traffic distribution method requires you to create a list of permitted link tags. When using this method, the path-selection algorithm uses the PQP exclusively to choose the best path from the list of permitted paths that match the link tags. Only qualified paths are eligible for selection. If multiple paths are qualified, the path-selection algorithm uses the sensitivity value assigned to the metrics to prioritize which metric is used first in the selection process. You typically choose this method when link cost is not a factor for the application. If the path conditions change and the best available path is disqualified, the path selection algorithm moves affected sessions to the best qualified path remaining. When path conditions change and the best available path is again qualified and the Fallback Hold Time expires, the path selection algorithm moves affected sessions back to the best available path. • Top-down priority—This traffic distribution method requires you to create a prioritized list of permitted link tags. When using this method, the path-selection algorithm chooses a qualified path that matches the highest priority link tag. You typically use this method when there is a significant cost differential between paths. You assign a low priority to paths with high-cost link tags so the more expensive links are used only when paths with low-cost link tags are disqualified. If the path conditions change and the highest priority path is disqualified, the path selection algorithm moves affected sessions to the next-highest priority qualified path remaining. When path conditions change and the highest priority path is again qualified and the Fallback Hold Time expires, the path selection algorithm moves affected sessions back to the highest priority path. Palo Alto Networks 35 Palo Alto Networks Design Details • Weighted session distribution—This traffic distribution method requires you create a list of permitted link tags and to assign a percentage weight to each entry in the list. The total percentage allocated across all link tags must add to 100. When using this method, the path-selection algorithm performs session distribution across paths for each link tag, the paths do not need to be qualified. When a link tag reaches its percentage limit, further sessions are distributed only to paths with link tags that remain under the percentage limit. You typically use this method for non-critical applications. This traffic distribution method does not rely on path metrics. Note The Weighted Session Distribution method does not reroute existing sessions when a path is disqualified. Existing sessions are only rerouted when their path state changes to DOWN. Palo Alto Networks does not recommend using this method for businesscritical applications. If you configure multiple physical interfaces with the same SD-WAN interface profile and link tag, then you adjust the behavior of these traffic distribution methods. For each method, if the path-selection algorithm chooses the shared link tag, it distributes sessions evenly across all paths with the link tag. In addition to the behavior of the various traffic distribution methods that was previously discussed, the path-selection algorithm also has implicit behavior for the following special cases: • Multiple physical interfaces with the same link tag—This is a special case in which multiple physical interfaces share the same SD-WAN link tag. In this case, if the path-selection algorithm chooses this link tag, it distributes sessions evenly across all qualified paths with the link tag. • No qualified paths exist—This is a special case that commonly occurs in the early stage of an SD-WAN deployment. If you don’t know the current loss, jitter, and latency for your WAN paths and you configure the PQP thresholds too low, this case may occur. In this case, the path-selection algorithm chooses the path that has the best metrics. • No matching SD-WAN policy—This is a special case that commonly occurs in the early stage of an SD-WAN deployment. If you don’t include SD-WAN policy rules for all applications running across the WAN this case may occur. In this case, the path-selection algorithm distributes the sessions that do not match any of the SD-WAN policy rules across all available paths, including high-cost paths. Note Rather than relying on the implicit round-robin method for unmatched sessions, Palo Alto Networks recommends creating an “Unmatched” SD-WAN policy rule that matches any application with any service. You should create a PQP with high thresholds for all metrics and then reference this PQP within the policy rule. You can use any TDP that you prefer with this policy rule. • All VIFs are down—This is a special case that occurs only rarely, such as during a widespread network outage. In this case, the device no longer follows SD-WAN policy rules and falls back to path selection based on traditional IP routing if an active path still exists. Palo Alto Networks 36 Palo Alto Networks Design Details SD-WAN Policy Rules SD-WAN policy rules are similar to security policy rules, which makes SD-WAN policies easy to configure. Both rule types match on source zone/address/user, destination zone/address, and application/service. However, the security policies are evaluated first before applying the SD-WAN policy. If an application is permitted by the security policy, the SD-WAN policy then processes the application. The SD-WAN policy rules are not processed for denied applications or flows. After an application matches an SD-WAN policy rule, the path-selection algorithm evaluates each path from the TDP against the real-time path conditions and thresholds in the PQP to determine the appropriate path. Note Palo Alto Networks SD-WAN automatically enables the Symmetric Return feature in order to avoid asymmetric traffic flows. You need to configure an SD-WAN policy only at the site that initiates the connection. If you initiate a connection from a remote site to a central site and the SD-WAN policy at the remote site assigns the flow to an SD-WAN path, the central-site device automatically uses the same path for the return traffic. MANAGING SD-WAN DEPLOYMENTS WITH PANORAMA The best method for ensuring up-to-date firewall configuration is to use Panorama for central management of firewall policies. Panorama simplifies consistent policy configuration across multiple independent firewalls through its device group and template stack capabilities. When multiple firewalls are part of the same device group, they receive a common ruleset. Because Panorama enables you to control all of your firewalls—whether they be on-premises or in the public cloud, a physical appliance or virtual—device groups also provide configuration hierarchy. With device group hierarchy, lower-level groups include the policies of the higher-level groups. This allows you to configure consistent rulesets that apply to all firewalls, as well as consistent rulesets that apply to specific firewall deployment locations such as the public cloud. Your SD-WAN configuration is simplified by using a minimum of two sets of templates and device groups. Use one set for central-site hub devices and a second set for remote-site devices. If you have remote sites grouped across multiple regions, you may want to separate your remote-site devices across additional templates and device groups. Panorama and the SD-WAN Plugin In addition to the traditional Panorama capabilities based on device group and template stacks, the SD-WAN plugin for Panorama also provides advanced configuration, monitoring, and reporting for SDWAN. Although the SD-WAN plugin supports the configuration of existing brownfield devices, this guide only covers the design details for new deployments. Palo Alto Networks 37 Palo Alto Networks Design Details After you add your central-site and remote-site devices to Panorama and associate them with the proper template stacks and device groups, you complete the device configurations using the SD-WAN plugin for Panorama. As you add devices to the SD-WAN plugin, you must assign the devices to a site and designate the site type as either Hub or branch. The site type determines which other sites to configure as IPSec peers. Palo Alto Networks recommends that you enable dynamic routing with BGP for the site. As part of the BGP configuration, you must provide the following site-specific information: BGP router-ID, IPv4 loopback address, and BGP autonomous system number. On central-site hubs, you must also provide a list of IPv4 prefixes for BGP to advertise. On remote-site devices, the list of IPv4 prefixes for BGP is optional. After you have added the devices, you complete the SD-WAN plugin configuration by creating an SD-WAN VPN cluster. Each VPN cluster is a logical grouping of central-site and remote-site devices. The SD-WAN plugin uses VPN clusters as the top level for SD-WAN monitoring and reporting. Note You must assign each device to a VPN cluster. If you leave a device unassigned, the SD-WAN plugin does not generate and push the SD-WAN configuration to the device. The SD-WAN plugin simplifies and automates several critical SD-WAN tasks through a process referred to as AutoVPN. When the SD-WAN plugin runs AutoVPN, the following tasks are completed: • Create predefined security zones and required interfaces—If the predefined security zones do not already exist, the plugin automatically creates them. If you have enabled BGP, the plugin also creates a loopback interface for use as the BGP router-ID. • Create SD-WAN VIFs and configure IPSec tunnels—The plugin automatically creates the required VIFs for each SD-WAN device. The plugin also generates the complete configuration for the IPSec tunnels, which includes the tunnel interfaces, IKE crypto profile, IKE gateways, IPSec crypto profile, and the IPSec tunnels. The plugin assigns tunnel IP addresses from a customer provided pool. • Configuration of BGP and static routes—The plugin automatically enables BGP on all of the selected SD-WAN devices. The BGP configuration includes peer group setup and redistribution of the central-site and remote-site prefixes. The plugin also creates static routes to BGP peers through the appropriate VPN VIFs and a default route through the DIA VIF. For sites that do not run BGP, you may configure static routing. Note The SD-WAN plugin regenerates AutoVPN configurations each time new devices are added to the plugin. Palo Alto Networks does not recommend performing local overrides of configuration elements that the SD-WAN plugin manages. Palo Alto Networks 38 Palo Alto Networks Design Details In addition to the configuration tasks listed previously, the SD-WAN plugin also provides centralized monitoring and reporting capabilities for SD-WAN. The monitoring dashboard provides application performance and link performance statistics for your SD-WAN VPN clusters on a per-site basis. Application performance monitoring includes a list of all observed applications for each site, matching SD-WAN policy rule and link tags, application health status and total sessions (impacted/total). The App Performance screen lists applications as Impacted only if the SD-WAN device is unable to identify a path that meets the defined application thresholds in the PQP. Link performance monitoring includes a list of paths for each site (IPSec tunnel or physical interface), link tag and type, link status, and path health (latency/jitter/packet loss). You can specify the time interval for the dashboard or customize the interval to investigate previous events. If you need more detail on an application or a link, you can drill down and get more specific information. Log Collection with Panorama The monitoring and reporting capabilities for SD-WAN require that you collect and store traffic logs using Panorama in either a standalone or log-collector based configuration. You can use either of the following two deployment mode options for Panorama which, if necessary, allows for the separation of management and log collection. • Panorama mode—Panorama controls both policy and log management functions for all of the managed devices. • Log Collector mode—One or more Log Collectors collect and manage logs from the managed devices. This assumes that another deployment of Panorama is operating in Panorama Mode or Management Only Mode. Figure 19 Panorama modes Panorama (Panorama Mode) Panorama (Management Mode) Panorama (Log Collector Mode) Configuration Logging Palo Alto Networks 39 Palo Alto Networks Design Details The separation of management and log collection enables the Panorama deployment to meet scalability, organizational, and geographical requirements. You must have one or more managed log collectors (which may include Panorama itself). On Panorama, you add each log collector a collector group. As you add new SD-WAN devices to Panorama, you must assign each device to a collector group. On your SD-WAN devices, you must send system logs, configuration logs, and traffic logs to Panorama. Otherwise, the SD-WAN monitoring screens on Panorama will be incomplete. To send the traffic logs, you must create a log forwarding profile and reference this profile within each of your security policy rules. If you do not include a log forwarding profile, the traffic matching that security policy rule is not monitored. Device Management You manage and monitor all devices in your SD-WAN by using Panorama and the Panorama SD-WAN plugin. There are two valid methods for device management. The default method is to use the management interface of your device to communicate with Panorama. This method is suitable for devices deployed to sites that already have an existing network connection to Panorama. An alternate method is to use a dataplane interface on your device to communicate with Panorama. This method requires additional configuration on your device, but it does not require an existing connection to a site and is suitable for remote-site management across a WAN. Note If you intend to manage your devices by using dataplane interfaces across a public network, then you must assign a public IP address to Panorama and each log collector. Palo Alto Networks 40 Palo Alto Networks Design Details Central-Site Device Management You typically deploy Panorama in a central site so that you can manage the next-generation firewalls deployed within your organization. If you take this approach, managing your central-site devices is straightforward because you can use the existing network. After you connect the management interfaces of your devices to the internal network, no additional configuration is required. By default, you use the management interface to communicate with Panorama and for control plane functions such as DNS, NTP, logging, and security service updates. You must create security policy rules on the existing security infrastructure that control traffic to the internet to permit the control plane applications that use cloud services. Figure 20 Central-site device management Configuration Logging Internet Updates (eth1/1) dataplane flows (MGT) (MGT) (eth1/x) Central Site If you are deploying a new device, you can arrange for technical on-site personnel to perform the initial device configuration. The initial configuration tasks require a direct connection to the SD-WAN device. After the initial tasks are complete, you can complete the remaining tasks centrally, using Panorama. Palo Alto Networks 41 Palo Alto Networks Design Details Remote-Site Device Management It is more complex to manage a device located at a remote site than it is for a device located at a central site. If you have an existing out-of-band (OOB) network dedicated to device management, then you can follow the same approach that was previously discussed for central site devices. Figure 21 Remote-site device management Configuration Logging Internet Updates (MGT) (eth1/1) dataplane flows (MGT) (eth1/x) Remote Site Central Site The following methods are available for remote sites without an OOB management network: • Manual device configuration performed by on-site personnel—This method is almost identical to the method discussed for central sites. The primary difference is the configuration. For a remote-site device without an OOB management network, you must manage the device in-band, through a dataplane interface (example: ethernet1/1). This method requires that you create service routes, NAT policy rules, and security policy rules for your device. When the basic configuration is complete, you can connect the device to the network, and the device initiates contact to Panorama. You can complete the remaining tasks centrally, using Panorama. The configuration that you develop using this method may be used in the alternate methods that follow. In this method, when you add the device to Panorama and apply the centralized configuration, you overwrite certain several local device parameters with identical, centrally defined values. The centralized configuration also inserts new policies using Panorama pre-rules, preempting the local policy rules. This approach allows you to maintain centralized control of the overall device configuration. Palo Alto Networks 42 Palo Alto Networks Design Details • USB bootstrap configuration—After you have created and validated a working remote-device configuration, you may use that remote-device configuration as a baseline configuration to simplify and automate the configuration process. First, you create an init-cfg.txt file that provides the basic information that your device needs to connect to your network. Then, you must save and export a configuration snapshot from a working device. Next, you create a bootstrap package that includes both the init-cfg.txt file and the configuration snapshot. In order to bootstrap a SD-WAN device, you must first install the bootstrap package onto a USB flash drive. To prepare the USB flash drive with the bootstrap package, you can use any running Palo Alto Networks next-generation firewall with a USB port. The on-site personnel can configure a factory-default SD-WAN device by inserting the USB flash drive and then powering on the device. The on-site personnel do not need to perform any device configuration. As the device boots up, they must also connect the device to the network so that the device can initiate contact to Panorama. You can complete the remaining tasks centrally, using Panorama. • Centralized staging—After you have created and validated a working remote-device configuration, central-site personnel perform the SD-WAN device configurations for each remote-site device at a central staging site. After the staging process is complete, the device is repacked and shipped to the remote-site. The on-site personnel do not need to perform any device configuration. They are required only to power on the device and connect the device to the network so that the device can initiate contact to Panorama. You can complete the remaining tasks centrally, using Panorama. SECURITY ZONES Each SD-WAN device requires a specific a set of security zones. You must create some of these zones manually, for which you typically use Panorama templates. The Panorama SD-WAN plugin uses several of the zones, so their names must match exactly. You should create all the zones in advance if you want to reference them in the security and NAT policies within Panorama device groups. If you don’t create the zones in advance, the SD-WAN plugin creates any required zones that do not already exist. Palo Alto Networks 43 Palo Alto Networks Design Details Central Site At a minimum, each central site must include the zones listed in Table 5. Table 5 Required zones for SD-WAN central-site devices Zone name Zone type SD-WAN usage zone-internal Layer3 Required for device loopback interfaces used for SD-WAN. Zone name must match exactly. zone-to-branch Layer3 Required for SD-WAN tunnels and VPN VIFs to remote sites. Zone name must match exactly. zone-public Layer3 Required for WAN-facing interfaces and DIA VIFs. You may choose the zone name. zone-private Layer3 Required for LAN-facing interfaces at central sites. You may choose the zone name. Figure 22 Central-site security zones (eth1/3) Zone: zone-private (loopback.901) Zone: zone-internal (eth1/1) Zone: zone-public (eth1/2) DIA VIF VPN VIF (to remote sites) Palo Alto Networks 44 Zone: zone-to-branch Palo Alto Networks Design Details Remote Site At a minimum, each remote site must include the zones listed in Table 6. If necessary, you may use additional private zones to provide segmentation at your remote sites. Table 6 Required zones for SD-WAN remote-site devices Zone name Zone type SD-WAN usage zone-internal Layer 3 Required for device loopback interface used for SD-WAN. Zone name must match exactly. zone-to-hub Layer 3 Required for SD-WAN tunnels and VPN VIFs to central sites. Zone name must match exactly. zone-public Layer 3 Required for WAN-facing interfaces and DIA VIFs. You may choose the zone name. zone-private Layer 3 Required for LAN-facing interfaces at remote sites. You may choose the zone name. Figure 23 Remote-site security zones VPN VIF (to central sites) Zone: zone-public DIA VIF (eth1/1) Zone: zone-internal Zone: zone-to-hub (eth1/2) (loopback.901) (eth1/3) Zone: zone-private AUTOVPN OVERLAY The Panorama SD-WAN plugin uses the link type information to determine which device peers are used for a link’s overlay network. If you designate a device interface as connected to any public link type (ADSL/DSL, cablemodem, Ethernet, fiber, LTE/4G or WiFi), then the SD-WAN plugin configures the device to connect to the overlay network for that link type. If a device has multiple interfaces with the same link class, each interface has a connection to the overlay network. When you run AutoVPN, the SD-WAN plugin creates an overlay network between each set of peers. On each peer, a VPN VIF terminates the overlay network, which is composed of the set of tunnels to each peer. The number of tunnels required depends on SD-WAN link types of the tunnel source interfaces. The SD-WAN plugin uses the link class for the link type listed in Table 3 to determine the tunnel peers. Palo Alto Networks 45 Palo Alto Networks Design Details If you have two WAN links at each site, one public (ADSL/DSL) and one private (MPLS), the SD-WAN plugin creates one tunnel between the public interfaces and a second tunnel between the private interfaces. The SD-WAN plugin does not create tunnels between public and private interfaces. The SD-WAN plugin automatically includes private interfaces, such as MPLS, as DIA VIF group members. Unless your private network has a path to the internet, you should exclude the private interfaces from sending DIA traffic and use the private interfaces for tunnel termination. Configure a traffic distribution profile for DIA that includes only interfaces connected to public networks. An SD-WAN path is equivalent to a tunnel within a VPN VIF. The SD-WAN devices perform pathmonitoring for each path. Figure 24 AutoVPN (MPLS and internet) INET (eth1/1) VPN VIF (to remote site) MPLS (eth1/2) DIA VIF source inte rface DIA VIF source inte rface INET (eth1/1) MPLS (eth1/2) VPN VIF (to central site) If you have two WAN links at each site, both public (ADSL/DSL and cablemodem), the SD-WAN plugin creates a full mesh of tunnels between the public interfaces. Palo Alto Networks 46 Palo Alto Networks Design Details Figure 25 AutoVPN (dual internet) INET-1 (eth1/1) VPN VIF (to remote site) INET-2 (eth1/2) DIA VIF source inte rface DIA VIF source inte rface INET-1 (eth1/1) INET-2 (eth1/2) VPN VIF (to central site) If you have three WAN links at each site, two public (ADSL/DSL and cablemodem), and one private (MPLS), the SD-WAN plugin creates a full mesh of tunnels between the public interfaces and a single tunnel between the private interfaces. The SD-WAN plugin does not create tunnels between public and private interfaces. Figure 26 SD-WAN (MPLS and dual internet) VPN VIF (to remote site) DIA VIF INET-1 (eth1/1) MPLS (eth1/3) source inte rface INET-2 (eth1/2) INET-2 (eth1/2) INET-1 (eth1/1) source inte rface MPLS (eth1/3) VPN VIF (to central site) DIA VIF Palo Alto Networks 47 Palo Alto Networks Design Details SD-WAN ROUTING The method you use to route traffic to and from your Layer 3 LAN devices (Ethernet switch or router) at your sites is independent of the routing method used within the SD-WAN. In order to enable end-to-end routing across your SD-WAN, you must share routing information from the SD-WAN with your LAN devices and you must advertise site-specific routes across the SD-WAN. Note Palo Alto Networks recommends that you use contiguous IP route blocks within your organization so that you may summarize the IP routes. This approach simplifies the routing configuration for your SD-WAN. Central-Site Routing At your central sites, you must configure routing on your Layer 3 LAN devices in order to send traffic destined for remote sites to the LAN interface of your SD-WAN device. You can use either static routing, OSPF, or BGP on your LAN device. If you use static routing, you must include all remote-site prefixes or include summary routes that match the remote-site prefixes. If you use BGP or OSPF, you must also configure the same protocol on your SD-WAN device and create a redistribution profile for the virtual router (on your SD-WAN device) that includes the remote-site prefixes or summary routes. The SD-WAN device then advertises prefixes listed in the redistribution profile to your LAN device. At each central site, you must also configure routing on your SD-WAN device to send traffic destined for the site to the nearest interface of your Layer 3 LAN device. You can use either static routing, OSPF, or BGP on your SD-WAN device. If you use static routing, you must include all central-site prefixes or include summary routes that match the central-site prefixes. If you use BGP or OSPF, you must also configure the same protocol on your Layer 3 LAN device and advertise the central-site prefixes or summary routes to your SD-WAN device. You use BGP between the SD-WAN devices to advertise the site-specific routes across the SD-WAN. As part of the AutoVPN process with the SD-WAN plugin, you configure a list of IPv4 prefixes for BGP to advertise. This list of prefixes must include the central-site prefixes or summary routes. Remote-Site Routing In many organizations, the remote sites are relatively small and may have only a few networking devices. A common network topology for a remote site is an SD-WAN device directly connected to a Layer 2 LAN switch. In this type of topology, the SD-WAN device is responsible for all routing at the remote site. You use BGP between the SD-WAN devices in order to advertise the site-specific routes across the SDWAN. As part of the AutoVPN process with the SD-WAN plugin, you enable BGP at the remote site. The AutoVPN process automatically configures the SD-WAN device to redistribute the directly connected routes from its LAN-facing interfaces. You may also configure a list of IPv4 prefixes for BGP to advertise. Palo Alto Networks 48 Palo Alto Networks Design Details Remote-Site to Remote-Site Routing If you wish to enable routing for traffic between your remote sites, you must explicitly enable it. The BGP routing configuration generated by the SD-WAN plugin advertises central-site prefixes to all sites and remote-site prefixes to central sites. If you need to route traffic between remote sites, you must designate a central site to act as a transit site and configure that site to advertise one or more remote-site summary routes to the remote-site devices. Figure 27 Remote-site to remote-site routing Central Site INET-1 INET-2 MPLS Local LAN 10.5.128.0 /22 Dest IP Prefix 10.5.128.0/22 10.6.0.0/16 10.61.0.0/16 BGP Redist Rule s remote-remote traffic remote-remote traffic (sdwan.902) (sdwan.902) Local LAN 10.6.128.0/24 Local LAN 10.61.128.0/24 Routes Dest IP Prefix 10.5.128.0/22 10.6.0.0/16 10.61.0.0/16 10.6.128.0/24 Routes Interface sdwan.902 sdwan.902 sdwan.902 ethernet1/3 Dest IP Prefix 10.5.128.0/22 10.6.0.0/16 10.61.0.0/16 10.61.128.0/24 Interface sdwan.902 sdwan.902 sdwan.902 ethernet1/3 Remote Sites SECURING THE SITES In addition to securing user traffic, as part of your SD-WAN deployment you must create security policies to support control-plane functions such as remote-management, DNS, NTP, routing protocols, and security service updates. The SD-WAN devices require a properly functioning control plane. Otherwise, they cannot forward user traffic and provide consistent visibility and comprehensive protection. Palo Alto Networks 49 Palo Alto Networks Design Details Control-Plane Policies The central-site SD-WAN devices use their management interfaces for remote-management, DNS, NTP, and security service updates. However, you must create security policy rules to permit BGP to and from other SD-WAN devices. You configure the remote-site SD-WAN devices to use their dataplane interfaces for remote-management, DNS, NTP and security service updates by using service routes. As part of the service route configuration, you must create a loopback interface for the service routes and then a set of NAT policy rules—one for each of the WAN dataplane interfaces. Each NAT rule translates the loopback interface’s IP address to a WAN dataplane interface IP address. Additionally, you must create security policy rules to permit remotemanagement, DNS, NTP and security service updates to access the zone-public zone. You must also create security policy rules to permit BGP to and from central-site SD-WAN devices. You can use the SD-WAN plugin to automatically generate BGP rules for your SD-WAN device groups, or you can create the BGP rules manually using your preferred settings. DIA Policies The DIA policies are required only on the remote-site SD-WAN devices. This assumes that internet access at the central sites is not provided through the SD-WAN devices. As part of the DIA configuration, you create a set of NAT policy rules—one for each of the WAN dataplane interfaces. Each NAT rule translates traffic from the zone-private zone to a WAN dataplane interface IP address. Additionally, you must create the required security policy rules to permit applications to access the zone-public zone. Figure 28 NAT policy for SD-WAN DIA Zone: zone-public (eth1/1) NAT (eth1/2) DIA VIF Zone: zone-private (eth1/3) Original P acke t Translated Packet Name Source Zone Destination Zone Destination Interface DIA-NAT-eth1_1 zone-private zone-public ethernet1/1 Source Translation dynamic-ip-and-port ethernet1/1 DIA-NAT-eth1_2 zone-private zone-public ethernet1/2 dynamic-ip-and-port ethernet1/2 Palo Alto Networks 50 Palo Alto Networks Design Details Site-Site Policies Traffic between SD-WAN sites requires only security policy rules and does not require any NAT policy rules. However, because each session must pass through multiple SD-WAN devices, you must create complementary security policy rules for each device in the traffic path. All security policy rules use rule type Universal except where noted: • Central-site to remote-site traffic—At the central site, create security policy rules to permit application traffic from the zone-private zone to the zone-to-branch zone. At the remote site, create security policy rules to permit application traffic from the zone-to-hub zone to the zoneprivate zone. • Remote-site to central-site traffic—At the remote site, create security policy rules to permit application traffic from the zone-private zone to the zone-to-hub zone. At the central site, create security policy rules to permit application traffic from the zone-to-branch zone to the zone-private zone. • Remote-site to remote-site traffic—At the source remote site, create security policy rules to permit application traffic from the zone-private zone to the zone-to-hub zone. At the central site, create security policy rules with Rule Type intrazone, to permit application traffic from the zone-to-branch zone. These rules implicitly include the destination zone of zone-to-branch. At the destination remote site, create security policy rules to permit application traffic from the zone-tohub zone to the zone-private zone. Palo Alto Networks 51 SD-WAN Design Models SD-WAN Design Models These design models describe how Palo Alto Networks Secure SD-WAN and CloudGenix SD-WAN allow organizations to extend their security perimeter to include remote sites, ensure data privacy for network traffic internal to the organization, and provide inspection and visibility for all traffic entering and leaving the network. For building your SD-WAN and securing your external connections, Palo Alto Networks provides a broad set of offerings from which you may select: • Next-generation firewall with PAN-OS Secure SD-WAN • CloudGenix Autonomous SD-WAN with ION • Prisma Access There are many ways to use the concepts discussed in the previous sections to build an SD-WAN and secure your DIA traffic. This guide includes two specific design models. The primary difference between the two models is how each model provides comprehensive security for the remote-site traffic. • PAN-OS Secure SD-WAN—This design model uses the next-generation firewall for both centralsite and remote-site locations. You connect the sites using the natively integrated SD-WAN capabilities of the next-generation firewall. You use Panorama to centrally manage all devices. This design model describes a network topology with two central sites and multiple remote sites in a single region. All sites have dual connections to a public network provided through a standard Ethernet handoff. The SD-WAN interconnects the sites in a hub-and-spoke topology and provides per-application path selection across all possible paths. Each remote site is configured for DIA, and the DIA traffic shares both connections to the public network. In this model, you can also use the next-generation firewall to provide segmentation in order to reduce the scope of compliance, to limit data exfiltration, and to reduce the attack surface. This model consolidates the SD-WAN functions and security functions onto a single device at remote sites, which reduces complexity and cost. Because a next-generation firewall is deployed at every site, you can provide visibility and inspection for all users and applications across every network segment. Although you may choose between hardware appliance and virtual form-factors, you still self-manage and self-deploy the devices in this model. Palo Alto Networks recommends this model if pervasive security is a high priority for your organization. Palo Alto Networks 52 SD-WAN Design Models • CloudGenix with Prisma Access—This design model uses the CloudGenix ION devices for both central-site and remote-site locations. You connect the sites by using CloudGenix SD-WAN secure application fabric, AppFabric. You centrally manage all devices by using the CloudGenix Portal. You manage the Prisma Access configuration by using the cloud services plugin for Panorama. This design model describes a network topology with two central sites and multiple remote sites in a single region. All sites have dual connections to a public network provided through a standard Ethernet handoff. You use AppFabric to interconnect sites with hub-and-spoke, partial-mesh, or full-mesh topologies depending on business and scale requirements. The SD-WAN provides perapplication path selection across all possible paths. For DIA traffic, each remote site is connected to Prisma Access, which is treated as a third-party endpoint. The ION devices use third-party VPN interfaces to build IPSec tunnels to Prisma Access remote-network connections. The DIA traffic shares both connections to the public network. In this model, the ION devices provide an application-aware, zone-based firewall at each remote site. In addition, Prisma Access complements the ION on-site capabilities and provides additional visibility and inspection for DIA traffic. The ION devices also support zero-touch provisioning and deployment. This model separates the SD-WAN functions from security functions for the remote sites. Cost and complexity remain low because both functions are managed and delivered from the cloud. For DIA traffic, Prisma Access provides the exact same security, visibility, and control as next-generation firewall at the remote site. Palo Alto Networks recommends this model if you require zero-touch provisioning and deployment for your remote-site devices and pervasive security for DIA and cloud-based applications is a high priority for your organization. This model also provides deep visibility into the performance of applications running across the SD-WAN. Palo Alto Networks 53 SD-WAN Design Models PAN-OS SECURE SD-WAN DESIGN MODEL In this model, each site connects to the SD-WAN, which is the private WAN for the organization and provides access to the headquarters and data center locations. This model assumes that the central sites are already interconnected through an existing public or private WAN connection. The SD-WAN provides two key functions in this design model: • Interconnection between central sites and remote sites using VPN VIFs and IPSec tunnels • Direct internet access from remote sites using DIA VIFs Figure 29 PAN-OS Secure SD-WAN design model Central Site Central Site INET-1 INET-2 Direct Internet Access SD-WAN Remote Sites Direct Internet Access Remote Sites The SD-WAN protects your internal user traffic by using VPN VIFs that provide intelligent path control with per-application path selection. The IPSec tunnels over the public networks ensure data privacy through strong encryption. The SD-WAN also provides per-application path selection and load sharing for public network traffic. At all locations, the next-generation firewall provides full visibility of the users, applications, and content for all network traffic. At your remote sites, you can also use the next-generation firewall to reduce the attack surface through segmentation. In this model, you use next-generation firewalls as SD-WAN devices at both the central-site and remotesite locations. You use Panorama to centrally configure, manage, and monitor the SD-WAN. Each device is first added to Panorama as a managed device and then added to the Panorama SD-WAN plugin as an SD-WAN device. This model assumes that other (not shown) security infrastructure devices provide internet access at the central sites. Palo Alto Networks 54 SD-WAN Design Models Central-Site Devices This model integrates SD-WAN hub devices into the existing networks at central sites. For device management, connect the management port to an existing network that has access to Panorama. At the central sites, you configure each SD-WAN device as a hub. The devices at each site connect to the private network and two public networks. On the interfaces that connect to the public networks, you must use public IP addresses with static address assignment and each interface must have a Next Hop Gateway. Configure BGP on the virtual router for each device to exchange routing information with peers on the private network. For each central-site SD-WAN device, include a complete list of central-site prefixes, or summary prefixes, when adding the device to the SD-WAN plugin. If you want to enable remote-site to remote-site traffic, choose one of your central sites to be a transit site and include the remote-site summary prefixes in the prefix list for the transit-site device. The SD-WAN devices at the central sites each use BGP to advertise their prefix lists to all other sites. On Panorama, you create security policy rules to permit BGP to and from other SD-WAN devices. On Panorama, you should create a set of SD-WAN policies to control applications initiated from the central sites. Each policy should include the zone-private source zone and the zone-to-branch destination zone. For sessions that are initiated from the central site, to control those sessions with an SD-WAN policy, you only need to configure an SD-WAN policy at that site. Note SD-WAN policies apply only to a single SD-WAN routing hop and are effective to control site-to-site flows such as “remote-site to central-site” or “centralsite to remote-site”. To control site-to-site flows such as remote-site to central-site to remotesite, you must apply SD-WAN policies that match the flow at both the remotesite and the central-site. Remote-Site Devices This model consolidates the SD-WAN functions and security functions onto a single device, which reduces complexity and cost. Because a next-generation firewall is deployed at every site, you can provide visibility and inspection for all users and applications across every network segment. At the remote sites, you configure each SD-WAN device as a branch. The devices at each site connect to the private network and two public networks. On the interfaces that connect to the public networks, you may use either public IP addresses or private IP addresses behind a NAT device. You can configure each public interface with either DHCP address assignment or static address assignment, and each interface must have a next-hop gateway. Palo Alto Networks 55 SD-WAN Design Models The SD-WAN devices at the remote sites each use BGP to advertise their directly connected LAN networks and learn routes from the SD-WAN hubs devices at the central sites. If you have additional prefixes to add, specify them in a prefix list for the site. On Panorama, you create security policy rules to permit BGP to and from central-site SD-WAN devices. On Panorama, you create a set of SD-WAN policies to control applications initiated from the remote sites. Each policy should include the zone-private source zone and the zone-to-hub destination zone. For sessions that are initiated from the remote site, to control those sessions with an SD-WAN policy, you only need to configure an SD-WAN policy at that site. For each SD-WAN policy, you must choose a PQP and a TDP and reference them in an SD-WAN policy rule. Where possible, you should use built-in PQPs to simplify your configuration. You must include a PQP for the SD-WAN policies that control your DIA traffic, even though the DIA VIF does not perform end-to-end path monitoring. The DIA VIF inherits the path monitoring state of the IPSec tunnels sourced from member interfaces. To support DIA traffic, you must configure each public interface to perform NAT. On Panorama, you should create a set of NAT policy rules—one for each of the public interfaces. Each NAT rule translates traffic from the zone-private zone to a public interface IP address. Additionally, you must create the required security policy rules to permit applications to access the zone-public zone. If required, create multiple private security zones to provide segmentation at the remote site. This approach allows you to partition the network into manageable, secure segments which reduces the scope of compliance and limits data exfiltration. Create the required security policy rules to permit applications to and from each additional security zone. Figure 30 Remote-site segmentation VPN VIF (to central sites) Zone: zone-public DIA VIF (eth1/1) Zone: zone-private2 (PCI) Palo Alto Networks Zone: zone-to-hub (eth1/2) (eth1/3) (eth1/4) 56 Zone: zone-private (non-PCI) SD-WAN Design Models Interconnecting the Sites The SD-WAN plugin builds the VPN tunnels between sites. You must have at least one SD-WAN hub device at a central site and at least one SD-WAN branch device at a remote site. The SD-WAN plugin generates a configuration that builds an overlay network for the public network, and for each type of private network in your organization. If any SD-WAN device has multiple interfaces that connect to the same overlay network, then a full-mesh of VPN tunnels is created between the central site and the remote site for that overlay network. In this model, all of the sites are in a single logical region. You configure the SD-WAN plugin with a single VPN cluster that maps to the region. A VPN cluster is the highest level organizational structure for grouping used by the monitoring dashboard. Within the cluster, you can compare site performance and view site-specific application performance and link-performance statistics. After you complete the AutoVPN process and push the plugin-generated configuration, the SD-WAN remote-site devices initiate IPSec connections to the central sites and begin to apply the SD-WAN policies pushed from Panorama. Configure SD-WAN Application Policies Palo Alto Networks recommends that you monitor your network traffic and then use the information you have collected to develop an SD-WAN application policy for your organization. In the absence of any configured policies, traffic flows across the SD-WAN by using traditional routing along with round-robin load sharing. You must configure SD-WAN policies to match your applications and explicitly distribute the application traffic across the public networks. You build these policies and apply them to the SD-WAN devices at the source sites. Since you use Panorama templates to configure these policies, you must create separate policies for each Panorama device group - one set of policies for central-site devices and another set of policies for remote-site devices. Palo Alto Networks 57 SD-WAN Design Models CLOUDGENIX WITH PRISMA ACCESS DESIGN MODEL In this model, each site connects to the CloudGenix AppFabric, which is the SD-WAN for the organization and provides access to the headquarters and data center locations. This model assumes that the central sites are already interconnected through an existing public or private WAN connection. At all sites, you must use an ION device as an on-premises device. After the basic device configuration is complete, you transition the site to control mode. One key difference from the PAN-OS Secure SD-WAN model is that you use Prisma Access to secure DIA traffic, and each remote site connects directly to Prisma Access by using third-party VPN connections. Figure 31 CloudGenix with Prisma Access design model CloudGenix Portal Central Site Central Site INET-1 INET-2 DIA traffic SD-WAN AppFabric Third-party VPN Prisma Access Remote Site Remote Sites Remote Site All aspects of the CloudGenix SD-WAN are managed through the CloudGenix Portal. For Prisma Access, Panorama, and Cortex™ Data Lake manage all policy and monitoring centrally, and Palo Alto Networks manages the Prisma Access infrastructure. The SD-WAN protects your internal user traffic using AppFabric, which uses IPSec tunnels over the public networks ensure data privacy through strong encryption. ION devices automatically choose the best WAN path for your applications based on business policy and real-time analysis of the application performance metrics and WAN links. This model assumes that other (not shown) security infrastructure devices provide internet access at the central sites. Palo Alto Networks 58 SD-WAN Design Models Central-Site Devices This model integrates ION devices into the existing networks at central sites. The devices at each central site connect to the private network and two public networks. For device management, connect the ION controller port to an existing network that has access to the internet. The device connects to CloudGenix Portal and, once online, appears as an unclaimed device. On CloudGenix Portal, for each central site, you create a site and configure the site as a data center. You then claim the device and assign it to the site. Configure an IP address for each interface and assign a circuit label. You may use either statically assigned or DHCP-assigned IP addresses. Palo Alto Networks recommends you use statically assigned IP addresses for the connections at central sites. You can place the ION devices behind a NAT device. If you do, you must provide the public IP addresses when you configure the interfaces. Configure BGP for each device to exchange routing information with peers on the private network. Remote-Site Devices This model deploys ION devices into new greenfield networks at remote sites. The devices at each remote site connect to the private network and two public networks. For device management, connect the ION internets port to the public networks. The device connects to CloudGenix Portal and, once online, appears as an unclaimed device. On CloudGenix Portal, for each remote site, you create a site and configure the site as a branch. You then claim the device and assign it to the site. Configure an IP address for each interface and assign a circuit label. You may use either statically assigned or DHCP-assigned IP addresses. Palo Alto Networks recommends you use a statically assigned IP address for the connection to the private network and DHCP-assigned IP addresses for the connections to the public networks. If necessary, configure BGP for each device to exchange routing information with peers on the private network. Interconnecting the Sites CloudGenix Portal automatically configures the AppFabric between your sites after you switch the sites to control mode. At this time, the portal also pushes security and application policies to all sites. By default, the AppFabric builds a hub-and-spoke topology that also supports remote-site to remote-site communication by routing through a central site. You may optionally configure direct connections between remote sites using the AppFabric’s secure fabric link overlay. Palo Alto Networks 59 SD-WAN Design Models Configuring SD-WAN Application Policies Palo Alto Networks recommends that monitor your network analytics and then use the information you have collected to develop an SD-WAN application policy for your organization. In a brownfield deployment, you have the option to first deploy your ION devices in analytics mode, which only monitors traffic. However, in a greenfield deployment, you deploy your devices in control mode. You must configure SD-WAN policies to match your applications and explicitly distribute the application traffic across the public networks and which applications to send to Prisma Access. Connecting the Remote Sites to Prisma Access The CloudGenix application policies can supersede destination-routing based policies. Rather than just configuring a default route to Prisma Access, you instead treat Prisma Access as a set of third-party endpoints. You then create policies that match applications and assign them to use the Prisma Access endpoints. You have two options for managing the integration from CloudGenix SD-WAN to Prisma Access. In small-scale deployments, typically less than 20 sites, you manually manage the integration to Prisma Access by using both Panorama and CloudGenix Portal. In large-scale deployments, you automate the Prisma Access integration through API-based integration by using CloudGenix CloudBlades Platform. A CloudBlade provides an abstraction layer between CloudGenix Portal and a particular cloud service, which simplifies integration with the service. The Prisma Access for Networks CloudBlade allows you to automatically configure the CloudGenix Portal, ION devices, Panorama, and Prisma Access. Manual Configuration Option On Panorama, for each remote site, you create a Prisma Access remote network connection at the location nearest your remote site. If you are using an active/active connection, you create an additional Prisma Access remote network connection at the location next-nearest your remote site. Note The active/active option consumes twice the licensed bandwidth of Prisma Access because both links are configured as primary tunnels. On CloudGenix Portal, for each remote site, you create and configure a third-party VPN from the interface to the nearest Prisma Access location. If you are using an active/active connection, you create and configure an additional third-party VPN from that interface to the next nearest Prisma Access location. After you create the third-party VPN connections, you then configure the associated SD-WAN policy to direct internet and cloud-based applications to Prisma Access. You must also configure security policy rules on Prisma Access for the associated applications. Palo Alto Networks 60 SD-WAN Design Models CloudBlade Configuration Option The Prisma Access CloudBlade automates the third-party VPN components of the manual configuration option, which makes this option highly-scalable. This option requires that you deploy either an onpremises Docker container or a public-cloud container to facilitate the communication between the Prisma Access CloudBlade and Panorama. On CloudGenix Portal, you also need to apply regional Prisma Access tags to remote-site device interfaces. The CloudBlade orchestrates both the third-party VPN configuration and the Prisma Access onboarding. After you create the third-party VPN connections, you then configure the associated SD-WAN policy to direct internet and cloud-based applications to Prisma Access. You must also configure security policy rules on Prisma Access for the associated applications. Palo Alto Networks 61 Summary Summary Palo Alto Networks provides organizations with several options for deploying SD-WAN. In addition to a broad WAN overview, this guide covered the technical aspects of PAN-OS Secure SD-WAN in-depth and described the best methods for integrating CloudGenix with Prisma Access. You can use this guide to determine which offering is most effective in meeting the business requirements of your organization. The PAN-OS Secure SD-WAN design model consolidates the SD-WAN functions and security functions onto a single device at remote sites, which reduces complexity and cost. All sites within your organization benefit from the comprehensive security features of the next-generations firewall. You can provide visibility and inspection for all users and applications across every network segment, including traffic flows to central sites, cloud-based applications, and the internet. The CloudGenix with Prisma Access design model combines the industry-leading SD-WAN capabilities of CloudGenix SD-WAN with the cloud-based comprehensive security capabilities of Prisma Access. The native SD-WAN and security functions of the ION devices are complemented by additional cloud-delivered security. You can provide visibility and inspection for all users and applications across including traffic flows to central sites, cloud-based applications and the internet. With either design model, your organization can achieve consistent security, regardless of location. Palo Alto Networks 62 You can use the feedback form to send comments about this guide. HEADQUARTERS Palo Alto Networks Phone: +1 (408) 753-4000 3000 Tannery Way Sales: +1 (866) 320-4788 Santa Clara, CA 95054, USA Fax: +1 (408) 753-4001 http://www.paloaltonetworks.com info@paloaltonetworks.com © 2020 Palo Alto Networks, Inc. Palo Alto Networks is a registered trademark of Palo Alto Networks. A list of our trademarks can be found at http://www.paloaltonetworks.com/company/trademarks.html. All other marks mentioned herein may be trademarks of their respective companies. Palo Alto Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. B-000221P-1-20a