SD-WAN Administrator’s Guide 3.1 docs.paloaltonetworks.com Contact Information Corporate Headquarters: Palo Alto Networks 3000 Tannery Way Santa Clara, CA 95054 www.paloaltonetworks.com/company/contact-support About the Documentation • For the most recent version of this guide or for access to related documentation, visit the Technical Documentation portal docs.paloaltonetworks.com. • To search for a specific topic, go to our search page docs.paloaltonetworks.com/search.html. • Have feedback or questions for us? Leave a comment on any page in the portal, or write to us at documentation@paloaltonetworks.com. Copyright Palo Alto Networks, Inc. www.paloaltonetworks.com © 2022-2023 Palo Alto Networks, Inc. Palo Alto Networks is a registered trademark of Palo Alto Networks. A list of our trademarks can be found at www.paloaltonetworks.com/company/ trademarks.html. All other marks mentioned herein may be trademarks of their respective companies. Last Revised September 18, 2023 SD-WAN Administrator’s Guide 3.1 2 ©2023 Palo Alto Networks, Inc. Table of Contents SD-WAN Overview........................................................................................... 5 About SD-WAN........................................................................................................................... 6 System Requirements for SD-WAN..................................................................................... 10 SD-WAN Configuration Elements.........................................................................................14 Plan Your SD-WAN Configuration....................................................................................... 16 Configure SD-WAN.........................................................................................21 Install the SD-WAN Plugin..................................................................................................... 22 Install the SD-WAN Plugin When Panorama is Internet-Connected................ 22 Install the SD-WAN Plugin When Panorama is not Internet-Connected......... 23 Set Up Panorama and Firewalls for SD-WAN................................................................... 26 Add Your SD-WAN Firewalls as Managed Devices.............................................. 26 Create an SD-WAN Network Template.................................................................. 28 Create the Predefined Zones in Panorama............................................................. 29 Create the SD-WAN Device Groups........................................................................ 31 Create a Link Tag...................................................................................................................... 34 Configure an SD-WAN Interface Profile.............................................................................35 Configure a Physical Ethernet Interface for SD-WAN.....................................................39 Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN............ 42 Configure Layer 3 Subinterfaces for SD-WAN..................................................................47 Configure a Virtual SD-WAN Interface............................................................................... 50 Create a Default Route to the SD-WAN Interface...........................................................52 Configure SD-WAN Link Management Profiles................................................................ 53 Create a Path Quality Profile......................................................................................53 Configure SaaS Monitoring......................................................................................... 55 SD-WAN Traffic Distribution Profiles...................................................................... 66 Create a Traffic Distribution Profile..........................................................................71 Create an Error Correction Profile............................................................................ 74 Configure an SD-WAN Policy Rule...................................................................................... 78 Allow Direct Internet Access Traffic Failover to MPLS Link.......................................... 83 Configure DIA AnyPath........................................................................................................... 84 Distribute Unmatched Sessions.............................................................................................91 Add SD-WAN Devices to Panorama....................................................................................93 Configure Certificate-Based Authentication for SD-WAN Devices.................. 93 Add an SD-WAN Device............................................................................................. 97 Bulk Import Multiple SD-WAN Devices................................................................109 Onboard PAN-OS Firewalls to Prisma Access..................................................... 112 Configure HA Devices for SD-WAN................................................................................. 126 SD-WAN Administrator’s Guide 3.1 3 ©2023 Palo Alto Networks, Inc. Table of Contents Create a VPN Cluster............................................................................................................ 127 Create a Full Mesh VPN Cluster with DDNS Service....................................................137 Create a Static Route for SD-WAN................................................................................... 141 Configure Advanced Routing for SD-WAN..................................................................... 143 Monitoring and Reporting...........................................................................149 Monitor SD-WAN Tasks....................................................................................................... 150 Monitor SD-WAN Application and Link Performance...................................................152 Monitor Prisma Access Hubs...............................................................................................156 Baseline Your Prisma Access Hub Application and Link Performance........... 156 Monitor Prisma Access Hub Application and Link Performance..................... 157 Generate an SD-WAN Report.............................................................................................162 Troubleshooting............................................................................................. 165 Use CLI Commands for SD-WAN Tasks...........................................................................166 Replace an SD-WAN Device............................................................................................... 170 Troubleshoot App Performance..........................................................................................172 Troubleshoot Link Performance..........................................................................................177 Upgrade your SD-WAN Firewalls.......................................................................................182 Upgrade the SD-WAN Plugin..............................................................................................183 Uninstall the SD-WAN Plugin............................................................................................. 184 SD-WAN Administrator’s Guide 3.1 4 ©2023 Palo Alto Networks, Inc. SD-WAN Overview Learn about SD-WAN and plan your configuration to ensure a successful deployment. • About SD-WAN • System Requirements for SD-WAN • SD-WAN Configuration Elements • Plan Your SD-WAN Configuration 5 SD-WAN Overview About SD-WAN Software-Defined Wide Area Network (SD-WAN) is a technology that allows you to use multiple internet and private services to create an intelligent and dynamic WAN, which helps lower costs ® ® and maximize application quality and usability. Beginning with PAN-OS 9.1, Palo Alto Networks offers strong security with an SD-WAN overlay in a single management system. Instead of using costly and time-consuming MPLS with components such as routers, firewalls, WAN path controllers, and WAN optimizers to connect your WAN to the internet, SD-WAN on a Palo Alto Networks firewall allows you to use less expensive internet services and fewer pieces of equipment. You don’t need to purchase and maintain other WAN components. • PAN-OS Security with SD-WAN Functionality • SD-WAN Link and Firewall Support • Prisma Access Hub Support • Centralized Management PAN-OS Security with SD-WAN Functionality The SD-WAN plugin is integrated with PAN-OS, so that you get the security features of a PANOS firewall and SD-WAN functionality from a single vendor. The SD-WAN overlay supports dynamic, intelligent path selection based on applications and services and the conditions of links that each application or service is allowed to use. The path health monitoring for each link includes latency, jitter, and packet loss. Granular application and service controls allow you to prioritize applications based on whether the application is mission-critical, latency-sensitive, or meets certain health criteria, for example. Dynamic path selection avoids brownout and node failure problems because sessions fail over to a better performing path in less than one second. The SD-WAN overlay works with all PAN-OS security features, such as User-ID™ and AppID™, to provide complete security control to branch offices. The full suite of App-ID capabilities (App-ID decoder, App-ID cache, and source/destination external dynamic list [EDL] IP address lists) identifies applications for application-based control of SD-WAN traffic. You can deploy the firewall with Zero Trust segmentation of traffic. You can configure and manage SD-WAN centrally from the Panorama web interface or the Panorama REST API. You may have cloud-based services and instead of having your internet traffic flow from branches to the hub to the cloud, you want the internet traffic to flow directly from branches to the cloud using a directly connected ISP. Such access from a branch to the internet is Direct Internet Access (DIA). You don’t need to spend your hub bandwidth and money on internet traffic. The branch firewall is already doing security, so you don’t need the hub firewall to enforce security on internet traffic. Use DIA on branches for SaaS, web browsing, or heavy-bandwidth applications that shouldn’t be backhauled to a hub. The following figure illustrates a DIA virtual interface consisting of three links from the branch to the cloud. The figure also illustrates a VPN tunnel virtual interface consisting of four links that connect the branch to the hub at the headquarters. SD-WAN Administrator’s Guide 3.1 6 ©2023 Palo Alto Networks, Inc. SD-WAN Overview SD-WAN Link and Firewall Support Link bundling allows you to group multiple physical links (that different ISPs use to communicate with the same destination) into a virtual SD-WAN interface. On the basis of applications and services, the firewall chooses from the links (path selection) for session load sharing and to provide failover protection in the event of a brownout or blackout. Thus you are providing the application with the best quality performance. The firewall automatically performs session load sharing over the links in a virtual SD-WAN interface to use available bandwidth advantageously. An SD-WAN interface must have all of the same type of connection (either DIA or VPN). VPN links support the hub-and-spoke topology. SD-WAN supports the following types of WAN connections: ADSL/DSL, cable modem, Ethernet, fiber, LTE/3G/4G/5G, MPLS, microwave/radio, satellite, WiFi, and anything that terminates as Ethernet to the firewall’s interface. You decide the appropriate strategy for how to use the links. You could use inexpensive broadband connections before expensive MPLS or LTE connections. Alternatively, you could use specific VPN tunnels to reach specific hubs in a region. See the system requirements for SD-WAN for a full list of firewall models that support SD-WAN software capabilities. If you are a new customer purchasing a Palo Alto Networks next-generation firewall, you will use the default virtual router for SD-WAN. If you are an existing customer, you can choose to either let PAN-OS overwrite any existing virtual routers or use a new virtual router and new zones for SD-WAN to keep SD-WAN content separate from your pre-existing configuration. SD-WAN Administrator’s Guide 3.1 7 ©2023 Palo Alto Networks, Inc. SD-WAN Overview Beginning with PAN-OS 11.0, SD-WAN plugin 3.1 supports advanced routing engine that uses industry-standard configuration methodology to facilitate administrator tasks. Although conceptually equivalent, the advanced routing engine uses logical routers rather than virtual routers to instantiate routing domains. When you enable advanced routing, logical routers are created and advanced routing engine is used for routing. When you disable Advanced Routing, virtual routers are created and legacy engine is used for routing. Prisma Access Hub Support With SD-WAN plugin 2.2 and later releases, PAN-OS Secure SD-WAN provides you with Prisma Access hub support to give you full control of how and where applications are secured. Prisma Access Hub support allows PAN-OS firewalls to connect to Prisma Access compute nodes (CNs) to achieve cloud-based security in an SD-WAN hub-and-spoke topology. This support enables a seamless link failover from on-premises security to Prisma Access and the ability to mix both to meet your security needs. In a mixed topology with both SD-WAN firewalls and Prisma Access hubs, the SD-WAN hubs are Prisma Access CNs (IPSec Termination Nodes) and the SD-WAN branches are PAN-OS firewalls. SD-WAN automatically creates IKE and IPSec tunnels that connect the branch to the hub. Using Traffic Distribution profiles, you can create SD-WAN policies to match specific internet applications and redirect them to a PAN-OS firewall or Prisma Access deployment of your choice. With Prisma Access hub support, on-premises and cloud security platforms work together to provide a complete solution with consistent security policies managed by Panorama. See the system requirements for SD-WAN for the minimum PAN-OS and SD-WAN plugin versions required for Prisma Access Hub support. Prisma Access hub support has the following limitations: • Importing and exporting an SD-WAN configuration related to Prisma Access are not supported. • Load, Partial Load, Revert, and Partial Revert for the Prisma Access configuration are not supported. • Onboarding to an existing Prisma Access Remote Network Security Proccessing Node (RNSPN) is not supported. For an existing branch that is connected to Prisma Access, you need to delete the branch and then onboard it again. • No SD-WAN CLI commands are available on Prisma Access firewalls. • On a CN, there is no path selection for traffic that originates on the CN. • Prisma Access statistics are not provided in SD-WAN reporting and statistics. Centralized Management Panorama™ provides the means to configure and manage SD-WAN, which makes configuring multiple options on many geographically-dispersed firewalls much faster and easier than configuring firewalls individually. You can change network configurations from a single location rather than configuring each firewall individually. Auto VPN configuration allows Panorama to configure branches and hubs with secure IKE/IPSec connections. A VPN cluster defines the hubs and branches that communicate with each other in a geographic region. The firewall uses VPN tunnels for path health monitoring between a branch and a hub to provide subsecond detection of brownout conditions. SD-WAN Administrator’s Guide 3.1 8 ©2023 Palo Alto Networks, Inc. SD-WAN Overview The Panorama dashboard provides visibility into your SD-WAN links and performance so that you can adjust path quality thresholds and other aspects of SD-WAN to improve its performance. Centralized statistics and reporting include application and link performance statistics, path health measurements and trend analysis, and focused views of application and link issues. Begin by understanding your SD-WAN use case, then review the SD-WAN configuration elements, traffic distribution methods, and plan your SD-WAN configuration. To greatly accelerate the configuration, the best practice is for you to export an empty SD-WAN device CSV and enter information such as branch office IP address, the virtual router to use, the firewall site name, zones to which the firewall belongs, and BGP route information. Panorama uses the CSV file to configure the SD-WAN hubs and branches and to automatically provision VPN tunnels between hubs and branches. SD-WAN supports dynamic routing through eBGP and is configured using Panorama’s SD-WAN plugin to allow all branches to communicate with the hub only or with the hub and other branches. If Panorama is managing a multi-vsys firewall, all SD-WAN enabled interfaces and configurations must be configured on vsys1. SD-WAN does not support an SD-WAN configuration across multiple virtual systems of a multi-VSYS firewall. SD-WAN interfaces must be configured in the same virtual router; they cannot be split among virtual routers. SD-WAN Administrator’s Guide 3.1 9 ©2023 Palo Alto Networks, Inc. SD-WAN Overview System Requirements for SD-WAN Review the minimum software versions, plugin versions, and resource requirements for the Panorama™ plugin for SD-WAN. Beginning with PAN-OS 11.0, you can configure advanced routing for SD-WAN with plugin version 3.1. Platform PAN-OS System Requirement Prisma SD-WAN Access Cloud Plugin Configuration Plugin Panorama 11.1.0 • (Panorama virtual appliance) System Disk —224GB system disk 4.0.0 and 5.0.0 3.2.0 4.0.0 and 5.0.0 3.1.2 4.0.0 3.1.2 4.0.0 3.1.1 3.2.1.h21 3.0.1-h6 11.0.3 • CPU—16 CPUs • Memory—64GB memory 11.0.2 • System Mode —Panorama mode and Management Only mode 11.0.2 11.0.1 11.0.1 is the recommended 11.0.x release. SD-WAN Administrator’s Guide 3.1 • (M-Series Appliance in Management Only mode only) A 8TB RAID logging disk pair enabled 10 ©2023 Palo Alto Networks, Inc. SD-WAN Overview Platform PAN-OS 11.0.0 10.2.7 10.2.6 10.2.5 10.2.4 10.2.4 is the recommended 10.2.x release. 10.2.3 System Requirement The above information applies to a maximum of 500 managed devices. For information about using a maximum of 1,000 managed devices, refer to System Requirements for the Panorama Virtual Appliance 10.2.1 Prisma SD-WAN Access Cloud Plugin Configuration Plugin 3.2.1-h3 3.1.0-h6 4.0.0 and 5.0.0 3.0.6 4.0.0 3.0.6 4.0.0 3.0.5 3.2.1-h21 3.0.4 3.2.1-h5 3.0.4 Not 3.0.1 supported in this release; 3.0.0 expected in a future release. Do not upgrade to PAN-OS 11.1 if you are using SD-WAN with Prisma Access Cloud Configuration Plugin. 10.2.0 10.1.11 4.0.0 and 5.0.0 2.2.6 10.1.11 4.0.0 2.2.5 10.1.10 4.0.0 2.2.4 SD-WAN Administrator’s Guide 3.1 11 ©2023 Palo Alto Networks, Inc. SD-WAN Overview Platform PAN-OS System Requirement Prisma SD-WAN Access Cloud Plugin Configuration Plugin 10.1.10 is the recommended 10.1.x release. 3.2.1-h5 2.2.4 10.1.8 3.2.1-h5 2.2.2 10.1.5-h1 2.1 2.2.1 10.1.0 2.1 2.2 10.1.9 10.1.9 is the recommended 10.1.x release. Next-Gen Firewall • PAN-OS 11.1 —11.1.0 N/A • PAN-OS 11.0 —11.0.0 • PAN-OS 10.2 —10.2.0 • PAN-OS 10.1 —10.1.4 • PAN-OS 10.0 —10.0.8 Prisma Access Compute Node 10.0.7* * Prisma Access Compute Nodes (IPSec Termination Nodes) must run PAN-OS 10.0.7 or a later 10.0 release. If necessary, work with your sales team to request an upgrade before attempting to onboard a branch office to the Prisma Access hub. The following firewall models support SD-WAN software capabilities: • PA-220 and PA-220R SD-WAN Administrator’s Guide 3.1 12 ©2023 Palo Alto Networks, Inc. SD-WAN Overview • PA-400 Series • PA-820 and PA-850 • PA-1400 Series • PA-3200 Series • PA-3400 Series • PA-5200 Series • PA-5400 Series • PA-7000 Series • VM-Series firewalls For more information about specific hardware availability, refer to the compatibility matrix. SD-WAN Administrator’s Guide 3.1 13 ©2023 Palo Alto Networks, Inc. SD-WAN Overview SD-WAN Configuration Elements The elements of an SD-WAN configuration work together, allowing you to: • Group physical Ethernet interfaces that share a common destination into a logical SD-WAN interface. • Specify link speeds. • Specify the thresholds at which a deteriorating path (or brownout or blackout) to an SD-WAN warrants selecting a new best path. • Specify the method of selecting that new best path. This view indicates the relationships between elements at a glance. The goal of an SD-WAN configuration is to control which links your traffic takes by specifying the VPN tunnels or direct internet access (DIA) that certain applications or services take from a branch to a hub or from a branch to the internet. You group paths so that if one path deteriorates, the firewall selects a new best path. • A Tag name of your choice identifies a link; you apply the Tag to the link (interface) by applying an Interface Profile to the interface, as the red arrow indicates. A link can have only one Tag. The two yellow arrows indicate that a Tag is referenced in the Interface Profile and the Traffic Distribution profile. Tags allow you to control the order that interfaces are used for traffic distribution. Tags allow Panorama to systematically configure many firewall interfaces with SDWAN functionality. • An SD-WAN Interface Profile specifies the Tag that you apply to the physical interface, and also specifies the type of Link that interface is (ADSL/DSL, cable modem, Ethernet, fiber, SD-WAN Administrator’s Guide 3.1 14 ©2023 Palo Alto Networks, Inc. SD-WAN Overview LTE/3G/4G/5G, MPLS, microwave/radio, satellite, WiFi, or other). The Interface Profile is also where you specify the maximum upload and download speeds (in Mbps) of the ISP’s connection. You can also change whether the firewall monitors the path frequently or not; the firewall monitors link types appropriately by default. • A Layer3 Ethernet Interface with an IPv4 address can support SD-WAN functionalities. You apply an SD-WAN Interface Profile to this interface (red arrow) to indicate the characteristics of the interface. The blue arrow indicates that physical Interfaces are referenced and grouped in a virtual SD-WAN Interface. • A virtual SD-WAN Interface is a VPN tunnel or DIA group of one or more interfaces that constitute a numbered, virtual SD-WAN Interface to which you can route traffic. The paths belonging to an SD-WAN Interface all go to the same destination WAN and are all the same type (either DIA or VPN tunnel). (Tag A and Tag B indicate that physical interfaces for the virtual interface can have different tags.) • A Path Quality Profile specifies maximum latency, jitter, and packet loss thresholds. Exceeding a threshold indicates that the path has deteriorated and the firewall needs to select a new path to the target. A sensitivity setting of high, medium, or low lets you indicate to the firewall which path monitoring parameter is more important for the applications to which the profile applies. The green arrow indicates that you reference a Path Quality Profile in one or more SD-WAN Policy Rules; thus, you can specify different thresholds for rules applied to packets having different applications, services, sources, destinations, zones, and users. • A Traffic Distribution Profile specifies how the firewall determines a new best path if the current preferred path exceeds a path quality threshold. You specify which Tags the distribution method uses to narrow its selection of a new path; hence, the yellow arrow points from Tags to the Traffic Distribution profile. A Traffic Distribution profile specifies the distribution method for the rule. • The preceding elements come together in SD-WAN Policy Rules. The purple arrow indicates that you reference a Path Qualify Profile and a Traffic Distribution profile in a rule, along with packet applications/services, sources, destinations, and users to specifically indicate when and how the firewall performs application-based SD-WAN path selection for a packet not belonging to a session. (You can also reference a SaaS Quality Profile and an Error Correction Profile in an SD-WAN policy rule.) Now that you understand the relationship between the elements, review the traffic distribution methods and then Plan Your SD-WAN Configuration. SD-WAN Administrator’s Guide 3.1 15 ©2023 Palo Alto Networks, Inc. SD-WAN Overview Plan Your SD-WAN Configuration Plan the complete topology of your SD-WAN-enabled branch and hub firewall interfaces so that you can create Panorama™ templates with CSV files and then push the configurations to the firewalls. STEP 1 | Plan the branch and hub locations, link requirements, and IP addresses. From Panorama you will export an empty SD-WAN device CSV and populate it with branch and hub information. 1. Decide the role of each firewall (branch or hub). 2. Determine which branches will communicate with which hubs; each functional group of branch and hub firewalls that communicate with each other is a VPN cluster. For example, your VPN clusters might be organized geographically or by function. 3. Determine the ISP link types that each branch and hub support: ADSL/DSL, cable modem, Ethernet, fiber, LTE/3G/4G/5G, MPLS, microwave/radio, satellite, and WiFi. 4. Determine the maximum download and upload bandwidth (Mbps) that the link types support and how you want to apply these speed controls to links, as described in Step 2. Record the ISP link’s maximum download and upload bandwidth (Mbps). This information will serve as reference egress maximums if you need to configure QoS to control the application bandwidth. 5. Gather the public IP addresses of branch firewalls, whether they are static or dynamically assigned. The firewall must have an internet-routable, public IP address so it can initiate and terminate IPSec tunnels and route application traffic to and from the internet. The ISP’s customer premise equipment must be directly connected to the Ethernet interface on the firewall. If you have a device that performs NAT located between the branch firewall and the hub, the NAT device can prevent the firewall from bringing up IKE peering and IPSec tunnels. If the tunnel fails, work with the administrator of the remote NAT device to resolve the issue. 6. Gather the private network prefixes and serial numbers of branch and hub firewalls. 7. Decide the link type of each firewall interface. Allocate the same link types on the same Ethernet interfaces across the branch firewalls to make configuration easier. For example, Ethernet1/1 is always cable modem. 8. Decide on the naming conventions for your sites and SD-WAN devices. Do not use the simple hostnames “hub” or “branch” because Auto VPN configuration uses these keywords to generate various configuration elements. 9. If you already have zones in place before configuring SD-WAN, decide how to map those zones to the predefined zones that SD-WAN uses for path selection. You will SD-WAN Administrator’s Guide 3.1 16 ©2023 Palo Alto Networks, Inc. SD-WAN Overview map existing zones to the predefined zones named zone-internal, zone-to-hub, zone-tobranch, and zone-internet. Information you will enter into a CSV (so that you can add multiple SD-WAN devices at once) includes: serial number, type of device (branch or hub), names of zones to map to predefined zones (pre-existing customers), loopback address, prefixes to redistribute, AS number, router ID, and virtual router name. STEP 2 | Plan link bundles and VPN security for private links. A link bundle lets you combine multiple physical links into one virtual SD-WAN interface for purposes of path selection and failover protection. By having a bundle of more than one physical link, you maximize application quality in case a physical link deteriorates. You create a bundle by applying the same link tag to multiple links (via an SD-WAN Interface Profile). The link tag identifies a bundle of links that have a similar type of access and similar type of SDWAN policy handling. For example, you can create a link tag named low cost broadband and include the cable modem and fiber broadband services. STEP 3 | Identify the applications that will use SD-WAN and QoS optimization. 1. Identify the critical and the latency-sensitive business applications for which you will provide SD-WAN control and policies. These are applications that require a good user experience, and are likely to fail under poor link conditions. Start with the most critical and latency-sensitive applications; you can add applications after SD-WAN is functioning smoothly. 2. Identify the applications that require QoS policies so you can prioritize bandwidth. These should be the same applications you identified as critical or latency-sensitive. Start with the most critical and latency-sensitive applications; you can add applications after SD-WAN is functioning smoothly. STEP 4 | Determine when and how you want links to fail over to a different link in the event the original link degrades or fails. 1. Decide on the path monitoring mode for a link, although the best practice is to retain the default setting for the link type: • Aggressive—The firewall sends probe packets to the opposite end of the SD-WAN link at a constant frequency (five probes per second by default). Aggressive mode is appropriate for links where monitoring path quality is critical; where you need fast detection and failover for brownout and blackout conditions. Aggressive mode provides subsecond detection and failover. • Relaxed—The firewall observes a configurable idle time between sending probe packets for seven seconds (at the probe frequency you configure), which makes path monitoring less frequent than aggressive mode. Relaxed mode is appropriate for links SD-WAN Administrator’s Guide 3.1 17 ©2023 Palo Alto Networks, Inc. SD-WAN Overview that have very low bandwidth, links that are expensive to operate, such as satellite or LTE, or when fast detection isn’t as important as preserving cost and bandwidth. 2. Prioritize the order in which the firewall selects the first link for a new session and the order in which links should be a candidate to replace a link that is failing over, if there is more than one candidate. For example, if you want an expensive backup LTE link to be the last link used (only when the inexpensive broadband links are oversubscribed or completely down), then use the Top Down Priority traffic distribution method and place the tag that is on the LTE link last in the list of tags for the Traffic Distribution profile. 3. For the applications and services, determine the path health thresholds at which you consider a path to have degraded enough in quality that you want the firewall to select a new path (fail over). The quality characteristics are latency (range is 10 to 2,000 ms), jitter (range is 10 to 1,000 ms), and packet loss percentage. These thresholds constitute a Path Quality profile, which you reference in an SD-WAN policy rule. When any single threshold (for packet loss, jitter, or latency) is exceeded (and the remaining rule criteria are met), the firewall chooses a new preferred path for the matching traffic. For example, you can create Path Quality profile AAA with latency/jitter/packet loss thresholds of 1000/800/10 to use in Rule 1 when FTP packets come from source zone XYZ, and create Path Quality profile BBB (with thresholds of 50/200/5) to use in Rule 2 when FTP packets come from source IP address 10.1.2.3. Best practice is to start with high thresholds and test how the application tolerates them. If you set the values too low, the application may switch paths too frequently. Consider whether the applications and services you are using are especially sensitive to latency, jitter, or packet loss. For example, a video application might have good buffering that mitigates latency and jitter, but would be sensitive to packet loss, which impacts the user experience. You can set the sensitivity of the path quality parameters in the profile to high, medium or low. If the sensitivity settings for latency, jitter, and packet loss are the same, the firewall examines the parameters in the order of packet loss, latency, jitter. 4. Decide if there are links among which to load share new sessions for an application or service. STEP 5 | Plan the BGP configurations that Panorama will push to branches and hubs to dynamically route traffic between them. 1. Plan BGP route information, including a four-byte autonomous system number (ASN). Each firewall site is in a separate AS and therefore must have a unique ASN. Each firewall must also have a unique Router ID. 2. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. 3. If you don’t want to use BGP dynamic routing, plan to use Panorama’s network configuration features to push out other routing configurations. You can do static routing between the branch and hubs. Simply omit all of the BGP information in the Panorama plugin and use normal virtual router static routes to perform static routing. SD-WAN Administrator’s Guide 3.1 18 ©2023 Palo Alto Networks, Inc. SD-WAN Overview STEP 6 | Consider the capacities of firewall models for virtual SD-WAN interfaces, SD-WAN policy rules, log size, IPSec tunnels (including proxy IDs), IKE peers, BGP and static route tables, BGP routing peers, and performance for your firewall mode (App-ID™, threat, IPSec, decryption). Ensure the branch and hub firewall models you intend to use support the capacities you require. SD-WAN Administrator’s Guide 3.1 19 ©2023 Palo Alto Networks, Inc. SD-WAN Overview SD-WAN Administrator’s Guide 3.1 20 ©2023 Palo Alto Networks, Inc. Configure SD-WAN After you Plan Your SD-WAN Configuration, install the SD-WAN plugin and set up the Panorama™ management server to centrally manage the SD-WAN configuration of your hub and branch firewalls. By leveraging Panorama, you reduce the management requirements and operational overhead for administrating your SD-WAN deployment, and can more easily monitor your link health and troubleshoot issues should they arise. If Panorama is managing a multi-vsys firewall, all SD-WAN enabled interfaces and configurations must be configured on vsys1. SD-WAN does not support an SD-WAN configuration across multiple virtual systems of a multi-VSYS firewall. • Install the SD-WAN Plugin • Set Up Panorama and Firewalls for SD-WAN • Create a Link Tag • Configure an SD-WAN Interface Profile • Configure a Physical Ethernet Interface for SD-WAN • (Optional) Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN • (Optional) Configure Layer 3 Subinterfaces for SD-WAN • Configure a Virtual SD-WAN Interface • Create a Default Route to the SD-WAN Interface • Configure SD-WAN Link Management Profiles • Configure an SD-WAN Policy Rule • Allow Direct Internet Access Traffic Failover to MPLS Link • Configure DIA AnyPath • Distribute Unmatched Sessions • Add SD-WAN Devices to Panorama • (Optional) Configure HA Devices for SD-WAN • Create a VPN Cluster • Create a Full Mesh VPN Cluster with DDNS Service • (Optional) Create a Static Route for SD-WAN • (Optional) Configure Advanced Routing for SD-WAN 21 Configure SD-WAN Install the SD-WAN Plugin A Panorama™ management server with an SD-WAN plugin is required to configure and manage an SD-WAN deployment. If your Panorama is internet-connected, you download the SDWAN plugin directly from Panorama and install it on the Panorama management server. If your Panorama is not internet-connected, you download the SD-WAN plugin from the Palo Alto ® Networks Customer Support Portal and install it on the Panorama management server. • Install the SD-WAN Plugin When Panorama is Internet-Connected • Install the SD-WAN Plugin When Panorama is not Internet-Connected Install the SD-WAN Plugin When Panorama is Internet-Connected A Panorama™ management server with an SD-WAN plugin installed is required to configure and manage an SD-WAN deployment. When Panorama is connected to the internet, you download and install the SD-WAN plugin directly from the Panorama web interface. The plugin needs to be installed only on the Panorama managing your SD-WAN firewalls, and not on the individual hub and branch firewalls. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > Plugins, search for the sd_wan plugin and Check Now for the most recent version of the plugin. STEP 3 | Download and Install the SD-WAN plugin. STEP 4 | After you successfully install the SD-WAN plugin, select Commit and Commit to Panorama. This step is required before you can commit any configuration changes to Panorama. STEP 5 | (Management Only mode only) Enable the logging disks required to store SD-WAN monitoring data. • M-Series Appliances—All M-Series appliances come with two pairs of 8TB logging disks in RAID 1 by default. When managing firewalls leveraging SD-WAN from Panorama in SD-WAN Administrator’s Guide 3.1 22 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Management Only mode, you must enable the first pair of logging disk pairs to store SDWAN monitoring data. 1. Log in to the Panorama CLI. 2. Enable the first pair of logging disk pairs included by default with your M-Series appliance. > request system raid add A1 3. Verify that logging Logging Disk Pair A is Available: > show system raid detail When the RAID set up is complete, the following response displays: Disk Pair A Status Disk id A1 model size status Available clean Present : ST91000640NS : 953869 MB : active sync 4. Make the logging disk pairs available for logging. 1. Select Panorama > Managed Collectors and edit the Log Collector. 2. Select Disks and Add each array. 3. Click OK to save your changes. 4. Select Commit > Commit to Panorama and Commit your changes. 5. Select Commit > Push to Devices, select the Collector Group, and Push your changes. • Panorama Virtual Appliances—If you deployed your Panorama virtual appliance in Management Only mode, you must increase the system disk to 224GB to store SD-WAN monitoring data. STEP 6 | Continue to Set Up Panorama and Firewalls for SD-WAN to begin configuring your SD-WAN deployment. Install the SD-WAN Plugin When Panorama is not InternetConnected A Panorama™ management server with an SD-WAN plugin is required to configure and manage an SD-WAN deployment. When Panorama is not connected to the internet, you must download the SD-WAN plugin from the Palo Alto Networks Customer Support Portal and upload the plugin to Panorama. The plugin needs to be installed only on the Panorama managing your SD-WAN firewalls, and not on the individual hub and branch firewalls. STEP 1 | Log in to the Palo Alto Networks Customer Support Portal. STEP 2 | Select Updates > Software Updates, and in the Filter By drop-down select Panorama Integration Plug In. SD-WAN Administrator’s Guide 3.1 23 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Locate and download the SD-WAN Plug-in. STEP 4 | Log in to the Panorama Web Interface. STEP 5 | Select Panorama > Plugins and Upload the SD-WAN plugin. STEP 6 | Browse and locate the SD-WAN plugin you downloaded from the Customer Support Portal and click OK. STEP 7 | Install the SD-WAN plugin. STEP 8 | After you successfully install the SD-WAN plugin, select Commit and Commit to Panorama. This step is required before you can commit any configuration changes to Panorama. STEP 9 | (Management Only mode only) Enable the logging disks required to store SD-WAN monitoring data. • M-Series Appliances—All M-Series appliances come with two pairs of 8TB logging disks in RAID 1 by default. When managing firewalls leveraging SD-WAN from Panorama in Management Only mode, you must enable the first pair of logging disk pairs to store SDWAN monitoring data. 1. Log in to the Panorama CLI. 2. Enable the first pair of logging disk pairs included by default with your M-Series appliance. > request system raid add A1 3. Verify that logging Logging Disk Pair A is Available: > show system raid detail When the RAID set up is complete, the following response displays: Disk Pair A Status Disk id A1 model size SD-WAN Administrator’s Guide 3.1 Available clean Present : ST91000640NS : 953869 MB 24 ©2023 Palo Alto Networks, Inc. Configure SD-WAN status : active sync 4. Make the logging disk pairs available for logging. 1. Select Panorama > Managed Collectors and edit the Log Collector. 2. Select Disks and Add each array. 3. Click OK to save your changes. 4. Select Commit > Commit to Panorama and Commit your changes. 5. Select Commit > Push to Devices, select the Collector Group, and Push your changes. • Panorama Virtual Appliances—If you deployed your Panorama virtual appliance in Management Only mode, you must increase the system disk to 224GB to store SD-WAN monitoring data. STEP 10 | Continue to Set Up Panorama and Firewalls for SD-WAN to begin configuring your SD-WAN deployment. SD-WAN Administrator’s Guide 3.1 25 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Set Up Panorama and Firewalls for SD-WAN Before you can begin configuring your SD-WAN deployment, you must add your hub and branch firewalls as managed devices, and create the necessary templates and device group configurations to successfully push your SD-WAN configuration to SD-WAN firewalls. • Add Your SD-WAN Firewalls as Managed Devices • Create an SD-WAN Network Template • Create the Predefined Zones in Panorama • Create the SD-WAN Device Groups Add Your SD-WAN Firewalls as Managed Devices Before you can begin configuring your SD-WAN deployment, you must first Install the SDWAN Plugin and add your hub and branch firewalls as managed devices to the Panorama™ management server. As part of adding your SD-WAN firewall as a managed device on the Panorama™ management server, you must activate the SD-WAN license to enable SD-WAN functionality for the firewall. As part of adding your SD-WAN firewalls as managed devices, you must configure your managed firewalls to forward logs to Panorama. Panorama collects information from multiple sources, such as configuration logs, traffic logs, and link characteristic measurements, to generate the visibility for SD-WAN application and link health information. Do not have your Panorama management server connection to be only reliant on the SDWAN overlay. To maintain a reliable connection, where the Panorama is always reachable to the PAN-OS firewalls, we recommend you to create a dedicated IPSec tunnel from the PAN-OS firewalls to reach Panorama (that is outside the SD-WAN overlay between hub/ branches where the Panorama is located). With this approach, you can ensure that the Panorama management server is always reachable even if there is any impact to the SDWAN overlay. STEP 1 | Launch the Firewall Web Interface. STEP 2 | Activate your SD-WAN license to enable SD-WAN functionality on the firewall. Each firewall you intend to use in your SD-WAN deployment requires a unique auth code to activate the license. For example, if you have 100 firewalls, you must purchase 100 SD-WAN SD-WAN Administrator’s Guide 3.1 26 ©2023 Palo Alto Networks, Inc. Configure SD-WAN licenses and activate each SD-WAN license on each firewall using one of the 100 unique auth codes. For VM-Series firewalls, you apply the SD-WAN auth code against the specific VMSeries firewall. If you deactivate the VM-Series firewall, the SD-WAN auth code can be activated on a different VM-Series firewall of the same model. Ensure that your SD-WAN license remains valid to continue leveraging SD-WAN. If the SD-WAN license expires, the following occurs: • A warning displays when you Commit any configuration changes but no commit failure occurs. • Your SD-WAN configuration no longer functions but is not deleted. • Firewalls no longer monitor and gather link health metrics and stop sending monitoring probes. • Firewalls no longer send app and link health metrics to Panorama. • SD-WAN path selection logic is disabled. • New sessions round robin on the virtual SD-WAN interface. • Existing sessions remain on the specific link they were on when the license expired. • If an internet outage occurs, traffic follows using standard routing and ECMP if configured. STEP 3 | Add the Panorama IP address to the firewall. 1. Select Device > Setup > Management and edit the Panorama Settings. 2. Enter the Panorama IP address in the first field. The Panorama FQDN is not supported for SD-WAN. 3. (Optional) If you have set up a high availability (HA) pair in Panorama, enter the IP address of the secondary Panorama in the second field. 4. Verify that you Enable pushing device monitoring data to Panorama. 5. Click OK. 6. Commit your changes. STEP 4 | Configure log forwarding to Panorama. Forwarding logs from your SD-WAN firewalls to Panorama is required to display Monitoring and Reporting data. By default, HTTP/2 inspection is automatically enabled if decryption is enabled for application traffic. The parent sessions using a HTTP/2 connection does not generate any traffic logs because they do not carry any application traffic. However, the child sessions generated by the streams within the HTTP/2 parent session still generate traffic logs. For more information on viewing logs for HTTP/2 connections, see the Palo Alto Networks Knowledgebase. SD-WAN Administrator’s Guide 3.1 27 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 5 | Add one or more firewalls to Panorama. For more details about adding firewalls to Panorama, see Add a Firewall as a Managed Device. 1. Log in to the Panorama Web Interface. 2. Select Panorama > Managed Devices > Summary and Add the firewall(s). 3. Enter the serial number(s) of the firewalls. 4. If you are adding firewalls when the required device groups and templates are already created, enable (check) Associate Devices to assign new firewalls to the appropriate device groups and template stack. 5. To add multiple firewalls using a CSV, click Import and Download Sample CSV to populate with the firewall information, and then Browse to import the firewalls. 6. Click OK. STEP 6 | Select Commit and Commit and Push your configuration. STEP 7 | Repeat Steps 2 through 5 on each firewall you intend to use in your SD-WAN deployment. Create an SD-WAN Network Template Create a template containing all the networking configuration objects for your SD-WAN hubs and branches. You must create a separate template and template stack for your hub firewalls and a separate template and template stack for your branch firewalls. It is a best practice to limit the number of templates and template stacks used to manage your SD-WAN device configuration. Limiting the number of templates and template stacks used across all hubs and branches greatly reduces the operational overhead of managing the configurations of multiple SD-WAN hubs and branches. Use template or template stack variables to help reduce the number of templates used. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create the SD-WAN hub network template. 1. Select Panorama > Templates and Add a new template. 2. Enter a descriptive Name for the template. 3. (Optional) Enter a Description for the template. 4. Click OK to save your configuration changes. STEP 3 | Create a hub template stack. 1. Select Panorama > Templates and click Add Stack to add a new template stack. 2. Enter a descriptive Name for the template stack. 3. (Optional) Enter a Description for the template. 4. Add the SD-WAN network template you created in the Step 2. 5. In the Devices section, select the check boxes for all your SD-WAN hub firewalls. 6. Click OK to save your configuration changes. SD-WAN Administrator’s Guide 3.1 28 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Create the SD-WAN branch network template. 1. Add a new template. 2. Enter a descriptive Name for the template. 3. (Optional) Enter a Description for the template. 4. Click OK to save your configuration changes. STEP 5 | Create a branch template stack. 1. Click Add Stack to add a new template stack. 2. Enter a descriptive Name for the template stack. 3. (Optional) Enter a Description for the template. 4. Add the SD-WAN network template you created in the Step 4. 5. In the Devices section, select the check boxes for all your SD-WAN branch firewalls. 6. Click OK to save your configuration changes. STEP 6 | Commit your configuration changes. Create the Predefined Zones in Panorama SD-WAN policy rules use predefined zones for internal path selection and traffic forwarding purposes. There are two use cases; your use case depends on whether you are enabling SD-WAN ® on your current PAN-OS firewalls that have existing security policy rules or whether you are starting a brand new PAN-OS deployment with no previous security policy rules. If your current firewalls have security policy rules in place, you map your existing zones to the predefined zones that SD-WAN policies use. The SD-WAN engine makes use of the predefined zones for forwarding traffic. Additionally, creating the predefined zones in the Panorama™ templates provides consistent visibility between the managed firewalls and Panorama: • Zone Internet—For traffic going to and coming from the untrusted internet. • Zone to Hub—For traffic going from branch firewalls to hub firewalls and for traffic going between hub firewalls. • Zone to Branch—For traffic going from hub firewalls to branch firewalls and for traffic between branch firewalls. • Zone Internal—For internal traffic at a specific location. • Zone to PA Hub—For internal traffic to reach Prisma Access hub. If you don’t create the predefined zones, the SD-WAN plugin will automatically create the predefined zones on your branch and hub firewall, but you won’t see them in Panorama. There are two main use cases for predefined zones: • Existing Zones—You already have pre-existing zones that you created for use in User-ID™ or various policies (security policy rules, QoS policy rules, zone protection, and packet buffer protection). You must map the pre-existing zones to the predefined zones that SD-WAN uses so the firewall can properly forward traffic. You should continue to use your pre-existing zones in all of your policies because the new predefined zones are used only for SD-WAN SD-WAN Administrator’s Guide 3.1 29 ©2023 Palo Alto Networks, Inc. Configure SD-WAN forwarding. You will map the zones when you to Add SD-WAN Devices to Panorama by creating your CSV file. (If you aren’t using a CSV file, you will map zones when you configure Panorama > SD-WAN > Devices and add existing zones to Zone Internet, Zone to Hub, Zone to Branch, and Zone Internal.) The result of mapping is that a branch or hub firewall can do a forwarding lookup to determine the egress SD-WAN interface and thus the egress zone. If you don’t map pre-existing zones to predefined zones, an allowed session won’t use SD-WAN. The mapping is necessary because existing customers have different zone names in place, and the firewall must narrow all of those zone names down to the predefined zones. You don’t necessarily have to map zones to all of the predefined zones, but you should map existing zones to at least the Zone to Hub and Zone to Branch zones. • No Existing Zones—You have a brand new deployment of Palo Alto Networks® firewalls and SD-WAN. In this case, you don’t have zones to map; we recommend you use the predefined zones in your PAN-OS policies and User-ID to simplify deployment. Before you begin configuring your SD-WAN deployment, for both use cases, you will create the required predefined zones in Panorama named zone-internet, zone-internal, zoneto-hub, zone-to-branch, and zone-to-pa-hub. When you onboard your branch and hub firewalls, you will Add SD-WAN Devices to Panorama. For pre-existing customers, the SD-WAN plugin will internally map pre-existing zones with these predefined zones when executing SDWAN policy rules, QoS policy rules, zone protection, User-ID, and packet buffer protection, and will use the predefined zones for zone logging and visibility in Panorama. For new customers, you are properly set up using the predefined zones. The predefined zones are also required in order to automatically set up VPN tunnels between your SD-WAN hubs and branches when you push the configuration from Panorama to your managed SD-WAN devices. The zone names are case-sensitive and must match the names provided in this procedure. Your commit fails on the firewall if the zone names don’t match those described in this procedure. In this example, we are creating the zone named zone-internet. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Zones and in the Template context drop-down, select the network template you previously created. STEP 3 | Add a new zone. STEP 4 | Enter zone-internet, for example, as the Name of the zone. STEP 5 | For zone Type, select Layer3. SD-WAN Administrator’s Guide 3.1 30 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Click OK. STEP 7 | Repeat the previous steps to create the remaining zones. In total, you must create the following zones: • zone-to-branch • zone-to-hub • zone-internal • zone-internet • zone-to-pa-hub STEP 8 | Commit and Commit and Push your configuration changes. STEP 9 | Commit your changes. Create the SD-WAN Device Groups Create device groups, one for your hubs and one for your branches, containing all the policy rules and configuration objects for your SD-WAN hubs and branches. After you create the device groups for your hubs and branches, you must create a Security policy rule in each device group allowing traffic between the hub and branch zones. Creating these Security policy rules ensures that traffic between the SD-WAN device zones is allowed when the SD-WAN plugin creates the VPN tunnels after you create a VPN cluster. Configure identical configurations across your hub firewalls and an identical configuration across your branch firewalls. This greatly reduces the operational overhead of having to manage the configurations of multiple SD-WAN hubs and branches, and allows you to troubleshoot, isolate, update configuration issues much more rapidly. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create the Predefined Zones in Panorama. SD-WAN Administrator’s Guide 3.1 31 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Create the SD-WAN hub device group. 1. Select Panorama > Device Groups and Add a device group. 2. Enter SD-WAN_Hub as the Name for the device group. 3. (Optional) Enter a Description for the template. 4. In the Devices section, select the check boxes to assign the SD-WAN hubs to the group. 5. For the Parent Device Group, select Shared. 6. Click OK. STEP 4 | Create the SD-WAN branch device group. 1. Select Panorama > Device Groups and Add a device group. 2. Enter SD-WAN_Branch as the Name for the device group. 3. (Optional) Enter a Description for the template. 4. In the Devices section, select the check boxes to assign the SD-WAN branches to the group. 5. For the Parent Device Group, select Shared. 6. Click OK. STEP 5 | Create a Security policy rule to control traffic flows from branch offices to the hub’s internal zone and from the hub’s internal zone to branch offices. 1. Select Policies > Security and in the Device Group context drop-down, select the SDWAN_Hub device group. 2. Add a new policy rule. 3. Enter a Name for the policy rule, such as SD-WAN access--hub DG. 4. Select Source > Source Zone and Add the zone-internal and zone-to-branch. 5. Select Destination > Destination Zone and Add the zone-internal and zone-to-branch. 6. Select Application and Add applications to allow. You must allow BGP if you are using BGP routing. 7. Select Actions and Allow to allow the applications you selected. 8. Select Target and specify the target devices to which Panorama™ should push this rule. SD-WAN Administrator’s Guide 3.1 32 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Create a Security policy rule to control traffic originating from the branch offices’ internal zone to the hub and from the hub to the branch offices’ internal zone. 1. Select Policies > Security and in the Device Group context drop-down, select the SDWAN_Branch device group. 2. Add a new policy rule. 3. Enter a Name for the policy rule, such as SD-WAN access--branch DG. 4. Select Source > Source Zone and Add the zone-internal and zone-to-hub. 5. Select Destination > Destination Zone and Add the zone-internal and zone-to-hub. 6. Select Application and Add applications to allow. You must allow BGP if you are using BGP routing. 7. Select Actions and Allow to allow the applications you selected. 8. Select Target and specify the target devices to which Panorama should push this rule. STEP 7 | Commit and push your configuration. 1. Commit and Commit and Push your configuration changes. 2. In the Push Scope section, click Edit Selections. 3. Enable (check) Include Device and Network Templates and click OK. 4. Commit and Push your configuration changes. There are two commit operations that are automatically performed when you commit and push the device group and template configuration. View the Tasks to verify that the second commit is successful. Of these two commit operations, the first always fails. SD-WAN Administrator’s Guide 3.1 33 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Create a Link Tag Create a link tag to identify one or more physical links that you want applications and services to use in a specific order during SD-WAN traffic distribution and failover protection. Grouping multiple physical links allows you to maximize the application and service quality if the physical link health deteriorates. When planning how to group your links, consider the use or purpose of the links and group them accordingly. For example, if you are configuring links intended for low-cost or non-businesscritical traffic, create a link tag and group these interfaces together to ensure that the intended traffic flows primarily on these links, and not on more expensive links that may impact businesscritical applications or services. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Objects > Tags and select the appropriate device group from the Device Group context drop-down. STEP 3 | Add a new tag. STEP 4 | Enter a descriptive Name for the tag. For example; Low Cost Paths, Expensive Paths, General Access, Private HQ, or Backup. STEP 5 | Enable (check) Shared to make the Link Tag available to all device groups on the Panorama™ management server and to the default vsys on a single vsys hub or branch, or to vsys1 on any multi-vsys hub or branch that you push to. By configuring a Shared Link Tag, Panorama is able to reference the Link Tags in the firewall configuration validation and successfully commits and pushes the configuration to branches and hubs. The commit fails if Panorama is unable to reference a Link Tag. STEP 6 | (Optional) Select a Color for the tag. STEP 7 | Enter helpful Comments about the tag. For example, Group two low cost broadband links and a backup link for general access to the internet. STEP 8 | Click OK to save your configuration changes. STEP 9 | Commit and Commit and Push your configuration changes. STEP 10 | Configure an SD-WAN Interface Profile. SD-WAN Administrator’s Guide 3.1 34 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure an SD-WAN Interface Profile Create an SD-WAN interface profile to define the characteristics of ISP connections and to specify the speed of links and how frequently the firewall monitors the link, and specify a Link Tag for the link. When you specify the same Link Tag on multiple links, you are grouping (bundling) those physical links into a link bundle or fat pipe. You must configure an SD-WAN interface profile and specify it for an Ethernet interface enabled with SD-WAN before you can save the Ethernet interface. Group links based on a common criterion. For example, group links by path preference from most preferred to least preferred, or group links by cost. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Network Profiles > SD-WAN Interface Profile and select the appropriate template from the Template context drop-down. STEP 3 | Add an SD-WAN interface profile. STEP 4 | Enter a user-friendly Name for the SD-WAN interface profile, which you’ll see in reporting, troubleshooting, and statistics. STEP 5 | Select the vsys Location if you have a multi-vsys Panorama™ management server. By default, vsys1 is selected. STEP 6 | Select the Link Tag that this profile will assign to the interface. STEP 7 | Add a Description for the profile. STEP 8 | Select the physical Link Type from the predefined list (ADSL/DSL, Cable modem, Ethernet, Fiber, LTE/3G/4G/5G, MPLS, Microwave/Radio, Satellite, WiFi, or Other). The firewall can support any CPE device that terminates and hands off as an Ethernet connection to SD-WAN Administrator’s Guide 3.1 35 ©2023 Palo Alto Networks, Inc. Configure SD-WAN the firewall; for example, WiFi access points, LTE modems, laser/microwave CPEs all can terminate with an Ethernet handoff. Private, point-to-point link types (MPLS, satellite, microwave, and Other) will form tunnels with only the same link type; for example, MPLS-to-MPLS and satellite-tosatellite. Tunnels will not be created between an MPLS link and an Ethernet link, for example. For existing PAN-OS deployments that have zones defined on the interfaces that will be used to support SD-WAN, Panorama may automatically configure the interface’s zone name to one of the predefined SD-WAN zones under the following conditions: 1. The SD-WAN interface is configured as a point-to-point private Link Type (MPLS, Satellite, or Microwave) in its Interface Profile. 2. The VPN Data Tunnel Support checkbox is disabled (unchecked) on the SD-WAN Interface Profile. This instructs PAN-OS to forward traffic in clear text outside of the SD-WAN VPN tunnel. On the Hub firewall, the zone name is configured as “zone-to-branch” when condition a) is met. On the Branch firewall, the zone name is configured as “zone-to-hub” when both condition a) and condition b) are met. Panorama automates this step to simplify configuration to ensure proper communication between the hub and branch firewalls. If you have preexisting firewall policies that reference the old zone name, you must update the policies to reflect the new predefined SD-WAN zone name. STEP 9 | Specify the Maximum Download (Mbps) speed from the ISP in megabits per second (range is 0 to 100,000; there is no default). You can enter a range using up to three decimal places, for example, 10.456. Ask your ISP for the link speed or sample the link’s maximum speeds with a tool such as speedtest.net and take an average of the maximums over a good length of time. STEP 10 | Specify the Maximum Upload (Mbps) speed to the ISP in megabits per second (range is 0 to 100,000; there is no default). You can enter a range using up to three decimal places, for example, 10.456. Ask your ISP for the link speed or sample the link’s maximum speeds with a tool such as speedtest.net and take an average of the maximums over a good length of time. STEP 11 | Select Eligible for Error Correction Profile interface selection to enable Forward Error Correction (FEC) or packet duplication for interfaces. You must enable this on both the encoding and decoding firewalls; you must also create an Error Correction profile to apply to the SD-WAN policy rule for specific applications. SD-WAN Administrator’s Guide 3.1 36 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 12 | VPN Data Tunnel Support determines whether the branch-to-hub traffic and return traffic flows through a VPN tunnel for added security (the default method) or flows outside of the VPN tunnel to avoid encryption overhead. • Leave VPN Data Tunnel Support enabled for public link types that have direct internet connections or internet breakout capability, such as cable modem, ADSL, and other internet connections. • You can disable VPN Data Tunnel Support for private link types such as MPLS, satellite, or microwave that do not have internet breakout capability. However, you must first ensure the traffic cannot be intercepted because it will be sent outside of the VPN tunnel. • The branch may have DIA traffic that needs to fail over to the private MPLS link connecting to the hub, and reach the internet from the hub. The VPN Data Tunnel Support setting determines whether the private data flows through the VPN tunnel or flows outside the tunnel, and the failed over traffic uses the other connection (that the private data flow doesn’t use). The firewall uses zones to segment DIA failover traffic from private MPLS traffic. STEP 13 | If you Configure DIA AnyPath, a principal virtual interface can have multiple hub virtual interfaces, so you must prioritize the order in which a particular hub is selected for failover. Specify such priority by setting the VPN Failover Metric for the VPN tunnels bundled in the hub virtual interface where this profile is applied. The lower the metric, the higher the priority of the interface to be selected during failover. If multiple hub virtual interfaces have the same metric value, SD-WAN sends new session traffic to them in round-robin fashion. SD-WAN Administrator’s Guide 3.1 37 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 14 | (Optional) Select the Path Monitoring mode in which the firewall monitors the interfaces where you apply this SD-WAN Interface Profile. The firewall selects what it considers the best monitoring method based on Link Type. Retain the default setting for the link type unless an interface (where you apply this profile) has issues that require more aggressive or more relaxed path monitoring. • Aggressive—(Default for all link types except LTE and Satellite) Firewall sends probe packets to the opposite end of the SD-WAN link at a constant frequency. Use this mode if you need fast detection and failover for brownout and blackout conditions. • Relaxed—(Default for LTE and Satellite link types) Firewall waits for a number of seconds (the Probe Idle Time) between sending sets of probe packets, making path monitoring less frequent. When the probe idle time expires, firewall sends probes for seven seconds at the Probe Frequency configured. Use this mode when you have low bandwidth links, links that charge by usage (such as LTE), or when fast detection isn’t as important as preserving cost and bandwidth. STEP 15 | Set the Probe Frequency (per second), which is the number of times per second that the firewall sends a probe packet to the opposite end of the SD-WAN link (range is 1 to 5; default is 5). The default setting provides subsecond detection of brownout and blackout conditions. If you change the Probe Frequency for a Panorama template, you should also adjust the Packet Loss percentage threshold in a Path Quality profile for a Panorama device group. STEP 16 | If you select Relaxed path monitoring, you can set the Probe Idle Time (seconds) that the firewall waits between sets of probe packets (range is 1 to 60; default is 60). STEP 17 | Enter the Failback Hold Time (seconds) that the firewall waits for a recovered link to remain qualified before the firewall reinstates that link as the preferred link after it has failed over (range is 20 to 120; default is 120). STEP 18 | Click OK to save the profile. STEP 19 | Commit and Commit and Push your configuration changes. STEP 20 | Monitor your application and link path health metrics, and generate reports of your application and link health performance. For more information, see Monitoring and Reporting. SD-WAN Administrator’s Guide 3.1 38 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure a Physical Ethernet Interface for SD-WAN In Panorama, configure a physical, Layer 3 Ethernet interface and enable SD-WAN functionality. To configure a physical interface, you must assign it an IPv4 address. You must also assign the interface a fully qualified next-hop gateway and assign an SD-WAN Interface Profile to the interface. (SD-WAN supports only a Layer 3 interface type; it does not support Layer 2 networks such as VPLS.) After you use Panorama to create a VPN cluster and export your hub and branch information in the CSV, an Auto VPN configuration in the SD-WAN plugin uses this information to generate a configuration for the associated branches and hubs that includes the predefined SD-WAN zones and creates secure VPN tunnels between SD-WAN branches and hubs. Auto VPN configuration also generates the BGP configuration if you enter BGP information in the CSV or in Panorama when you add an SD-WAN branch or hub. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Interfaces > Ethernet, select the appropriate template from the Template context drop-down, select a slot number, such as Slot1, and select an interface (for example, ethernet1/1). STEP 3 | Select the Interface Type as Layer3. STEP 4 | For a legacy routing engine, select a Virtual Router or create a new virtual router. For Advanced Routing Engine, select a Logical Router or create a new logical router. STEP 5 | Assign the Security Zone that is appropriate for the interface you’re configuring. For example, if you’re creating an uplink to an ISP, you must know that the Ethernet interface you chose is going to an untrusted zone. STEP 6 | If you're enabling SD-WAN on an IPv4 interface, select the IPv4 tab, Enable SD-WAN. With SD-WAN plugin 3.2.0 and later releases, you can configure up to four IP addresses for an SD-WAN enabled interface. The SD-WAN plugin uses only the first IP Address from the configured IP address list to create the SD-WAN tunnel. The SD-WAN considers only the first IP address for the Next Hop Gateway and ignores the remaining IP addresses in the list. (HA deployments only) If you wish to downgrade from SD-WAN plugin version 3.2.0 to 3.1.0 or earlier, remove the HA active/passive configurations on both the firewalls before attempting a downgrade procedure, such as downgrading PAN-OS and SD-WAN plugin versions. STEP 7 | For an IPv4 interface, select Type of address: • Static—In the IP field, Add an IPv4 address and prefix length for the interface. You can use a defined variable, such as $uplink, with a range of addresses. Enter the fully qualified IPv4 address of the Next Hop Gateway (the next hop from the IPv4 address you just entered). The Next Hop Gateway must be on the same subnet as the IPv4 address. The Next Hop Gateway is the IP address of the ISP’s default router that the ISP gave you when you SD-WAN Administrator’s Guide 3.1 39 ©2023 Palo Alto Networks, Inc. Configure SD-WAN bought the service. It is the next hop IP address to which the firewall sends traffic to reach the ISP’s network, and ultimately, the internet and the hub. • PPPoE—Enable PPPoE authentication for DSL links, enter the Username and Password, and Confirm Password. • DHCP Client—It’s critical that DHCP assigns a default gateway, also known as the next hop gateway for the ISP connection. The ISP will provide all the necessary connectivity information, such as dynamic IP address, DNS servers, and the default gateway. Although DHCP Client is supported for a hub or branch interface, on a hub interface it’s preferable for you to assign a Static address instead of DHCP Client. Using DHCP on a hub requires the Palo Alto Networks DDNS service. Using a Static address at the hub site creates a more stable environment because DDNS isn’t involved when resolving the DHCP IP address changes, and because the DDNS service can take a few minutes to register the new IP address when it changes. If you have multiple branch sites connecting to a hub site, having stability is critical to keeping the network up and running. If you select DHCP Client, be sure to disable the option Automatically create default route pointing to default gateway provided by server, which is enabled by default. STEP 8 | On the SD-WAN tab, select an SD-WAN Interface Profile that you already created (or create a new SD-WAN Interface Profile) to apply to this interface. The SD-WAN Interface Profile has an associatedm link tag, so the interfaces where this profile is applied will have the associated link tag. An interface can have only one link tag. SD-WAN Administrator’s Guide 3.1 40 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 9 | Click OK to save the Ethernet interface. STEP 10 | Commit and Commit and Push your configuration changes. STEP 11 | (SD-WAN manual configuration only) Configure a Virtual SD-WAN Interface. Auto VPN configuration will perform this task if you’re using Auto VPN. SD-WAN Administrator’s Guide 3.1 41 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN Physical firewalls running PAN-OS 11.1 and SD-WAN Plugin 2.1.0 support SD-WAN on aggregated Ethernet (AE) interfaces so that an SD-WAN firewall in a data center, for example, can have an aggregate interface group (bundle) of physical Ethernet interfaces that provide link redundancy. SD-WAN supports AE interfaces with or without subinterfaces. You can create an AE interface with subinterfaces that you can tag for different ISP services in order to provide endto-end traffic segmentation. Thus, your ISP services can reach multiple labs or buildings without needing a dedicated pair of fibers for each connection. A Layer 3 AE interface group connects to a router, as shown in the following figure: VM-Series firewalls do not support AE interfaces. An SD-WAN hub or branch firewall that has an AE interface should not belong to the same VPN cluster as a VM-Series SD-WAN hub or branch firewall because AE interfaces are not supported on VM-Series firewalls. PPPoE is not supported on subinterfaces. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Configure an SD-WAN Interface Profile for each ISP connection (subinterface) in the AE interface group to define its link attributes. SD-WAN Administrator’s Guide 3.1 42 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Create an AE interface group. 1. Select Network > Interfaces > Ethernet, select a Panorama Template, and Add Aggregate Group. 2. For Interface Name, enter the number to identify the aggregate group; range is 1 to 16. 3. For Interface Type, select Layer3. 4. Click OK. STEP 4 | Assign physical interfaces to the aggregate group. 1. Select Network > Interfaces > Ethernet and select the interface you want to assign to the aggregate group. 2. Select the Interface Type as Aggregate Ethernet. 3. Select the Aggregate Group you created; for example, ae1. 4. On the Advanced tab, select the Link Speed, Link Duplex, and Link State. 5. Click OK. 6. Repeat this step for each interface you want to assign to the aggregate group. SD-WAN Administrator’s Guide 3.1 43 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 5 | For the aggregate group, create a subinterface that uses a static IP address. 1. Select Network > Interfaces > Ethernet, highlight the aggregate interface, such as ae1, and click Add Subinterface at the bottom of the screen. 2. For Interface Name, enter a number after the period, such as 107. 3. Enter the VLAN Tag to differentiate between the subinterfaces. For ease of use, make the tag the same number as the subinterface ID. 4. Select the IPv4 tab and Enable SD-WAN. 5. Select the Type of address: Static. 6. Add the IP address (and subnet mask) of the subinterface. 7. Enter the IP address of the Next Hop Gateway. 8. Click OK. SD-WAN Administrator’s Guide 3.1 44 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Alternatively, for the aggregate group, create a subinterface that uses DHCP to get its address. 1. Select Network > Interfaces > Ethernet and in the Template field, select a Template Stack. 2. Highlight the aggregate interface, such as ae1, and click Add Subinterface at the bottom of the screen. 3. Highlight the subinterface and click Override at the bottom of the screen. 4. Highlight the subinterface and for Interface Name, enter a number after the period, such as 1. 5. Enter the VLAN Tag to differentiate between the subinterfaces. For ease of use, make the tag the same number as the subinterface ID. 6. Select the IPv4 tab and Enable SD-WAN. 7. Select the Type of address: DHCP Client. 8. Select Enable. 9. Uncheck (do not select) Automatically create default route pointing to default gateway provided by server. 10. Select the Advanced tab and DDNS tab. 11. Select Settings and Enable. The Hostname is automatically generated by the Panorama SD-WAN plugin. 12. Select the Vendor as Palo Alto Networks DDNS. 13. Click OK. SD-WAN Administrator’s Guide 3.1 45 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 7 | Apply an SD-WAN Interface Profile to the subinterface. 1. Highlight the subinterface you created and select the SD-WAN tab. 2. Select the SD-WAN Interface Profile you created for this link or create a new profile. 3. Click OK. STEP 8 | Repeat the prior steps to create additional Layer3 subinterfaces for the aggregate interface group and apply an SD-WAN Interface Profile to each subinterface. STEP 9 | Commit. SD-WAN Administrator’s Guide 3.1 46 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure Layer 3 Subinterfaces for SD-WAN Firewalls running PAN-OS 11.1 and SD-WAN Plugin 2.1.0 support SD-WAN on Layer 3 subinterfaces so that the firewall can segment traffic using VLAN tags. The following task shows how to create a Layer3 subinterface that uses a static IP address and how to create one that uses DHCP to get its address. It shows how to assign a VLAN tag to the subinterface and enable SDWAN on the subinterface. Create an SD-WAN interface profile to define each ISP connection and assign the profile to the corresponding subinterface (a virtual SD-WAN interface). If you configure SD-WAN Layer 3 subinterfaces on VM-Series firewalls, the VMware configuration must have respective portgroups attached to those interfaces that allow all VLANs. PPPoE is not supported on subinterfaces. STEP 1 | Configure an SD-WAN Interface Profile for each ISP connection (subinterface) to define its link attributes. STEP 2 | Create a Layer 3 subinterface that uses a static IP address. 1. Select Network > Interfaces > Ethernet and in the Template field select a template. 2. Select an interface. 3. For Interface Type, select Layer3 and click OK. 4. Highlight the interface and click Add Subinterface at the bottom of the screen. 5. After the Interface Name and period, enter the subinterface number. 6. Enter a Tag for the subinterface (range is 1 to 4,094). For ease of use, make the tag the same number as the subinterface ID. 7. On the IPv4 tab, Enable SD-WAN. 8. Select the Type of address: Static. 9. Add the IP address and subnet mask. 10. Enter the IP address of the Next Hop Gateway. 11. Click OK. SD-WAN Administrator’s Guide 3.1 47 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Alternatively, create a Layer 3 subinterface that uses DHCP to get its address. 1. Select Network > Interfaces > Ethernet and in the Template field, select a template stack (not a template). 2. Select an interface. 3. For Interface Type, select Layer3 and click OK. 4. Highlight the interface and click Add Subinterfaces at the bottom of the screen. 5. Highlight the subinterface and click Override. 6. Highlight the subinterface and after the Interface Name and period, enter the subinterface number. 7. Enter a Tag for the subinterface (range is 1 to 4,094). For ease of use, make the tag the same number as the subinterface ID. 8. On the IPv4 tab, Enable SD-WAN. 9. Select Type of address: DHCP Client and Enable. 10. Uncheck (do not select) Automatically create default route pointing to default gateway provided by server. 11. Select the Advanced tab and then the DDNS tab. 12. Select Settings and Enable. The Hostname is automatically generated by the Panorama SD-WAN plugin. 13. Select the Vendor as Palo Alto Networks DDNS. 14. Click OK. SD-WAN Administrator’s Guide 3.1 48 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Apply an SD-WAN Interface Profile to the subinterface. 1. Highlight the subinterface you created and select the SD-WAN tab. 2. Select the SD-WAN Interface Profile you created for this link or create a new profile. 3. Click OK. STEP 5 | Repeat the prior steps to add more subinterfaces to the interface. STEP 6 | Commit. SD-WAN Administrator’s Guide 3.1 49 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure a Virtual SD-WAN Interface If you use Auto VPN configuration through Panorama, it creates the SD-WAN interfaces for you, in which case you don’t create and configure a virtual SD-WAN interface. If you aren’t using Auto VPN configuration with Panorama, create and configure a virtual SDWAN interface to specify one or more physical, SD-WAN-capable ethernet interfaces that go to the same destination, such as to a specific hub or to the internet. In fact, all links in a virtual SDWAN interface must be the same type: all VPN tunnel links or all direct internet access (DIA) links. The first figure illustrates an example of an SD-WAN interface named SDWAN.901 that bundles two physical interfaces, which use different carriers: Ethernet1/1 (the cable modem link) and Ethernet1/2 (the fiber service link). Both links are a VPN tunnel from the branch to the hub. In this figure, both links in the SD-WAN interface happen to use the same link tag (Cheap Broadband), but links in an SD-WAN interface can have different link tags. In the following figure, SDWAN.902 bundles Ethernet1/1 and Ethernet1/2 links, which are both DIA links from the branch to the internet: STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Interfaces > SD-WAN and select the appropriate template from the Template context drop-down. SD-WAN Administrator’s Guide 3.1 50 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Add a logical SD-WAN interface by entering a number (in the range 1 to 9,999) after the sdwan. prefix. Auto VPN configuration creates SD-WAN interfaces numbered .901, .902, and so on. Hence, if you want to create the SD-WAN interfaces manually, do not use sdwan.90x format for SD-WAN interface name. STEP 4 | Enter a descriptive Comment. Add a helpful comment, such as Branch to internet or Branch to western USA hub if you are on the Branch template. Your comment makes troubleshooting easier rather than trying to decipher auto-generated names in logs and reports. STEP 5 | On the Config tab, assign the SD-WAN interface to a Virtual Router. STEP 6 | Assign the SD-WAN interface to a Security Zone. The virtual SD-WAN interface and all of its interface members must be in the same Security zone, thus ensuring the same Security policy rules apply to all paths from the branch to the same destination. STEP 7 | On the Advanced tab, Add Interfaces, which are members that go to the same destination, by selecting one or more Layer 3 Ethernet interfaces (for DIA) or one more virtual VPN tunnel interfaces (for hub). If you enter more than one interface, they must all be the same type (either VPN tunnel or DIA). The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA or a hub location. During routing, the route table determines which virtual SD-WAN interface (egress interface) the packet will exit based on the destination IP address in the packet. Then the SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule that the packet matches determine which path to use (and the order in which to consider new paths if a path deteriorates.) STEP 8 | Click OK to save your configuration change. STEP 9 | Commit and Commit and Push your configuration changes. SD-WAN Administrator’s Guide 3.1 51 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Create a Default Route to the SD-WAN Interface If you are using a service route to access Panorama™, to bring up the firewall you must create a default route that points to an SD-WAN interface you created. Auto VPN creates a virtual SD-WAN interface named sdwan.901 for DIA and creates a virtual SD-WAN interface named sdwan.902 for VPN tunnels. Auto VPN also creates its own default route that uses the sdwan.901 interface as its egress interface and uses a low metric, so that the sdwan.901 interface is preferred over the default route you created. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select the Template you are working on. STEP 3 | Select Network > Virtual Routers and select a virtual router, such as sd-wan. STEP 4 | Select Static Routes and Add a static route by Name. STEP 5 | For Destination, enter 0.0.0.0/0. STEP 6 | For egress Interface, select one of the logical SD-WAN interfaces you created to bring up the firewall. The egress interface you select can be any logical SD-WAN interface except sdwan.901 or sdwan.902. STEP 7 | For Next Hop, select None. STEP 8 | For Metric, enter a value greater than 50, so that this default route is not preferred over the default route that Auto VPN creates with a low metric. STEP 9 | Click OK. STEP 10 | Select Commit and Commit and Push your configuration changes. STEP 11 | Commit your changes. STEP 12 | Repeat this task for other templates on firewalls that use a service route to access Panorama. SD-WAN Administrator’s Guide 3.1 52 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure SD-WAN Link Management Profiles Create and configure a Path Quality, SaaS Quality, Traffic Distribution, and Error Correction Profile to manage SD-WAN link failovers. • Create a Path Quality Profile • Configure SaaS Monitoring • SD-WAN Traffic Distribution Profiles • Create a Traffic Distribution Profile • Create an Error Correction Profile Create a Path Quality Profile Create a Path Quality profile for each set of business-critical and latency-sensitive applications, application filters, application groups, services, service objects and service group objects that has unique network quality (health) requirements based on latency, jitter, and packet loss percentage. Applications and services can share a Path Quality profile. Specify the maximum threshold for each parameter, above which the firewall considers the path deteriorated enough to select a better path. As an alternative to creating a Path Quality profile, you can use any of the predefined Path Quality profiles, such as general-business, voip-video, file-sharing, audio-streaming, photo-video, and remote-access, and more. The predefined profiles are set up to optimize the latency, jitter, and packet loss thresholds for the type of applications and services suggested by the name of the profile. The predefined Path Quality profiles for a Panorama device group are based on the default Probe Frequency settings in the SD-WAN Interface profile for a Panorama template. If you change the default Probe Frequency setting, you must adjust the Packet Loss percentage threshold in the Path Quality profile for the firewalls in a Device Group that are affected by the Panorama template where you changed the Interface profile. The firewall treats the latency, jitter, and packet loss thresholds as OR conditions, meaning if any one of the thresholds is exceeded, the firewall selects the new best (preferred) path. Any path that has latency, jitter, and packet loss less than or equal to all three thresholds is considered qualified and the firewall selected the path based on the associated Traffic Distribution profile. By default, the firewall measures latency and jitter every 200ms and takes an average of the last three measurements to measure path quality in a sliding window. You can modify this behavior by selecting aggressive or relaxed path monitoring when you Configure an SD-WAN Interface Profile. If a path fails over because it exceeded the configured packet loss threshold, the firewall still sends probing packets on the failed path and calculates its packet loss percentage as the path recovers. It can take approximately three minutes for the packet loss percentage on a recovered path to fall below the packet loss threshold configured in the Path Quality profile. For example, suppose an SD-WAN policy rule for an application has a Path Quality profile that specifies a packet loss threshold of 1% and a Traffic Distribution profile that specifies Top Down distribution with tag 1 (applied to tunnel.1) first on the list and tag 2 (applied to tunnel.2) next on the list. SD-WAN Administrator’s Guide 3.1 53 ©2023 Palo Alto Networks, Inc. Configure SD-WAN When tunnel.1 exceeds 1% packet loss, the data packets fail over to tunnel.2. After tunnel.1 recovers to 0% packet loss (based on probing packets), it can take up to three minutes for the monitored packet loss rate for tunnel.1 to drop below 1%, at which time the firewall then selects tunnel.1 as the best path again. The sensitivity setting indicates which parameter (latency, jitter, or packet loss) is more important (preferred) for the applications to which the profile applies. When the firewall evaluates link quality, it considers a parameter with a high setting first. For example, when the firewall compares two links, suppose one link has 100ms latency and 20ms jitter; the other link has 300ms latency and 10 ms jitter. If the sensitivity for latency is high, the firewall chooses the first link. If the sensitivity for jitter is high, the firewall chooses the second link. If the parameters have the same sensitivity (by default the parameters are set to medium), the firewall evaluates packet loss first, then latency, and jitter last. As the SD-WAN Traffic Distribution Profiles concept states, the new path selection occurs in less than one second if you leave Path Monitoring and Probe Frequency with default settings; otherwise, new path selection could take more than one second. To achieve subsecond failover based on packet loss, you must set the latency sensitivity to high and the latency threshold to no more than 250ms. Reference the Path Quality profile in an SD-WAN policy rule to control the threshold at which the firewall replaces a deteriorating path with a new path for matching application packets. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select a Device Group. STEP 3 | Select Objects > SD-WAN Link Management > Path Quality Profile. STEP 4 | Add a Path Quality profile by Name using a maximum of 31 alphanumeric characters. STEP 5 | For Latency, double-click the Threshold value and enter the number of milliseconds allowed for a packet to leave the firewall, arrive at the opposite end of the SD-WAN tunnel, and a response packet to return to the firewall before the threshold is exceeded (range is 10 to 2,000; default is 100). STEP 6 | For Latency, select the Sensitivity (low, medium, or high). Default is medium. Click the arrow at the end of the Threshold column to sort thresholds in ascending or descending numerical order. STEP 7 | For Jitter, double-click the Threshold value and enter the number of milliseconds (range is 10 to 1,000; default is 100). SD-WAN Administrator’s Guide 3.1 54 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 8 | For Jitter, select the Sensitivity (low, medium, or high). Default is medium. STEP 9 | For Packet Loss, double-click the Threshold value and enter the percentage of packets lost on the link before the threshold is exceeded (range is 1 to 100.0; default is 1). Setting the Sensitivity for Packet Loss has no effect, so leave the default setting. If you change the Probe Frequency in an SD-WAN Interface profile for a Panorama template, you should also adjust the Packet Loss threshold for a Panorama device group. STEP 10 | Click OK. STEP 11 | Commit and Commit and Push your configuration changes. STEP 12 | Commit your changes. STEP 13 | Repeat this task for every Device Group. Configure SaaS Monitoring Configure a SaaS quality profile to monitor Direct Internet Access (DIA) links between a SaaS application and your branch firewall. SaaS application path monitoring is supported only for SD-WAN enabled PAN-OS firewalls. SaaS application path monitoring is not supported for Prisma Access Hubs. • Create a SaaS Quality Profile • Use Case: Configure SaaS Monitoring for a Branch Firewall • Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to the Same SaaS Application Destination • Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to a Different SaaS Application Destination Create a SaaS Quality Profile If your branch firewall has a Direct Internet Access (DIA) link to a Software-as-a-Service (SaaS) application, create a SaaS Quality profile to specify how one or more SaaS applications should be monitored. SaaS Quality profiles are associated with an SD-WAN policy rule to determine how the branch firewall determines the path quality thresholds for latency, jitter, and packet loss and selects the preferred path for an outgoing packet. The SaaS Quality profile supports up to four static IP addresses, or one fully qualified domain name (FQDN) or URL per SaaS Quality profile. When multiple static IP addresses are configured, the branch firewall monitors one IP address at a time in a cascading order based on how the IP addresses are ordered in the SaaS Quality profile. For example, if you add IP1, IP2, IP3, and IP4, the branch firewall monitors IP1 to determine if the path quality thresholds have been exceeded, then continues to IP2, and so forth. SD-WAN Administrator’s Guide 3.1 55 ©2023 Palo Alto Networks, Inc. Configure SD-WAN SD-WAN monitoring and reporting data displays the SaaS application and SaaS application IP, FQDN, or URL as it is currently configured in the SaaS Quality profile associated with an SD-WAN policy rule regardless of the time filter applied when viewing your SD-WAN monitoring data. For example, three days ago you initially configured the IP address of your SaaS application as 192.168.10.50 in a SaaS Quality profile and had traffic match the SDWAN policy rule to which the SaaS Quality profile is associated. Today, you reconfigured this existing SaaS Quality profile and changed the SaaS application IP address to 192.168.10.20. When you go review the SD-WAN monitoring data, all existing monitoring data for this SaaS application display the IP address 192.168.10.20. STEP 1 | Log in to the Panorama web interface. STEP 2 | Select Objects > SD-WAN Link Management > SaaS Quality Profile and specify the Device Group containing your SD-WAN configuration. STEP 3 | Add a new SaaS quality profile. STEP 4 | Enter a descriptive Name for the SaaS Quality profile. STEP 5 | (Optional) Enable (check) Shared to make the SaaS Quality profile shared across all device groups. STEP 6 | (Optional) Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. Disable override can only be enabled if Shared is disabled in the previous step. STEP 7 | Configure the SaaS Monitoring Mode. • Automatically monitor the SaaS application path health. Enabled by default, Adaptive monitoring allows the branch firewall to passively monitor the SaaS application session for send and receive activity to determine if the path quality SD-WAN Administrator’s Guide 3.1 56 ©2023 Palo Alto Networks, Inc. Configure SD-WAN thresholds have been exceeded. The SaaS application path health quality is automatically determined without any additional health checks on the SD-WAN interface. Adaptive SaaS monitoring is supported only for TCP SaaS applications. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per critical SaaS application that you need monitored. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. SaaS monitoring is resource-intensive and may impact firewall performance if monitoring a large number of SaaS applications. It is a best practice to only monitor those business-critical SaaS applications that need good usability. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. Configure a FQDN address object for the SaaS application. 2. Select IP Address/Object > FQDN and Add the FQDN. 3. Select the FQDN address object for the SaaS application. 4. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. SD-WAN Administrator’s Guide 3.1 57 ©2023 Palo Alto Networks, Inc. Configure SD-WAN • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. The minimum probe interval supported for a SaaS application HTTP/HTTPS is 3 seconds. 4. Click OK to save your configuration changes. STEP 8 | Select Commit and Commit and Push your configuration changes. Use Case: Configure SaaS Monitoring for a Branch Firewall If your organization is leveraging a business-critical SaaS application at a branch firewall location, you can configure a SaaS Quality profile and associate it with a SD-WAN policy rule to monitor the latency, jitter, and packet loss health metrics of the critical SaaS application and swap links from an SD-WAN branch firewall to a SaaS application on a Direct Internet Access (DIA) link to ensure application usability. If the business-critical SaaS application DIA link health metric thresholds are exceeded, the link is swapped to the next DIA link configured in the Traffic Distribution profile for all new sessions. The existing session on the degraded DIA link is not swapped over to the next DIA link. SD-WAN Administrator’s Guide 3.1 58 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 1 | Set up your SD-WAN deployment. 1. Install the SD-WAN Plugin. 2. Set Up Panorama and Firewalls for SD-WAN. 3. Add SD-WAN Devices to Panorama. 4. (High availability configurations only) Configure HA Devices for SD-WAN. 5. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. Creating a single Link Tag for multiple DIA links allows you to aggregate bandwidth between bundled links and allow the firewall to distribute sessions between multiple links. STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create a Path Quality profile to configure the latency, jitter, and packet loss thresholds and sensitivity in order to specify when the branch firewall should swap to the next DIA link. STEP 7 | Create a SaaS Quality profile to specify your SaaS application and the frequency the DIA link is monitored. STEP 8 | Create a Traffic Distribution profile to specify the order the branch firewall swaps to DIA links in the event of link health degradation. SD-WAN Administrator’s Guide 3.1 59 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 9 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to the Same SaaS Application Destination If your organization is leveraging a SaaS application at a branch firewall location but the branch firewall has no healthy DIA links to swap to, you can configure the hub firewall as a failover alternative to maintain a healthy connection to your SaaS application. If the SaaS application DIA link health metric thresholds are exceeded and the branch firewall has no healthy DIA links available, the link is swapped to the next hub firewall for all new sessions. The existing session on the degraded DIA link is not swapped over to the hub firewall. For example, say your branch and hub firewalls are located in the same region and access a SaaS application using the same destination IP. You can configure the hub firewall to act as a failover in the event there are no healthy DIA links from the branch firewall to the SaaS application by configuring an identically named SaaS Quality profile on both the branch and hub firewalls to automatically failover to the hub firewall if no healthy DIA links are available from the branch firewall. This allows you to maintain a health path for your SaaS application and maintain accurate end-to-end SaaS application monitoring data without congesting your network bandwidth. STEP 1 | Set up your SD-WAN deployment. 1. Install the SD-WAN Plugin. 2. Set Up Panorama and Firewalls for SD-WAN. 3. Add SD-WAN Devices to Panorama. 4. (High availability configurations only) Configure HA Devices for SD-WAN. 5. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. SD-WAN Administrator’s Guide 3.1 60 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create identically named SaaS quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. The easiest way to accomplish this is to create a single SaaS Quality profile in the Shared device group. SD-WAN Administrator’s Guide 3.1 61 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Alternatively, you can create two SaaS Quality profiles with identical names in different device groups and push them to your hub and branch firewalls. 1. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and from the Device Group drop-down select Shared. 2. Add a new SaaS Quality profile. 3. Enter a descriptive Name for the SaaS Quality profile. 4. Enable (check) Shared to make the SaaS Quality profile shared across all device groups. This is required to make the SaaS Quality profile available to all device groups your branch and hub firewalls belong to. 5. Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. 6. Configure the SaaS Monitoring Mode using one of the following methods. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per SaaS application. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. Configure a FQDN address object for the SaaS application. 2. Select IP Address/Object > FQDN and Add the FQDN. 3. Select the FQDN address object for the SaaS application. 4. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. STEP 7 | Create a Traffic Distribution profile to specify the order the branch firewall swaps from DIA links to VPN links to the hub firewall in the event of link health degradation. SD-WAN Administrator’s Guide 3.1 62 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 8 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to a Different SaaS Application Destination If your organization is leveraging a SaaS application at a branch firewall location but the branch firewall has no healthy DIA links to swap to, you can configure the hub firewall as a failover alternative to maintain a healthy connection to your SaaS application using a SaaS Quality profile pointing to a different SaaS application destination. If the SaaS application DIA link health metric thresholds are exceeded and the branch firewall has no healthy DIA links available, the link is swapped to the next hub firewall for all new sessions. The existing session on the degraded DIA link is not swapped over to the hub firewall. For example, say your branch and hub firewalls are located on opposite sides of the country and access a SaaS cloud application deployed in a cloud provider such as GCP. You can configure the hub firewall to act as a failover in the event there are no healthy DIA links from the branch firewall to the SaaS application. To accomplish this, configure an identically named SaaS Quality profile on both the branch and hub firewalls to automatically failover to the hub firewall if no healthy DIA links are available from the branch firewall. The SaaS Quality profile configured on the hub firewall to points to the on-ramp location closest to the hub to take advantage of local resources closest to it. This allows you flexibility in specifying healthy failover paths and the ability to maintain accurate end-to-end SaaS application monitoring data without congesting your network bandwidth. STEP 1 | Set up your SD-WAN deployment. 1. Install the SD-WAN Plugin. 2. Set Up Panorama and Firewalls for SD-WAN. 3. Add SD-WAN Devices to Panorama. 4. (High availability configurations only) Configure HA Devices for SD-WAN. 5. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. SD-WAN Administrator’s Guide 3.1 63 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create identically named SaaS quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. Create two SaaS SD-WAN Administrator’s Guide 3.1 64 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Quality profiles with identical names each pointing to a different SaaS application destination in different device groups and push them to your hub and branch firewalls. 1. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and select the target device group containing the branch firewall from the Device Group drop-down. 2. Add a new SaaS Quality profile. 3. Enter a descriptive Name for the SaaS Quality profile. 4. Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. 5. Configure the SaaS Monitoring Mode using one of the following methods. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per SaaS application. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. Configure a FQDN address object for the SaaS application. 2. Select IP Address/Object > FQDN and Add the FQDN. 3. Select the FQDN address object for the SaaS application. 4. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. 6. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and select the target device group containing the hub firewall from the Device Group drop-down. 7. Repeat Steps 6.2 through 6.5 to create an identically named SaaS Quality profile for a SaaS application at a different destination. SD-WAN Administrator’s Guide 3.1 65 ©2023 Palo Alto Networks, Inc. Configure SD-WAN This step is required to make in identically named SaaS Quality profile in the device group your hub firewall belongs to. STEP 7 | Create a Traffic Distribution profile to specify the order the branch firewall swaps from DIA links to VPN links to the hub firewall in the event of link health degradation. STEP 8 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. SD-WAN Traffic Distribution Profiles In an SD-WAN topology, the firewall detects a brownout, a blackout, and path deterioration per application and selects a new path to ensure you experience the best performance for your critical business applications. Having multiple ISP links allows you to scale your traffic capacity and reduce costs. The new path selection occurs in less than one second if you leave Path Monitoring and Probe Frequency with default settings; otherwise, new path selection could take more than one second. To implement such path selection, the firewall uses SD-WAN policy rules, which reference a Traffic Distribution profile that specifies how to select paths for session load distribution and for failover to a better path when path quality for an application deteriorates. Decide which traffic distribution method an application or service (that matches an SD-WAN policy rule) should use: • Best Available Path—Select this method if cost is not a factor and you will allow applications to use any path out of the branch. The firewall uses path quality metrics to distribute traffic and to fail over to one of the links belonging to a Link Tag in the list, thus providing the best application experience to users. • Top-Down Priority—If you have expensive or low-capacity links that you want used only as a last resort or as a backup link, use the Top-Down Priority method and place the tags that include those links last in the list of Link Tags in the profile. The firewall uses the top Link Tag in the list first to determine the links on which to session load traffic and on which to fail over. If none of the links in the top Link Tag are qualified based on the Path Quality profile, the firewall selects a link from the second Link Tag in the list. If none of the links in the second Link Tag are qualified, the process continues as necessary until the firewall finds a qualified link in the last Link Tag. If all associated links are overloaded and no link meets quality thresholds, the firewall uses the Best Available Path method to select a link on which to forward traffic. At the start of a failover event, the firewall starts at the top of the Top-Down Priority list of Link Tags to find a link to which it fails over. • Weighted Session Distribution—Select this method if you want to manually load traffic (that matches the rule) onto your ISP and WAN links and you don’t require failover during brownout conditions. You manually specify the link’s load when you apply a static percentage of new sessions that the interfaces grouped with a single Link Tag will get. The firewall distributes new sessions using round robin among the links having the specified Link Tags, until the link assigned the lowest percentage reaches that percentage of sessions. The firewall then uses the SD-WAN Administrator’s Guide 3.1 66 ©2023 Palo Alto Networks, Inc. Configure SD-WAN remaining link(s) in the same manner. You might select this method for applications that aren’t sensitive to latency and that require a lot of the link’s bandwidth capacity, such as large branch backups and large file transfers. If the link experiences brownout, the firewall doesn’t redirect the matching traffic to a different link. In the event of a failing path condition, the traffic distribution method you choose for application(s) in an SD-WAN policy rule, along with the Link Tags on groups of links, determine if and how the firewall selects a new path (performs link failover) as follows: Path Condition Top-Down Priority Best Available Path Weighted Session Distribution Session on existing path failed a path health threshold (brownout) Affected session fails over to better path (if available) Affected session fails over to better path (if available) Affected sessions don’t fail over Top-Down or Best Available Path recovered: existing path is still qualified (good) Affected session fails back to previous path Affected session stays on existing path, doesn’t fail back Affected sessions don’t fail over Top-Down or Best Available Path recovered: existing path fails a health check All sessions fail back to previous path Selective sessions fail back to previous path until affected existing path recovers Affected sessions don’t fail over Existing path is down (blackout) All sessions fail over to next path on list All sessions fail over to next best path All sessions fail over to other tags based on weight settings Brownout with no qualified (better) path Take best available path Take best available path Take best available path Additionally, the firewall automatically performs session load sharing among interface members of a single Link Tag. After those interfaces approach their maximum Mbps, new sessions flow over to interfaces having a different Link Tag (based on the traffic distribution method) if those interfaces have better health metrics. Path Condition Top-Down Priority Best Available Path Weighted Session Distribution Multiple links with the same SD-WAN Tag Share session load equally among links within SD-WAN Tag Share session load based on best path within SD-WAN Tag Share session load based on % weight SD-WAN Administrator’s Guide 3.1 67 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Path Condition Top-Down Priority Best Available Path Weighted Session Distribution assigned to SD-WAN Tag Multiple links with different SD-WAN Tags Share session load based on list priority, load link(s) in first SDWAN Tag first. Share session load based on best path from all SD-WAN Tags Share session load based on % weight assigned to SD-WAN Tags The following figure illustrates an example of a Traffic Distribution profile that uses the TopDown Priority method. The #1, #2, and #3 are the order of Link Tags of links the firewall examines, if necessary, to find a healthy path to complete an application session failover. For each separate failover event that arises, the firewall starts at the beginning of the Top-Down list of Link Tags. 1. In this Top-Down Priority example, packets from a branch carrying a specific application (for example, office365-enterprise-access) arrive at the firewall. The firewall uses the route table to determine the next hop to the destination and the outgoing interface, which is the virtual SD-WAN interface tunnel named sdwan.901. The Security policy rule allows the packets. The packets then match an SD-WAN policy rule (named Office365 to Hub1) that specifies the destination zone for the hub. The firewall uses the SD-WAN policy rule’s Path Quality profile, Traffic Distribution profile, and that profile’s Link Tags to determine which interface member SD-WAN Administrator’s Guide 3.1 68 ©2023 Palo Alto Networks, Inc. Configure SD-WAN (link) from sdwan.901 to use. The Traffic Distribution profile lists three Link Tags in this order: #1 Cheap Broadband, #2 HQ Backhaul, and #3 Backup (which is the order of Link Tags the firewall examines links to find a link to which it can fail over). 2. Assuming all paths are qualified (by the Path Quality profile), the firewall distributes the packets to one of the physical links tagged with first Link Tag in the Traffic Distribution profile list: Cheap Broadband. The sdwan.901 tunnel has two member interfaces (two carriers): the cable modem VPN tunnel and the fiber service VPN tunnel. The firewall first examines a link by round-robin, and chooses the first link it finds that is qualified, for example, the cable modem link. 3. If the first Cheap Broadband link (cable modem) isn’t a qualified link, the firewall selects the second Cheap Broadband link (fiber service). 4. If the second Cheap Broadband link (fiber service) isn’t a qualified link, the firewall selects the link tagged with the #2 link tag HQ Backhaul, which is a more expensive MPLS link to the same hub. 5. If the MPLS link isn’t a qualified link, the firewall selects the link tagged with the #3 link tag Backup, which is an even more expensive 5G LTE link to the same hub. 6. If the firewall doesn’t find a qualified link to fail over to, it uses the Best Available method to select a link. 7. Upon the start of a new failover event, the firewall starts at the top of the Top-Down list of Link Tags to find a link to which it will fail over. Keep in mind that SD-WAN traffic distribution is one of the later steps in the packet flow logic. Let’s zoom out to see a broader view of the packet flow. Packet flow details for the figure are as follows: 1. When a session for an application arrives at the firewall, the firewall performs session lookup to determine if the session is an existing session or new session. SD-WAN Administrator’s Guide 3.1 69 ©2023 Palo Alto Networks, Inc. Configure SD-WAN 2. A new session goes through session setup: 1. Forwarding lookup—The firewall gets the egress zone, egress interface, and virtual system from the Layer 3 route table or Layer 2 Forwarding Database lookup, etc. For applications that match an SD-WAN policy rule, the firewall uses the virtual SD-WAN interface as the egress interface. 2. NAT Policy lookup—If session matches a NAT rule, firewall does another forwarding lookup to determine the final (translated) egress interface and zone. 3. Security Policy lookup—If a Security Policy rule allows the session, the session is created and installed in the session table. The firewall then performs additional classification using AppID™ and User-ID™. 3. Content Inspection—The firewall performs Threat Inspection (Anti-Spyware for IPS ® [Vulnerability Protection], Antivirus, URL Filtering, WildFire , etc.) on payload and headers as needed. 4. The Forwarding/Egress stage performs path selection and forwards the packets. This stage is where SD-WAN path selection occurs. 1. Packet Forwarding Process—The firewall uses the ingress interface to determine the forwarding domain; performs routing, switching, or virtual wire forwarding. 2. SD-WAN path selection occurs when the application matches an SD-WAN policy rule; the Path Quality profile determines path qualification; the Traffic Distribution profile determines the method of path selection and the order in which paths are considered during the selection. 3. IPSec/SSL-VPN tunnel encryption occurs if needed. 4. Packet Egress Process - QoS shaping, DSCP rewrite, and IP fragmentation are applied (if needed). 5. Transmit Packet—The firewall forwards the packet over the selected egress interface. Now we zoom back in to examine the SD-WAN path selection logic in more detail. SD-WAN Administrator’s Guide 3.1 70 ©2023 Palo Alto Networks, Inc. Configure SD-WAN 1. The firewall consults the route table during Forwarding lookup; based on the destination IP address matching a Layer 3 prefix, the firewall determines the egress SD-WAN virtual interface. The packet is either going directly to the public internet or going back to the hub through a secure VPN link. 2. The firewall monitors each path by performing health checks that run over a VPN tunnel. Each DIA circuit has a VPN tunnel that monitors health information. 3. The application in the SD-WAN policy rule is associated with a Path Quality profile, and the firewall compares the path’s actual average latency, jitter, and packet loss values to the threshold values. 4. Any path that has a higher latency, jitter, or packet loss value than the threshold is not selected. 5. All qualifying paths in the virtual SD-WAN interface are then subjected to the Traffic Distribution profile’s method and path priority (ordering) logic. SD-WAN link tags group ISP services together, and the order of these tags in the Traffic Distribution profile prioritizes the paths during path selection. 6. Thus, the Path Quality Profile and the Traffic Distribution profile together determine the next best path to use and the firewall forwards the traffic out that link. Create a Traffic Distribution Profile Based on your SD-WAN configuration plan, create the SD-WAN Traffic Distribution Profiles you need based on how you want the applications in your SD-WAN policy rules to be session loaded and to fail over. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Ensure you already configured the Link Tags in an SD-WAN interface profile and committed and pushed them. The Link Tags must be pushed to your hubs and branches in order for SD-WAN Administrator’s Guide 3.1 71 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Panorama™ to successfully associate the Link Tags you specify in this Traffic Distribution profile to an SD-WAN interface profile. STEP 3 | Select a Device Group. STEP 4 | Create a Traffic Distribution profile. 1. Select Objects > SD-WAN Link Management > Traffic Distribution Profile. 2. Add a Traffic Distribution profile by Name using a maximum of 31 alphanumeric characters. 3. Select Shared only if you want to use this traffic distribution profile across all Device Groups (both hubs and branches). 4. Select one traffic distribution method and add a maximum of four Link Tags that use this method for this profile. • Best Available Path—Add one or more Link Tags. During the initial packet exchanges, before App-ID has classified the application in the packet, the firewall uses the path in the tag that has the best health metrics (based on the order of tags). After the firewall identifies the application, it compares the health (path quality) of the path it was using to the health of the first path (interface) in the first Link Tag. If the original path’s health is better, it remains the selected path; otherwise, the firewall replaces the original path. The firewall repeats this process until it has evaluated all the paths in the Link Tag. The final path is the path the firewall selects when a packet arrives that meets the match criteria. When a link becomes unqualified and must fail over to the next best path, the firewall can migrate a maximum of 1,000 sessions per minute from the unqualified link to the next best path. For example, suppose tunnel.901 has 3,000 sessions; 2,000 of those sessions match SD-WAN policy rule A and 1,000 sessions match SD-WAN policy rule B (both rules have a traffic distribution policy configured with Best Path Available). If tunnel.901 becomes unqualified, it takes three minutes to migrate the 3,000 sessions from the unqualified link to the next best path. • Top Down Priority—Add one or more Link Tags. The firewall distributes new sessions (that meet the match criteria) to links using the top-to-bottom order of the Link Tags you added. The firewall examines the first tag configured for this profile, and SD-WAN Administrator’s Guide 3.1 72 ©2023 Palo Alto Networks, Inc. Configure SD-WAN examines the paths that use that tag, selecting the first path it finds that is qualified (that is at or below the Path Quality thresholds for this rule). If no qualified path is found from that Link Tag, firewall examines the paths that use the next Link Tag. If the firewall finds no path after examining all paths in all of the Link Tags, the firewall uses the Best Available Path method. The first path selected is the preferred path until one of the Path Quality thresholds for that path is exceeded, at which point the firewall again starts at the top of the Link Tag list to find the new preferred path. If you have only one link at the hub, that link supports all of the virtual interfaces and DIA traffic. If you want to use the link types in a specific order, you must apply a Traffic Distribution profile to the hub that specifies Top Down Priority, and then order the Link Tags to specify the preferred order. If you apply a Traffic Distribution profile that instead specifies Best Available Path, the firewall will use the link, regardless of cost, to choose the best performing path to the branch. In summary, Link Tags in a Traffic Distribution profile, the Link Tag applied to a hub virtual interface, and a VPN Failover Metric in an SD-WAN Interface Profile work only when the Traffic Distribution profile specifies Top Down Priority. • Weighted Session Distribution—Add one or more Link Tags and then enter the Weight percentage for each Link Tag so that the weights total 100%. The firewall performs session load distribution between Link Tags until their percentage maximums are reached. If there is more than one path in the Link Tag, the firewall distributes equally using round-robin until the path health metrics are reached, and then distributes sessions to the other member(s) that are not at the limit. If multiple physical interfaces have the same tag, the firewall distributes matching sessions evenly among them. If all paths fail a health (path quality) threshold, the firewall selects the path that has the best health statistics. If no SD-WAN links are available (perhaps due to a blackout), the firewall uses static or dynamic routing to route the matching packets. If a packet is routed to a virtual SD-WAN interface, but the firewall cannot find a preferred path for the session based on the SD-WAN policy’s Traffic Distribution profile, the firewall implicitly uses the Best Available Path method to find the preferred path. The firewall distributes any application sessions that don’t match an SD-WAN policy rule based on the firewall’s implicit, final rule, which distributes the sessions in round-robin order among all available links, regardless of the Traffic Distribution profile. If you prefer to control how the firewall distributes unmatched sessions, create a final catch-all rule to Distribute Unmatched Sessions to specific links in the order you specify. 5. (Optional) After adding Link Tags, use the Move Up or Move Down arrows to change the order of tags in the list, so they reflect the order in which you want the firewall to use links for this profile and for the selected applications in the SD-WAN policy rule. 6. Click OK. STEP 5 | Commit and Commit and Push your configuration changes. SD-WAN Administrator’s Guide 3.1 73 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Commit your changes. Create an Error Correction Profile Forward error correction (FEC) is a method of correcting certain data transmission errors that occur over noisy communication lines, thereby improving data reliability without requiring retransmission. FEC is helpful for applications that are sensitive to packet loss or corruption, such as audio, VoIP, and video conferencing. With FEC, the receiving firewall can recover lost or corrupted packets by employing parity bits that the sending encoder embeds in an application flow. Repairing the flow avoids the need for SD-WAN data to fail over to another path or for TCP to resend packets. FEC can also help with UDP applications by recovering the lost or corrupt packets, since UDP does not retransmit packets. SD-WAN FEC supports branch and hub firewalls acting as encoders and decoders. The FEC mechanism has the encoder add redundant bits to a bitstream, and the decoder uses that information to correct received data if necessary, before sending it to the destination. SD-WAN also supports packet duplication as an alternative method of error correction. Packet duplication performs a complete duplication of an application session from one tunnel to a second tunnel. Packet duplication requires more resources than FEC and should be used only for critical applications that have low tolerance for dropped packets. Modern applications that have their own embedded recovery mechanisms may not need FEC or packet duplication. Apply FEC or packet duplication only to applications that can really benefit from such a mechanism; otherwise, much additional bandwidth and CPU overhead are introduced without any benefit. Neither FEC nor packet duplication is helpful if your SD-WAN problem is congestion. FEC and packet duplication functionality require Panorama to run PAN-OS 10.0.2 or a later release and SD-WAN Plugin 2.0 or a later release that is compatible with the PAN-OS release. The encoder and decoder must both be running PAN-OS 10.0.2 or a later release. If one branch or hub is running an older software release than what is required, traffic with an FEC or packet duplication header is dropped at that firewall. Beginning with PAN-OS 10.0.3, FEC and packet duplication are supported in a full mesh topology, in addition to the hub-spoke topology already supported. Neither FEC nor packet duplication should be used on DIA links; they are only for VPN tunnel links between branches and hubs. FEC and packet duplication is supported only for SD-WAN enabled PAN-OS firewalls. FEC and packet duplication is not supported for Prisma Access Hubs. To configure FEC or packet duplication on the encoder (the side that initiates FEC or packet duplication), use Panorama to: • Create an SD-WAN Interface Profile that specifies Eligible for Error Correction Profile interface selection and apply the profile to one or more interfaces. • Create an Error Correction Profile to implement FEC or packet duplication. • Apply the Error Correction Profile to an SD-WAN policy rule and specify a single application to which the rule applies. SD-WAN Administrator’s Guide 3.1 74 ©2023 Palo Alto Networks, Inc. Configure SD-WAN • Push the configuration to encoders. (The decoder [the receiving side] requires no specific configuration for FEC or packet duplication; the mechanisms are enabled by default on the decoder as long as the encoder initiates the error correction.) FEC and packet duplication support an MTU of 1,340 bytes. A packet larger than that will not go through the FEC or packet duplication process. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Configure an SD-WAN Interface Profile, where you select Eligible for Error Correction Profile interface selection to indicate that the firewall can automatically use the interfaces (where the SD-WAN Interface Profile is applied) for error correction. Whether this option defaults to selected or not depends on the Link Type you select for the profile. You can have Eligible for Error Correction Profile interface selection unchecked in a profile and apply the profile to an expensive 5G LTE link, for example, so that costly error correction is never performed on that link. STEP 3 | Configure a Physical Ethernet Interface for SD-WAN and apply the SD-WAN Interface Profile that you created to an Ethernet interface. STEP 4 | Create an Error Correction Profile for FEC or packet duplication. 1. Select Objects > SD-WAN Link Management > Error Correction Profile. 2. Add an Error Correction profile and enter a descriptive Name of up to 31 alphanumeric characters; for example, EC_VOIP. 3. Select Shared to make the Error Correction profile available to all device groups on Panorama and to the default vsys on a single-vsys hub or branch, or to vsys1 on a multivsys hub or branch to which you push this configuration. 4. Specify the Activate when packet loss exceeds (%) setting—When packet loss exceeds this percentage, FEC or packet duplication is activated for the configured applications in SD-WAN Administrator’s Guide 3.1 75 ©2023 Palo Alto Networks, Inc. Configure SD-WAN the SD-WAN policy rule where this Error Correction profile is applied. Range is 1 to 99; the default is 2. 5. Select Forward Error Correction or Packet Duplication to indicate which error correction method the firewall uses when an SD-WAN policy rule references this SDWAN Interface Profile; the default is Forward Error Correction. If you select Packet Duplication, SD-WAN selects an interface over which to send duplicate packets. (SDWAN selects one of the interfaces you configured with Eligible for Error Correction Profile interface selection in the prior step.) 6. (Forward Error Correction only) Select the Packet Loss Correction Ratio: 10% (20:2), 20% (20:4), 30% (20:6), 40% (20:8), or 50% (20:10)—Ratio of parity bits to data packets; the default is 10% (20:2). The higher the ratio of parity bits to data packets that the sending firewall (encoder) sends, the higher the probability that the receiving firewall (decoder) can repair packet loss. However, a higher ratio requires more redundancy and therefore more bandwidth overhead, which is a tradeoff for achieving error correction. The parity ratio applies to the encoding firewall’s outgoing traffic. For example, if the hub firewall parity ratio is 50% and the branch firewall parity ratio is 20%, the hub firewall will receive 20% and the branch firewall will receive 50%. 7. Specify the Recovery Duration (ms)—Maximum number of milliseconds that the receiving firewall (decoder) can spend performing packet recovery on lost data packets using the parity packets it received (range is 1 to 5,000; default is 1,000). The firewall immediately sends data packets it receives to the destination. During the Recovery Duration, the decoder performs packet recovery for any lost data packets. When the recovery duration expires, all the parity packets are released. You configure the recovery duration in the Error Correction Profile for the encoder, which sends the Recovery Duration value to the decoder. A Recovery Duration setting on the decoder has no impact. Start by using the default Recovery Duration setting and adjust it if necessary, based on your testing with normal and intermittent brown-outs. 8. Click OK. STEP 5 | Configure an SD-WAN Policy Rule, reference the Error Correction Profile you created in the rule, and specify a critical application to which the rule applies. Specify only one application in the SD-WAN policy rule when configuring FEC or packet duplication. You should not combine multiple applications in a single policy rule for FEC or packet duplication. SD-WAN Administrator’s Guide 3.1 76 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Commit and Commit and Push your configuration changes to the encoding firewalls (branches and hubs). SD-WAN Administrator’s Guide 3.1 77 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure an SD-WAN Policy Rule An SD-WAN policy rule specifies application(s) and/or service(s) and a traffic distribution profile to determine how the firewall selects the preferred path for an incoming packet that doesn’t belong to an existing session and that matches all other criteria, such as source and destination zones, source and destination IP addresses, and source user. The SD-WAN policy rule also specifies a path quality profile of thresholds for latency, jitter, and packet loss. When one of the thresholds is exceeded, the firewall selects a new path for the application(s) and/or service(s). When monitoring your SD-WAN traffic, traffic originating from a source behind the hub device is evaluated against the SD-WAN policies pushed to the hub device as it enters the hub device, and because the path selection decision has already been made, the branch device does not evaluate the traffic against its SD-WAN policies as it passes through the branch device to the final target device. Conversely, traffic originating from a source behind the branch device is evaluated against the SD-WAN policies pushed to the branch device and not by hub device. The Panorama™ management server aggregates the logs from both the hub and branch, and for the same traffic, two session entries are displayed but only the SD-WAN device that originally evaluated the traffic contains the SD-WAN details. In an SD-WAN policy rule, you can reference an Error Correction profile so that you can apply Forward Error Correction (FEC) or packet duplication for specified critical applications that have a low tolerance for dropped or corrupted packets. In an SD-WAN policy rule, you also specify the devices to which you want Panorama to push the rule. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Policies > SD-WAN and select the appropriate device group from the Device Group context drop-down. STEP 3 | Add an SD-WAN policy rule. STEP 4 | On the General tab, enter a descriptive Name for the rule. STEP 5 | On the Source tab, configure the source parameters of the policy rule. 1. Add the Source Zone or select Any source zone 2. Add one or more source addresses, set an external dynamic list (EDL), or select Any Source Address. 3. Add one or more source users or select any Source User. STEP 6 | On the Destination tab, configure the destination parameters of the policy rule. 1. Add the Destination Zone or select Any destination zone. 2. Add one or more destination addresses, set an EDL, or select Any Destination Address. SD-WAN Administrator’s Guide 3.1 78 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 7 | On the Application/Service tab, attach your SD-WAN Link Management profiles and specify your applications and services. PAN-OS 10.0.2 supports associating only a SaaS Quality Profile or an Error Correction but not both. If you associate one of these profiles with an SD-WAN policy rule, you cannot associate the other. For example, if you associate a SaaS Quality profile with an SD-WAN policy rule, you are unable to associate an Error Correction profile with the same SD-WAN policy rule. 1. Select the Path Quality or Create a Path Quality Profile. 2. Select the SaaS Quality Profile or Create a SaaS Quality Profile if the branch firewall has a Direct Internet Access (DIA) link to a SaaS application. The default is None (disabled). 3. Select the Error Correction Profile or Create an Error Correction Profile to apply forward error correction (FEC) or packet duplication to the applications that match the SD-WAN policy rule. The default is None (disabled). 4. Add Applications and select one or more applications from the list or select Any applications. All applications you select are subject to the health thresholds specified in the Path Quality profile you selected. If a packet matches one of these applications and that application exceeds one of the health thresholds in the Path Quality profile (and the packet matches the remaining rule criteria), the firewall selects a new preferred path. Add only business-critical applications and applications that are sensitive to path conditions for their usability. If you associate a SaaS Quality profile in Adaptive mode with the SD-WAN policy, add the specific SaaS applications you want to monitor. Using adaptive monitoring for all applications that match the SD-WAN policy rule may impact the performance of the SD-WAN firewall. If you associate a SaaS Quality profile with a specified SaaS application, add the SaaS application to the SD-WAN rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. 5. Add Services and select one or more services from the list or select Any services. All services you select are subject to the health thresholds specified in the Path Quality profile you selected. If a packet matches one of these services and that service exceeds SD-WAN Administrator’s Guide 3.1 79 ©2023 Palo Alto Networks, Inc. Configure SD-WAN one of the health thresholds in the Path Quality profile (and the packet matches the remaining rule criteria), the firewall selects a new preferred path. Add only business-critical services and services that are sensitive to path conditions for their usability. STEP 8 | On the Path Selection tab, select a Traffic Distribution profile or Create a Traffic Distribution Profile. When an incoming packet (unassociated with a session) matches all the match criteria in the rule, the firewall uses this Traffic Distribution profile to select a new preferred path. SD-WAN Administrator’s Guide 3.1 80 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 9 | On the Target tab, use one of the following methods to specify the target firewalls in the device group to which Panorama pushes the SD-WAN policy rule: • Select Any (target to all devices) (the default) to push the rule to all devices. Alternatively, select Devices or Tags to specify the devices to which Panorama pushes the SD-WAN policy rule. • On the Devices tab, select one or more filters to restrict the selections that appear in the Name field; then select one or more devices to which Panorama pushes the rule, as in this example: • On the Tags tab, Add one or more Tags and select the tag(s) to specify that Panorama push the rule to devices that are tagged with the selected tags, as in this example: • If you specified Devices or Tags, you can select Target to all but these specified devices and tags to have Panorama push the SD-WAN policy rule to all devices except for the specified devices or tagged devices. STEP 10 | Click OK. SD-WAN Administrator’s Guide 3.1 81 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 11 | Commit and Commit and Push your configuration changes. STEP 12 | (Best Practice) Create a catch-all SD-WAN policy rule to Distribute Unmatched Sessions so that you can control which links any unmatched sessions use and view unmatched sessions in logging and reports in the SD-WAN plugin. If you don’t create a catch-all rule to distribute unmatched sessions, the firewall distributes them in round-robin order among all available links because there is no traffic distribution profile for unmatched sessions. Round-robin distribution of unmatched sessions can increase your costs unexpectedly and result in loss of application visibility. STEP 13 | After configuring your SD-WAN policy rules, Create a Security Policy Rule to allow traffic (for example, bgp as an Application) from branches to the internet, from branches to hubs, and from hubs to branches. STEP 14 | (Optional) Configure QoS for critical applications. If the SD-WAN applications need guaranteed bandwidth capacities or if you do not want other applications taking bandwidth from critical business applications, create QoS rules to control the bandwidth properly. STEP 15 | To automatically set up BGP routing between VPN cluster members, in the SD-WAN plugin, Configure BGP routing between branches and hubs to dynamically route traffic that will be subject to the SD-WAN failover and load sharing. Alternatively, if you want to manually configure BGP routing on each firewall or use a separate Panorama template to configure BGP routing (for more control), leave the BGP information in the plugin blank. Instead, configure BGP routing. STEP 16 | Configure NAT for public-facing virtual SD-WAN interfaces. SD-WAN Administrator’s Guide 3.1 82 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Allow Direct Internet Access Traffic Failover to MPLS Link At an SD-WAN branch office, the firewall performs split tunneling so that any applications having a public IP address take the Direct Internet Access (DIA) interface to the internet, and applications having private IP addresses that belong to the hub take the VPN interface. The firewall automatically fails over DIA applications to the MPLS private connection to the hub when necessary, so that the traffic destined for the internet takes an alternative path through the hub to reach the internet. To allow this to work, you must do the following: STEP 1 | Create an MPLS link between your branch and hub. When you create the SD-WAN Interface profile, the link type must be MPLS for both the hub and branch. STEP 2 | If you want the private traffic to go through the VPN tunnel, enable VPN Data Tunnel Support in the SD-WAN Interface profile. If you disable VPN Data Tunnel Support, the private data will go outside of the VPN tunnel. STEP 3 | Configure an SD-WAN Policy Rule for specific applications, Create a Path Quality Profile, and Create a Traffic Distribution Profile that specifies the Top Down Priority method. The Traffic Distribution profile must also specify an MPLS link as one of the failover options (identified by a tag). Verify that the applications in the SD-WAN policy rule reference the correct Path Quality and Traffic Distribution profiles, and that the Traffic Distribution profile specifies Top Down Priority. After the VPN Data Tunnel Support is enabled on both the hub and branch and the MPLS link is operational, the firewall automatically uses the MPLS connection to fail over DIA traffic when necessary. STEP 4 | In the hub configuration, ensure the hub has a path to the internet and routing is properly set up for the hub traffic to reach the internet. The firewall uses the DIA virtual interface and the VPN virtual interface to ensure that the public internet traffic is kept separate from your private traffic in the same path; that is, the internet traffic and private traffic do not go through the same VPN tunnel. Full segmentation with proper zoning is in full effect. SD-WAN Administrator’s Guide 3.1 83 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure DIA AnyPath When your SD-WAN direct internet access (DIA) links from an ISP experience a blackout or brownout, you need those links to fail over to another link to ensure business continuity. DIA links can fail over to an MPLS link, but you may not have an MPLS link. DIA links must be able to fail over to another link that has a direct path or indirect path (through a hub or branch) to the internet; the DIA traffic can take any path available to get to the internet and isn’t restricted to DIA. DIA AnyPath supports a DIA link failing over to a private VPN tunnel going to a hub firewall to then reach the internet. Furthermore, if your topology is full mesh (branch-to-branch) and there is no hub, the DIA traffic can fail over to a branch firewall to reach the internet. DIA AnyPath requires PAN-OS 10.0.3 or a later PAN-OS release and the compatible SD-WAN Plugin release, which is shown in the SD-WAN table in the Panorama Plugins section of the Compatibility Matrix. There are several use cases for wanting an internet link to fail over to a VPN tunnel (DIA AnyPath): • You want to transition away from an expensive MPLS link to one or more public internet connections, usually from different vendors. • You have multiple hubs in a VPN cluster to allow a waterfall-type of failover from the primary hub to a succession of backup hubs. • In a split tunneling scenario, you want only a particular bandwidth-intensive application to go directly to the internet through the branch’s DIA link instead of going back to the data center hub through the VPN tunnel, thus saving WAN bandwidth costs. In the event of a DIA brownout or blackout, this application traffic fails over to the data center hub to reach the internet; and if necessary can subsequently fail over to a second hub to reach the internet. • In a different split tunneling scenario, you want most of your internet traffic to go out the DIA link rather than backhaul the traffic to the data center for internet breakout. However, you want specific applications (that may need additional scanning or logging by another security device) to go back to the data center. You create an SD-WAN policy rule to direct those applications to a primary path to the hub, rather than the normal DIA link as determined by the default route in the firewall’s route table. In the event of a brownout or blackout, those applications fail over to take the branch’s DIA interface. DIA AnyPath introduces the concept of a principal virtual interface, which can include both DIA links and nested hub virtual interfaces and branch virtual interfaces (VPN tunnels) that each include their own links. The principal virtual interface can have a maximum of nine DIA (Ethernet) interfaces, hub virtual interfaces, and branch virtual interfaces. You assign a Link Tag to a hub when you add the hub device to Panorama. Assuming you use the SD-WAN plugin, Auto VPN assigns that Link Tag to the hub virtual interface, which allows you to specify the tag in a Traffic Distribution profile to control the failover order among virtual interfaces. A principal virtual interface is referred to as a DIA-VIF in CLI commands. SD-WAN Administrator’s Guide 3.1 84 ©2023 Palo Alto Networks, Inc. Configure SD-WAN A principal virtual interface can have interface members that belong to different Security zones. However, the best practice is to have all member interfaces in the principal virtual interface belong to the same Security zone. Another best practice is to have at least one member interface in the principal virtual interface be of the link type Ethernet, Cable mode, ADSL, Fiber, LTE, or WiFi. The following topology example shows Branch1 with two ISP connections and an MPLS link. Branch1 also has a Hub1 virtual interface with three VPN tunnels connecting to Hub1, and a Hub2 virtual interface of three VPN tunnels connecting to Hub2. Branch1 also has a branch2 virtual interface with three VPN tunnels connecting to Branch2 and a branch3 virtual interface with three VPN tunnels connecting to Branch3. The goal of DIA AnyPath is to configure the order in which DIA can fail over to VPN tunnels to reach the internet directly or indirectly and thus maintain business continuity. When you configure a principal virtual interface, it automatically becomes the default route so that internet traffic is routed properly to any of the members of the principal virtual interface (both DIA links and VPN tunnels). The path selection is based on SD-WAN Path Quality profiles and Traffic Distribution profiles, which you would set to use the Top Down Priority distribution method to control the failover order. In the example topology, a Traffic Distribution profile can list the tag for the principal virtual interface first, then the tag for the Hub1 virtual interface, and then the tag for the Hub2 virtual interface. Zooming in to a deeper level of failover priority, a hub virtual interface has multiple tunnel members, so you need a way to prioritize the failover order of the members, such as prioritizing that a broadband VPN tunnel be used before an LTE VPN tunnel. You specify the priority SD-WAN Administrator’s Guide 3.1 85 ©2023 Palo Alto Networks, Inc. Configure SD-WAN using the VPN Failover Metric in the SD-WAN Interface Profile that you apply to the Ethernet interface. The lower the metric value, the higher the priority for the tunnel to be selected upon failover. In the topology example, in the Hub1 virtual interface, a lower VPN Failover Metric for t11 than for t12 causes internet traffic to fail over to t11 before t12. If multiple tunnels in a virtual interface have the same metric, SD-WAN sends new session traffic to the tunnels in round-robin fashion. STEP 1 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 86 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 2 | Specify the failover priority for a VPN tunnel bundled in a hub virtual interface or branch virtual interface. 1. Select or Configure an SD-WAN Interface Profile. Best Practice is to configure at least one interface with the link type of Ethernet, Cable modem, ADSL, Fiber, LTE, or WiFi. 2. You must enable VPN Data Tunnel Support. 3. Specify the VPN Failover Metric for a VPN tunnel; range is 1 to 65,535; default is 10. The lower the metric value, the higher the priority of VPN tunnel (link) where you apply this profile. For example, set the metric to a low value and apply the profile to a broadband interface; then create a different profile that sets a high metric to apply to an expensive LTE interface to ensure it is used only after broadband has failed over. If you have only one link at the hub, that link supports all of the virtual interfaces and DIA traffic. If you want to use the link types in a specific order, you must apply a Traffic Distribution profile to the hub that specifies Top Down Priority, and then order the Link Tags to specify the preferred order. If you apply a Traffic Distribution profile that instead specifies Best Available Path, the firewall will use the link, regardless of cost, to choose the best performing path to the branch. In summary, Link Tags in a Traffic Distribution profile. the Link Tag applied to a hub virtual interface (Step 6 in this task), and a VPN Failover Metric work only when the Traffic Distribution profile specifies Top Down Priority. 4. Click OK. STEP 3 | Configure a Physical Ethernet Interface for SD-WAN and on the SD-WAN tab, apply the SDWAN Interface Profile you created in the prior step. Best Practice is to have all interfaces in the principal virtual interface belong to the same Security zone. SD-WAN Administrator’s Guide 3.1 87 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Repeat Steps 2 and 3 to configure additional SD-WAN Interface Profiles with a different VPN failover metric and apply the profiles to different Ethernet interfaces to determine the order in which failover occurs to the links. STEP 5 | Create a Link Tag for a hub virtual interface. STEP 6 | Add the Link Tag to a hub that you want to participate in DIA AnyPath. 1. In Panorama > SD-WAN > Devices, Add an SD-WAN Device to add a hub to be managed by Panorama. 2. Select the hub. 3. Select the Link Tag that you created in the prior step, which Auto VPN applies to the whole hub virtual interface, not an individual link. Thus, you can reference this Link Tag in the Traffic Distribution Profile to indicate the hub virtual interface for the failover order for DIA AnyPath. On the branch device, Auto VPN uses this tag to populate the Link Tag field on the SD-WAN virtual interface that terminates on the hub device. 4. Click OK. STEP 7 | Repeat Steps 5 and 6 to create a Link Tag for each hub virtual interface and add the tag to each hub that will participate in DIA AnyPath. Do the same for any branch virtual interface. STEP 8 | Create a Traffic Distribution Profile to implement DIA AnyPath. 1. Create a Traffic Distribution Profile. 2. Select Top Down Priority. 3. Add the Link Tags so they appear in the order that you want their associated links used for failover. For example, if your use case is for certain applications to use DIA first, list the DIA tag first, then a hub virtual interface tag, then a second hub virtual interface tag. If your use case is for certain applications to first go to the hub and then the internet, list a hub virtual interface first, then perhaps a second hub virtual interface, and list a DIA tag last. If you have full mesh with no hub, use the DIA tag and branch virtual interface tags in the order you want. SD-WAN Administrator’s Guide 3.1 88 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 9 | Create identically named SaaS Quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. The easiest way to configure failover to a hub firewall with the same SaaS application destination is to create a single SaaS Quality profile in the Shared device group. Alternatively, you can create two SaaS Quality profiles with identical names in different device groups and push them to your hub and branch firewalls. To failover to a hub firewall with different SaaS application destinations, create two SaaS Quality profiles with identical names each pointing to a different SaaS application destination in different device groups and push them to your hub and branch firewalls. You must also create an SD-WAN policy rule that references this SaaS Quality profile in order to allow the hub to advertise link quality statistics for the SaaS Quality profile to the branch. Doing so will provide end-to-end SaaS monitoring through the hub. Without this SD-WAN policy rule, you would have only the link measurements from the branch to the hub, but not from the hub to the SaaS application. STEP 10 | Allow the hubs to participate in DIA AnyPath. 1. Create a VPN Cluster and select a hub. 2. Select Allow DIA VPN for the hub. A maximum of four hubs (any combination of PANOS hubs participating in DIA AnyPath and Prisma Access hubs) are supported. If they are HA hubs, a total of eight hubs are supported. If you Allow DIA VPN for one HA peer in a pair, you must also enable it for the other HA peer. SD-WAN Administrator’s Guide 3.1 89 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 11 | Create an SD-WAN policy rule for specific application(s) to use DIA AnyPath. 1. Configure an SD-WAN Policy Rule. 2. On the Application/Service tab, specify the applications and services for which you want to implement DIA AnyPath. 3. Associate the SaaS Quality Profile you created in the previous step. If you are configuring a SaaS Quality profile with different SaaS application destination, you must associate the SaaS Quality profile with the SD-WAN policy rule in each branch and hub device group. 4. On the Path Selection tab, select the Traffic Distribution Profile you created for the applications. STEP 12 | Route new sessions that don’t match any SD-WAN policy rule and sessions that arrive during a Panorama or firewall configuration change. 1. Create an appropriate Path Quality profile and Traffic Distribution profile to handle such sessions. 2. Configure an SD-WAN Policy Rule that is a catch-all rule for these sessions. 3. Place the rule last in the list. STEP 13 | Commit and Push to Devices. STEP 14 | Create a Security Policy Rule to allow DIA traffic to the Destination Zones named zoneinternet and zone-to-hub and specify the Applications subject to the rule. Commit and push to the branches. STEP 15 | Use the following CLI commands to monitor DIA information: 1. show sdwan connection <dia-vif-name> 2. show sdwan path-monitor stats dia-vif all 3. show sdwan path-monitor dia-anypath 4. show sdwan path-monitor dia-anypath packet-buffer all 5. show sdwan path-monitor stats conn-idx <IDX> SD-WAN Administrator’s Guide 3.1 90 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Distribute Unmatched Sessions The firewall attempts to match sessions that arrive at an SD-WAN virtual interface to an SDWAN policy rule; the firewall examines the SD-WAN policy rules in order from the top down, just as it does for Security policy rules. • If there is an SD-WAN rule match, the firewall executes the path monitoring and traffic distribution for that SD-WAN policy rule. • If there is no match to any SD-WAN policy rule in the list, the session matches an implied SD-WAN policy rule at the end of the list that uses the round-robin method to distribute unmatched sessions among all links in one SD-WAN interface, which is based on the route lookup. Furthermore, if there is no SD-WAN policy rule for a specific application, the firewall doesn’t track that application’s performance in the SD-WAN-specific visibility tools such as logging and reports in the SD-WAN plugin. To illustrate the implied policy rule: • Suppose the firewall has three SD-WAN policy rules: one rule specifies five voice applications, one rule specifies six video conferencing applications, and one rule specifies ten SaaS applications. • A session, for example, a video application session, arrives at the firewall and doesn’t match any of the SD-WAN policy rules. Because the session didn’t match a rule, the firewall has no path quality profile or traffic distribution profile to apply to the session. • Therefore, firewall matches the video application to the implied rule and distributes each video session among all of the available SD-WAN link tags and their associated links on the firewall, which could be two broadband links, an MPLS link, and an LTE link. Session 1 goes to one member of the broadband interface, session 2 goes to another member of the broadband interface, session 3 goes to MPLS, session 4 goes to LTE, session 5 goes to the first member of the broadband interface, session 6 goes to the second member of the broadband interface, and the round-robin distribution continues. You may not want to let your unmatched sessions resort to matching the implied SD-WAN rule because you have no control over that session distribution. Instead, we recommend you create a catch-all SD-WAN policy rule and place it last in the list of SD-WAN policy rules. A catch-all SDWAN policy rule lets you: • Control which links the unmatched sessions use. • View all of the applications on the firewall (including unmatched application sessions) in logging and reports in the SD-WAN plugin. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create a Path Quality Profile that sets very high latency, jitter, and packet loss thresholds that will never be exceeded. For example, 2,000ms latency, 1,000ms jitter, and 99% packet loss. SD-WAN Administrator’s Guide 3.1 91 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Create a Traffic Distribution Profile that specifies the SD-WAN link tags you want to use, in the order in which you want the links associated with those link tags to be used by unmatched sessions. If you don’t want unmatched applications to use a specific path (physical interface) at all, omit the tag that includes that link from the list of link tags in the traffic distribution profile. For example, if you don’t want an unmatched application such as movie streaming to use the expensive LTE link, omit the link tag for the LTE link from the list of link tags in the traffic distribution profile. STEP 4 | Add a catch-all SD-WAN policy rule and on the Application/Service tab, specify the Path Quality Profile that you created. STEP 5 | Select Any for the Applications and Service. STEP 6 | On the Path Selection tab, select the Traffic Distribution Profile you created. STEP 7 | Move the rule down to the last position in the list of SD-WAN policy rules. STEP 8 | Commit and Commit and Push your configuration changes. STEP 9 | Commit your changes. SD-WAN Administrator’s Guide 3.1 92 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Add SD-WAN Devices to Panorama Add a single SD-WAN hub or branch firewall or use a CSV to bulk import multiple SD-WAN hub and branch firewalls with pre-shared key or certificate authentication type. • Configure Certificate-Based Authentication for SD-WAN Devices • Add an SD-WAN Device • Bulk Import Multiple SD-WAN Devices • Onboard PAN-OS Firewalls to Prisma Access Configure Certificate-Based Authentication for SD-WAN Devices You can authenticate an SD-WAN device using either of the following two authentication types: • Pre-shared key (default authentication type) • Certificate (SD-WAN plugin 3.2.0 and later releases) When you create a new SD-WAN cluster or refresh the key with an SD-WAN plugin version earlier than 3.2.0, the SD-WAN plugin generates the pre-shared key automatically. In addition to the pre-shared key authentication type, we provide certificate-based authentication with SDWAN plugin 3.2.0 and later releases for next-generation firewalls to fulfill your security needs. Take your security to the next level with stronger authentication and validation for all SD-WAN sites with certificate-based authentication. We support certificate-based authentication on all software and hardware devices running legacy or advanced routing engines that support SD-WAN. Follow the steps mentioned in the Upgrade and Downgrade Considerations before you upgrade or downgrade your current SD-WAN plugin. Use the following workflow to configure certificate-based authentication for your SD-WAN device: STEP 1 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 93 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 2 | Generate a certificate for SD-WAN devices on Panorama. 1. Select Panorama > Certificate Management > Certificates. 2. Create a self-signed root CA certificate or import a certificate from your enterprise CA. Based on the root CA, generate devices certificate for an SD-WAN device. We do not support SCEP-generated certificate. The generated certificate must be unique for each SD-WAN device. That is, you can't generate a certificate and share it among multiple SD-WAN devices. Keep the following in mind while generating the branch and hub firewall certificates that is used for SD-WAN tunnel authentication: • Two different hub devices can use the same hub certificate. • Two different branch devices can use the same branch certificates if the following conditions are met: • Branch devices are not part of the same VPN cluster • There is no common hub device between the VPN clusters that these branch devices would be part of • (HA deployments only) Two different branch devices can also have the same branch certificates if they are configured as HA members. • If the hub device is common between VPN clusters, certificates for branch devices part of these VPN clusters should have unique certificates with all attributes having unique values. If you don't ensure the uniqueness of the certificate and its values, then commit will fail on the hub device (no commit failure on Panorama). Also ensure that the leaf certificates (branch and hub firewall certificates) used for SD-WAN tunnel authentication are generated meeting the following criteria: • Key usage should have digital signatures • All certificates must be signed by the same root CA • The device certificate must be directly signed by the root CA. • Certificate format should be PKCS12 The certificate attributes are used for determining the local ID and peer ID for IKE gateways. Hence, the leaf certificates, that is, the branch and hub firewall certificates that is used for SD-WAN tunnel authentication must be generated with the following SD-WAN Administrator’s Guide 3.1 94 ©2023 Palo Alto Networks, Inc. Configure SD-WAN three certificate attributes and each certificate attribute should be assigned with three unique attribute values. Otherwise, a commit error will be thrown. • FQDN (Host Name) • IP address (IP) • User FQDN (Alt Email) It's mandatory to have unique Host Name, IP, and Alt Email certificate attributes among all certificates. That is, none of the certificates should have these attribute values in common. In the below example, NewCertificate is generated with the total of nine mandatory certificate attributes. The Host Name certificate attribute is configured with three unique attribute values: pan-fw01.yourcompany.com, pan-fw02.yourcompany.com, and pan-fw03.yourcompany.com. The IP certificate attribute is configured with three unique attribute values: 192.0.2.0, 192.0.2.1, and 192.0.2.2. The Alt Email certificate attribute is configured with three unique attribute values: sales@yourcompany.com, IT@yourcompany.com, and customercare@yourcompany.com. SD-WAN Administrator’s Guide 3.1 95 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | (Optional) Configure a Certificate Profile that includes the root CA and intermediate CA for secure server communication. 1. Select Panorama > Certificate Management > Certificate Profile. 2. Configure a certificate profile. If you configure an intermediate CA as part of the Certificate Profile, you must also include the root CA. This Certificate Profile defines how the SD-WAN hubs and the branches authenticate mutually. STEP 4 | Import the CA certificates to validate the identity of the SD-WAN devices. 1. Panorama > Certificate Management > Certificates 2. Import the CA certificate and the key pair on Panorama for each SD-WAN device in a cluster or import multiple certificates using Multiple Certificates (.tar). Use CSV to bulk import the certificates into the Panorama management server. The CSV allows you to import multiple certificates at once, rather than adding each certificate manually. 3. Commit your changes. It's important to commit after importing the certificates for the imported certificates to be available for further configuration. STEP 5 | Configure a certificate-based authentication type while adding an SD-WAN hub or branch firewall to be managed by the Panorama management server. When adding your devices, you specify what type of device it is (branch or hub), an authentication type for the device, and you give each device its site name for easy identification. 1. Select Panorama > SD-WAN > Devices to add an SD-WAN device (SD-WAN hub or branch firewall) to be managed by the Panorama management server. 2. Select the VPN Tunnel tab and configure the authentication type. For certificatebased authentication, select Certificate and configure the certificate-related fields. It's mandatory to select an authentication type while adding an SD-WAN device. STEP 6 | Configure certificate-based authentication when onboarding PAN-OS firewalls to Prisma Access. 1. Select Panorama > SD-WAN > Devices to select the SD-WAN branch firewall to connect to the Prisma Access hub and configure the connection. 2. Select Prisma Access Onboarding and Add a compute node to a Region. In the VPN Tunnel, it's mandatory to select the authentication type to authenticate the CN (Prisma Access hub). For certificate-based authentication, select Certificate as the SD-WAN Administrator’s Guide 3.1 96 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Authentication type and configure the certificate-related fields. It's mandatory to select an authentication type while onboarding PAN-OS firewalls to Prisma Access. Ensure that you select the same authentication type for all the branch devices and the Prisma Access hub that is added. A commit failure occurs on Panorama if you try to use different authentication types for the branch and the Prisma hub. STEP 7 | Configure certificate-based authentication while creating a VPN cluster. 1. Select Panorama > SD-WAN > VPN Clusters. 2. Select the VPN cluster Type. 3. Select the Authentication Type as Certificate. It's mandatory to specify the authentication type to add a device in a VPN cluster. A VPN cluster should have the same authentication type selected for all its devices. You can't change the authentication type of an SD-WAN device that has been added to a VPN cluster already. If you want to change, then remove the VPN cluster and its SD-WAN devices and configure it again with the authentication type of your choice. By default, we support the pre-shared key authentication type for the devices in a VPN cluster (if you have not selected the certificate type manually). STEP 8 | Commit your configuration changes. STEP 9 | Select Push to Devices to push your configuration changes to your managed firewalls. Add an SD-WAN Device Add an SD-WAN hub or branch firewall to be managed by the Panorama™ management server. When adding your devices, you specify what type of device it is (branch or hub) and you give each device its site name for easy identification. Before adding your devices, plan your SD-WAN configuration to ensure that you have all the required IP addresses and you understand the SDWAN topology. This helps in reducing any configuration errors. ® If you have preexisting zones for your Palo Alto Networks firewalls, you’ll be mapping them to the predefined zones used in SD-WAN. If you want to have active/passive HA running on two branch firewalls or two hub firewalls, don’t add those firewalls as SD-WAN devices at this time. You’ll add them as HA peers separately when you Configure HA Devices for SD-WAN. If you’re using BGP routing, you must add a Security policy rule to allow BGP from the internal zone to the hub zone and from the hub zone to the internal zone. If you want to use 4-byte ASNs, you must first enable 4-byte ASNs for the virtual router. When viewing SD-WAN devices, if no data is present or the screen indicates that SDWAN is undefined, check in the Compatibility Matrix that the Panorama release you’re using supports the SD-WAN plugin release you’re trying to use. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Devices and Add a new SD-WAN firewall. SD-WAN Administrator’s Guide 3.1 97 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Select the managed firewall Name to add as an SD-WAN device. You must add your SDWAN firewalls as managed devices before you can add them as an SD-WAN device. STEP 4 | Select the Type of SD-WAN device. • Hub—A centralized firewall deployed at a primary office or location to which all branch devices connect using a VPN connection. Traffic between branches passes through the hub before continuing to the target branch, and connects branches to centralized resources at the hub location. The hub device processes traffic enforces policy rules, and manages link swapping at the primary office or location. • Branch—A firewall deployed at a physical branch location that connects the hub using a VPN connection and provides security at the branch level. The branch device processes traffic, enforces policy rules, and manages link swapping at the branch location. STEP 5 | Select the Router Name to use for routing between the SD-WAN hub and branches. By default, an sdwan-default virtual router is created and enables Panorama to automatically push router configurations. (Advanced Routing enabled) If you have configured advanced routing and the logical routers are created successfully, Router Name displays both virtual and logical router names: • If the virtual router and logical router names are the same, then the Router Name displays the same name because advanced routing creates a logical router with the same name as the virtual router, by default. It's important that the logical router name and virtual router name are the same for the same template when using the advanced routing engine. • If the virtual router and logical router names are different (which happens only when you update the logical router name manually), then the router name displays both the virtual and logical router names. You can select either virtual router (for legacy engine) or logical router (for advanced routing engine) based on your requirement. If you haven't enabled Advanced Routing, then you’ll have only virtual routers to select from the Router Name (for the legacy engine). STEP 6 | Enter the SD-WAN Site name to identify the geographical location or purpose of the device. The SD-WAN Site name supports all upper-case and lower-case alphanumerical and special characters. Spaces aren’t supported in the Site name and result in monitoring (Panorama > Monitoring) data for that site not to be displayed. All SD-WAN devices, including SD-WAN devices in a high availability (HA) configuration, must have a unique Site name. STEP 7 | Select the Link Tag you created for the hub virtual interface (or branch virtual interface), which Auto VPN will assign to the virtual interface. You’ll use this Link Tag in a Traffic Distribution profile to allow the hub (or branch) to participate in DIA AnyPath. STEP 8 | If you’re adding a hub that is behind a device performing NAT for the hub, you must specify the IP address or FQDN of the public-facing interface on that upstream NAT-performing device, so that Auto VPN configuration can use that address as the tunnel endpoint of the SD-WAN Administrator’s Guide 3.1 98 ©2023 Palo Alto Networks, Inc. Configure SD-WAN hub. It’s the IP address that the branch office’s IKE and IPSec flows must be able to reach. (You must have already configured a physical Ethernet interface for SD-WAN.) 1. On the Upstream NAT tab, enable Upstream NAT. 2. Add an SD-WAN interface; select an interface you already configured for SD-WAN. 3. Select IP Address or FQDN and enter the IPv4 address without a subnet mask (for example, 192.168.3.4) or the FQDN of the upstream device, respectively. 4. Click OK. Additionally, on the upstream device that is performing NAT you must set up the inbound destination NAT with a one-to-one NAT policy, and you must not configure port translation to the IKE or IPSec traffic flows. If the IP address on the upstream device changes, you must configure the new IP address and push it to the VPN cluster. You must use the CLI commands clear vpn ipsec-sa, clear vpn ike-sa, and clear session all on both the branch and hub. You must also clear session all on the virtual router where you configured the NAT policy for the IP addresses. Upstream NAT isn’t supported on Layer 2 interfaces. STEP 9 | If you’re adding a branch that is behind a device performing NAT for the branch, you must specify the IP address or FQDN of the public-facing interface on that upstream NATperforming device, or select DDNS to indicate that the IP address for the interface on the NAT device is obtained from the Palo Alto Networks DDNS service. Thus, Auto VPN Configuration uses that public IP address as the tunnel endpoint for the branch. It is the IP SD-WAN Administrator’s Guide 3.1 99 ©2023 Palo Alto Networks, Inc. Configure SD-WAN address that the branch office’s IKE and IPSec flows must be able to reach. (You must have already configured a physical Ethernet interface for SD-WAN.) 1. On the Upstream NAT tab, enable Upstream NAT. 2. Add an SD-WAN interface; select an interface you already configured for SD-WAN. 3. If you select the NAT IP Address Type to be Static IP, select IP Address or FQDN and enter the IPv4 address without a subnet mask (for example, 192.168.3.4) or the FQDN of the upstream device, respectively. 4. Alternatively, select the NAT IP Address Type to be DDNS. 5. Click OK. Additionally, on the upstream device that is performing NAT you must set up the inbound destination NAT with a one-to-one NAT policy, and you must not configure port translation to the IKE or IPSec traffic flows. If the IP address on the upstream device changes, you must configure the new IP address and push it to the VPN cluster. You must use the CLI commands clear ipsec, clear ike-sa, and clear session all on both the branch and hub. You must also clear session all on the virtual router where you configured the NAT policy for the IP addresses. There is a second location in the web interface where you can configure Upstream NAT for a branch, but the following location isn't preferred and you shouldn't configure Upstream NAT for a branch in both places. The secondary, non-preferred location to configure Upstream NAT is on Panorama at Network > Interfaces > Ethernet, select a template in the Template field, select an Ethernet interface, and select the SD-WAN tab. At this point you can Enable Upstream NAT, and select a NAT IP Address Type. This second method takes precedence. If Upstream NAT is first configured for the Ethernet interface on Panorama through the template stack, then the SD-WAN plugin won’t change the settings, even if you use different settings on the plugin device configuration page. Only if there is no Upstream NAT configured on Panorama through the template stack, then the plugin configuration for Upstream NAT takes effect. Upstream NAT is not supported on Layer 2 interfaces. STEP 10 | If your application traffic is tagged with Type of Service (ToS) bits or Differentiated Services Code Point (DSCP) markings, copy the ToS field from the inner IPv4 header to the outer VPN header of encapsulated packets going through the VPN tunnel to preserve QoS information. 1. Select the VPN Tunnel tab. 2. Select Copy ToS Header. 3. Click OK. SD-WAN Administrator’s Guide 3.1 100 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 11 | (Mandatory) (SD-WAN plugin 3.2.0 and later releases) Specify how to authenticate the peer. Select the Authentication type: Pre Shared Key or Certificate. If you choose a pre-shared key, the pre-shared key will be automatically generated. • You must use a unique certificate for each device in the SD-WAN cluster. • You can't change the authentication type after adding an SD-WAN device to the VPN cluster. • (HA deployments only) If you have set up a high availability (HA) pair in Panorama, then both the active and passive firewalls must use the same certificate. During the RMA process, you must configure the replacement firewall with the same certificate as the active firewall. If the certificate of an active firewall is revoked and a new certificate is pushed, then the passive firewall must also be updated with the new certificate. That is, both active and passive firewalls must have the same certificate configured in a high availability deployment. SD-WAN Administrator’s Guide 3.1 101 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 12 | (Only if enabling Certificate authentication type) Configure certificate-based authentication. 1. Select a Local Certificate—one that is already on the Panorama, Import a certificate, or Generate a new certificate. • If you need to Import a certificate, then first Import a Certificate for IKEv2 Gateway Authentication and then return to this task. We do not support SCEP-generated certificate. • If you want to Generate a new certificate, then first generate a certificate on the Panorama and then return to this task. The generated certificate must be unique for each SD-WAN device. That is, you can't generate a certificate and share it among multiple SD-WAN devices. Keep the following in mind while generating the branch and hub firewall certificates that is used for SD-WAN tunnel authentication: • Two different hub devices can use the same hub certificate. • Two different branch devices can use the same branch certificates if the following conditions are met: • Branch devices are not part of the same VPN cluster • There is no common hub device between the VPN clusters that these branch devices would be part of • (HA deployments only) Two different branch devices can also have the same branch certificates if they are configured as HA members. • If the hub device is common between VPN clusters, certificates for branch devices part of these VPN clusters should have unique certificates with all attributes SD-WAN Administrator’s Guide 3.1 102 ©2023 Palo Alto Networks, Inc. Configure SD-WAN having unique values. If you don't ensure the uniqueness of the certificate and its values, then commit will fail on the hub device (no commit failure on Panorama). Also ensure that the leaf certificates (branch and hub firewall certificates) used for SD-WAN tunnel authentication are generated meeting the following criteria: • Key usage should have digital signatures • All certificates must be signed by the same root CA • The device certificate must be directly signed by the root CA. • Certificate format should be PKCS12 • The certificate attributes are used for determining the local ID and peer ID for IKE gateways. Hence, the leaf certificates, that is, the branch and hub firewall certificates that is used for SD-WAN tunnel authentication must be generated with the following three certificate attributes and each certificate attribute should be assigned with three unique attribute values. Otherwise, a commit error will be thrown. • FQDN (Host Name) • IP address (IP) • User FQDN (Alt Email) It is mandatory to have unique Host Name, IP, and Alt Email certificate attributes among all certificates. That is, none of the certificates should have these attribute values in common. In the below example, NewCertificate is generated with the total of nine mandatory certificate attributes. The Host Name certificate attribute is configured with three unique attribute values: pan-fw01.yourcompany.com, pan-fw02.yourcompany.com, and pan-fw03.yourcompany.com. The IP certificate attribute is configured with three unique attribute values: 192.0.2.0, 192.0.2.1, and 192.0.2.2. The Alt Email certificate SD-WAN Administrator’s Guide 3.1 103 ©2023 Palo Alto Networks, Inc. Configure SD-WAN attribute is configured with three unique attribute values: sales@yourcompany.com, IT@yourcompany.com, and customercare@yourcompany.com. 2. (Optional) Choose a Certificate Profile. A certificate profile contains information about how to authenticate the peer gateway. 3. (Optional) Enable strict validation of peer’s extended key use to control strictly how the key can be used. STEP 13 | (Optional) Configure BGP routing. To automatically set up BGP routing between the VPN cluster members, enter the BGP information below. If you want to manually configure BGP routing on each firewall or use a separate Panorama template to configure BGP routing for more control, leave the BGP information below blank. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your preexisting BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. If the BGP configuration generated by the plugin conflicts with your preexisting BGP configuration, the preexisting BGP configuration takes precedence. If you want the pushed configuration to take precedence, you must enable the force template value when doing a Panorama push. 1. Select the BGP tab and enable BGP to configure BGP routing for SD-WAN traffic. 2. Enter the BGP Router ID, which must be unique among all routers. 3. (Substeps 3 through 6 pertain to the web interface for SD-WAN plugin 3.1.0 only. For later 3.1 releases, skip to Step 13.) Specify a static IPv4 Loopback Address for BGP SD-WAN Administrator’s Guide 3.1 104 ©2023 Palo Alto Networks, Inc. Configure SD-WAN peering. Auto VPN configuration automatically creates a loopback interface with the same IPv4 address that you specify. If you specify an existing loopback address, the commit will fail, so you should specify an IPv4 address that isn’t already a loopback address. 4. Enter the AS Number. The autonomous system number specifies a commonly defined routing policy to the internet. The AS number must be unique for every hub and branch location. 5. Disable the Remove Private AS option (the default is enabled) if you have endpoints that need to exchange routes with a hub or branch firewall in an SD-WAN BGP topology and therefore you don’t want to remove private AS numbers (64512 to 65534) from the AS_PATH attribute in BGP Updates. In this case, you want to allow the private AS numbers to leave the SD-WAN private AS in BGP Updates. The Remove Private AS setting applies to all BGP peer groups on the branch or hub firewall. If you need this setting to differ among BGP peer groups or peers, you must configure the setting outside of the SD-WAN plugin. If you change the Remove Private AS setting, commit to all SD-WAN cluster nodes, and later downgrade to an SD-WAN plugin version earlier than 2.0.2, then you must perform all configuration related to Remove Private AS outside of the SD-WAN plugin or directly on the firewalls. 6. Add the Prefix(es) to Redistribute. On a hub device, you must add at least one prefix to redistribute over the SD-WAN tunnel. Branch devices don’t have this mandatory configuration requirement because subnets connected to branch locations are redistributed by default. STEP 14 | (SD-WAN plugin 3.1.1 and later 3.1 releases using the legacy routing engine) To configure BGP to use IPv4, select IPV4 BGP. Whether your BGP environment is only IPv4 or dual stack (IPv4 and IPv6), you must enable IPv4 BGP. 1. Enable IPv4 BGP support. For an upgraded configuration (an already existing SD-WAN IPv4 configuration), Enable IPv4 BGP support is selected by default. Otherwise, explicitly Enable IPv4 BGP support. 2. Specify a static IPv4 Loopback Address for BGP peering. Auto VPN configuration automatically creates a Loopback interface with the same IPv4 address that you specify. If you specify an existing loopback address, the commit will fail, so you must specify an IPv4 address that isn’t already a loopback address. 3. Disable the Remove Private AS option (the default is enabled) if you have endpoints that need to exchange routes with a hub or branch firewall in an SD-WAN BGP topology and therefore you don’t want to remove private AS numbers (64512 to 65534) from SD-WAN Administrator’s Guide 3.1 105 ©2023 Palo Alto Networks, Inc. Configure SD-WAN the AS_PATH attribute in BGP Updates. In this case, you want to allow the private AS numbers to leave the SD-WAN private AS in BGP Updates. The Remove Private AS setting applies to all BGP peer groups on the branch or hub firewall. If you need this setting to differ among BGP peer groups or peers, you must configure the setting outside of the SD-WAN plugin. If you change the Remove Private AS setting, commit to all SD-WAN cluster nodes, and later downgrade to an SD-WAN plugin version earlier than 2.0.2, then you must perform all configuration related to Remove Private AS outside of the SD-WAN plugin or directly on the firewalls. 4. Add the Prefix(es) to Redistribute. On a hub device, you must enter at least one prefix to redistribute over the SD-WAN tunnel. Branch devices don't have this mandatory configuration requirement because subnets connected to branch locations are redistributed by default. STEP 15 | (SD-WAN plugin 3.1.1 and later 3.1 releases using the legacy routing engine) To configure BGP to use IPv6, select IPV6 BGP. 1. Enable IPv6 BGP support. 2. Specify a static IPv6 Loopback Address for BGP peering. Auto VPN configuration automatically creates a loopback interface with the same IPv6 address that you specify. SD-WAN Administrator’s Guide 3.1 106 ©2023 Palo Alto Networks, Inc. Configure SD-WAN If you specify an existing loopback address, the commit will fail, so you must specify an IPv6 address that isn't already a loopback address. 3. Add the Prefix(es) to Redistribute over the SD-WAN tunnel. On a hub device, you must enter at least one prefix to redistribute over the SD-WAN tunnel. Branch devices don't have this mandatory configuration requirement because subnets connected to branch locations are redistributed by default. STEP 16 | Click OK. STEP 17 | Select Group HA Peers at the bottom of the screen to display branches (or hubs) that are HA peers together. STEP 18 | Have Panorama create and push to firewalls a Security policy rule that allows BGP to run between branches and hubs. 1. At the bottom of the screen, for SD-WAN plugin 3.1.0, select BGP Policy or for SDWAN plugin 3.1.1 and later releases on a legacy routing engine, select IPv4 BGP Policy SD-WAN Administrator’s Guide 3.1 107 ©2023 Palo Alto Networks, Inc. Configure SD-WAN or IPv6 BGP Policy (depending on which type of BGP address you used) and Add a policy rule. 2. Enter a Policy Name for the Security policy rule that Panorama will automatically create. 3. Select Type as Hub or Branch. 4. Select Device Groups to specify the device groups to which Panorama pushes the Security policy rule. 5. Click OK. SD-WAN Administrator’s Guide 3.1 108 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 19 | Select Push to Devices to push your configuration changes to your managed firewalls. Bulk Import Multiple SD-WAN Devices Add multiple SD-WAN devices to quickly onboard branch and hub firewalls, rather than manually adding each device one at a time. When adding your devices, you specify what type of device it is (branch or hub) and you give each device its site name for easy identification. Before adding your devices, plan your SD-WAN configuration to ensure you have all the required IP addresses and that you understand the SD-WAN topology. This helps reduce any configuration errors. If you want to have active/passive HA running on two branch firewalls or two hub firewalls, don’t add those firewalls as SD-WAN devices in your CSV file. You’ll add them as HA peers separately when you Configure HA Devices for SD-WAN. If you’re using BGP routing, you must add a Security policy rule to allow BGP from the internal zone to the hub zone and from the hub zone to the internal zone. If you want to use 4-byte autonomous system numbers (ASNs), you must first enable 4-byte ASNs for the virtual router. If you have preexisting zones for your Palo Alto Networks firewalls, you’ll be mapping them to the predefined zones used in SD-WAN. STEP 1 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 109 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 2 | Select Panorama > SD-WAN > Devices > Device CSV and Export an empty SD-WAN device CSV. The CSV allows you to import multiple branch and hub devices at once, rather than adding each device manually. STEP 3 | Populate the SD-WAN device CSV with the branch and hub information and save the CSV. All fields are required unless noted otherwise. Enter the following for each hub and branch: • device-serial—The serial number of the branch or hub firewall. • type—Specify whether the device is a branch or a hub. • site—Enter the SD-WAN device site name to help you identify the geographical location or purpose of the device. The SD-WAN Site name supports all upper-case and lower-case alphanumerical and special characters. Spaces aren’t supported in the Site name and result in monitoring (Panorama > SD-WAN > Monitoring) data for that site not to be displayed. All SD-WAN devices, including SD-WAN devices in a high availability (HA) configuration, must have a unique Site name. • router-name—Enter the virtual router to use for routing between the SD-WAN hub and branches. By default, Panorama creates an sdwan-default virtual router and enables Panorama to automatically push router configurations. • vif-link-tag—Specify a link tag to identify the hub when applications and services use this link during SD-WAN traffic distribution and failover. • (Optional) router-id—Specify the BGP router ID, which must be unique among all virtual or logical routers. Enter the Loopback Address as the router ID. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. • (Optional) as-number—Enter the ASN of the private AS to which the virtual router on the hub or branch belongs. The SD-WAN plugin supports only private autonomous systems. SD-WAN Administrator’s Guide 3.1 110 ©2023 Palo Alto Networks, Inc. Configure SD-WAN The ASN must be unique for every hub and branch. The 4-byte ASN range is 4,200,000,000 to 4,294,967,294 or 64512.64512 to 65535.65534. The 2-byte ASN range is 64512 to 65534. Use a 4-byte private ASN. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. • (Optional) ipv4-bgp-enable—Specify yes or no to enable or disable BGP for IPv4 addresses. • (Optional) loopback-address—Specify a static loopback IPv4 address for BGP peering. SDWAN plugin 3.1.1 and later 3.1 releases support an IPv6 loopback address for BGP peering. • (Optional) remove-private-as—Specify no to disable the Remove Private AS option (default is enabled) if you have endpoints that need to exchange routes with a hub or branch firewall in an SD-WAN BGP topology and therefore you don’t want to remove private AS numbers (64512 to 65534) from the AS_PATH attribute in BGP Updates. This setting applies to all BGP peer groups on the branch or hub firewall. If you need this setting to differ among BGP peer groups or peers, you must configure the setting outside of the SD-WAN plugin. • (Optional) prefix-redistribute—Enter IP prefixes that the branch informs the hub it can reach. To add more than one prefix, separate prefixes with a space, an ampersand (&), and a space; for example, 192.2.10.0/24 & 192.168.40.0/24. By default, the branch firewall advertises all locally connected internet prefixes to the hub. Palo Alto Networks doesn’t redistribute the branch office default route(s) learned from the ISP. • (Optional) ipv6-bgp-enable—Specify yes/no to enable/disable BGP for IPv6 addresses. • (Optional) ipv6-loopback-address—Specify a static loopback IPv6 address for BGP peering. • (Optional) ipv6-prefix-redistribute—Enter IPv6 prefixes to redistribute to the hub router from the branch. By default, all locally connected internet IPv6 prefixes are advertised to the hub location. • (Optional) copy-tos-header—Specify yes/no to enable/disable this option to copy the Type of Service (TOS) header from the inner IP header to the outer IP header of the encapsulated packets in order to preserve the original TOS information. • authentication-type—Specify the authentication type that the device (hub or branch) supports: pre-shared key or certificate authentication. • (Only for Certificate authentication type) certificate-name—Enter a certificate name. The name is case-sensitive and can have up to 63 characters on the firewall or up to 31 SD-WAN Administrator’s Guide 3.1 111 ©2023 Palo Alto Networks, Inc. Configure SD-WAN characters on the Panorama. It must be unique and use only letters, numbers, hyphens, and underscores. For pre-shared key authentication type, this field should be left empty. STEP 4 | Import the SD-WAN device CSV into Panorama. Verify that there are no pending commits on Panorama or the import fails. 1. On Panorama, Select Panorama > SD-WAN > Devices > Device CSV and Import the CSV you edited in the previous step. 2. Browse and select the SD-WAN device CSV. 3. Click OK to import the SD-WAN devices. STEP 5 | Verify that your SD-WAN devices were successfully added. STEP 6 | Commit your configuration changes. STEP 7 | Select Push to Devices to push your configuration changes to your managed firewalls. Onboard PAN-OS Firewalls to Prisma Access SD-WAN plugin 2.2 provides Prisma Access hub support, in which PAN-OS firewalls connecting to Prisma Access compute nodes (CNs) achieve cloud-based security in an SD-WAN hub-andspoke topology. In this topology, the SD-WAN hubs are Prisma Access CNs (IPSec Termination Nodes) and the SD-WAN branches are PAN-OS firewalls. A maximum of four hubs (any combination of PAN-OS hubs participating in DIA AnyPath and Prisma Access hubs) are supported. SD-WAN automatically creates IKE and IPSec tunnels that connect the branch to the hub. Review the system requirements for SD-WAN and Prisma Access. It's important to configure Prisma Access first, and then configure SD-WAN. • If you're starting a brand new Prisma Access configuration, read the Prisma Access Administrator’s Guide and complete Phase 1 and then Phase 2 configuration steps. • If you already have Prisma Access running, ensure Phase 1 is complete, and then complete Phase 2. SD-WAN Administrator’s Guide 3.1 112 ©2023 Palo Alto Networks, Inc. Configure SD-WAN The following flowchart shows the order of the two configuration phases and basic steps within each phase. The full Prisma Access prerequisites with links and the configuration steps for SDWAN follow the flowchart. PHASE 1—PRISMA ACCESS PHASE 2—SD-WAN (COMPLETE PHASE 1 FIRST) (BEGIN ONLY AFTER COMPLETING PHASE 1) 1. Set up the infrastructure subnet, infrastructure BGP AS, template stack and device group for a tenant. 1. Configure a branch firewall with an interface that has SD-WAN enabled. 2. Set up template stacks, templates, device groups, trust and untrust zones, and bandwidth allocation for specific regions. 3. Specify the BGP local address pool for loopback addresses. 3. Ensure your Prisma Access deployment is licensed for remote networks. 4. Ensure your deployment allocates bandwidth per compute location, instead of by location. 5. Ensure you have assigned bandwidth to the compute location that corresponds to the location to which you want to onboard. 6. Perform a local commit and push to the Prisma Access cloud. 2. Log in to the Panorama web interface. 4. Select the SD-WAN branch firewall to connect to the Prisma Access hub and configure the connection. 5. Commit and Push the configuration to the cloud. 6. Verify that onboarding is complete. 7. Synchronize the branch firewall to Prisma Access. 8. Commit to Panorama. 9. Push to Devices. 10.View the new interface that was created. 11.Verify the IPSec tunnel is up. 12.Verify the IKE gateway is up. 13.Create an SD-WAN policy rule to generate monitoring data. 14.Commit and Commit and Push to branch firewalls. 15.Monitor Prisma Access hub application and link performance. Before you connect SD-WAN to Prisma Access, you must have a branch firewall with an interface that has SD-WAN enabled. Additionally, ensure you have performed the following Prisma Access prerequisites for one or more tenants; these are the Phase 1 steps: 1. For Panorama > Cloud Services > Configuration, set up the infrastructure subnet, infrastructure BGP AS, template stack and device group for a tenant on the Service Setup page. 2. On the Remote Networks page, set up template stacks, templates, device groups, trust and untrust zones, and bandwidth allocation for specific regions. SD-WAN Administrator’s Guide 3.1 113 ©2023 Palo Alto Networks, Inc. Configure SD-WAN 3. Ensure your Prisma Access deployment is licensed for remote networks by selecting Panorama > Licenses and checking your license information. • Licenses available after November 17, 2020 show the amount of licensed bandwidth you have for remote networks in the Net Capacity area. • Licenses available before November 17, 2020 show the available remote network bandwidth in the GlobalProtect Cloud Service for Remote Networks area under Total Mbps. 4. Ensure your deployment allocates bandwidth per compute location, instead of by location. 5. Ensure you have assigned bandwidth to the compute location that corresponds to the location to which you want to onboard. Prisma Access allocates one IPSec termination node per 500 Mbps of bandwidth you allocate to a region. 6. Perform a local commit and push to the Prisma Access cloud. After you have performed the preceding steps for Phase 1 with Prisma Access, perform the following Phase 2 steps for SD-WAN. STEP 1 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 114 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 2 | Specify the BGP local address pool for loopback addresses. 1. Select Panorama > SD-WAN > VPN Clusters. 2. At the bottom of the screen, select BGP Prisma Address Pool. 3. Add an unused private subnet (prefix and netmask) for the local BGP addresses for Prisma Access. 4. Click OK. 5. Commit. Do not simply change an existing address pool if Prisma Access is already onboarded. If you need to change an address pool, perform the following steps during a maintenance window to update the SD-WAN branch and the Prisma Access CN with your address pool changes: 1. Use Panorama to access an SD-WAN branch and delete the existing onboarding that the address pool change will impact; then do a local Commit. 2. Update the VPN address pool, and then do a local Commit. 3. Perform the Prisma Access onboarding again, and then do a local Commit and Push. SD-WAN Administrator’s Guide 3.1 115 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Select the SD-WAN branch firewall to connect to the Prisma Access hub and configure the connection. 1. Select Panorama > SD-WAN > Devices. 2. Select the branch firewall on which you enabled SD-WAN, whose name then populates the Name field. 3. Select the Type of device as Branch. 4. Select the Router Name. 5. Enter the Site. All SD-WAN devices must have a unique Site name. 6. Select Prisma Access Onboarding and Add. 7. Select a local, SD-WAN-enabled Interface on the firewall to connect to the Prisma Access hub. 8. Select a Prisma Access Tenant (select default for a single tenant environment). All SD-WAN interfaces on a branch firewall must use the same Prisma Access tenant. 9. Enter a helpful Comment. SD-WAN Administrator’s Guide 3.1 116 ©2023 Palo Alto Networks, Inc. Configure SD-WAN 10. Add a compute node to a Region by selecting the region where the CN (Prisma Access hub) is located. There can be multiple regions per interface. 11. Select an IPSec Termination Node (GP gateway) from the list of nodes; the list is based on the nodes that Prisma Access spun up for the region earlier. You are choosing the hub SD-WAN Administrator’s Guide 3.1 117 ©2023 Palo Alto Networks, Inc. Configure SD-WAN to which this branch connects. SD-WAN Auto VPN configuration builds IKE and IPSec relationships and tunnels with this node. 12. Enable BGP for communication between the branch and hub (Enable is the default). 13. Advertise Default Route to allow the Prisma Access hub’s default route to be advertised to the branch firewall. 14. Summarize Mobile User Routes before advertising to have the Prisma Access hub advertise summarized mobile user IP subnet routes, thereby reducing the number of advertisements to the branches. 15. Don’t Advertise Prisma Access Routes to prevent the IPSec Termination Node/hub from advertising its Prisma Access routes to the SD-WAN branches. 16. Enter the Secret for authentication of BGP communications and Confirm Secret. 17. (SD-WAN plugin 3.2.0 and later releases) Configure the VPN tunnel parameters and authentication type to authenticate the PAN-OS firewall and Prisma Access hub. 1. (Optional) If you want to preserve the Type of Service (ToS) information in the encapsulated packets, select Copy TOS Header. If there are multiple sessions inside the tunnel (each with a different ToS value), copying the ToS header can cause the IPSec packets to arrive out of order. 2. Select the Authentication: Pre Shared Key or Certificate. Ensure that you select the same authentication type for all the branch devices and the Prisma Access device that is added. The pre-shared key is automatically generated if selected as an authentication type for a region. 18. Select Certificate to configure certificate-based authentication. 19. (Only if enabling Certificate authentication type) The certificate must be present on the Panorama before performing Prisma Access Onboarding of the SD-WAN branch firewall. SD-WAN Administrator’s Guide 3.1 118 ©2023 Palo Alto Networks, Inc. Configure SD-WAN We do not support SCEP-generated certificate. Select a Local Certificate—one that is already on the Panorama. Ensure the following for the certificate that you have in the Panorama for the successful Prisma Access Onboarding process: • The certificate must be unique for each SD-WAN device. That is, you can't share the certificate among multiple SD-WAN devices. Keep the following in mind while generating the branch and hub firewall certificates that is used for SD-WAN tunnel authentication: • Two different hub devices can use the same hub certificate. • Two different branch devices can use the same branch certificates if the following conditions are met: • Branch devices are not part of the same VPN cluster • There is no common hub device between the VPN clusters that these branch devices would be part of • (HA deployments only) Two different branch devices can also have the same branch certificates if they are configured as HA members. • If the hub device is common between VPN clusters, certificates for branch devices part of these VPN clusters should have unique certificates with all attributes having unique values. If you don't ensure the uniqueness of the certificate and its values, then commit will fail on the hub device (no commit failure on Panorama). Also ensure that the leaf certificates (branch and hub firewall certificates) used for SD-WAN tunnel authentication are generated meeting the following criteria: • Key usage should have digital signatures • All certificates must be signed by the same root CA • The device certificate must be directly signed by the root CA • Certificate format should be PKCS12 • The certificate attributes are used for determining the local ID and peer ID for IKE gateways. Hence, the leaf certificates, that is, the branch and hub firewall certificates that is used for SD-WAN tunnel authentication must be generated with the following three certificate attributes and each certificate attribute should be assigned with three unique attribute values. Otherwise, a commit error will be thrown. • FQDN (Host Name) • IP address (IP) • User FQDN (Alt Email) It's mandatory to have unique Host Name, IP, and Alt Email certificate attributes among all certificates. That is, none of the certificates should have these attribute values in common. In the below example, NewCertificate is generated with the total of nine mandatory certificate attributes. The Host Name certificate attribute is configured with three SD-WAN Administrator’s Guide 3.1 119 ©2023 Palo Alto Networks, Inc. Configure SD-WAN unique attribute values: pan-fw01.yourcompany.com, pan-fw02.yourcompany.com, and pan-fw03.yourcompany.com. The IP certificate attribute is configured with three unique attribute values: 192.0.2.0, 192.0.2.1, and 192.0.2.2. The Alt Email certificate attribute is configured with three unique attribute values: sales@yourcompany.com, IT@yourcompany.com, and customercare@yourcompany.com. 20. (Optional) (Only if enabling Certificate authentication type) Choose a Certificate Profile. A Certificate Profile contains information about how to authenticate the peer gateway. 21. (Optional) Enable strict validation of peer’s extended key use to control strictly how the key can be used. 22. Select a Link Tag for the hub. When you want to enable ECMP for a Prisma Access hub, onboard more than one branch interface to the same compute node (CN) and use the same Link Tag on those branch interfaces. 23. Click OK. The display will include a Peer AS number and the Tunnel Monitor IP address provided by Prisma Access. SD-WAN Administrator’s Guide 3.1 120 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Commit and Push the configuration to the cloud, where Prisma Access spins up the correct number of IPSec Termination Nodes based on requested bandwidth. When more than one IPSec tunnel is going to the same CN, the Prisma Access configuration has ECMP enabled with symmetric return, as shown in this Prisma Access example: SD-WAN Administrator’s Guide 3.1 121 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 5 | Verify that onboarding is complete. 1. Select Panorama > Cloud Services > Status and verify that the Remote Networks Deployment Status displays success. 2. Select the Remote Networks Deployment Status details. 3. Confirm that the Prisma Access node completion displays 100%. SD-WAN Administrator’s Guide 3.1 122 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Synchronize the branch firewall to Prisma Access to retrieve the service IP address(es) of the CNs. 1. Select Panorama > SD-WAN > Devices. 2. Select the SD-WAN branch device. 3. Select Prisma Access Onboarding and Sync To Prisma (and respond to message to continue). Repeat for each branch device. After the sync to Prisma is successful, you will see the Prisma Access configuration parameters on the SD-WAN branch firewall. If not, wait for approximately 15 minutes and repeat the Sync to Prisma. If necessary, go to the Prisma Access plugin and verify that the CN onboarding has finished (you can see the CN with the bandwidth and IP addresses assigned). After that verification, retry Sync To Prisma. STEP 7 | Commit to Panorama. STEP 8 | Push to Devices to push to the local branch firewall. Edit Selections to select the Push Scope Selection. Select the correct Template and Device Group. STEP 9 | On the branch firewall, select Network > Interfaces > SD-WAN and see the new interface that was created with the Link Tag you created, assigned to the Security Zone named zoneto-pa-hub, and with the IPSec tunnel connecting to the CN. STEP 10 | Select Network > IPSec Tunnels and verify the IPSec tunnel is up. STEP 11 | Select Network > Network Profiles > IKE Gateways and verify the IKE gateway is up. SD-WAN Administrator’s Guide 3.1 123 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 12 | Create an SD-WAN policy rule to generate monitoring data. This step is required to baseline Prisma Access Hub latency, jitter, and packet loss data for accurate traffic distribution. SD-WAN monitoring data is generated from traffic that matches your SD-WAN policy rules. 1. Create a Traffic Distribution Profile. 2. Create a Path Quality Profile with high latency, jitter, and packet loss thresholds. A Path Quality profile is required to create an SD-WAN policy rule. Creating a Path Quality profile with high thresholds allows you to baseline latency, jitter, and packet loss for the Prisma Access hub without causing app to swap to a different link. 3. Configure an SD-WAN Policy Rule. STEP 13 | Commit and Commit and Push to branch firewalls. STEP 14 | (Only if enabling Pre Shared Key authentication type) Refresh the Prisma IKE pre-shared key. If you need to change the current Prisma IKE key that is used to secure the IPSec connection between the branch and the Prisma hub, perform this step to randomly generate a new key for the tunnel and update both sides of the tunnel. Perform this step when the hub and branch are not busy. Do not create an IKE gateway manually with a name beginning with “gw_” because such names are reserved for Prisma IKE creation during onboarding. This step to refresh the Prisma IKE pre-shared key refreshes all such named IKE gateways if there are any apart from those created by Prisma Access. 1. Select Panorama > SD-WAN > Devices and select a device. 2. At the bottom of the screen, select Refresh Prisma IKE Key. 3. A message appears notifying you that Refreshing the IKE key will update all SD-WAN tunnels between the branch and the Prisma Access hub and will require a simultaneous configuration push to all branch and Prisma Access hub devices. Best practice recommendation is to perform the refresh during a maintenance window as traffic can be affected. Do you wish to continue? Select Yes if you wish to continue. SD-WAN Administrator’s Guide 3.1 124 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 15 | Commit and Commit and Push to branch firewalls. STEP 16 | Monitor Prisma Access Hub Application and Link Performance to understand the baseline latency, jitter, and packet loss for the links to Prisma Access. This step is required to gather accurate latency, jitter, and packet loss data to fine-tune your Prisma Access hub Path Quality profiles. SD-WAN Administrator’s Guide 3.1 125 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure HA Devices for SD-WAN You can configure two firewalls as a branch in active/passive HA mode (or two firewalls as a hub in active/passive HA mode) to be part of your SD-WAN environment. In this case, Panorama™ needs to push the same configuration to the active peer and the passive peer, rather than treat the two firewalls individually. To make that happen, you configure active/passive HA before adding the devices for SD-WAN, so that Panorama is aware the devices are HA peers and pushes the same configuration to them. (Only HA active/passive mode is supported.) Read through the following procedure before you begin so you don’t Commit after adding your HA peers as SD-WAN devices. In HA, the firewall does not synchronize SD-WAN session distribution statistics. After an HA failover, the session distribution statistics display only statistics of new sessions; statistics of existing sessions are lost. STEP 1 | Before you enable SD-WAN on your HA peers, configure Active/Passive HA on two firewall models that support SD-WAN. STEP 2 | Add the HA peers as SD-WAN devices, but don’t perform the last step to Commit. STEP 3 | In Panorama, select Panorama > Managed Devices > Summary. STEP 4 | At the bottom of the screen, select Group HA Peers. Confirm that under the Status display, the HA Status column includes the two firewalls, one Active and one Passive. Panorama is aware of the HA status and will push the same SD-WAN configuration to the two HA peers when you commit. STEP 5 | Commit and Commit and Push. SD-WAN Administrator’s Guide 3.1 126 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Create a VPN Cluster In your SD-WAN configuration, you must configure one or more VPN clusters to determine which branches communicate with which hubs and creates a secure connection between the branch and hub devices. VPN clusters are logical groupings of devices, so consider things such as geographical location or function when logically grouping your devices. ® PAN-OS supports both hub-spoke and full mesh SD-WAN VPN topologies. In a hub-spoke topology, a centralized firewall hub at a primary office or location acts as the gateway between branch devices. The hub-to-branch connection is a VPN tunnel. In this configuration, traffic between branches must pass through the hub. The first time you Configure a Virtual SD-WAN Interface with direct internet access (DIA) links for an SD-WAN hub or branch firewall, a VPN cluster called autogen_hubs_cluster is automatically created and the SD-WAN firewall is automatically added to the VPN cluster. This allows the Panorama™ management server to Monitor SD-WAN Application and Link Performance for devices that are protected by the SD-WAN firewall and accessing resources outside of your corporate network. Additionally, any SD-WAN firewall with DIA links that you configure in the future are automatically added to the autogen_hubs_cluster VPN cluster containing all hubs and branches with DIA links to allow Panorama to monitor application and link performance. The autogen_hubs_cluster is purely for monitoring application and link health, and not to create VPN tunnels between the hubs and branches with DIA links. If you need to connect hubs and branches with VPN tunnels, you must create a new VPN cluster and add all the required hubs and branches to that cluster. When you select Pre Shared Key as an Authentication Type, a strong, random IKE pre-shared key is created for all hubs and branches in the VPN cluster to secure the VPN tunnels, and each firewall has a master key that encrypts the pre-shared key. The system secures the pre-shared key, even from the administrator. You can refresh the IKE pre-shared key, which Panorama sends to all members of the cluster. Refresh the pre-shared key when cluster members are not busy. When you select Certificate as the Authentication Type, then the hubs and branches in the SDWAN VPN cluster is based on the certificate-based authentication. After the SD-WAN plugin is upgraded to 2.1.0, the hub and branch firewalls in a single VPN cluster must all run either PAN-OS 10.0.4 (or a later 10.0 release) or 10.1.0, not a combination of the two releases. When viewing VPN clusters, if no data is present or the screen indicates that SD-WAN is undefined, check in the Compatibility Matrix that the Panorama release you are using supports the SD-WAN plugin release you are trying to use. STEP 1 | Plan your branch and hub VPN topology to determine which branches communicate with each of your hubs. For more information, see Plan Your SD-WAN Configuration. STEP 2 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 127 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 3 | Specify IP address ranges for the IPSec VPN tunnels that Auto VPN configuration creates. Auto VPN configuration creates a VPN tunnel between a hub and branches and assigns IP addresses to the tunnel endpoints. Enter subnet ranges that you want Auto VPN to use as VPN tunnel addresses. You can enter up to 20 IP prefix/netmask ranges. Auto VPN draws from that pool for VPN tunnel addresses, drawing from the largest range first (and the drawing from the next largest range when necessary). You must configure at least one range for the pool. If you don’t perform this step before pushing the configuration to a hub or branch, the Commit and Push will fail. If you upgrade from an earlier SD-WAN plugin release, you must check that your ranges are still correct. If it is not, enter new ranges. After you Commit, all tunnels are dropped and new tunnels are used, so perform this task during a time you have low traffic. 1. Select Panorama > SD-WAN > VPN Clusters. 2. At the bottom of the screen, select VPN Address Pool. 3. Add one or more (up to 20) Member IP address and netmask ranges, for example, 192.168.0.0/16. 4. Click OK. Do not simply change an existing address pool if Prisma Access is onboarded. If you need to change an address pool, perform the following steps during a maintenance window to update the branch and the Prisma Access CN with your address pool changes: 1. Use Panorama to access an SD-WAN branch and delete the existing onboarding that the address pool change will impact; then do a local Commit. 2. Update the VPN address pool, and then do a local Commit. 3. Perform the Prisma Access onboarding again, and then do a local Commit and Push. SD-WAN Administrator’s Guide 3.1 128 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 4 | Configure the VPN cluster. Repeat this step to create VPN clusters as needed. 1. Select Panorama > SD-WAN > VPN Clusters and Add a VPN cluster. 2. Enter a descriptive name for the VPN cluster. Underscores and spaces are not supported in the VPN cluster name and result in monitoring (Panorama > SD-WAN > Monitoring) data for the cluster not to be displayed. Choose the name of the VPN cluster carefully so you do not need to change the name in the future. SD-WAN monitoring data is generated based on the old cluster name and cannot be reconciled to a new cluster name, and will cause issues with the number of reported clusters when monitoring your VPN clusters or generating reports. 3. Select the VPN cluster Type. Only Hub-Spoke VPN cluster type is supported in PAN-OS 10.0.2 and earlier 11.1 releases. Beginning with PAN-OS 10.0.3, you can Create a Full Mesh VPN Cluster with DDNS Service. 4. (SD-WAN plugin 3.2.0 and later releases) Select the Authentication Type: Pre Shared Key or Certificate. It's mandatory to specify the authentication type to add a device in a VPN cluster. A VPN cluster should have the same authentication type selected for all its devices. When you select the authentication type for a VPN cluster, only the branches and hubs that are configured with the same authentication type (as the VPN cluster) can be added to the VPN cluster. For example, when you select certificate as the authentication type for a VPN cluster, then all the hubs and branches that are added to the cluster should be configured with certificate as the authentication type. You can't modify the authentication type or VPN cluster name of a VPN cluster that has been configured already. To make a change, remove the VPN cluster and its SDWAN devices and configure it again using the new authentication type or VPN cluster SD-WAN Administrator’s Guide 3.1 129 ©2023 Palo Alto Networks, Inc. Configure SD-WAN name. By default, we support the pre-shared authentication type for the devices in a VPN cluster (if the certificate method is not selected manually). • Once you've configured the VPN cluster, you cannot change the cluster name or its authentication type (both cluster and device level). • You cannot have different authentication types within a single VPN cluster. That is, a VPN cluster authentication type must match with all the SD-WAN devices in the VPN cluster. Any difference would result in a commit failure. • You can have different VPN clusters with different authentication types configured. • In a VPN cluster, you cannot have SD-WAN devices with different authentication types selected. If an SD-WAN hub is part of two VPN clusters, then the two clusters should be configured with the same authentication type. If you want to change the authentication type to Certificate for an existing VPN cluster, delete the VPN cluster and create it again with the authentication type of your choice. After creating a VPN cluster with certificate authentication type, if you want to downgrade to a PAN-OS or SD-WAN plugin version that does not support certificate authentication type, follow these steps: • Delete the existing VPN cluster. SD-WAN device authentication will automatically change to pre-shared key on downgrade. • Downgrade to the PAN-OS or SD-WAN plugin version of your preference. See the system requirements for SD-WAN for the minimum PAN-OS and SD-WAN plugin versions required for configuring the certificate authentication type. Follow the steps mentioned in the Upgrade and Downgrade Considerations before you upgrade or downgrade your current SD-WAN plugin. 5. Add one or more branch devices that you determined need to communicate with each other. • Select Group HA Peers to display the branch devices that are HA peers together. • Select the branch devices to add to the cluster. SD-WAN Administrator’s Guide 3.1 130 ©2023 Palo Alto Networks, Inc. Configure SD-WAN • Click OK. 6. Add one or more hub devices that you determined need to communicate with the branch devices. Up to four SD-WAN hub firewalls can be added to a VPN cluster. SD-WAN hubs in an HA configuration are considered as a single SD-WAN hub firewall. MPLS and satellite link types will form tunnels with only the same link type; for example, MPLS-to-MPLS and satellite-to-satellite. Tunnels will not be created between an MPLS link and an Ethernet link, for example. • Select Group HA Peers to display the hub devices that are HA peers together. • Select the hubs to add to the cluster and click OK. • For any new or previously existing VPN cluster that has more than one hub, you must prioritize the hubs to determine a) that traffic be sent to a particular hub, and b) the subsequent hub failover order. The hub failover priority range is 1 to 4. If you upgrade, the default priority is set to 4. The plugin internally translates the hub failover priority to a BGP local preference number as shown in the following table. The lower the priority value, the higher the priority and local preference. A cluster supports a maximum of four hubs. An active/passive HA pair counts as one hub. Multiple hubs can have the same priority; an HA pair must have the same priority. Panorama uses the branch’s BGP template to push the local preference of the hubs to the branches in the cluster. Hub Failover Priority Local Preference 1 250 2 200 3 150 SD-WAN Administrator’s Guide 3.1 131 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Hub Failover Priority Local Preference 4 100 If multiple hubs have the same priority, Panorama enables ECMP in two places on each branch firewall to determine how branches select the path. ECMP is enabled for the virtual router (Network > Virtual Routers > ECMP) and ECMP Multiple AS Support is enabled for BGP (Network > Virtual Routers > BGP > Advanced).If all hubs in the cluster have a unique priority, ECMP is disabled on the branches. If a hub priority configuration changes, Panorama reevaluates whether to enable or disable ECMP. • If you selected Group HA Peers, select the pair and click in the Hub Failover Priority field; enter a single Priority (range is 1 to 4), which applies to both hubs in the HA pair, and click OK. The Hub Failover Priority for HA Peers window appears only for configured HA pairs. If you add a new HA pair, you must configure the Hub Failover Priority for each of the two new peers independently. You will get an error message if you assign different priorities to hubs that are ungrouped HA peers and then you select Group HA Peers and Submit. • For hubs that are not HA pairs, select a hub and click in the Hub Failover Priority field; enter a priority (range is 1 to 4). SD-WAN Administrator’s Guide 3.1 132 ©2023 Palo Alto Networks, Inc. Configure SD-WAN 7. Click OK to save the VPN cluster. STEP 5 | Advertise additional prefixes at the branch to the hub. The firewall automatically redistributes (advertises) all non-public, connected routes from the branch to the hub. You can also redistribute any additional prefixes from the branch to the hub. The Prefix(es) to Redistribute field accepts a list of prefixes, rather than just a single prefix. 1. Select Panorama > SD-WAN > Devices and select a branch firewall. 2. Select BGP and Add one or more IP addresses with netmask to Prefix(es) to Redistribute. 3. Click OK. STEP 6 | Commit and Commit to Panorama. STEP 7 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) If your hub firewalls in a hub-spoke VPN cluster have DHCP or PPPoE interfaces, you must use DDNS. Select Network > Interfaces > Ethernet and in the Template field, select Template-stack for a hub. STEP 8 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Select the interfaces whose IP address indicates Dynamic-DHCP Client or PPPOE, click Override on the bottom of the screen, and click OK to close. STEP 9 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Verify on Panorama that the DDNS settings were configured. 1. Select Network > Interfaces > Ethernet and select the same interface again. 2. Select Advanced > DDNS. 3. See that the DDNS settings were automatically configured with a Hostname and the Vendor set to Palo Alto Networks DDNS. 4. Click OK. STEP 10 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Commit and Commit to Panorama. SD-WAN Administrator’s Guide 3.1 133 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 11 | Push the configuration to the hub(s). When Panorama creates virtual SD-WAN interfaces for hubs, Panorama doesn't necessarily create the interfaces using contiguous interface numbers. It might randomly skip an interface number, for example, sdwan.921, sdwan.922, sdwan.924, sdwan.925. Despite the discontiguous numbering, Panorama creates the correct number of SD-WAN interfaces. Use the operational CLI command show interface sdwan? to see the SD-WAN interfaces. 1. Select Commit and Push to Devices. 2. Edit Selections on the lower left side of the screen. 3. Deselect Filter Selected. 4. Click on Deselect All. 5. Select your hub Device Group. Select Include Device and Network Templates at the bottom of the screen. You must push to hubs before pushing to branches. Most branches have dynamic IP addresses through their service providers, so branches must initiate the IKE/IPSec connection because the hub doesn’t have the branches’ IP address.To ensure that the hub is ready to receive the IKE/IPSec connections, the hub’s configuration must be committed and pushed before the branch’s configuration. Thus, SD-WAN Administrator’s Guide 3.1 134 ©2023 Palo Alto Networks, Inc. Configure SD-WAN when the branch configurations are pushed and the branches initiate the connection to the hub, the hub is ready. 6. Select the Templates tab and Deselect All. 7. The Push Scope is the Device Group. Push the configuration to the hub(s). STEP 12 | Push the configuration to the branch(es) by repeating the prior step, but selecting your branch Device Group. SD-WAN Administrator’s Guide 3.1 135 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 13 | Refresh the IKE preshared key. If you need to change the current IKE key that is used to secure the IPSec connections between VPN cluster devices, perform this step to randomly generate a new key for the cluster. Perform this step when cluster members are not busy. 1. Select Panorama > SD-WAN > VPN Clusters and select a cluster. 2. At the bottom of the screen, select Refresh IKE Key. 3. A message appears notifying you that Refreshing the IKE key will generate a new security association (SA) for every SD-WAN firewall in the VPN cluster. This may cause a service disruption. Do you wish to continue? Yes | No Select Yes if you wish to continue. 4. Commit. After you Refresh IKE Key, you must commit to the entire cluster; a partial commit will bring down tunnels. 5. Push to Devices. SD-WAN Administrator’s Guide 3.1 136 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Create a Full Mesh VPN Cluster with DDNS Service Beginning with PAN-OS 10.0.3, SD-WAN supports a full mesh topology, in addition to the hubspoke topology. The mesh can consist of branches with or without hubs. Use full mesh when the branches need to communicate with each other directly. Examples of use cases for full mesh include retailers that have branches and hubs, and enterprises that operate with or without hubs. Some firewall interfaces use DHCP to get their IP address. Branch offices often use a consumergrade internet service and receive a dynamic IP address, which of course can change. For this reason, the firewalls require Dynamic DNS (DDNS) so that a DDNS service can detect the publicfacing IP address of the firewall interface that is running SD-WAN. When you push the DDNS setting to all firewalls, that notifies each firewall to register its external interface IP address with the Palo Alto Networks DDNS cloud service so that the IP address is converted to an FQDN. DDNS is also required because the CPE device from the ISP may be performing source NAT. (The dynamic IP address may or may not be source-NAT translated). The DDNS service allows the firewall to register the public-facing IP address with the DDNS server. When you have devices connect for branch-to-branch mesh, Auto VPN contacts the DDNS service for those firewalls to pull their public IP addresses that are registered in the DDNS cloud and uses those public IP addresses to create the IKE peering and the VPN tunnels. If the CPE device is performing source NAT, when you add an SD-WAN branch device to be managed by Panorama, you will enable Upstream NAT and the NAT IP Address Type will be DDNS. For the CPE device or upstream routing device using source NAT, you are responsible for creating the one-to-one destination NAT rule (with no port translation) on that device to translate the external IP address back to the private IP address assigned to the firewall’s interface. This translation allows the IKE and IPSec protocols to come back into the firewall. (Palo Alto Networks doesn’t have access rights to the upstream CPE or upstream router that is performing source NAT.) SD-WAN full mesh with DDNS service requires the following: • PAN-OS 10.0.3 or a later 11.1 release • SD-WAN Plugin 2.0.1 or a later 2.0 release • ZTP Plugin 1.0.1 or a later 1.0 release that is downloaded, installed, and configured in order to leverage the DDNS that is associated with ZTP. Panorama must be ZTP-registered and communicating with the ZTP Service. • Applications and Threats Content Release Version 8354 or a later version • All firewalls participating in full mesh DDNS must be registered under the same Customer Support Portal (CSP) account. • All firewalls participating in full mesh DDNS must have the latest device certificate installed. Properly authenticating the firewalls, Panorama, and the cloud services are important security procedures that require the device certificate, and the CSP and ZTP services. SD-WAN Administrator’s Guide 3.1 137 ©2023 Palo Alto Networks, Inc. Configure SD-WAN • If you have a firewall or other network device that controls outgoing traffic positioned in front of the Palo Alto Networks firewall, you must change the configuration on that device to allow traffic from the DDNS-enabled interfaces to the following FQDNs: • https://myip.ngfw-ztp.paloaltonetworks.com/ • https://ngfw-ztp.paloaltonetworks.com/ (to reach whatsmyIP service) (to reach DDNS registration service) STEP 1 | Install the latest device certificate for Panorama and for all managed firewalls that are hubs or branches. STEP 2 | Install ZTP Plugin 1.0.1 to set up Zero Touch Provisioning. 1. In the Panorama Administrator’s Guide, read the ZTP Overview. 2. Install the ZTP Plugin. 3. Configure the ZTP Installer Administrator Account. 4. Select Panorama > Zero Touch Provisioning > Setup and edit the General settings to enable Dynamic IP Registration. 5. Click OK. The General settings indicate On ZTP Service with a Tenant ID number. 6. Select ZTP Service Status and confirm that the firewall Serial Number is listed. STEP 3 | If you haven’t already done so, install the SD-WAN Plugin 2.0.1 or a later 2.0 release. STEP 4 | Commit on Panorama. STEP 5 | Log in to the Panorama Web Interface. STEP 6 | Create the VPN Address Pool as shown in Create a VPN Cluster. SD-WAN Administrator’s Guide 3.1 138 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 7 | Create the full mesh VPN cluster. 1. Select Panorama > SD-WAN > VPN Clusters. 2. Select the Type to be Mesh. 3. Add the Branches that need to communicate with each other. 4. (Optional) Add one or more Hubs if you also want a hub in the mesh. 5. Click OK. STEP 8 | Commit and Commit to Panorama. If your firewalls have static IP addresses, you are done. If your branch or hub firewalls in a VPN mesh have DHCP or PPPoE interfaces, you must use DDNS, so continue this procedure as follows. STEP 9 | Select Network > Interfaces > Ethernet and in the Template field, select Template-stack for a particular branch. STEP 10 | Select the interface whose IP address indicates Dynamic-DHCP Client or PPPOE, click Override on the bottom of the screen, and click OK to close. STEP 11 | Verify on Panorama that the DDNS settings were configured. 1. Select Network > Interfaces > Ethernet and select the same interface again. 2. Select Advanced > DDNS. 3. See that the DDNS settings were automatically configured with a Hostname based on the interface name, and the Vendor set to Palo Alto Networks DDNS. For example, on the Ethernet1/2 interface, the resulting Hostname is 0102. 4. Click OK. SD-WAN Administrator’s Guide 3.1 139 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 12 | If the VPN cluster includes any hubs that have a DHCP or PPPoE interface, repeat Steps 9 through 11, but in the Template field, select Template-stack for a particular hub. Even if your hub is not in a full mesh cluster, but is in a hub-spoke cluster, if the hub uses DHCP or PPPOE to get its IP address for an SD-WAN interface, you must perform the Override steps to enable DDNS. STEP 13 | Commit to Panorama and Push to Devices. STEP 14 | Verify on the branch firewall that the branch is configured with DDNS. 1. Log into the branch firewall. 2. Select Network > Interfaces > Ethernet and for the Ethernet interface that you configured, scroll over the DDNS Info icon in the Features column to see the Vendor, Hostname, IP address, and other DDNS information. STEP 15 | On another branch in the cluster, view that the Peer Address of the interface is a systemgenerated FQDN for the DDNS registration. 1. Log onto another branch and select Network > Network Profiles > IKE Gateways. 2. See that the Peer Address is a secure name, not easily referenced and showing no company information; for example 0101.8ced8460fcc5177cd3665ce41b6345323a15a612b8e52ec1d9ec057a582cb4.t13855f6c9a92d STEP 16 | View FQDNs of branches and hubs and update DDNS information. 1. Access the CLI. 2. View FQDNs (generated by DDNS) for other branches and hubs: show dns-proxy fqdn all 3. Update the DDNS addresses: request system fqdn refresh SD-WAN Administrator’s Guide 3.1 140 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Create a Static Route for SD-WAN In addition to (or as an alternative to) BGP routing, you can create static routes to route your SDWAN traffic. You can configure static routes either using Panorama™ or directly on the firewall hub or branch. If you are going to use Panorama, you should be familiar with the process to Configure a Template or Template Stack Variable. You will create a variable to use as the destination in your static route, as shown in the following procedure. You will push a static route (that goes to the hub) to the branch. You will push a static route (that goes to the branch) to the hub. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Configure a Template or Template Stack Variable and enter the variable Name in the following format: $peerhostname_clustername.customname. For example, $branchsanjose_clusterca.10 or $DIA_cluster2.location3. After the dollar sign ($), the elements in the variable are: • peerhostname—Hostname of the destination hub or branch to which the static route goes. For a static route to the internet, the peerhostname must be DIA. An alternative to the peer’s hostname is to use the peer’s serial number. If the peer is part of an HA pair, you can use the hostname or serial number of either one of the two HA firewalls. • clustername—Name of the VPN cluster to which the destination hub or branch belongs. • customname—Text string of your choice; you cannot use a period (.) in the customname. You can have more than one static route going to the same peer, which means the variables will have the same peerhostname and clustername; you differentiate the variables by using a different customname. STEP 3 | Select the variable Type to be IP Netmask and enter the destination IP address with a slash and netmask length, such as 192.168.2.1/24. STEP 4 | Click OK to save the variable. STEP 5 | Select Network > Virtual Routers and select a virtual router. STEP 6 | Select Static Routes > IPv4 andAdd a Name for the static route. STEP 7 | For Destination, select the variable you created. STEP 8 | For Interface, select from the dropdown list, which includes only interfaces from the template; for example, Ethernet1/1, Tunnel.x, or sdwan.xx. STEP 9 | For Next Hop, select IP Address and enter the IP address of the next hop for the static route (the hub or branch to which the static route goes). STEP 10 | Click OK. STEP 11 | Commit and Commit and Push your changes. Auto VPN configuration replaces the sdwan keyword in the Interface field of the static route with the egress virtual SD-WAN interface that it determines based on the Destination variable. SD-WAN Administrator’s Guide 3.1 141 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Thus, the static route in the routing table indicates that traffic going to the peer host in the identified VPN cluster will egress the virtual SD-WAN interface to reach the specified next hop. STEP 12 | Configure a static route for the return traffic. SD-WAN Administrator’s Guide 3.1 142 ©2023 Palo Alto Networks, Inc. Configure SD-WAN Configure Advanced Routing for SD-WAN Advanced Routing Engine allows the firewall to scale and provide stable, high-performing, and highly available routing functions to large data centers, ISPs, enterprises, and cloud users. The Advanced Routing Engine relies on industry-standard configuration methodology, which facilitates the administrator tasks. It allows the creation of profiles that are used for different functions (such as, filtering, redistribution, and metric changes), all of which can be used across logical routers. These profiles provide finer granularity to filter routes for each dynamic routing protocol and improve route redistribution across multiple protocols. Although conceptually equivalent, the advanced routing engine uses logical routers rather than virtual routers to instantiate routing domains. Unlike virtual routers, logical routers are not created by default; you must create one before configuring the routing functions. You can use an advanced routing engine or a legacy engine based on your network requirements: • When you enable Advanced Routing, logical routers are created and advanced routing engine is used for routing. • When you disable Advanced Routing, virtual routers are created and legacy engine is used for routing. The advanced route engine supports multiple logical routers (known as a virtual router on the legacy route engine). The advanced route engine has more convenient menu options and there are more BGP settings that you can easily configure in a profile (authentication, timers, address family, or redistribution profile) that applies to a BGP peer group or peer, for example. The Advanced Routing Engine supports static routes, MP-BGP, OSPFv2, OSPFv3, RIPv2, Protocol Independent Multicast Sparse Mode (PIM-SM), PIM Source-Specific Multicast (SSM), BFD, redistribution, route filtering into the RIB, access lists, prefix lists, and route maps. You’ll need the following to configure advanced routing engine on SD-WAN: Platform Panorama TM Firewalls running PAN-OS Release SD-WAN Plugin 11.1 and later 3.1.0 and later SD-WAN plugin creates logical router or virtual router based on the value of the advanced routing option. When Advanced Routing is enabled, a logical router is created; Otherwise, virtual router is created. When you enable advanced routing in template stack and perform a Panorama commit and push to the firewall, SD-WAN plugin runs the migration script to create the SD-WAN related objects (static, interfaces, redistribution profile, BGP) in Logical Router. The migration script creates the logical router name same as the virtual router name for the same template. Hence the hubs and branches have always the same router name. SD-WAN Administrator’s Guide 3.1 143 ©2023 Palo Alto Networks, Inc. Configure SD-WAN After migration, Panorama does not allow you to delete the migrated virtual routers. The Panorama SD-WAN plugin 3.1.0 can concurrently manage firewalls using the Advanced Routing Engine and firewalls using the legacy routing engine. The benefit is that you can migrate select managed firewalls to the new Advanced Routing Engine while still maintaining your current legacy routing engine configuration on others. While the SD-WAN plugin 3.1.0 manages a firewall regardless of the routing engine, only one routing engine configuration can be in effect at a time on a managed firewall. You can use the Advanced Routing option to enable or disable the advanced routing engine. Each time you change the engine that the firewall uses (you enable or disable Advanced Routing to access the advanced engine or legacy engine, respectively), you must commit the configuration and reboot the firewall for the changes to take effect. Before you switch to the advanced route engine, make a backup of your current configuration. Similarly, if you configure Panorama with a template stack that enables or disables Advanced Routing, after you commit and push the template stack to devices, you must reboot the devices in the template stack for the change to take effect. When configuring Panorama, create device groups and template stack for devices that all use the same Advanced Routing setting (all enabled or all disabled). Panorama won’t push configurations with Advanced Routing enabled to smaller firewalls that don’t support Advanced Routing. For those firewalls, Panorama will push a legacy configuration if one is present. Ensure that you downgrade to an appropriate SD-WAN plugin and PAN-OS version, and disable Advanced Routing if you plan to use a virtual router. Use a separate template where the Advanced Routing is disabled (in this case, virtual routers are created) when downgrading the SDWAN plugin. If you have configured Advanced Routing and want to switch to a virtual router, then disable Advanced Routing to return to the previously saved virtual router configuration. Commit and push any changes made to the firewall after disabling advanced routing before attempting a downgrade procedure, such as downgrading PAN-OS and SD-WAN plugin versions. If you enable Advanced Routing, SD-WAN interfaces must be configured in the same logical router; they cannot be split among logical routers. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Upgrade Panorama to 11.1 and install the SD-WAN plugin 3.1.0. STEP 3 | Add your hub and branch firewalls as managed devices to the Panorama server. TM management STEP 4 | Make a backup of your current configuration before you enable Advanced Routing. STEP 5 | In the Device section, select appropriate template stack from the Template context dropdown. SD-WAN Administrator’s Guide 3.1 144 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 6 | Enable advanced routing engine. 1. Select Device > Setup > Management and edit the General Settings. 2. Enable Advanced Routing. SD-WAN plugin will create logical router or virtual router based on the value of the advanced routing option. When Advanced Routing is enabled, a logical router is created. Otherwise, virtual router is created. 3. Click OK. 4. A warning message about the migration appears; click Yes to proceed. On clicking Yes, a built-in migration script will migrate your existing configuration to the advanced routing engine. If you select Skip, an empty configuration is created for the advanced routing engine. The Migration Configuration displays the color codes that indicate the migration status. SD-WAN Administrator’s Guide 3.1 145 ©2023 Palo Alto Networks, Inc. Configure SD-WAN In the Virtual Router, review the STATUS of templates in template stacks. The STATUS should be green for successful migration. Otherwise, take necessary action for any templates that did not pass the migration. The successful migration automatically converts each virtual router to a corresponding logical router. It is mandatory to commit the configuration and restart the firewall for the changes to take effect. 5. Commit. 6. Select Device > Setup > Operations and Reboot Device. STEP 7 | Select Commit > Commit to Panorama and commit your changes. SD-WAN Administrator’s Guide 3.1 146 ©2023 Palo Alto Networks, Inc. Configure SD-WAN STEP 8 | Commit and push your configuration changes to your managed firewalls. Push to Devices to view the logical routers added in the selected SD-WAN firewalls. 1. Select Commit > Push to Devices and Edit Selections. 2. Select Templates and choose the templates stack and template from the list. 3. Enable Force Template Values to overwrite local configuration with the updated template values. Before you use this option, check for overridden values on the firewalls to ensure your commit does not result in any unexpected network outages or issues caused by replacing those overridden values. 4. Click OK and Push to devices. STEP 9 | Log back into the firewall. STEP 10 | Select Network. Notice the menu items, which are more industry-standard and more detailed than the single item (Virtual Routers) on the legacy menu. Routing includes Logical Routers and Routing Profiles, which include BGP, BFD, OSPF, OSPFv3, RIPv2, Filters, and Multicast. STEP 11 | You must enable Advanced Routing for each template stack individually when you have more than one template stack in your configuration. Repeat Steps 5 through 10 for other template stacks on firewalls that you intend to update for advanced routing. According to our design requirement, the logical router name must be the same as the virtual router name for the same template when using the advanced routing engine. This means that hubs and branches have always the same router name. When manually creating logical routers rather than using a migration script, you must make sure the logical router name and virtual router name are the same. STEP 12 | Select virtual or logical router in your SD-WAN deployment. Select Panorama > SD-WAN > Devices, to add an SD-WAN device (SD-WAN hub or branch firewall) to be managed by the Panorama management server. In addition to existing configuration options for adding an SD-WAN device, you can now select a logical router (for advanced routing engine) or virtual router (for legacy engine) for a Router Name. It is important that the logical router name and the virtual router name are same for the same template when using the advanced routing engine. Select the Router Name (logical or virtual router) to use for routing between the SD-WAN hub and branches: • If the virtual router and logical router names are the same, then the Router Name displays one name. • If virtual router and logical router names are different, then the Router Name displays both virtual and logical router name. You can select either virtual router (for legacy engine) or logical router (for advanced routing engine) based on your requirement. SD-WAN Administrator’s Guide 3.1 147 ©2023 Palo Alto Networks, Inc. Configure SD-WAN SD-WAN Administrator’s Guide 3.1 148 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting Monitor and generate reports of the application and link health status in your VPN clusters to identify and resolve issues. In order for the Panorama™ management server to display SD-WAN application and link health information, you must enable the SD-WAN firewalls to push device monitoring data to Panorama and configure log forwarding to Panorama when you Add Your SD-WAN Firewalls as Managed Devices. If you have not configured your SD-WAN firewalls to forward logs to Panorama, the SD-WAN Monitoring displays no application or link health information. In order for Panorama to gather SD-WAN monitoring data, you must push the SD-WAN configuration from Panorama to your SD-WAN firewalls. If no SD-WAN monitoring data is displayed, verify that you successfully pushed the SD-WAN configuration. • Monitor SD-WAN Tasks • Monitor SD-WAN Application and Link Performance • Monitor Prisma Access Hubs • Generate an SD-WAN Report 149 Monitoring and Reporting Monitor SD-WAN Tasks Monitor commits, pushes, and other SD-WAN tasks executed from the Panorama™ management server to gain insight and detailed information regarding the specific task. If a task succeeds with warnings or fails, you can view detailed warnings and description to better understand how to resolve the misconfiguration. Additionally, you can view the last push state details to review detailed information as to what caused the task warnings or errors. STEP 1 | Log in to the Panorama web interface. STEP 2 | After editing the SD-WAN configuration, Commit your changes to view the job status. The job status window displays the operation performed, the result, and any details and warnings related to the job status. SD-WAN Administrator’s Guide 3.1 150 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 3 | View the last push details for jobs that succeed with warnings or failed. 1. Click Tasks ( ) at the bottom of the web interface to open the Task Manager. 2. Click the job Type for the SD-WAN task. 3. Click the job Status to view the last push state details for the task. 4. Review the last push state details to identify and resolve the configuration issues. SD-WAN Administrator’s Guide 3.1 151 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting Monitor SD-WAN Application and Link Performance Monitor the application and link performance in your VPN clusters to troubleshoot issues by viewing summary information across all VPN clusters and then successively drilling down to isolate the issues to affected sites, applications, and links. Visibility on SD-WAN traffic is shown on the SD-WAN firewall receiving the traffic. For example, for traffic from the hub firewall to the branch firewall, the SD-WAN monitoring data is reflected on the branch firewall. The landing dashboard displays: • App Performance • Impacted—One or more applications in the VPN cluster for which none of the paths have jitter, latency, or packet loss performance that meet the specified thresholds in the Path Quality Profile in the list of paths from which the firewall can choose. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. • Link Performance • Error—One or more sites in the VPN cluster have connectivity issues such as when a tunnel or a virtual interface (VIF) is down. • Warning—Number of VPN clusters, hubs, and branches that have links with jitter, latency, or packet loss performance measurements that exceed the moving seven-day average value of the metric. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. If a hub or branch firewall have an SD-WAN policy rule configured with Forward Error Correction, an Error Correction Initiated message is displayed to notify you that the hub or branch firewall detected and corrected errors in transmitted data for an application. SD-WAN hubs display Error Correction Initiated only if traffic originated from the SD-WAN hub to the SD-WAN branch and matched an SD-WAN policy rule with an error correction profile attached. From the landing dashboard, narrow the view to impacted applications or links that have the Error or Warning status. Then select an affected site to view site-level details. From the site, view application-level or link-level details. See Monitor Prisma Access Hub Application and Link Performanceto monitor application and link performance for Prisma Access hubs. If no data is present or the screen indicates that SD-WAN is undefined, check in the Compatibility Matrix that the Panorama release you are using supports the SD-WAN plugin release you are trying to use. STEP 1 | Log in to the Panorama Web Interface. SD-WAN Administrator’s Guide 3.1 152 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 2 | Select Panorama > SD-WAN > Monitoring to view at-a-glance health status summaries of your VPN clusters, hubs, and branches. STEP 3 | Click an App Performance or Link Performance summary that indicates Impacted, Error, or Warning counts to view a detailed list of sites and their status based on latency, jitter, and packet loss. STEP 4 | Click a site that displays Warning or Error to see one VPN cluster. The site data display App Performance and Link Performance, including the impacted applications. Additionally, use SD-WAN Administrator’s Guide 3.1 153 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting the Sites filter to view VPN clusters based on link notifications, latency deviations, jitter deviations, packet loss deviations, or impacted applications. For SaaS applications over a Direct Internet Access (DIA) link, the SaaS Monitoring column indicates whether the app is created in a SaaS Quality profile and associated with one or more SD-WAN policy rules. • Disabled—The app is not a SaaS application configured in a SaaS Quality profile. • Enabled—The app is a SaaS application configured in a SaaS Quality profile and is associated with one or more SD-WAN policies. If you associated an error correction profile with an SD-WAN policy rule for an application, the Error Correction Applied columns displays if and what type of error correction was applied. Additionally, you can view the Error Corrected Sessions/Impacted Sessions/Total Sessions to understand how many sessions were error corrected by the branch or hub firewall out of the total number of sessions for the specified timeframe. Click PDF/CSV to export the detailed health information for the applications and links in the Site in PDF or CSV format STEP 5 | Click the branch or hub that has an application that needs attention. SD-WAN Administrator’s Guide 3.1 154 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 6 | Click an impacted application to view application-level or link-level details. For example, view the link characteristics for an application to understand the latency, jitter and packet loss for the application over the specified link. Additionally, you can view when error correction was applied for the link. SD-WAN Administrator’s Guide 3.1 155 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting Monitor Prisma Access Hubs Baseline and monitor your Prisma Access Hub application and link performance to understand how to configure and modify your SD-WAN link management profiles. • Baseline Your Prisma Access Hub Application and Link Performance • Monitor Prisma Access Hub Application and Link Performance Baseline Your Prisma Access Hub Application and Link Performance Before you Configure SD-WAN Link Management Profiles, Palo Alto Networks recommends you baseline your Prisma Access Hub application and link performance to better understand the normal payload activity for your Prisma Access Hub to avoid unnecessary link swapping for applications and traffic that do not require it. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Onboard PAN-OS Firewalls to Prisma Access. STEP 3 | Select Panorama > SD-WAN > Monitoring and modify the SD-WAN monitoring time range. The longer the time frame you use to baseline your Prisma Access Hub app and link performance, the more accurate the baseline will be. At a minimum, use three days worth of app and link performance to baseline the latency, jitter, and packet loss data you use to create your SD-WAN link management profiles. Palo Alto Networks recommends assessing seven days of app and link performance data to baseline the latency, jitter, and packet loss for the Prisma Access Hub. STEP 4 | Filter the SD-WAN monitoring to display only your Prisma Access Hub-Spoke VPN clusters. 1. Click an App Performance or Link Performance summary that indicates Impacted, Error, or Warning counts to view a detailed list of sites and their status based on latency, jitter, and packet loss. 2. In the VPN Cluster filter, select Prisma Access Hub-Spoke. 3. Click a Site to view in-depth health details for the Prisma Access Hub. SD-WAN Administrator’s Guide 3.1 156 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 5 | Review the link characteristics for a Prisma Access Hub application. 1. Click an application in the App Performance section to view the Traffic Characteristics and the links used for the applications traffic. 2. Click each Link to view the baseline latency, jitter, and packet loss measured for the application across the link. Repeat this for all links until you gather sufficient baseline data to modify your Prisma Access Hub path quality profile. STEP 6 | Modify your Prisma Access Hub Path Quality profile based on the latency, jitter, and packet loss baselines you gathered. STEP 7 | Continue configuring SD-WAN as needed. STEP 8 | Monitor Prisma Access Hub Application and Link Performance to further fine-tune your SDWAN link management profiles. Monitor Prisma Access Hub Application and Link Performance Monitor the application and link performance for your Prisma Access Hub to troubleshoot issues by viewing summary information across all VPN clusters and then successively drilling down to isolate the issues to affected sites, applications, and links. Visibility on SD-WAN traffic is shown on the Prisma Access deployment or SD-WAN firewall receiving the traffic. For example, for traffic from the hub firewall to the branch firewall, the SD-WAN monitoring data is reflected on the branch firewall. The landing dashboard displays: SD-WAN Administrator’s Guide 3.1 157 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting • App Performance • Impacted—One or more applications in the VPN cluster for which none of the paths have jitter, latency, or packet loss performance that meet the specified thresholds in the Path Quality Profile in the list of paths from which the firewall can choose. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. • Link Performance • Error—One or more sites in the VPN cluster have connectivity issues such as when a tunnel or a virtual interface (VIF) is down. • Warning—Number of VPN clusters, hubs, and branches that have links with jitter, latency, or packet loss performance measurements that exceed the moving seven-day average value of the metric. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. From the landing dashboard, narrow the view to impacted applications or links that have the Error or Warning status. Then select an affected site to view site-level details. From the site, view application-level or link-level details. See Monitor SD-WAN Application and Link Performance to monitor application and link performance across all your SD-WAN sites. If no data is present or the screen indicates that SD-WAN is undefined, check in the Compatibility Matrix that the Panorama release you are using supports the SD-WAN plugin release you are trying to use. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring to view at-a-glance health status summaries of your VPN clusters, hubs, and branches. SD-WAN Administrator’s Guide 3.1 158 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 3 | Filter the SD-WAN monitoring to display only your Prisma Access Hub-Spoke VPN clusters. 1. Click an App Performance or Link Performance summary that indicates Impacted, Error, or Warning counts to view a detailed list of sites and their status based on latency, jitter, and packet loss. 2. In the VPN Cluster filter, select Prisma Access Hub-Spoke. 3. Click a Site to view in-depth health details for the Prisma Access Hub. STEP 4 | Review the Prisma Access Hub in-depth health details. The site data displays the Prisma Access Onboarding details, as well App Performance and Link Performance including the impacted applications. For SaaS applications over a Direct Internet Access (DIA) link, the SaaS Monitoring column indicates whether the app is created in a SaaS Quality profile and associated with one or more SD-WAN policy rules. • Disabled—The app is not a SaaS application configured in a SaaS Quality profile. • Enabled—The app is a SaaS application configured in a SaaS Quality profile and is associated with one or more SD-WAN policies. If you associated an error correction profile with an SD-WAN policy rule for an application, the Error Correction Applied columns displays if and what type of error correction was applied. Additionally, you can view the Error Corrected Sessions/Impacted Sessions/Total Sessions to SD-WAN Administrator’s Guide 3.1 159 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting understand how many sessions were error corrected by the branch or hub firewall out of the total number of sessions for the specified timeframe. Click PDF/CSV to export the detailed health information for the applications and links in the Site in PDF or CSV format SD-WAN Administrator’s Guide 3.1 160 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting STEP 5 | Click an impacted application to view application-level or link-level details. For example, view the link characteristics for an application to understand the latency, jitter and packet loss for the application over the specified link. Additionally, you can view when error correction was applied for the link. SD-WAN Administrator’s Guide 3.1 161 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting Generate an SD-WAN Report Configure and generate an SD-WAN report detailing the top applications or links with the highest frequency of path quality degradation. The order application or links appear in a report is based on the amount of data impacted; the more data is impacted, the higher the application or link appears in the report. SD-WAN reports are generated as needed and cannot be scheduled. Use the SD-WAN reports to verify the correct application or link throughput, or ensure that application or link impact is not noticed by users. For example, if your ISP guaranteed a certain amount of throughput on a link, generate a Link Performance report for that link to verify that the guaranteed bandwidth is honored. From the Panorama™ management server, you can only generate reports for applications or links across all SD-WAN enabled firewalls. To generate a report for applications or links processed by an individual firewall, you must create and generate the report locally on the firewall. If no data is present or the screen indicates that SD-WAN is undefined, check in the Compatibility Matrix that the Panorama release you are using supports the SD-WAN plugin release you are trying to use. STEP 1 | Log in to the Panorama web interface. STEP 2 | Select Panorama > SD-WAN > Reports and Add a new report. STEP 3 | Configure the SD-WAN report parameters. 1. Enter a descriptive Name for the report. 2. Choose the Report Type to generate: • Select App Performance to generate a report detailing only application health performance. • Select Link Performance to generate a report detailing only the link health performance. 3. Select the VPN Cluster for which to generate the report. By default, all is selected. 4. Select a Site within the selected VPN cluster for which to generate the report. By default, all is selected. If you selected all Clusters then this field is grayed out and a Site cannot be selected. 5. (App Performance only) Select the Application for which to generate the report. If you selected all Clusters and Sites then this field is grayed out and an individual application cannot be selected. 6. (Link Performance only) Select the Link Tag for which to generate the report. Selecting a link tag generates a report for all links in grouped using the tag in the cluster or site. By default, all is selected. 7. (Link Performance only) Select the Link Type for which to generate the report. Selecting a link type generates a report for all links of the specified type in the cluster or site. By default, all is selected. 8. Select the Top N applications or links to include in the report. This setting determines the number of applications or links experiencing health degradation to include in the SD-WAN Administrator’s Guide 3.1 162 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting report. By default, the report includes the top 5 applications or links experiencing health degradation. 9. Specify the Time Period within which to generate the report. By default, None is selected and queries the entire health status history of the applications or links. STEP 4 | Click Run Now to generate the report. STEP 5 | View the generated report and Export XML to export the report in XML format to your local device. When ready, click Close. STEP 6 | In the Reports pop-up, click OK to save your configured report. STEP 7 | Commit > Commit to Panorama and Commit your changes. SD-WAN Administrator’s Guide 3.1 163 ©2023 Palo Alto Networks, Inc. Monitoring and Reporting SD-WAN Administrator’s Guide 3.1 164 ©2023 Palo Alto Networks, Inc. Troubleshooting Use the Panorama™ management server Command Line Interface (CLI) to view SD-WAN information and perform operations. • Use CLI Commands for SD-WAN Tasks • Replace an SD-WAN Device • Troubleshoot App Performance • Troubleshoot Link Performance • Upgrade your SD-WAN Firewalls • Upgrade the SD-WAN Plugin • Uninstall the SD-WAN Plugin 165 Troubleshooting Use CLI Commands for SD-WAN Tasks Use the following CLI commands to view and clear SD-WAN information and view SD-WAN global counters. You can also view VPN tunnel information, BGP information, and SD-WAN interface information. If you want to ... Use ... View or Clear SD-WAN Information • View path names and IDs for an SD-WAN interface, their state, local and peer IP addresses, and tunnel interface number. • View the number and percentage of sessions distributed to each tunnel member of a virtual SD-WAN interface. • View the names of SD-WAN policy rules that send traffic to the specified virtual SD-WAN interface, along with the traffic distribution method, configured latency, jitter, and packet loss thresholds, link tags identified for the rule, and member tunnel interfaces. • View SD-WAN events such as path selection and path quality measurements. > show sdwan connection all | <s dwan-interface> > show sdwan session distributi on policy-name <sdwan-policy-na me> > show sdwan rule vif sdwan.x > show sdwan event For PAN-0S 10.0.0 and 10.0.1, when you make an SDWAN configuration change (such as a Path Quality profile change) that results in a different SD-WAN path being selected, the traffic log does not count or log the path change. • Clear SD-WAN events. SD-WAN Administrator’s Guide 3.1 > clear sdwan event 166 ©2023 Palo Alto Networks, Inc. Troubleshooting If you want to ... Use ... • View latency, jitter, and packet loss on a virtual SD-WAN interface (specify interface number or name). > show sdwan path-monitor stats vif <sdwan.x> Latency, jitter, and packet loss measurements are taken and averaged over three timeframes. Each timeframe has a health version, which increments when a health parameter value (that exceeds the threshold) changes. In addition to the real time measurement, there is a current use measurement, which displays the value of the parameter the last time the real-time value change exceeded the threshold. • View the name of the SD-WAN policy rule that the specified session matches, the source and destination tunnel interfaces, the configured latency, jitter, and packet loss percentage for the rule, and the traffic distribution method. > show sdwan path-monitor stats vif <sdwan-interface-name> > show sdwan session path-select session-id <session-id> For PAN-0S 10.0.0 and 10.0.1, when you make an SDWAN configuration change (such as a Path Quality profile change) that results in a different SD-WAN path being selected, the traffic log does not count or log the path change. • View monitoring mode for the virtual SDWAN link (Aggressive or Relaxed) and update intervals. > show sdwan path-monitor parame ter path-name <sdwan-path-name> • View monitoring mode for the virtual SDWAN interface (Aggressive or Relaxed), update intervals, and probe statistics. > show sdwan path-monitor parame ter vif <sdwan.x> View Global Counters to Troubleshoot SD-WAN • On a branch, verify that the number of SDWAN probe Request packets transmitted SD-WAN Administrator’s Guide 3.1 167 > show counter global filter del ta yes ©2023 Palo Alto Networks, Inc. Troubleshooting If you want to ... Use ... equals the number of probe Reply packets received. On a branch firewall, most SD-WAN tunnels are the initiator, which means the tunnel will have SD-WAN path-monitor probing enabled. • On a hub, verify that the number of SDWAN probe Request packets received equals the number of probe Reply packets transmitted. On a hub firewall, most SD-WAN tunnels are the responder, which means the tunnel will have SD-WAN path-monitor probing disabled. flow_sdwan_prob_req_tx flow_sdwan_prob_reply_rx > show counter global filter del ta yes flow_sdwan_prob_req_rx flow_sdwan_prob_reply_tx View VPN Tunnel Information • View all tunnels created on firewall. > show vpn flow • View details of individual tunnels identified by name. • View details of individual tunnels identified by ID. > show vpn flow name <name> > show vpn flow tunnel-id <tunne l-id> • View Internet Key Exchange (IKE) Phase 1 and Phase 2 details for all tunnels. > show vpn ike-sa • View IKEv2 security associations (SAs) and IKEv2 IPSec child SAs of a specific gateway. > show vpn ike-sa gateway <gate way> • View tunnel details. > show vpn tunnel View BGP Information SD-WAN Administrator’s Guide 3.1 168 ©2023 Palo Alto Networks, Inc. Troubleshooting If you want to ... Use ... • View BGP summary for a virtual router. > show routing protocol bgp sum mary virtual-router <virtual-rou ter> • View BGP peer summary. > show routing protocol bgp peer peer-name <peer-name> virtual-r outer <virtual-router> • View summary of local routing information base (RIB). > show routing protocol bgp loc -rib View SD-WAN Interface Information among RIB and FIB • View new SD-WAN egress interface. > show routing route • View SD-WAN interfaces in forwarding information base (FIB). SD-WAN Administrator’s Guide 3.1 > show routing fib 169 ©2023 Palo Alto Networks, Inc. Troubleshooting Replace an SD-WAN Device The return merchandise authentication (RMA) process enables you to replace either failed or malfunctioning SD-WAN devices with new or reused functional SD-WAN devices at a branch or a data center site. An SD-WAN device can fail or malfunction for a number of reasons, such as a device chip failure, device misconfiguration, or from daily wear and tear. If the SD-WAN device is unusable due to a malfunction or overall failure, use the RMA process to replace the failed or malfunctioning device. A commit failure occurs on Panorama™ and managed devices if you try to replace an SD-WAN firewall from an existing deployment without following a proper RMA process. Before you begin the RMA process: • Review Before Starting RMA Firewall Replacement. • The SD-WAN generates configurations, such as IPSec gateways and keyIDs, based on the device serial number. Hence, you must update the serial number of the replacement firewall for SD-WAN to recognize the new firewall and to avoid commit failures. Find whether your SD-WAN configuration has IPSec or VPN object references to the old firewall: • To replace a branch firewall in a high availability (HA) deployment, login to the hub firewall and select Network > Network Profiles > IKE Gateways. Search for the serial number (without white spaces) of the old firewall. If you get one or more search results, it indicates that the SD-WAN is referencing the old firewall serial number in the gateway configuration. In this case, we recommend you to disconnect the old branch firewall from Panorama and HA deployment. • To replace a firewall in a full mesh deployment without hubs, search for the old firewall serial numbers on any of the branch firewalls. If you get one or more search results, it indicates that the SD-WAN is referencing the old firewall serial number in the gateway configuration. In this case, we recommend you to disconnect the old branch firewall from Panorama and mesh deployment. • To replace a standalone firewall, it is not necessary to search for the serial number. Use the following workflow to restore the configuration on a managed firewall when there is an RMA. STEP 1 | Select Panorama > SD-WAN > Devices and delete the old firewall. STEP 2 | Select Panorama > SD-WAN > VPN Clusters and delete the old firewall. STEP 3 | Commit the changes to Panorama. STEP 4 | (HA deployments only) Push the changes to all hubs and the other HA peers (except the old firewall that needs to be replaced). Before proceeding, ensure that the commit succeeds on both hubs and standalone firewalls. If the search for the old firewall serial number does not return any gateway configurations, you can skip this step. STEP 5 | Configure an RMA replacement firewall. STEP 6 | (HA deployments only) Establish a HA connection between the replacement firewall and the standalone firewall. A firewall with a lower numerical value, and therefore a higher priority, SD-WAN Administrator’s Guide 3.1 170 ©2023 Palo Alto Networks, Inc. Troubleshooting is designated as active. To avoid your replacement firewall taking over as an active HA peer, ensure that it isn’t assigned with a higher device priority. STEP 7 | Select Panorama > SD-WAN > Devices and add the new branch firewall. STEP 8 | Select Panorama > SD-WAN > VPN Clusters and add the new branch firewall. STEP 9 | Commit the changes to Panorama. STEP 10 | Select Commit > Push to Devices and push the entire Panorama managed configuration to the hubs and both HA peers at the branch. When you Push to Devices, Panorama attempts to push the changes to all the devices in the cluster for both HA and hub-and-spoke deployments. To avoid pushing the changes to all devices, select Edit Selections in the Push Scope and disable all other devices in Device Groups devices and Templates. • In hub-and-spoke deployments, select the hub firewalls and HA template stack of the branch system to which you intend to push the configuration. As a result, sites that aren’t selected could become out of sync. • In full mesh deployments, it’s mandatory to push the changes to all devices in the cluster. SD-WAN Administrator’s Guide 3.1 171 ©2023 Palo Alto Networks, Inc. Troubleshooting Troubleshoot App Performance Understanding what is causing degraded performance for your apps and services is integral to ensuring the user experience is not impacted. Understanding why your VPN clusters are impacted and app traffic failed over to different links helps in fine tuning your SD-WAN configuration. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring and view the Impacted VPN clusters. STEP 3 | Filter the VPN clusters based on your preferred metric from the Site drop-down and select time frame. In this example, we are viewing All Sites containing impacted VPN clusters in the last 12 hours. SD-WAN Administrator’s Guide 3.1 172 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 4 | In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. SD-WAN Administrator’s Guide 3.1 173 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 5 | In the App Performance section, click an app to view detailed Traffic Characteristic information about the app traffic such as the internet service(s) and links used: • Review the pie chart to understand the breakdown of app traffic across the your internet services. • Review the linegraph to understand how many bytes of data were transferred over each internet service over time. • Review the Links Used section to understand which links the app traffic used and to understand how many of the bytes were impacted out of the total bytes in the selected time frame. SD-WAN Administrator’s Guide 3.1 174 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 6 | Investigate which health metric caused the app to swap links. The dotted line indicates the seven day average for the health metric. 1. In the Links Used section of the Traffic Characteristics tab, click an ethernet Link to view detailed Link Characteristics (latency, jitter, and packet loss) over the time frame specified in Step 2 to investigate what health metric caused the app to swap links. 2. In the Traffic Characteristics tab, select another link to view the Link Characteristics for secondary app link to better understand what caused the VPN cluster to become impacted. SD-WAN Administrator’s Guide 3.1 175 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 7 | After you have identified why the app traffic is impacted, consider the following to resolve the issue: • Consider adding additional links to the Traffic Distribution Profile. By adding additional links for app traffic to fail over to, you help ensure that the app traffic and user experience are not impacted by links with degraded health. • Reconfigure the health thresholds in your Path Quality Profile. It may be that the health thresholds are too strict, resulting in unnecessary link failover. For example, if you have an app that can experience up to 18% packet loss before user experience is impacted, having a 10% packet loss threshold would result in the app failing over to a different link without a need to. • Consult your internet service provider (ISP) to determine if there are impacts to your network outside of your control that they can resolve. SD-WAN Administrator’s Guide 3.1 176 ©2023 Palo Alto Networks, Inc. Troubleshooting Troubleshoot Link Performance Understanding what is causing degraded link performance is integral for ensuring the user experience when using apps and services is not impacted. Understanding why your VPN clusters have links that are impacted helps in fine tuning your SD-WAN configuration to ensure that the user experiences when using apps and services are not affected by links with degraded health. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring and view the Impacted VPN clusters. STEP 3 | Filter the VPN clusters based on your preferred metric from the Site drop-down and select time frame. In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. In this example, we are viewing All Sites containing impacted VPN clusters in the last 24 hours. SD-WAN Administrator’s Guide 3.1 177 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 4 | In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. SD-WAN Administrator’s Guide 3.1 178 ©2023 Palo Alto Networks, Inc. Troubleshooting STEP 5 | In the App Performance section, click an app to view detailed Traffic Characteristic information about the app traffic such as the internet service(s) and links used: • Review the pie chart to understand the breakdown of app traffic across the your internet services. • Review the linegraph to understand how many bytes of data were transferred over each internet service over time. • Review the Links Used section to understand which links the app traffic used and to understand how many of the bytes were impacted out of the total bytes in the selected time frame. STEP 6 | Investigate which health metric caused the app to swap links. The dotted line indicates the threshold you configure when you Create a Path Quality Profile. 1. In the Links Used section of the Traffic Characteristics tab, click an ethernet Link to view detailed Link Characteristics (latency, jitter, and packet loss) over the time frame specified in Step 2 to investigate what health metric caused the app to swap links. In this example, we are viewing ethernet 1/1 and can see that the percentage of packets lost SD-WAN Administrator’s Guide 3.1 179 ©2023 Palo Alto Networks, Inc. Troubleshooting regularly exceeded the configured threshold in the Path Quality Profile for the app and can conclude that this is the reason the app traffic failed over to the next best link. 2. In the Traffic Characteristics tab, select another link to view the Link Characteristics. In this example, we are viewing ethernet 1/4 and can see that after the app traffic failed SD-WAN Administrator’s Guide 3.1 180 ©2023 Palo Alto Networks, Inc. Troubleshooting over, ethernet 1/4 experienced jitter for the app that exceeded the configured threshold. This forced the app traffic to fail over back to ethernet 1/1. Since both links had health metrics that were exceeded, the app traffic had no healthy link to fail over to resulting in the VPN cluster becoming impacted. STEP 7 | After you have identified why the app traffic is impacted, consider the following to resolve the issue: • Consider adding additional links to the Traffic Distribution Profile. By adding additional links for app traffic to fail over to, you help ensure that the app traffic and user experience are not impacted by links with degraded health. • Reconfigure the health thresholds in your Path Quality Profile. It may be that the health thresholds are too strict, resulting in unnecessary link fail over. For example, if you have an app that can experience up to 18% packet loss before user experience is impacted, having a 10% packet loss threshold would result in the app failing over to a different link without a need to. • Consult your internet service providor (ISP) to determine if there are impacts to your network outside of your control that they can resolve. SD-WAN Administrator’s Guide 3.1 181 ©2023 Palo Alto Networks, Inc. Troubleshooting Upgrade your SD-WAN Firewalls Review the Panorama Plugin for SD-WAN 2.1 Release Notes and then use the following procedure to upgrade your Panorama and managed SD-WAN firewalls. STEP 1 | Install Content and Software Updates for Panorama. STEP 2 | Upgrade your managed Log Collectors. • Upgrade Log Collectors when Panorama is Internet-Connected. • Upgrade Log Collectors when Panorama is not Internet-Connected. STEP 3 | Upgrade your SD-WAN hub firewalls. You must upgrade your hub firewalls from PAN-OS 10.0.0 to PAN-OS 10.0.1 or later release before you upgrade your branch firewalls. Upgrading branch firewalls before hub firewalls may result in incorrect monitoring data (Panorama > SD-WAN > Monitoring) and for SD-WAN links to erroneously display as down. • Upgrade Firewalls when Panorama is Internet-Connected. • Upgrade Firewalls when Panorama is not Internet-Connected. STEP 4 | Upgrade your SD-WAN branch firewalls. • Upgrade Firewalls when Panorama is Internet-Connected. • Upgrade Firewalls when Panorama is not Internet-Connected. SD-WAN Administrator’s Guide 3.1 182 ©2023 Palo Alto Networks, Inc. Troubleshooting Upgrade the SD-WAN Plugin Upgrade the SD-WAN plugin version installed on your Panorama™ management server and firewalls leveraging SD-WAN. See the Palo Alto Networks Panorama Plugin Compatibility Matrix and review the minimum PANOS version required for your target SD-WAN plugin version. STEP 1 | Log in to the Panorama web interface. STEP 2 | Upgrade the SD-WAN plugin version on Panorama. For Panorama in a high availability (HA) configuration, repeat this step on the Panorama HA peer. 1. Select Panorama > Plugins and Check Now for the latest sd_wan plugin version. 2. Download and Install the latest version of the SD-WAN plugin. 3. After you successfully install the SD-WAN plugin, select Commit and Commit to Panorama. This step is required before you can commit any configuration changes to Panorama. STEP 3 | After the new plugin version successfully installs, view the Panorama Dashboard and in the General Information widget verify that the SD-WAN plugin displays the SD-WAN plugin version you upgraded to. SD-WAN Administrator’s Guide 3.1 183 ©2023 Palo Alto Networks, Inc. Troubleshooting Uninstall the SD-WAN Plugin To uninstall the SD-WAN plugin from the Panorama management server, you must remove your SD-WAN plugin configuration from Panorama before you can successfully uninstall the SD-WAN plugin. STEP 1 | Log in to the Panorama web interface. STEP 2 | Remove any security policy rules that allow BGP to run between your SD-WAN hubs and branches. 1. Select Panorama > SD-WAN > Devices > BGP Policy and Remove the security policy rules. 2. Click OK to save your configuration changes. STEP 3 | Select Panorama > Plugins and select Remove Config for the SD-WAN plugin. STEP 4 | Select Commit and Commit and Push your configuration changes to your managed firewalls. STEP 5 | Uninstall the SD-WAN plugin. Click OK when prompted to continue uninstalling the SD-WAN plugin. SD-WAN Administrator’s Guide 3.1 184 ©2023 Palo Alto Networks, Inc.