SD-WAN Administrator’s Guide 2.1 paloaltonetworks.com/documentation Contact Information Corporate Headquarters: Palo Alto Networks 3000 Tannery Way Santa Clara, CA 95054 www.paloaltonetworks.com/company/contact-support About the Documentation • For the most recent version of this guide or for access to related documentation, visit the Technical Documentation portal www.paloaltonetworks.com/documentation. • To search for a specific topic, go to our search page www.paloaltonetworks.com/documentation/ document-search.html. • Have feedback or questions for us? Leave a comment on any page in the portal, or write to us at documentation@paloaltonetworks.com. Copyright Palo Alto Networks, Inc. www.paloaltonetworks.com © 2021-2021 Palo Alto Networks, Inc. Palo Alto Networks is a registered trademark of Palo Alto Networks. A list of our trademarks can be found at www.paloaltonetworks.com/company/ trademarks.html. All other marks mentioned herein may be trademarks of their respective companies. Last Revised June 3, 2021 2 SD-WAN ADMINISTRATOR’S GUIDE | Table of Contents SD-WAN Overview............................................................................................ 5 About SD-WAN........................................................................................................................................... 7 SD-WAN Configuration Elements........................................................................................................ 10 Plan Your SD-WAN Configuration....................................................................................................... 12 Configure SD-WAN..........................................................................................15 Install the SD-WAN Plugin.....................................................................................................................17 Install the SD-WAN Plugin When Panorama is Internet-Connected.............................. 17 Install the SD-WAN Plugin When Panorama is not Internet-Connected....................... 17 Set Up Panorama and Firewalls for SD-WAN...................................................................................19 Add Your SD-WAN Firewalls as Managed Devices............................................................ 19 Create an SD-WAN Network Template.................................................................................20 Create the Predefined Zones in Panorama............................................................................21 Create the SD-WAN Device Groups...................................................................................... 23 Create a Link Tag......................................................................................................................................26 Configure an SD-WAN Interface Profile............................................................................................ 27 Configure a Physical Ethernet Interface for SD-WAN.................................................................... 30 Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN............................32 Configure Layer 3 Subinterfaces for SD-WAN................................................................................. 36 Configure a Virtual SD-WAN Interface...............................................................................................38 Create a Default Route to the SD-WAN Interface.......................................................................... 41 Create a Path Quality Profile.................................................................................................................42 Configure SaaS Monitoring.................................................................................................................... 44 Create a SaaS Quality Profile....................................................................................................44 Use Case: Configure SaaS Monitoring for a Branch Firewall............................................ 46 Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to the Same SaaS Application Destination.............................................................47 Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to a Different SaaS Application Destination..........................................................50 SD-WAN Traffic Distribution Profiles................................................................................................. 53 Create a Traffic Distribution Profile.....................................................................................................58 Create an Error Correction Profile....................................................................................................... 60 Configure an SD-WAN Policy Rule......................................................................................................63 Allow Direct Internet Access Traffic Failover to MPLS Link..........................................................67 Configure DIA AnyPath...........................................................................................................................68 Distribute Unmatched Sessions............................................................................................................ 74 Add SD-WAN Devices to Panorama................................................................................................... 76 Add an SD-WAN Device............................................................................................................76 Bulk Import Multiple SD-WAN Devices.................................................................................80 Configure HA Devices for SD-WAN................................................................................................... 83 Create a VPN Cluster.............................................................................................................................. 84 Create a Full Mesh VPN Cluster with DDNS Service......................................................................92 Create a Static Route for SD-WAN..................................................................................................... 96 Monitoring and Reporting.............................................................................. 99 Monitor SD-WAN Tasks.......................................................................................................................101 Monitor SD-WAN Application and Link Performance.................................................................. 103 Generate an SD-WAN Report............................................................................................................ 107 TABLE OF CONTENTS iii Troubleshooting.............................................................................................. 109 Use CLI Commands for SD-WAN Tasks.......................................................................................... 111 Troubleshoot App Performance..........................................................................................................114 Troubleshoot Link Performance......................................................................................................... 118 Upgrade your SD-WAN Firewalls...................................................................................................... 122 Upgrade the SD-WAN Plugin............................................................................................................. 123 Uninstall the SD-WAN Plugin............................................................................................................. 124 iv TABLE OF CONTENTS SD-WAN Overview Learn about SD-WAN and plan your configuration to ensure a successful deployment. > About SD-WAN > SD-WAN Configuration Elements > Plan Your SD-WAN Configuration 5 6 SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 2021 Palo Alto Networks, Inc. About SD-WAN Software-Defined Wide Area Network (SD-WAN) is a technology that allows you to use multiple internet and private services to create an intelligent and dynamic WAN, which helps lower costs and maximize ® ® application quality and usability. Beginning with PAN-OS 9.1, Palo Alto Networks offers strong security with an SD-WAN overlay in a single management system. Instead of using costly and time-consuming MPLS with components such as routers, firewalls, WAN path controllers, and WAN optimizers to connect your WAN to the internet, SD-WAN on a Palo Alto Networks firewall allows you to use less expensive internet services and fewer pieces of equipment. You don’t need to purchase and maintain other WAN components. • PAN-OS Security with SD-WAN Functionality • SD-WAN Link and Firewall Support • Centralized Management PAN-OS Security with SD-WAN Functionality The SD-WAN plugin is integrated with PAN-OS, so that you get the security features of a PAN-OS firewall and SD-WAN functionality from a single vendor. The SD-WAN overlay supports dynamic, intelligent path selection based on applications and services and the conditions of links that each application or service is allowed to use. The path health monitoring for each link includes latency, jitter, and packet loss. Granular application and service controls allow you to prioritize applications based on whether the application is mission-critical, latency-sensitive, or meets certain health criteria, for example. Dynamic path selection avoids brownout and node failure problems because sessions fail over to a better performing path in less than one second. The SD-WAN overlay works with all PAN-OS security features, such as User-ID™ and App-ID™, to provide complete security control to branch offices. The full suite of App-ID capabilities (App-ID decoder, AppID cache, and source/destination external dynamic list [EDL] IP address lists) identifies applications for application-based control of SD-WAN traffic. You can deploy the firewall with Zero Trust segmentation of traffic. You can configure and manage SD-WAN centrally from the Panorama web interface or the Panorama REST API. You may have cloud-based services and instead of having your internet traffic flow from branches to the hub to the cloud, you want the internet traffic to flow directly from branches to the cloud using a directly connected ISP. Such access from a branch to the internet is Direct Internet Access (DIA). You don’t need to spend your hub bandwidth and money on internet traffic. The branch firewall is already doing security, so you don’t need the hub firewall to enforce security on internet traffic. Use DIA on branches for SaaS, web browsing, or heavy-bandwidth applications that shouldn’t be backhauled to a hub. The following figure illustrates a DIA virtual interface consisting of three links from the branch to the cloud. The figure also illustrates a VPN tunnel virtual interface consisting of four links that connect the branch to the hub at the headquarters. SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 7 2021 Palo Alto Networks, Inc. SD-WAN Link and Firewall Support Link bundling allows you to group multiple physical links (that different ISPs use to communicate with the same destination) into a virtual SD-WAN interface. On the basis of applications and services, the firewall chooses from the links (path selection) for session load sharing and to provide failover protection in the event of a brownout or blackout. Thus you are providing the application with the best quality performance. The firewall automatically performs session load sharing over the links in a virtual SD-WAN interface to use available bandwidth advantageously. An SD-WAN interface must have all of the same type of connection (either DIA or VPN). VPN links support the hub-and-spoke topology. SD-WAN supports the following types of WAN connections: ADSL/DSL, cable modem, Ethernet, fiber, LTE/3G/4G/5G, MPLS, microwave/radio, satellite, WiFi, and anything that terminates as Ethernet to the firewall’s interface. You decide the appropriate strategy for how to use the links. You could use inexpensive broadband connections before expensive MPLS or LTE connections. Alternatively, you could use specific VPN tunnels to reach specific hubs in a region. The following firewall models support SD-WAN software capabilities: • • • • • • • PA-220 PA-220R PA-400 Series PA-820 PA-850 PA-3200 Series PA-5200 Series 8 SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 2021 Palo Alto Networks, Inc. • • • • • • • PA-5400 Series PA-7000 Series VM-50 VM-100 VM-300 VM-500 VM-700 If you are a new customer purchasing a Palo Alto Networks next-generation firewall, you will use the default virtual router for SD-WAN. If you are an existing customer, you can choose to either let PAN-OS overwrite any existing virtual routers or use a new virtual router and new zones for SD-WAN to keep SDWAN content separate from your pre-existing configuration. Centralized Management Panorama™ provides the means to configure and manage SD-WAN, which makes configuring multiple options on many geographically-dispersed firewalls much faster and easier than configuring firewalls individually. You can change network configurations from a single location rather than configuring each firewall individually. Auto VPN configuration allows Panorama to configure branches and hubs with secure IKE/IPSec connections. A VPN cluster defines the hubs and branches that communicate with each other in a geographic region. The firewall uses VPN tunnels for path health monitoring between a branch and a hub to provide subsecond detection of brownout conditions. The Panorama dashboard provides visibility into your SD-WAN links and performance so that you can adjust path quality thresholds and other aspects of SD-WAN to improve its performance. Centralized statistics and reporting include application and link performance statistics, path health measurements and trend analysis, and focused views of application and link issues. Begin by understanding your SD-WAN use case, then review the SD-WAN configuration elements, traffic distribution methods, and plan your SD-WAN configuration. To greatly accelerate the configuration, the best practice is for you to export an empty SD-WAN device CSV and enter information such as branch office IP address, the virtual router to use, the firewall site name, zones to which the firewall belongs, and BGP route information. Panorama uses the CSV file to configure the SD-WAN hubs and branches and to automatically provision VPN tunnels between hubs and branches. SD-WAN supports dynamic routing through eBGP and is configured using Panorama’s SD-WAN plugin to allow all branches to communicate with the hub only or with the hub and other branches. SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 9 2021 Palo Alto Networks, Inc. SD-WAN Configuration Elements The elements of an SD-WAN configuration work together, allowing you to: • Group physical Ethernet interfaces that share a common destination into a logical SD-WAN interface. • Specify link speeds. • Specify the thresholds at which a deteriorating path (or brownout or blackout) to an SD-WAN warrants selecting a new best path. • Specify the method of selecting that new best path. This view indicates the relationships between elements at a glance. The goal of an SD-WAN configuration is to control which links your traffic takes by specifying the VPN tunnels or direct internet access (DIA) that certain applications or services take from a branch to a hub or from a branch to the internet. You group paths so that if one path deteriorates, the firewall selects a new best path. • A Tag name of your choice identifies a link; you apply the Tag to the link (interface) by applying an Interface Profile to the interface, as the red arrow indicates. A link can have only one Tag. The two yellow arrows indicate that a Tag is referenced in the Interface Profile and the Traffic Distribution profile. Tags allow you to control the order that interfaces are used for traffic distribution. Tags allow Panorama to systematically configure many firewall interfaces with SD-WAN functionality. • An SD-WAN Interface Profile specifies the Tag that you apply to the physical interface, and also specifies the type of Link that interface is (ADSL/DSL, cable modem, Ethernet, fiber, LTE/3G/4G/5G, MPLS, microwave/radio, satellite, WiFi, or other). The Interface Profile is also where you specify the maximum upload and download speeds (in Mbps) of the ISP’s connection. You can also change whether the firewall monitors the path frequently or not; the firewall monitors link types appropriately by default. • A Layer3 Ethernet Interface with an IPv4 address can support SD-WAN functionalities. You apply an SD-WAN Interface Profile to this interface (red arrow) to indicate the characteristics of the interface. 10 SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 2021 Palo Alto Networks, Inc. • • • • The blue arrow indicates that physical Interfaces are referenced and grouped in a virtual SD-WAN Interface. A virtual SD-WAN Interface is a VPN tunnel or DIA group of one or more interfaces that constitute a numbered, virtual SD-WAN Interface to which you can route traffic. The paths belonging to an SD-WAN Interface all go to the same destination WAN and are all the same type (either DIA or VPN tunnel). (Tag A and Tag B indicate that physical interfaces for the virtual interface can have different tags.) A Path Quality Profile specifies maximum latency, jitter, and packet loss thresholds. Exceeding a threshold indicates that the path has deteriorated and the firewall needs to select a new path to the target. A sensitivity setting of high, medium, or low lets you indicate to the firewall which path monitoring parameter is more important for the applications to which the profile applies. The green arrow indicates that you reference a Path Quality Profile in one or more SD-WAN Policy Rules; thus, you can specify different thresholds for rules applied to packets having different applications, services, sources, destinations, zones, and users. A Traffic Distribution Profile specifies how the firewall determines a new best path if the current preferred path exceeds a path quality threshold. You specify which Tags the distribution method uses to narrow its selection of a new path; hence, the yellow arrow points from Tags to the Traffic Distribution profile. A Traffic Distribution profile specifies the distribution method for the rule. The preceding elements come together in SD-WAN Policy Rules. The purple arrow indicates that you reference a Path Qualify Profile and a Traffic Distribution profile in a rule, along with packet applications/services, sources, destinations, and users to specifically indicate when and how the firewall performs application-based SD-WAN path selection for a packet not belonging to a session. (You can also reference a SaaS Quality Profile and an Error Correction Profile in an SD-WAN policy rule.) Now that you understand the relationship between the elements, review the traffic distribution methods and then Plan Your SD-WAN Configuration. SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 11 2021 Palo Alto Networks, Inc. Plan Your SD-WAN Configuration Plan the complete topology of your SD-WAN-enabled branch and hub firewall interfaces so that you can create Panorama™ templates with CSV files and then push the configurations to the firewalls. STEP 1 | Plan the branch and hub locations, link requirements, and IP addresses. From Panorama you will export an empty SD-WAN device CSV and populate it with branch and hub information. 1. Decide the role of each firewall (branch or hub). 2. Determine which branches will communicate with which hubs; each functional group of branch and hub firewalls that communicate with each other is a VPN cluster. For example, your VPN clusters might be organized geographically or by function. 3. Determine the ISP link types that each branch and hub support: ADSL/DSL, cable modem, Ethernet, fiber, LTE/3G/4G/5G, MPLS, microwave/radio, satellite, and WiFi. 4. Determine the maximum download and upload bandwidth (Mbps) that the link types support and how you want to apply these speed controls to links, as described in Step 2. Record the ISP link’s maximum download and upload bandwidth (Mbps). This information will serve as reference egress maximums if you need to configure QoS to control the application bandwidth. 5. Gather the public IP addresses of branch firewalls, whether they are static or dynamically assigned. The firewall must have an internet-routable, public IP address so it can initiate and terminate IPSec tunnels and route application traffic to and from the internet. The ISP’s customer premise equipment must be directly connected to the Ethernet interface on the firewall. If you have a device that performs NAT located between the branch firewall and the hub, the NAT device can prevent the firewall from bringing up IKE peering and IPSec tunnels. If the tunnel fails, work with the administrator of the remote NAT device to resolve the issue. 6. Gather the private network prefixes and serial numbers of branch and hub firewalls. 7. Decide the link type of each firewall interface. Allocate the same link types on the same Ethernet interfaces across the branch firewalls to make configuration easier. For example, Ethernet1/1 is always cable modem. 8. Decide on the naming conventions for your sites and SD-WAN devices. Do not use the simple hostnames “hub” or “branch” because Auto VPN configuration uses these keywords to generate various configuration elements. 9. If you already have zones in place before configuring SD-WAN, decide how to map those zones to the predefined zones that SD-WAN uses for path selection. You will map existing zones to the predefined zones named zone-internal, zone-to-hub, zone-to-branch, and zone-internet. Information you will enter into a CSV (so that you can add multiple SD-WAN devices at once) includes: serial number, type of device (branch or hub), names of zones to map to predefined zones (pre-existing customers), loopback address, prefixes to redistribute, AS number, router ID, and virtual router name. STEP 2 | Plan link bundles and VPN security for private links. A link bundle lets you combine multiple physical links into one virtual SD-WAN interface for purposes of path selection and failover protection. By having a bundle of more than one physical link, you maximize application quality in case a physical link deteriorates. You create a bundle by applying the same link tag 12 SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 2021 Palo Alto Networks, Inc. to multiple links (via an SD-WAN Interface Profile). The link tag identifies a bundle of links that have a similar type of access and similar type of SD-WAN policy handling. For example, you can create a link tag named low cost broadband and include the cable modem and fiber broadband services. STEP 3 | Identify the applications that will use SD-WAN and QoS optimization. 1. Identify the critical and the latency-sensitive business applications for which you will provide SDWAN control and policies. These are applications that require a good user experience, and are likely to fail under poor link conditions. Start with the most critical and latency-sensitive applications; you can add applications after SD-WAN is functioning smoothly. 2. Identify the applications that require QoS policies so you can prioritize bandwidth. These should be the same applications you identified as critical or latency-sensitive. Start with the most critical and latency-sensitive applications; you can add applications after SD-WAN is functioning smoothly. STEP 4 | Determine when and how you want links to fail over to a different link in the event the original link degrades or fails. 1. Decide on the path monitoring mode for a link, although the best practice is to retain the default setting for the link type: • Aggressive—The firewall sends probe packets to the opposite end of the SD-WAN link at a constant frequency (five probes per second by default). Aggressive mode is appropriate for links where monitoring path quality is critical; where you need fast detection and failover for brownout and blackout conditions. Aggressive mode provides subsecond detection and failover. • Relaxed—The firewall observes a configurable idle time between sending probe packets for seven seconds (at the probe frequency you configure), which makes path monitoring less frequent than aggressive mode. Relaxed mode is appropriate for links that have very low bandwidth, links that are expensive to operate, such as satellite or LTE, or when fast detection isn’t as important as preserving cost and bandwidth. 2. Prioritize the order in which the firewall selects the first link for a new session and the order in which links should be a candidate to replace a link that is failing over, if there is more than one candidate. For example, if you want an expensive backup LTE link to be the last link used (only when the inexpensive broadband links are oversubscribed or completely down), then use the Top Down Priority traffic distribution method and place the tag that is on the LTE link last in the list of tags for the Traffic Distribution profile. 3. For the applications and services, determine the path health thresholds at which you consider a path to have degraded enough in quality that you want the firewall to select a new path (fail over). The quality characteristics are latency (range is 10 to 2,000 ms), jitter (range is 10 to 1,000 ms), and packet loss percentage. These thresholds constitute a Path Quality profile, which you reference in an SD-WAN policy rule. When any single threshold (for packet loss, jitter, or latency) is exceeded (and the remaining rule criteria are met), the firewall chooses a new preferred path for the matching traffic. For example, you can create Path Quality profile AAA with latency/jitter/packet loss thresholds of 1000/800/10 to use in Rule 1 when FTP packets come from source zone XYZ, and create Path Quality profile BBB (with thresholds of 50/200/5) to use in Rule 2 when FTP packets come from source IP address 10.1.2.3. Best practice is to start with high thresholds and test how the application tolerates them. If you set the values too low, the application may switch paths too frequently. Consider whether the applications and services you are using are especially sensitive to latency, jitter, or packet loss. For example, a video application might have good buffering that mitigates latency and jitter, but would be sensitive to packet loss, which impacts the user experience. You can set SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview © 13 2021 Palo Alto Networks, Inc. the sensitivity of the path quality parameters in the profile to high, medium or low. If the sensitivity settings for latency, jitter, and packet loss are the same, the firewall examines the parameters in the order of packet loss, latency, jitter. 4. Decide if there are links among which to load share new sessions for an application or service. STEP 5 | Plan the BGP configurations that Panorama will push to branches and hubs to dynamically route traffic between them. 1. Plan BGP route information, including a four-byte autonomous system number (ASN). Each firewall site is in a separate AS and therefore must have a unique ASN. Each firewall must also have a unique Router ID. 2. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. 3. If you don’t want to use BGP dynamic routing, plan to use Panorama’s network configuration features to push out other routing configurations. You can do static routing between the branch and hubs. Simply omit all of the BGP information in the Panorama plugin and use normal virtual router static routes to perform static routing. STEP 6 | Consider the capacities of firewall models for virtual SD-WAN interfaces, SD-WAN policy rules, log size, IPSec tunnels (including proxy IDs), IKE peers, BGP and static route tables, BGP routing peers, and performance for your firewall mode (App-ID™, threat, IPSec, decryption). Ensure the branch and hub firewall models you intend to use support the capacities you require. 14 SD-WAN ADMINISTRATOR’S GUIDE | SD-WAN Overview Configure SD-WAN After you Plan Your SD-WAN Configuration, install the SD-WAN plugin and set up the Panorama™ management server to centrally manage the SD-WAN configuration of your hub and branch firewalls. By leveraging Panorama, you reduce the management requirements and operational overhead for administrating your SD-WAN deployment, and can more easily monitor your link health and troubleshoot issues should they arise. > > > > > > > > > > > > > > > > > > > > > > > Install the SD-WAN Plugin Set Up Panorama and Firewalls for SD-WAN Create a Link Tag Configure an SD-WAN Interface Profile Configure a Physical Ethernet Interface for SD-WAN (Optional) Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN (Optional) Configure Layer 3 Subinterfaces for SD-WAN Configure a Virtual SD-WAN Interface Create a Default Route to the SD-WAN Interface Create a Path Quality Profile Configure SaaS Monitoring SD-WAN Traffic Distribution Profiles Create a Traffic Distribution Profile Create an Error Correction Profile Configure an SD-WAN Policy Rule Allow Direct Internet Access Traffic Failover to MPLS Link Configure DIA AnyPath Distribute Unmatched Sessions Add SD-WAN Devices to Panorama (Optional) Configure HA Devices for SD-WAN Create a VPN Cluster Create a Full Mesh VPN Cluster with DDNS Service (Optional) Create a Static Route for SD-WAN 15 16 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Install the SD-WAN Plugin A Panorama™ management server with an SD-WAN plugin is required to configure and manage an SDWAN deployment. If your Panorama is internet-connected, you download the SD-WAN plugin directly from Panorama and install it on the Panorama management server. If your Panorama is not internet-connected, ® you download the SD-WAN plugin from the Palo Alto Networks Customer Support Portal and install it on the Panorama management server. • Install the SD-WAN Plugin When Panorama is Internet-Connected • Install the SD-WAN Plugin When Panorama is not Internet-Connected Install the SD-WAN Plugin When Panorama is Internet-Connected A Panorama™ management server with an SD-WAN plugin installed is required to configure and manage an SD-WAN deployment. When Panorama is connected to the internet, you download and install the SD-WAN plugin directly from the Panorama web interface. The plugin needs to be installed only on the Panorama managing your SD-WAN firewalls, and not on the individual hub and branch firewalls. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > Plugins, search for the sd_wan plugin and Check Now for the most recent version of the plugin. STEP 3 | Download and Install the SD-WAN plugin. STEP 4 | After you successfully install the SD-WAN plugin, select Commit and Commit to Panorama. This step is required before you can commit any configuration changes to Panorama. STEP 5 | Continue to Set Up Panorama and Firewalls for SD-WAN to begin configuring your SD-WAN deployment. Install the SD-WAN Plugin When Panorama is not InternetConnected A Panorama™ management server with an SD-WAN plugin is required to configure and manage an SDWAN deployment. When Panorama is not connected to the internet, you must download the SD-WAN plugin from the Palo Alto Networks Customer Support Portal and upload the plugin to Panorama. The plugin needs to be installed only on the Panorama managing your SD-WAN firewalls, and not on the individual hub and branch firewalls. STEP 1 | Log in to the Palo Alto Networks Customer Support Portal. STEP 2 | Select Updates > Software Updates, and in the Filter By drop-down select Panorama Integration Plug In. STEP 3 | Locate and download the SD-WAN Plug-in. STEP 4 | Log in to the Panorama Web Interface. STEP 5 | Select Panorama > Plugins and Upload the SD-WAN plugin. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 17 2021 Palo Alto Networks, Inc. STEP 6 | Browse and locate the SD-WAN plugin you downloaded from the Customer Support Portal and click OK. STEP 7 | Install the SD-WAN plugin. STEP 8 | After you successfully install the SD-WAN plugin, select Commit and Commit to Panorama. This step is required before you can commit any configuration changes to Panorama. STEP 9 | Continue to Set Up Panorama and Firewalls for SD-WAN to begin configuring your SD-WAN deployment. 18 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Set Up Panorama and Firewalls for SD-WAN Before you can begin configuring your SD-WAN deployment, you must add your hub and branch firewalls as managed devices, and create the necessary templates and device group configurations to successfully push your SD-WAN configuration to SD-WAN firewalls. • • • • Add Your SD-WAN Firewalls as Managed Devices Create an SD-WAN Network Template Create the Predefined Zones in Panorama Create the SD-WAN Device Groups Add Your SD-WAN Firewalls as Managed Devices Before you can begin configuring your SD-WAN deployment, you must first Install the SD-WAN Plugin and add your hub and branch firewalls as managed devices to the Panorama™ management server. As part of adding your SD-WAN firewall as a managed device on the Panorama™ management server, you must activate the SD-WAN license to enable SD-WAN functionality for the firewall. As part of adding your SD-WAN firewalls as managed devices, you must configure your managed firewalls to forward logs to Panorama. Panorama collects information from multiple sources, such as configuration logs, traffic logs, and link characteristic measurements, to generate the visibility for SD-WAN application and link health information. STEP 1 | Launch the Firewall Web Interface. STEP 2 | Activate your SD-WAN license to enable SD-WAN functionality on the firewall. Each firewall you intend to use in your SD-WAN deployment requires a unique auth code to activate the license. For example, if you have 100 firewalls, you must purchase 100 SD-WAN licenses and activate each SD-WAN license on each firewall using one of the 100 unique auth codes. For VM-Series firewalls, you apply the SD-WAN auth code against the specific VM-Series firewall. If you deactivate the VM-Series firewall, the SD-WAN auth code can be activated on a different VM-Series firewall of the same model. Ensure that your SD-WAN license remains valid to continue leveraging SD-WAN. If the SD-WAN license expires, the following occurs: • • • • • • • • A warning displays when you Commit any configuration changes but no commit failure occurs. Your SD-WAN configuration no longer functions but is not deleted. Firewalls no longer monitor and gather link health metrics and stop sending monitoring probes. Firewalls no longer send app and link health metrics to Panorama. SD-WAN path selection logic is disabled. New sessions round robin on the virtual SD-WAN interface. Existing sessions remain on the specific link they were on when the license expired. If an internet outage occurs, traffic follows using standard routing and ECMP if configured. STEP 3 | Add the Panorama IP address to the firewall. 1. Select Device > Setup > Management and edit the Panorama Settings. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 19 2021 Palo Alto Networks, Inc. 2. Enter the Panorama IP address in the first field. The Panorama FQDN is not supported for SD-WAN. 3. (Optional) If you have set up a high availability (HA) pair in Panorama, enter the IP address of the secondary Panorama in the second field. 4. Verify that you Enable pushing device monitoring data to Panorama. 5. Click OK. 6. Commit your changes. STEP 4 | Configure log forwarding to Panorama. Forwarding logs from your SD-WAN firewalls to Panorama is required to display Monitoring and Reporting data. By default, HTTP/2 inspection is automatically enabled if decryption is enabled for application traffic. The parent sessions using a HTTP/2 connection does not generate any traffic logs because they do not carry any application traffic. However, the child sessions generated by the streams within the HTTP/2 parent session still generate traffic logs. For more information on viewing logs for HTTP/2 connections, see the Palo Alto Networks Knowledgebase. STEP 5 | Add one or more firewalls to Panorama. For more details about adding firewalls to Panorama, see Add a Firewall as a Managed Device. 1. 2. 3. 4. Log in to the Panorama Web Interface. Select Panorama > Managed Devices > Summary and Add the firewall(s). Enter the serial number(s) of the firewalls. If you are adding firewalls when the required device groups and templates are already created, enable (check) Associate Devices to assign new firewalls to the appropriate device groups and template stack. 5. To add multiple firewalls using a CSV, click Import and Download Sample CSV to populate with the firewall information, and then Browse to import the firewalls. 6. Click OK. STEP 6 | Select Commit and Commit and Push your configuration. STEP 7 | Repeat Steps 2 through 5 on each firewall you intend to use in your SD-WAN deployment. Create an SD-WAN Network Template Create a template containing all the networking configuration objects for your SD-WAN hubs and branches. You must create a separate template and template stack for your hub firewalls and a separate template and template stack for your branch firewalls. It is a best practice to limit the number of templates and template stacks used to manage your SD-WAN device configuration. Limiting the number of templates and template stacks used across all hubs and branches greatly reduces the operational overhead of managing the configurations of multiple SD-WAN hubs and branches. Use template or template stack variables to help reduce the number of templates used. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create the SD-WAN hub network template. 1. Select Panorama > Templates and Add a new template. 20 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 2. Enter a descriptive Name for the template. 3. (Optional) Enter a Description for the template. 4. Click OK to save your configuration changes. STEP 3 | Create a hub template stack. 1. 2. 3. 4. 5. 6. Select Panorama > Templates and click Add Stack to add a new template stack. Enter a descriptive Name for the template stack. (Optional) Enter a Description for the template. Add the SD-WAN network template you created in the Step 2. In the Devices section, select the check boxes for all your SD-WAN hub firewalls. Click OK to save your configuration changes. STEP 4 | Create the SD-WAN branch network template. 1. 2. 3. 4. Add a new template. Enter a descriptive Name for the template. (Optional) Enter a Description for the template. Click OK to save your configuration changes. STEP 5 | Create a branch template stack. 1. 2. 3. 4. 5. 6. Click Add Stack to add a new template stack. Enter a descriptive Name for the template stack. (Optional) Enter a Description for the template. Add the SD-WAN network template you created in the Step 4. In the Devices section, select the check boxes for all your SD-WAN branch firewalls. Click OK to save your configuration changes. STEP 6 | Commit your configuration changes. Create the Predefined Zones in Panorama SD-WAN policy rules use predefined zones for internal path selection and traffic forwarding purposes. There are two use cases; your use case depends on whether you are enabling SD-WAN on your current ® PAN-OS firewalls that have existing security policy rules or whether you are starting a brand new PANOS deployment with no previous security policy rules. If your current firewalls have security policy rules in place, you map your existing zones to the predefined zones that SD-WAN policies use. The SD-WAN engine makes use of the predefined zones for forwarding traffic. Additionally, creating the predefined zones in the Panorama™ templates provides consistent visibility between the managed firewalls and Panorama: • Zone Internet—For traffic going to and coming from the untrusted internet. • Zone to Hub—For traffic going from branch firewalls to hub firewalls and for traffic going between hub firewalls. • Zone to Branch—For traffic going from hub firewalls to branch firewalls and for traffic between branch firewalls. • Zone Internal—For internal traffic at a specific location. If you don’t create the predefined zones, the SD-WAN plugin will automatically create the predefined zones on your branch and hub firewall, but you won’t see them in Panorama. There are two main use cases for predefined zones: SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 21 2021 Palo Alto Networks, Inc. • Existing Zones—You already have pre-existing zones that you created for use in User-ID™ or various policies (security policy rules, QoS policy rules, zone protection, and packet buffer protection). You must map the pre-existing zones to the predefined zones that SD-WAN uses so the firewall can properly forward traffic. You should continue to use your pre-existing zones in all of your policies because the new predefined zones are used only for SD-WAN forwarding. You will map the zones when you to Add SD-WAN Devices to Panorama by creating your CSV file. (If you aren’t using a CSV file, you will map zones when you configure Panorama > SD-WAN > Devices and add existing zones to Zone Internet, Zone to Hub, Zone to Branch, and Zone Internal.) The result of mapping is that a branch or hub firewall can do a forwarding lookup to determine the egress SD-WAN interface and thus the egress zone. If you don’t map pre-existing zones to predefined zones, an allowed session won’t use SD-WAN. The mapping is necessary because existing customers have different zone names in place, and the firewall must narrow all of those zone names down to the predefined zones. You don’t necessarily have to map zones to all of the predefined zones, but you should map existing zones to at least the Zone to Hub and Zone to Branch zones. • No Existing Zones—You have a brand new deployment of Palo Alto Networks® firewalls and SD-WAN. In this case, you don’t have zones to map; we recommend you use the predefined zones in your PAN-OS policies and User-ID to simplify deployment. Before you begin configuring your SD-WAN deployment, for both use cases, you will create the required predefined zones in Panorama named zone-internet, zone-internal, zone-to-hub, and zoneto-branch. When you onboard your branch and hub firewalls, you will Add SD-WAN Devices to Panorama. For pre-existing customers, the SD-WAN plugin will internally map pre-existing zones with these predefined zones when executing SD-WAN policy rules, QoS policy rules, zone protection, User-ID, and packet buffer protection, and will use the predefined zones for zone logging and visibility in Panorama. For new customers, you are properly set up using the predefined zones. The predefined zones are also required in order to automatically set up VPN tunnels between your SDWAN hubs and branches when you push the configuration from Panorama to your managed SD-WAN devices. The zone names are case-sensitive and must match the names provided in this procedure. Your commit fails on the firewall if the zone names don’t match those described in this procedure. In this example, we are creating the zone named zone-internet. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Zones and in the Template context drop-down, select the network template you previously created. STEP 3 | Add a new zone. STEP 4 | Enter zone-internet, for example, as the Name of the zone. STEP 5 | For zone Type, select Layer3. STEP 6 | Click OK. 22 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 7 | Repeat the previous steps to create the remaining zones. In total, you must create the following zones: • • • • zone-to-branch zone-to-hub zone-internal zone-internet STEP 8 | Commit and Commit and Push your configuration changes. STEP 9 | Commit your changes. Create the SD-WAN Device Groups Create device groups, one for your hubs and one for your branches, containing all the policy rules and configuration objects for your SD-WAN hubs and branches. After you create the device groups for your hubs and branches, you must create a Security policy rule in each device group allowing traffic between the hub and branch zones. Creating these Security policy rules ensures that traffic between the SD-WAN device zones is allowed when the SD-WAN plugin creates the VPN tunnels after you create a VPN cluster. Configure identical configurations across your hub firewalls and an identical configuration across your branch firewalls. This greatly reduces the operational overhead of having to manage the configurations of multiple SD-WAN hubs and branches, and allows you to troubleshoot, isolate, update configuration issues much more rapidly. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create the Predefined Zones in Panorama. STEP 3 | Create the SD-WAN hub device group. 1. 2. 3. 4. Select Panorama > Device Groups and Add a device group. Enter SD-WAN_Hub as the Name for the device group. (Optional) Enter a Description for the template. In the Devices section, select the check boxes to assign the SD-WAN hubs to the group. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 23 2021 Palo Alto Networks, Inc. 5. For the Parent Device Group, select Shared. 6. Click OK. STEP 4 | Create the SD-WAN branch device group. 1. 2. 3. 4. 5. 6. Select Panorama > Device Groups and Add a device group. Enter SD-WAN_Branch as the Name for the device group. (Optional) Enter a Description for the template. In the Devices section, select the check boxes to assign the SD-WAN branches to the group. For the Parent Device Group, select Shared. Click OK. STEP 5 | Create a Security policy rule to control traffic flows from branch offices to the hub’s internal zone and from the hub’s internal zone to branch offices. 1. Select Policies > Security and in the Device Group context drop-down, select the SD-WAN_Hub device group. 2. Add a new policy rule. 3. Enter a Name for the policy rule, such as SD-WAN access--hub DG. 4. Select Source > Source Zone and Add the zone-internal and zone-to-branch. 5. Select Destination > Destination Zone and Add the zone-internal and zone-to-branch. 6. Select Application and Add applications to allow. You must allow BGP if you are using BGP routing. 7. Select Actions and Allow to allow the applications you selected. 8. Select Target and specify the target devices to which Panorama™ should push this rule. STEP 6 | Create a Security policy rule to control traffic originating from the branch offices’ internal zone to the hub and from the hub to the branch offices’ internal zone. 1. Select Policies > Security and in the Device Group context drop-down, select the SD-WAN_Branch device group. 2. Add a new policy rule. 3. Enter a Name for the policy rule, such as SD-WAN access--branch DG. 4. Select Source > Source Zone and Add the zone-internal and zone-to-hub. 5. Select Destination > Destination Zone and Add the zone-internal and zone-to-hub. 6. Select Application and Add applications to allow. You must allow BGP if you are using BGP routing. 7. Select Actions and Allow to allow the applications you selected. 8. Select Target and specify the target devices to which Panorama should push this rule. STEP 7 | Commit and push your configuration. 1. 2. 3. 4. Commit and Commit and Push your configuration changes. In the Push Scope section, click Edit Selections. Enable (check) Include Device and Network Templates and click OK. Commit and Push your configuration changes. There are two commit operations that are automatically performed when you commit and push the device group and template configuration. View the Tasks to verify that 24 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. the second commit is successful. Of these two commit operations, the first always fails. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 25 2021 Palo Alto Networks, Inc. Create a Link Tag Create a link tag to identify one or more physical links that you want applications and services to use in a specific order during SD-WAN traffic distribution and failover protection. Grouping multiple physical links allows you to maximize the application and service quality if the physical link health deteriorates. When planning how to group your links, consider the use or purpose of the links and group them accordingly. For example, if you are configuring links intended for low-cost or non-business-critical traffic, create a link tag and group these interfaces together to ensure that the intended traffic flows primarily on these links, and not on more expensive links that may impact business-critical applications or services. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Objects > Tags and select the appropriate device group from the Device Group context drop-down. STEP 3 | Add a new tag. STEP 4 | Enter a descriptive Name for the tag. For example; Low Cost Paths, Expensive Paths, General Access, Private HQ, or Backup. STEP 5 | Enable (check) Shared to make the Link Tag available to all device groups on the Panorama™ management server and to every virtual system (vsys) on any multi-vsys hub or branch that you push to. By configuring a Shared Link Tag, Panorama is able to reference the Link Tags in the firewall configuration validation and successfully commits and pushes the configuration to branches and hubs. The commit fails if Panorama is unable to reference a Link Tag. STEP 6 | (Optional) Select a Color for the tag. STEP 7 | Enter helpful Comments about the tag. For example, Group two low cost broadband links and a backup link for general access to the internet. STEP 8 | Click OK to save your configuration changes. STEP 9 | Commit and Commit and Push your configuration changes. STEP 10 | Configure an SD-WAN Interface Profile. 26 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Configure an SD-WAN Interface Profile Create an SD-WAN interface profile to define the characteristics of ISP connections and to specify the speed of links and how frequently the firewall monitors the link, and specify a Link Tag for the link. When you specify the same Link Tag on multiple links, you are grouping (bundling) those physical links into a link bundle or fat pipe. You must configure an SD-WAN interface profile and specify it for an Ethernet interface enabled with SD-WAN before you can save the Ethernet interface. Group links based on a common criterion. For example, group links by path preference from most preferred to least preferred, or group links by cost. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Network Profiles > SD-WAN Interface Profile and select the appropriate template from the Template context drop-down. STEP 3 | Add an SD-WAN interface profile. STEP 4 | Enter a user-friendly Name for the SD-WAN interface profile, which you’ll see in reporting, troubleshooting, and statistics. STEP 5 | Select the vsys Location if you have a multi-vsys Panorama™ management server. By default, vsys1 is selected. STEP 6 | Select the Link Tag that this profile will assign to the interface. STEP 7 | Add a Description for the profile. STEP 8 | Select the physical Link Type from the predefined list (ADSL/DSL, Cable modem, Ethernet, Fiber, LTE/3G/4G/5G, MPLS, Microwave/Radio, Satellite, WiFi, or Other). The firewall can support any CPE device that terminates and hands off as an Ethernet connection to the firewall; for example, WiFi access points, LTE modems, laser/microwave CPEs all can terminate with an Ethernet handoff. Private, point-to-point link types (MPLS, satellite, microwave, and Other) will form tunnels with only the same link type; for example, MPLS-to-MPLS and satellite-to-satellite. Tunnels will not be created between an MPLS link and an Ethernet link, for example. STEP 9 | Specify the Maximum Download (Mbps) speed from the ISP in megabits per second (range is 0 to 100,000; there is no default). You can enter a range using up to three decimal places, for example, 10.456. Ask your ISP for the link speed or sample the link’s maximum speeds with a tool such as speedtest.net and take an average of the maximums over a good length of time. STEP 10 | Specify the Maximum Upload (Mbps) speed to the ISP in megabits per second (range is 0 to 100,000; there is no default). You can enter a range using up to three decimal places, for example, 10.456. Ask your ISP for the link speed or sample the link’s maximum speeds with a tool such as speedtest.net and take an average of the maximums over a good length of time. STEP 11 | Select Eligible for Error Correction Profile interface selection to enable Forward Error Correction (FEC) or packet duplication for interfaces. You must enable this on both the SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 27 2021 Palo Alto Networks, Inc. encoding and decoding firewalls; you must also create an Error Correction profile to apply to the SD-WAN policy rule for specific applications. STEP 12 | VPN Data Tunnel Support determines whether the branch-to-hub traffic and return traffic flows through a VPN tunnel for added security (the default method) or flows outside of the VPN tunnel to avoid encryption overhead. • Leave VPN Data Tunnel Support enabled for public link types that have direct internet connections or internet breakout capability, such as cable modem, ADSL, and other internet connections. • You can disable VPN Data Tunnel Support for private link types such as MPLS, satellite, or microwave that do not have internet breakout capability. However, you must first ensure the traffic cannot be intercepted because it will be sent outside of the VPN tunnel. • The branch may have DIA traffic that needs to fail over to the private MPLS link connecting to the hub, and reach the internet from the hub. The VPN Data Tunnel Support setting determines whether the private data flows through the VPN tunnel or flows outside the tunnel, and the failed over traffic uses the other connection (that the private data flow doesn’t use). The firewall uses zones to segment DIA failover traffic from private MPLS traffic. STEP 13 | If you Configure DIA AnyPath, a principal virtual interface can have multiple hub virtual interfaces, so you must prioritize the order in which a particular hub is selected for failover. Specify such priority by setting the VPN Failover Metric for the VPN tunnels bundled in the hub virtual interface where this profile is applied. The lower the metric, the high the priority of the interface to be selected during failover. If multiple hub virtual interfaces have the same metric value, SD-WAN sends new session traffic to them in round-robin fashion. STEP 14 | (Optional) Select the Path Monitoring mode in which the firewall monitors the interfaces where you apply this SD-WAN Interface Profile. The firewall selects what it considers the best monitoring method based on Link Type. Retain the default setting for the link type unless an interface (where you apply this profile) has issues that require more aggressive or more relaxed path monitoring. 28 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. • Aggressive—(Default for all link types except LTE and Satellite) Firewall sends probe packets to the opposite end of the SD-WAN link at a constant frequency. Use this mode if you need fast detection and failover for brownout and blackout conditions. • Relaxed—(Default for LTE and Satellite link types) Firewall waits for a number of seconds (the Probe Idle Time) between sending sets of probe packets, making path monitoring less frequent. When the probe idle time expires, firewall sends probes for seven seconds at the Probe Frequency configured. Use this mode when you have low bandwidth links, links that charge by usage (such as LTE), or when fast detection isn’t as important as preserving cost and bandwidth. STEP 15 | Set the Probe Frequency (per second), which is the number of times per second that the firewall sends a probe packet to the opposite end of the SD-WAN link (range is 1 to 5; default is 5). The default setting provides subsecond detection of brownout and blackout conditions. If you change the Probe Frequency for a Panorama template, you should also adjust the Packet Loss percentage threshold in a Path Quality profile for a Panorama device group. STEP 16 | If you select Relaxed path monitoring, you can set the Probe Idle Time (seconds) that the firewall waits between sets of probe packets (range is 1 to 60; default is 60). STEP 17 | Enter the Failback Hold Time (seconds) that the firewall waits for a recovered link to remain qualified before the firewall reinstates that link as the preferred link after it has failed over (range is 20 to 120; default is 120). STEP 18 | Click OK to save the profile. STEP 19 | Commit and Commit and Push your configuration changes. STEP 20 | Monitor your application and link path health metrics, and generate reports of your application and link health performance. For more information, see Monitoring and Reporting. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 29 2021 Palo Alto Networks, Inc. Configure a Physical Ethernet Interface for SDWAN In Panorama™, configure a physical, Layer 3 Ethernet interface and enable SD-WAN functionality. To configure a physical interface, you must assign it an IPv4 address and next hop gateway, and assign an SDWAN Interface Profile to the interface. After you use Panorama to create a VPN cluster and export your hub and branch information in the CSV, Auto VPN configuration in the SD-WAN plugin uses this information to generate a configuration for the associated branches and hubs that includes the predefined SD-WAN zones and creates secure VPN tunnels between SD-WAN branches and hubs. Auto VPN configuration also generates the BGP configuration if you enter BGP information the CSV or in Panorama when you add an SD-WAN branch or hub. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Interfaces > Ethernet, select the appropriate template from the Template context drop-down, select a slot number, such as Slot1, and select an interface (for example, ethernet1/1). STEP 3 | Select the Interface Type as Layer3. STEP 4 | Select a Virtual Router or create a new Virtual Router. STEP 5 | Assign the Security Zone that is appropriate for the interface you’re configuring. For example, if you are creating an uplink to an ISP, you must know that the Ethernet interface you chose is going to an untrusted zone. STEP 6 | On the IPv4 tab, Enable SD-WAN. STEP 7 | Select Type of address: • Static—In the IP field, Add an IPv4 address and prefix length for the interface. You can use a defined variable, such as $uplink, with a range of addresses. Enter the IPv4 address of the Next Hop Gateway (the next hop from the IPv4 address you just entered). The Next Hop Gateway must be on the same subnet as the IPv4 address. The Next Hop Gateway is the IP address of the ISP’s default router that the ISP gave you when you bought the service. It is the next hop IP address to which the firewall sends traffic to reach the ISP’s network, and ultimately, the internet and the hub. • PPPoE—Enable PPPoE authentication for DSL links, enter the Username and Password, and Confirm Password. • DHCP Client—It is critical that DHCP assigns a default gateway, also known as the next hop gateway for the ISP connection. The ISP will provide all the necessary connectivity information, such as dynamic IP address, DNS servers, and the default gateway. If you select DHCP Client, be sure to disable the option Automatically create default route pointing to default gateway provided by server, which is enabled by default. 30 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 8 | On the SD-WAN tab, select an SD-WAN Interface Profile that you already created (or create a new SD-WAN Interface Profile) to apply to this interface. The SD-WAN Interface Profile has an associated link tag, so the interfaces where this profile is applied will have the associated link tag. An interface can have only one link tag. STEP 9 | Click OK to save the Ethernet interface. STEP 10 | Commit and Commit and Push your configuration changes. STEP 11 | (SD-WAN manual configuration only) Configure a Virtual SD-WAN Interface. Auto VPN configuration will perform this task if you are using Auto VPN. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 31 2021 Palo Alto Networks, Inc. Configure an Aggregate Ethernet Interface and Subinterfaces for SD-WAN Physical firewalls running PAN-OS 10.1 and SD-WAN Plugin 2.1.0 support SD-WAN on aggregated Ethernet (AE) interfaces so that an SD-WAN firewall in a data center, for example, can have an aggregate interface group (bundle) of physical Ethernet interfaces that provide link redundancy. SD-WAN supports AE interfaces with or without subinterfaces. You can create an AE interface with subinterfaces that you can tag for different ISP services in order to provide end-to-end traffic segmentation. Thus, your ISP services can reach multiple labs or buildings without needing a dedicated pair of fibers for each connection. A Layer 3 AE interface group connects to a router, as shown in the following figure: VM-Series firewalls do not support AE interfaces. An SD-WAN hub or branch firewall that has an AE interface should not belong to the same VPN cluster as a VM-Series SD-WAN hub or branch firewall because AE interfaces are not supported on VM-Series firewalls. PPPoE is not supported on subinterfaces. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Configure an SD-WAN Interface Profile for each ISP connection (subinterface) in the AE interface group to define its link attributes. STEP 3 | Create an AE interface group. 32 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 1. 2. 3. 4. Select Network > Interfaces > Ethernet, select a Panorama Template, and Add Aggregate Group. For Interface Name, enter the number to identify the aggregate group; range is 1 to 16. For Interface Type, select Layer3. Click OK. STEP 4 | Assign physical interfaces to the aggregate group. 1. Select Network > Interfaces > Ethernet and select the interface you want to assign to the aggregate group. 2. Select the Interface Type as Aggregate Ethernet. 3. Select the Aggregate Group you created; for example, ae1. 4. On the Advanced tab, select the Link Speed, Link Duplex, and Link State. 5. Click OK. 6. Repeat this step for each interface you want to assign to the aggregate group. STEP 5 | For the aggregate group, create a subinterface that uses a static IP address. 1. Select Network > Interfaces > Ethernet, highlight the aggregate interface, such as ae1, and click Add Subinterface at the bottom of the screen. 2. For Interface Name, enter a number after the period, such as 107. 3. Enter the VLAN Tag to differentiate between the subinterfaces. For ease of use, make the tag the same number as the subinterface ID. 4. Select the IPv4 tab and Enable SD-WAN. 5. Select the Type of address: Static. 6. Add the IP address (and subnet mask) of the subinterface. 7. Enter the IP address of the Next Hop Gateway. 8. Click OK. STEP 6 | Alternatively, for the aggregate group, create a subinterface that uses DHCP to get its address. 1. Select Network > Interfaces > Ethernet and in the Template field, select a Template Stack. 2. Highlight the aggregate interface, such as ae1, and click Add Subinterface at the bottom of the screen. 3. Highlight the subinterface and click Override at the bottom of the screen. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 33 2021 Palo Alto Networks, Inc. 4. Highlight the subinterface and for Interface Name, enter a number after the period, such as 1. 5. Enter the VLAN Tag to differentiate between the subinterfaces. For ease of use, make the tag the same number as the subinterface ID. 6. Select the IPv4 tab and Enable SD-WAN. 7. Select the Type of address: DHCP Client. 8. Select Enable. 9. Uncheck (do not select) Automatically create default route pointing to default gateway provided by server. 10.Select the Advanced tab and DDNS tab. 11.Select Settings and Enable. The Hostname is automatically generated by the Panorama SD-WAN plugin. 12.Select the Vendor as Palo Alto Networks DDNS. 13.Click OK. STEP 7 | Apply an SD-WAN Interface Profile to the subinterface. 1. Highlight the subinterface you created and select the SD-WAN tab. 2. Select the SD-WAN Interface Profile you created for this link or create a new profile. 34 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 3. Click OK. STEP 8 | Repeat the prior steps to create additional Layer3 subinterfaces for the aggregate interface group and apply an SD-WAN Interface Profile to each subinterface. STEP 9 | Commit. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 35 2021 Palo Alto Networks, Inc. Configure Layer 3 Subinterfaces for SD-WAN Firewalls running PAN-OS 10.1 and SD-WAN Plugin 2.1.0 support SD-WAN on Layer 3 subinterfaces so that the firewall can segment traffic using VLAN tags. The following task shows how to create a Layer3 subinterface that uses a static IP address and how to create one that uses DHCP to get its address. It shows how to assign a VLAN tag to the subinterface and enable SD-WAN on the subinterface. Create an SD-WAN interface profile to define each ISP connection and assign the profile to the corresponding subinterface (a virtual SD-WAN interface). If you configure SD-WAN Layer 3 subinterfaces on VM-Series firewalls, the VMware configuration must have respective portgroups attached to those interfaces that allow all VLANs. PPPoE is not supported on subinterfaces. STEP 1 | Configure an SD-WAN Interface Profile for each ISP connection (subinterface) to define its link attributes. STEP 2 | Create a Layer 3 subinterface that uses a static IP address. Select Network > Interfaces > Ethernet and in the Template field select a template. Select an interface. For Interface Type, select Layer3 and click OK. Highlight the interface and click Add Subinterface at the bottom of the screen. After the Interface Name and period, enter the subinterface number. Enter a Tag for the subinterface (range is 1 to 4,094). For ease of use, make the tag the same number as the subinterface ID. 7. On the IPv4 tab, Enable SD-WAN. 8. Select the Type of address: Static. 9. Add the IP address and subnet mask. 10.Enter the IP address of the Next Hop Gateway. 11.Click OK. 1. 2. 3. 4. 5. 6. STEP 3 | Alternatively, create a Layer 3 subinterface that uses DHCP to get its address. 1. Select Network > Interfaces > Ethernet and in the Template field, select a template stack (not a template). 36 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 2. 3. 4. 5. 6. 7. Select an interface. For Interface Type, select Layer3 and click OK. Highlight the interface and click Add Subinterfaces at the bottom of the screen. Highlight the subinterface and click Override. Highlight the subinterface and after the Interface Name and period, enter the subinterface number. Enter a Tag for the subinterface (range is 1 to 4,094). For ease of use, make the tag the same number as the subinterface ID. 8. On the IPv4 tab, Enable SD-WAN. 9. Select Type of address: DHCP Client and Enable. 10.Uncheck (do not select) Automatically create default route pointing to default gateway provided by server. 11.Select the Advanced tab and then the DDNS tab. 12.Select Settings and Enable. The Hostname is automatically generated by the Panorama SD-WAN plugin. 13.Select the Vendor as Palo Alto Networks DDNS. 14.Click OK. STEP 4 | Apply an SD-WAN Interface Profile to the subinterface. 1. Highlight the subinterface you created and select the SD-WAN tab. 2. Select the SD-WAN Interface Profile you created for this link or create a new profile. 3. Click OK. STEP 5 | Repeat the prior steps to add more subinterfaces to the interface. STEP 6 | Commit. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 37 2021 Palo Alto Networks, Inc. Configure a Virtual SD-WAN Interface If you use Auto VPN configuration through Panorama, it creates the SD-WAN interfaces for you, in which case you don’t create and configure a virtual SD-WAN interface. If you aren’t using Auto VPN configuration with Panorama, create and configure a virtual SD-WAN interface to specify one or more physical, SD-WAN-capable ethernet interfaces that go to the same destination, such as to a specific hub or to the internet. In fact, all links in a virtual SD-WAN interface must be the same type: all VPN tunnel links or all direct internet access (DIA) links. The first figure illustrates an example of an SD-WAN interface named SDWAN.1 that bundles two physical interfaces, which use different carriers: Ethernet1/1 (the cable modem link) and Ethernet1/2 (the fiber service link). Both links are a VPN tunnel from the branch to the hub. In this figure, both links in the SD-WAN interface happen to use the same link tag (Cheap Broadband), but links in an SD-WAN interface can have different link tags. In the following figure, SDWAN.2 bundles Ethernet1/1 and Ethernet1/2 links, which are both DIA links from the branch to the internet: 38 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Network > Interfaces > SD-WAN and select the appropriate template from the Template context drop-down. STEP 3 | Add a logical SD-WAN interface by entering a number (in the range 1 to 9,999) after the sdwan. prefix. Auto VPN configuration creates SD-WAN interfaces numbered .901, .902, and so on, so do not use these numbers. STEP 4 | Enter a descriptive Comment. Add a helpful comment, such as Branch to internet or Branch to western USA hub if you are on the Branch template. Your comment makes troubleshooting easier rather than trying to decipher auto-generated names in logs and reports. STEP 5 | On the Config tab, assign the SD-WAN interface to a Virtual Router. STEP 6 | Assign the SD-WAN interface to a Security Zone. The virtual SD-WAN interface and all of its interface members must be in the same Security zone, thus ensuring the same Security policy rules apply to all paths from the branch to the same destination. STEP 7 | On the Advanced tab, Add Interfaces, which are members that go to the same destination, by selecting one or more Layer 3 Ethernet interfaces (for DIA) or one more virtual VPN tunnel SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 39 2021 Palo Alto Networks, Inc. interfaces (for hub). If you enter more than one interface, they must all be the same type (either VPN tunnel or DIA). The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA or a hub location. During routing, the route table determines which virtual SD-WAN interface (egress interface) the packet will exit based on the destination IP address in the packet. Then the SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule that the packet matches determine which path to use (and the order in which to consider new paths if a path deteriorates.) STEP 8 | Click OK to save your configuration change. STEP 9 | Commit and Commit and Push your configuration changes. 40 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Create a Default Route to the SD-WAN Interface If you are using a service route to access Panorama™, to bring up the firewall you must create a default route that points to an SD-WAN interface you created. Auto VPN creates a virtual SD-WAN interface named sdwan.901 for DIA and creates a virtual SD-WAN interface named sdwan.902 for VPN tunnels. Auto VPN also creates its own default route that uses the sdwan.901 interface as its egress interface and uses a low metric, so that the sdwan.901 interface is preferred over the default route you created. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select the Template you are working on. STEP 3 | Select Network > Virtual Routers and select a virtual router, such as sd-wan. STEP 4 | Select Static Routes and Add a static route by Name. STEP 5 | For Destination, enter 0.0.0.0/0. STEP 6 | For egress Interface, select one of the logical SD-WAN interfaces you created to bring up the firewall, such as sdwan.1. The egress interface you select can be any logical SD-WAN interface except sdwan.901 or sdwan.902. STEP 7 | For Next Hop, select None. STEP 8 | For Metric, enter a value greater than 50, so that this default route is not preferred over the default route that Auto VPN creates with a low metric. STEP 9 | Click OK. STEP 10 | Select Commit and Commit and Push your configuration changes. STEP 11 | Commit your changes. STEP 12 | Repeat this task for other templates on firewalls that use a service route to access Panorama. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 41 2021 Palo Alto Networks, Inc. Create a Path Quality Profile Create a Path Quality profile for each set of business-critical and latency-sensitive applications, application filters, application groups, services, service objects and service group objects that has unique network quality (health) requirements based on latency, jitter, and packet loss percentage. Applications and services can share a Path Quality profile. Specify the maximum threshold for each parameter, above which the firewall considers the path deteriorated enough to select a better path. As an alternative to creating a Path Quality profile, you can use any of the predefined Path Quality profiles, such as general-business, voip-video, file-sharing, audio-streaming, photo-video, and remote-access, and more. The predefined profiles are set up to optimize the latency, jitter, and packet loss thresholds for the type of applications and services suggested by the name of the profile. The predefined Path Quality profiles for a Panorama device group are based on the default Probe Frequency settings in the SD-WAN Interface profile for a Panorama template. If you change the default Probe Frequency setting, you must adjust the Packet Loss percentage threshold in the Path Quality profile for the firewalls in a Device Group that are affected by the Panorama template where you changed the Interface profile. The firewall treats the latency, jitter, and packet loss thresholds as OR conditions, meaning if any one of the thresholds is exceeded, the firewall selects the new best (preferred) path. Any path that has latency, jitter, and packet loss less than or equal to all three thresholds is considered qualified and the firewall selected the path based on the associated Traffic Distribution profile. By default, the firewall measures latency and jitter every 200ms and takes an average of the last three measurements to measure path quality in a sliding window. You can modify this behavior by selecting aggressive or relaxed path monitoring when you Configure an SD-WAN Interface Profile. If a path fails over because it exceeded the configured packet loss threshold, the firewall still sends probing packets on the failed path and calculates its packet loss percentage as the path recovers. It can take approximately three minutes for the packet loss percentage on a recovered path to fall below the packet loss threshold configured in the Path Quality profile. For example, suppose an SD-WAN policy rule for an application has a Path Quality profile that specifies a packet loss threshold of 1% and a Traffic Distribution profile that specifies Top Down distribution with tag 1 (applied to tunnel.1) first on the list and tag 2 (applied to tunnel.2) next on the list. When tunnel.1 exceeds 1% packet loss, the data packets fail over to tunnel.2. After tunnel.1 recovers to 0% packet loss (based on probing packets), it can take up to three minutes for the monitored packet loss rate for tunnel.1 to drop below 1%, at which time the firewall then selects tunnel.1 as the best path again. The sensitivity setting indicates which parameter (latency, jitter, or packet loss) is more important (preferred) for the applications to which the profile applies. When the firewall evaluates link quality, it considers a parameter with a high setting first. For example, when the firewall compares two links, suppose one link has 100ms latency and 20ms jitter; the other link has 300ms latency and 10 ms jitter. If the sensitivity for latency is high, the firewall chooses the first link. If the sensitivity for jitter is high, the firewall chooses the second link. If the parameters have the same sensitivity (by default the parameters are set to medium), the firewall evaluates packet loss first, then latency, and jitter last. Reference the Path Quality profile in an SD-WAN policy rule to control the threshold at which the firewall replaces a deteriorating path with a new path for matching application packets. STEP 1 | Log in to the Panorama Web Interface. 42 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 2 | Select a Device Group. STEP 3 | Select Objects > SD-WAN Link Management > Path Quality Profile. STEP 4 | Add a Path Quality profile by Name using a maximum of 31 alphanumeric characters. STEP 5 | For Latency, double-click the Threshold value and enter the number of milliseconds allowed for a packet to leave the firewall, arrive at the opposite end of the SD-WAN tunnel, and a response packet to return to the firewall before the threshold is exceeded (range is 10 to 2,000; default is 100). STEP 6 | For Latency, select the Sensitivity (low, medium, or high). Default is medium. Click the arrow at the end of the Threshold column to sort thresholds in ascending or descending numerical order. STEP 7 | For Jitter, double-click the Threshold value and enter the number of milliseconds (range is 10 to 1,000; default is 100). STEP 8 | For Jitter, select the Sensitivity (low, medium, or high). Default is medium. STEP 9 | For Packet Loss, double-click the Threshold value and enter the percentage of packets lost on the link before the threshold is exceeded (range is 1 to 100.0; default is 1). Setting the Sensitivity for Packet Loss has no effect, so leave the default setting. If you change the Probe Frequency in an SD-WAN Interface profile for a Panorama template, you should also adjust the Packet Loss threshold for a Panorama device group. STEP 10 | Click OK. STEP 11 | Commit and Commit and Push your configuration changes. STEP 12 | Commit your changes. STEP 13 | Repeat this task for every Device Group. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 43 2021 Palo Alto Networks, Inc. Configure SaaS Monitoring Configure a SaaS quality profile to monitor Direct Internet Access (DIA) links between a SaaS application and your branch firewall. • Create a SaaS Quality Profile • Use Case: Configure SaaS Monitoring for a Branch Firewall • Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to the Same SaaS Application Destination • Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to a Different SaaS Application Destination Create a SaaS Quality Profile If your branch firewall has a Direct Internet Access (DIA) link to a Software-as-a-Service (SaaS) application, create a SaaS Quality profile to specify how one or more SaaS applications should be monitored. SaaS Quality profiles are associated with an SD-WAN policy rule to determine how the branch firewall determines the path quality thresholds for latency, jitter, and packet loss and selects the preferred path for an outgoing packet. The SaaS Quality profile supports up to four static IP addresses, or one fully qualified domain name (FQDN) or URL per SaaS Quality profile. When multiple static IP addresses are configured, the branch firewall monitors one IP address at a time in a cascading order based on how the IP addresses are ordered in the SaaS Quality profile. For example, if you add IP1, IP2, IP3, and IP4, the branch firewall monitors IP1 to determine if the path quality thresholds have been exceeded, then continues to IP2, and so forth. SD-WAN monitoring and reporting data displays the SaaS application and SaaS application IP, FQDN, or URL as it is currently configured in the SaaS Quality profile associated with an SD-WAN policy rule regardless of the time filter applied when viewing your SD-WAN monitoring data. For example, three days ago you initially configured the IP address of your SaaS application as 192.168.10.50 in a SaaS Quality profile and had traffic match the SD-WAN policy rule to which the SaaS Quality profile is associated. Today, you reconfigured this existing SaaS Quality profile and changed the SaaS application IP address to 192.168.10.20. When you go review the SD-WAN monitoring data, all existing monitoring data for this SaaS application display the IP address 192.168.10.20. STEP 1 | Log in to the Panorama web interface. STEP 2 | Select Objects > SD-WAN Link Management > SaaS Quality Profile and specify the Device Group containing your SD-WAN configuration. STEP 3 | Add a new SaaS quality profile. STEP 4 | Enter a descriptive Name for the SaaS Quality profile. STEP 5 | (Optional) Enable (check) Shared to make the SaaS Quality profile shared across all device groups. STEP 6 | (Optional) Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. 44 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Disable override can only be enabled if Shared is disabled in the previous step. STEP 7 | Configure the SaaS Monitoring Mode. • Automatically monitor the SaaS application path health. Enabled by default, Adaptive monitoring allows the branch firewall to passively monitor the SaaS application session for send and receive activity to determine if the path quality thresholds have been exceeded. The SaaS application path health quality is automatically determined without any additional health checks on the SD-WAN interface. Adaptive SaaS monitoring is supported only for TCP SaaS applications. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per critical SaaS application that you need monitored. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. SaaS monitoring is resource-intensive and may impact firewall performance if monitoring a large number of SaaS applications. It is a best practice to only monitor those business-critical SaaS applications that need good usability. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. 2. 3. 4. Configure a FQDN address object for the SaaS application. Select IP Address/Object > FQDN and Add the FQDN. Select the FQDN address object for the SaaS application. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 45 2021 Palo Alto Networks, Inc. • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. The minimum probe interval supported for a SaaS application HTTP/HTTPS is 3 seconds. 4. Click OK to save your configuration changes. STEP 8 | Select Commit and Commit and Push your configuration changes. Use Case: Configure SaaS Monitoring for a Branch Firewall If your organization is leveraging a business-critical SaaS application at a branch firewall location, you can configure a SaaS Quality profile and associate it with a SD-WAN policy rule to monitor the latency, jitter, and packet loss health metrics of the critical SaaS application and swap links from an SD-WAN branch firewall to a SaaS application on a Direct Internet Access (DIA) link to ensure application usability. If the business-critical SaaS application DIA link health metric thresholds are exceeded, the link is swapped to the next DIA link configured in the Traffic Distribution profile for all new sessions. The existing session on the degraded DIA link is not swapped over to the next DIA link. STEP 1 | Set up your SD-WAN deployment. 1. 2. 3. 4. Install the SD-WAN Plugin. Set Up Panorama and Firewalls for SD-WAN. Add SD-WAN Devices to Panorama. (High availability configurations only) Configure HA Devices for SD-WAN. 46 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 5. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. Creating a single Link Tag for multiple DIA links allows you to aggregate bandwidth between bundled links and allow the firewall to distribute sessions between multiple links. STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create a Path Quality profile to configure the latency, jitter, and packet loss thresholds and sensitivity in order to specify when the branch firewall should swap to the next DIA link. STEP 7 | Create a SaaS Quality profile to specify your SaaS application and the frequency the DIA link is monitored. STEP 8 | Create a Traffic Distribution profile to specify the order the branch firewall swaps to DIA links in the event of link health degradation. STEP 9 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to the Same SaaS Application Destination If your organization is leveraging a SaaS application at a branch firewall location but the branch firewall has no healthy DIA links to swap to, you can configure the hub firewall as a failover alternative to maintain a healthy connection to your SaaS application. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 47 2021 Palo Alto Networks, Inc. If the SaaS application DIA link health metric thresholds are exceeded and the branch firewall has no healthy DIA links available, the link is swapped to the next hub firewall for all new sessions. The existing session on the degraded DIA link is not swapped over to the hub firewall. For example, say your branch and hub firewalls are located in the same region and access a SaaS application using the same destination IP. You can configure the hub firewall to act as a failover in the event there are no healthy DIA links from the branch firewall to the SaaS application by configuring an identically named SaaS Quality profile on both the branch and hub firewalls to automatically failover to the hub firewall if no healthy DIA links are available from the branch firewall. This allows you to maintain a health path for your SaaS application and maintain accurate end-to-end SaaS application monitoring data without congesting your network bandwidth. STEP 1 | Set up your SD-WAN deployment. 1. 2. 3. 4. 5. Install the SD-WAN Plugin. Set Up Panorama and Firewalls for SD-WAN. Add SD-WAN Devices to Panorama. (High availability configurations only) Configure HA Devices for SD-WAN. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create identically named SaaS quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. The easiest way to accomplish this is to create a single SaaS Quality profile in the Shared device group. Alternatively, you can create two SaaS Quality profiles with identical names in different device groups and push them to your hub and branch firewalls. 48 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 1. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and from the Device Group drop-down select Shared. 2. Add a new SaaS Quality profile. 3. Enter a descriptive Name for the SaaS Quality profile. 4. Enable (check) Shared to make the SaaS Quality profile shared across all device groups. This is required to make the SaaS Quality profile available to all device groups your branch and hub firewalls belong to. 5. Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. 6. Configure the SaaS Monitoring Mode using one of the following methods. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per SaaS application. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. 2. 3. 4. Configure a FQDN address object for the SaaS application. Select IP Address/Object > FQDN and Add the FQDN. Select the FQDN address object for the SaaS application. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. STEP 7 | Create a Traffic Distribution profile to specify the order the branch firewall swaps from DIA links to VPN links to the hub firewall in the event of link health degradation. STEP 8 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 49 2021 Palo Alto Networks, Inc. Use Case: Configure a Hub Firewall Failover for SaaS Monitoring from a Branch Firewall to a Different SaaS Application Destination If your organization is leveraging a SaaS application at a branch firewall location but the branch firewall has no healthy DIA links to swap to, you can configure the hub firewall as a failover alternative to maintain a healthy connection to your SaaS application using a SaaS Quality profile pointing to a different SaaS application destination. If the SaaS application DIA link health metric thresholds are exceeded and the branch firewall has no healthy DIA links available, the link is swapped to the next hub firewall for all new sessions. The existing session on the degraded DIA link is not swapped over to the hub firewall. For example, say your branch and hub firewalls are located on opposite sides of the country and access a SaaS cloud application deployed in a cloud provider such as GCP. You can configure the hub firewall to act as a failover in the event there are no healthy DIA links from the branch firewall to the SaaS application. To accomplish this, configure an identically named SaaS Quality profile on both the branch and hub firewalls to automatically failover to the hub firewall if no healthy DIA links are available from the branch firewall. The SaaS Quality profile configured on the hub firewall to points to the on-ramp location closest to the hub to take advantage of local resources closest to it. This allows you flexibility in specifying healthy failover paths and the ability to maintain accurate end-to-end SaaS application monitoring data without congesting your network bandwidth. STEP 1 | Set up your SD-WAN deployment. 1. 2. 3. 4. 5. Install the SD-WAN Plugin. Set Up Panorama and Firewalls for SD-WAN. Add SD-WAN Devices to Panorama. (High availability configurations only) Configure HA Devices for SD-WAN. Create a VPN Cluster. STEP 2 | Create a Link Tag to group the SaaS application DIA links. Create multiple Link Tags for your DIA links in order to apply different SD-WAN monitoring settings for each SaaS application DIA link based on the link type. Additionally, you can create a single Link Tag for multiple DIA links to group the links into a single link bundle. STEP 3 | Configure an SD-WAN Interface profile to define the characteristics of your ISP connection and specify the speed of the DIA link, how frequently the branch firewall monitors the link, and select the Link Tag to specify to which link the SD-WAN Interface profile applies. If you created multiple Link Tags, you must configure an SD-WAN Interface profile for each Link Tag. If you created a link bundle by assigning multiple DIA links to a single Link Tag, specifying that link tag applies the SD-WAN Interface profile settings to all DIA links in the bundle. STEP 4 | Configure a physical Ethernet interface for each SaaS application DIA link. All physical Ethernet interfaces for DIA links must be Layer3. STEP 5 | Configure a Virtual SD-WAN Interface that groups all physical Ethernet interfaces for the SaaS application DIA links into a single interface group. 50 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. The firewall virtual router uses this virtual SD-WAN interface to route SD-WAN traffic to a DIA location. The SD-WAN path health and Traffic Distribution profiles in the SD-WAN policy rule then determine which path to use and the order in which to consider new paths if a path health deteriorates. STEP 6 | Create identically named SaaS quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. Create two SaaS Quality profiles with identical names each pointing to a different SaaS application destination in different device groups and push them to your hub and branch firewalls. 1. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and select the target device group containing the branch firewall from the Device Group drop-down. 2. Add a new SaaS Quality profile. 3. Enter a descriptive Name for the SaaS Quality profile. 4. Enable (check) Disable override to disable overriding the SaaS Quality profile configuration on the local firewall. 5. Configure the SaaS Monitoring Mode using one of the following methods. • Configure the Static IP address for the SaaS application. Create a SaaS Quality profile per SaaS application. If a SaaS application has multiple IP addresses, configure a SaaS Quality profile with the multiple static IP addresses for that SaaS application. 1. Select IP Address/Object > Static IP Address and Add an IP address. 2. Enter the IP address of the SaaS application or select a configured address object. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. • Configure the fully qualified domain name (FQDN) for the SaaS application. 1. 2. 3. 4. Configure a FQDN address object for the SaaS application. Select IP Address/Object > FQDN and Add the FQDN. Select the FQDN address object for the SaaS application. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 5. Click OK to save your configuration changes. • Configure the URL for the SaaS application. URL monitoring is only supported for traffic over ports 80, 443, 8080, 8081, and 143. 1. Select HTTP/HTTPS. 2. Enter the Monitored URL of the SaaS application. 3. Enter the Probe Interval by which the branch firewall probes the SaaS application path for health information. 4. Click OK to save your configuration changes. 6. Select Objects > SD-WAN Link Management > SaaS Quality Profile, and select the target device group containing the hub firewall from the Device Group drop-down. 7. Repeat Steps 6.2 through 6.5 to create an identically named SaaS Quality profile for a SaaS application at a different destination. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 51 2021 Palo Alto Networks, Inc. This step is required to make in identically named SaaS Quality profile in the device group your hub firewall belongs to. STEP 7 | Create a Traffic Distribution profile to specify the order the branch firewall swaps from DIA links to VPN links to the hub firewall in the event of link health degradation. STEP 8 | Configure an SD-WAN policy rule to specify the SaaS application and link health metrics, and determine how the firewall selects the preferred link for the critical SaaS application traffic. In the Application tab, add the SaaS application you are monitoring to the SD-WAN policy rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. 52 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. SD-WAN Traffic Distribution Profiles In an SD-WAN topology, the firewall detects a brownout, a blackout, and path deterioration per application and selects a new path to ensure you experience the best performance for your critical business applications. Having multiple ISP links allows you to scale your traffic capacity and reduce costs. The new path selection occurs in less than one second if you leave Path Monitoring and Probe Frequency with default settings; otherwise, new path selection could take more than one second. To implement such path selection, the firewall uses SD-WAN policy rules, which reference a Traffic Distribution profile that specifies how to select paths for session load distribution and for failover to a better path when path quality for an application deteriorates. Decide which traffic distribution method an application or service (that matches an SD-WAN policy rule) should use: • Best Available Path—Select this method if cost is not a factor and you will allow applications to use any path out of the branch. The firewall uses path quality metrics to distribute traffic and to fail over to one of the links belonging to a Link Tag in the list, thus providing the best application experience to users. • Top-Down Priority—If you have expensive or low-capacity links that you want used only as a last resort or as a backup link, use the Top-Down Priority method and place the tags that include those links last in the list of Link Tags in the profile. The firewall uses the top Link Tag in the list first to determine the links on which to session load traffic and on which to fail over. If none of the links in the top Link Tag are qualified based on the Path Quality profile, the firewall selects a link from the second Link Tag in the list. If none of the links in the second Link Tag are qualified, the process continues as necessary until the firewall finds a qualified link in the last Link Tag. If all associated links are overloaded and no link meets quality thresholds, the firewall uses the Best Available Path method to select a link on which to forward traffic. At the start of a failover event, the firewall starts at the top of the Top-Down Priority list of Link Tags to find a link to which it fails over. • Weighted Session Distribution—Select this method if you want to manually load traffic (that matches the rule) onto your ISP and WAN links and you don’t require failover during brownout conditions. You manually specify the link’s load when you apply a static percentage of new sessions that the interfaces grouped with a single Link Tag will get. The firewall distributes new sessions using round robin among the links having the specified Link Tags, until the link assigned the lowest percentage reaches that percentage of sessions. The firewall then uses the remaining link(s) in the same manner. You might select this method for applications that aren’t sensitive to latency and that require a lot of the link’s bandwidth capacity, such as large branch backups and large file transfers. If the link experiences brownout, the firewall doesn’t redirect the matching traffic to a different link. In the event of a failing path condition, the traffic distribution method you choose for application(s) in an SD-WAN policy rule, along with the Link Tags on groups of links, determine if and how the firewall selects a new path (performs link failover) as follows: Path Condition Top-Down Priority Best Available Path Weighted Session Distribution Session on existing path failed a path health threshold (brownout) Affected session fails over to better path (if available) Affected session fails over to better path (if available) Affected sessions don’t fail over Top-Down or Best Available Path Affected session fails back to previous path Affected session stays on existing path, doesn’t fail back Affected sessions don’t fail over SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 53 2021 Palo Alto Networks, Inc. Path Condition Top-Down Priority Best Available Path Weighted Session Distribution Top-Down or Best Available Path recovered: existing path fails a health check All sessions fail back to previous path Selective sessions fail back to previous path until affected existing path recovers Affected sessions don’t fail over Existing path is down (blackout) All sessions fail over to next path on list All sessions fail over to next best path All sessions fail over to other tags based on weight settings Brownout with no qualified (better) path Take best available path Take best available path Take best available path recovered: existing path is still qualified (good) Additionally, the firewall automatically performs session load sharing among interface members of a single Link Tag. After those interfaces approach their maximum Mbps, new sessions flow over to interfaces having a different Link Tag (based on the traffic distribution method) if those interfaces have better health metrics. Path Condition Top-Down Priority Best Available Path Weighted Session Distribution Multiple links with the same SD-WAN Tag Share session load equally among links within SD-WAN Tag Share session load based on best path within SD-WAN Tag Share session load based on % weight assigned to SD-WAN Tag Multiple links with different SD-WAN Tags Share session load based on list priority, load link(s) in first SDWAN Tag first. Share session load based on best path from all SD-WAN Tags Share session load based on % weight assigned to SD-WAN Tags The following figure illustrates an example of a Traffic Distribution profile that uses the Top-Down Priority method. The #1, #2, and #3 are the order of Link Tags of links the firewall examines, if necessary, to find a healthy path to complete an application session failover. For each separate failover event that arises, the firewall starts at the beginning of the Top-Down list of Link Tags. 54 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 1. In this Top-Down Priority example, packets from a branch carrying a specific application (for example, office365-enterprise-access) arrive at the firewall. The firewall uses the route table to determine the next hop to the destination and the outgoing interface, which is the virtual SD-WAN interface tunnel named sdwan.1. The Security policy rule allows the packets. The packets then match an SD-WAN policy rule (named Office365 to Hub1) that specifies the destination zone for the hub. The firewall uses the SD-WAN policy rule’s Path Quality profile, Traffic Distribution profile, and that profile’s Link Tags to determine which interface member (link) from sdwan.1 to use. The Traffic Distribution profile lists three Link Tags in this order: #1 Cheap Broadband, #2 HQ Backhaul, and #3 Backup (which is the order of Link Tags the firewall examines links to find a link to which it can fail over). 2. Assuming all paths are qualified (by the Path Quality profile), the firewall distributes the packets to one of the physical links tagged with first Link Tag in the Traffic Distribution profile list: Cheap Broadband. The sdwan.1 tunnel has two member interfaces (two carriers): the cable modem VPN tunnel and the fiber service VPN tunnel. The firewall first examines a link by round-robin, and chooses the first link it finds that is qualified, for example, the cable modem link. 3. If the first Cheap Broadband link (cable modem) isn’t a qualified link, the firewall selects the second Cheap Broadband link (fiber service). 4. If the second Cheap Broadband link (fiber service) isn’t a qualified link, the firewall selects the link tagged with the #2 link tag HQ Backhaul, which is a more expensive MPLS link to the same hub. 5. If the MPLS link isn’t a qualified link, the firewall selects the link tagged with the #3 link tag Backup, which is an even more expensive 5G LTE link to the same hub. 6. If the firewall doesn’t find a qualified link to fail over to, it uses the Best Available method to select a link. 7. Upon the start of a new failover event, the firewall starts at the top of the Top-Down list of Link Tags to find a link to which it will fail over. Keep in mind that SD-WAN traffic distribution is one of the later steps in the packet flow logic. Let’s zoom out to see a broader view of the packet flow. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 55 2021 Palo Alto Networks, Inc. Packet flow details for the figure are as follows: 1. When a session for an application arrives at the firewall, the firewall performs session lookup to determine if the session is an existing session or new session. 2. A new session goes through session setup: 1. Forwarding lookup—The firewall gets the egress zone, egress interface, and virtual system from the Layer 3 route table or Layer 2 Forwarding Database lookup, etc. For applications that match an SDWAN policy rule, the firewall uses the virtual SD-WAN interface as the egress interface. 2. NAT Policy lookup—If session matches a NAT rule, firewall does another forwarding lookup to determine the final (translated) egress interface and zone. 3. Security Policy lookup—If a Security Policy rule allows the session, the session is created and installed in the session table. The firewall then performs additional classification using App-ID™ and User-ID™. 3. Content Inspection—The firewall performs Threat Inspection (Anti-Spyware for IPS [Vulnerability ® Protection], Antivirus, URL Filtering, WildFire , etc.) on payload and headers as needed. 4. The Forwarding/Egress stage performs path selection and forwards the packets. This stage is where SDWAN path selection occurs. 1. Packet Forwarding Process—The firewall uses the ingress interface to determine the forwarding domain; performs routing, switching, or virtual wire forwarding. 2. SD-WAN path selection occurs when the application matches an SD-WAN policy rule; the Path Quality profile determines path qualification; the Traffic Distribution profile determines the method of path selection and the order in which paths are considered during the selection. 3. IPSec/SSL-VPN tunnel encryption occurs if needed. 4. Packet Egress Process - QoS shaping, DSCP rewrite, and IP fragmentation are applied (if needed). 5. Transmit Packet—The firewall forwards the packet over the selected egress interface. Now we zoom back in to examine the SD-WAN path selection logic in more detail. 56 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 1. The firewall consults the route table during Forwarding lookup; based on the destination IP address matching a Layer 3 prefix, the firewall determines the egress SD-WAN virtual interface. The packet is either going directly to the public internet or going back to the hub through a secure VPN link. 2. The firewall monitors each path by performing health checks that run over a VPN tunnel. Each DIA circuit has a VPN tunnel that monitors health information. 3. The application in the SD-WAN policy rule is associated with a Path Quality profile, and the firewall compares the path’s actual average latency, jitter, and packet loss values to the threshold values. 4. Any path that has a higher latency, jitter, or packet loss value than the threshold is not selected. 5. All qualifying paths in the virtual SD-WAN interface are then subjected to the Traffic Distribution profile’s method and path priority (ordering) logic. SD-WAN link tags group ISP services together, and the order of these tags in the Traffic Distribution profile prioritizes the paths during path selection. 6. Thus, the Path Quality Profile and the Traffic Distribution profile together determine the next best path to use and the firewall forwards the traffic out that link. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 57 2021 Palo Alto Networks, Inc. Create a Traffic Distribution Profile Based on your SD-WAN configuration plan, create the SD-WAN Traffic Distribution Profiles you need based on how you want the applications in your SD-WAN policy rules to be session loaded and to fail over. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Ensure you already configured the Link Tags in an SD-WAN interface profile and committed and pushed them. The Link Tags must be pushed to your hubs and branches in order for Panorama™ to successfully associate the Link Tags you specify in this Traffic Distribution profile to an SD-WAN interface profile. STEP 3 | Select a Device Group. STEP 4 | Create a Traffic Distribution profile. 1. Select Objects > SD-WAN Link Management > Traffic Distribution Profile. 2. Add a Traffic Distribution profile by Name using a maximum of 31 alphanumeric characters. 3. Select Shared only if you want to use this traffic distribution profile across all Device Groups (both hubs and branches). 4. Select one traffic distribution method and add a maximum of four Link Tags that use this method for this profile. • Best Available Path—Add one or more Link Tags. During the initial packet exchanges, before AppID has classified the application in the packet, the firewall uses the path in the tag that has the best health metrics (based on the order of tags). After the firewall identifies the application, it compares the health (path quality) of the path it was using to the health of the first path (interface) in the first Link Tag. If the original path’s health is better, it remains the selected path; otherwise, the firewall replaces the original path. The firewall repeats this process until it has evaluated all the paths in the Link Tag. The final path is the path the firewall selects when a packet arrives that meets the match criteria. When a link becomes unqualified and must fail over to the next best path, the firewall can migrate a maximum of 1,000 sessions per minute from the unqualified link to the next best path. For example, suppose tunnel.901 has 3,000 sessions; 2,000 of those sessions match SD-WAN policy rule A and 1,000 sessions match SD-WAN policy rule B (both rules have a traffic distribution policy configured with 58 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Best Path Available). If tunnel.901 becomes unqualified, it takes three minutes to migrate the 3,000 sessions from the unqualified link to the next best path. • Top Down Priority—Add one or more Link Tags. The firewall distributes new sessions (that meet the match criteria) to links using the top-to-bottom order of the Link Tags you added. The firewall examines the first tag configured for this profile, and examines the paths that use that tag, selecting the first path it finds that is qualified (that is at or below the Path Quality thresholds for this rule). If no qualified path is found from that Link Tag, firewall examines the paths that use the next Link Tag. If the firewall finds no path after examining all paths in all of the Link Tags, the firewall uses the Best Available Path method. The first path selected is the preferred path until one of the Path Quality thresholds for that path is exceeded, at which point the firewall again starts at the top of the Link Tag list to find the new preferred path. If you have only one link at the hub, that link supports all of the virtual interfaces and DIA traffic. If you want to use the link types in a specific order, you must apply a Traffic Distribution profile to the hub that specifies Top Down Priority, and then order the Link Tags to specify the preferred order. If you apply a Traffic Distribution profile that instead specifies Best Available Path, the firewall will use the link, regardless of cost, to choose the best performing path to the branch. In summary, Link Tags in a Traffic Distribution profile, the Link Tag applied to a hub virtual interface, and a VPN Failover Metric in an SD-WAN Interface Profile work only when the Traffic Distribution profile specifies Top Down Priority. • Weighted Session Distribution—Add one or more Link Tags and then enter the Weight percentage for each Link Tag so that the weights total 100%. The firewall performs session load distribution between Link Tags until their percentage maximums are reached. If there is more than one path in the Link Tag, the firewall distributes equally using round-robin until the path health metrics are reached, and then distributes sessions to the other member(s) that are not at the limit. If multiple physical interfaces have the same tag, the firewall distributes matching sessions evenly among them. If all paths fail a health (path quality) threshold, the firewall selects the path that has the best health statistics. If no SD-WAN links are available (perhaps due to a blackout), the firewall uses static or dynamic routing to route the matching packets. If a packet is routed to a virtual SD-WAN interface, but the firewall cannot find a preferred path for the session based on the SD-WAN policy’s Traffic Distribution profile, the firewall implicitly uses the Best Available Path method to find the preferred path. The firewall distributes any application sessions that don’t match an SD-WAN policy rule based on the firewall’s implicit, final rule, which distributes the sessions in round-robin order among all available links, regardless of the Traffic Distribution profile. If you prefer to control how the firewall distributes unmatched sessions, create a final catch-all rule to Distribute Unmatched Sessions to specific links in the order you specify. 5. (Optional) After adding Link Tags, use the Move Up or Move Down arrows to change the order of tags in the list, so they reflect the order in which you want the firewall to use links for this profile and for the selected applications in the SD-WAN policy rule. 6. Click OK. STEP 5 | Commit and Commit and Push your configuration changes. STEP 6 | Commit your changes. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 59 2021 Palo Alto Networks, Inc. Create an Error Correction Profile Forward error correction (FEC) is a method of correcting certain data transmission errors that occur over noisy communication lines, thereby improving data reliability without requiring retransmission. FEC is helpful for applications that are sensitive to packet loss or corruption, such as audio, VoIP, and video conferencing. With FEC, the receiving firewall can recover lost or corrupted packets by employing parity bits that the sending encoder embeds in an application flow. Repairing the flow avoids the need for SD-WAN data to fail over to another path or for TCP to resend packets. FEC can also help with UDP applications by recovering the lost or corrupt packets, since UDP does not retransmit packets. SD-WAN FEC supports branch and hub firewalls acting as encoders and decoders. The FEC mechanism has the encoder add redundant bits to a bitstream, and the decoder uses that information to correct received data if necessary, before sending it to the destination. SD-WAN also supports packet duplication as an alternative method of error correction. Packet duplication performs a complete duplication of an application session from one tunnel to a second tunnel. Packet duplication requires more resources than FEC and should be used only for critical applications that have low tolerance for dropped packets. Modern applications that have their own embedded recovery mechanisms may not need FEC or packet duplication. Apply FEC or packet duplication only to applications that can really benefit from such a mechanism; otherwise, much additional bandwidth and CPU overhead are introduced without any benefit. Neither FEC nor packet duplication is helpful if your SD-WAN problem is congestion. FEC and packet duplication functionality require Panorama to run PAN-OS 10.0.2 or a later release and SDWAN Plugin 2.0 or a later release that is compatible with the PAN-OS release. The encoder and decoder must both be running PAN-OS 10.0.2 or a later release. If one branch or hub is running an older software release than what is required, traffic with an FEC or packet duplication header is dropped at that firewall. Beginning with PAN-OS 10.0.3, FEC and packet duplication are supported in a full mesh topology, in addition to the hub-spoke topology already supported. Neither FEC nor packet duplication should be used on DIA links; they are only for VPN tunnel links between branches and hubs. To configure FEC or packet duplication on the encoder (the side that initiates FEC or packet duplication), use Panorama to: • Create an SD-WAN Interface Profile that specifies Eligible for Error Correction Profile interface selection and apply the profile to one or more interfaces. • Create an Error Correction Profile to implement FEC or packet duplication. • Apply the Error Correction Profile to an SD-WAN policy rule and specify a single application to which the rule applies. • Push the configuration to encoders. (The decoder [the receiving side] requires no specific configuration for FEC or packet duplication; the mechanisms are enabled by default on the decoder as long as the encoder initiates the error correction.) FEC and packet duplication support an MTU of 1,340 bytes. A packet larger than that will not go through the FEC or packet duplication process. STEP 1 | Log in to the Panorama Web Interface. 60 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 2 | Configure an SD-WAN Interface Profile, where you select Eligible for Error Correction Profile interface selection to indicate that the firewall can automatically use the interfaces (where the SD-WAN Interface Profile is applied) for error correction. Whether this option defaults to selected or not depends on the Link Type you select for the profile. You can have Eligible for Error Correction Profile interface selection unchecked in a profile and apply the profile to an expensive 5G LTE link, for example, so that costly error correction is never performed on that link. STEP 3 | Configure a Physical Ethernet Interface for SD-WAN and apply the SD-WAN Interface Profile that you created to an Ethernet interface. STEP 4 | Create an Error Correction Profile for FEC or packet duplication. 1. Select Objects > SD-WAN Link Management > Error Correction Profile. 2. Add an Error Correction profile and enter a descriptive Name of up to 31 alphanumeric characters; for example, EC_VOIP. 3. Select Shared to make the Error Correction profile available to all device groups on Panorama and to every virtual system on a multi-vsys hub or branch to which you push this configuration. Panorama can reference a Shared Error Correction profile in the firewall configuration validation and successfully commit and push the configuration to branches and hubs. The commit fails if Panorama cannot reference an Error Correction profile. 4. Specify the Activate when packet loss exceeds (%) setting—When packet loss exceeds this percentage, FEC or packet duplication is activated for the configured applications in the SD-WAN policy rule where this Error Correction profile is applied. Range is 1 to 99; the default is 2. 5. Select Forward Error Correction or Packet Duplication to indicate which error correction method the firewall uses when an SD-WAN policy rule references this SD-WAN Interface Profile; the default is Forward Error Correction. If you select Packet Duplication, SD-WAN selects an interface over which to send duplicate packets. (SD-WAN selects one of the interfaces you configured with Eligible for Error Correction Profile interface selection in the prior step.) 6. (Forward Error Correction only) Select the Packet Loss Correction Ratio: 10% (20:2), 20% (20:4), 30% (20:6), 40% (20:8), or 50% (20:10)—Ratio of parity bits to data packets; the default is 10% (20:2). The higher the ratio of parity bits to data packets that the sending firewall (encoder) sends, the higher SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 61 2021 Palo Alto Networks, Inc. the probability that the receiving firewall (decoder) can repair packet loss. However, a higher ratio requires more redundancy and therefore more bandwidth overhead, which is a tradeoff for achieving error correction. The parity ratio applies to the encoding firewall’s outgoing traffic. For example, if the hub firewall parity ratio is 50% and the branch firewall parity ratio is 20%, the hub firewall will receive 20% and the branch firewall will receive 50%. 7. Specify the Recovery Duration (ms)—Maximum number of milliseconds that the receiving firewall (decoder) can spend performing packet recovery on lost data packets using the parity packets it received (range is 1 to 5,000; default is 1,000). The firewall immediately sends data packets it receives to the destination. During the Recovery Duration, the decoder performs packet recovery for any lost data packets. When the recovery duration expires, all the parity packets are released. You configure the recovery duration in the Error Correction Profile for the encoder, which sends the Recovery Duration value to the decoder. A Recovery Duration setting on the decoder has no impact. Start by using the default Recovery Duration setting and adjust it if necessary, based on your testing with normal and intermittent brown-outs. 8. Click OK. STEP 5 | Configure an SD-WAN Policy Rule, reference the Error Correction Profile you created in the rule, and specify a critical application to which the rule applies. Specify only one application in the SD-WAN policy rule when configuring FEC or packet duplication. You should not combine multiple applications in a single policy rule for FEC or packet duplication. STEP 6 | Commit and Commit and Push your configuration changes to the encoding firewalls (branches and hubs). 62 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Configure an SD-WAN Policy Rule An SD-WAN policy rule specifies application(s) and/or service(s) and a traffic distribution profile to determine how the firewall selects the preferred path for an incoming packet that doesn’t belong to an existing session and that matches all other criteria, such as source and destination zones, source and destination IP addresses, and source user. The SD-WAN policy rule also specifies a path quality profile of thresholds for latency, jitter, and packet loss. When one of the thresholds is exceeded, the firewall selects a new path for the application(s) and/or service(s). When monitoring your SD-WAN traffic, traffic originating from a source behind the hub device is evaluated against the SD-WAN policies pushed to the hub device as it enters the hub device, and because the path selection decision has already been made, the branch device does not evaluate the traffic against its SD-WAN policies as it passes through the branch device to the final target device. Conversely, traffic originating from a source behind the branch device is evaluated against the SD-WAN policies pushed to the branch device and not by hub device. The Panorama™ management server aggregates the logs from both the hub and branch, and for the same traffic, two session entries are displayed but only the SD-WAN device that originally evaluated the traffic contains the SD-WAN details. In an SD-WAN policy rule, you can reference an Error Correction profile so that you can apply Forward Error Correction (FEC) or packet duplication for specified critical applications that have a low tolerance for dropped or corrupted packets. In an SD-WAN policy rule, you also specify the devices to which you want Panorama to push the rule. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Policies > SD-WAN and select the appropriate device group from the Device Group context drop-down. STEP 3 | Add an SD-WAN policy rule. STEP 4 | On the General tab, enter a descriptive Name for the rule. STEP 5 | On the Source tab, configure the source parameters of the policy rule. 1. Add the Source Zone or select Any source zone 2. Add one or more source addresses, set an external dynamic list (EDL), or select Any Source Address. 3. Add one or more source users or select any Source User. STEP 6 | On the Destination tab, configure the destination parameters of the policy rule. 1. Add the Destination Zone or select Any destination zone. 2. Add one or more destination addresses, set an EDL, or select Any Destination Address. STEP 7 | On the Application/Service tab, attach your SD-WAN Link Management profiles and specify your applications and services. PAN-OS 10.0.2 supports associating only a SaaS Quality Profile or an Error Correction but not both. If you associate one of these profiles with an SD-WAN policy rule, you cannot associate the other. For example, if you associate a SaaS Quality profile with an SD-WAN policy rule, you are unable to associate an Error Correction profile with the same SD-WAN policy rule. 1. Select the Path Quality or Create a Path Quality Profile. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 63 2021 Palo Alto Networks, Inc. 2. Select the SaaS Quality Profile or Create a SaaS Quality Profile if the branch firewall has a Direct Internet Access (DIA) link to a SaaS application. The default is None (disabled). 3. Select the Error Correction Profile or Create an Error Correction Profile to apply forward error correction (FEC) or packet duplication to the applications that match the SD-WAN policy rule. The default is None (disabled). 4. Add Applications and select one or more applications from the list or select Any applications. All applications you select are subject to the health thresholds specified in the Path Quality profile you selected. If a packet matches one of these applications and that application exceeds one of the health thresholds in the Path Quality profile (and the packet matches the remaining rule criteria), the firewall selects a new preferred path. Add only business-critical applications and applications that are sensitive to path conditions for their usability. If you associate a SaaS Quality profile in Adaptive mode with the SD-WAN policy, add the specific SaaS applications you want to monitor. Using adaptive monitoring for all applications that match the SD-WAN policy rule may impact the performance of the SD-WAN firewall. If you associate a SaaS Quality profile with a specified SaaS application, add the SaaS application to the SD-WAN rule to ensure the SaaS monitoring settings are applied only to the desired SaaS application. 5. Add Services and select one or more services from the list or select Any services. All services you select are subject to the health thresholds specified in the Path Quality profile you selected. If a packet matches one of these services and that service exceeds one of the health thresholds in the Path Quality profile (and the packet matches the remaining rule criteria), the firewall selects a new preferred path. Add only business-critical services and services that are sensitive to path conditions for their usability. 64 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 8 | On the Path Selection tab, select a Traffic Distribution profile or Create a Traffic Distribution Profile. When an incoming packet (unassociated with a session) matches all the match criteria in the rule, the firewall uses this Traffic Distribution profile to select a new preferred path. STEP 9 | On the Target tab, use one of the following methods to specify the target firewalls in the device group to which Panorama pushes the SD-WAN policy rule: • Select Any (target to all devices) (the default) to push the rule to all devices. Alternatively, select Devices or Tags to specify the devices to which Panorama pushes the SD-WAN policy rule. • On the Devices tab, select one or more filters to restrict the selections that appear in the Name field; then select one or more devices to which Panorama pushes the rule, as in this example: • On the Tags tab, Add one or more Tags and select the tag(s) to specify that Panorama push the rule to devices that are tagged with the selected tags, as in this example: SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 65 2021 Palo Alto Networks, Inc. • If you specified Devices or Tags, you can select Target to all but these specified devices and tags to have Panorama push the SD-WAN policy rule to all devices except for the specified devices or tagged devices. STEP 10 | Click OK. STEP 11 | Commit and Commit and Push your configuration changes. STEP 12 | (Best Practice) Create a catch-all SD-WAN policy rule to Distribute Unmatched Sessions so that you can control which links any unmatched sessions use and view unmatched sessions in logging and reports in the SD-WAN plugin. If you don’t create a catch-all rule to distribute unmatched sessions, the firewall distributes them in round-robin order among all available links because there is no traffic distribution profile for unmatched sessions. Round-robin distribution of unmatched sessions can increase your costs unexpectedly and result in loss of application visibility. STEP 13 | After configuring your SD-WAN policy rules, Create a Security Policy Rule to allow traffic (for example, bgp as an Application) from branches to the internet, from branches to hubs, and from hubs to branches. STEP 14 | (Optional) Configure QoS for critical applications. If the SD-WAN applications need guaranteed bandwidth capacities or if you do not want other applications taking bandwidth from critical business applications, create QoS rules to control the bandwidth properly. STEP 15 | To automatically set up BGP routing between VPN cluster members, in the SD-WAN plugin, Configure BGP routing between branches and hubs to dynamically route traffic that will be subject to the SD-WAN failover and load sharing. Alternatively, if you want to manually configure BGP routing on each firewall or use a separate Panorama template to configure BGP routing (for more control), leave the BGP information in the plugin blank. Instead, configure BGP routing. STEP 16 | Configure NAT for public-facing virtual SD-WAN interfaces. 66 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Allow Direct Internet Access Traffic Failover to MPLS Link At an SD-WAN branch office, the firewall performs split tunneling so that any applications having a public IP address take the Direct Internet Access (DIA) interface to the internet, and applications having private IP addresses that belong to the hub take the VPN interface. The firewall automatically fails over DIA applications to the MPLS private connection to the hub when necessary, so that the traffic destined for the internet takes an alternative path through the hub to reach the internet. To allow this to work, you must do the following: STEP 1 | Create an MPLS link between your branch and hub. When you create the SD-WAN Interface profile, the link type must be MPLS for both the hub and branch. STEP 2 | If you want the private traffic to go through the VPN tunnel, enable VPN Data Tunnel Support in the SD-WAN Interface profile. If you disable VPN Data Tunnel Support, the private data will go outside of the VPN tunnel. STEP 3 | Configure an SD-WAN Policy Rule for specific applications, Create a Path Quality Profile, and Create a Traffic Distribution Profile that specifies the Top Down Priority method. The Traffic Distribution profile must also specify an MPLS link as one of the failover options (identified by a tag). Verify that the applications in the SD-WAN policy rule reference the correct Path Quality and Traffic Distribution profiles, and that the Traffic Distribution profile specifies Top Down Priority. After the VPN Data Tunnel Support is enabled on both the hub and branch and the MPLS link is operational, the firewall automatically uses the MPLS connection to fail over DIA traffic when necessary. STEP 4 | In the hub configuration, ensure the hub has a path to the internet and routing is properly set up for the hub traffic to reach the internet. The firewall uses the DIA virtual interface and the VPN virtual interface to ensure that the public internet traffic is kept separate from your private traffic in the same path; that is, the internet traffic and private traffic do not go through the same VPN tunnel. Full segmentation with proper zoning is in full effect. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 67 2021 Palo Alto Networks, Inc. Configure DIA AnyPath When your SD-WAN direct internet access (DIA) links from an ISP experience a blackout or brownout, you need those links to fail over to another link to ensure business continuity. DIA links can fail over to an MPLS link, but you may not have an MPLS link. DIA links must be able to fail over to another link that has a direct path or indirect path (through a hub or branch) to the internet; the DIA traffic can take any path available to get to the internet and isn’t restricted to DIA. DIA AnyPath supports a DIA link failing over to a private VPN tunnel going to a hub firewall to then reach the internet. Furthermore, if your topology is full mesh (branchto-branch) and there is no hub, the DIA traffic can fail over to a branch firewall to reach the internet. DIA AnyPath requires PAN-OS 10.0.3 or a later 10.1 release and SD-WAN Plugin 2.0.1 or a later 2.0 release. There are several use cases for wanting an internet link to fail over to a VPN tunnel (DIA AnyPath): • You want to transition away from an expensive MPLS link to one or more public internet connections, usually from different vendors. • You have multiple hubs in a VPN cluster to allow a waterfall-type of failover from the primary hub to a succession of backup hubs. • In a split tunneling scenario, you want only a particular bandwidth-intensive application to go directly to the internet through the branch’s DIA link instead of going back to the data center hub through the VPN tunnel, thus saving WAN bandwidth costs. In the event of a DIA brownout or blackout, this application traffic fails over to the data center hub to reach the internet; and if necessary can subsequently fail over to a second hub to reach the internet. • In a different split tunneling scenario, you want most of your internet traffic to go out the DIA link rather than backhaul the traffic to the data center for internet breakout. However, you want specific applications (that may need additional scanning or logging by another security device) to go back to the data center. You create an SD-WAN policy rule to direct those applications to a primary path to the hub, rather than the normal DIA link as determined by the default route in the firewall’s route table. In the event of a brownout or blackout, those applications fail over to take the branch’s DIA interface. DIA AnyPath introduces the concept of a principal virtual interface, which can include both DIA links and nested hub virtual interfaces and branch virtual interfaces (VPN tunnels) that each include their own links. The principal virtual interface can have a maximum of nine DIA (Ethernet) interfaces, hub virtual interfaces, and branch virtual interfaces. You assign a Link Tag to a hub when you add the hub device to Panorama. Assuming you use the SD-WAN plugin, Auto VPN assigns that Link Tag to the hub virtual interface, which allows you to specify the tag in a Traffic Distribution profile to control the failover order among virtual interfaces. A principal virtual interface is referred to as a DIA-VIF in CLI commands. A principal virtual interface can have interface members that belong to different Security zones. However, the best practice is to have all member interfaces in the principal virtual interface belong to the same Security zone. Another best practice is to have at least one member interface in the principal virtual interface be of the link type Ethernet, Cable mode, ADSL, Fiber, LTE, or WiFi. The following topology example shows Branch1 with two ISP connections and an MPLS link. Branch1 also has a Hub1 virtual interface with three VPN tunnels connecting to Hub1, and a Hub2 virtual interface of three VPN tunnels connecting to Hub2. Branch1 also has a branch2 virtual interface with three VPN 68 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. tunnels connecting to Branch2 and a branch3 virtual interface with three VPN tunnels connecting to Branch3. The goal of DIA AnyPath is to configure the order in which DIA can fail over to VPN tunnels to reach the internet directly or indirectly and thus maintain business continuity. When you configure a principal virtual interface, it automatically becomes the default route so that internet traffic is routed properly to any of the members of the principal virtual interface (both DIA links and VPN tunnels). The path selection is based on SD-WAN Path Quality profiles and Traffic Distribution profiles, which you would set to use the Top Down Priority distribution method to control the failover order. In the example topology, a Traffic Distribution profile can list the tag for the principal virtual interface first, then the tag for the Hub1 virtual interface, and then the tag for the Hub2 virtual interface. Zooming in to a deeper level of failover priority, a hub virtual interface has multiple tunnel members, so you need a way to prioritize the failover order of the members, such as prioritizing that a broadband VPN tunnel be used before an LTE VPN tunnel. You specify the priority using the VPN Failover Metric in the SD-WAN Interface Profile that you apply to the Ethernet interface. The lower the metric value, the higher the priority for the tunnel to be selected upon failover. In the topology example, in the Hub1 virtual interface, a lower VPN Failover Metric for t11 than for t12 causes internet traffic to fail over to t11 before t12. If multiple tunnels in a virtual interface have the same metric, SD-WAN sends new session traffic to the tunnels in round-robin fashion. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Specify the failover priority for a VPN tunnel bundled in a hub virtual interface or branch virtual interface. 1. Select or Configure an SD-WAN Interface Profile. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 69 2021 Palo Alto Networks, Inc. Best Practice is to configure at least one interface with the link type of Ethernet, Cable modem, ADSL, Fiber, LTE, or WiFi. 2. You must enable VPN Data Tunnel Support. 3. Specify the VPN Failover Metric for a VPN tunnel; range is 1 to 65,535; default is 10. The lower the metric value, the higher the priority of VPN tunnel (link) where you apply this profile. For example, set the metric to a low value and apply the profile to a broadband interface; then create a different profile that sets a high metric to apply to an expensive LTE interface to ensure it is used only after broadband has failed over. If you have only one link at the hub, that link supports all of the virtual interfaces and DIA traffic. If you want to use the link types in a specific order, you must apply a Traffic Distribution profile to the hub that specifies Top Down Priority, and then order the Link Tags to specify the preferred order. If you apply a Traffic Distribution profile that instead specifies Best Available Path, the firewall will use the link, regardless of cost, to choose the best performing path to the branch. In summary, Link Tags in a Traffic Distribution profile. the Link Tag applied to a hub virtual interface (Step 6 in this task), and a VPN Failover Metric work only when the Traffic Distribution profile specifies Top Down Priority. 4. Click OK. STEP 3 | Configure a Physical Ethernet Interface for SD-WAN and on the SD-WAN tab, apply the SD- WAN Interface Profile you created in the prior step. Best Practice is to have all interfaces in the principal virtual interface belong to the same Security zone. STEP 4 | Repeat Steps 2 and 3 to configure additional SD-WAN Interface Profiles with a different VPN failover metric and apply the profiles to different Ethernet interfaces to determine the order in which failover occurs to the links. STEP 5 | Create a Link Tag for a hub virtual interface. 70 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 6 | Add the Link Tag to a hub that you want to participate in DIA AnyPath. 1. In Panorama > SD-WAN > Devices, Add an SD-WAN Device to add a hub to be managed by Panorama. 2. Select the hub. 3. Select the Link Tag that you created in the prior step, which Auto VPN applies to the whole hub virtual interface, not an individual link. Thus, you can reference this Link Tag in the Traffic Distribution Profile to indicate the hub virtual interface for the failover order for DIA AnyPath. On the branch device, Auto VPN uses this tag to populate the Link Tag field on the SD-WAN virtual interface that terminates on the hub device. 4. Click OK. STEP 7 | Repeat Steps 5 and 6 to create a Link Tag for each hub virtual interface and add the tag to each hub that will participate in DIA AnyPath. Do the same for any branch virtual interface. STEP 8 | Create a Traffic Distribution Profile to implement DIA AnyPath. 1. Create a Traffic Distribution Profile. 2. Select Top Down Priority. 3. Add the Link Tags so they appear in the order that you want their associated links used for failover. For example, if your use case is for certain applications to use DIA first, list the DIA tag first, then a hub virtual interface tag, then a second hub virtual interface tag. If your use case is for certain applications to first go to the hub and then the internet, list a hub virtual interface first, then perhaps a second hub virtual interface, and list a DIA tag last. If you have full mesh with no hub, use the DIA tag and branch virtual interface tags in the order you want. STEP 9 | Create identically named SaaS Quality profiles for both the hub and branch firewalls. Two identically named SaaS Quality profiles must be configured on the hub and branch firewalls to successfully leverage the hub firewall as an alternative failover. The easiest way to configure failover to a hub firewall with the same SaaS application destination is to create a single SaaS Quality profile in the Shared device group. Alternatively, you can create two SaaS Quality profiles with identical names in different device groups and push them to your hub and branch firewalls. To failover to a hub firewall with different SaaS application destinations, create two SaaS Quality profiles with identical names each pointing to a different SaaS application destination in different device groups and push them to your hub and branch firewalls. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 71 2021 Palo Alto Networks, Inc. You must also create an SD-WAN policy rule that references this SaaS Quality profile in order to allow the hub to advertise link quality statistics for the SaaS Quality profile to the branch. Doing so will provide end-to-end SaaS monitoring through the hub. Without this SD-WAN policy rule, you would have only the link measurements from the branch to the hub, but not from the hub to the SaaS application. STEP 10 | Allow the hubs to participate in DIA AnyPath. 1. Create a VPN Cluster and select a hub. 2. Select Allow DIA VPN for the hub. A maximum of four hubs in a VPN cluster can participate in DIA AnyPath. If they are HA hubs, a total of eight hubs are supported. If you Allow DIA VPN for one HA peer in a pair, you must also enable it for the other HA peer. STEP 11 | Create an SD-WAN policy rule for specific application(s) to use DIA AnyPath. 1. Configure an SD-WAN Policy Rule. 2. On the Application/Service tab, specify the applications and services for which you want to implement DIA AnyPath. 3. Associate the SaaS Quality Profile you created in the previous step. If you are configuring a SaaS Quality profile with different SaaS application destination, you must associate the SaaS Quality profile with the SD-WAN policy rule in each branch and hub device group. 4. On the Path Selection tab, select the Traffic Distribution Profile you created for the applications. STEP 12 | Route new sessions that don’t match any SD-WAN policy rule and sessions that arrive during a Panorama or firewall configuration change. 1. Create an appropriate Path Quality profile and Traffic Distribution profile to handle such sessions. 2. Configure an SD-WAN Policy Rule that is a catch-all rule for these sessions. 3. Place the rule last in the list. STEP 13 | Commit and Push to Devices. STEP 14 | Create a Security Policy Rule to allow DIA traffic to the Destination Zones named zoneinternet and zone-to-hub and specify the Applications subject to the rule. Commit and push to the branches. STEP 15 | Use the following CLI commands to monitor DIA information: 1. show sdwan connection <dia-vif-name> 2. show sdwan path-monitor stats dia-vif all 72 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 3. show sdwan path-monitor dia-anypath 4. show sdwan path-monitor dia-anypath packet-buffer all 5. show sdwan path-monitor stats conn-idx <IDX> SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 73 2021 Palo Alto Networks, Inc. Distribute Unmatched Sessions The firewall attempts to match sessions that arrive at an SD-WAN virtual interface to an SD-WAN policy rule; the firewall examines the SD-WAN policy rules in order from the top down, just as it does for Security policy rules. • If there is an SD-WAN rule match, the firewall executes the path monitoring and traffic distribution for that SD-WAN policy rule. • If there is no match to any SD-WAN policy rule in the list, the session matches an implied SD-WAN policy rule at the end of the list that uses the round-robin method to distribute unmatched sessions among all links in one SD-WAN interface, which is based on the route lookup. Furthermore, if there is no SD-WAN policy rule for a specific application, the firewall doesn’t track that application’s performance in the SD-WAN-specific visibility tools such as logging and reports in the SDWAN plugin. To illustrate the implied policy rule: • Suppose the firewall has three SD-WAN policy rules: one rule specifies five voice applications, one rule specifies six video conferencing applications, and one rule specifies ten SaaS applications. • A session, for example, a video application session, arrives at the firewall and doesn’t match any of the SD-WAN policy rules. Because the session didn’t match a rule, the firewall has no path quality profile or traffic distribution profile to apply to the session. • Therefore, firewall matches the video application to the implied rule and distributes each video session among all of the available SD-WAN link tags and their associated links on the firewall, which could be two broadband links, an MPLS link, and an LTE link. Session 1 goes to one member of the broadband interface, session 2 goes to another member of the broadband interface, session 3 goes to MPLS, session 4 goes to LTE, session 5 goes to the first member of the broadband interface, session 6 goes to the second member of the broadband interface, and the round-robin distribution continues. You may not want to let your unmatched sessions resort to matching the implied SD-WAN rule because you have no control over that session distribution. Instead, we recommend you create a catch-all SD-WAN policy rule and place it last in the list of SD-WAN policy rules. A catch-all SD-WAN policy rule lets you: • Control which links the unmatched sessions use. • View all of the applications on the firewall (including unmatched application sessions) in logging and reports in the SD-WAN plugin. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Create a Path Quality Profile that sets very high latency, jitter, and packet loss thresholds that will never be exceeded. For example, 2,000ms latency, 1,000ms jitter, and 99% packet loss. STEP 3 | Create a Traffic Distribution Profile that specifies the SD-WAN link tags you want to use, in the order in which you want the links associated with those link tags to be used by unmatched sessions. If you don’t want unmatched applications to use a specific path (physical interface) at all, omit the tag that includes that link from the list of link tags in the traffic distribution profile. For example, if you don’t want an unmatched application such as movie streaming to use the expensive LTE link, omit the link tag for the LTE link from the list of link tags in the traffic distribution profile. 74 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 4 | Add a catch-all SD-WAN policy rule and on the Application/Service tab, specify the Path Quality Profile that you created. STEP 5 | Select Any for the Applications and Service. STEP 6 | On the Path Selection tab, select the Traffic Distribution Profile you created. STEP 7 | Move the rule down to the last position in the list of SD-WAN policy rules. STEP 8 | Commit and Commit and Push your configuration changes. STEP 9 | Commit your changes. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 75 2021 Palo Alto Networks, Inc. Add SD-WAN Devices to Panorama Add a single SD-WAN hub or branch firewall or use a CSV to bulk import multiple SD-WAN hub and branch firewalls. • Add an SD-WAN Device • Bulk Import Multiple SD-WAN Devices Add an SD-WAN Device Add an SD-WAN hub or branch firewall to be managed by the Panorama™ management server. When adding your devices, you specify what type of device it is (branch or hub) and you give each device its site name for easy identification. Before adding your devices, plan your SD-WAN configuration to ensure you have all the required IP addresses and that the SD-WAN topology is well understood. This helps in reducing any configuration errors. ® If you have pre-existing zones for your Palo Alto Networks firewalls, you will be mapping them to the predefined zones used in SD-WAN. If you want to have Active/Passive HA running on two branch firewalls or two hub firewalls, do not add those firewalls as SD-WAN devices at this time. You will add them as HA peers separately when you Configure HA Devices for SD-WAN. If you are using BGP routing, you must add a security policy rule to allow BGP from the internal zone to the hub zone and from the hub zone to the internal zone. If you want to use 4-byte ASNs, you must first enable 4-byte ASNs for the virtual router. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Devices and Add a new SD-WAN firewall. STEP 3 | Select the managed firewall Name to add as an SD-WAN device. You must add your SD-WAN firewalls as managed devices before you can add them as an SD-WAN device. STEP 4 | Select the Type of SD-WAN device. • Hub—A centralized firewall deployed at a primary office or location to which all branch devices connect using a VPN connection. Traffic between branches passes through the hub before continuing to the target branch, and connects branches to centralized resources at the hub location. The hub device processes traffic, enforces policy rules, and manages link swapping at the primary office or location. • Branch—A firewall deployed at a physical branch location that connects the hub using a VPN connection and provides security at the branch level. The branch device processes traffic, enforces policy rules, and manages link swapping at the branch location. STEP 5 | Select the Virtual Router Name to use for routing between the SD-WAN hub and branches. By default, an sdwan-default virtual router is created and enables Panorama to automatically push router configurations. STEP 6 | Enter the SD-WAN Site name to identify the geographical location or purpose of the device. 76 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. The SD-WAN Site name supports all upper-case and lower-case alphanumerical and special characters. Spaces are not supported in the Site name and result in monitoring (Panorama > Monitoring) data for that site not to be displayed. If you are adding an HA pair, you must use the same Site name for the two peers. STEP 7 | (PAN-OS 10.0.3 and later 10.1 releases) Select the Link Tag you created for the hub virtual interface (or branch virtual interface), which Auto VPN will assign to the virtual interface. You will use this Link Tag in a Traffic Distribution profile to allow the hub (or branch) to participate in DIA AnyPath. STEP 8 | If you are adding a hub that is behind a device performing NAT for the hub, you must specify the IP address or FQDN of the public-facing interface on that upstream NAT-performing device, so that Auto VPN Configuration can use that address as the tunnel endpoint of the hub. It is the IP address that the branch office’s IKE and IPSec flows must be able to reach. (You must have already configured a physical Ethernet interface for SD-WAN.) 1. On the Upstream NAT tab, enable Upstream NAT. 2. Add an SD-WAN interface; select an interface you already configured for SD-WAN. 3. Select IP Address or FQDN and enter the IPv4 address without a subnet mask (for example, 192.168.3.4) or the FQDN of the upstream device, respectively. 4. Click OK. Additionally, on the upstream device that is performing NAT you must set up the inbound Destination NAT with a one-to-one NAT policy, and you must not configure port translation to the IKE or IPSec traffic flows. If the IP address on the upstream device changes, you must configure the new IP address and push it to the VPN cluster. You must use the CLI commands clear ipsec, clear ike-sa, and clear session all on both the branch and hub. You must also clear session all on the virtual router where you configured the NAT policy for the IP addresses. Upstream NAT is not supported on Layer 2 interfaces. STEP 9 | (PAN-OS 10.0.3 and later 10.1 releases) If you are adding a branch that is behind a device performing NAT for the branch, you must specify the IP address or FQDN of the public-facing interface on that upstream NAT-performing device, or select DDNS to indicate that the IP address for the interface on the NAT device is obtained from the Palo Alto Networks DDNS service. Thus, Auto VPN Configuration uses that public IP address as the tunnel endpoint for the branch. It is the IP address that the branch office’s IKE and IPSec flows must be able to reach. (You must have already configured a physical Ethernet interface for SD-WAN.) 1. On the Upstream NAT tab, enable Upstream NAT. 2. Add an SD-WAN interface; select an interface you already configured for SD-WAN. 3. If you select the NAT IP Address Type to be Static IP, select IP Address or FQDN and enter the IPv4 address without a subnet mask (for example, 192.168.3.4) or the FQDN of the upstream device, respectively. 4. Alternatively, select the NAT IP Address Type to be DDNS. 5. Click OK. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 77 2021 Palo Alto Networks, Inc. Additionally, on the upstream device that is performing NAT you must set up the inbound Destination NAT with a one-to-one NAT policy, and you must not configure port translation to the IKE or IPSec traffic flows. If the IP address on the upstream device changes, you must configure the new IP address and push it to the VPN cluster. You must use the CLI commands clear ipsec, clear ike-sa, and clear session all on both the branch and hub. You must also clear session all on the virtual router where you configured the NAT policy for the IP addresses. There is a second location in the UI where you can configure Upstream NAT for a branch, but the following location is not preferred and you should not configure Upstream NAT for a branch in both places. The secondary, non-preferred location to configure Upstream NAT is on Panorama at Network > Interfaces > Ethernet, select a template in the Template field, select an Ethernet interface, and select the SDWAN tab. At this point you can Enable Upstream NAT, and select a NAT IP Address Type. This second method takes precedence. If Upstream NAT is first configured for the Ethernet interface on Panorama through the template stack, then the SD-WAN plugin will not change the settings, even if you use different settings on the plugin device configuration page. Only if there is no Upstream NAT configured on Panorama through the template stack, then the plugin configuration for Upstream NAT takes effect. Upstream NAT is not supported on Layer 2 interfaces. STEP 10 | (Required for pre-existing customers) Map your pre-existing zones to predefined zones used for SD-WAN. When you map your existing zones to an SD-WAN zone, you must modify your security policy rules and add the SD-WAN zones to the correct Source and Destination zones. 1. 2. 3. 4. Select Zone Internet and Add the pre-existing zones that will egress SD-WAN traffic to the internet. Select Zone to Hub and Add the pre-existing zones that will egress SD-WAN traffic to the hub. Select Zone to Branch and Add the pre-existing zones that will egress SD-WAN traffic to the branch. Select Zone Internal and Add the pre-existing zones that will egress SD-WAN traffic to an internal zone. STEP 11 | (Optional) Configure Border Gateway Protocol (BGP) routing. To automatically set up BGP routing between the VPN cluster members, enter the BGP information below. If you want to manually configure BGP routing on each firewall or use a separate Panorama template to configure BGP routing for more control, leave the BGP information below blank. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. 1. Select the BGP tab and enable BGP to configure BGP routing for SD-WAN traffic. 2. Enter the BGP Router ID, which must be unique among all routers. 3. Specify a static IPv4 Loopback Address for BGP peering. Auto VPN configuration automatically creates a Loopback interface with the same IPv4 address that you specify. If you specify an existing 78 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. loopback address, the commit will fail, so you should specify an IPv4 address that is not already a loopback address. 4. Enter the AS Number. The autonomous system number specifies a commonly defined routing policy to the internet. The AS number must be unique for every hub and branch location. 5. Disable the Remove Private AS option (the default is enabled) if you have endpoints that need to exchange routes with a hub or branch firewall in an SD-WAN BGP topology and therefore you don’t want to remove private AS numbers (64512 to 65534) from the AS_PATH attribute in BGP Updates. In this case, you want to allow the private AS numbers to leave the SD-WAN private AS in BGP Updates. The Remove Private AS setting applies to all BGP peer groups on the branch or hub firewall. If you need this setting to differ among BGP peer groups or peers, you must configure the setting outside of the SD-WAN plugin. If you change the Remove Private AS setting, commit to all SD-WAN cluster nodes, and subsequently downgrade to an SD-WAN Plugin version earlier than 2.0.2, then all configuration related to Remove Private AS must be done outside of the SD-WAN plugin or directly on the firewalls. 6. Enter Prefix(es) to Redistribute. On a hub device, you must enter at least one prefix to redistribute. Branch devices do not have this option; subnets connected to branch locations are redistributed by default. STEP 12 | Click OK. STEP 13 | Select Group HA Peers at the bottom of the screen to display branches (or hubs) that are HA peers together. STEP 14 | Have Panorama create and push to firewalls a Security policy rule that allows BGP to run between branches and hubs. 1. Select BGP Policy at the bottom of the screen and Add. 2. Enter a Policy Name for the Security policy rule that Panorama will automatically create. 3. Select Device Groups to specify the device groups to which Panorama pushes the Security policy rule. 4. Click OK. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 79 2021 Palo Alto Networks, Inc. STEP 15 | Select Push to Devices to push your configuration changes to your managed firewalls. Bulk Import Multiple SD-WAN Devices Add multiple SD-WAN devices to quickly onboard branch and hub firewalls, rather than manually adding each device one at a time. When adding your devices, you specify what type of device it is (branch or hub) and you give each device its site name for easy identification. Before adding your devices, plan your SDWAN configuration to ensure you have all the required IP addresses and that the SD-WAN topology is well understood. This helps reduce any configuration errors. If you want to have Active/Passive HA running on two branch firewalls or two hub firewalls, do not add those firewalls as SD-WAN devices in your CSV file. You will add them as HA peers separately when you Configure HA Devices for SD-WAN. If you are using BGP routing, you must add a security policy rule to allow BGP from the internal zone to the hub zone and from the hub zone to the internal zone. If you want to use 4-byte autonomous system numbers (ASNs), you must first enable 4-byte ASNs for the virtual router. If you have pre-existing zones for your Palo Alto Networks firewalls, you will be mapping them to the predefined zones used in SD-WAN. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Devices > Device CSV and Export an empty SD-WAN device CSV. The CSV allows you to import multiple branch and hub devices at once, rather than adding each device manually. 80 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 3 | Populate the SD-WAN device CSV with the branch and hub information and save the CSV. All fields are required unless noted otherwise. You must enter the following for each hub and branch: • device-serial—The serial number of the branch or hub firewall. • type—Specify whether the device is a branch or a hub. • site—Enter the SD-WAN device site name to help you identify the geographical location or purpose of the device. The SD-WAN Site name supports all upper-case and lower-case alphanumerical and special characters. Spaces are not supported in the Site name and result in monitoring (Panorama > SD-WAN > Monitoring) data for that site not to be displayed. • (Required for pre-existing customers) Map your pre-existing zones to predefined zones used for SDWAN. When you map your existing zones to an SD-WAN zone, you must modify your security policy rules and add the SD-WAN zones to the correct Source and Destination zones. • zone-internet—Enter the names of pre-existing zones that SD-WAN traffic will egress to reach the internet. • zone-to-branch —Enter the names of pre-existing zones that SD-WAN traffic will egress to reach a branch. • zone-to-hub—Enter the names of pre-existing zones that SD-WAN traffic will egress to reach a hub. • zone-internal—Enter the names of pre-existing zones that SD-WAN traffic will egress to reach an internal zone. • (Optional) loopback-address—Specify a static loopback IPv4 address for Border Gateway Protocol (BGP) peering. • (Optional) prefix-redistribute—Enter IP prefixes that the branch informs the hub it can reach. To add more than one prefix, separate prefixes with a space, an ampersand (&), and a space; for example, 192.2.10.0/24 & 192.168.40.0/24. By default, the branch firewall advertises all locally connected internet prefixes to the hub. Palo Alto Networks does not redistribute the branch office default route(s) learned from the ISP. • (Optional) as-number—Enter the ASN of the private AS to which the virtual router on the hub or branch belongs. The SD-WAN plugin supports only private autonomous systems. The ASN must be unique for every hub and branch. The 4-byte ASN range is 4,200,000,000 to 4,294,967,294 or 64512.64512 to 65535.65534. The 2-byte ASN range is 64512 to 65534. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 81 2021 Palo Alto Networks, Inc. Use a 4-byte private ASN. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. • (Optional) router-id—Specify the BGP router ID, which must be unique among all virtual routers. Enter the Loopback Address as the router ID. Before implementing SD-WAN with BGP routing in an environment where BGP is already in use, ensure that the BGP configuration generated by the SD-WAN plugin doesn’t conflict with your existing BGP configuration. For example, you must use the existing BGP AS number and router ID values for the corresponding SD-WAN device values. • vr-name—Enter the name of the virtual router to use for routing between the SD-WAN hub and branches. By default, Panorama creates an sdwan-default virtual router and can automatically push router configurations. STEP 4 | Import the SD-WAN device CSV into Panorama. Verify that there are no pending commits on Panorama or the import fails. 1. On Panorama, Select Panorama > SD-WAN > Devices > Device CSV and Import the CSV you edited in the previous step. 2. Browse and select the SD-WAN device CSV. 3. Click OK to import the SD-WAN devices. STEP 5 | Verify that your SD-WAN devices were successfully added. STEP 6 | Commit your configuration changes. STEP 7 | Select Push to Devices to push your configuration changes to your managed firewalls. 82 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. Configure HA Devices for SD-WAN You can configure two branches or two hubs in active/passive HA mode to be part of your SD-WAN environment. In this case, Panorama™ needs to push the same configuration to the active peer and the passive peer, rather than treat the two firewalls individually. To make that happen, you configure active/ passive HA before adding the devices for SD-WAN, so that Panorama is aware the devices are HA peers and pushes the same configuration to them. Read through the following procedure before you begin so you don’t Commit after adding your HA peers as SD-WAN devices. STEP 1 | Before you enable SD-WAN on your HA peers, configure Active/Passive HA on two firewall models that support SD-WAN. STEP 2 | Add the HA peers as SD-WAN devices, but don’t perform the last step to Commit. STEP 3 | In Panorama, select Panorama > Managed Devices > Summary. STEP 4 | At the bottom of the screen, select Group HA Peers. Confirm that under the Status display, the HA Status column includes the two firewalls, one Active and one Passive. Panorama is aware of the HA status and will push the same SD-WAN configuration to the two HA peers when you commit. STEP 5 | Commit and Commit and Push. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 83 2021 Palo Alto Networks, Inc. Create a VPN Cluster In your SD-WAN configuration, you must configure one or more VPN clusters to determine which branches communicate with which hubs and creates a secure connection between the branch and hub devices. VPN clusters are logical groupings of devices, so consider things such as geographical location or function when logically grouping your devices. ® PAN-OS supports both hub-spoke and full mesh SD-WAN VPN topologies. In a hub-spoke topology, a centralized firewall hub at a primary office or location acts as the gateway between branch devices. The hub-to-branch connection is a VPN tunnel. In this configuration, traffic between branches must pass through the hub. The first time you Configure a Virtual SD-WAN Interface with direct internet access (DIA) links for an SD-WAN hub or branch firewall, a VPN cluster called autogen_hubs_cluster is automatically created and the SD-WAN firewall is automatically added to the VPN cluster. This allows the Panorama™ management server to Monitor SD-WAN Application and Link Performance for devices that are protected by the SD-WAN firewall and accessing resources outside of your corporate network. Additionally, any SD-WAN firewall with DIA links that you configure in the future are automatically added to the autogen_hubs_cluster VPN cluster containing all hubs and branches with DIA links to allow Panorama to monitor application and link performance. The autogen_hubs_cluster is purely for monitoring application and link health, and not to create VPN tunnels between the hubs and branches with DIA links. If you need to connect hubs and branches with VPN tunnels, you must create a new VPN cluster and add all the required hubs and branches to that cluster. A strong, random IKE preshared key is created for all hubs and branches in the VPN cluster to secure the VPN tunnels, and each firewall has a master key that encrypts the preshared key. The system secures the preshared key, even from the administrator. You can refresh the IKE preshared key, which Panorama sends to all members of the cluster. Refresh the preshared key when cluster members are not busy. After the SD-WAN Plugin is upgraded to 2.1.0, the hub and branch firewalls in a single VPN cluster must all run either PAN-OS 10.0.x or 10.1.0, not a combination of the two releases. STEP 1 | Plan your branch and hub VPN topology to determine which branches communicate with each of your hubs. For more information, see Plan Your SD-WAN Configuration. STEP 2 | Log in to the Panorama Web Interface. STEP 3 | Specify IP address ranges for the IPSec VPN tunnels that Auto VPN configuration creates. Auto VPN configuration creates a VPN tunnel between a hub and branches and assigns IP addresses to the tunnel endpoints. Enter subnet ranges that you want Auto VPN to use as VPN tunnel addresses.You can enter up to 20 IP prefix/netmask ranges. Auto VPN draws from that pool for VPN tunnel addresses, drawing from the largest range first (and the drawing from the next largest range when necessary). You must configure at least one range for the pool. If you don’t perform this step before pushing the configuration to a hub or branch, the Commit and Push will fail. 84 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. If you upgrade from an earlier SD-WAN Plugin release, you must check that your ranges are still correct. If it is not, enter new ranges. After you Commit, all tunnels are dropped and new tunnels are used, so perform this task during a time you have low traffic. 1. Select Panorama > SD-WAN > VPN Clusters. 2. At the bottom of the screen, select VPN Address Pool. 3. Add one or more (up to 20) Member IP address and netmask ranges, for example, 192.168.0.0/16. 4. Click OK. STEP 4 | Configure the VPN cluster. Repeat this step to create VPN clusters as needed. 1. Select Panorama > SD-WAN > VPN Clusters and Add a VPN cluster. 2. Enter a descriptive name for the VPN cluster. Underscores and spaces are not supported in the VPN cluster name and result in monitoring (Panorama > SD-WAN > Monitoring) data for the cluster not to be displayed. Choose the name of the VPN cluster carefully so you do not need to change the name in the future. SD-WAN monitoring data is generated based on the old cluster name and cannot be reconciled to a new cluster name, and will cause issues with the number of reported clusters when monitoring your VPN clusters or generating reports. 3. Select the VPN cluster Type. Only Hub-Spoke VPN cluster type is supported in PAN-OS 10.0.2 and earlier 10.1 releases. Beginning with PAN-OS 10.0.3, you can Create a Full Mesh VPN Cluster with DDNS Service. 4. Add one or more branch devices that you determined need to communicate with each other. • Select Group HA Peers to display the branch devices that are HA peers together. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 85 2021 Palo Alto Networks, Inc. • Select the branch devices to add to the cluster. • Click OK. 5. Add one or more hub devices that you determined need to communicate with the branch devices. Up to four SD-WAN hub firewalls can be added to a VPN cluster. SD-WAN hubs in an HA configuration are considered as a single SD-WAN hub firewall. MPLS and satellite link types will form tunnels with only the same link type; for example, MPLS-to-MPLS and satellite-to-satellite. Tunnels will not be created between an MPLS link and an Ethernet link, for example. • Select Group HA Peers to display the hub devices that are HA peers together. • Select the hubs to add to the cluster and click OK. • For any new or previously existing VPN cluster that has more than one hub, you must prioritize the hubs to determine a) that traffic be sent to a particular hub, and b) the subsequent hub failover order. The hub failover priority range is 1 to 4. If you upgrade, the default priority is set to 4. The plugin internally translates the hub failover priority to a BGP local preference number as shown in the following table. The lower the priority value, the higher the priority and local preference. A cluster supports a maximum of four hubs. An active/passive HA pair counts as one 86 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. hub. Multiple hubs can have the same priority; an HA pair must have the same priority. Panorama uses the branch’s BGP template to push the local preference of the hubs to the branches in the cluster. Hub Failover Priority Local Preference 1 250 2 200 3 150 4 100 If multiple hubs have the same priority, Panorama enables ECMP in two places on each branch firewall to determine how branches select the path. ECMP is enabled for the virtual router (Network > Virtual Routers > ECMP) and ECMP Multiple AS Support is enabled for BGP (Network > Virtual Routers > BGP > Advanced).If all hubs in the cluster have a unique priority, ECMP is disabled on the branches. If a hub priority configuration changes, Panorama reevaluates whether to enable or disable ECMP. • If you selected Group HA Peers, select the pair and click in the Hub Failover Priority field; enter a single Priority (range is 1 to 4), which applies to both hubs in the HA pair, and click OK. The Hub Failover Priority for HA Peers window appears only for configured HA pairs. If you add a new HA pair, you must configure the Hub Failover Priority for each of the two new peers independently. You will get an error message if you assign different priorities to hubs that are ungrouped HA peers and then you select Group HA Peers and Submit. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 87 2021 Palo Alto Networks, Inc. • For hubs that are not HA pairs, select a hub and click in the Hub Failover Priority field; enter a priority (range is 1 to 4). 6. Click OK to save the VPN cluster. STEP 5 | Advertise additional prefixes at the branch to the hub. The firewall automatically redistributes (advertises) all non-public, connected routes from the branch to the hub. You can also redistribute any additional prefixes from the branch to the hub. The Prefix(es) to Redistribute field accepts a list of prefixes, rather than just a single prefix. 1. Select Panorama > SD-WAN > Devices and select a branch firewall. 2. Select BGP and Add one or more IP addresses with netmask to Prefix(es) to Redistribute. 3. Click OK. STEP 6 | Commit and Commit to Panorama. STEP 7 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) If your hub firewalls in a hub-spoke VPN cluster have DHCP or PPPoE interfaces, you must use DDNS. Select Network > Interfaces > Ethernet and in the Template field, select Template-stack for a hub. STEP 8 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Select the interfaces whose IP address indicates Dynamic-DHCP Client or PPPOE, click Override on the bottom of the screen, and click OK to close. STEP 9 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Verify on Panorama that the DDNS settings were configured. 1. Select Network > Interfaces > Ethernet and select the same interface again. 2. Select Advanced > DDNS. 3. See that the DDNS settings were automatically configured with a Hostname and the Vendor set to Palo Alto Networks DDNS. 4. Click OK. STEP 10 | (SD-WAN Plugin 2.0.1 and later 2.0 releases) Commit and Commit to Panorama. STEP 11 | Push the configuration to the hub(s). 88 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. When Panorama creates virtual SD-WAN interfaces for hubs, Panorama doesn't necessarily create the interfaces using contiguous interface numbers. It might randomly skip an interface number, for example, sdwan.921, sdwan.922, sdwan.924, sdwan.925. Despite the discontiguous numbering, Panorama creates the correct number of SD-WAN interfaces. Use the operational CLI command show interface sdwan? to see the SDWAN interfaces. 1. Select Commit and Push to Devices. 2. Edit Selections on the lower left side of the screen. 3. Deselect Filter Selected. 4. Click on Deselect All. 5. Select your hub Device Group. Select Include Device and Network Templates at the bottom of the screen. You must push to hubs before pushing to branches. Most branches have dynamic IP addresses through their service providers, so branches must initiate the IKE/IPSec connection because the hub doesn’t have the branches’ IP address.To ensure that the hub is ready to receive the IKE/IPSec connections, the hub’s configuration must be committed and pushed before the branch’s configuration. Thus, when the branch configurations are pushed and the branches initiate the connection to the hub, the hub is ready. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 89 2021 Palo Alto Networks, Inc. 6. Select the Templates tab and Deselect All. 7. The Push Scope is the Device Group. Push the configuration to the hub(s). STEP 12 | Push the configuration to the branch(es) by repeating the prior step, but selecting your branch Device Group. STEP 13 | Refresh the IKE preshared key. If you need to change the current IKE key that is used to secure the IPSec connections between VPN cluster devices, perform this step to randomly generate a new key for the cluster. Perform this step when cluster members are not busy. 1. Select Panorama > SD-WAN > VPN Clusters and select a cluster. 2. At the bottom of the screen, select Refresh IKE Key. 90 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 3. A message appears notifying you that Refreshing the IKE key will generate a new security association (SA) for every SD-WAN firewall in the VPN cluster. This may cause a service disruption. Do you wish to continue? Yes | No Select Yes if you wish to continue. 4. Commit. After you Refresh IKE Key, you must commit to the entire cluster; a partial commit will bring down tunnels. 5. Push to Devices. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 91 2021 Palo Alto Networks, Inc. Create a Full Mesh VPN Cluster with DDNS Service Beginning with PAN-OS 10.0.3, SD-WAN supports a full mesh topology, in addition to the hub-spoke topology. The mesh can consist of branches with or without hubs. Use full mesh when the branches need to communicate with each other directly. Examples of use cases for full mesh include retailers that have branches and hubs, and enterprises that operate with or without hubs. Some firewall interfaces use DHCP to get their IP address. Branch offices often use a consumer-grade internet service and receive a dynamic IP address, which of course can change. For this reason, the firewalls require Dynamic DNS (DDNS) so that a DDNS service can detect the public-facing IP address of the firewall interface that is running SD-WAN. When you push the DDNS setting to all firewalls, that notifies each firewall to register its external interface IP address with the Palo Alto Networks DDNS cloud service so that the IP address is converted to an FQDN. DDNS is also required because the CPE device from the ISP may be performing source NAT. (The dynamic IP address may or may not be source-NAT translated). The DDNS service allows the firewall to register the public-facing IP address with the DDNS server. When you have devices connect for branch-to-branch mesh, Auto VPN contacts the DDNS service for those firewalls to pull their public IP addresses that are registered in the DDNS cloud and uses those public IP addresses to create the IKE peering and the VPN tunnels. If the CPE device is performing source NAT, when you add an SD-WAN branch device to be managed by Panorama, you will enable Upstream NAT and the NAT IP Address Type will be DDNS. For the CPE device or upstream routing device using source NAT, you are responsible for creating the one-to-one destination NAT rule (with no port translation) on that device to translate the external IP address back to the private IP address assigned to the firewall’s interface. This translation allows the IKE and IPSec protocols to come back into the firewall. (Palo Alto Networks doesn’t have access rights to the upstream CPE or upstream router that is performing source NAT.) SD-WAN full mesh with DDNS service requires the following: • PAN-OS 10.0.3 or a later 10.1 release • SD-WAN Plugin 2.0.1 or a later 2.0 release • ZTP Plugin 1.0.1 or a later 1.0 release that is downloaded, installed, and configured in order to leverage the DDNS that is associated with ZTP. Panorama must be ZTP-registered and communicating with the ZTP Service. • Applications and Threats Content Release Version 8354 or a later version • All firewalls participating in full mesh DDNS must be registered under the same Customer Support Portal (CSP) account. • All firewalls participating in full mesh DDNS must have the latest device certificate installed. Properly authenticating the firewalls, Panorama, and the cloud services are important security procedures that require the device certificate, and the CSP and ZTP services. • If you have a firewall or other network device that controls outgoing traffic positioned in front of the Palo Alto Networks firewall, you must change the configuration on that device to allow traffic from the DDNS-enabled interfaces to the following FQDNs: • https://myip.ngfw-ztp.paloaltonetworks.com/ • https://ngfw-ztp.paloaltonetworks.com/ (to reach whatsmyIP service) (to reach DDNS registration service) 92 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 1 | Install the latest device certificate for Panorama and for all managed firewalls that are hubs or branches. STEP 2 | Install ZTP Plugin 1.0.1 to set up Zero Touch Provisioning. 1. 2. 3. 4. In the Panorama Administrator’s Guide, read the ZTP Overview. Install the ZTP Plugin. Configure the ZTP Installer Administrator Account. Select Panorama > Zero Touch Provisioning > Setup and edit the General settings to enable Dynamic IP Registration. 5. Click OK. The General settings indicate On ZTP Service with a Tenant ID number. 6. Select ZTP Service Status and confirm that the firewall Serial Number is listed. STEP 3 | If you haven’t already done so, install the SD-WAN Plugin 2.0.1 or a later 2.0 release. STEP 4 | Commit on Panorama. STEP 5 | Log in to the Panorama Web Interface. STEP 6 | Create the VPN Address Pool as shown in Create a VPN Cluster. STEP 7 | Create the full mesh VPN cluster. 1. 2. 3. 4. 5. Select Panorama > SD-WAN > VPN Clusters. Select the Type to be Mesh. Add the Branches that need to communicate with each other. (Optional) Add one or more Hubs if you also want a hub in the mesh. Click OK. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 93 2021 Palo Alto Networks, Inc. STEP 8 | Commit and Commit to Panorama. If your firewalls have static IP addresses, you are done. If your branch or hub firewalls in a VPN mesh have DHCP or PPPoE interfaces, you must use DDNS, so continue this procedure as follows. STEP 9 | Select Network > Interfaces > Ethernet and in the Template field, select Template-stack for a particular branch. STEP 10 | Select the interface whose IP address indicates Dynamic-DHCP Client or PPPOE, click Override on the bottom of the screen, and click OK to close. STEP 11 | Verify on Panorama that the DDNS settings were configured. 1. Select Network > Interfaces > Ethernet and select the same interface again. 2. Select Advanced > DDNS. 3. See that the DDNS settings were automatically configured with a Hostname based on the interface name, and the Vendor set to Palo Alto Networks DDNS. For example, on the Ethernet1/2 interface, the resulting Hostname is 0102. 4. Click OK. STEP 12 | If the VPN cluster includes any hubs that have a DHCP or PPPoE interface, repeat Steps 9 through 11, but in the Template field, select Template-stack for a particular hub. Even if your hub is not in a full mesh cluster, but is in a hub-spoke cluster, if the hub uses DHCP or PPPOE to get its IP address for an SD-WAN interface, you must perform the Override steps to enable DDNS. STEP 13 | Commit to Panorama and Push to Devices. STEP 14 | Verify on the branch firewall that the branch is configured with DDNS. 94 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. 1. Log into the branch firewall. 2. Select Network > Interfaces > Ethernet and for the Ethernet interface that you configured, scroll over the DDNS Info icon in the Features column to see the Vendor, Hostname, IP address, and other DDNS information. STEP 15 | On another branch in the cluster, view that the Peer Address of the interface is a system- generated FQDN for the DDNS registration. 1. Log onto another branch and select Network > Network Profiles > IKE Gateways. 2. See that the Peer Address is a secure name, not easily referenced and showing no company information; for example 0101.8ced8460fcc5177cd3665ce41b6345323a15a612b8e52ec1d9ec057a582cb4.t13855f6c9a92d6[...]e18a0d9 STEP 16 | View FQDNs of branches and hubs and update DDNS information. 1. Access the CLI. 2. View FQDNs (generated by DDNS) for other branches and hubs: show dns-proxy fqdn all 3. Update the DDNS addresses: request system fqdn refresh SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 95 2021 Palo Alto Networks, Inc. Create a Static Route for SD-WAN In addition to (or as an alternative to) BGP routing, you can create static routes to route your SD-WAN traffic. You can configure static routes either using Panorama™ or directly on the firewall hub or branch. If you are going to use Panorama, you should be familiar with the process to Configure a Template or Template Stack Variable. You will create a variable to use as the destination in your static route, as shown in the following procedure. You will push a static route (that goes to the hub) to the branch. You will push a static route (that goes to the branch) to the hub. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Configure a Template or Template Stack Variable and enter the variable Name in the following format: $peerhostname_clustername.customname. For example, $branchsanjose_clusterca.10 or $DIA_cluster2.location3. After the dollar sign ($), the elements in the variable are: • peerhostname—Hostname of the destination hub or branch to which the static route goes. For a static route to the internet, the peerhostname must be DIA. An alternative to the peer’s hostname is to use the peer’s serial number. If the peer is part of an HA pair, you can use the hostname or serial number of either one of the two HA firewalls. • clustername—Name of the VPN cluster to which the destination hub or branch belongs. • customname—Text string of your choice; you cannot use a period (.) in the customname. You can have more than one static route going to the same peer, which means the variables will have the same peerhostname and clustername; you differentiate the variables by using a different customname. STEP 3 | Select the variable Type to be IP Netmask and enter the destination IP address with a slash and netmask length, such as 192.168.2.1/24. STEP 4 | Click OK to save the variable. STEP 5 | Select Network > Virtual Routers and select a virtual router. STEP 6 | Select Static Routes > IPv4 and Add a Name for the static route. STEP 7 | For Destination, select the variable you created. STEP 8 | For Interface, select from the dropdown list, which includes only interfaces from the template; for example, Ethernet1/1, Tunnel.x, or sdwan.xx. STEP 9 | For Next Hop, select IP Address and enter the IP address of the next hop for the static route (the hub or branch to which the static route goes). STEP 10 | Click OK. STEP 11 | Commit and Commit and Push your changes. Auto VPN configuration replaces the sdwan keyword in the Interface field of the static route with the egress virtual SD-WAN interface that it determines based on the Destination variable. Thus, the static route in the routing table indicates that traffic going to the peer host in the identified VPN cluster will egress the virtual SD-WAN interface to reach the specified next hop. 96 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 2021 Palo Alto Networks, Inc. STEP 12 | Configure a static route for the return traffic. SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN © 97 2021 Palo Alto Networks, Inc. 98 SD-WAN ADMINISTRATOR’S GUIDE | Configure SD-WAN Monitoring and Reporting Monitor and generate reports of the application and link health status in your VPN clusters to identify and resolve issues. In order for the Panorama™ management server to display SDWAN application and link health information, you must enable the SD-WAN firewalls to push device monitoring data to Panorama and configure log forwarding to Panorama when you Add Your SD-WAN Firewalls as Managed Devices. If you have not configured your SD-WAN firewalls to forward logs to Panorama, the SD-WAN Monitoring displays no application or link health information. In order for Panorama to gather SD-WAN monitoring data, you must push the SD-WAN configuration from Panorama to your SD-WAN firewalls. If no SDWAN monitoring data is displayed, verify that you successfully pushed the SDWAN configuration. > Monitor SD-WAN Tasks > Monitor SD-WAN Application and Link Performance > Generate an SD-WAN Report 99 100 SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 2021 Palo Alto Networks, Inc. Monitor SD-WAN Tasks Monitor commits, pushes, and other SD-WAN tasks executed from the Panorama™ management server to gain insight and detailed information regarding the specific task. If a task succeeds with warnings or fails, you can view detailed warnings and description to better understand how to resolve the misconfiguration. Additionally, you can view the last push state details to review detailed information as to what caused the task warnings or errors. STEP 1 | Log in to the Panorama web interface. STEP 2 | After editing the SD-WAN configuration, Commit your changes to view the job status. The job status window displays the operation performed, the result, and any details and warnings related to the job status. STEP 3 | View the last push details for jobs that succeed with warnings or failed. 1. Click Tasks ( ) at the bottom of the web interface to open the Task Manager. 2. Click the job Type for the SD-WAN task. 3. Click the job Status to view the last push state details for the task. 4. Review the last push state details to identify and resolve the configuration issues. SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 101 2021 Palo Alto Networks, Inc. 102 SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 2021 Palo Alto Networks, Inc. Monitor SD-WAN Application and Link Performance Monitor the application and link performance in your VPN clusters to troubleshoot issues by viewing summary information across all VPN clusters and then successively drilling down to isolate the issues to affected sites, applications, and links. Visibility on SD-WAN traffic is shown on the SD-WAN firewall receiving the traffic. For example, for traffic from the hub firewall to the branch firewall, the SD-WAN monitoring data is reflected on the branch firewall. The landing dashboard displays: • App Performance • Impacted—One or more applications in the VPN cluster for which none of the paths have jitter, latency, or packet loss performance that meet the specified thresholds in the Path Quality Profile in the list of paths from which the firewall can choose. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. • Link Performance • Error—One or more sites in the VPN cluster have connectivity issues such as when a tunnel or a virtual interface (VIF) is down. • Warning—Number of VPN clusters, hubs, and branches that have links with jitter, latency, or packet loss performance measurements that exceed the moving seven-day average value of the metric. • OK—Number of VPN clusters, hubs, and branches that are experiencing no jitter, latency, or packet loss performance issues. If a hub or branch firewall have an SD-WAN policy rule configured with Forward Error Correction, an Error Correction Initiated message is displayed to notify you that the hub or branch firewall detected and corrected errors in transmitted data for an application. SD-WAN hubs display Error Correction Initiated only if traffic originated from the SD-WAN hub to the SD-WAN branch and matched an SD-WAN policy rule with an error correction profile attached. From the landing dashboard, narrow the view to impacted applications or links that have the Error or Warning status. Then select an affected site to view site-level details. From the site, view application-level or link-level details. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring to view at-a-glance health status summaries of your VPN clusters, hubs, and branches. SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 103 2021 Palo Alto Networks, Inc. STEP 3 | Click an App Performance or Link Performance summary that indicates Impacted, Error, or Warning counts to view a detailed list of sites and their status based on latency, jitter, and packet loss. STEP 4 | Click a site that displays Warning or Error to see one VPN cluster. The site data display App Performance and Link Performance, including the impacted applications. Additionally, use the Sites filter to view VPN clusters based on link notifications, latency deviations, jitter deviations, packet loss deviations, or impacted applications. For SaaS applications over a Direct Internet Access (DIA) link, the SaaS Monitoring column indicates whether the app is created in a SaaS Quality profile and associated with one or more SD-WAN policy rules. 104 SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 2021 Palo Alto Networks, Inc. • Disabled—The app is not a SaaS application configured in a SaaS Quality profile. • Enabled—The app is a SaaS application configured in a SaaS Quality profile and is associated with a single SD-WAN policy. • Multiple—The app is a SaaS application configured in a SaaS Quality profile and is associated with a multiple SD-WAN policies. If you associated an error correction profile with an SD-WAN policy rule for an application, the Error Correction Applied columns displays if and what type of error correction was applied. Additionally, you can view the Error Corrected Sessions/Impacted Sessions/Total Sessions to understand how many sessions were error corrected by the branch or hub firewall out of the total number of sessions for the specified timeframe. Click PDF/CSV to export the detailed health information for the applications and links in the Site in PDF or CSV format STEP 5 | Click the branch or hub that has an application that needs attention. STEP 6 | Click an impacted application to view application-level or link-level details. For example, view the link characteristics for an application to understand the latency, jitter and packet loss for the application over the specified link. Additionally, you can view when error correction was applied for the link. SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 105 2021 Palo Alto Networks, Inc. 106 SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 2021 Palo Alto Networks, Inc. Generate an SD-WAN Report Configure and generate an SD-WAN report detailing the top applications or links with the highest frequency of path quality degradation. The order application or links appear in a report is based on the amount of data impacted; the more data is impacted, the higher the application or link appears in the report. SD-WAN reports are generated as needed and cannot be scheduled. Use the SD-WAN reports to verify the correct application or link throughput, or ensure that application or link impact is not noticed by users. For example, if your ISP guaranteed a certain amount of throughput on a link, generate a Link Performance report for that link to verify that the guaranteed bandwidth is honored. From the Panorama™ management server, you can only generate reports for applications or links across all SD-WAN enabled firewalls. To generate a report for applications or links processed by an individual firewall, you must create and generate the report locally on the firewall. STEP 1 | Log in to the Panorama web interface. STEP 2 | Select Panorama > SD-WAN > Reports and Add a new report. STEP 3 | Configure the SD-WAN report parameters. 1. Enter a descriptive Name for the report. 2. Choose the Report Type to generate: • Select App Performance to generate a report detailing only application health performance. • Select Link Performance to generate a report detailing only the link health performance. 3. Select the VPN Cluster for which to generate the report. By default, all is selected. 4. Select a Site within the selected VPN cluster for which to generate the report. By default, all is selected. If you selected all Clusters then this field is grayed out and a Site cannot be selected. 5. (App Performance only) Select the Application for which to generate the report. 6. 7. 8. 9. If you selected all Clusters and Sites then this field is grayed out and an individual application cannot be selected. (Link Performance only) Select the Link Tag for which to generate the report. Selecting a link tag generates a report for all links in grouped using the tag in the cluster or site. By default, all is selected. (Link Performance only) Select the Link Type for which to generate the report. Selecting a link type generates a report for all links of the specified type in the cluster or site. By default, all is selected. Select the Top N applications or links to include in the report. This setting determines the number of applications or links experiencing health degradation to include in the report. By default, the report includes the top 5 applications or links experiencing health degradation. Specify the Time Period within which to generate the report. By default, None is selected and queries the entire health status history of the applications or links. STEP 4 | Click Run Now to generate the report. SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting © 107 2021 Palo Alto Networks, Inc. STEP 5 | View the generated report and Export XML to export the report in XML format to your local device. When ready, click Close. STEP 6 | In the Reports pop-up, click OK to save your configured report. STEP 7 | Commit > Commit to Panorama and Commit your changes. 108 SD-WAN ADMINISTRATOR’S GUIDE | Monitoring and Reporting Troubleshooting Use the Panorama™ management server Command Line Interface (CLI) to view SD-WAN information and perform operations. > > > > > > Use CLI Commands for SD-WAN Tasks Troubleshoot App Performance Troubleshoot Link Performance Upgrade your SD-WAN Firewalls Upgrade the SD-WAN Plugin Uninstall the SD-WAN Plugin 109 110 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. Use CLI Commands for SD-WAN Tasks Use the following CLI commands to view and clear SD-WAN information and view SD-WAN global counters. You can also view VPN tunnel information, BGP information, and SD-WAN interface information. If you want to ... Use ... View or Clear SD-WAN Information • View path names and IDs for an SD-WAN interface, their state, local and peer IP addresses, and tunnel interface number. > show sdwan connection all | <sdwaninterface> • View the number and percentage of sessions distributed to each tunnel member of a virtual SD-WAN interface. > show sdwan session distribution policy-name <sdwan-policy-name> • View the names of SD-WAN policy rules that send traffic to the specified virtual SD-WAN interface, along with the traffic distribution method, configured latency, jitter, and packet loss thresholds, link tags identified for the rule, and member tunnel interfaces. • View SD-WAN events such as path selection and path quality measurements. > show sdwan rule vif sdwan.x > show sdwan event For PAN-0S 10.0.0 and 10.0.1, when you make an SD-WAN configuration change (such as a Path Quality profile change) that results in a different SDWAN path being selected, the traffic log does not count or log the path change. • Clear SD-WAN events. > clear sdwan event • View latency, jitter, and packet loss on a virtual SD-WAN interface (specify interface number or name). Latency, jitter, and packet loss measurements are taken and averaged over three timeframes. Each timeframe has a health version, which increments when a health parameter value (that exceeds the threshold) changes. In addition to the real time measurement, there is a current > show sdwan path-monitor stats vif <sdwan.x> > show sdwan path-monitor stats vif <sdwan-interface-name> SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 111 2021 Palo Alto Networks, Inc. If you want to ... Use ... use measurement, which displays the value of the parameter the last time the real-time value change exceeded the threshold. • View the name of the SD-WAN policy rule that the specified session matches, the source and destination tunnel interfaces, the configured latency, jitter, and packet loss percentage for the rule, and the traffic distribution method. > show sdwan session path-select session-id <session-id> For PAN-0S 10.0.0 and 10.0.1, when you make an SD-WAN configuration change (such as a Path Quality profile change) that results in a different SDWAN path being selected, the traffic log does not count or log the path change. • View monitoring mode for the virtual SDWAN link (Aggressive or Relaxed) and update intervals. > show sdwan path-monitor parameter path-name <sdwan-path-name> • View monitoring mode for the virtual SDWAN interface (Aggressive or Relaxed), update intervals, and probe statistics. > show sdwan path-monitor parameter vif <sdwan.x> View Global Counters to Troubleshoot SD-WAN • On a branch, verify that the number of SDWAN probe Request packets transmitted equals the number of probe Reply packets received. On a branch firewall, most SD-WAN tunnels are the initiator, which means the tunnel will have SD-WAN path-monitor probing enabled. • On a hub, verify that the number of SD-WAN probe Request packets received equals the number of probe Reply packets transmitted. On a hub firewall, most SD-WAN tunnels are the responder, which means the tunnel will have SD-WAN path-monitor probing disabled. > show counter global filter delta yes flow_sdwan_prob_req_tx flow_sdwan_prob_reply_rx > show counter global filter delta yes flow_sdwan_prob_req_rx flow_sdwan_prob_reply_tx View VPN Tunnel Information 112 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. If you want to ... Use ... • View all tunnels created on firewall. > show vpn flow • View details of individual tunnels identified by name. > show vpn flow name <name> • View details of individual tunnels identified by ID. > show vpn flow tunnel-id <tunnel-id> • View Internet Key Exchange (IKE) Phase 1 and Phase 2 details for all tunnels. > show vpn ike-sa • View IKEv2 security associations (SAs) and IKEv2 IPSec child SAs of a specific gateway. > show vpn ike-sa gateway <gateway> • View tunnel details. > show vpn tunnel View BGP Information • View BGP summary for a virtual router. > show routing protocol bgp summary virtual-router <virtual-router> • View BGP peer summary. > show routing protocol bgp peer peer-name <peer-name> virtual-router <virtual-router> • View summary of local routing information base (RIB). > show routing protocol bgp loc-rib View SD-WAN Interface Information among RIB and FIB • View new SD-WAN egress interface. > show routing route • View SD-WAN interfaces in forwarding information base (FIB). > show routing fib SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 113 2021 Palo Alto Networks, Inc. Troubleshoot App Performance Understanding what is causing degraded performance for your apps and services is integral to ensuring the user experience is not impacted. Understanding why your VPN clusters are impacted and app traffic failed over to different links helps in fine tuning your SD-WAN configuration. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring and view the Impacted VPN clusters. STEP 3 | Filter the VPN clusters based on your preferred metric from the Site drop-down and select time frame. In this example, we are viewing All Sites containing impacted VPN clusters in the last 12 hours. STEP 4 | In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. 114 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. STEP 5 | In the App Performance section, click an app to view detailed Traffic Characteristic information about the app traffic such as the internet service(s) and links used: • Review the pie chart to understand the breakdown of app traffic across the your internet services. • Review the linegraph to understand how many bytes of data were transferred over each internet service over time. • Review the Links Used section to understand which links the app traffic used and to understand how many of the bytes were impacted out of the total bytes in the selected time frame. STEP 6 | Investigate which health metric caused the app to swap links. SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 115 2021 Palo Alto Networks, Inc. The dotted line indicates the seven day average for the health metric. 1. In the Links Used section of the Traffic Characteristics tab, click an ethernet Link to view detailed Link Characteristics (latency, jitter, and packet loss) over the time frame specified in Step 2 to investigate what health metric caused the app to swap links. 2. In the Traffic Characteristics tab, select another link to view the Link Characteristics for secondary app link to better understand what caused the VPN cluster to become impacted. 116 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. STEP 7 | After you have identified why the app traffic is impacted, consider the following to resolve the issue: • Consider adding additional links to the Traffic Distribution Profile. By adding additional links for app traffic to fail over to, you help ensure that the app traffic and user experience are not impacted by links with degraded health. • Reconfigure the health thresholds in your Path Quality Profile. It may be that the health thresholds are too strict, resulting in unnecessary link fail over. For example, if you have an app that can experience up to 18% packet loss before user experience is impacted, having a 10% packet loss threshold would result in the app failing over to a different link without a need to. • Consult your internet service providor (ISP) to determine if there are impacts to your network outside of your control that they can resolve. SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 117 2021 Palo Alto Networks, Inc. Troubleshoot Link Performance Understanding what is causing degraded link performance is integral for ensuring the user experience when using apps and services is not impacted. Understanding why your VPN clusters have links that are impacted helps in fine tuning your SD-WAN configuration to ensure that the user experiences when using apps and services are not affected by links with degraded health. STEP 1 | Log in to the Panorama Web Interface. STEP 2 | Select Panorama > SD-WAN > Monitoring and view the Impacted VPN clusters. STEP 3 | Filter the VPN clusters based on your preferred metric from the Site drop-down and select time frame. In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. In this example, we are viewing All Sites containing impacted VPN clusters in the last 24 hours. STEP 4 | In the Sites column, select the impacted hub or branch firewall to view the impacted apps and the corresponding link performance. 118 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. STEP 5 | In the App Performance section, click an app to view detailed Traffic Characteristic information about the app traffic such as the internet service(s) and links used: • Review the pie chart to understand the breakdown of app traffic across the your internet services. • Review the linegraph to understand how many bytes of data were transferred over each internet service over time. • Review the Links Used section to understand which links the app traffic used and to understand how many of the bytes were impacted out of the total bytes in the selected time frame. STEP 6 | Investigate which health metric caused the app to swap links. SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 119 2021 Palo Alto Networks, Inc. The dotted line indicates the threshold you configure when you Create a Path Quality Profile. 1. In the Links Used section of the Traffic Characteristics tab, click an ethernet Link to view detailed Link Characteristics (latency, jitter, and packet loss) over the time frame specified in Step 2 to investigate what health metric caused the app to swap links. In this example, we are viewing ethernet 1/1 and can see that the percentage of packets lost regularly exceeded the configured threshold in the Path Quality Profile for the app and can conclude that this is the reason the app traffic failed over to the next best link. 2. In the Traffic Characteristics tab, select another link to view the Link Characteristics. In this example, we are viewing ethernet 1/4 and can see that after the app traffic failed over, ethernet 1/4 experienced jitter for the app that exceeded the configured threshold. This forced the app traffic to fail over back to ethernet 1/1. Since both links had health metrics that were exceeded, the app traffic had no healthy link to fail over to resulting in the VPN cluster becoming impacted. 120 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. STEP 7 | After you have identified why the app traffic is impacted, consider the following to resolve the issue: • Consider adding additional links to the Traffic Distribution Profile. By adding additional links for app traffic to fail over to, you help ensure that the app traffic and user experience are not impacted by links with degraded health. • Reconfigure the health thresholds in your Path Quality Profile. It may be that the health thresholds are too strict, resulting in unnecessary link fail over. For example, if you have an app that can experience up to 18% packet loss before user experience is impacted, having a 10% packet loss threshold would result in the app failing over to a different link without a need to. • Consult your internet service providor (ISP) to determine if there are impacts to your network outside of your control that they can resolve. SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 121 2021 Palo Alto Networks, Inc. Upgrade your SD-WAN Firewalls Review the Panorama Plugin for SD-WAN 2.1 Release Notes and then use the following procedure to upgrade your Panorama and managed SD-WAN firewalls. STEP 1 | Install Content and Software Updates for Panorama. STEP 2 | Upgrade your managed Log Collectors. • Upgrade Log Collectors when Panorama is Internet-Connected. • Upgrade Log Collectors when Panorama is not Internet-Connected. STEP 3 | Upgrade your SD-WAN hub firewalls. You must upgrade your hub firewalls from PAN-OS 10.0.0 to PAN-OS 10.0.1 or later release before you upgrade your branch firewalls. Upgrading branch firewalls before hub firewalls may result in incorrect monitoring data (Panorama > SD-WAN > Monitoring) and for SD-WAN links to erroneously display as down. • Upgrade Firewalls when Panorama is Internet-Connected. • Upgrade Firewalls when Panorama is not Internet-Connected. STEP 4 | Upgrade your SD-WAN branch firewalls. • Upgrade Firewalls when Panorama is Internet-Connected. • Upgrade Firewalls when Panorama is not Internet-Connected. 122 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 2021 Palo Alto Networks, Inc. Upgrade the SD-WAN Plugin Upgrade the SD-WAN plugin version installed on your Panorama™ management server and firewalls leveraging SD-WAN. STEP 1 | Log in to the Panorama web interface. STEP 2 | Upgrade the SD-WAN plugin version on Panorama. For Panorama in a high availability (HA) configuration, repeat this step on the Panorama HA peer. 1. Select Panorama > Plugins and Check Now for the latest sd_wan plugin version. 2. Download and Install the latest version of the SD-WAN plugin. STEP 3 | Upgrade the SD-WAN plugin version on your hub and branch firewalls. 1. Select Panorama > Device Deployment > Plugins and Check Now for the latest sd_wan plugin version. 2. Download the latest version of the SD-WAN plugin. 3. Install the SD-WAN plugin and select your hub and branch firewalls from the list of Devices. 4. Click OK to install the new SD-WAN plugin version on the selected hub and branch firewalls. SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting © 123 2021 Palo Alto Networks, Inc. Uninstall the SD-WAN Plugin To uninstall the SD-WAN plugin from the Panorama management server, you must remove your SD-WAN plugin configuration from Panorama before you can successfully uninstall the SD-WAN plugin. STEP 1 | Log in to the Panorama web interface. STEP 2 | Remove any security policy rules that allow BGP to run between your SD-WAN hubs and branches. 1. Select Panorama > SD-WAN > Devices > BGP Policy and Remove the security policy rules. 2. Click OK to save your configuration changes. STEP 3 | Select Panorama > Plugins and select Remove Config for the SD-WAN plugin. STEP 4 | Select Commit and Commit and Push your configuration changes to your managed firewalls. STEP 5 | Uninstall the SD-WAN plugin. Click OK when prompted to continue uninstalling the SD-WAN plugin. 124 SD-WAN ADMINISTRATOR’S GUIDE | Troubleshooting