Fire Jumper Academy
FTD Troubleshooting for FE’s
Ed Finger
edfinge@cisco.com
Day 1
• 9AM-Noon: Lecture
• 1PM-5PM: Lab
Day 2
• 9AM-Noon: Lecture
• 1PM-5PM: Lab
Special Thank You to Nazmul Rajib
• Reference Diagrams from the Book
• Cisco Firepower Threat Defense (FTD)
• Nazmul Rajib
Training Resource for this Class
amazon.com/dp/1587144808
ciscopress.com/title/9781587144806
Agenda
•
Cisco Firepower Technology
•
Firewall Licensing and Registration
•
Firewall Deployment in Routed and
Transparent Modes
•
Capturing Traffic for Advanced
Analysis
•
Blocking Traffic Using Inline Interface
Mode
•
Inspecting Traffic Without Blocking IT
•
Handling Encapsulated Traffic
(Prefilter Policy)
•
Bypassing Inspections and
Trusting Traffic
•
Rate Limiting Traffic
•
Block-Listing Suspicious
Addresses by Using Security
Intelligence
•
Filtering URLs Based on Category,
Risk, and Reputation
•
Discover Network Applications and
Control Application Traffic
•
Controlling File Transfer and
Blocking the Spread of Malware
Cisco Secure Firewall
Technology
Firepower System
• Sourcefire acquisition by Cisco 2013
• Leader in:
• Intrusion Detection System (IDS)
• Intrusion Prevention System (IPS)
• Next Generation Firewall (NGFW)
• IPS based on SNORT
• Developed by Martin Roesch
Firepower System
• Firewall Management Center (FMC)
• Unified management over firewalls, application control, intrusion
prevention, URL filtering and advanced malware protection
• Firewall Threat Defense (FTD)
• Unified software image
• Next Generation Firewall (NGFW)
• Firewall System Components
Firepower System Components
• Firewall Core Software
• Unified image for Cisco ASA and Firewall Services (FTD)
• SNORT engine
• Intrusion detection, prevention, Snort 2.x and Snort 3.x
• Web Server for GUI
• SNORT/Sourcefire rules
• Special ruleset to detect and prevent intrusion
• Vulnerability database (VDB)
• Stores vulnerability information and fingerprints of applications, services and operating
systems
Firepower System Components
• Geolocation database (GeoDB)
• Stores geographical information and associated IP addresses
• URL filtering database
• Categorizes websites based on targeted audiences or business purposes
• Stored in the URL filtering database
• Security Intelligence Feed
• Talos monitors the internet to identify potential malicious IP addresses, domain names, and
URLs
• Local Malware detection
• Malware license the FTD can detect viruses in files
• ClamAV engine to analyze files locally
Operational Modes
FTD Deployment Modes
NGFW
Routed
10.1.1.0/24
inside
NGIPS
FTD
10.1.2.0/24
outside
DMZ 10.1.3.0/24
Transparent
inside
FTD
DMZ
Integrated Routing
Inline Tap
10.1.1.0/24
FTD
inside
and Bridging
10.1.1.0/24 DMZ
outside
Inline
10.1.2.0/24
outside
Eth1/1
Eth1/1
Passive
Eth1/1
FTD
FTD
FTD
Eth1/2
Eth1/2
Mode of Operation – IPS Only
▪
Passive Mode
•
•
▪
Detection only mode. Does not interrupt network traffic flow.
Support physical interfaces and Ether Channels only.
Sensor sits between two
physical ports on a switch or
two different switches
Inline Mode
•
Blocks malicious traffic in real time.
• Support physical interfaces and Ether Channels only
• Cannot use redundant interfaces, VLANs, and sub-interfaces
▪
Inline-Tap
•
•
Cabling is like the Inline mode, but acts like the Passive mode.
Switch between inline and inline-tap modes is done simply in the
GUI.
Transparent Interfaces
Sensor is Layer 2 Bridge
Architecture
Architecture Diagram
Firewall Threat Defense Introduction
Firepower (mainly L7
functionality)
LINA/ASA (L2-L4 functionality)
Feature migration
Full feature set
Firewall Threat Defense
On-box
Firewall Device Manager
Off-box
Firewall Management Center
FTD Software Architecture – The Big Picture
• LINA engine (multiple instances of Data Path) - Focused on L2-L4 functionality
• Snort engine (multiple instances of Snort) - Focused on L7 functionality
1. A packet enters the ingress interface and it is handled by the LINA engine
2. If the policy dictates so the packet is inspected by the Snort engine
3. Snort engine returns a verdict (allow or block) for the packet
4. The LINA engine drops or forwards the packet based on Snort’s verdict
FTD Flow Load Balancing
Internet
A
FTD
NUMA
LINA
Load Balancer
B
C
LINA A
LINA B
LINA C
A
B
C
C
SNORT
A
A
SNORT
B
DAQ A
DAQ B
DAQ C
B
DAQ Initiates Flows to SNORT Instance
SNORT
C
FXOS Architecture and Packet Flow
1.
2.
3.
4.
5.
6.
MIO receives the packet on physical port
MIO sends the packet to Decorator (DDoS - vDP Radware)
MIO receives the packet from Decorator (DDoS - vDP Radware)
MIO sends the packet to the Security Module (FTD). If there is FTD Clustering the MIO will get
the packet back and send it to the peer unit
FTD sends the packet back to MIO
MIO sends the packet to the egress physical port
Packet Flow on FP2100
1.
2.
3.
4.
5.
6.
The packet arrives on the Internal Switch
Internal Switch forwards the packet to the LINA (ASA) Engine (Network Processing Unit - NPU)
LINA (ASA) Engine forwards the packet to Snort Engine (CPU)
Snort Engine forwards the packet back to LINA (ASA) Engine (NPU)
LINA (ASA) Engine forwards the packet to the Internal Switch
The Internal Switch sends the packet back to physical network
FTD Diagnostic vs. Management Interface
Management Interface
Diagnostic
br1/management0
•
•
•
Mandatory interface (terminate
sftunnel)
Assigned IP address is used for
FTD/FMC communication
SSH/HTTPS access to the FTD
> show network
•
•
•
Not recommended to
configure, use Data interface
Remote access to ASA (LINA)
Source for ASA syslog, AAA
messages, etc.
# show interface ip brief
SNORT Architecture
Detection Engine
• Consists of two components to perform inspection
• Rules builder
• Inspection component
• Rules builder
• On Snort startup, assembles rules into rule chains
• Optimizes rule matching by the inspection component
• Sources, destinations and port sources and destinations redundancies are eliminated
• Implements rules chains as linked lists in a decision tree
• Inspection component
• Matches traffic to a rule chain
• Further inspects traffic against the options in the matching rule chain
• A packet is tested if it is TCP > Then passed to TCP rules, Source Address, etc. Until packet either matches an attach
signature or tests clean.
23
Detection Engine
• Consists of two components to perform inspection
• Rules builder to build attack signatures by parsing Snort rules
• Inspection component
• Rules builder
• On Snort startup, assembles rules into rule chains
• Optimizes rule matching by the inspection component
• Sources, destinations and port sources and destinations redundancies are eliminated
• Implements rules chains as linked lists in a decision tree
• Inspection component
• Matches traffic to a rule chain
• Further inspects traffic against the options in the matching rule chain
• A packet is tested if it is TCP > Then passed to TCP rules, Source Address, etc. Until packet either matches an attach
signature or tests clean.
24
SNORT 2 Architecture
SNORT 2 Architecture
Frag(x)
Stream(x)
8080, 8000, 8180, etc.
SNORT 2 Architecture
SNORT 2 Components Supported
acl
AC policies in the packet path
pdts
Communication path between Lina and
Snort
daq
Data Acquisition Layer for accepting
the packet into SNORT
Snort-engine
Snort Generic Component
Snort-file processor
Snort File Processor Component
Snort-firewall
Snort Component responsible for
enforcing Access Control Policy (AC
Policy)
Snort 3 - Overview
• Snort 3 is now supported with FMC as well as FDM
• Snort 3 Device Management
• Ability to toggle device Snort versions (Snort 2<->Snort 3) from FMC device
management
• Upgrade / Migration Changes
• Simplified Migration of Snort 2 to Snort 3 policies after upgrading to FP 7.0
• Support for synchronizing common intrusion policies between Snort 2 and
Snort 3 versions
Snort 3 Primer
• Inspectors replace preprocessors
• Event driven Plugins accomplish much of the processing objectives:
• Codec - to decode and encode packets
• Inspector - replaces Snort 2 preprocessors, for normalization, etc.
• IpsOption - for detection in Snort rules
• IpsAction - for custom actions
• Logger - for handling events
• Mpse - for fast pattern matching
• So - for dynamic rules
• Lightweight Security Package(LSP) is the new release manager for Snort 3, replacing
Security Rule Updates (SRU) – fast, smaller, higher frequency
Snort 3 – Benefits
• Much more efficient memory utilization and faster reload times from
multi-threaded architecture
• Faster/deeper pattern lookups with HyperScan for higher efficacy
• Improved human-readable signature language
• LSP Light Weight Security Package – Smaller and Faster
• Event-driven plugins replace preprocessors for quicker verdicts
Snort 3.0 Architecture
• Threaded to utilize multiple cores
• 1 control thread (main)
• N packet threads per process
• Reloads faster (1 vs N)
• A single config and network map
• Uses less memory
• Supports more IPS rules and larger netmap
• Rules written in text like Snort 2
• More uniform syntax in Snort 3
• Easier to read, write, and verify
• “Snort2lua” converts 2.9 IPS rules to 3.0 format
• LuaJIT will be added later by TALOS
Presentation ID
34
Snort 3 Traffic Inspection
Custom Intrusion Rules
• Users can upload custom intrusion rules, written in Snort 3 rule
syntax
• snort2lua tool on the FMC can be used to convert Snort 2 rules to
Snort 3 syntax
• Each custom rule must have a SID (>1000000) and REV
information
• GID need not be provided by the user
• GID will be auto-generated per domain as in case of Snort 2
• Auto generated GID will be different from Snort 2 GID to avoid
SID collision`
Snort 3 – Changes in IPS Rule
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"BLACKLIST URI request for
known malicious URI"; flow:established,to_server; content:"/setup_b.asp?prj="; nocase;
http_uri; content:"&pid="; nocase; http_uri; content:"&mac="; nocase; http_uri;
pcre:"/\/setup_b\.asp\?prj=\d\x26pid=[^\r\n]*\x26mac=/Ui"; metadata:service http;
sid:19626; rev:2;)
alert http
(
msg:"BLACKLIST URI request for known malicious URI";
flow:established,to_server;
http_uri;
regex:"/setup_b\.asp\?prj=\d&pid=.*&mac=", nocase, fast_pattern;
sid:19626; rev:4;
)
Snort Tools
FTD ASP Drop Output
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Tools – show asp drop – FTD
•
Command Reference for all show asp drops:
•
https://www.cisco.com/c/en/us/td/docs/security/asa/asa-commandreference/show_asp_drop/show_asp_drop.pdf
•
Snort-drop: This counter is incremented and the packet is dropped
as requested by Snort module.
•
Snort-down : This counter is incremented and the packet is dropped
as the Snort module is down.
•
Snort-busy: This counter is incremented and the packet is dropped
as the Snort module is busy and unable to handle the frame.
Show Snort Stats
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Tools – show snort statistics – FTD CLI Guide
Use this command to show Snort inspection results of your access
policy and intrusion rule configurations. This command is typically used
when debugging unexpected Snort inspection behavior. The statistics
include the following:
•
Passed Packets—The number of packets sent to Snort, including
packets sent from Lina.
•
Blocked Packets—The number of packets blocked in Lina and not
sent to Snort.
Tools – show snort statistics – FTD CLI Guide
•
Injected Packets—The number of packets Snort created and added
to the traffic stream. For example, if you configure a block with reset
action, Snort generates packets to reset the connection.
•
Packets bypassed (Snort Down or Snort Busy)—If you configure
the system to allow packets that require Snort inspection and Snort
cannot perform the inspection, these counters are the number of
packets that bypassed inspection when Snort was either down or too
busy to handle the packets.
Tools – show snort statistics – FTD CLI Guide
•
Fast-forwarded flows—The number of flows that were fast
forwarded by policy, and thus not inspected.
•
Blacklisted flows—The number of flows from policy configuration
that were dropped by Snort.
•
Start-of-flow events—The Lina process sends start-of-flow events to
Snort when it fast paths a flow without sending it to Snort. These
events help Snort keep track of the connections and report the
connection events.
•
End-of-flow events—The Lina process sends end-of-flow events to
Snort when a fast path flow ends.
Tools – show snort statistics – FTD CLI Guide
•
Denied flow events—The Lina process sends denied flow events to
Snort when it decides to drop a flow before sending it to Snort.
•
Frames forwarded to Snort before drop—The number of to-bedropped packets forwarded to Snort. When the Lina decides to drop
the frame for some reason, it is also sent to Snort for visibility.
•
Inject packets dropped—The number of packets that Snort added to
the traffic stream that were dropped.
Detailed Packet
Processing Flow
General Traffic Flow through FTD
Firewall Data-Path Troubleshooting Framework flow diagram
Packet
Ingress
Platform
DAQ
SI
ACP
SSL
Auth
Software
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
IPS
NAP
Packet Flow
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
FTD Packet Processing: ACL best practices
•
•
•
•
•
Use Prefilter Policy Fastpath rules for big ‘fat’ flows
Place more specific rules at the top of the Access Control Policy
Place rules that require Snort inspection at the bottom of the policy
Avoid excessive logging
Be aware of rule expansion
> show access-list | include elements
access-list CSM_FW_ACL_; 7 elements; name hash: 0x4a69e3f3
admin@NGFW1$ cat /var/sf/detection_engines/UUID/ngfw.rules | grep "Start of AC rule" -A 10000000 | grep v " of AC rule." | wc -l
32
Traffic Flow Troubleshooting
• Tools
used to troubleshoot Traffic Flow:
•
System Support Trace
• System Support Firewall-Engine-Debug
• Capture
• Capture-Traffic
• Packet-Tracer
• Show
asp drops
• Show snort stats
Designing a Firewall Management Network
Requires that management interfaces can communicate with
each other.
FMC can manage from any location
Firewall Management Network
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Management Interfaces
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Management Interface Multiple Networks
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Management Interface Best Practices
• Segment management traffic from the data traffic.
• Keeping the management network out-of-band secures the control-plane
traffic in case the data network is attacked.
• Default management port TCP 8305
• Cisco strongly recommends not to change the default port number
Firewall Licensing and
Registration
Smart License Architecture
• Offers two major benefits over the Classic License.
• Enables you to administer all your Firewall licenses from a single place.
• Enables full functionalities of the FMC and FTD without installing any licenses for the first 90 days of
an initial deployment.
• In the Smart Licensing architecture, the Firewall manager uses a process—Smart
Agent—to communicate and register with the Cisco License Authority.
• Cisco offers two options to connect a Firewall manager to the Cisco License Authority.
• Cisco License Authority directly over the Internet
• Smart Software Manager On-Prem.
Cisco Smart Software Manager (SSM)
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
License Types
•
Base
• Update the Firewall System
• Control applications and users
• Perform switching, routing and NAT
• Threat
• Detect and prevent intrusion attempts
• Block-List traffic based on intelligence
• Block transfer of certain types of files
• Malware
• Protect network from malware, enable AMP for network and enable AMP Threat Grid features
• URL Filtering
• Filter URLs based on Reputation and Category
Connection between FMC and License Server
• Command on the FMC
• Login to FMC console port
• For this lab Username: admin Password: C1sco12345
• Type
• Expert
• sudo tail –f /var/log/sam.log
• You will be prompted for a password use: C1sco12345
Registration Process
• FMC initial configuration
• Management Interface IP address: pre-configured DHCP (7.x) will fall back to 192.168.95.1
• Username: Admin Password: Admin123
• Device Management Add Device from the GUI
• Change Network Parameters from CLI prompt: sudo configure-network password: C1sco12345
• FMC Troubleshooting
• CLI Commands
• Expert
• route –n
• sudo ping x.x.x.x (For lab testing use 198.19.10.1, 198.19.10.100)
• sudo traceroute <IP Address of FTD>
• sudo netstat –antp | grep –i 8305 [will show currently registered FTD’s and their tunnel ports]
• sudo sftunnel_status.pl <FTD IP Address>
Registration Process
• FTD Management Interface
• Uses management interface to send and receive control traffic
• During FTD Installation Process when you provide IP address for the Management Interface it is assigned
to the Logical management interface
• Uses the logical management interface to complete the registration process with the FMC
• Establishes the Secure Tunnel
• Apply Policies to the FTD
• Transfer events from FTD to FMC
• FTD Data Interfaces
• Each data interface required to be on a different network
• Diagnostic Interface must be on same subnet as logical management interface br1
• When diagnostic interface is configured with IP address it is considered a Data Interface
• Must be on separate subnet from inside interface need a router to send traffic between different subnets
Registration Process
• FTD Troubleshooting
• Show network
• Shows Hostname, Domains, DNS servers, Management Port, Gateway, IP Address of Management
Interface
• Show interface ip brief
• Ping system x.x.x.x
• capture-traffic (for SNORT and br1)
• sftunnel-status
• Expert mode Commands
• ifconfig
• sudo netstat –antp | grep –i 8305
Troubleshooting Commands
• Restart the sftunnel Process
• sudo manage_procs.pl
• Option 3 Restart Comm. Channel
• On FMC to run debug log:
• sudo tail –f /var/log/messages | grep [Management IP Address e.g. 198.19.10.81]
Common Causes
Registration
Failures
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
1.
RMA’s
2.
Registration Keys
3.
NAT
4.
Sftunnel Certificate
Expired
5.
Sftunnel.conf file
corrupted
6.
Database Issues
Registration Failure Troubleshooting
•
Logs to check (sudo –i)
•
/etc/sf/sftunnel.conf
• /var/log/messages
•
Scripts for registration
issues
•
•
•
•
•
register_appliance.pl
remove_peer.pl
remove_managers.pl
manage_procs.pl
sfcfa_revoke
Registration Failure: Corrupt sftunnel.conf
•
Registration may fail immediately or error if sftunnel.conf is corrupt.
•
Common error messages
“Communication channel for management interface is not configured”
during registration
• ”Could not establish a connection with sensor. Make sure the registration
keys match, that the software versions are compatible, and that the
network is not blocking the connection”
•
Check sftunnel.conf file:
# cat /var/sf/sftunnel.conf.in
# cat /var/sf/sftunnel.conf
Registration Failure: Fix Corrupt sftunnel.conf
1.
Stop sftunnel process.
# sudo pmtool disablebyid sftunnel
2.
Backup corrupt sftunnel file.
# cp /etc/sf/sftunnel.conf.in /etc/sf/sftunnel.conf.bad
3.
Replace corrupt file with a known good sftunnel.conf from TAC.
4.
Run the command to populate the data in the replacement file.
# perl -MFlyLoader -e
"SF::PeerManager::ConfigFiles::create_sftunnel_config()"
5.
Restart sftunnel process.
# pmtool enablebyid sftunnel
6.
Attempt registration again.
Registration Failures : Database Issues
•
Verify the EM_peers table is not corrupt
•
Verify the sensor table is not corrupt
•
Verify process for registration are running
•
•
•
•
•
SFDataCorrelator
SFTunnel
SFMGR
SFIPPROXY
Mysqld
Firewall Deployment in
Routed and Transparent
Modes
Policy Deployment
Failures
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
•
Troubleshooting
deployment failures
•
Low Bandwidth causing
deployment failure
Policy Deployment Failure Troubleshooting
•
Collect UI Error for deployment failure
•
Collect IP of sensor(s) failing deployment
•
Optionally
•
Trouble Shoot files from FMC & Sensor(s)
• Pigtail deployment logs
•
Logs to monitor for policy deployments
•
action_queue.log
• usmsharedsvcs.log (FMC)
• policy_deployment.log
• pigtail (shows a combination of log file data)
Capturing Traffic for
Advanced Analysis
Getting to know the tools Capture and Trace
Best Practice for Capturing Traffic
• The primary objective of a Firewall system is not to capture live traffic
• Capture of live traffic only for troubleshooting purposes.
• Use a dedicated system for capturing traffic for a long period of time.
• Capturing live traffic on a production system can degrade system performance.
• Capture traffic during a maintenance window.
• Redirect packets to a file instead of console.
• Copy the file a computer and open it by using any packet analyzer.
• If not storing the packet in a file, limit the number of packets captured.
Packet Capture
•
Allows Real packets that are captured on ingress interface to
be traced through the system
•
Includes information from Snort and preprocessors about
verdicts and actions
•
Multiple packet captures are possible at a time
•
Can modify, delete, clear and save captures
•
Traffic data can be downloaded in pcap or ASCII format
Capturing Traffic for Advanced Analysis
• Traffic Capture Essentials
• FTD blocks a packet from ingress to egress interface performed by:
• Firewall engine or Firepower engine.
• Analyze packets from both engines to determine the problem.
• Registration or communications issues between FTD and the FMC
• Capture-traffic from the Firepower management interfaces.
TRAFFIC TRANSVERSAL
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Traffic Capture
• Firepower Engine
• Firewall Engine
• FMC
CAPTURE vs CAPTURE-TRAFFIC
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Troubleshooting Traffic Capture Packet
Processing by an Interface
• Packet Processing by an Interface
• If traffic not showing up in Capture. Check filtering condition in the capture or capture-traffic command
is correct.
• Check the statistics of the FTD interface and determine whether any packets are dropped by the
interface.
• > show interface GigabitEthernet 1/1
• Interface GigabitEthernet1/1 "INSIDE_INTERFACE", is down, line protocol is down
• 0 no buffer
Received 68 broadcasts, 0 runts, 0 giants
0 input errors, 0 CRC, 0 frame, 0 overrun,
• FTD pulls packets from the ingress interface.
• If the FIFO queuing algorithm becomes full. Incoming packets get dropped. The number of these dropped
packets in the overrun counter.
• The no buffer counter in the interface statistics provides the number of dropped packets.
FIFO QUEUE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
FTD Captures
• FTD provides 2 types of captures
1. LINA-level capture – ‘capture’ command
2. Snort-level capture – ‘capture-traffic’ command
• Where do these captures take place?
• capture-traffic command can be also used to capture FTD controlplane traffic (br1, management0)
FTD Packet Processing: Intrusion Policy
•
•
Intrusion Policy (Snort Rules) is the same as on classic Firepower devices
Packets marked as dropped by a Snort Rule (Signature) can be tracked by:
1. FMC GUI (Connection Events, Intrusion Events)
2. Capture with Trace:
firepower# show cap CAPI%intf=INSIDE% trace | in Drop
Drop-reason: (ips) Blocked or blacklisted by the IPS preprocessor
3.
From CLISH mode using ‘system support trace’
> system support trace
192.168.0.10-45988 > 192.168.2.10-80
192.168.0.10-45988 > 192.168.2.10-80
192.168.0.10-45988 > 192.168.2.10-80
192.168.0.10-45988 > 192.168.2.10-80
6
6
6
6
Snort: processed decoder alerts or actions queue, drop
AS 1 I 12 Deleting session
NAP id 2, IPS id 1, Verdict BLACKLIST
===> Blocked by IPS
FTD Packet Processing: L3/L4 ACL - Allow
•
Tracing a real packet will show the Snort Verdict
firepower# show capture CAPI packet-number 1 trace
1: 09:17:18.996149
1.1.1.1 > 2.2.2.2: icmp: echo request
!
Phase: 4
Type: ACCESS-LIST
...
This packet will be sent to snort for additional processing where a verdict will be reached
!
Phase: 13
Type: EXTERNAL-INSPECT
...
Application: 'SNORT Inspect'
Phase: 14
Type: SNORT
...
Snort Verdict: (pass-packet) allow this packet
FTD Data-Plane: Capture + Capture w/Trace
As from 6.2 release the capture can trace Snort features known as
preprocessors and get the verdict:
> show capture CAPI packet-number 6 trace
..
Type: SNORT
Result: DROP
Additional Information:
Packet: TCP, ACK, seq 157078375, ack 664745083
AppID: service HTTP (676), application unknown (0)
The Snort ACP Rule that
Firewall: starting rule matching, zone -1 -> -1, geo 0(0) -> 0, vlan 0, sgt 65535,
username 'No Authentication Required', , url http://192.168.2.10/image.bmp
triggered the Drop Verdict
Firewall: block rule, 'Rule1' , drop
Snort: processed decoder alerts or actions queue, drop
NAP id 2, IPS id 0, Verdict BLACKLIST, Blocked by Firewall
The Snort Verdict
Snort Verdict: (black-list) black list this flow..
..
Result:
input-interface: INSIDE
input-status: up
input-line-status: up
Action: drop
Drop-reason: (firewall) Blocked or blacklisted by the firewall preprocessor
The Drop Reason
FTD Data-Plane – Capture w/Trace
Snort Verdicts of Capture w/Trace and possible Root Causes
Drop Reason
Possible Root Cause
(firewall) Blocked by the
firewall preprocessor
The packet is dropped by Snort ACL Block rule.
Check 'system support firewall-engine-debug' to
confirm this
ACP L7 Rule
(ips) Blocked the IPS preprocessor
The packet is dropped by a Signature from the Snort
Intrusion Policy
Intrusion Policy
(file-process) Blocked by the
file process preprocessor
The packet is dropped by Snort File Policy
(reputation) Blocked by the
reputation preprocessor
The packet is dropped by the Snort Security
Intelligence (SI)
(snort-drop) Snort requested to drop
the frame
The packet matches an existing LINA Engine flow.
Check the first packet of the flow that was blocked to
see the full verdict
File Policy
SI (IP reputation)
Existing
LINA Engine
flow
FTD Data-Plane – Capture + Capture w/Trace
Source interface
IP Protocol
Circular buffer
Trace ingress packets
Current GUI limitations
•
Cannot specify Src and Dst ports
•
Only basic IP Protocols can be matched
•
Cannot enable capture for LINA Engine ASP Drops
Workaround – Use the FTD CLI
Traffic Capture Troubleshooting FTD
• Run the capture-traffic command on the shell
• Select the Router to capture traffic from the data interfaces.
• > capture-traffic
• Select 1 Router or Global (Version dependent)
• Options
• –c Stops after a certain number of packets are captured
• –e Displays the Ethernet header in the capture
• –n Does not resolve the host name and port name
• –s Defines the size (snaplength) of the captured packets
• –v Show extra packet data. –vv shows even more data
• –w Saves the captured packets in a file instead of displaying them on the console
• –X Shows packet contents in hex and ASCII
• Host, net Filters traffic to and from a single host or the entire network, respectively
• Port, portrange Filters traffic to and from a single or range of ports respectively
• Src, dst Selects a direction for traffic flow –source or destination. Used in conjunction with the host and port options
• And, or, not Combines or isolates traffic with a precise condition
• Vlan Captures traffic related to a particular VLAN. To capture VLAN tagged traffic, you must enter the vlan option fillowed by
the desired vlan_id
Troubleshooting Traffic Capture ICMP FTD
> capture icmp_traffic interface INSIDE_INTERFACE match icmp any
any
> show capture icmp_traffic
> capture-traffic
> Router or Global
Options: icmp
ICMP-BLOCK-RULE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Traffic Capture Troubleshooting FTD
• Example
• –n –c 5 host x.x.x.x and port 80
• –n –c 5 scr x.x.x.x and dst port 80
• Server side traffic-originated web server
• –n –c 5 dst x.x.x.x and src port 80
• Capture with default packet data
• –n –c 5 host x.x.x.x
• Print option
• –n –c 5 –vv host x.x.x.x
• Layer 2 Header
• –n –c 5 –e host x.x.x.x
• Download .pcap File
• –w traffic.pcap –s 1518 host x.x.x.x
Traffic Capture Troubleshooting Firewall
Engine
• Determine the name of the interface from which you want to capture traffic
• > show nameif
• Capture <capture name> interface <int_name> match protocol_name
source_detail destination_detail
Traffic Capture Troubleshooting FMC
• admin@FMC:~$ ifconfig
• Check eth0
• admin@FMC:~$ sudo tcpdump –i eth0 host x.x.x.x
• sudo tcpdump –I eth0 host x.x.x.x –w /var/common/fmc_traffic.pcap
• To store any user-generated file, use the /var/common directory on the FMC. If you save a file on this directory, the
FMC allows you to download it directly using the GUI.
• FMC GUI Download
• Go to the Health > Monitor page and find the FMC in the appliance health status list.
• Click on the FMC name. A detail view of the FMC health status appears.
• Click the Advanced Troubleshooting button. The File Download tab appears.
• Enter the name of the .pcap file (including the file extension) in the File field.
• Click the Download button. The browser prompts you to save the file on your computer.
• FMC CLI Download
• scp /var/common/fmc_traffic.pcap admin@x.x.x.x: /home/user
Locations of
Capture
FTD: /ngfw/var/common
LINA: /mnt/disk0/
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Packet Capture Procedures on Cisco
Firepower Device
Packet Tracer
• Test policy configuration by modeling a packet based on
source/destination address and protocol characteristics.
• Policy lookup to test:
• access rules
• NAT
• Routing
• Access policies
• Rate limiting
• Packet flow based on:
• Interfaces
• SA/DA
• Ports and protocols
FTD Data-Plane - Packet-Tracer
•
Packet-tracer shows the LINA Engine Datapath checks done on a virtual packet
Source interface
summary, detailed or xml format
FTD Data-Plane - Packet-Tracer
When to use Packet-Tracer:
• To verify if traffic to a specific port is allowed by LINA Engine Data-path and Snort
Examples of Snort features that can be verified by Packet-Tracer:
• Security Intelligence (IP Reputation)
• L3/L4 IPS Intrusion Rules
Examples of Snort features that currently cannot be verified by Packet-Tracer:
• L7-related (SI DNS/URL, App ID, File Policy, L7 IPS Intrusion Rules)
• Identity-based rules
FTD Packet Processing: L3/L4 ACL - Allow
•
packet-tracer shows that LINA engine will send the packet to Snort engine for a Verdict
> packet-tracer input inside icmp 1.1.1.1 8 0 2.2.2.2
Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group CSM_FW_ACL_ global
access-list CSM_FW_ACL_ advanced permit ip host 1.1.1.1 host 2.2.2.2 rule-id 268435456
access-list CSM_FW_ACL_ remark rule-id 268435456: ACCESS POLICY: FTD5506-1 - Mandatory/1
access-list CSM_FW_ACL_ remark rule-id 268435456: L7 RULE: ACP_Rule1_Allow_ICMP_App
Additional Information:
This packet will be sent to snort for additional processing where a verdict will be reached
Blocking Traffic Using Inline
Interface Mode
Inline Mode Essentials
• You can deploy an FTD
• IDS Intrusion Detection System
• IPS Intrusion Prevention System
• To prevent any potential intrusion attempt in real time, you must
deploy an FTD device in Inline Mode.
• The ingress and egress interfaces are bundled into an interface
pair.
• Each pair must be associated with an inline set, which is a logical
group of one or more interface pairs.
INLINE-PAIR
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
FTD-DEPLOYMENT
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Inline Mode vs Transparent Mode
• Both Inline Mode and Transparent Mode act as bumps in the wire.
• Not seen by connected devices.
• Inline Mode- Interfaces in an interface pair can send and receive any traffic, if
policies permit.
• No IP addresses needed on member interfaces of an inline-pair.
• Transparent Mode places the inside and outside networks into a virtual bridge
group and creates a Layer 2 bridging network.
• Traffic originates from an FTD device using a Bridged Virtual Interface (BVI) as its
source interface.
• The BVI, inside network, and outside network must all be configured with the IP
addresses from a single subnet.
• Transparent Mode supports NAT
• FTD does not support NAT in the Inline Mode.
CONFIGURING INLINE MODE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Verifying Packet Flow by Using Real Packet
Capture
• Example
• > capture inside_icmp trace interface INSIDE-INTERFACE
match icmp any any
> capture outside_icmp trace interface OUTSIDE_INTERFACE match icmp
any any
> show capture
> show capture inside_icmp
> show capture outside_icmp
> show capture inside_icmp packet-number 1 trace
> show capture inside_icmp packet-number 2 trace
Blocking a Specific Port
• Blocking a Specific Port
• Policies > Access Control > Access Control page
• Blocking port SSH 22 or another
• > show access-list
• Analyzing a Packet Drop
> capture inside_ssh trace interface INSIDE_INTERFACE match tcp
any any eq 22
> show capture inside_ssh
> show capture inside_ssh packet-number 2 trace
Inspecting Traffic
Without Blocking It
Traffic Inspection Essentials
• Deploying a FTD for detection-only two options
• Passive Mode
• Inline Tap Mode
• Passive Monitoring Technology
• Promiscuous Mode: Configure the interface in passive mode, Set the interface into
Promiscuous Mode.
• Promiscuous Mode allows an interface to see any packet in a network segment—even
packets that are not sent to that interface. This capability empowers an FTD device to monitor
the network activities without being an active part of a network.
• SPAN port: Some switch models replicate traffic from multiple switch ports and send the
copies to a specific switch port. An FTD device can monitor a network without being an
active part of the network flow; this feature is called port mirroring.
Best Practices for Detection-Only Deployment
• To observe any network activities and tune the security policies
• Deploy FTD device in Detection-Only Mode Choose:
• Inline Tap Mode over traditional Passive Mode. This allows you to switch to
Inline Mode faster, without touching any physical cables.
• FTD device in Detection-Only Mode permanently: Choose Passive Mode over
Inline Tap Mode to eliminate any chance of traffic interruption due to an
accidental outage of the FTD device (Fail-to-wire can correct this).
• If the utilization of a network is medium to high, use a TAP instead of a SPAN
port.
• FTD device in Passive Mode cannot block traffic
• IP addresses, and DHCP server configurations are not necessary in Inline, Inline
Tap, and Passive Modes.
PROMISCUOUS MODE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Handling Encapsulated
Traffic (Pre-filter Policy)
Pre-filter Packet Processing
Prefilter
• Prefilter and access control policies both allow you to block and trust
traffic, though the prefiltering "trust“ functionality is called "fastpath"
because it skips more inspection.
• Bypass capability Fastpath rule action
• Fastpathed traffic in the prefilter stage bypasses all further inspection
and handling, including:
• Security Intelligence
• authentication requirements imposed by an identity policy
• SSL decryption
• access control rules
• deep inspection of packet payloads
• Discovery
• rate limiting
118
ENCAPSULATED TRAFFIC
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
FTD Packet Processing: Prefilter Policy
•
•
Prefilter Policy got introduced in 6.1 version
Serves 2 main purposes
1. Adds additional flexibility when it comes to handling tunneled traffic:
•
GRE
•
IP-in-IP
•
IPv6-in-IP
•
Teredo Port 3544
2. Provides Early Access Control (EAC) which allows a flow to bypass completely the Snort engine
FTD Packet Processing: Prefilter Policy
Prefilter Rules are deployed to LINA as L3/L4 ACEs and are placed above the normal L3/L4 ACEs
firepower# show access-list
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
access-list CSM_FW_ACL_ line
1 remark rule-id 268434457: PREFILTER POLICY: FTD_Prefilter_Policy
2 remark rule-id 268434457: RULE: Fastpath_Rule1
Prefilter
3 advanced trust ip host 192.168.75.16 any rule-id 268434457 event-log both (hitcnt=0)
Rules
4 remark rule-id 268434456: PREFILTER POLICY: FTD_Prefilter_Policy
5 remark rule-id 268434456: RULE: DEFAULT TUNNEL ACTION RULE
6 advanced permit ipinip any any rule-id 268434456 (hitcnt=0) 0xf5b597d6
7 advanced permit 41 any any rule-id 268434456 (hitcnt=0) 0x06095aba
Tunnel Prefilter
8 advanced permit gre any any rule-id 268434456 (hitcnt=2) 0x52c7a066
Rules
9 advanced permit udp any any eq 3544 rule-id 268434456 (hitcnt=0) 0xcf6309bc
10 remark rule-id 268434445: ACCESS POLICY: FTD5506-1 - Mandatory/1
12 advanced deny ip host 10.1.1.1 any rule-id 268434445 event-log flow-start (hitcnt=0) 0x8bf72c63
L3/L4
14 remark rule-id 268434434: L4 RULE: DEFAULT ACTION RULE
ACEs
15 advanced permit ip any any rule-id 268434434 (hitcnt=410) 0xa1d3780e
Prefilter vs ACP rules = first match is applied
}
}
}
FTD Packet Processing: Prefilter Policy
(tunneled)
•
How works the analyze action:
firepower# show access-list
access-list CSM_FW_ACL_ line 5 remark rule-id 268435473: RULE: Tunnel_Rule1
access-list CSM_FW_ACL_ line 6 advanced permit gre any any rule-id 268435473
root@FTD5506-1:/var/sf/detection_engines/UUID# cat ngfw.rules
# Start of tunnel and priority rules.
# These rules are evaluated by LINA. Only tunnel tags are used ..
268435473 allow any any any any any any any 47 (tunnel 2)
268434456 allow any any any any any any any 4 (tunnel -1)
268434456 allow any any any any any any any 41 (tunnel -1)
# End of tunnel and priority rules.
# Start of AC rule.
268435474 allow 2 any any any any any any any
268435468 allow any any any any any any any any (log dcforward) #
End of AC rule.
Bypassing Inspections
and Trusting Traffic
FASTPATH
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
PreFilter FASTPATH and ACP TRUST
Rate Limiting Traffic
RATE-LIMITING
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Rate Limiting
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
BEST PRACTICES
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Block List Suspicious
Addresses by Using
Security Intelligence
SI-ESSENTIALS
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
SI-ARCHITECTURE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
SI-ARCHITECTURE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
TXT-FILES-FOR-BLOCK LIST
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Block List Configuration
• FMC, navigate to Object > Object Management.
• Select the Security Intelligence > Network Lists and Feeds option.
• Then click the Add Network Lists and Feeds button to upload your text file to the FMC.
• After you select List from the Type dropdown, you can browse your text file. Upon successful upload, the FMC can
show the number of addresses you uploaded.
• Once the file is uploaded, navigate to Policies > Access Control > Access Control and edit the access control policy
that you want to deploy to an FTD.
• When the policy editor page appears, select the Security Intelligence tab. A list of available objects and zones appears.
• On the Network subtab, select the custom object you want to block-list. You can also select a specific zone for
inspection. By default, FTD inspects traffic from Any zone.
•
Immediate Block-Listing Using a Connection Event
Block-List Configuration
• The Firewall System enables you to Block-List an address instantly by right-clicking your mouse.
• Adding an Address to a Block-List
• Navigate to Analysis > Connections > Events.
• Right-click the address you want to block.
• From the context menu that appears, select Add IP to Block List. A confirmation window appears.
Click the Add to Block List to confirm.
• Deleting an Address from a Blocklist
• Any addresses that you Block-List by using the Block List IP Now option are included in the Global-Block-List
category.
• If you want to remove the blocking attribute from an address and allow the address again,
• Object Management > Security Intelligence > Network Lists and Feeds > Global-Block-List and edit to remove
the address.
• If you delete the entire Global-Block-List object by using the Security Intelligence configuration page, the access
control policy does not enforce the Blocklist IP Now function.
Block-List Configuration
• Monitoring a Block-List
• Occasionally, you might want to monitor the activities of certain hosts in a network instead of
blocking them completely.
• Follow these steps to enable monitoring functionality using Security Intelligence:
• Assumes that you have already blocked certain traffic. This time, you just want to change the
action (monitor only instead of block) for certain traffic.
• In the Access Control Policy editor page, go to the Security Intelligence tab.
• From the Block-List field, right-click a category that you want to monitor.
• FTD does not support the Monitor-only mode for the Global-Block-List category.
• From the context menu that appears, select the Monitor-only (do not block) option. The icon changes
from a red × to a green down arrow.
• When the configuration is complete, click Save to save the changes and redeploy the access
control policy.
Block-List Configuration Cont’d
• Verification and Troubleshooting Tools
• Check whether Security Intelligence’s health module is enabled on the health policy.
• System > Health > Policy, edit a health policy, and select Security Intelligence from the left
panel. You must redeploy a health policy if you change any settings.
• Check whether the FMC has a valid Threat license and whether the license is applied on the
desired FTD device.
• Verifying the Download of the Latest Files
• admin@FMC:~$ ls -halp /var/sf/iprep_download/
• ! After updating the Cisco Intelligence Feed from the cloud, FMC shows the
blocklist files:
• ls –halp /var/sf/iprep_download/
FTD Block-List Configuration Cont
•
FTD
• ! After a fresh installation, FTD shows only the empty Block-List (.blf) and Do-Not-Block-List (.wlf) files. At this point,
FMC has not applied a Cisco Intelligence Feed yet:
• > expert
• admin@firepower:~$ ls -halp /var/sf/iprep_download/
• ! Upon a successful deployment of an Access Control policy, FTD received the necessary Block-List (.blf) and Do-NotBlock-List (.wlf) files from an FMC. Each of these files represent a Security Intelligence category.
• admin@firepower:~$ ls -halp /var/sf/iprep_download/
• If you have enabled Security Intelligence based on URL, you can find the list of Block-Listed and Do-Not-BlockListed URLs in similar format. The files are located in the /var/sf/siurl_download directory.
•
Verifying URL-Based Security Intelligence Rules
• Security Intelligence at /var/sf/siurl_download.
• admin@firepower:~$ ls -halp /var/sf/siurl_download/
• admin@firepower:~$ cat /var/sf/siurl_download/url.rules
FTD Packet Processing: SI (IP)
•
Security Intelligence (SI) can Drop or Allow IP addresses early in the packet processing lifetime within
the Snort engine
•
Do-Not-Block-List overrides the Block-List
•
The Blocklist can be populated in 2 ways:
•
1.
Manually by the FMC administrator
2.
Automatically by Intelligence Feed (Talos or custom) or List
Snort returns to LINA a verdict about a packet being blocked
FTD Packet Processing: SI (IP)
•
The files containing the IPs from Talos SI Feed are in /ngfw/var/sf/iprep_download directory
root@FTD5506-1:/ngfw/var/sf/iprep_download# ls -alt | grep blf
-rw-r--r-- 1 root root 1252278 Jun 12 16:06 3e2af68e-5fc8-4b1c-b5bc-b4e7cab598ba.blf
-rw-r--r-- 1 root root 227696 Jun 12 16:05 032ba433-c295-11e4-a919-d4ae5275a468.blf
•
If a packet is being dropped by Snort SI the LINA capture trace shows the Verdict
> show capture CAPI packet-number 1 trace
1: 16:07:45.147743
192.168.75.14 > 38.229.186.248: icmp: echo request
Phase: 14
Type: SNORT
Subtype:
Result: DROP
Additional Information:
Snort Verdict: (black-list) black list this flow
FTD Packet Processing: SI (DNS/URL)
Security Intelligence (URL)
• Works similarly to IP Security Intelligence and provides 3 actions
1. Allow (Do-Not-Block-List)
2. Block (Block-List)
3. Monitor (Block-List)
•
•
In case Talos URL Feed is used part of the db is stored locally and updated daily
For non-cached URLs a Cloud lookup is done
Filtering URLs Based on
Category, Risk and
Reputation
OPERATIONAL
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
URL-FILTERING
Settings> Tools> Scheduling> Add Task
URL-FILTERING-DIAGRAM
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
UNCATAGORIZED URL
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Controlling File Transfer
and Blocking the Spread
of Malware
BLOCKING-ARCHITECTURE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
MALWARE-CLOUD
ARCHITECTURAL
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
MALWARE-TEXT-FILE
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Operations
Getting to know the tools
FTD Logging – ASA-level logs
ASA-level logs provide information about the FTD LINA Engine
Remote Syslog server
which receives the logs
> show running-config logging
Same as on classic ASA
> show logging
Apr 23 2017 11:11:45: %ASA-7-609002: Teardown local-host nlp_int_tap:ff02::1 duration 0:00:02
FTD Logging – Snort-level logs
Connection Events provide information about Snort Data-Plane
Table View provides flexibility –
allows addition/removal of columns
On FTD the Connection Events are stored in:
/var/sf/detection_engines/UUID/instance-X/unified_events-2.logXXX
When they get archived they go to:
/var/sf/detection_engines/UUID/instance-X/backup/unified_events-2.logXXX
FTD Logging – System logs
• Files like /ngfw/var/log/messages contain valuable information about FTD
Control and Data-Plane status
• You can use ‘system support view-files’ command from CLISH to see the
contents of files under /ngfw/var/log directory
> system support view-files
===View Logs===
============================
Directory: /ngfw/var/log
----------sub-dirs---------packages
removed_packages
...
-----------files-----------2017-04-23 13:09:14.360532 | 3148401
| messages
([b] to go back or [s] to select a file to view, [Ctrl+C] to exit)
Type ‘s’ to select a file
Type a sub-dir name to list its contents: s
Type the name of the file to view ([b] to go back, [Ctrl+C] to exit)
> messages
Specify the file name
Apr 23 04:02:14 Firepower-module1 CROND[54253]: pam_unix(crond:session): session closed
FTD CPU usage
FTD provides 2 levels of CPU usage:
• System level
> system support utilization
top - 13:00:15 up 2 days, 3 min, 0 users, load average: 24.29, 24.17, 24.15
Tasks: 719 total,
1 running, 717 sleeping,
0 stopped,
1 zombie
Cpu(s): 29.8%us, 4.9%sy, 0.0%ni, 65.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
...
PID USER
PR NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND
Expected
13672 root
0 -20 27.2g 2.5g 2.0g S 1794 1.0 51978:46 lina
13863 root
20
0 5778m 72m 15m S
2 0.0 49:11.73 SFDataCorrelator
> show cpu system
Time
CPU
%usr
%nice
%sys %iowait
%irq
%soft %steal %guest %gnice
19:53:46
all
29.79
0.00
4.92
0.04
0.00
0.01
0.00
0.00
0.00
• LINA engine level
> show cpu usage
CPU utilization for 5 seconds = 2%; 1 minute: 1%; 5 minutes: 0%
> show processes cpu-usage sorted non-zero
PC
Thread
-
5Sec
0.1%
1Min
0.1%
5Min
0.1%
Process
DATAPATH-0-5729
behavior!
%idle
65.24
FTD CPU usage
• Baseline your total CPU utilization
• Alerts about High CPU don't necessary indicate a problem unless there is also
latency and/or packet loss
• Bypass flows that don't need Snort Inspection using Prefilter Policy Fastpath
a) Big/fat flows (e.g backups, file transfers)
b) Encrypted traffic that you don't decrypt (e.g. IPSec, SSH)
• Intrusion Policy tuning and poorly-written Snort rules can affect performance.
Try "Connectivity over Security" to narrow down
FTD Memory usage
FTD provides 2 levels of memory usage:
• System level
> system support utilization
...
Mem:
8184696k total, 3583248k used,
Swap: 3998716k total,
4920k used,
2991088k free,
3993796k free,
171812k buffers
1438548k cached
total = used + free +
buffers + cached
PID USER
PR NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND
13672 root
0 -20 27.2g 2.5g 2.0g S 1794 1.0 51978:46 lina
> show memory system
total
used
free
shared
buffers
cached
Mem:
8184696
3582772
2992228
0
171828
1437868
-/+ buffers/cache:
3582772
4601924
free + buffers
Swap:
3998716
4920
3993796
+ cached
• LINA engine level
firepower# show memory [detail]
Free memory:
2645848803 bytes (57%)
Used memory:
1985224704 bytes (43%)
-----------------------------Total memory:
4631073507 bytes (100%)
Least free memory:
Most used memory:
131126033472 bytes (92%)
11392566208 bytes ( 8%)
FTD Memory usage (FDM)
• FDM Monitoring > System Dashboard
Memory utilization graph
• The same output from FTD CLI:
> system support utilization
top - 19:43:24 up 1 day, 12:09, 1 user, load average: 12.09, 12.08, 12.07
Tasks: 142 total,
3 running, 139 sleeping,
0 stopped,
0 zombie
Cpu(s): 29.8%us, 4.9%sy, 0.0%ni, 65.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem:
8184696k total, 3582780k used, 2996544k free,
171436k buffers
Swap: 3998716k total,
4920k used, 3993796k free, 1433936k cached
FTD Logging – System logs
You can run pigtail from root to select individual logs
Getting to know the tools
FTD Debugs
• FTD debugs can be categorized in 3 types
1. LINA Engine Control-Plane debugs
2. LINA Engine Data-Plane debugs
3. Snort Engine Data-Plane debugs
Health Monitor Reports for Troubleshooting
System Tuning
Performance Tuning
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
•
Iterative Process with
Firepower, may take
several tuning efforts
•
Feature Set Enabled
•
CPU, Memory,
Throughput, Latency
•
Best Practices
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Diagram courtesy of Cisco Firepower Threat Defense FTD Nazmul Rajib
Performance | Best practices
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
CPU
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
•
Trust Known traffic
•
Avoid Elephant Flows
•
Avoid Discovery of
0.0.0.0/0 network
Memory
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
•
Validate Asynchronous
Traffic and update TCP
Stream Configuration
•
Check for swap and
buffer memory using top
•
Trust Known traffic to
reduce the number of
active TCP sessions.
Throughput
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
•
Trust small packets
(Voice, DNS, ICMP)
•
Avoid Rules with generic
PCRE (Pearl Compatible
Regular Expressions)
•
Avoid Elephant Flows
•
Check Firepower
Datasheets for appliance
ratings
•
Have Cisco Sales use
the Performance
Estimator
•
Set Connection logging
DB Limits to default.
•
Always Log at End of
connection when
possible.
•
Avoid un-necessary
custom widgets on the
Dashboard.
Latency on FMC
© 2022 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Performance Tuning Policy Config Best
Practices
•
Properly Defined Variable Set
•
•
IPS Base Policy using one of the
three defaults, Connectivity,
Balanced, Security
Avoid using local/customer IPS
rules.
•
Review Performance Statistics
Graphs from FMC.
•
Do not use Maximum Detection
•
Flow-IP to eliminate TopTalkers
•
Network Discovery Policy not
set to default.
•
Avoid inspecting trusted traffic
like backups and encrypted data.
•
AC Rules configured optimally
Thank You