FortiEDR Collector Installation Secure Advance Team Feb. 2021 Rev 3.5 © Fortinet Inc. All Rights Reserved. 1 Overview - Quick and Easy Installation The FortiEDR Collector comes as a standard installer package that is easily installed and retains only a limited amount of metadata on the device in order to keep CPU usage to virtually zero and the storage requirements to a minimum. The Collector resides deep inside device’s operating system, including desktops, laptops and servers. Upon every attempt made, the FortiEDR Collector collects all required metadata and sends it to the FortiEDR Core signed by a FortiEDR digital signature. The FortiEDR Collector then holds it until authorization is received from the FortiEDR Core: • Pass: Legitimate requests are allowed. • Block: Malicious attempts are blocked. The policy enforcement is performed autonomously by the FortiEDR Collector only when the FortiEDR Core is temporarily inaccessible (State: Running Autonomously). 2 System Requirements - Collectors Verify all devices, workstations, virtual machines and servers on which a FortiEDR component will be installed comply with the system requirements provided on “Installation and Administration Guide”. Component System Requirements Processor The FortiEDR Collector runs on Intel or AMD x86 – both 32-bit and 64-bit. Hypervisors-compatible. FortiEDR is designed to use less than 1% CPU for the FortiEDR Collector. Physical Memory FortiEDR Collector requires at least 60 MB of RAM. Disk Space FortiEDR Collector installation requires at least 20 MB of disk space. Supported Operating Systems The FortiEDR Collector can be installed on any of the following operating systems (both 32-bit and 64-bit versions): • Windows XP SP2/SP3, 7, 8, 8.1and 10. • Windows Server 2003 SP2, R2 SP2, 2008 SP2, 2008 R2 SP1, 2012, 2012 R2, 2016 • and 2019. • MacOS Versions: Yosemite (10.10), El Capitan (10.11), Sierra (10.12), High Sierra • (10.13), Mojave (10.14), Catalina (10.15) and Big Sur (11). • Linux Versions: RedHat Enterprise Linux and CentOS 6.8, 6.9, 6.10, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7 and 7.8 and Ubuntu LTS 16.04.5, 16.04.6, 18.04.1 and 18.04.2 server, 64-bit only. • VDI Environments: VMware Horizons 6 and 7 and Citrix XenDesktop 7. • The FortiEDR Core, Repository Server, FortiEDR Aggregator and FortiEDR Central Manager components are supplied in ISO format, which includes a CentOS 7 image. FortiEDR Core, FortiEDR Aggregator and FortiEDR Central Manager can be installed on a virtual machine or a dedicated workstation or server. Please, check admin guide or contact to support for any update in Supported OS NOTE: For windows 7 and 2008 servers you will require the following Security Updates: • Windows 7 - KB2921916, KB3033929 • Windows Server 2008 - KB4474419 3 Components Connectivity The FortiEDR platform is comprised of the following components. The connections are established between the FortiEDR Collector and other FortiEDR components: • To the FortiEDR Aggregator: The FortiEDR Collector initially sends registration information to the FortiEDR Aggregator via SSL and then it sends ongoing health and status information. • From the FortiEDR Aggregator: The FortiEDR Collector receives its configuration from the FortiEDR Aggregator. • To the FortiEDR Core: The FortiEDR Collector sends compressed operating system metadata to the FortiEDR Core and then ongoing health and status information. • From the FortiEDR Core: The FortiEDR Collector receives connection establishment authorization or denial (blocking) from the FortiEDR Core. FortiEDR Core FortiEDR Aggregator FortiEDR Collector Port: 555 Port: 8081 FortiEDR Cloud Services 4 Connectivity Requirements Ensure that these ports are not blocked by your firewall product (if one is deployed). Component System Requirements Connectivity • • • • • Supported Browsers The FortiEDR Central Manager console can be accessed using the Google Chrome, Firefox Mozilla, Microsoft Edge and Apple Safari browsers FortiEDR Core listens to communication on port 555. FortiEDR Aggregator listens to communication on port 8081. Browser connection to the FortiEDR Core is via port 443. Network connectivity between all system components is required. Allow up to 5 Mbps of additional network workload for each 1,000 Collectors. Connectivity - please verify that the following ports are open / not blocked by other security products: IP:Port -> X.X.X.X:555 between the Collector and the Core IP:Port -> X.X.X.X:8081 between the Collector and the Aggregator DNS:Port -> *.ensilo.com:8081 between the Collector and the Aggregator DNS:Port -> *.ensilo.com:443 for the Central Manager UI console access reputation.ensilo.com should be opened to allow connection for FortiEDR Cloud Services (FCS) As a security best practice, it is recommended to update the firewall rules so that they only have a narrow opening. For example: • Only open the TCP outbound port 555 to the Core IP address. • Only open the TCP outbound port 8081 to the Aggregator IP address 5 Connectivity List 6 Requesting and Obtaining a Collector Installer - I In the Licensing window you can click the button to obtain a Collector installer file that can be used to install a Collector. The requested installer is then emailed to you. 7 Requesting and Obtaining a Collector Installer II The FortiEDR Collector comes as a standard installer package that can be easily customized and emailed to the specified email address. 1. Select the checkbox of the installer(s) you want to Request and specify the version for each of them. Multiple installers can be requested at the same time 2. Select the aggregator to which this Collector is registered. NOTE: please choose the DNS option when selecting the aggregator 3. In a multi-tenant system, select the organization 4. Select the Collector Group to which the installed Collector is assigned, or leave the field empty for the Collector to be assigned to the default Collector Group 5. Advanced area, specify the following: - VDI (Virtual Desktop Infrastructure) Installation: If you are installing the Collector on a VDI environment - Use System Proxy Settings: If you use a web proxy to filter requests in this device’s network - Enforce Reboot: Check this checkbox in order to delay data collection until a device reboot is applied. This is only required in rare cases. NOTE: As a security best practice, in advance, it is recommended to save a downloaded collector Installer without any custom setting and replace it after new updated versions. This allows you to have always an installer when the Link to download installers is not immediately available. 8 Requesting and Obtaining a Collector Installer III After the installer is generated by FortiEDR, it is emailed to the specified email address. Note that the link to download installers is only available for several hours. Be sure to download the installers within the required time period so that the link does not expire. Hoster View name: Organization name: To extract the installer from the zip file you need the Device Registration Password. Please check slide 18 “Device Registration Password” NOTE: Files will come from ens-ecs-inst@fortinet.com or DoNotReply@fortinet.com, so please make sure emails from that address are not filtered/blocked. 9 Creating Custom Collectors - Manually You can also create a custom installer for your customers. Contact Fortinet support (at https://support.fortinet.com) to be provided with The Silent Installer Generator utility and empty MSI files. 1. Open the Silent Installer Generator file (FortiEDRCollectorSilentInstallerGenerator_#.#.#.###.exe). 2. (Only for Windows) Therefore, be sure to create a copy of the empty MSI files before running the Silent Installer Generator 3. (Only for Windows) In the browser window, locate the copied MSI files that you prepared. The Silent Installer Generator requires empty fields. The target MSI files must not have been previously modified. 4. Fill in the Aggregator DNS/address:port fields, Registration Password, Collector Groups. NOTE: please choose the DNS option when selecting the aggregator (recommended) You will see the date and timestamp change on the files that you just modified NOTE: Request empty MSI files after any collector installer update available. 10 Before you start – Update / Patch • Update: Keeping your servers or devices up to date with patches is a requirement to ensure a proper level of security. Making sure your devices are "correctly patched and updated" is a necessary step towards being fully protected. Servers that run out of date and insecure versions of software are responsible for the majority of compromises. NOTE: Old Windows 7 and Server 2008 don’t support SHA256 code signing Require Windows update or specific KB installation: • Windows 7 - KB2921916, KB3033929 • Windows Server 2008 - KB4474419 Our SHA1 code signing has expired in the beginning of January 2020 and it cannot be re-issued. The new certificate is signed with SHA256, which may not be accepted by old Win7 and Server 2008 that were not patched since 2015. On such old and non-patched devices the installer/upgrade will fail and the device will stay running with older Collector version. • On Windows 7 one can overcome this problem by patching the device with KB2921916. • https://fortiedr.sharefile.com/share/view/s6170af9accfd46f283910bc16a5789a9 • On Windows Server 2008 R2 SP1 one can overcome this problem by patching the device with KB2921916 • On Windows Server 2008 R2 (no SP) and Windows 7 starter there is no patch by Microsoft to overcome this issue. More information available here: https://docs.microsoft.com/en-us/security-updates/SecurityAdvisories/2015/3033929?redirectedfrom=MSDN 11 Before you start – Take Backups / Snapshots • Backup: While not strictly a security measure, backups can be crucial in saving compromised systems and data. You should always create the Restore Point and continue installing any application (including FortiEDR Collectors). If it causes issues on the system, you can always go back to the Clean installation restore point. 12 Before you start – Running with other AV - I Multiple antivirus programs will interfere with each other if they're installed on the same system. Using the default Windows removal tool to uninstall your current AV is often insufficient. In some situations, a legitimate antivirus program cannot uninstall because it or the software used to uninstall it is corrupt, bad, or missing. In these situations, you must download an uninstaller program from the antivirus scanner company to remove all files associated with the antivirus. List of anti-malware product removal tools: https://answers.microsoft.com/en-us/protect/forum/protect_other-protect_start-windows_other/list-of-antimalware-product-removal-tools/2bcb53f7-7ab4-4ef9-ab3a-6aebfa322f75 If you decide to remove your AV products on your system prior to the installation of FortiEDR, you should use the remover tool from former antivirus scanner company. On the other hand, keeping you actual AV program requires to exclude the path directories described in the following slide. 13 Before you start - Running with other AV - II We have a set of paths we recommend excluding if you plan to run other AVs in parallel. NOTE: the folder path name depends on the installer version used: - Version 3.x: “enSiloCollector” - Version 4.x: “FortiEDRCollector” Paths/Directories to exclude for Windows (4.x version): %ProgramData%\FortiEDR\ %ProgramFiles%\Fortinet\ %ProgramFiles%\Fortinet\FortiEDR\FortiEDRCollector.exe %ProgramFiles%\Fortinet\FortiEDR\FortiEDRCollectorService.exe %windir%\System32\drivers\FortiEDRAvDriver_*.sys %windir%\System32\drivers\FortiEDRBaseDriver_*.sys %windir%\System32\drivers\FortiEDRElamDriver_*.sys %windir%\System32\drivers\FortiEDRIotDriver_*.sys %windir%\System32\drivers\FortiEDRWinDriver_*.sys Paths to be excluded for MacOS (3.x version): - entire /Library/enSilo directory or - /Library/enSilo/enSiloCollector /Library/enSilo/enSiloCollectorTray /Library/enSilo/enSiloConfig If driver has to be whitelisted, then /Library/enSilo/enSiloDriver or /Library/Extensions/enSiloDriver.kext Paths to be excluded for Linux (4.x or 3.x version): /sbin/FortiEDRCollector or /sbin/enSiloCollector and anything under /opt/FortiEDRCollector or /sbin/enSiloCollector 14 Before you start – System Components Ensure that the status of the Cores, Aggregators, and FCS is Running. This is essential before any installation. NOTE: For Collector being able to Run Autonomously, previously it had to connect to FortiEDR Core and Aggregator. 15 Before you start – License Capacity - I The License Capacity field in the Licenses window shows the total number of license seats for the entire FortiEDR system, which are divided into Workstations, Servers and IoT Devices. The Default (hoster) organization initially receives the total allocation of licenses. The Administrator is responsible for allocating these licenses among organizations. In a single-organization FortiEDR system, licenses do not need to be allocated between organizations, as there is only one organization. 16 Before you start – License Capacity - II Before installing Collectors, you need to verify the number of available licenses that are remaining. Only the number of FortiEDR Collectors allowed by the license can register with the FortiEDR Central Manager. Additional FortiEDR Collectors cannot register with the FortiEDR Central Manager and will lose protection. 17 Before you start – Device Registration Password The device registration password is required in order to install or uninstall components from the system. 18 Installing a FortiEDR Collector on Windows Run the FortiEDR Collector installation file. • FortiEDRCollectorInstaller32.msi file if you are using a 32-bit operating system • FortiEDRCollectorInstaller64.msi file if you are using a 64-bit operating system. Collector Configuration will be filled with all custom settings previously selected in the request of Collector Installer. You can fill them if empty or change them is as necessary. For a multi-organization FortiEDR system, enter the name of the organization in the Organization field. Windows may possibly display a message requesting that you confirm the installation. Please do so. 19 Installing a FortiEDR Collector on MAC - I Double-click the *.dmg file named FortiEDRCollectorInstallerOSX_1.3.0.xxx.dmg. Collector Configuration will be filled with all custom settings previously selected in the request of Collector Installer. You can fill them if empty or change them is as necessary. For a multi-organization FortiEDR system, enter the name of the organization in the Organization field. 20 Installing a FortiEDR Collector on MAC - II Starting 10.14.5, Apple requires all new macOS apps to be notarized in order for them to run. On installation from scratch or in case the operating system was upgraded to 10.14.5 before collector was upgraded to this build 113, user must approve enSilo's in 'Security Preferences' in order for collector to run on the OS: 21 Installing a FortiEDR Collector in Linux - I Before installing the collector, you should confirm the supported OS and Kernel version: 1. Type any one of the following command to find OS name and version in Linux: cat /etc/os-release lsb_release -a hostnamectl 2. Type the following command to find Linux kernel version: uname -r You should confirm and provide the build and kernel version of the Ubuntu or CentOS servers to Fortinet support (at https://support.fortinet.com) on the availability of the supported Collector. Check next slide to verify the actual supported linux and kernel version. 22 Installing a FortiEDR Collector on Linux - II Linux - Supported versions and kernels • Ubuntu LTS 16.04.5, 16.04.6 server o kernel versions 4.4.0-131, 4.4.0-142, 4.4.0-145, 4.4.0-169, 4.4.0-173, 4.4.0-184 • Ubuntu LTS 18.04.1, 18.04.2, 18.04.3, 18.04.4 server o kernel versions 4.15.0-34, 4.15.0-36, 4.15.0-54, 4.15.0-55, 4.15.0-66, 4.15.0-70, 4.15.0-72, 4.15.0-74, 4.15.0-76, 4.15.0-88 generic, 4.15.0-108 generic o kernel versions 5.3.0-59, 5.3.0-61 generic • Ubuntu LTS 20.04, 20.04.1 server o kernel versions 5.4.0-42, 5.4.0-47, 5.4.0-48 generic • RHEL (CentOS) version 6: o CentOS & RHEL 6.8 - 2.6.32-642 o CentOS & RHEL 6.9 - 2.6.32-696 o CentOS & RHEL 6.10 - 2.6.32-754 • RHEL (CentOS) version 7: o CentOS & RHEL 7.2 - 3.10.0-327 Please, contact to Fortinet support o CentOS & RHEL 7.3 - 3.10.0-514 (at https://support.fortinet.com) for any o CentOS & RHEL 7.4 - 3.10.0-693 update o CentOS & RHEL 7.5 - 3.10.0-862 Use always the last installer version for o CentOS & RHEL 7.6 - 3.10.0-957 supported versions and kernels o CentOS & RHEL 7.7 - 3.10.0-1062 o CentOS & RHEL 7.8 - 3.10.0-1127 • RHEL (CentOS) version 8: o CentOS & RHEL 8.0 - 4.18.0-80 o CentOS & RHEL 8.1 - 4.18.0-147 o CentOS & RHEL 8.2 - 4.18.0-193 o CentOS & RHEL 8.3 - 4.18.0-240 23 Installing a FortiEDR Collector on Linux - III Unzip the file: gzip -d [FILE NAME] Give it executable permissions chmod +x [FILE NAME] Extract the content of the package (after gunzip): ./FortiEDRSilentInstall_X.X_.sh --noexec --target /tmp Create the folder path. mkdir -pv FortiEDRCollector/Config/Collector Copy the file CollectorBootstrap.jsn to folder path cp /tmp/CollectorBootstrap.jsn /opt/FortiEDRCollector/Config/Collector Run the FortiEDR Collector installation file for 64-bit servers using the following command: • CentOS: sudo yum install FortiEDRCollectorInstaller_%Linux_distribution%-%version_number%.x86_64.rpm • Ubuntu: sudo apt-get install FortiEDRCollectorInstaller_Ubuntu-%version_number%.deb NOTE: the folder path name depends on: After the installation is completed, run the following: - Version 3.x: “enSiloCollector” sudo /opt/FortiEDRCollector/scripts/fortiedrconfig.sh - Version 4.x: “FortiEDRCollector” - Specify the FortiEDR Aggregator domain name or IP address. - Enter the FortiEDR Aggregator port information (usually 8081). - For a multi-tenant setup, enter the organization. Otherwise, leave the organization empty. - Enter Collector Group information or leave empty to be registered to the default Collector Group. - Enter the device registration password - Do you want to connect via proxy (Y/N)? prompt, type Y if your setup includes a web proxy. Finally, you can check the status with the following command: /opt/FortiEDRCollector/control.sh --status 24 Installing a FortiEDR Collector on Linux - IV For FIPS, Security-Enhanced Linux and Secure Boot Kernel Module Signature: Run the FortiEDR Collector installation file for 64-bit servers using the following command: rpm -ivh --nodigest --nofiledigest /opt/FortiEDRCollectorInstaller_CentOS8-4.1.0-674.x86_64.rpm Verify status. updatedb Locate control.sh /opt/FortiEDRCollector/control.sh --status SELinux preventing module load access: Click YES to generate a local policy module to allow access Fortinet Secure Boot Kernel Module Signature: Follow the steps describe in the link (https://fortiedr.sharefile.com/share/view/sf3297d655b4f492dbf3d4fb8d997d336). Public Key Enrollment: The user is prompted to enter a password. It is very important to remember this password because it is required in the next steps after reboot The user is requested to reboot the machine and run the following command: reboot –f When the machine loads, the first screen to display is a blue screen (shim screen) asking to Hit Any Key to continue the process.This screen disappears quite quickly, so you should stay in front of the screen in order to press the key While the shim screen is displayed, select "Enroll Mok". A new screen displays with two options. Select "Continue" and press enter to continue to the next step A new screen displays asking for the password that was entered before the reboot (at step a). Please enter the password. The machine now reboots with key in the kernel key chain, and the modules signed with it such that the FortiEDR collector is loaded successfully. Verify status. /opt/FortiEDRCollector/control.sh –status After the installation is completed, run the following: /opt/FortiEDRCollector/scripts/fortiedrconfig.sh 25 Verifying Collector Installation - I 1. Look for the collector status icon in the System Tray. Each device protected by FortiEDR can display an icon in the system tray to indicate its state. The FortiEDR icon indicates the current state of the device, as follows: System Tray icon with Collector Status: • Protection On: The FortiEDR Collector is up. • Protection Off/Disable: FortiEDR Collector was disabled in the FortiEDR Central Manager. • Degraded: Specifies that the FortiEDR Collector is prevented from performing to its full capacity • Isolated: Device is isolated (blocked) from communicating with the outside world (for both sending and receiving). Linux – • /opt/FortiEDRCollector/control.sh –status 26 Verifying Collector Installation - II 2. Search for the name of the computer in the collector sub-tab under Inventory in the Central Manager. • Running: The FortiEDR Collector is up and all is well. • Running (Autonomously): The Core is temporarily inaccessible. • Disconected: The device is offline, power down, or not connected to the Aggregator. • Disconected (Expired): The device has not connected to the system in more than 30 days • Disabled: The FortiEDR collector was disable • Degraded: The FortiEDR collector is prevented from performing to its full capacity • Unmanaged: The FortiEDR collector is NOT installed 27 Verifying Collector Installation - III 3. Selecting SYSTEM EVENTS in the ADMINISTRATION tab displays all the events relevant to the FortiEDR system, including Collectors. 28 Verifying Collector Installation – IV (Windows) 4. Look for FortiEDR collector in the Task Manager (PC) or Activity Monitor (Mac): 5. Check Windows Services to verify that the FortiEDR Collector Service is running, as shown below: NOTE: In general, a FortiEDR Collector does not require the device on which it is installed to reboot after its installation. However, in some cases, you may want to couple the installation of the FortiEDR Collector with a reboot of the device. Rebooting may prevent a threat actor from attempting to exfiltrate data on a previously existing connection that was established before installation of the FortiEDR Collector. 29 Verifying Collector Installation – Degraded Degraded shows that there is some sort of issue that is preventing the collector from performing at full capacity. It may be caused by compatibility issues or a lack of resources on the device. This should be investigated to identify the source of the problem. If you hover the cursor over the collector state, you will see a brief description of the problem, which can help you determine the best course of action. Remember to Select SYSTEM EVENTS in the ADMINISTRATION tab to display all collector events. Warnings Examples state was changed to "Degraded“: • There is no available configuration. • The Collector driver could not properly load. • Please approve Fortinet Endpoint Protection and Response Platform Kernel extension in the device. • Collector version update failed. • Unsupported operating system version. • Lost connection. 30 Verifying Collector Installation – Running (Autonomously) Autonomous mode refers to a status where the Collector cannot connect to the Core either entirely or its Connection to the Core suffers from multiple timeouts/errors. In this scenario the Collector continues to run and protect the device. The Collector is the one setting its status as "Running Autonomously" in case it can't communicate with the Core for 1 minute. When the Collector fails to establish a connection with either the Core or the Aggregator it keeps trying to establish a connection every few seconds to few minutes. Time varies according to the amount of errors on previous tries. The Collector’s keep alive interval is 30 seconds from either the last keep alive request or the last event sent form the Collector to the Core (TCP port 555). 31 Verifying Collector Installation – Running (DEP disabled) Data Execution Prevention (DEP) is a system-level memory protection feature that is built into the operating system starting with Windows XP and Windows Server 2003. DEP enables the system to mark one or more pages of memory as non-executable. Marking memory regions as non-executable means that code cannot be run from that region of memory, which makes it harder for the exploitation of buffer overruns. DEP prevents code from being run from data pages such as the default heap, stacks, and memory pools. If an application attempts to run code from a data page that is protected, a memory access violation exception occurs, and if the exception is not handled, the calling process is terminated. In 32-bit Windows running on x64-based systems, PAE also enables several advanced system and processor features, including hardware-enabled Data Execution Prevention (DEP). Without DEP Windows is extremely vulnerable to buffer overrun attacks and must be enabled in any secure modern operating system. It is highly recommended to turn it ON HOW TO ENABLE: https://www.dell.com/support/article/en-il/sln288643/what-is-data-execution-prevention-dep?lang=en https://support.microsoft.com/en-us/help/912923/how-to-determine-that-hardware-dep-is-available-and-configured-on-your 32 Verifying Network Communications – I (Windows) One Common deployment issue is when Collector can not communicate. This is normally due to a blocked port along the way. If you do not see connection, make sure there are no firewall blocking these ports. 1. Check existing connections to the Core and Aggregator. (Type netstat –an to get a full list of connections). On a Windows PC run C:\Users\..>netstat -ano 10 | find /i "8081" TCP 192.168.1.112:61741 X.X.X.X:8081 ESTABLISHED 4932 C:\Users\..>netstat -ano 10 | find /i "555" TCP 192.168.1.112:61704 X.X.X.X:555 ESTABLISHED 4 On a linux run: [root@dev-edr-core ~]# netstat -ln You should see connection Established. 2. Run telnet and check network connectivity . On a Windows PC run C:\Users\..>telnet X.X.X.X 8081 (IP Aggregator + Port 8081) C:\Users\..>telnet X.X.X.X 555 (IP Core + Port 555) You should see a blank command prompt labeled Telnet X.X.X.X. This means the connection was successful 33 Connectivity Test Utility Contact Fortinet support (at https://support.fortinet.com) to be provided with a connectivity test utility ConnectivityTestApp.exe. This utility simulates a simple security event generated from the FortiEDR Collector. • Place the provided ConnectivityTestApp.exe utility on the communicating device. • Make sure Windows Defender or AV will allow run it as administrator. • Run it by double-clicking. • Upon activation of the utility: - A popup message displays execution blocked. - A security event should appear on the FortiEDR Central Manager . 34 Event Viewer Test - I (Windows) The Windows Event Viewer records whenever a FortiEDR Collector blocks communication from a device or file modification related to ransomware activity. This information is recorded in the Windows Event Viewer log located in the following location: Event Viewer 🡪 Windows Logs 🡪 Application. 🡪 Filter Current log 🡪 FortiEDR Collector 35 Event Viewer Test - II (Windows) NOTE : The Connectivity Test Utility should be recorded in the Windows Event Viewer log 36 Event Viewer Test - III (MacOS) The MacOS console records whenever a FortiEDR Collector blocks communication from a device or file modification related to ransomware activity. This information is recorded in the MacOS console log located in the following location: Applications 🡪 Utilities 🡪 Console 🡪 All Messages 37 Upgrading the Collector – Remotely After a Collector has been installed in the system, you can upgrade it remotely from the Licensing panel of Administration tab. We recommend deploying Collector updates in smaller batches (by groups and OS) to avoid any problems. FortiEDR gradually updates all the Collectors. 38 Upgrading the Collector – Automatic Updates The Automatic Collector Updates feature updates the revision for a given FortiEDR version. This means that all Collectors in all Collector Groups in all environments and operating systems are updated to the latest FortiEDR revision available. 39 Upgrading the Collector - Manually To upgrade the Collector manually (not via the user interface): Windows Open the FortiEDRCollectorInstallaler32_x.x.x.xxx.msi or FortiEDRCollectorInstallaler64_x.x.x.xxx.msi file and follow the displayed instructions. Linux 1. Copy the installer file to the Collector machine: - CentOS: FortiEDRCollectorInstaller_%Linux_distribution%-%version_number%.x86_64.rpm - Ubuntu: FortiEDRCollectorInstaller_Ubuntu-%version_number%.deb). 2. Stop the Collector using its password: NOTE: the folder path name depends on: - Version 3.x: “enSiloCollector” /opt/FortiEDRCollector/control.sh --stop <registration password> - Version 4.x: “FortiEDRCollector” 3. Do one of the following: - CentOS: Run sudo yum install FortiEDRCollectorInstaller_%Linux_distribution% %version_number%.x86_64.rpm. - Ubuntu: Run sudo apt-get install FortiEDRCollectorInstaller_Ubuntu-%version_number%.deb. 4. Answer Y when asked if you want to upgrade. 40 Uninstalling Collector - Remotely From the Central Manager Inventory page (recommended). Use the button to uninstall a Collector from a device. Use caution when using this option, as a Collector cannot be reinstalled remotely after removal using the FortiEDR user interface. You can uninstall a FortiEDR Collector from a device and then delete it from the FortiEDR INVENTORY (page 73) if you would like to add another FortiEDR Collector. When a user attempts to uninstall the Collector from a Windows OS device, he/she must supply the registration NOTE: For troubleshooting, it is recommended to disable a Collector using the Enable/Disable option rather than uninstalling it. 41 Uninstalling Collector – Locally - I You can uninstall a FortiEDR Collector from a device and then delete it from the FortiEDR INVENTORY if you would like to add another FortiEDR Collector. When a user attempts to uninstall the Collector from a Windows OS device, he/she must supply the registration password. You can find this password in the FortiEDR Management Console. If the collector is still listed as Runing, wait a minute or two, and then refresh the browser. 42 Uninstalling Collector – Locally - II You can uninstall a FortiEDR Collector using the custom package Installer. Open the FortiEDRCollectorInstallaler32_x.x.x.xxx.msi or FortiEDRCollectorInstallaler64_x.x.x.xxx.msi file NOTE: In order to stop manually the FortiEDR service from running on a Windows OS device, enter the following command: C:\Program Files\Fortinet\FortiEDR\FortiEDRCollectorService.exe --stop and then provide the registration password in the pop-up windows. 43 Removing all local Configuration Files After uninstalling or any installation Errors, you may want to check that all local configuration files were removed before installing collector again. Run Cleanup Tool: Contact Fortinet support (at https://support.fortinet.com) to be provided with FortiEDRCleanupTool.exe 44 Verifying Collector was unistalled Like when a new collector is added to the system, you can track events affecting the protection of user devices in System Events. 45 Deleting a Collector You should delete uninstalled collectors from management console to reduce clutter. Deleting a collector does not uninstall it from the device. If the collector is still present on the remote machine, it will re-register. NOTE: you must always uninstall the collector before deleting it. 46 Exporting logs The Export Logs feature enables you to retrieve technical information from the FortiEDR devices deployed in the organization, such as from Collectors, Cores, Aggregators and the Management server. The retrievable technical content describes the activities of each FortiEDR device. Typically, the technical content contains logs and statistical information. Click the Download link and send the retrieved logs to Fortinet technical support. If the device is offline, obtain the logs locally (see next slide). NOTE: If you downloaded the collector logs through the management console, they are encrypted, so you will not be able to analyze them yourself. You should send them to Fortinet technical support to decrypt and analyze them for you. 47 Obtaining Collector logs locally NOTE: the folder path name depends on: Windows: - Version 3.x: “enSiloCollector” - \ProgramData\FortiEDR - Version 4.x: “FortiEDRCollector” - Locate Windows Task Scheduler and Windows system logs at: - Windows XP: \Documents and Settings\All Users\Application Data - Windows Vista and later: C:\Windows\Tasks. - ProcMon tool: In rare occasions, you will need to run ProcMon (Sysinternals tool from Microsoft) while performing the installation. Save the ProcMon log to a PML file and get the installer logs using the parameter: /l*vx log.txt Mac: - /Library/FortiEDR/Logs/Collector - / Library/FortiEDR/Logs/Driver Linux: - Linux 2.6x: /opt/FortiEDR/logs - Linux 2.7x: /opt/FortiEDRCollector/logs /opt/FortiEDRCollector/control.sh --status Please, send the retrieved logs to Fortinet technical support for further analysis. 48 Troubleshooting Guide After a FortiEDR Collector is first launched, it registers with the FortiEDR Central Manager and is displayed in the INVENTORY tab. If it does not appear to have registered, then perform the following: 1. Check that the device on which the FortiEDR Collector is installed is powered on and has an Internet connection. 2. Require Windows update or specific KB installation, and Review OS requirements. 3. Installer requires Administrator rights. Also, verify device registration password is correct. 4. Ensure that the status of the Cores, Aggregators, and FCS is Running. 5. Validate License Capability for Workstations, Servers and IoT. License exceeded or expired. 6. Validate that ports 8081 and 555 are available and that no other third-party product is blocking these ports. 7. Perform a connectivity test in order to validate connectivity between all of the FortiEDR components: FortiEDR Collector 🡪 FortiEDR Core 🡪 FortiEDR Aggregator 🡪 FortiEDR Central Manager Console. 8. Choose the DNS option when selecting the aggregator. If you still have issues try using the IP address. 9. In general, a FortiEDR Collector does not require the device to reboot after its installation. However, in some cases, you may want to couple the installation of the FortiEDR Collector with a reboot of the device. 10. Validate Warning Messages from SYSTEM EVENTS in the ADMINISTRATION tab for all collector events. 11. Check for new installer versions. Always install the last available updates. 12. Verify the set of excluding paths if you plan to run other AVs in parallel or uninstall it using removal tools. 13. Use the connectivity test utility. ConnectivityTestApp.exe. 14. Uninstall previous version fail - Windows msi DB or Registry issues -> run cleanup tool: FortiEDRCleanupTool.exe 15. Remove all local Configuration files before reinstalling. 16. Retrieve logs and ProcMon logs from devices and send them to Fortinet Technical Support. 49 Crashes and Blue Screen Of Death (BSOD) A system crash (also known as a “bug check” or a "Stop error") occurs when Windows can't run correctly. The dump file that is produced from this event is called a system crash dump. A manual kernel or complete memory dump file is useful when you troubleshoot several issues because the process captures a record of system memory at the time of a crash. By default, you can get a full memory dump from C:\Windows\MEMORY.DMP and summit it to Fortinet support (at https://support.fortinet.com). You should also send the crash dump: C:\Windows\system32\crashdumps Here are the steps for configuring Windows to create full dumps: (https://docs.microsoft.com/en-us/windows/client-management/generate-kernel-or-complete-crash-dump). In frozen state to see where it is stuck and collect the memory dump we can force a Kernel memory dump/Complete memory dump from a keyboard on a Windows 10. (https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/forcing-a-system-crash-from-thekeyboard) NOTE: If you can not reconnect in safe mode ask Fortinet Support for guidance to remove FortiEDR BaseDriver. 50 References • FortiEDR Installation and Administration Guide Version 4.2 • FortiEDR Study Guide 4.0 • FortiEDR Cloud-based PoC Guidelines • FortiEDR Troubleshooting Version 4.1 (Ido Kelson) 51