University of Connecticut Information Technology Security Incident Response Plan Revision History Current Revision Number: 1.00 Last Revision Date: 02/15/2010 Document Release History Date 02/15/10 Revision 1.0 Revised By Jason Pufahl/Mick DiGrazia Description Initial Document Document Ownership and Approvals The University IT Security Office has responsibility for the Incident Response Plan and is thereby responsible for its maintenance and future revisions. All revisions to this document require the explicit approval of the following: Name Date Jason Pufahl Mick DiGrazia Review Cycle This document must be reviewed at least annually to ensure its applicability to the current University environment and to address the changing technologies, organization, and needs of the University. Furthermore, upon resolution of High-Risk security incidents, the Incident Response Plan should be reviewed to determine if the plan can be revised to help provide improved incident response in the future. -i- Introduction Maintaining the confidentiality, integrity, and availability of systems and data is a risk management issue for all organizations, including the University of Connecticut. Furthermore, as more personally identifiable information is collected and systems and processes become increasingly more complex, regulations continue to place requirements for the protection of that information on the University. The University has had security incidents in the past and should expect that they will occur in the future as additional technologies with differing demands are brought into the environment to service the information technology needs of the University. The Information Technology Security Office has created this Incident Response Plan to assist in incident identification, response, and notification. Scope The University Incident Response Plan addresses events affecting any University information technology resource which may negatively impact the confidentiality, integrity, and/or availability of the resource. Because of the variety of potential incidents, any attempt to take into consideration the technical procedures required to remediate each incident would be incomplete. For that reason, this document does not attempt to outline the technical procedures associated with incident handling. The Incident Handling Plan provides a methodology or framework within which University of Connecticut incident handlers can work to ensure a complete and consistent approach to incident response. The University Incident Response Plan is designed to assist in handling security events and is not intended to address scenarios for which processes are in place such as: • • • • Violations of the Digital Millennium Copyright Act (DMCA). Excessive bandwidth usage. Incidents involving only non-University information technology resources (including student resources), such as personally-owned data or computers. Investigations of misuse of University IT resources by University employees, affiliates, or students. Audience This document is primarily for University departmental information security contacts and systems administrators with direct involvement in the identification and resolution of security incidents on the systems, data, and applications which they manage. University management may also be interested in the document in order to plan for and understand the methods and processes used to remediate security incidents within their scope of responsibility. - ii - Table of Contents Executive Overview Revision History .............................................................................................................................. i Document Release History .......................................................................................................... i Document Ownership and Approvals .......................................................................................... i Review Cycle ............................................................................................................................... i Introduction ..................................................................................................................................... ii Scope ........................................................................................................................................... ii Audience ..................................................................................................................................... ii Table of Contents ........................................................................................................................... iii Executive Overview ....................................................................................................................... iii Definitions....................................................................................................................................... 1 Incident Response Plan ................................................................................................................... 2 UCIRT......................................................................................................................................... 2 UCIRT Members .................................................................................................................... 2 UCIRT Roles and Responsibilities ......................................................................................... 2 Support Roles .......................................................................................................................... 3 Reporting and Notification ......................................................................................................... 4 Communication ........................................................................................................................... 4 Documentation ............................................................................................................................ 4 Incident Handling Details ........................................................................................................... 5 Phase 1: Preparation ................................................................................................................ 5 Phase 2: Detection................................................................................................................... 5 Phase 3: Containment ............................................................................................................. 6 Phase 4: Remediation .............................................................................................................. 6 Phase 5: Resolution ................................................................................................................. 6 Phase 6: Closure and Lessons Learned ................................................................................... 6 Step-By-Step Incident Handling Procedures .............................................................................. 7 Appendix A: Incident Handling Contacts ....................................................................................... 8 ITSO ............................................................................................................................................ 8 UCIRT Executive........................................................................................................................ 8 UCIRT Technical ........................................................................................................................ 8 UCIRT Functional ...................................................................................................................... 8 Appendix B: Incident Severity Classification Guidelines .............................................................. 9 Appendix C: Incident Reporting Methods .................................................................................... 10 Phone Numbers ......................................................................................................................... 10 Email Addresses........................................................................................................................ 10 Appendix D: Incident Handler Log Template .............................................................................. 11 Appendix E: Incident Response Quick Reference Guide ............................................................. 12 Assessing the Suspicious Situation ................................................................................... 12 If You Believe a Compromise is Likely... ....................................................................... 12 Windows Initial System Examination ............................................................................. 12 Unix Initial System Examination ..................................................................................... 12 Incident Response Communications ................................................................................ 12 Key Incident Response Steps ............................................................................................. 12 Other Incident Response Resources ................................................................................ 12 Appendix F: Incident Report Content ........................................................................................... 13 - iii - Definitions The following terms are used throughout the Incident Response Plan: Departments and Contacts Definitions Administrator A University service provider charged with managing information technology resources. User An individual who uses an information technology resource or service. ITSO ISO UCIRT ISC Information Technology Security Office: The central University Information Security Office. Information Security Officer for the University: The director of the Information Security Office. UConn Computer Incident Response Team: A team of subject matter experts assembled to respond to an incident and implement the Incident Response Plan. Information Security Contact: An individual responsible for the support and/or maintenance of an information technology resource. Incident-Related Definitions Term Definition Event Any observable occurrence within an information technology resource. Incident Any event, successful or unsuccessful, that has the potential to negatively impact the confidentiality, availability, or integrity of any UConn IT resource. Incident Response An action plan for handling the misuse of Information Technology Resources. -1- Incident Response Plan The University’s Incident Response Plan is documented to provide a well-defined, consistent, and organized approach for handling security incidents, as well as taking appropriate action when an incident at an external organization is traced back to and reported to the University. The plan identifies and describes the roles and responsibilities of the UConn Computer Incident Response Team (UCIRT), which is responsible for activating the Incident Response Plan. UCIRT The UConn Computer Incident Response Team (UCIRT), is led by the Information Security Officer (ISO) or a designee named by the ISO. The mission of UCIRT is to provide an immediate, effective, and skillful response to any unexpected incident with information security implications (i.e., negatively impacting the confidentiality, integrity, or availability of University systems or data). The UCIRT is expected to follow the Incident Response Plan and is authorized to take appropriate steps to contain and remediate an incident. UCIRT Members UCIRT members will be convened as necessary based on the incident scope and severity. UCIRT Roles and Responsibilities UCIRT – Executive Representation • Provides preparedness resources for UCIRT to respond to security incidents. • Coordinates UCIRT assignment and response to High-Risk incidents. • Determines if production services should be taken offline until incident resolution. • Enacts University Security Breach Protocol when appropriate. • Manages the incident Lessons Learned process. UCIRT – ITSO • Responds and directly handles High-Risk incidents. • Classifies High-Risk incidents according to the Incident Severity Classification Guidelines. • Provides proper training to UCIRT on incident handling. • Manages necessary communication between the UCIRT, law enforcement, and the Connecticut Education Network (CEN). • Ensures appropriate evidence gathering, chain of custody, and preservation of information/evidence by UCIRT. • Prepares a written summary of the incident, including corrective actions taken. UCIRT – Information Security Contacts (ISC) • Performs initial incident classification as High-Risk or Low-Risk. • Escalates all High-Risk incidents to the ISO. • Collects/preserves pertinent information regarding the incident. • Works to contain, remediate, resolve, and document security incidents. -2- • Determines if localized production services should be taken offline until incident resolution. UCIRT – Functional Representation • Manages necessary communication between the University, public, and media. • Provides specific expertise to: o Assist with proper evidence handling. o Ensure compliance with state and federal laws. o Ensure compliance with University policy. o Manage disciplinary actions for incidents involving University staff or students. Support Roles Administrators • Implement controls specified by Data Owners and monitors systems for signs of attack or unauthorized/inappropriate access. • Provide physical and procedural safeguards for information resources. • Performs regular system maintenance and monitoring. • Reports unexpected events to ISC, which may lead to incident handling. • Collects pertinent information regarding the incident at the request of the ISC. User • Provides background information on events which may help UCIRT and ITSO understand the cause of an incident. -3- Reporting and Notification The ITSO is the central point of contact for reporting incidents. There are several methods available for reporting incidents, listed in Appendix C. The ISO will be notified of all High-Risk incidents. If the ISO is not available, the CIO or another delegate will be notified. An analysis of the incident will take place by the ISO to determine whether UCIRT activation is necessary and determine the appropriate personnel involvement. The ISO will also prioritize incident response in the event multiple incidents require handling. Communication security incidents must be treated on a ‘need to know’ basis. Individuals who are not directly involved in the handling of the incident must not be given information about the existence of the incident or the methods used to contain or remediate the incident. Communication Low-Risk incidents must be recorded, either initially or upon resolution, in the RT system. High-Risk incidents should be recorded initially and updated throughout the incident response process in the RT system. Documentation Documentation of actions performed is expected throughout the incident handling process and is expected for all incidents. Incident handlers must record all steps performed and all changes made to production systems and observations. All documentation should be sequential and time/date-stamped, and should include exact commands entered into systems, results of commands, actions taken (e.g., logging on, disabling accounts, applying router filters, etc.) as well as observations about the system and/or incident. Documentation should occur as the incident is being handled, not after. All documentation should be provided to the ITSO upon incident resolution. A sample Incident Handler Log Template is provided in Appendix D. A list of questions that should be considered for an incident report is provided in Appendix F. -4- Incident Handling Details Although technical procedures vary depending on the categorization and type of incident, each incident must include the following six (6) phases: 1. Preparation: Ready the UCIRT to handle incidents. 2. Detection: Gather and analyze events; determine the existence of a threat and the impact to confidentiality, availability, or integrity of a UConn IT resource. 3. Containment: Stop the damage from attackers and preserve evidence. 4. Remediation: Remove artifacts left from attacker. 5. Resolution: Return systems to production and monitor. 6. Closure and lessons learned: Document findings and implement lessons learned to improve operations and/or incident handling. Based on the investigation, it may be necessary to repeat some of the phases; however, once an incident is detected the process should be followed to completion. Phase 1: Preparation The Preparation phase involves readying the UCIRT to handle incidents. Some required elements for incident handling are indicated below: • Communications • Software/Hardware • Data • Space • Documentation • Supplies • People • Training • Policy • Transportation Preparation should be done at regular intervals prior an actual incident occurring. Phase 2: Detection Incident detection occurs internally in all areas and at all levels of the University, as well as externally, through reports from non-University incident handlers. All High-Risk incidents should immediately be reported to ITSO once detected. Administrators and users must be familiar with their systems to determine if an event constitutes an incident. Effective incident detection occurs when: 1. The administrator or user is familiar with normal operations. 2. Systems are equipped with effective auditing and logging tools. 3. Administrators review systems and logs to identify deviations from normal operations. Security contacts must analyze all available information in order to understand the scope of an incident and effectively contain and remediate the incident. The incident must be fully diagnosed prior to beginning subsequent phases of the Incident Response Plan. -5- Phase 3: Containment The first priority of UCIRT, in every incident, is to contain the incident as quickly as possible. An incident is considered contained when no additional harm can be caused and the incident handler is able to focus on remediation. Containment consists of three stages: • Short-term containment: stopping the progress of the incident or attacker. • Information gathering. • Long-term containment: making changes to the production system. Phase 4: Remediation The goal of the Remediation phase is to clean up a system and remove any artifacts (e.g., rootkits) left from the attacker. During the Remediation phase, the UCIRT team must also determine and document the cause and symptoms of the incident: isolating the attack based on information gathered during the detection phase, and determining how the attack was executed. Phase 5: Resolution During the Resolution phase, the UCIRT restores normal business operations. It is critical to carefully handle incident Resolution and verify system performance and security before being brought back online. Tests must be completed and baseline system activity (gathered in the Preparation phase) must be compared to ensure the system is verified before operations are restored. Phase 6: Closure and Lessons Learned In the Closure and Lessons Learned phase, the ITSO documents findings from the incident and the handling of the incident is reviewed by the UCIRT. The expected outcome of this phase is improved operations and improved incident response procedures. -6- Step-By-Step Incident Handling Procedures The following is the process to follow once an incident is suspected. Detection: 1. Administrators or Users notify their Information Security Contact (ISC) upon discovery of potential incident. 2. ISC confirms the incident. 3. ISC classifies the incident based on criteria in Appendix B. a. ISC escalates any High-Risk incidents to be handled by the Information Security Office. b. ISC continues through the Incident Response Document to assist in handling any Low-Risk incidents. 4. Begin the documentation process. 5. Communicate incident to department management as appropriate. Containment: 6. Take necessary steps to prevent incident from spreading. 7. Document containment steps. Remediation: 8. Determine incident cause based on information gathered during the detection phase. 9. Determine how attack was executed. 10. Remove threat. 11. Perform a vulnerability assessment and remediate vulnerabilities. 12. Return systems to trusted state. Resolution: 13. Compare system against original baseline gathered during preparation phase. 14. Business units test the service/system to verify functionality. 15. Restore system to production environment. 16. Perform ongoing system monitoring to ensure system integrity and detect incident recurrence. Closure: 17. Finalize incident handling documentation. 18. Email documentation and incident description to university incident handling system. -7- Appendix A: Incident Handling Contacts ITSO Name Jason Pufahl (CISO) Mike Lang Mick DiGrazia Department UITS Information Security UITS Information Security UITS Information Security Phone# 860-486-3743 860-486-6816 860-486-1336 Cell Phone# 860-420-9897 Email Address jason.pufahl@uconn.edu mike.lang@uconn.edu mick.digrazia@uconn.edu Phone# 860-486-2096 Cell Phone# Email Address david.gilbertson@uconn.edu Phone# Cell Phone# Email Address Phone# Cell Phone# Email Address UCIRT Executive Name David Gilbertson Title Chief Information Officer UCIRT Technical Name Title UCIRT Functional Name Title -8- Appendix B: Incident Severity Classification Guidelines Upon detection incidents will be classified as one of two classes by members of UCIRT: • Low-Risk Incident • High-Risk Incident Criteria Incidents which meet any of the following criteria will be considered High-Risk Incidents: • • • • • • There is a reasonable expectation that Protected or Confidential Data was accessible to unauthorized individuals as a result of the incident (during the initial classification phase it is not necessary to determine whether or not confidential data was actually accessed). There is a reasonable expectation that the incident has or may result in financial theft or loss of intellectual property. There is a possibility that the incident has or could result in compromise of additional University systems or data. There is a possibility that physical harm could result to any person or to University property as a result of the incident. There is a possibility that the incident could affect the availability of University or department mission-critical infrastructure, systems, applications, or data. The data or systems involved in the incident are impacted by state or federal regulation, grants, or University policy. Incidents which do not meet any of the High-Risk Incident criteria should be considered Low-Risk Incidents and handled using the Step-By-Step Incident Handling Procedures. -9- Appendix C: Incident Reporting Methods Any of the following resources may be used to report an incident for investigation and remediation by the UCIRT. Phone Numbers • • • • • • (860)486-4357 (HELP), option #3: The UITS Help Center (860)486-8255: Security Incident Report Line (888)685-2637: Office of Audit, Compliance and Ethics Anonymous Reportline (860)486-4526: Office of Audit, Compliance and Ethics (860)486-4800: UConn Police Department A phone call to any member of the ITSO Email Addresses • • • security(at)uconn(dot)edu helpcenter(at)uconn(dot)edu An email sent to any member of the ITSO - 10 - Appendix D: Incident Handler Log Template Date: Incident Record (RT) Number: System/Application: Handler: Page: Note: all recorded steps must be hand-written using the form below Time Command/Action Output/Result/Observations - 11 - Appendix E: Incident Response Quick Reference Guide arp –a, netstat –nr Examine network configuration Look at autostart services chkconfig --list (Linux), ls /etc/rc*.d (Solaris), smf (Solaris 10+) lusrmgr, net users, net localgroup administrators, net group administrators List processes ps aux (Linux, BSD), ps -ef (Solaris), lsof +L1 Find recently-modified files (affects lots of files!) ls –lat /, find / -mtime -2d -ls Tips for examining a suspect system to decide whether to escalate for formal incident response. Assessing the Suspicious Situation List users and groups To retain attacker’s footprints, avoid taking actions that access many files or installing tools. Look at scheduled jobs schtasks Look at auto-start programs msconfig Look at network configuration details and connections; note anomalous settings, sessions or ports. Look at the list of users for accounts that do not belong or should have been disabled. Look at a listing of running processes or scheduled jobs for those that do not belong there. List processes Look for unusual programs configured to run automatically at system’s start time. Verify integrity of OS files (affects lots of files!) Check ARP and DNS settings; look at contents of the hosts file for entries that do not belong there. Research recently-modified files (affects lots of files!) Look for unusual files and verify integrity of OS and application files. Use a network sniffer, if present on the system or available externally, to observe for unusual activity. taskmgr, wmic process list full net start, tasklist /svc List services Check DNS settings and the hosts file ipconfig /all, ipconfig /displaydns, more %SystemRoot%\ System32\Drivers\etc\hosts sigverif 3. Containment: Stop the damage from attackers and preserve evidence Unix Initial System Examination 4. Remediation: Remove artifacts left from attacker 5. Resolution: Restore the system to normal operations, possibly via reinstall or backup. 6. Closure: Document findings and implement lessons learned to improve operations and/or incident handling wtmp, who, last, lastlog If You Believe a Compromise is Likely... Examine network configuration Look at event logs eventvwr Key Incident Response Steps Avoid using Windows Explorer, as it modifies useful file system details; use command-line. List recent security events Windows Initial System Examination If you suspect the network was compromised, communicate out-of-band, e.g. landline phones. Detection: Detect the incident, determine its scope, and involve the appropriate parties. Examine recently-reported problems, intrusion detection and related alerts for the system. Take thorough notes to track what you observed, when, and under what circumstances. Avoid sending sensitive data over email or instant messenger without encryption. 2. dir /a/o-d/p %SystemRoot%\ System32 Look at event log files in directories (locations vary) If stopping an on-going attack, unplug the system from the network; do not reboot or power down. Do not share incident details with people outside the team responding to the incident. Preparation: Gather and learn the necessary tools, become familiar with your environment. A rootkit might conceal the compromise from tools; trust your instincts if the system just doesn’t feel right. Do not panic or let others rush you; concentrate to avoid making careless mistakes. Incident Response Communications 1. /var/log, /var/adm, /var/spool Involve the ITSO for next steps, and notify your manager. rpm -Va (Linux), pkgchk (Solaris) netstat –nao, netstat –vb, net session, net use List network connections and related details Look at system, security, and application logs for unusual events. Verify integrity of installed packages (affects lots of files!) List network connections and related details List users arp –an, route print netstat –nap (Linux), netstat –na (Solaris), lsof –i more /etc/passwd Look at scheduled jobs more /etc/crontab, ls /etc/cron.*, ls /var/at/jobs Check DNS settings and the hosts file more /etc/resolv.conf, more /etc/hosts Other Incident Response Resources Windows Intrusion Discovery Cheat Sheet http://sans.org/resources/winsacheatsheet.pdf Checking Windows for Signs of Compromise http://www.ucl.ac.uk/cert/win_intrusion.pdf Linux Intrusion Discovery Cheat Sheet http://sans.org/resources/linsacheatsheet.pdf Checking Unix/Linux for Signs of Compromise http://www.ucl.ac.uk/cert/nix_intrusion.pdf Authored by Lenny Zeltser, who leads a security consulting team at SAVVIS, and teaches malware 12 analysis at SANS Institute. Special thanks for feedback to Lorna Hutcheson, Patrick Nolan, Raul Siles, Ed Skoudis, Donald Smith, Koon Yaw Tan, Gerard White, and Bojan Zdrnja. Creative Commons v3 “Attribution” License for this cheat sheet v. 1.7. Appendix F: Incident Report Content • • • • • • Executive Summary o Include overview of the incident. o Include Risk Level (High, Low). o Determine if compromise has been contained. Background o Initial Analysis. o Investigative Procedures. o Individual(s) performing investigation. o Include forensic tools used during investigation. Findings o Identify ALL systems analyzed. o Domain Name System (DNS) names. o Internet Protocol (IP) addresses. o Operating System (OS) version. Established how and source of compromise. Function of system(s) o Function of system(s). o Timeframe of compromise. o Any data exported by intruder. Remediation steps taken. 13