GUIDANCE Resilient Backups For many organizations the loss or unavailability of critical data results in significant costs, often rising sharply over short periods of time. These costs can extend beyond immediate considerations such as the lack of productivity or lost revenue, to long-term reputational damage, long-term loss of market share or business extinction in some cases. It is important that businesses consider their own risk tolerance and the availability requirements of their critical data, and choose a backup solution that is best matched to their needs. Understanding the organisation's most critical business functions is crucial, as it allows for mitigations to be put in place and the survival of the business during disruption related events. This document remains the property of the Australian Government. The information contained in this document is for the use of the intended recipient only and may contain confidential or privileged information. If this document has been received in error, that error does not constitute a waiver of any confidentiality, privilege or copyright in respect of this document or the information it contains. This document and the information contained herein cannot be disclosed, disseminated or reproduced in any manner whatsoever without prior written permission from the Assistant Secretary, E-Security Policy and Coordination, Attorney-General's Department, 3 - 5 National Circuit, Barton ACT 2600. The material and information in this document is general information only and is not intended to be advice. The material and information is not adapted to any particular person’s circumstances and therefore cannot be relied upon to be of assistance in any particular case. You should base any action you take exclusively on your own methodologies, assessments and judgement, after seeking specific advice from such relevant experts and advisers as you consider necessary or desirable. To the extent permitted by law, the Australian Government has no liability to you in respect of damage that you might suffer that is directly or indirectly related to this document, no matter how arising (including as a result of negligence). Executive Summary It is important for businesses to have resilient backup solutions in place. There have been a number of recent media articles highlighting the real world business impacts of data loss – losses that may have been lessened or mitigated if robust backup processes and architectures had been in place. The aim of this document is to provide guidance on: the importance of effective backups issues to consider when assessing backup options, and issues to consider when assessing backup effectiveness. Simply having a backup solution in place may not be sufficient. Organisations must understand their specific business needs, the limitations of their current backup solution and how to manage those limitations. Consideration of whether the solution provides adequate business continuity and resilience in the context of the organisation’s risk appetite is paramount. Resilient Backups For many organisations the loss or unavailability of critical data results in significant costs, often rising sharply over short periods of time. These costs often go beyond immediate considerations of lack of productivity or lost revenue to long-term reputational damage, long-term loss of market share or business extinction in some cases. It is important that businesses consider their own risk tolerance and the availability requirements of their critical data, and choose a backup solution that is best matched to their needs. Understanding the organisation's most critical business functions is also important so that mitigations can be put in place to ensure the survival of the business during disruption related events. Considerations for In-House Backup Solutions In-house back-up solutions are primarily considered by medium to large enterprises, although some small businesses pursue them as well. There is an increasing trend away from tape backup solutions to real-time replication. Although this presents a number of speed and flexibility advantages, it is not without risk to rely on this solution for a primary source of backup as discussed in the previous section. You should consider utilising both real-time replication and offline backup; replication for agile recovery from data centre outages and offline backup to mediums such as tape for recovery in the event of corruption or compromise. Backup frequency and retention periods It is important to understand the impact of not having access to critical and non-critical data for a set period of time, or permanently in the case of complete loss, will have on the ongoing operation of your business. Understanding this will help to set backup schedules and procedures to best support your business operations. In particular it is important to understand the impact on your business if you were to lose all data that has been created or changed since your last backup. Data corruption is a commonly discussed example, but what if the servers that contained your critical data were seized by law enforcement as part of investigating an intrusion? Would your business be in a position to resume critical business functions within an acceptable timeframe? Risks such as these should be considered and incorporated into your backup and restoration plans. Also consider the different classes of data your business stores and uses according to the levels of criticality these data sources represent to your business. This will assist in determining backup budget allocations and ensure that data is protected based on its risk and value to the organisation. For some businesses, it may be acceptable to lose a week’s worth of data (such as email or transactional information) and for others the loss of more than 1 hour of data may be unacceptable. In both cases, different backup processes and procedures will be required along with differing backup solutions. Understanding organisational requirements for data retention is also of importance. Are backups replaced every week, month, or perhaps year? Especially in the case of corruption or compromise, you may not discover that your organisation has been impacted for at least months. If you do not have a “known clean” backup, you may have to rebuild your operating environments from their foundations. The question of what point it is safe to restore data backups such as email and office documents will be even harder to answer. Adequate retention rates based on your organisation's risk profile and business requirements will enable more predictable recovery options. Some reports1 state that up to 60% of breaches remain undiscovered for months or more, which may provide some insight into how long individual backups should be retained for. You should also consider scanning your backups with anti-virus software before restoration if practicable, using the latest signatures available. Backup security Backed-up data represents a concentration of business information sitting at rest, often for extended periods of time. You should consider encrypting this information to provide an additional layer of protection in case of theft or compromise. Backed-up data stores are generally more vulnerable than the servers the information originated from and so should be risk assessed and mitigated appropriately. The physical security surrounding the backup stores should also be considered along with the location of the backups. The distance that the backed-up data should be stored from the original source generally increases as a function of a number of factors including data criticality along with local geographic and social stability. Both the way in which your backups are stored and where they are stored should be considered in relation to your business requirements and risk appetite. Testing regimes Backups may only need to be restored infrequently. It is quite common for organisations to discover problems with their dataflow or backup procedures which create problems with the restoration process, or prevent it altogether. Therefore, organisations should routinely test data restoration from backup archives, both online and offsite. Backing up data on internet facing systems Internet facing systems are generally situated within Demilitarised Zones (DMZs) which are segregated from the corporate network. Backing up these services will likely require opening a hole through one of the corporate firewalls to allow the transfer of data to the backup store. This represents a potential vector for an attack migrating from your DMZ to corporate network. If you choose not to use an enterprise solution you may need to consider independent backup devices connected to machines within your DMZ. However this will involve greater overheads including tape rotation and additional equipment. The compromises inherent in backing up different systems within your network should be considered and appropriate risk mitigations put in place where required. Database backups Backing up the server file system is generally not enough for modern databases. The database contents should generally be backed up separately via dedicated database backup software. When this is not available, some organisations may choose to export their data to flat text files. These files can be used to rebuild the binary database in the event of a corruption or total data loss, which will require additional time to restore. These issues should be considered as part of your organisation's back up plans. 1 Verizon, 2011 Data Breach Investigations Report, viewed 27 June 2011 <http://www.verizonbusiness.com/go/2011dbir/us/> Credential currency considerations System backups and restores can sometimes have unanticipated consequences. If system level backup is restored, the passwords on the system will be those that were current at the time of backup. If a robust password change policy is in place, these may have now changed. In these instances, consider using a password management policy for system passwords which keeps a record of old system passwords (eg. consol administrator) for the length of the backup retention period. Transaction IDs or other “serial number” type system information should also be considered. These too will have been reset upon system restore, and you may encounter duplicate transaction numbers. Appropriate controls should be put in place to mitigate these circumstances. System updates and security patches When performing a system level restore it is important to understand which system updates and security patches (for both the system level and applications) were not applied when the backup was taken. This is especially important for custom applications that have had security updates applied as opposed to customoff-the-shelf (COTS) system and application patches. Having a robust update and patch procedure in place is important to ensure that critical updates and patches can be applied to restored systems when they are reinstated. Considerations for Outsourced and Online Backup Solutions Outsourced and online backup solutions are increasingly popular, especially amongst small to medium enterprises. This option is commonly used for the backup of website and online data (sometimes as part of the web design and implementation service) although can be used for other business data as well. Assumptions are often made about the performance of the outsourcer delivering the solution. It is important that all contracts are reviewed thoroughly to ensure critical services and response times are agreed upon, and that the ability to deliver these services within contracted parameters are tested periodically. Considerations for online website backups: Ensure that your service provider gives you a copy of your website data upon completing the design and implementation, and after every update. This data should be stored in a suitably secured offsite location on archive quality optical media or tape (depending on size of data) or other suitably reliable source. If backups are created dynamically and stored online, ensure that the service contracts provide sufficient controls for: o data resiliency – be satisfied that the way in which the data is stored provides adequate permanency, fault tolerance and geographic diversity. o length of storage – data snapshots should be stored for adequate lengths of time, in line with your business requirements. o number of snapshots – data recovery points should be available which align with the information currency requirements of your business. You must consider if the business can reasonably accept the loss of any new or changed data over a period of time equal to the difference between snapshot points. o speed in which data can be restored – data should be restored within maximum allowable outage (MAO) timeframes. It should be noted that real time replication solutions provide data and service availability protection in the event of a failure of the primary data centre but cannot be considered a backup solution. If data corruption occurs in the primary data centre due to error or compromise this will be automatically replicated to the disaster recovery site. A separate backup solution, invoked periodically and stored separately, should be used. Considerations for email and other files: Undertake a risk assessment to determine the impact of losing access to this data, both for a set period of time and permanently. Ensure service contracts stipulate backup outcomes appropriate for your business needs and risk posture. Consider periodically taking a snapshot of all your email and online files, and storing these in an offline manner (such as on archive quality optical media in a safe for small amounts of data or using commercial offline storage solutions for larger data stores). Online backups still need to be physically stored somewhere. Therefore, ensure that the storage location has sufficient physical security controls in place, and that handling and audit procedures for your data are acceptable. Check if backups are encrypted and suggest asking for this as part of the contract if they are not. Recommendations Summary CERT Australia suggests that you: Assess the different types of data your organisation stores as part of conducting its business in terms of its criticality to core business functions. Understand the business impact of the unavailability of data, for a period of time and permanently, and ensure that backup solutions, processes and procedures are in place which reflect these impacts. Ensure that organisational backup frequency and retention periods are adequate to cover your business risk and operational requirements. Ensure that both online and offline backups are protected via appropriate logical and physical security including considering the encryption of backup data. Please contact CERT Australia on 1300 172 499 or info@cert.gov.au if you have any questions concerning this publication, its content or its application. Related Information AS/NZS 5050:2010 Business Continuity – Managing Disruption-Related Risk Feedback CERT Australia is interested in any feedback that you may have with respect to this document and or the service that we provide. If you would like to provide us with your comments, please do not hesitate to e-mail us at info@cert.gov.au or contact us on 1300 172 499. About CERT Australia CERT Australia’s primary responsibility is to develop close working relationships with critical infrastructure organisations and businesses that operate systems that are important to Australia’s national interest. In this way, CERT Australia is able to help ensure that important services that all Australians rely on in their daily lives are secure and resilient. Blank page