University of South Australia
School of Computer and Information Science
SECURITY METRICS IN SCADA SYSTEM
Duc Nam Nguyen (ID: 110057721)
Master of Science (Computer and Information Science)
Supervisor: Dr. Elena Sitnikova
Submission Date: June 17th 2012
Abstract
Supervisory Control and Data Acquisition (SCADA) networks are used to control and
monitor critical infrastructure such as oil, gas, electricity, water etc. in most
developed organizations and countries. Due to the development of hacking tools, the
ability of hackers or terrorists, the lack of security functions in history of SCADA
system, the gap to apply security solutions from IT system,
the systems have
become ‘open’ and easy to be attacked and controlled. The problem in these
systems is how to know or decide these systems are secured. Unfortunately, at this
time there is no a widely accepted definition, standards which can define one system
is secured or not. To find answer for this concern, the research uses available
materials and analyse the architecture as well as weaknesses of SCADA system.
Possible answer is security metric which is quantifiable measurement of an aspect in
SCADA system. It also helps operators for making decisions and improves
performance of SCADA systems. The answer of research question is completely
given in the paper based on analyse theoretically. The outcomes of this research is
supporting further studies on security area in Process Control System which cover
SCADA system.
i
Acknowledgements
I am wholeheartedly thankful to my supervisor, Dr. Elena Sitnikova, who has
encouraged, guided and supported me to finish this minor thesis. I know I made a lot
of annoys for your work during the time I am doing my thesis. I will never forget your
efforts to help me to finish my thesis.
Besides my supervisor, it is a pleasure to thank Professor Jiuyong Li for
helping me to arrange other parts of the thesis such as presentation and minor thesis
1.
My sincere thanks also go to Nguyen Thanh Nhan who has supported me in
the start of this thesis. Sorry because I could not help you in your thesis.
Last but not the least; I would like to thank my parent who always encourages
me even sometimes I wanted to give up.
ii
Contents
Abstract ....................................................................................................................... i
Acknowledgements .....................................................................................................ii
Contents ..................................................................................................................... iii
List of Figures ............................................................................................................. v
List of Tables ..............................................................................................................vi
Abbreviations ............................................................................................................ vii
Chapter 1. Introduction .............................................................................................. 1
1.1
Research Problems....................................................................................... 1
1.2
Motivation ...................................................................................................... 3
1.3
Research question ........................................................................................ 4
1.4
Contribution ................................................................................................... 4
1.5
Chapter Review............................................................................................. 4
Chapter 2. SCADA system ......................................................................................... 5
2.1 Definition of SCADA system ............................................................................. 5
2.2 SCADA System General Layout ....................................................................... 5
2.3 Modern architecture of SCADA systems ........................................................... 7
2.4 How a SCADA system can be attacked? .......................................................... 9
2.4.1 Possible threats on SCADA system ............................................................ 9
2.4.2 Well-known attacks on SCADA system .................................................... 12
2.5 Discussion....................................................................................................... 15
2.6 Chapter Review .............................................................................................. 17
Chapter 3. Security metrics ...................................................................................... 18
3.1 Definition of security metric ............................................................................. 18
3.2 Challenges and opportunities for security metrics in SCADA system ............. 19
3.3 Category of security metrics ............................................................................ 21
3.3.1 Organizational metrics .............................................................................. 21
3.3.2 Operational metrics ................................................................................... 22
3.3.3 Technical metrics ...................................................................................... 23
3.4 Recommendations of developing security metrics .......................................... 23
3.4.1 The relationships between security metrics and risk analysis ................... 24
3.4.2 Framework for metric development and evaluation .................................. 25
3.5 Chapter Review .............................................................................................. 26
iii
Chapter 4. Security metrics in SCADA system ......................................................... 27
4.1 Number of attacks ........................................................................................... 27
4.2 Probability of attacks ....................................................................................... 27
4.3 Shortest Length of attacks .............................................................................. 28
4.4 Attack surface metric ...................................................................................... 28
4.5 Rogue change days ........................................................................................ 29
4.6 Security evaluation deficiency count ............................................................... 29
4.7 Data transmission exposure............................................................................ 29
4.8 Known vulnerability days ................................................................................. 30
4.9 Password crack time ....................................................................................... 30
4.10 Detection mechanism deficiency count ......................................................... 30
4.11 Restoration time ............................................................................................ 30
4.12 Chapter review .............................................................................................. 31
Chapter 5. Conclusion .............................................................................................. 32
References: .............................................................................................................. 33
iv
List of Figures
Figure 1: SCADA system general layout (Stouffer et al. 2008) .................................. 6
Figure 2: SCADA System Implementation Example (Stouffer et al. 2008) ................. 8
Figure 3: SCADA System Implementation Example (Rail Monitoring and Controlling)
(Stouffer et al. 2008)................................................................................................... 9
Figure 4: Attacks map into SCADA systems (Kang et al. 2009) ............................... 11
Figure 5: Metric Categorization Identified at Workshop on Information Security
System Rating and Ranking (ACSA 2001) ............................................................... 21
Figure 6: Framework for metric development (Grantz et al. 2003) ........................... 25
Figure 7: Security framework for SCADA system (Nhan 2012) ................................ 26
v
List of Tables
Table 1: possible threats on SCADA system (edited from GAO 2005 and Stouffer et
al. 2008) ................................................................................................................... 10
Table 2: Common SCADA security threats (Kang et al. 2009) ................................. 12
vi
Abbreviations
CERT
Computer Emergency Response Team
CNN
Cable News Network
DoS
Denial of Services
GAO
Government Accountability Office
HMI
Human Machine Interface
ICS
Industrial Control System
IED
Intelligent Electronic Device
IT
Information Technology
IP
Internet Protocol
I3P
Institute for Information Infrastructure Protection
ISO
International Standards Organization
LAN
Local Area Network
MTU
Master Terminal Unit
NERC
North American Electric Reliability Corporation
PLC
Programmable Logic Controller
PCS
Process Control System
RTU
Remote Terminal Unit
SCADA
Supervisory Control and Data Acquisition
TCP
Transmission Control Protocol
vii
Chapter 1. Introduction
Supervisory Control and Data Acquisition (SCADA) systems are widely used
to control and monitor resources of the critical infrastructure in many organizations
and nations. SCADA systems are built for many fields such as oil and gas, air traffic
and
railways,
power
generation
and
transmission,
water
management,
manufacturing, production plants and electricity (Slay & Sitnikova 2009). The original
SCADA systems were built as standalone networks. The control system and system
devices communicated with each other in an isolated network and very limited
information which is sent to outside. There was no concern about security in SCADA
at that time. In recent years, due to the improvements of computer science, new
technologies and requirements from business, SCADA systems are not able to work
independently. They needed to transfer data to communicate with corporate
networks, allow real time access and control processes by supervisor from a
personal computer which can be located far from system (Kang 2009, Stouffer et al.
2008). The communication links in SCADA system becomes more complicated and
they are targets for hackers to attack SCADA system. There are many designs for
each kind of networks or purposes and these designs have differences but they have
some commons such as allowing for remote access to systems, allowing for
connections between SCADA networks and internal network. These networks are
also built on common technologies such as Windows, Ethernet and Web Services
(Slay & Miller 2006). In recent years, concerns about cyber-attacks on SCADA
systems which are used to control and monitor the major critical infrastructure, is
extremely increasing in many organizations and nations.
1.1 Research Problems
Slay and Miller (2007) mentions the most well-known attack on SCADA
system was the attack on the Maroochy Shire Council’s sewage control system in
Queensland, Australia in January 2000. Another example is the Davis-Besse Ohio
Nuclear Power Plant in January 2003 (Poulsen 2003). In August 2005, Zotob which
is a worm spreads by exploiting the Microsoft Windows Plug and Play Buffer
Overflow Vulnerability, crashed thirteen automobile manufacturing plants in United
State and many other corporations and forced them unavailable for an hour (Robert
1
2005). In March 2007, the results of an experiment named “Aurora Generator Test”
which is provided by Idaho National Labs, elevate cyber-attack threats to new level.
In 2008, an emergency shutdown of the Hatch Nuclear Power Plant in Georgia due
to a software update was cause of loss of millions dollars (Krebs 2008). In April and
June 2009, the Wall Street Journal reported that Russian and Chinese spies
attacked the United State electrical grid and the North American Electric Reliability
Corporation (NERC) and a defence contractor evaluate abilities of the company to
resist cyber-attacks (Bauch et al. 2010). In 2010, the Stuxnet worm which includes
specialized malware payload, is designed to target Siemens SCADA systems and 60
per cent of Stuxnet worm is located in Iran and affected many computers in other
countries (Keizer 2010). And one debated situation was event of Northwest Rail
Company in December 2012 in United State. On December 1st, train service was
slow for a short while and schedules delayed for 15 minutes and second event
happened on December 2nd. The investigation could not find the culprit and
concluded that there were no attacks on control system. Many scientists started
worrying about vulnerabilities of SCADA system which is used to control countries’
critical infrastructure (Rashid 2012).
In United State, eighty per cent of the power is generated by investor-owned
public utilities and according to the security firm Riptech, 70% of their clients had at
least one major attack in the first six months of 2002 and it is 57% in the last six
months of 2001 (Barnes et al. 2004). According to the report by the Center of
Strategic and International Studies, in 2009, there were nearly half of the
respondents confirm that their systems were not attacked by large scale denial of
service attacks or network infiltrations but in 2010, 80 per cent faced a large scale
denial of service attacks and 85 per cent got experiences to network infiltrations. In
addition, a quarter of daily or weekly report from interviewees is about denial of
service attacks on a large scale and two-third report (at least monthly) said that
malware designed had been found for sabotage on their systems (Baker et al. 2010).
It is easy to recognize that cyber-attacks have become increasing day by day on
SCADA systems.
The owners and researchers in SCADA systems are struggling to find answer
for questions such as “How secure is our system?”, “How secure does it need to
be?” and “How we can improve security for SCADA system?”. Security metrics are
2
tools which can help to provide possible answers for these questions. Security
metrics are used widely in information technology field. It provides a practical
approach to measuring information security. Based on collection, analysis and report
of data in the systems, security metrics can evaluate security level of the system and
also provide decision making and accountability (Payne 2006). However, IT security
metrics cannot always be applied or implemented in SCADA systems even the
technologies are the same. According to research report from I3P, the information
security community developed some security metrics, some of them can directly and
immediately applied to SCADA system and most security metrics cannot be used
immediately to SCADA system. To explain this difficulty, scientists stated differences
between SCADA system and traditional IT system. The SCADA system has different
risks and priorities. It pays more concerns about risks to the health and safety of
human lives and damage to the environment while typical IT system pays more
concerns about production loss and impacts on economy (Stouffer et al. 2008, I3P
SCADA project). It does not mean that SCADA system does not have concerns
about financial issues, it only differs about priority. In addition, the purpose of safety
and efficiency in SCADA system can make conflicts with the operations of control
system due to differences of performance, availability and other requirements
(Stouffer 2008 and I3P SCADA project).
1.2 Motivation
SCADA systems are used to control important infrastructure in modern world,
so losing control security in SCADA systems bring huge impacts on human lives and
economy. The current trends on securing SCADA system focused on developing
new SCADA architectures or the technologies or methods to adapt with threats from
cyber-attacks (Slay & Sitnikova 2009). Security metrics are good tools to identify the
SCADA system is secured or not. However security metrics are not easy to apply
immediately to SCADA system. It is a gap of applying security metrics in SCADA
system. There were a lot of efforts from owners and researchers to define useful
metrics for SCADA systems (I3P SCADA Research Project 2003, Payne 2006,
Krautsevich et al. 2010, and Wing & Manadhata 2004-2005-2007).
3
1.3 Research question
This research answers the question “How the security of SCADA systems be
measured?” As mentioned above, the use of security metrics in security fields is
becoming more commonly and widely, so the goal of this research explains how to
use security metrics to identify security level in SCADA systems.
1.4 Contribution
The results of this research will help in extending the study of SCADA system
in security area. It will provide a big literature review of security metrics in SCADA
system. If the research is successful in providing the overall picture of the security of
SCADA system including general knowledge of SCADA architecture, challenges of
SCADA system in cyber-attacks, the needs and difficulties of using security metrics
to identify security level of system and some suggestions of developing and using
metrics for SCADA system, these will be motivations for further researches and in
supporting the development of useful metrics which are very limited in the public
domain for securing the SCADA system.
This research has a general picture of SCADA system and security metrics for
who wants to have basic information of the system and supports them in their further
works. It also includes some technical metrics which are used in some SCADA
systems. These metrics could be used in other studies to develop a new metric (or
combination of metrics) for specific SCADA system.
1.5 Chapter Review
This chapter provides introduction on problems of modern SCADA system,
basic ideas for security metrics, difficulties when applying security metrics to SCADA
system, thesis question and research contributions. Next chapter will provide in
details about SCADA system, its characteristics and reasons why security metrics for
IT system cannot be applied directly to SCADA system.
4
Chapter 2. SCADA system
2.1 Definition of SCADA system
Supervisory Control and Data Acquisition (SCADA) systems are computerbased control systems which are used to monitor and control physical processes.
These systems generally contain a set of network devices such as controllers,
sensors actuators and communication devices. SCADA systems are designed to
collect data, transfer it to a central computer facility, display the data to the operator
and allow the operators to monitor or control entire system form a location in real
time (Coates et al. 2010, Dos et al. 2008). SCADA systems are used in distribution
systems such as electrical power grids, water distribution and wastewater collection
systems, oil and natural gas pipelines and railway transportation systems (Slay and
Miller 2006, Barnes et al. 2004, Cai et al. 2008). In the past, these systems were
designed and worked in isolated environments. The control systems and devices
communicated with each other in an isolated network and sharing information with
outside is very rare (Chandia et al. 2007). So the systems’ security functions were
not concerned by the owners or researchers. These days, the connections between
SCADA systems and open networks are complicating. The increasing of
components of control systems which are connected with the outside using Internetbased standards and the integrating of control networks to corporate networks in
order to share data, are raising a big question about protecting SCADA network from
cyber-attacks for owners, users and scientists (Coates et al. 2010). The demand
about identifying these systems are secured or not and protecting these systems are
also highly increasing.
2.2 SCADA System General Layout
In general layout of SCADA system, there are three main parts: Control
Center, Communication Link and Field Sites. Figure 1 shows a general layout of
SCADA system (Kang et al. 2009, Stouffer et al. 2008).
5
Figure 1: SCADA system general layout (Stouffer et al. 2008)
The Control Center of SCADA system includes control server, data historian,
human machine interface, engineering workstations and communication routers. A
SCADA Control Server (normally called Master Terminal Unit – MTU) has tasks to
monitor and control for field sites over long distance communications network
including alarming, processing system status and reporting. Based on information
from field sites, the MTU will process data and help to provide automatic commands
or give suggestions for operator to remote stations and control devices in field sites
(Kang et al. 2009, Stouffer et al. 2008). Human Machine Interface (HMI) allows
operators to monitor the states of processes by control or modify settings and control
operations when system has an emergency situation. The HMI also provides
process status and historical information. A HMI could be a workstation in the control
center, a laptop on a wireless LAN or a browser on a system connected to SCADA
control server network (Kang et al. 2009, Stouffer 2008). The Data Historian is used
to store process data in the system. The data could be historical data collected from
firewalls, systems messages, intrusion detection system log, traffic capturing. This
data can be used to analysing, studying or reporting in different levels (Kang et al.
2009, Stouffer et al. 2008). The Communication Router is used to transfer data
between control center and other parts of SCADA system (Stouffer et al. 2008).
Communication links between the control site and various field sites vary
among different implementations. SCADA devices can connect to others by some
6
ways such as optical fiber, radio, telephone line, microwave, satellite or Ethernet
(Kang et al. 2009, Stouffer et al. 2008, and Chikuni et al. 2007).
The tasks of Field Site are performing and controlling local process. A Remote
Terminal Unit (RTU) is a special purpose data acquisition and control unit which is
used to support the operations of SCADA stations. If wire connections are not
possible, wireless radio interfaces are implemented in RTU to support wireless
connections (Kang et al. 2009, Stouffer et al. 2008). A Programmable Logic
Controller (PLC) is a small industrial computer which is used to calculate logic
functions by electrical hardware such as relays, switches and mechanical counters.
In SCADA systems, PLCs are often used as field devices due to its economical,
versatile, flexible and configurable (Kang et al. 2009, Stouffer et al. 2008). An
Intelligent Electronic Device (IED) is a smart sensor which is used to process data,
communicate to other devices and perform local processing and controlling. An IED
could combine an analog input sensor, analog output, low level control capabilities,
and a communication system and program memory. The use of IEDs is allowing for
control automatically at local process (Kang et al. 2009, Stouffer et al. 2008).
The general layout of SCADA system provided a general picture of SCADA
components and its tasks. The next section will introduce modern architecture of
SCADA system which is developed from general layout of SCADA system.
2.3 Modern architecture of SCADA systems
Figure 2 shows an example implementation of a SCADA system. It still has
Control Center and three Field Sites which are mentioned in the previous section.
The improvement of modern architecture of SCADA system is the opening of the
system architecture and the utilizing of open standards and protocols. There are
several differences to the general layout of SCADA system in section 2.2. A Backup
Control Center was installed to provide redundancy if the functions of a primary
control center are lost. And a Regional Control Center is used for higher level of
supervisory control. One Field Site is located near to Control Center and use the
wide area network (WAN) to communicate. The employees in corporate network can
access to all devices in Control Centers through the WAN and access remotely to
Field Sites to troubleshooting and maintaining. In this example, the connections
7
between Control Center and two Field Sites are using radio frequency to
communicate (Stouffer et al. 2008).
Figure 2: SCADA System Implementation Example (Stouffer et al. 2008)
Stouffer et al. (2008) also introduce an example of SCADA system
implementation in rail monitoring and controlling. Figure 3 show that the Rail
Monitoring and Control system has a control center and three sections of a rail
system. The SCADA system collects and processes information from the rail
sections including status of the trains, signal systems, traction electrification and
ticket vending machines. This information is also transferred to operation consoles at
the HMI station inside the rail control center. The SCADA system monitors operator
inputs at the rail control center and give high level commands to the rail section
components (Stouffer et al 2008).
8
Figure 3: SCADA System Implementation Example (Rail Monitoring and Controlling)
(Stouffer et al. 2008)
Modern architecture of SCADA system still keeps the basic idea of the
general layout and it has some improvements to adapt with specific situation of
SCADA system in real world.
2.4 How a SCADA system can be attacked?
In this section, this paper will provide information for possible threats on
SCADA system, analysis potential attacks and give some well-known attacks in
approximately 20 years.
2.4.1 Possible threats on SCADA system
In 2005, Government Accountability Office in United State analysed data from
the Federal Bureau of Investigation, Central Intelligent Agency and the Software
Engineering Institute’s CERT Coordination Center to provide 9 emergency cyber
security threats for industrial control system (including SCADA). And in 2008,
9
Stouffer added industrial spies to have totally 10 emergency cyber security threats.
In this thesis, possible threats on SCADA system will be concluded into 6 major
kinds of threats which are listed in Table 1:
Threat Agent
Foreign intelligent
services
Description
Other countries use cyber tools to attack on critical
infrastructure of target country. It could include warfare
doctrines, programs and capabilities. The consequences are
impacts on the supply, communications and economic
infrastructure which could have negative effects to human lives
and environment.
Insiders
Insiders do not need to have great knowledge about computer
intrusions, but they can use their experiences when using
system often to allow them to gain access and damage system
or steal data. Insiders might be employees, contractors or
business partners. Insiders could make one of the highest
impacts on SCADA system.
Phishers
The culprit could be an individual or small groups attacked
Spammers
system to gain money or steal information or send unsolicited
Spyware/malware email with false information to sell products, spread
authors
spyware/malware or attack directly targets. This threat is based
on spreading virus or worm to target’s system and could harm
data, hard drives etc. Some well-known worms and virus:
Zotob, Slammer (Sapphire), Blaster etc.
Terrorists
This kind of threat is trying to give as much damage as possible
to countries’ critical infrastructure to threaten national security.
The methods could be virus, worm, denial of service etc.
Criminal group
Criminal groups attack system to gain money. Their higher
priority is money not damage like terrorists.
Hacker
Attackers break into networks for proving themselves in
community or conquering a challenge (target system). Another
kind of hackers could be bot-network operators. They take over
multiple systems to coordinate attacks and spread phishing
schemes, spams and malware. Attackers now can easily
download attack scripts and protocols from internet and choose
a system as a target. While the tools which are used for attack
become complicating, they also become easier to use.
Table 1: possible threats on SCADA system (edited from GAO 2005 and Stouffer et
al. 2008)
The previous part of this section provided results from analysing data in
United State by Government Accountability Office. Another point of view is looking
for common threats by analysing architecture of SCADA system. Kang et al. (2009)
stated in his paper common threats on SCADA system based on analysing
weaknesses in SCADA architecture. The author divided attacks on SCADA system
10
into 3 main categories: attacks upon the power system, attacks by the power system
and
attacks through the power system. In the first category, the target of attack is
infrastructure itself, the goal of attack could be a single component such as a critical
substation or transmission tower. The target in the second category is the people
who are using infrastructure and the target in the last category is the individual
infrastructure such as lines, pipes, cables and tunnels etc.
Figure 4: Attacks map into SCADA systems (Kang et al. 2009)
Figure 4 show that there are about 32 types of security threats on SCADA
networks. These threats can be found in Table 2. These are common security
threats on SCADA systems, these threats can be combined to one threat. However
there are more threats in the real world and they are very complicated to analyze as
well as understand.
1.Authorization
violation
2.Bombs
3.Browsing
4.Bypassing
9.Information
leakage
10.Intercept/alter
11.Interfernce
database query
analysis
12.Masquerade
11
17.Sabotage
25.Traffic analysis
18.Scavenging
19.Spying
26.Trap Door
27.Trojan Horse
20.Service
28.Tunneling
controls
5.Data
Modifications
Spoofing
21.Sniffers
13.Physical
Intrusion
6.Denial of Service
14.Replay
22.Substitution
7.Eavesdropping
8.Illegitimate use
15.Repudiation
16.Resourse
Exhaustion
23.Terrorism
24.Theft
29.Unauthorized
access violations of
permission
30.Unauthorized
access
piggybacking
31.Virus
32.Worm
Table 2: Common SCADA security threats (Kang et al. 2009)
The improvement of modern SCADA systems creates a big hole for many
vulnerabilities and possible attacks. An attack could be done from corporate network,
virtual private network, wireless network, di-up modem and local network as well.
Possible attacks on an SCADA system could separate into: backdoors and holes in
network parameters, vulnerabilities in common protocols, database attacks,
communications hijacking, attacks on field devices and “man in middle” attacks
(Stouffer et al. 2008 and Kuipers and Fabro 2006).
From different methods to identify possible attacks in SCADA system, there
has still same point which is exploiting the weaknesses of SCADA system as well as
the impacts from the need of expanding SCADA system to other networks. The next
section will introduce famous attacks on SCADA system and also point out the target
of each attack.
2.4.2 Well-known attacks on SCADA system
In March 1997, a teenager in Worcester, Massachusetts using a dial-up
connection to Worcester Air Traffic control system broke into the system and
disabled part of public switched telephone. The consequences were phone services
at the control tower, airport security, airport fire department, the weather service and
carriers stopped. The attack blocked phone services to 600 homes and businesses
in the nearby town of Rutland (CNN 1998).
In June 1999, due to the poor performance of SCADA system in controlling
gasoline pipeline, 237,000 gallons of gasoline from a 16-inch pipeline flowed through
Whatcom Falls Park in Bellingham, Washington. The consequences were the
gasoline ignited and burned in 1.5 miles length and causing 3 deaths and 8 injuries.
12
The total economic loss was $45 million. According the investigation from the
National Transportation Safety Board, one of five reasons of the accident was the
Olympic Pipe Line Company used SCADA system in practicing their database
development work while the system need to be used to monitor the pipeline. This
made the system unavailable to control the pipeline in critical event (National
Transportation Safety Board 1999).
Slay and Miller (2007) mentions the most well-known attack on SCADA
system was the attack on the Maroochy Shire Council’s sewage control system in
Queensland, Australia in January 2000. After the control system for the sewage plant
was completely implemented by a contractor company, the system had some
problems such as fails on start or stop operation when required, fails on alarming
and loss of communications between control centers and pumping stations. At the
beginning, the managers thought there was a leak in the pipes and the valves were
opening without commanded .They wasted several months to discover that the
valves were activated by the spoofed controllers. And the culprit is an ex-employee
of the contractor company who installed the control system. The consequences were
the flooding of nearby hotels, park, and river with approximately 264,000 gallons of
raw sewage. When investigating this attack, the police discovered that it is hard to
detect this kind of attack due to the slow of the responses and the culprit totally
finished 46 documented attacks (Slay and Miller 2007).
In January 2003, The Nuclear Regulatory Commission stated that the
Microsoft SQL Server worm called Slammer or Sapphire was the cause of the
disable a safety monitoring system about 5 hours at the idled Davis-Besse nuclear
power plant in Oak Harbor, Ohio. And it took about six hours to become available
again. The investigation showed that the worm entered the Davis-Besse plant
through an unsecured network. Slammer exploited a buffer overflow vulnerability in
computers which running Microsoft’s SQL Server or Microsoft Data Engine, it could
double in size every 8.5 seconds (Poulsen 2003).
On August 14th 2003, parts of Midwest, Northeast United States and Ontario,
Canada had an electric power blackout. The investigation stated that reasons were
the failure of the alarm processor in SCADA system and incomplete information on
topology changes. It affected 50 million people in eight states in United State and
13
Ontario province in Canada. It took four days to restore in United State and more
than a week in Ontario. Total economic loss was between $4 billion and $10 billion
dollars (NERC 2004).
In August 2003, the Sobig virus was attached in an email and when opened, it
sends itself to other email addresses of the victim’s address book. The virus can
create an open backdoor which helps hackers to control victims’ computer without
detection. Spammers can use the backdoor to upload applications which can send
spam email continually. The impacts of Sobig virus were infecting the computer
system at CSX Corp.’s Jacksonville, Florida headquarters, shutting down signals,
dispatches and other systems. And consequences were short distance train from
Richmond, Virginia to Washington and New York was delayed for more than two
hours and long distance trains were delayed about 5 hours (Niland 2003).
In August 2005, based on vulnerability of Microsoft Windows Plug and Play
Buffer Overflow, a worm named Zotob crashed thirteen of DaimlerChrysler’s
automobile manufacturing plants and made them disable for an hour. Plants in
Illinois, Indiana, Wisconsin, Ohio, Delaware and Michigan also were shut due to
Zotob. Zotob slows the computers process, makes them crash and reboot
continually. Zotob can create an open backdoor control channel and allow hackers to
control the infected computer. It also prevents the infected computer to connect to
antivirus website by adding some codes into computer’s operating system files
(Robert 2005).
In March 2008, there was an emergency shutdown due to software update on
the plant’s business network of the Hatch Nuclear Power Plant in Georgia. The
business network and the SCADA system used two way communication and the
update synchronized information on both systems. After reboot, the SCADA safety
systems found a lack of data and alarmed that the water level in the cooling systems
had dropped, thus automatically shut the system and started emergency plan. The
manager did not recognize that the update in business work would synchronize data
between two networks and it was the cause of accident. The consequences were the
loss of millions of dollars to recover this mistake (Krebs 2008).
Stuxnet is a computer worm which discovered in June 2010 and categorized
in July 2010. It exploits the vulnerabilities in Microsoft Windows to spread copies of
14
itself through networks and removable drives. It installs server and client
components to create backdoor to any clients which it can connect to. It connects to
a remote server to test internet connection, send and receive commands from
hackers. The primary goal of Stuxnet worm is to gain access control of industrial
facilities (including SCADA system). If it can find these systems on the effected
computer, it can steal code and design project from system. It also can upload its
own code to the Programmable Logic Controller (PLC) which is a component of
SCADA system. It is not easy to find these codes on PLC. According to records of
Stuxnet from Symantec, 60 per cent of Stuxnet appeared in Iran and the left infected
other countries such as Indonesia, India, United State and others (Symantec 2010).
2.5 Discussion
This section discusses about the differences between SCADA systems and IT
systems. Thus the audiences could be easy to understand why security techniques
(include security metrics) cannot be applied directly to SCADA systems. As
mentioned in production section, SCADA systems have higher priority concern for
risk of health and safety of human lives than financial issues. In addition, some
targets of health and safety bring conflicts with design of security in the system.
In general, the most significant different between the SCADA system and IT
system is the high availability requirement for monitoring and control operations. For
SCADA systems, the priority for security goals from lowest to highest is
confidentiality, integrity and availability while the priority or IT system is availability,
integrity and confidentiality. Stouffer et al. (2008) listed 13 considerations when
analysing security for industrial control system (including SCADA):
Performance requirements: IT system is non-real time system which requires
high throughput, consistent responses and can accept some levels of delay and
jitter. While SCADA system is real time system which requires time-critical
responses, modest throughput and cannot accept high delay or jitter.

Availability requirements: the SCADA system normally works 24 hours/7 days.
The system must ensure high availability for processing. Unexpected stops of
the system are not acceptable and expected stops must be planned and
scheduled. Many SCADA systems cannot be easily stopped and started
15
without negative effects to production. Thus some strategies which is used for
IT system such as reboot a component, port scanning, service fingerprinting
are not acceptable for SCADA system.

Risk management requirements: higher priority concerns in IT system are
data confidentiality and integrity while in SCADA are human and environment
safety and fault tolerance. The security employees for SCADA system must
understand the link between security and safety which are very different in
other systems.

Architecture security focus: in IT system, the primary focus of security is
protecting the information which normally stored in main office. Thus it gives
more convenient to protect this information. While SCADA system needs to
protect information not only stored in main office but also in end stations
(PLC, IED and RTU).

Physical interaction: typical IT system does not have physical interaction with
the environment while SCADA system has complex interactions with physical
processes and consequences. All security functions must be proved that there
are no conflicts with operations of the system.

Time-Critical Response: in SCADA system, information flow must not be
interrupted while requires an access control into system. The response time to
human interaction is very critical while in IT system, access control can be
implemented without regard for information flow.

System operation: typical IT system has advantages for finding hardware and
software for replacing while SCADA system uses specialized hardware and
software, thus it is difficult to upgrade for SCADA system.

Resource constraints: SCADA system is designed to support the industrial
process and may not have enough resources or capabilities to support
security functions. In addition, when adding new security functions to SCADA
system, it may make conflicts with system vendor license and services
agreements. In IT system, it is specified that the system will have enough
resources to support third party applications for security functions.

Communications: SCADA system uses specialized protocols such as
Modbus/TCP, EtherNet/IP and DNP317 to communicate between control
16
center and field sites while IT system uses other standard communications
protocols.

Change management: upgrading software is a main requirement for both IT
system and SCADA system to prevent exploitation of vulnerabilities of
software. For IT system, software updates are applied in a timely fashion
based on security policy and procedures (normally automatic). While in
SCADA system, software updates must be tested by vendor of the system,
and only implemented in planned outages.

Managed support: typical IT system allows supports from many different
vendors while SCADA system usually allows supports from a single vendor.

Component lifetime: typical IT system has a lifetime from 3 to 5 years due to
the quick improvements of new technologies while SCADA system has a
lifetime from 15 to 20 years due to its special design and requirements of
software and hardware.

Access to components: it is easy to access the typical IT components while
SCADA components are isolated and require physical efforts to gain access
to them.
To conclude, SCADA system with its specialized design (including hardware,
software and communication protocols) and high requirements for availability makes
difficulties to apply security solutions from typical IT system to SCADA system.
2.6 Chapter Review
This chapter provided general picture about SCADA system which includes its
architecture, improvements in modern system, weaknesses of the system,
differences between SCADA system and typical IT system. Thus audiences can
understand even security in SCADA system becomes seriously but it still has a gap
to apply typical solutions for SCADA system. Next chapter will provide one solution –
security metric and analysis the effects of this solution in SCADA system.
17
Chapter 3. Security metrics
The term “metrics” describes a broad category of tools which are used to
evaluate data in many organizations. Metrics are widely used by industry for many
years. The simplest form of a metric could be a measurement which is compared to
a scale or benchmark to produce results (I3P report 2005). For example, a system
has 100 computers and 98 computers have completed software update. Thus there
are 2 per cent of computers which have not completed software update. This
measurement can be used to determine a security metric by comparing it with the
system‘s benchmarks. The security goal could be “no more than 1% computers have
not completed software update”. The difference between metrics and measurements
are measurements provides single point to point view for specific sector or
dimension of a system while metrics are used to comparing the results from
measurements to a pre-identify baseline. Measurements are made from counting
while metrics are created from comparing and analyzing (Jansen 2009).There is a
rich of set for security metrics in IT systems which have been developed around 40
years. However, the security metrics for SCADA system are not widely available.
The reason comes from the special environment of SCADA systems. There were a
lot of efforts to create, develop and apply security metrics from IT system to SCADA
system. However there are limited security metrics for security metrics which are
published in security community. This chapter will provide short definition of security
metric, its categories, challenges for applying a security metric in SCADA system,
how to put a security metric into a SCADA system and some existing security
metrics.
3.1 Definition of security metric
At a high level, security metrics are quantifiable measurements of security
aspect of a system. Security metrics could include tools which are designed to help
making decision automatically, improve performance and evaluate system and
accountability through collection, analysis and report from related data (Jansen
2009). A measurement can quickly identify aspects of a system or process. This
measurement could be a form of a number, trend line, relative position etc. However
the measurements by themselves normally have little meanings while when using
18
measurements and comparing them with benchmarks, the result will bring more
meaningful and that is a metric. Benchmarks provide a frame of reference for
comparing measurements. Benchmarks could be industry standards, past
performance, corporate security goals, expected performance and peer averages. A
metric which is involved with security aspects of system, is called security metric
(Jansen 2009, Glantz et al. 2003, I3P report 2005, I3P SCADA project).
3.2 Challenges and opportunities for security metrics in
SCADA system
The current trends of improving security in SCADA system is developing new
SCADA structure which can adapt with threatens from cyber-attacks (Fernandez, EB
& Larrondo-Petrie 2010 and Yakkali & Subramanian 2010). Most of the SCADA
systems are designed for long time ago and worked in isolated environment. Thus
they could not provide security functions. However, to develop and build a new
SCADA system will spend a lot of money for the owners (organizations, individuals
or governments). The new development of SCADA structure would be useful when
start building a totally new system. A question has been appeared what is possible
solution for the old SCADA systems? Researchers started finding and improving
vulnerabilities in old SCADA systems. Due to the increase of requirements of using
new devices and the evolution of new technologies in SCADA systems to improve
effectiveness and efficiency of the processes, it becomes more complicating to fix
every hole or vulnerability in SCADA systems. One big gap to apply security
solutions in SCADA system is that any new security solution need to be tested
carefully before implemented into the systems while SCADA system always requires
very high about accountability. Any small mistake or hole from new security solution
will bring huge impacts on SCADA system (Kang et al. 2009, Stouffer et al. 2008,
and I3P report 2005). Exploiting security solutions from IT system to use it in SCADA
system does not bring good signal for researches due to the differences between
two systems and the special of SCADA system itself. One well-known example for
this could be using password in IT system and SCADA system. In IT system, it
always encourages the users to use as much complicating passwords as possible to
prevent the cyber-attacks on passwords. However in SCADA system, if the staffs
use too complicating password to log in the system, one possible situation is when a
critical event happens, complicating password would make confuse to the staff while
19
at that time, the staffs need to be logged into the system as soon as possible to
make effective decision (Stouffer et al. 2008, I3P report 2005). And scientists came
to an idea about security metrics which are used widely in IT system. First time, they
also tried to apply security metrics which are developed for IT system into SCADA
systems. Some metrics could be applied directly to SCADA systems while most of
metrics could not. The reasons are due to the differences between two systems and
the connection link between security and safety in SCADA system. Then the
evolution on developing new security metrics for SCADA system started. This
solution for securing SCADA system also can answer the research question “How
security of SCADA system be measured?” Thus security of SCADA system can be
measured, the scientists can answer the question “Is this SCADA system secured or
not?” From this stage, security community can prevent or reduce the risks in SCADA
system.
One big effort from security community on developing security metrics for
SCADA system is the I3P Process Control System (including SCADA) in 2003 and
the I3P SCADA project recently. In 2003, with the supports from 10 institutions and
the National Institute of Standards and Technology, the I3P PCS research project is
organized into six major tasks and these tasks are separated to research in 10
institutions:

Task 1: Assess dependence on PCS and its security

Task 2: Account for the type and magnitude of PCS interdependencies

Task 3: Develop metrics for the assessment and management of PCS
security

Task 4: Develop tools and technologies to enable inherently secure
PCS security

Task 5: Develop cross domain solutions for information sharing

Task 6: Transfer technology of these solutions into industry
Task 3 of the project was developing metrics for securing PCS; however the
results from this stage were very limited. They could not provide some specific
security
metrics
for
PCS
(including
SCADA)
but
they
provided
some
recommendations when developing a security metric. The research project also
20
provided the categories of security metrics for PCS which can be used for SCADA
systems as well.
3.3 Category of security metrics
The security metrics for SCADA system could be categorized into three broad
classes: Organizational metrics, Operational metrics and Technical metrics (ACSA
2001 and I3P report 2005). The I3P project identified three sets of SCADA
stakeholders include industry (operators and users of systems), vendors (developer
of system, system integrators and IT security service providers) and government
(I3P report 2005).
Figure 5: Metric Categorization Identified at Workshop on Information Security System
Rating and Ranking (ACSA 2001)
3.3.1 Organizational metrics
Organizational
metrics
are
used
to
evaluate
the
effectiveness
of
organizational programs and processes. They can be used to assess the adequacy
of the standards, policies and procedures of the organization to improve security.
They also help to provide decisions of the uses of organization’s resources. These
decisions could cover investment in architectures or technologies, security programs
processes and divided resources between security program components and
21
activities. This kind of metric might answer some questions about standards, policies
or procedures such as “Has organization use some security standards?” and “Is
there a security policy for SCADA system and its component?” etc. Organizational
security metrics could separate into security program metrics and security process
metrics. Organizational security metrics are more relevant to industry and vendors.
They are normally used in mandating performance or reporting and they are quite
tied to standards of good practice. An example of organizational metrics could be the
percentage of staffs that finished security training course reaches the security
program goal of security awareness and training. The metric could be identified like
“the percentage of staffs that finished security training course must more than 98 per
cent”. The SCADA system should use these metrics as indicators to reach the
security goals of the organizations. (I3P report 2005).
There are three broad categories of security metrics which were accepted by
security community in SCADA system. There still have the combinations of three
categories to provide relevant security metrics for specific situation.
3.3.2 Operational metrics
Operational metrics are used to evaluate and manage risks to operational
environments of the systems. Most metrics of risk or its component sector are
operational metrics. They could be metrics that describe the threat environment,
metrics used in risk management, metrics that support incident response etc.
Operational metrics could be used to assess organizations’ security posture or
indicate changes in system processes to improve risk management of the systems.
One advantage of this metric is that operational data of the SCADA systems is
already collected and could be analysed to provide operational security metrics.
Operational metrics can be used to assess how effective of the organization’s
policies and procedures to the responsible staffs. The operational metrics could be
defined based on answers of some questions such as “Are cyber security
inspections of SCADA system done by staffs who finished training course about
cyber security?”, “Was the certified training course meets industry standards?” and
are the results of each inspection reported and stored into a database?” etc. (I3P
report 2005).
22
3.3.3 Technical metrics
Technical security metrics are used to evaluate and compare technical
objects such as algorithms, specifications, architectures, designs. Technical security
metrics could be used to help organizations to select products which are the best
suitable for their security goals. For example, the organization could use technical
security metrics to identify required or desired characteristics of the SCADA systems
or its product. Technical security metrics could be used to assess the adequacy of
the security being implemented to protect every component of SCADA systems. This
kind of metrics normally is done every day to secure the system. This metrics could
be involved with some questions such as “how many attacks in the system by
day/week or month?”, “how many successful accesses into the system by day/week
or month? And is there any raising number accesses could make concern?” etc. (I3P
report 2005).
3.4 Recommendations of developing security metrics
A report from I3P SCADA security workshop (2005) could not provide specific
security metrics for SCADA systems however the supporters and scientists agreed
on some recommendations on developing a security metric for SCADA system. They
include 5 major points:
Two major approaches should be used to develop security metrics were
bottom up and top down. The bottom up approach is to identify the objectives of the
system first, then find a suitable metric for the objective and from the metric, they
could determine for the measurement (Payne 2001 and I3P report 2005). An
example of bottom up approach: the objective could be “reduce the number of virus
infections within the company by 30% comparing by 2011”. Then the metric could be
“current ratio of virus alerts to actual infections as compared to the baseline in 2011”
and the measurement could be “number of virus alerts issued to the organization by
month or year”. The top down approach goes from choosing measurement, then
identifying metric and finding objective. For example, the measurement could be
“average number of high vulnerabilities detected per server using Symantec
scanning tool”, the metric could be “changes in number of critical vulnerabilities
detected on servers since last reporting” and the objective could be “reduce the level
23
of detectable vulnerabilities on servers within the company” (Payne 2001 and I3P
report 2005).
The security metrics for evaluating SCADA products, topologies, protocols
and designs could bring more attractive to industry. They could be applied for new
and existing systems. This metrics could be a consumer report metric or a product
design metric (Payne 2001 and I3P report 2005).
Return on investment metric is used to evaluate products, security policies
and security performance. The advantages of this are to provide well understood
form to manager and can be directly applied to bottom line decisions (Payne 2001
and I3P report 2005).
Develop a new security metric based on existing security research and
standards could bring more benefits for both scientists and industry. One specific
recommendation was ISO-17799, Code of Practice for Information Security
Management (ISO 2002).
The most important thing is that develop a security metric is for a real-time
monitoring system. The operators do not have enough time to manually fix all
network traffic and access patterns during system processes. A metric could alerts
the operator to provide an easy to understand state of system to the operators
(Payne 2001 and I3P report 2005).
These recommendations stated some important points when developing a
security metric for a SCADA system. It could help developer to focus on some
important directions when working with SCADA system.
3.4.1 The relationships between security metrics and risk analysis
Security metrics are built to evaluate security states of the system. It also
helps to make automatically decisions or give suggestions for operators. Many of the
decisions which are supported by security metrics go into risk management
decisions. Security risk metrics could be one of the most difficult to determine
because they are relevant to some subjective factors such as capabilities of
adversaries and political or financial consequences of a threat (I3P report 2005,
Jansen 2009, I3P SCADA project).
24
3.4.2 Framework for metric development and evaluation
One possible method to develop and evaluate security metric is developing a
framework for the SCADA system. Figure 6 is an example of a framework for metric
development (Grantz et al. 2003). The framework started identifying target audience.
Different target audience will need a different metric. Target audience could be
different positions in an organization such as managers, engineers and operators
etc. or different organization in a sector such as oil and gas corporations, policy
makers, regulatory agencies, vendors, consultants etc. From target audience, the
next step is to identify metric objectives for each target audience. Measurable
attributes components could cover risk (threats, vulnerabilities and consequences),
engineering (technical measurements to support design and configuration), value
(costs and benefits), assurance (confidence) and performance. From measurable
attributes and metric objectives, the framework will compare with benchmarks of the
system then provide results which are normally visualized for easy analysis. And the
out of the framework is supporting decision making or analysing security levels of the
SCADA system (Grantz et al. 2003).
Figure 6: Framework for metric development (Grantz et al. 2003)
Another example is a security framework which uses data mining techniques
to create models of models for predictions or filtering data (Nhan 2012). The
framework for SCADA system in Figure 7 uses data mining techniques, historical
data and security metrics to filter historical data, predict future event, support security
25
assessment, identify state of the system and report or alarm to manager when
necessary (Nhan 2012).
Figure 7: Security framework for SCADA system (Nhan 2012)
From two frameworks, the audiences could understand how to implemented
or use security metrics in SCADA system. The next section will provide seven
dimensions which are represented for seven important aspects of cyber security in
control system (including SCADA) (Department of Homeland Security 2009).
3.5 Chapter Review
This chapter brings to the audiences general picture of security metrics in
both IT system and SCADA system. It also provides important sectors which
developers of security metrics in SCADA system need to be concentrated. In
addition, it explains how to use security metrics in SCADA system using framework.
Next chapter will focus on answer the research question, security metrics are used to
measure the security metrics for SCADA system but the answer for the questions
“what
are
they?”
and
“how
does
26
it
look
like?”
will
be
given.
Chapter 4. Security metrics in SCADA system
This section will provide some existing security metrics which can be applied
on SCADA system.
4.1 Number of attacks
The idea of this metric is based on number of attacks the system can identify
and count how many attacks on an existing system. This is the most common way to
start building security metric in a system. And the system which has higher number
of attacks; is less secure than others. This idea is developed and used by Jansen
2009, Ortalo et al. 1999 and Pamula et al. 2006.
The original idea of this metric is counting the number of bugs found on
system A and system B. The goal of original idea is to determine or measure
whether system A is “more secure” than system B. Krautsevich et al. (2010) defines
his metric based on number of attacks: system A is more secured than system B if
number of attacks on system A is smaller than number of attacks on system B.
However, in SCADA system, the purpose of using security metrics is to determine a
system is secured or not. This security metric can be applied directly to SCADA
system by pre-determined a safe point for SCADA system. For example, the
benchmark for this metric is the number of attacks on the system in week 20
identified that system is secured. Thus for every week, after finished measurements
of number of attacks, the security metric could be “if the number of attacks on
system in week N is smaller than the number of attacks in week 20, the system is
secured and if the number of attacks on system in week N is bigger than the number
of attacks in week 20, the system is not secured”. It can be applied for months or
years depend on requirements of security of the system. The number of attacks on
system in week 20 could be defined as security goal for the system.
4.2 Probability of attacks
The probability to accomplish an attack successfully is a well-known metric.
The metric identifies how probable is that a hacker is capable of reaching final target.
According to Krautsevich et al. (2010), the author states that to reach the final goal,
the attacker must to do a number of events. In order to get the probability to do an
attack successfully, every action required for the attack must be done successfully.
The author defines maximal probability of attack is equal to the result which is
27
calculated by multiple all probability of each action in one attack. The idea comes
from security metrics for IT system but it can be applied directly to SCADA system
and is useful for industry. For example, to attack successfully a system with attack A,
it requires n actions from a1 to an. The maximal probability of attack A is Pa=
P(a1).P(a2).,,,P(an). The benchmark for probability of attack A is P (xa). If Pa is smaller than
P(xa), the system is not secured. If P(xa) is smaller than Pa, the system is secured. This
security metric can be applied for each attack.
4.3 Shortest Length of attacks
The idea of this metric is the less step attacker has to make, the simpler is to
reach the target and the less secure the system is. According to Ortalo et al. (2005)
and Krautsevich et al. (2010), in the SCADA system it is possible to find some nodes
or targets as the privileges of possible attackers and a path exists between these
nodes to final nodes or final targets which is a cause of a potential security breach.
So the length of this path is long and has many inter-nodes, the attackers have to
spend more time, efforts to attack the target. For each component of SCADA
system, it is possible to evaluate the shortest length from each component to
hackers. Depend on the value of the measurement for the shortest length, it is
possible to set a value of SL for each component or device. For example, the
number of nodes (steps) needed to reach device A is SL a. The operators analyse the
data log files of critical events and find that hackers need SL a’ nodes or steps to
attack device A. If SLa’ is smaller than SLa, the SCADA system is not secured and in
contrast.
4.4 Attack surface metric
This metric has been proposed by Manadhata and Wing (2004, 2005 and
2007). The idea in this metric is the system which has bigger “surface”, is easier to
be attacked than the system which has smaller “surface”. So the system which has
smaller “surface” is more secure than bigger one. There are two challenges to use
this metric in SCADA system: the first one is to define surface of a system, the
second is how to define a safe surface for a system. The system’s surface is smaller
than safe surface is secured, otherwise it is not secured. The authors consider about
three dimensions of an attack surface: targets, channels and access rights. They
state that the more targets or the more channels or the more generous the access
28
rights, the larger the attack surface. This security metric covers a large area in
SCADA system. Assume that attack surface metric is combinations of security
metrics in SCADA system. Each component metric will be used to evaluate one
dimension or aspect of the system, thus the result is the more secured aspects, the
more secured system. The measurement could be the percentage of secured aspect
out of total aspects. The ideal value for this metric is 100% however it could be set
as acceptable value – AS. If the percentage of secured aspects is more than AS, the
system is secured and in contrast.
4.5 Rogue change days
A rogue change is any change to the system configuration without sending
any alarms or notifications to the security staffs. Rogue Change Days are equal to
the number of rogue changes multiplied by the number of days which is used to
recognize the changes by security staffs. For example, if three laptops were added
to the SCADA system without the understandings of security staffs and they did not
discover this change until 7 days. The value of rogue change days is equal to 3
multiplied by 7 and it is 21 rogue change days to the calculation. The ideal value of
this security metric is zero. Any changes in the SCADA system need to be reported
to responsible staffs (Department of Homeland Security 2009).
4.6 Security evaluation deficiency count
The security evaluation deficiency count is the number of devices in the
system which is not covered by security protection and evaluation. The
measurement of this metric is counting how many devices in the system which is
evaluated by a security program. The ideal value of this metric is zero (Department
of Homeland Security 2009).
4.7 Data transmission exposure
To reduce the impacts of hackers on data transmission in SCADA system, all
data which is transferred in or out the system need to be encrypted. The
measurement is how many transmission channels which are used by the SCADA
system do not have any encryptions. For example, if using Telnet to transfer data
between the SCADA system and the Internet, the value of this metric will be added
one because Telnet does not use encryptions while transfer data. The ideal value of
this metric is zero (Department of Homeland Security 2009).
29
4.8 Known vulnerability days
The known vulnerability days metric is calculated by total known and
unpatched vulnerabilities multiplied by their exposure days. The value of this metric
is increasing each day due to the discovery of unknown vulnerabilities increases too.
For example, if there are 5 unpatched vulnerabilities and these vulnerabilities were
discovered 15 days ago. The value of measurement is 5 multiplied by 15 and result
is 75 known vulnerabilities days. The ideal value of this metric is also zero
(Department of Homeland Security 2009).
4.9 Password crack time
The password crack time is the shortest time needed to crack a single
password of any account on the system. The measurement of this metric is the
minimum amount of time for attackers to crack password successful. It will help to
identify the best lifetime for password on the system. For example, the amount of
time for cracking password is 7 days, 9 days and 16 days. So the password crack
time is the shortest time – 7 days for cracking password. The ideal value of this
metric is infinity (Department of Homeland Security 2009).
4.10 Detection mechanism deficiency count
The detection mechanism deficiency count metric is the number of computers
or devices which do not have antivirus programs or any kinds of malware detection
or attack detection mechanisms in the system. For example, there are 140 out of 150
computers which are installed antivirus programs and the antivirus software is
updated. Thus there have 10 computers that do not have any detection mechanisms.
The value of the measurement of this metric is 10. The ideal value of this metric is
zero (Department of Homeland Security 2009).
4.11 Restoration time
The restoration time metric is calculated by the amount of time needed to
restore all devices of a SCADA system from consequences of an attack multiplied by
the number of devices which are affected by the attack. For example, there are 3
RTUs and 1 PLCs stopped working due to an attack on SCADA system and it took 2
hours to restore each devices. The measurement for this metric is 4 multiplied by
30
120 minutes and the result is 480 minutes. The ideal value of this metric is zero
(Department of Homeland Security 2009).
4.12 Chapter review
This chapter introduced some existing security metrics that could come from
security metrics in IT system or could be developed to apply to SCADA system.
Security metrics which are mentioned in this chapter can be applied directly to
SCADA system.
31
Chapter 5. Conclusion
To summary, this minor thesis is completely answering the research question
“how can the security in SCADA system be measured?” To answer this question, the
thesis started introducing and analysing the SCADA system to point out the gap
between SCADA system and IT system. With the increasing of using SCADA system
in controlling critical infrastructure and the raising of new technologies, security in
SCADA system SCADA system with its specialized design and devices bring
difficulties to apply security solutions to measure its security level. Security metric is
the possible solution to measure security level in SCADA system. The thesis also
mentioned some security metrics which can be applied directly to SCADA system as
stimulations to answer the research question.
The answer for the research question was concluded by researching and
analysing SCADA system and its component as well as the development of security
metrics theoretically without any experiments. it could be seen as a limitation of this
research.
The need of developing security metrics in SCADA system is increasing due
to the high requirements of security in SCADA system. This research is the
beginning of study on security metrics area and it will bring benefits for who want to
start working in security environment of SCADA system and general is Process
Control System.
The next step for this research is applying metrics which were developed in
the paper to real system and continuing to improve effectiveness of useful security
metrics. The next step of this research requires supports from security communities,
organizations as well as SCADA project from all over the world.
32
References:
Applied Computer Security Associates (ACSA) 2001, Proceedings of Workshop on
Information-Security-System Rating and Ranking (WISSRR), Williamsburg, Virginia.
ACSA, Silver Spring, Maryland.
Baker, S, Fillipiak, N & Timlin, K 2010, “In the Dark”, McAfee second annual critical
infrastructure protection report, Centre for Strategic and International protection
report.
Barnes, K ,Johnson, B & Nickelson, R 2004, “Review of Supervisory Control and
Data Acquisition (SCADA) Systems”, Idaho National Engineering and Environmental
Laboratory Bechtel BWXT, Idaho, LLC.
Cai, N, Wang, J & Yu, X 2008, ‘SCADA system security: Complexity, history and
new developments’, 6th IEEE International Conference on Industrial Informatics,
Daejeon, Korea.
Chandia, R , Gonzalez, J, Kilpatrick, T, Papa, M & Shenoi, S 2007, ‘Security
Strategies for SCADA networks’, IFIP International Federation for Information
Processing, no. 9, vol. 253, pp. 117-131.
Chikuni, E & Dondo, M 2007, ‘Investigating the security of electrical power systems
SCADA’, AFRICON 2007, pp. 1-7.
CNN Interactive, ‘Teen Hacker Faces Federal Charges’, viewed on 15th November
2011, http://edition.cnn.com/TECH/computing/9803/18/juvenile.hacker/index.html
Coates, G, Hopkinson, K, Graham, S & Kurkowski, S 2010, ‘A Trust System
Architecture for SCADA Network Security’, IEEE Transaction on Power Delivery,
vol.25, no.1, pp. 158-169.
Department of Homeland security 2009, ‘Primer Control Systems Cyber Security
Framework and Technical Metrics’, Control Systems Security Program, National
Cyber Security Division.
Dos Anjos, T, Brito, A & Motta Pires, P 2008, ‘A model for security management of
SCADA systems’, IEEE International Conference on Emerging Technologies and
Factory Automation, pp. 448-51.
Fernandez, EB & Larrondo-Petrie, MM 2010, 'Designing Secure SCADA Systems
Using Security Patterns', System Sciences (HICSS), 2010 43rd Hawaii International
Conference on, 5-8 Jan. 2010, pp. 1-8.
Glantz, C, Stoddard, M, Mcintyre, A, Santos, J, Bodeu, D, O’neil, L and Gennert, B
2003, ‘The Development of Security Metrics for Process Control System’, report to
I3P.
33
Graham, J. & Patel, S 2004, ‘Security Considerations in SCADA Communication
Protocols’, Intelligent Systems Research Laboratory, viewed 23November 2011,
http://www.louisville.edu/speed/cecs/facilities/ISLab/tech%20papers/ISRL-04-01.pdf
Government Accountability Office (GAO) 2005, Department of Homeland Security’s
(DHS’s) Role in Critical Infrastructure Protection (CIP) Cybersecurity, GAO-05-434
(Washington, D.C.: May, 2005), viewed 8 December 2011
http://www.gao.gov/new.items/d05434.pdf
International Standards Organization (ISO) 2002, Information technology -- Systems
Security Engineering -- Capability Maturity Model, Geneva, Switzerland.
Institute for Information Infrastructure Protection (I3P) 2003, Cyber Security
Research and Development Agenda.
I3P SCADA project, SCADA pedia, <http://www.digitalbond.com/scadapedia/>.
I3P report 2005, Process Control System Security Metrics State of Practice, viewed
on 15th November http://stuweb.ee.mtu.edu/~ssmoily/section_4.pdf.
Kang, D, Lee, J, Kim, S & Park, J 2009, Analysis on cyber threats to SCADA
systems, Transmission and Distribution Conference and Exposition: Asia and
Pacific, pp. 1-4.
Keizer, G 2010, ‘Is Stuxnet the ‘best’ malware ever?’, viewed on 15 th November
2011, http://www.infoworld.com/print/137598.
Kuipers, D & Fabro, M 2006, Control Systems Cyber Security: Defense in Depth
Strategies, Homeland Security External Report -06-1478, viewed on 15th November
<http://www.inl.gov/technicalpublications/Documents/3375141.pdf>
Krautsevich, L, Martinelli, F & Yautsiukhin, A 2010, ‘Formal approach to security
metrics. What does “more secure” mean to you?’, ECSA 2010 August, Copenhagen,
Denmark, pp23-26.
Krebs, B 2008, ‘Cyber Incident Blamed for Nuclear Power Plant Shutdown’,
Washingtonpost. Newsweek Interactive.
Manadhata, J & Wing J. M. 2004, ‘Measuring a system’s attack surface’, Technical
Report CMU-TR-04-102, CMU.
Manadhata, J & Wing J. M. 2005, ‘An attack surface metric’, Technical Report CMUCS-05-155, CMU.
Manadhata, J & Wing J. M. 2007, ‘An approach to measuring a system’s attack
surface’, Technical Report CMU-CS-07-146, CMU.
Miller, M & Slay, J 2006, ‘A Security Architecture for SCADA Networks’, the 17th
Australian Conference on Information Systems, Adelaide .
34
NERC, ‘Technical Analysis of August 14, 2003, Blackout: What Happened, Why and
What did we learn?’, viewed on 15th November 2011,
http://www.nerc.com/docs/docs/blackout/NERC_Final_Blackout_Report_07_13_04.p
df.
Nhan, N. T 2012, Final Document of Minor Thesis, Submission to the University of
South Australia, 30th March.
Niland, M 2003, ‘Computer Virus Brings Down Train Signals’, viewed on 15th
November 2011, http://www.informationweek.com/news/13100807.
NTSB (National Transportation Safety Board) 1999, ‘Pipeline Accident Report:
Pipeline Rupture and Subsequent Fire in Bellingham’, Washington.
Ortalo, R, Deswarte, Y & Kaaniche, M 1999, ‘Experimenting with quantitative
evaluation tools for monitoring operational security’, IEEE TSE, pp. 663-650.
Pamula, J 2006, ‘A weakest adversary security metric for network configuration
security analysis’, Proceeding of QoP-06 ACM Press.
Payne, S. C., 2006, ‘A Guide to Security Metrics’, SANS Institute InfoSec Reading
Room, http://www.sans.org/reading_room/whitepapers/auditing/guide-securitymetrics_55.
Poulsen, K 2003, “Slammer worm crashed Ohio nuke plant network”, Security
Focus, viewed 23 November 2011, http://www.securityfocus.com/news/6767
Rashid, F. Y 2012, ‘SCADA Systems in Railways Vulnerable to Attack’, IT Security &
Network Security News.
Robert, PF 2005, “Zotob, PnP Worms Slam 13 Daimler Chrysler Plants”, IT Security
and Network Security News.
Slay, J & Sitnikova, E 2009, 'The Development of a Generic Framework for the
Forensic Analysis of SCADA and Process Control Systems Forensics in
Telecommunications, Information and Multimedia', in Sorell, M (ed), vol. 8, Springer
Berlin Heidelberg, pp. 77-82
Slay, J and Miller, M 2007, “Lessons learned from the Maroochy water breach”, In
Critical Infrastructure Protection (November 2007), vol. 253/2007, Springer Boston,
pp. 73–82.
Stouffer, K, Falco, J and Scarfone, K 2008, Guide to Industrial Control System (ICS)
Security, Nist Special Publication, vol. 800, no.82, pp. i-F14.
Symantec 2010, ‘W32.Stuxnet’, viewed on 15th November 2011,
http://www.symantec.com/security_response/writeup.jsp?docid=2010-071400-312399.
35
Yakkali, H & Subramanian, N 2010, “Efficient design of SCADA system using
minimum spanning tree and the NFR Framework”, Proceedings of the 2010 42 nd
IEEE Southeastern symposium on system theory, Tyler, Texas, pp.346-351.
36