Journal of Risk Research
ISSN: 1366-9877 (Print) 1466-4461 (Online) Journal homepage: https://www.tandfonline.com/loi/rjrr20
Using qualitative types of risk assessments
in conjunction with FRAM to strengthen the
resilience of systems
Kjartan Bjørnsen, Anders Jensen & Terje Aven
To cite this article: Kjartan Bjørnsen, Anders Jensen & Terje Aven (2020) Using qualitative types
of risk assessments in conjunction with FRAM to strengthen the resilience of systems, Journal of
Risk Research, 23:2, 153-166, DOI: 10.1080/13669877.2018.1517382
To link to this article: https://doi.org/10.1080/13669877.2018.1517382
Published online: 06 Dec 2018.
Submit your article to this journal
Article views: 399
View related articles
View Crossmark data
Citing articles: 6 View citing articles
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=rjrr20
JOURNAL OF RISK RESEARCH
2020, VOL. 23, NO. 2, 153–166
https://doi.org/10.1080/13669877.2018.1517382
Using qualitative types of risk assessments in conjunction
with FRAM to strengthen the resilience of systems
Kjartan Bjørnsen, Anders Jensen and Terje Aven
Department of Safety, Economics and Planning, University of Stavanger, Stavanger, Norway
ABSTRACT
ARTICLE HISTORY
In this paper, we look into how some qualitative types of risk assessments can be used in conjunction with functional resonance analysis
method (FRAM) to strengthen the resilience of systems. In FRAM, variation in relation to meeting specified functions is central, but risk and
uncertainty considerations are not an integral part. We suggest to add
to FRAM an assessment of the modelling choices and judgements using
strength of knowledge considerations and a qualitative sensitivity analysis. In this way, an improved basis for assessing and strengthening system resilience with FRAM is established. We illustrate the idea with a
simple example from the oil and gas industry.
Received 13 March 2018
Accepted 17 August 2018
KEYWORDS
Qualitative risk assessment;
complex systems;
knowledge; resilience;
FRAM
1. Introduction
The concept of resilience has received considerable attention in recent years, with a number of
books and research papers published on the subject; see, for example, Hollnagel, Woods, and
€m, Van Winsen, and Henriqson (2015), Righi,
Leveson (2007), Renn (2008), Haimes (2009), Bergstro
Saurin, and Wachs (2015) and Hosseini, Barker, and Ramirez-Marquez (2016). Several interpretations of the resilience concept have been suggested and Woods (2015) provides an overview of
some of the most common ones. In particular, he suggests the following four main categories of
interpretations:
1.
2.
3.
4.
resilience as rebound from trauma and return to equilibrium
resilience as a synonym for robustness
resilience as the opposite of brittleness
resilience as network architectures that can sustain the ability to adapt to future surprises as
conditions evolve.
The basic understanding of resilience in this paper is for the most part in line with (4) and
the ideas outlined in the Society for Risk Analysis (SRA) glossary (SRA 2015): a system is resilient
if it has a strong ability to sustain and restore its basic functionality following some event.
To assess and strengthen the resilience of systems, there exist a number of different
approaches and methods, see for instance Linkov et al. (2013), Hosseini, Barker, and RamirezMarquez (2016), Florin and Linkov (2016) and Linkov and Palma-Oliveira (2017). One approach is
to use the functional resonance analysis method (FRAM), introduced by Hollnagel (2004), to gain
understanding of how a system functions and to use this understanding to assess resilience.
CONTACT Kjartan Bjørnsen
k.bjoernsen@gmail.com
ß 2018 Informa UK Limited, trading as Taylor & Francis Group
University of Stavanger, Stavanger, Norway
154
K. BJØRNSEN ET AL.
Resilience is then looked at as an emerging property of the system (similarly to safety; Yang,
Tian, and Zhao 2017).
Using FRAM to assess resilience is often seen as an alternative approach to traditional risk
assessments for achieving desired system performance and avoiding adverse outcomes. For
example, in relation to organizational change, Hollnagel (2013) compares a traditional qualitative
risk assessment approach, based on hazard identification and likelihood judgements, to FRAM.
Lundblad et al. (2008) propose using FRAM as an alternative to traditional human reliability analysis methods in the context of a nuclear power plant. Pereira (2013) discusses the use of FRAM
as an alternative to risk assessment for a radiopharmaceutical dispatches process while Rosa,
Haddad, and Carvalho (2015) compare FRAM to traditional risk assessment methods for assessing
occupational safety in construction. Others also use FRAM as a supplement to traditional risk
assessments in order to assess domains where risk assessments are found unsuited (e.g.
Belmonte et al. (2011) and Smith et al. (2017))
The efforts to find alternative approaches to traditional risk assessments are motivated by a
rather strong critique of such assessments. Many authors have argued that traditional risk assessments, based on hazard identification, event chains and probability estimation, are unsuited in
the context of complex systems (e.g. Dekker et al. (2008), Leveson (2004, 2011) and Hollnagel
(2014)). It is argued that in such a context, reliable probability estimates cannot be produced
due to the large uncertainties of what types of events can occur, and how the events can occur.
The underlying assumption of such risk assessments, that desired system performance can be
achieved simply by preventing failures, errors and mishaps, is questioned (see Hollnagel (2014),
Dekker (2014) and Leveson (2011)). Using FRAM to assess resilience is seen to represent a way
forward in this regard, as this approach does not rely on these traditional perspectives
and approaches.
The focus of FRAM is on the functions of the system rather than on the components. The
potential interactions between these functions are modelled in a network in order to better
understand how the system operates as a whole, and to propose ways to ensure desired system
performance. A central idea in FRAM is to understand variation in relation to meeting the functions. In what situations are the functions met, partly met or not fulfilled. Coarse qualitative
judgements are conducted in order to characterize the variation. However, explicit judgements
or assessment of risks and uncertainties are not an integral part of FRAM. Some authors have
pointed to this fact and called for such judgements and assessments (Bjerga, Aven, and Zio
2016). The key argument supporting the need for such judgements and assessments is that the
model of the system studied with its functions and variation that is generated by FRAM (the
FRAM model) is just that – a model which not necessarily describes the system at hand accurately. Any model of how a system works can be more or less good in representing how the system works and can also be based on erroneous assumptions, data and opinions. We thus face
potential deviations between the model and the real system and its performance and, consequently, also uncertainties and risks. Efforts to improve resilience based on FRAM could thus turn
out to be misguided.
However, these authors have not developed or presented an approach or method for how to
deal with this issue. For example, it is not straightforward how to characterize the risk and uncertainties in FRAM to improve resilience assessments. The present paper addresses this issue and
shows how a certain qualitative risk assessment approach can be applied in conjunction with
FRAM. This approach is not a traditional qualitative risk assessment approach, in the sense that it
considers specific hazards and threats, constructs pathways and assigns likelihoods, as described
in, for example, Altenbach (1995), Coleman and Marks (1999), Wooldridge (2008), Gritzalis et al.
(2018) and Bromfield, Corin, and Atapattu (2018). Such qualitative assessments would also be
subject to the above critique of risk assessments, and thus be problematic in the context of complex systems. Here we adopt a new type of qualitative analysis, highlighting uncertainties and
knowledge aspects, based on a broader perspective on risk, in line with Aven (2017a).
JOURNAL OF RISK RESEARCH
155
By incorporating this type of analysis to FRAM, the current work contributes to the further
development of FRAM as a method for assessing resilience and hazards in complex systems.
Several authors have contributed to this end by, for example, allowing for exhaustive searches of
scenarios by combining FRAM with model checking methods (Zheng, Tian, and Zhao 2016; Tian
et al. 2016), reducing subjectivity in the characterization of variation (Rosa, Haddad, and Carvalho
2015) and proposing a framework for using Monte Carlo simulations to evaluate the variation
(Patriarca, Di Gravio, and Costantino 2017). The current work is distinct from these contributions in
the sense that this work does not aim at improving the ability of FRAM to accurately represent an
actual system, or identify scenarios within the constructed model. Rather, the work suggests how
risk and uncertainties can be characterized and taken into account in FRAM, such that an improved
basis for understanding and strengthening resilience is established. In this way, the work contributes to improving the practical application of FRAM, and also, more broadly, to the academic discussion of how risk assessments can complement and contribute to resilience assessments and
vice versa (see discussions in Park et al. (2013), Linkov et al. (2013, 2014) and Aven (2017a)).
The research approach applied in this work can be described as analytical, fundamental and
conceptual, as described by Kothari (2004). It is analytical in the sense that we reflect on already
available information, fundamental in the sense that it is generic and relevant in many different
settings, and conceptual in the sense that it concerns primarily abstract concepts and ideas. The
fundamental research approach entails that the ideas in this paper are not illustrated on a detailed
and realistic case, but rather on a quite simple and illustrative example. We refer to the discussion
by Aven (2018) about the role of fundamental and conceptual research in risk analysis science.
The paper is organized as follows. In Section 2, we give an introduction to FRAM, highlighting
its relationship to resilience. In Section 3, we discuss the need for considering risk and uncertainty and propose a method for including the announced risk and uncertainty assessment in
FRAM, before illustrating the method in Section 4 with an example. In Section 5, we provide a
discussion of this method and, finally, in Section 6, we make some concluding remarks.
2. FRAM
The purpose of this section is to present the main ideas of FRAM and illustrate how it relates to
the concept of resilience. To do this we use simple example. The example is revisited throughout
the paper. We first look more closely into the concept of resilience, to be used for the coming
analysis. As mentioned in Section 1, the basic idea of resilience is that a system is resilient if it
has a strong ability to sustain and restore its basic functionality following some event (SRA 2015).
To illustrate this idea, we consider a gas processing plant. Several types of events can occur
disturbing the gas processing activity. For the gas processing plant to be resilient, it must, in line
with the idea above, have a strong ability to sustain as well as restore the gas processing activity
following an event. To be more concrete, let X represent the state of the system defined by
X ¼ 2: normal functioning
X ¼ 1: gas leak
X ¼ 0: ignited gas leak.
If the system is in state X ¼ 0, a leakage has occurred and the gas has been ignited: the processing is stopped. If the system is in state X ¼ 1, a gas leak has occurred, but the gas is not
ignited (but it could be at a later stage). The gas processing is stopped. The desired state is state
X ¼ 2, where no gas leak has occurred and gas is being processed. The idea of resilience relates
to the gas processing plant’s ability to remain in State 2 following an event, for example,
changes in pressure and temperature, and its ability to return to state X ¼ 2 and not to enter
state X ¼ 0, if it has entered state X ¼ 1 (a leakage has occurred).
156
K. BJØRNSEN ET AL.
We now look more closely into FRAM. For a more thorough introduction to FRAM, the reader
is referred to Hollnagel (2012, 2016). We leave recently proposed modifications of FRAM out of
the discussion (see Rosa, Haddad, and Carvalho (2015), Tian et al. (2016) and Patriarca, Di Gravio,
and Costantino (2017)), as these are less relevant to the suggestions made in this paper.
The first step of FRAM is to build the FRAM model, comprising a list, consisting of the functions
that are necessary for everyday work to succeed, and a corresponding description of these that
defines how the functions potentially can be coupled together (Hollnagel 2012). After constructing
the FRAM model, the next step is to characterize how the fulfillment of the functions varies
(Hollnagel 2012). We note here that Hollnagel (2012) refers to this step as characterizing the variability of functions. However, we find the notion of a function varying somewhat unclear in this setting,
as this term could also refer to the purposes or goals of subsystems varying in relation to the larger
context. For example, we can think of a pump on an offshore oil and gas installation that, depending on the situation, can be used to supply firewater to the processing area, or to supply drinking
water to the living quarter. Such variation in function is, however, not the focus of Step 2 in FRAM.
Rather, the focus is the degree to which subsystems succeed in performing their functions.
To illustrate, consider a separator, whose function is to process gas, that is, to separate liquids
from gas. We look at a set of time periods and measure whether or not the given function is fulfilled. The function is fulfilled if the gas exiting the separator contains no liquids; otherwise, it is
not fulfilled or partly fulfilled. Over time, we will thus observe variation in the degree to which
this function is fulfilled.
Having characterized the variation, the next step is to assess how the variation resonates
through the FRAM model and influence the overall system performance. The idea is that not fulfilling one function can influence the degree to which other functions are fulfilled, thus creating
what Hollnagel (2012) refers to as functional resonance. The method for identifying functional
resonance entails using the FRAM model; however, no specific approach is suggested. The goal
of the fourth and final step of FRAM is to prevent undesirable system performance and facilitate
desired system performance by proposing measures to manage the functional resonance.
Performing FRAM can contribute to strengthen system resilience in the sense that adverse
outcomes can be prevented on the basis of resilience engineering principles, such as focusing
on normal performance variation rather than errors and failures (Lundblad et al. 2008; Hollnagel
2013). It can also be considered to strengthen the resilience in the sense that a system’s ability
to adapt and function under changing conditions can be assessed and improved with FRAM
(Aguilera et al. 2016; Pereira 2013). We can illustrate this process of strengthening resilience with
FRAM by again considering the above gas processing plant.
In this setting, FRAM can be used to investigate how variation in the degree to which functions are met can ultimately cause changes in the system state and propose ways to prevent the
system from entering undesirable system states as well as ‘quickly’ returning to a desirable state
if entering an undesirable state. For example, consider the separator in the gas processing plant
mentioned above. Then we are interested in how the degree to which the function to separate
liquids from gas is met, affects the state of the system. Using the FRAM model and assessing
possible functional resonance, several events are possible, both ‘everyday’ events and ‘extreme’
events. For example, if the function to separate liquids from gas is not met, that is, the gas contains liquid, a pressure build-up can occur, ultimately causing the system to change state.
Following the assessment of possible functional resonance, we continue to identify measures to
strengthen the system’s ability to sustain and restore its functionality (resilience).
3. Adding risk and uncertainty judgements to FRAM
In FRAM, the causes of variation in the fulfillment of a function are not restricted to other
upstream functions. External and internal factors, such as weather conditions and psychological
JOURNAL OF RISK RESEARCH
157
factors, can also cause variation (Hollnagel 2012). In addition, the way the functions are coupled
is not considered static, but can differ depending on the situation (Hollnagel 2012). Hence, the
number of potential scenarios of functional resonance can be extensive and some judgements
about which scenarios are realistic, or worth exploring, are required. For example, a default
assumption in FRAM is that technical equipment has relatively stable performance (Hollnagel
2012). In addition, the variation associated with different functions is characterized with crude
likelihood judgements prior to the search of scenarios (Hollnagel 2012). When model checking is
used with FRAM to make exhaustive searches for scenarios, rules about how variation can propagate between functions have to be established (Tian et al. 2016). These types of judgements can
always be erroneous, and it is in relation to these we suggest that broad risk assessments can
strengthen the resilience assessment with FRAM.
Traditional types of quantitative and qualitative risk assessment methods (e.g. Kumamoto
and Henley (1996) and Coleman and Marks (1999)) are difficult to use to characterize the risk
associated with erroneous judgements in FRAM. These traditional approaches focus on specific
hazards, pathways and consequences in order to arrive at reliable probability estimates.
However, new developments within the risk field provide more suitable approaches. It is argued
that traditional ways of conceptualizing and expressing risk based on probabilities (e.g. Kaplan
and Garrick (1981)) are too narrow (Aven 2012). The main argument for this conclusion is that
any probability is conditional on some background knowledge, which could be more or less
strong. For example, a probability assignment of, say, 0.2, may be based on a large amount of
relevant data, or on scarce and less relevant data. The number in itself does not reflect this
aspect of uncertainty but the two situations are clearly different. The issue has led to new
approaches of characterizing and assessing risk by supplementing traditional approaches with
strength of knowledge (SoK) considerations (e.g. Berner and Flage (2016), Aven and Flage
(2017) and Aven (2017b)), and new approaches that focus primarily on SoK considerations
(Aven, 2017a).
The SoK concept is applicable in conjunction with FRAM since the results of FRAM are conditional on potentially weak background knowledge in the same way as the results of a risk assessment. For example, there may be unjustified assumptions underlying the structure of the model
or the crude likelihood judgements about variation may be based on scarce information. It is by
focusing on the strength of this background knowledge, and the consequences of it being erroneous or inaccurate, we suggest that risk can be characterized in FRAM to benefit resilience
assessments. In the following, a concrete approach is proposed, which we suggest to be added
as an additional step in FRAM, prior to the final step of identifying measures to strengthen system resilience.
The proposed approach has two main purposes: first, to identify modelling choices and judgements that are based on less strong knowledge and, second, to reflect these elements’ importance for the FRAM results. Note that the main goal of the method is not to reveal and detect
errors made in the assessment but rather to reflect upon the consequences related to deviations
in the current beliefs and to provide a basis for further discussion and analysis. The suggested
approach is carried out as follows. First, we identify modelling choices and judgements in the
FRAM assessment. This can be done using the guidewords (1–5) provided by Bjerga, Aven, and
Zio (2016). How to use this list of guidewords depends on the specific context; however, as a
general suggestion, the following statements can be considered in relation to (1–5) in order to
identify modelling choices and judgements:
1.
2.
Functions
a. All relevant functions are included?
b. The level of detail is sufficient to identify and prevent unwanted events?
System parts
a. Critical system parts are reflected by the functions?
158
3.
4.
5.
K. BJØRNSEN ET AL.
System environment
a. All relevant circumstances are taken into account?
Dependencies
a. All relevant dependencies are reflected in the model?
b. The way functions interact is known?
Variation in the system
a. The basis for the crude likelihood judgements is strong?
b. The basis for selecting events is strong?
Using the resulting list, the next step is to assess the SoK on which the modelling choices
and judgements are based. Several approaches can be used for this purpose, for example, the
approach suggested in Flage, Aven, and Zio (2009) (see also Berner and Flage (2016) and Aven
and Flage (2017)). Following the approach, aspects such as models, data, assumptions and expert
judgements are evaluated to give SoK characterizations. A similar approach is the NUSAP
(Numeral, Unit, Spread, Assessment and Pedigree) system (e.g. Funtowicz and Ravetz (1990),
Kloprogge, Van der Sluijs, and Petersen (2011) and Van Der Sluijs et al. (2005)).
Next, we assess how changes in modelling choices and judgements would affect the FRAM
results. Since FRAM, currently, is a primarily qualitative method, traditional quantitative sensitivity
methods such as described in Saltelli, Chan, and Scott (2009) are difficult to apply. However, the
point of this step is to make judgements about the importance of these elements in regard to
the FRAM assessment, not to provide quantitative sensitivities. Rather, we suggest qualitative
judgements similar to those in Flage and Aven (2009). In their paper, sensitivity is scored, based
on how relative changes in base case influence the results and conclusions of the
risk assessment.
To summarize our suggested approach, we first identify modelling choices and judgements,
before we assess the SoK in relation to each one. Following this, an assessment of the importance of these modelling choices and judgements is carried out. Finally, these assessments are
used to provide a basis for further discussion and analysis. In Section 4, we provide an example,
illustrating how to apply this approach as an additional step in FRAM. We further aim at illustrating how adding the approach can improve a resilience assessment with FRAM.
4. Example: including qualitative risk judgements in FRAM of a gas
processing plant
Again, we consider the gas processing plant introduced in the previous sections. Before we
move on, it should be noted that we have aimed to keep the example (and also the FRAM
model) quite simple to illustrate the main ideas and make it accessible for those without technical understanding of gas processing plants. In a real application, the model would be more
detailed and could also include several abstraction levels. The reader is referred to Patriarca,
€m, and Di Gravio (2017) for an approach on how to do this.
Bergstro
We are interested in the resilience of the gas processing plant with respect to gas leaks. Recall
the system states defined in Section 2. For the system to be resilient, it must possess a strong
ability to remain in State 2 following some event. Additionally, if the system enters State 1, it
needs the ability to quickly return to State 2. We use FRAM to investigate how variation in relation
to not meeting functions can influence the system performance and to propose ways to
strengthen the resilience.
This FRAM approach is divided into five steps: first, to identify and describe the necessary
functions; second, to assess variation in the degree of function achievement; third, to assess how
this variation affects the state of the system; fourth, to carry out qualitative risk judgements; and
finally, fifth, to use the previous results to propose measures to strengthen the resilience.
JOURNAL OF RISK RESEARCH
159
Figure 1. FRAM model of the gas processing plant processing incoming gas.
Step 1: identify and describe functions
We start FRAM by first identifying the functions that are necessary for the gas processing plant
to perform as desired. An instantiation of the FRAM model is illustrated in Figure 1.
The functions are coupled by using the input and output aspects from Hollnagel (2012, 46):
Input (I): that which the function processes or transforms or that which starts the function.
Output (O): that which is the result of the function, either an entity or a state change.
Preconditions (P): conditions that must exist before a function can be carried out.
Resources (R): that which the function needs when it is carried out (execution condition) or consumes to produce the output.
Time (T): temporal constraints affecting the function (with regard to starting time, finishing time
or duration).
Control (C): how the function is monitored or controlled.
In Figure 1, the functions are represented by hexagons, each corner representing an aspect:
input, output, precondition, resources, time and control. The functional couplings are represented by lines, connecting one function’s aspect(s) to another function’s aspect(s). For example,
160
K. BJØRNSEN ET AL.
Figure 2. Variation in system states.
the function ‘To prevent ignition’ is coupled with the function ‘To restart production’, using the
aspects, output and input. Likewise, the function ‘To supply gas’ is also coupled with the function ‘To restart production’, but, here, the functions are coupled using the time aspect and the
input aspect.
Step 2: functions and variation
Next, we assess variation in function achievement. To understand what variation means in this
setting, we can consider a set of thought-constructed time periods and measure whether a given
function is fulfilled or not in the particular period.
For instance, for the function ‘To supply gas’, the function is fulfilled if the flow X [stdm3/min]
is in the interval [a,b], where a,b 2 R. Over time, function fulfillment can be characterized using
the following categories:
V-1. X 2 ½a; b
V-2. X 2 ðb; 1Þ
V-3. X 2 ð1; aÞ.
The V-1 category is desirable and, if observed, the function is fulfilled. However, over time, we
will observe V-1, V-2 and V-3 and interpret this as variation in the degree to which the function
is fulfilled. In FRAM, coarse qualitative judgements can also be conducted in order to characterize
the future variation. For example, category V-3 could be characterized as ‘unlikely’
(Hollnagel 2012).
Step 3: accumulation of variation
In this step, we are interested in how the variation in function achievement leads to variation in
the state of the system. To understand what variation in the state of the system means, we can
consider a set of thought-constructed time periods and observe the state of the system. Recall
the defined system states:
X ¼ 2: normal functioning state
X ¼ 1: gas leak
X ¼ 0: ignited gas leak.
It is preferable to be in State 2 and not to enter States 1 or 0. Additionally, if the system
enters State 1, we want it to quickly return to State 2. Over time, we observe both States 1 and
JOURNAL OF RISK RESEARCH
161
2 and interpret this as variation in system states. An illustration is provided in Figure 2. Note
that, if the process enters State 0, the process ends. The states are illustrated in Figure 2: States
0, 1 and 2 on the vertical axis, time periods on the horizontal axis.
Using the network in Figure 1 and the characterizations of variation in Step 2, we continue
identifying events where function achievement accumulates through the FRAM model, causing
changes in system states (identifying functional resonance). One example could be the functions
‘To supply gas’ and ‘to maintain equipment’ not being fulfilled, causing the function ‘To contain
gas’ to not be fulfilled, ultimately leading the system to change from State 2 to State 1. In the
following Step 4, we illustrate how to supplement FRAM with risk considerations.
Step 4: further risk-based considerations
In the previous steps of the assessment, a number of modelling choices and judgements related
to dependencies, system parts, functions, system environment and the variation in the system
were made. Thus, we face potential deviations between the model and the real system and its
performance and, consequently, also uncertainties and risks. In this section, we illustrate how the
suggested approach in Section 3 can be applied as an additional step of FRAM to assess and
characterize these uncertainties and risks.
We start by identifying modelling choices and judgements in FRAM, using the list in Section 3
as a basis. One approach is to use each question in the list during, for example, a brainstorming
session, in order to identify modelling choices and judgements. To illustrate, by using the first
question 1. a. – All relevant functions are included, the modelling choice of not including functions representing management is identified. Another example is question 4. a. – All relevant
dependencies are reflected in the model. From this question, the choice of not including any
dependency from the function ‘To maintain equipment’ to the function ‘To monitor for leakage’
is identified. Accordingly, the resulting list of modelling choices and judgements is
1.
2.
No management functions
No dependency from the function ‘To maintain equipment’ to the function ‘To monitor
for leakage’.
We proceed to assess the SoK related to (1) and (2), using the approach outlined in Aven and
Flage (2017) as a guide for the assessment, but other methods are available (see Section 3).
Following the approach, the knowledge supporting (1) and (2) is categorized as weak, if one or
more of the following is true (whenever relevant):
The assumptions made represent strong simplifications.
Data/information are/is non-existent or highly unreliable.
There is strong disagreement among experts.
The phenomena involved are poorly understood, models are non-existent or known/believed
to give poor predictions.
If, on the other hand, all of the following are true, then the knowledge supporting (1) and (2)
is categorized as strong (whenever relevant):
The assumptions made are seen as very reasonable.
A large amount of reliable and relevant data/information is available.
There is broad agreement among experts.
The phenomena involved are well understood, models are known to give prediction with
required accuracy.
162
K. BJØRNSEN ET AL.
Figure 3. Assessment of modelling choices and judgements.
Cases in between are categorized as moderate knowledge.
For (1) – ‘no management functions’, the SoK can be categorized as moderate. The rationale
is that the modelling choice only represents a minor simplification, there are some data supporting this choice, and experts agree that this is reasonable. The phenomenon involved is also fairly
well understood; hence, the categorization is moderate. As for (2) – ‘No dependency from the
function “to maintain equipment” to the function “to monitor for leakage”’, the SoK can be evaluated as weak, as there is little data available.
Following the SoK considerations, we proceed to investigate the importance of (1) and (2). A
central question is: what would happen if (1) and (2) were changed? To illustrate, how would
the assessment have looked if the management functions were included? And how would the
FRAM assessment have looked if a dependency from the function ‘to maintain equipment’ to the
function ‘To process gas’ was included? There are several ways in which this can be assessed. In
this example, we are, for the most part, interested in illustrating the main idea, and a simple
scoring system suffices. For example:
Low: the FRAM results are slightly changed.
Medium: the FRAM results are moderately changed.
High: the FRAM results are significantly changed.
For (1) – ‘no management functions’, the degree of importance can be categorized as
medium, as this is believed to moderately change the FRAM assessment. For (2) – ‘No dependency from the function “to maintain equipment” to the function “to monitor for leakage”’, the
degree of importance is characterized as high, as this is believed to significantly change the
assessment. The assessment is illustrated in Figure 3. The SoK is represented by the vertical axis,
and the degree of importance is represented by the horizontal axis.
Similar considerations can be carried out for the remaining modelling choices
and judgements.
Step 5: proposing ways to manage variation
On the basis of the events found in Step 3, we proceed to identify ways to manage variation in
function achievement which can cause undesirable system states. Different strategies can be
used. For example, measures that strengthen the system’s ability to remain in State 2 (strengthen
robustness and flexibility) or measures that strengthen the system’s ability to return to State 2
given State 1 (strengthen the recoverability). Concrete examples could be to propose regular
maintenance courses for the staff and better monitoring of the gas flow into the production
facility in order to dampen variation in the function fulfillment of ‘To contain gas’. Such a process
JOURNAL OF RISK RESEARCH
163
of identifying measures would follow the standard FRAM. However, having introduced the additional Step 4, we also benefit from the insights gained in this step when seeking to
strengthen resilience.
The insights gained from Step 4 can be used to strengthen resilience in different ways. For
example, we can consider 1. – ‘No dependency from the function “to maintain equipment” to
the function “to monitor for leakage”’. This was based on weak knowledge and judged to have
high importance. This means that, if there is dependency between the two functions, it is likely
that there are relevant events leading to undesirable system states not covered in the FRAM
assessment. A measure for this issue is to carry out a new assessment of events, with the
dependency included, and subsequently propose ways to handle these events. However, there
are also other ways to handle this issue, for example, by collecting data and monitoring the
dependency during the operation of the system. There may also be measures that can be taken
such that the potential interdependency becomes less important. All of these suggested measures can be seen as a way to strengthen resilience. However, there is a difference between these
measures and the measures found in the standard FRAM assessment. While the measures found
in FRAM are based on the current beliefs and best judgements, the measures found here are
identified on the basis of risk and uncertainty considerations, that is, by considering what happens if the current beliefs and best judgements are wrong or inaccurate.
5. Discussion
Above we have illustrated how FRAM can be used to assess and strengthen system resilience.
The main aim of this article has been to argue and illustrate that for such application of FRAM,
adding some types of qualitative risk assessments contribute to a better understanding of resilience. In this sense, the current work can be seen as addressing the challenges of complex systems. One such challenge is the lack of accurate prediction models for complex systems, and
hence the difficulty in specifying measures to ensure desired performance of the system. By adding qualitative types of uncertainty judgements to FRAM, this challenge is not solved as events
contrary to the knowledgebase, so called black swans, may still occur. For example, the SoK
assessment incorporated in FRAM is also based on some knowledge that can be wrong or
inaccurate. Nevertheless, a richer knowledge basis is established for understanding functional resonance and suggesting measures to strengthen the resilience through the assessment.
The key idea is that judgements made in FRAM might be based on poor knowledge and that
this should also be reflected in FRAM when it is used to assess resilience. In order to do this, we
have added an extra step for including some types of qualitative risk assessments, highlighting
knowledge aspects. The aim of the step was to identify and assess the knowledge supporting
the modelling choices and judgements related to dependencies, system parts, functions, system
environment and the variation in the system. By carrying out such an assessment, the basis for
identifying events and measures is broadened, as we not only take into account the current
beliefs and best judgements but also investigate what happens if the current beliefs and best
judgements are wrong or inaccurate. This investigation could lead to the identification of events
that, due to weak knowledge and large consequences, should have measures taken against
them, even if the current knowledge does not support the occurrence of the events. Hence, the
current work contributes to better understanding of resilience, in the sense that surprises relative
to our knowledge are addressed.
The qualitative risk assessments proposed in conjunction with FRAM in this paper can also be
of value for other applications of FRAM than resilience assessments. For example, Smith et al.
(2017) show how FRAM can complement safety assessments using fault tree analysis (FTA) and
Bayesian networks (BN). In such a context, identifying modelling choices and judgements in the
FRAM model that are supported by weak knowledge can be of value as these choices and
164
K. BJØRNSEN ET AL.
judgements may influence, and hence contribute to uncertainty, in the FTA and BN results. In
this way, considerations of risk and uncertainty in FRAM are not only beneficial when FRAM is
used to strengthen system resilience, but also when FRAM is used as a basis for risk and safety
assessments.
In the suggested approach, we included in Step 4 both an assessment of the strength of the
knowledge and an importance assessment. Assessing an element to be important as well as
uncertain means that it can be possible to generate alternative FRAM models for which a number of new measures can be identified. Such a strategy was exemplified in Step 5 in Section 4.
Following this line of thinking, it is a challenge to identify measures that are efficient for the
new models as well as the initial model, that is, finding robust measures. The current work does
not directly address this challenge, and this could be considered a limitation. However, the proposed approach can be seen as a step towards facilitating the identification of such robust measures with FRAM. A second limitation or drawback of the proposed method is that it increases
the workload of FRAM. The extent of the risk and uncertainty judgements to be added must be
balanced against the expected insights gained, and will depend on the scope of the analysis and
the resources available.
6. Conclusion
A challenge of complex systems is the difficulty in foreseeing events and predicting their effect
on system performance. The resilience concept represents an attractive way of thinking in this
regard. The key idea is that desired system performance is not achieved through prevention of
specified undesirable events but, rather, through increasing the ability of the system to sustain
and restore its basic functionality following an event. FRAM can be used to assess and
strengthen the resilience of a system. The main contribution of this paper has been to illustrate
how certain types of qualitative risk assessments can be applied in conjunction with FRAM in
such a context. Considerations of risk and uncertainties in FRAM have previously been identified
as a need, but traditional qualitative and quantitative risk assessment methods are less suited.
We have suggested a solution to the challenge based on the newly developed SoK concept. In
particular, we suggested including an additional step in FRAM, in which modelling choices and
judgements concerning functions, system parts, system environment, dependencies and variation
are evaluated in terms of SoK and importance. This strengthens the resilience assessment when
using FRAM, as surprising events relative to the current knowledge can be identified and mitigated. In this sense, adding risk and uncertainty considerations to FRAM contributes to better
meeting the challenges of complex systems.
The suggested approach was illustrated with a quite simple example of an oil and gas processing system. Hence, a demonstration of the approach on a realistic case has not been provided
in this paper. However, a simple example serves the primary purpose of this work, which is to
provide general motivation and principles for including risk and uncertainty considerations in
resilience assessments with FRAM. The suggested approach should of this reason be considered
as a point of departure for generating adapted versions of FRAM, suited for different types of
settings. The issue of demonstrating and adapting the approach for more realistic cases is considered a subject for future research. Such research should also compare the increased workload
of the approach against the benefits gained, as well as the managerial implications of implementing the approach in practice.
Acknowledgements
The authors are grateful to an anonymous reviewer for the useful comments and suggestions to the original version of this paper.
JOURNAL OF RISK RESEARCH
165
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
The work has been funded by the Norwegian Research Council as a part of the Petromaks 2 program under grant
number 228335/E30. The support is gratefully acknowledged.
References
Aguilera, M. V., B. Cabrera da Fonseca, T. K. Bastos Ferris, M. C. R. Vidal, and P. V. R. de Carvalho. 2016. “Modelling
Performance Variabilities in Oil Spill Response to Improve System Resilience.” Journal of Loss Prevention in the
Process Industries 41: 18–30.
Altenbach, T. J. 1995. “A Comparison of Risk Assessment Techniques from Qualitative to Quantitative.” Joint
American Society of Mechanical Engineers (ASME)/Japan Society of Mechanical Engineers (JSME) pressure vessels
and piping conference, Honolulu, July 23-27.
Aven, T. 2012. “The Risk Concept—Historical and Recent Development Trends.” Reliability Engineering & System
Safety 99: 33–44. doi:https://doi.org/10.1016/j.ress.2011.11.006
Aven, T. 2017a. “How Some Types of Risk Assessments Can Support Resilience Analysis and Management.”
Reliability Engineering and System Safety 167: 536–543. doi:10.1016/j.ress.2017.07.005.
Aven, T. 2017b. “Improving Risk Characterisations in Practical Situations by Highlighting Knowledge Aspects, with
Applications to Risk Matrices.” Reliability Engineering & System Safety 167: 42–48. doi:10.1016/j.ress.2017.05.006.
Aven, T. 2018. “Reflections on the Use of Conceptual Research in Risk Analysis.” Risk Analysis. doi:10.1111/
risa.13139.
Aven, T., and R. Flage. 2017. “Risk assessment with broad uncertainty and knowledge characterisations: An illustrative case study.” Chap. 1 in Knowledge in Risk Assessments, edited by, Wiley.
€n, L. Heurley, and R. Capel. 2011. “Interdisciplinary Safety Analysis of Complex SocioBelmonte, F., W. Scho
Technological Systems Based on the Functional Resonance Accident Model: An Application to Railway Traffic
Supervision.” Reliability Engineering & System Safety 96 (2): 237–249. doi:10.1016/j.ress.2010.09.006.
€m, J., R. Van Winsen, and E. Henriqson. 2015. “On the Rationale of Resilience in the Domain of Safety: A
Bergstro
Literature Review.” Reliability Engineering and System Safety 141: 131–141. doi:10.1016/j.ress.2015.03.008.
Berner, C., and R. Flage. 2016. “Strengthening Quantitative Risk Assessments by Systematic Treatment of Uncertain
Assumptions.” Reliability Engineering and System Safety 151: 46–59. doi:10.1016/j.ress.2015.10.009.
Bjerga, T., T. Aven, and E. Zio. 2016. “Uncertainty Treatment in Risk Analysis of Complex Systems: The Case of
STAMP and FRAM.” Reliability Engineering and System Safety 156: 203–209.
Bromfield, K. E., S. Corin, and A. Atapattu. 2018. “A Qualitative Binary Risk Assessment Model for Regulating the
Biosecurity and Environmental Risk of Endophytes.” Biological Control 116: 46–52.
Coleman, M. E., and H. M. Marks. 1999. “Qualitative and Quantitative Risk Assessment.” Food Control 10 (4–5):
289–297.
Dekker, S. 2014. Safety Differently: Human Factors for a New Era. Boca Raton, FL: CRC Press.
Dekker, S., E. Hollnagel, D. Woods, and R. Cook. 2008. Resilience Engineering: New Directions for Measuring and
Maintaining Safety in Complex Systems. Sweden.
Flage, R., T. Aven, and E. Zio. 2009. “Alternative representations of uncertainty in system reliability and risk analysisReview and discussion.” Joint ESREL (European Safety and Reliability) and SRA-Europe (Society for Risk Analysis
Europe) Conference, Valencia.
Flage, R., and T. Aven. 2009. “Expressing and Communicating Uncertainty in Relation to Quantitative Risk Analysis.”
Reliability and Risk Analysis: Theory & Applications 2 (13): 9–18.
Florin, M. V., and I. Linkov. 2016. “International Risk Governance Council (IRGC) Resource Guide on Resilience.”
https://www.irgc.org/risk-governance/resilience/.
Funtowicz, S. O., and J. R. Ravetz. 1990. Uncertainty and Quality in Science for Policy. Netherlands: Springer.
Gritzalis, D., G. Iseppi, A. Mylonas, and V. Stavrou. 2018. “Exiting the Risk Assessment Maze: A Meta-Survey.” ACM
Computing Surveys 51 (1): 1. doi:https://doi.org/10.1145/3145905
Haimes, Y. Y. 2009. “On the Definition of Resilience in Systems.” Risk Analysis 29 (4): 498–501. doi:10.1111/j.15396924.2009.01216.x.
Hollnagel, E. 2004. Barriers and Accident Prevention. Surrey, UK: Ashgate.
Hollnagel, E. 2012. FRAM, The Functional Resonance Analysis Method: Modelling Complex Socio-technical Aystems.
Surrey, UK: Ashgate.
166
K. BJØRNSEN ET AL.
Hollnagel, E. 2013. “An Application of the Functional Resonance Analysis Method (FRAM) to Risk Assessment of
Organisational Change”. Swedish Radiation Safety Authority. Report 2013:09.
Hollnagel, E. 2014. Safety-I and Safety–II: The Past and Future of Safety Management. Surrey, UK: Ashgate.
Hollnagel, E. 2016. “A FRAM glossary.” Accessed 26 May 2017. http://functionalresonance.com/a-fram-glossary.html.
Hollnagel, E., D. D. Woods, and N. Leveson. 2007. Resilience Engineering: Concepts and Precepts. Surrey, UK: Ashgate.
Hosseini, S., K. Barker, and J. E. Ramirez-Marquez. 2016. “A Review of Definitions and Measures of System
Resilience.” Reliability Engineering and System Safety 145: 47–61. doi:10.1016/j.ress.2015.08.006.
Kaplan, S., and B. J. Garrick. 1981. “On the Quantitative Definition of Risk.” Risk Analysis 1 (1): 11–27. doi:10.1111/
j.1539-6924.1981.tb01350.x.
Kloprogge, P., J. P. Van der Sluijs, and A. C. Petersen. 2011. “A Method for the Analysis of Assumptions in ModelBased Environmental Assessments.” Environmental Modelling and Software 26 (3): 289–301.
Kothari, C. R. 2004. Research Methodology: Methods and Techniques. 2nd ed. New Age International Publishers.
Kumamoto, H., and E. Henley. 1996. Probabilistic Risk Assessment and Management for Engineers and Scientists. 2nd
ed. Piscataway, NJ: IEEE.
Leveson, N. 2011. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press.
Leveson, N. 2004. “A New Accident Model for Engineering Safer Systems.” Safety Science 42 (4): 237–270. doi:
http://dx.doi.org/10.1016/S0925-7535(03)00047-X.
€ger, J. H. Lambert, A., et al. 2014. “Changing the
Linkov, I., Bridges, T. F. Creutzig, J. Decker, C. Fox-Lent, W. Kro
Resilience Paradigm.” Nature Climate Change 4 (6): 407–409. doi:10.1038/nclimate2227.
Linkov, I., D. A. Eisenberg, M. E. Bates, D. Chang, M. Convertino, J. H. Allen, S. E. Flynn, and T. P. Seager. 2013.
“Measurable Resilience for Actionable Policy.” Environmental Science and Technology 47 (18): 10108–10110. doi:
10.1021/es403443n.
, eds. 2017. Resilience and Risk. Amsterdam: Springer.
Lundblad, K., J. Speziali, R. Woltjer, and J. Lundberg. 2008. “FRAM as a Risk Assessment Method for Nuclear Fuel
Transportation.” International Conference Working on Safety.
Park, J., T. P. Seager, P. S. C. Rao, M. Convertino, and I. Linkov. 2013. “Integrating Risk and Resilience Approaches to
Catastrophe Management in Engineering Systems.” Risk Analysis 33 (3): 356–367. doi:10.1111/j.15396924.2012.01885.x.
€m, and G. Di Gravio. 2017. “Defining the Functional Resonance Analysis Space: Combining
Patriarca, R., J. Bergstro
Abstraction Hierarchy and FRAM.” Reliability Engineering & System Safety 165 (Supplement C): 34–46. doi:https://
doi.org/10.1016/j.ress.2017.03.032.
Patriarca, R., G. Di Gravio, and F. Costantino. 2017. “A Monte Carlo Evolution of the Functional Resonance Analysis
Method (FRAM) to Assess Performance Variability in Complex Systems.” Safety Science 91: 49–60.
Pereira, A. 2013. Introduction to the use of FRAM on the effectiveness assessment of a radiopharmaceutical dispatches process. In Proceedings of the International Nuclear Atlantic Conference.
Renn, O. 2008. Risk Governance: Coping with Uncertainty in a Complex World. London: Earthscan.
Righi, A. W., T. A. Saurin, and P. Wachs. 2015. “A Systematic Literature Review of Resilience Engineering: Research
Areas and a Research Agenda Proposal.” Reliability Engineering & System Safety 141: 142–152. doi:http://dx.doi.
org/10.1016/j.ress.2015.03.007.
Rosa, L., V. Haddad, and A. Carvalho. 2015. “Assessing Risk in Sustainable Construction Using the Functional
Resonance Analysis Method (FRAM).” Cognition, Technology & Work 17 (4): 559–573.
Saltelli, A., K. Chan, and E. M. Scott. 2009. Sensitivity Analysis. Wiley.
Smith, D., B. Veitch, F. Khan, and R. Taylor. 2017. “Understanding Industrial Safety: Comparing Fault Tree, Bayesian
Network, and FRAM Approaches.” Journal of Loss Prevention in the Process Industries 45: 88–101. doi:https://doi.
org/10.1016/j.jlp.2016.11.016.
2015. “SRA glossary.” Society for Risk Analysis. Accessed 08 October 2015. Available at: http://www.sra.org/sites/
default/files/pdf/SRA-glossary-approved22june2015-x.pdf.
Van Der Sluijs, J. P., M. Craye, S. Funtowicz, P. Kloprogge, J. Ravetz, and J. Risbey. 2005. “Combining Quantitative
and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System.” Risk
Analysis 25 (2): 481–492.
Tian, J., J. Wu, Q. Yang, and T. Zhao. 2016. “FRAMA: A Safety Assessment Approach Based on Functional Resonance
Analysis Method.” Safety Science 85: 41–52.
Woods, D. D. 2015. “Four Concepts for Resilience and the Implications for the Future of Resilience Engineering.”
Reliability Engineering and System Safety 141: 5–9. doi:10.1016/j.ress.2015.03.018.
Wooldridge, M. 2008. “Qualitative Risk Assessment.” In Microbial Risk Analysis of Foods, edited by, 1–28.
Washington, DC: ASM Press. doi:10.1128/9781555815752.ch1
Yang, Q., J. Tian, and T. Zhao. 2017. “Safety Is an Emergent Property: Illustrating Functional Resonance in Air Traffic
Management with Formal Verification.” Safety Science 93: 162–177. doi:10.1016/j.ssci.2016.12.006.
Zheng, Z., J. Tian, and T. Zhao. 2016. “Refining Operation Guidelines with Model-Checking-Aided FRAM to Improve
Manufacturing Processes: A Case Study for Aeroengine Blade Forging.” Cognition, Technology & Work 18 (4):
777–791.