MASTER'S THESIS
Control Gates for Assuring Information
Security during System Development
A Case Study
Fredrik Emanuelsson
Master of Science (120 credits)
Information Security
Luleå University of Technology
Department of Computer Science, Electrical and Space Engineering
Acknowledgement
Thanks to Professor Tero Päivärinta and Heidi Hartikainen for all help during this thesis. Thanks
to all students at the program opposing and helping out with valuable comments.
Many valuable sources have been used during this work. Some of the important is Gurpret
Dhillon for his work on the informal aspects of information security and Åge Garnes for his work
on project management and reducing risk in projects. Dan Harnesk classes in information
security discussing work from Siponen.
Abstract
Has information security its Achilles’ heel in the impact analysis? The hypothetical question
appeared during the case study of the governmental IT department working with integration.
Complexity and lack of support from the business line was causing inadequate risk and impact
assessments. This case study reveals that risk and impact assessment can fail due to complexity,
lack of criteria for prioritization, unclear roles and responsibilities. Information security risk
formulas requires input but in the real case it was impossible to assess the required input due to
the complexity of the governmental IT projects with many vendors and complex portfolios,
unclear roles and responsibilities, lack of support from the business department and complex
organizational dynamics. The contradictory results indicate that calculating business impact was
more difficult than assessing risk/cost or benefits. The difficulties of setting business
prioritizations became visible when the client hired consultants to conduct prioritizing
assessments. Later the consultants answered that they were unable to prioritize. This incident
match what is known in information security research that even if the policies prescribes
existence of information security standards that ensures quality it’s unsure whether or not they
are successful in the real case. At last the IT department solved the problem by using experienced
consultants who made their own priorities. Research shows that support from top-management is
critical for success which is visible in the case by weak collaboration from the business
department in the cost/benefit/risk and impact assessments which affected the possibility to
succeed with the important assessments in the early stage of the project. The case highlights some
difficulties in getting the business department involved in the strategic IT-meetings during
development.
Keywords: control gates information security SDLC quality assurance risk management business
impact cost benefit assessment
1|Page
Contents
Introduction ...................................................................................................................................... 5
1. Problem and objectives ................................................................................................................ 6
2. Research question and delimitation.............................................................................................. 8
3. Theoretical frame ......................................................................................................................... 9
3.1 Control gate in the system development life cycle ................................................................ 9
3.2 Risk/Impact/Cost/Benefit assessments ................................................................................. 13
3.3 NIST SecSDLC .................................................................................................................... 15
3.3 Meaning of social relations in project management ............................................................. 21
4. Methodology .............................................................................................................................. 23
4.1 Research logic ...................................................................................................................... 23
4.1.1 Literature ........................................................................................................................... 23
4.2 Research design .................................................................................................................... 23
4.2.1 Selection of research methodology ................................................................................... 25
4.2.2 Define unit of analysis....................................................................................................... 27
4.2.3 Data collection................................................................................................................... 27
4.2.4 Collected data is interpreted using explanation building, linking data to the question ..... 27
4.2.5 Logical decision models for data processing .................................................................... 28
4.3 Reliability and validity ......................................................................................................... 29
4.3.1 Construct validity .............................................................................................................. 30
4.3.2 Internal validity ................................................................................................................. 30
4.3.3 External validity ................................................................................................................ 31
4.3.5 Review and approval ......................................................................................................... 32
5 Empirical study ........................................................................................................................... 33
5.1 Case site description ............................................................................................................. 33
5.2 Results .................................................................................................................................. 33
5.3 Interpretations of data........................................................................................................... 35
5.3.2 Email conversation Company model for systems deliveries ............................................ 35
2|Page
5.3.3 Concepts in the development process ............................................................................... 37
5.3.4 Delivery planning. ............................................................................................................. 39
5.3.5 Control gates in the SecSDLC .......................................................................................... 39
5.4 Interviews. ............................................................................................................................ 42
5.4.1 New elements that arrive and QA ..................................................................................... 42
5.4.2 Establishment of the product queue meetings ................................................................... 43
5.4.3 Impact evaluation .............................................................................................................. 43
5.4.4 Product elements are roughly estimated and prioritized ................................................... 44
5.4.5 Incident and problem handling .......................................................................................... 44
6. Analysis ...................................................................................................................................... 46
6.1 Data in line with the NIST 800-64, 800-12, 800-55,800-30 ................................................ 46
6.2 Data not in line with the NIST 800-12,55,64 ....................................................................... 47
6.3 Analysing chain of evidence and building explanations ...................................................... 48
6.3.1 Analysing data ................................................................................................................... 48
6.3.2 Holistic view for the single case........................................................................................ 51
6.4 Summary analysis ................................................................................................................ 55
7. Conclusions, discussions and further research ........................................................................... 57
7.1 Short simple answer to the research question ...................................................................... 58
7.2 NIST 800-12 concepts ......................................................................................................... 59
7.3 Control gates NIST 800-64, 800-55 ..................................................................................... 63
7.4 Informal aspects ................................................................................................................... 64
7.5 Management of projects and information security ............................................................... 67
7.6 Recommendations ................................................................................................................ 67
7.7 Further research .................................................................................................................... 68
References ...................................................................................................................................... 70
Appendix ........................................................................................................................................ 74
A. First part is questions about new elements that arrive and QA. ............................................ 74
C. Third part is questions about impact evaluation. ................................................................... 75
D. Forth part is questions analysing if the product elements are roughly estimated and
prioritized. .................................................................................................................................. 75
E. Fifth part is questions related to incident and problem handling. .......................................... 76
3|Page
F. Overview of data that can be related to control gates or context in the lens of the theoretical
framework. ................................................................................................................................. 76
List of figures
Figure 1. Cooper (2001) Stage gate model…………………………………………………...10
Figure 2. Cooper (2001) Stage gate …………………………………………………………….11
Figure 3. Whitman, Mattord (2009) Concept of the system development life cycle…………….11
Figure 4. Guttman, Roback (1995) Computer security plan………………………………..……12
Figure 5. Company documentation screenshot intranet ………………………………………....34
Figure 6. Company documentation screenshot email conversation……………………………...35
Figure 7. Company documentation planning board……………………………………………...35
Figure 8. Company documentation screenshot from vendor…………………………………….36
Figure 9. Company documentation screenshot delivery planning……………………………….37
List of tables
Table 1. Attributes of risk………..……………………………………………………………..14
Table 2. Research methodology …………….…………………………………………………23
Table 3. Research tactics………………..………………………………………………………24
Table 4. Empirical Data Questions about new elements………………………………………..70
Table 5. Empirical Data Questions about product queue meetings………………………….…70
Table 6. Empirical Data Questions about impact evaluations……………………………….…71
Table 7. Empirical Data Questions about prioritizing……………………………………….….71
Table 8. Empirical Data Questions about Problem handling…………………………………...71
Table 9. Empirical Data Questions about Problem handling…………………………………...72
Table 10. Empirical Data Analysis Overview……………………………………………….….72
4|Page
Introduction
Today it’s common that organizations develop custom made software to fit their needs.
Developing the new software solution for the organization often means integration with previous
already existing software including many interfaces to subsystem which can also span external
systems outside the organization. This complex situation makes quality assurances of the system
development process an important issue. Development of the software means adapting to existing
programs and can also include external supplier’s software. The complex situation with many
interfaces towards the subsystems that arises makes the quality assurance become a difficult task
that must be addressed systematically and incrementally.
“In such an environment assuring software quality is a non-trivial task, and thus over the years
numerous techniques have been introduced in an attempt to increase software quality”
(Ambartsoumian, Dhaliwal & Meservy 2011, p.29)
One best practice method for incremental quality assurance is called the stage gate method and
was introduced by Cooper(2001) where the assurance of quality is done after every step in the
process from idea to delivery. This case study describes experiences from implementing quality
assurance steps for IT- projects in the governmental sector.
In the first two chapters the problem and research question is presented. Next in chapter three
fundamental concepts is explained. Control gates, SDLC, Information security and
risk/cost/benefit/impact analysis are explained together with some important concepts from
project management. In Chapter four the methodology of this research is described. Chapter five
describes the empirical part with the results and finally findings are discussed.
5|Page
1. Problem and objectives
Research findings highlights despite that we have information security standards implemented it
does not mean we increase the quality or secure the core business. (Siponen 2006)
It is just so that an information security standard does not provide us with the information how
we proceed with the process in real life. As Siponen(2006) argues in his article with very accurate
title “Information Security Standards Focus on the Existence of Process, Not Its Content The
existence of prescribed security processes in organizations does not mean the goals of the
processes are achieved”. We find similar statement in NIST 800-12 standard. ”Simply issuing
policy, with no follow-up to implement that policy, may not suffice” (Guttman, Roback 1995)
The writer means that even if we have security management standards in place in our working
processes they do not answer how people should carry out their work, they just ensure the
processes exist. “Clearly, it is not important that something is done, but what really matters here
is how well it is done.” (Siponen 2006) He refers to four prestigious information security
management standards:
1. BS ISO/IEC17799: 2000
2. GAISP
3. SSE-CMM (2003)
4. The Standard of Good Practice for Information Security
So we do not know how to implement standards in a real case, this is up to organizations to find
out.
Control gates is introduced as a mean to reduce risk and improve control of projects.(Cooper
2001) Control gates are elaborated in the NIST standard Security consideration 800-64 with clear
milestones to be checked for every phase of the system development life cycle. In the first control
gate the NIST standard includes for example financial reviews, budgetary constraints, business
requirements, risk management and impact level. “Engineering security into a product’s initiation
6|Page
phase typically costs less than acquiring technologies later” (Richard et al. 2008) p.6 The idea of
the control gate is well defined but the NIST standards does not provide with full understanding
how it should be implemented in a real case. There is a need for example cases to give more
understanding about control gate implementation.
7|Page
2. Research question and delimitation
This study was delimited to quality assurance (from now on called QA) of cost/benefit/impact
and risk assessments in the SecSDLC. This delimitation was interpreted to checkpoints (control
gates) of quality assurance in other words checkpoints in the process. From the NIST800-64
standard the first and second phase control gates has been used because the study focused on the
early phase in the system development life cycle. The term checkpoint is equal to QA in this
study. (control gate is also used for the word checkpoint) The research question is ‘How can
control gates be implemented and what are the experiences from when you try to use control
gates for cost/benefit/risk and impact analysis?’ This case study aimed to give an example how
the real situation looked like within an IT-department in the governmental sector. Consequences
of this research question meant to understand the business perspective in the control gates to be
able to fulfill the steps for the SecSDLC, because we cannot prioritize if we don’t assess the core
business needs. We must investigate what is worth protecting in the organization. We must also
understand how to measure and control assets like intellectual property. One part of the control
gates is to understand cost/benefit of the gate to be able to make a go/no go decision. Therefore
the identified risks must be are clearly understood before the system development advances to the
next life cycle phase. Because if the risk isn’t clearly understood it not possible to calculate cost
of countermeasures and benefit of not implementing them. This study will not describe technical
measures like antivirus and firewalls but focus on the process and describe the content of the
control gate assuring quality.
8|Page
3. Theoretical frame
In relation to the research question the main concepts were the control gate and
risk/impact/cost/benefit assessments. The main concepts were defined and then elaborated
towards NIST800-64, 800-12, 800-55, 800-30, 800-37 standard documents which were further
elaborated in relation to theories based on concepts from Meier et al. (2011) , Dhillon(2006,2007)
and Garnes(2004,2009).
Meier(2011) has a relation to de-facto software industry, and Garnes(2004,2009) is expert in
project management with focus on risk. Dhillon(2006,2007) adds perspective to (Meier et al.
2011) by discussing the informal and social aspects of information security.
3.1 Control gate in the system development life cycle
In Coopers stage gate model the system development process is controlled at the gates.
Coopers model uses the gates to make go/no go decisions during the development process.
By using the concept it’s possible to assure quality in big projects and gives the management a
tool for control of the entire process. In NIST800-64 information security standard the stage gate
is named control gate. So stage gate is equal to control gate.
9|Page
Figure 1. Cooper (2001) Stage gate model. Quality is assured at the control gates.
Figure 2. Cooper(2001) Stage gate.
The gates reduce the risk for bad quality and increase chances for success. Problems in the
project can be corrected in an early phase of the project which saves cost. A gate can result in that
a project is killed or continue by reasons presented in the control gate which is the main idea with
the control gates. Not all projects should be implemented. Coopers concept is flexible and can be
adapted to the real case situation. Whitman,Mattord(2009) highlights the difference between
Security Systems Development Life Cycle (SecSDLC) and the Systems Development Life Cycle
(SDLC). In this comparative matrix the author use six phases spanning from Investigation to
Maintenance. Comparing the SDLC and the SecSDLC, (Whitman, Mattord 2009 ,p.25)
10 | P a g e
Figure 3. Concept of the system development life cycle. (Whitman, Mattord 2009, p.25)
11 | P a g e
Figure 4. In the picture Computer Security plan is to be integrated during the system life cycle
(Guttman, Roback 1995, p.78)
To perform the practical implementation of the control gates in big organizations can fast become
a difficult task due to the complexity. This means there is a need to have a simple tool to be able
to produce efficient information that can be the base for control to the management. In other
words the management needs information to control projects. Risk/cost/benefit and impact
analysis are examples of control information which is necessary to make go/no go decisions for
the project.
” Projects don’t arrive at their conclusion perfectly executed and delivering all the benefits
promised in the Business Case, at the advertised cost. They must be measured along the way to
ensure they are developing to plan” (Nielsen 2008) This is also a well-known fact in information
security which can be related to NIST 800-55 standard “what gets measured gets done, when a
Board of Directors requires the CEO to report regularly the values of specified metrics, it
communicates to the CEO what the directors consider important” (Legrant et al. 10 jan 2005, p.5)
Information security NIST 800-64 standard defines a control gate. “Identifies general control
gates, or established points in the life cycle, when the system will be evaluated and when
12 | P a g e
management will determine whether the project should continue as is, change direction, or be
discontinued. Control gates should be flexible and tailored to the specific organization. Control
gates are valuable in that they provide the organization with the opportunity to verify that security
considerations are being addressed, adequate security “ (Richard et al. 2008)
3.2 Risk/Impact/Cost/Benefit assessments
Risk management is according to Whitman,Mattord(2009) the process of identifying and
controlling the risks facing an organization which includes assessments and calculations of
risks/impacts/costs/benefits. The purpose of risk management and information security is to
reduce the competitive disadvantages and optimize the competitive advantages over other
actors/companies on the market. “Risk management is the process of identifying vulnerabilities in
an organization’s information systems and taking carefully reasoned steps to ensure the
confidentiality, integrity, and availability of all components in the organization’s information
system.” (Whitman,Mattord 2009, p.112)
Gallagher(2011) NIST800-30 elaborate risk as a measure of what extent a resource is threatened
by a potential event and the impact in case that event occurs. Risks involve loss of
confidentiality, integrity or availability of information or information systems and reflect the
impact on the organization/company. “A risk assessment is the process of identifying,
prioritizing, and estimating information security risks. Assessing information security risk
requires the careful analysis of threat and vulnerability information to determine the extent to
which circumstances or events could adversely impact an organization and the likelihood that
such circumstances or events will occur….risk is the combination of the likelihood of a threat
event’s occurrence and its potential adverse impact” (Gallagher 2011,NIST800-30 ,p.6,10)
Assessments methods can be qualitative or quantitative. Where quantitative refers to using actual
number values on cost and losses, and qualitative refers to risk scales spanning from low to
moderate to high risk values instead of numerical values. Third option is a semi-quantitative scale
using 1-100 for assessing the qualitative risk. Gallagher(2011) NIST800-30 describes the process
to start with Identification of threats, sources and events that can be harmful. Next step is to
13 | P a g e
identify vulnerabilities and determine likelihood of occurrence. Finally the process is to
determine the impact and conclude the risk.
The risk assessment process can produce a sum in numerical value of the risk using the following
formula:
Risk = (likelihood of occurrence of a vulnerability) * (value of the asset) - (current controls) +
uncertainty
(Whitman,Mattord 2009, p.132)
Risk modeling using Bayesian index is discussed by Chan(2011) as a tool to deal with the
complexity. Likelihood ratios go from low to high and the risk attribute is ranked in relation to
other risk attributes.
Attributes of is risk
Analysis ability of ‘impact
risk management for
continuing business’
Without analysis ability of
‘impact risk management for
continuing business’
Support and coordination
from top management
Without support and
coordination from top
management
Likelihood ratios
(+) 2.419 to 1
Rank
24
(−) 1 to 3.220
14
(+) 7.299 to 1
1
(−) 1 to 11.959
1
Table 1 Attributes of risk (Chan 2011, p.633)
Similar tables are of help to categorize and determine proper ratios and rank when calculating
risk. In the process of using the method the attributes can be selected and during assessments only
the answer yes or no is needed to select if the attribute is presented or not. The ratios are
predefined by a board of experts.
As Dhillon(2007) discuss the following options can be considered after risk has been calculated.

Do nothing. If the risk is considered acceptable.

Risk avoidance. If the risk is recognized but ways to avoid it can be done.
14 | P a g e

Risk prevention. If the risk can be prevented.

Risk planning. If the risk can be handled by planning and prioritization.

Risk recognition. If the risk is known and research is done to be able to handle it.

Risk insurance. Transfer the risk to someone else by insurances.
Risk need to be addressed in an early stage of the project. “When security requirements are
considered as an integral subset of other information system requirements, the resulting system
has fewer weaknesses and deficiencies, and therefore, fewer vulnerabilities that can be exploited
in the future.” NIST 800-37 (Gallagher2010, p.9)
3.3 NIST SecSDLC
According to Richard(2008) the System development life cycle in the standard has five phases.
1. Initiation
2. Development/Acquisition
3. Implementation/ Assessment
4. Operation/Maintenance
5. Disposal
During the first phase early integration, threats and requirements are considered. Security is
considered from a business perspective. This means that the business requirements and
cost/benefit/impact analysis need to be ensured. In the second phase a detailed risk assessment is
required and also functional, testing requirements. Risk, impact and cost/benefit analysis are part
of the first and second phase to be able to make go/no go decisions. “Security discussions should
be performed as part of (not separately from) the development project to ensure solid
understandings among project personnel of business decisions and their risk implications to the
overall development project “ (Richard et al. 2008, p. 13)
The idea and purpose with the NIST 800-64 standard document discussed by Richard(2008) is to
integrate security into the system development life cycle. This process starts with the business
view and moves towards implementation of the application. This study will stay within the first
15 | P a g e
two phases of the SDLC described in the NIST 800-64 and therefore include these concepts in
the theoretical framework.
As Richard(2008) say the idea with the initial phases is to identify control gates to be able to
determine if the system will change direction or be discontinued. “Control gates should be
flexible and tailored to the specific organization. Control gates are valuable in that they provide
the organization with the opportunity to verify that security considerations are being addressed,
adequate security is being built in, and identified risks are clearly understood before the system
development advances to the next life cycle phase. ” (Richard et al. 2008, p. 11)
This statement from the author is more or less identical which was displayed by Cooper(2001)
which indicates that the control gates in NIST standard is adopted from Coopers stage gate
concept. Richard(2008) explains that the NIST800-64 considers interdependencies between tasks
so the information security activities does not impact negatively and affect the process contra
productive.
NIST Interdependencies in the early phases
According to Richard(2008) the first step regarding mapping of interdependencies is to create a
schedule of all coming information security activities to be able to plan and look at resources
related to the activities. The next step is to approach a holistic perspective on all activities in
relation to the business impact analysis. The enterprise architecture needs to be considered and
evaluated together with the plans for how the system will share information with other systems.
The impact analysis and risk analysis is highlighted as key assessments in the early phases. “The
BIA is a key step in the contingency planning process….The risk assessment is a primary tool to
identify if the tailored security controls are effective to address an organization’s risk tolerance.
” (Richard et al. 2008, p. 17, 24)
NIST 800-64 Control gate Phase one
Richard(2008) discusses that in the first phase the business requirements needs to be assured to
be able to determine the acquisition strategy. There need to be a rough concept to be able to
verify that the system can be completed within budget and that the concept is in line with the
organization goals. There is a need for an early Performance specification. The first phase
16 | P a g e
requires an enterprise architecture which is harmonized with the organizations IT-architecture,
and general business requirements. A financial review is required which includes cost, risk and
impact assessments of the system.
NIST 800-64 Control gate Phase two
Richard(2008) says that in the second phase there need to be an design review that includes
integration with other systems. In the second phase there is a need for a system functional review
which includes test cases. Financial reviews need to be enough detailed so it’s possible to
monitor cost/benefit. The second phase include follow up on the risk management. Phase two
together with phase one covers the early stage of the project with cost/benefit, risk and impact
analysis with focus on business requirements.
Roles and responsabilities
Information security roles and responsibilities needs to be clear in an early stage of the project. It
is important that employees knows who is responsible for what. The standard suggests that key
roles are identified in the first phase. ” With any development project, it is important to involve
appropriate information security personnel as early as possible, preferably in the initiation
phase.” (Richard et al. 2008, p.8)
In the NIST 800-55 standard we see a statement that relates information security measurement
with the importance of roles and responsibilities as a critical success factors. “Clearly assign
information security responsibilities, and lay the foundation needed to reliably measure progress
and compliance” (Chew, Swanson & Stine 2008, p.3)
NIST800-12 definition of information security
In this study the concepts presented in the NIST 800-12 standard handbook by Guttman(1995)
represents the main concepts together with NIST800-64 and defines information security with
control gates. The NIST800-12 is composed in 1995 and uses the term ‘computer security’ which
could be discussed as old fashion today when newer research uses information security as
concept. The motivation to use these ‘older’ concepts is that the basic concepts are well rooted in
the NIST framework where the NIST800-12 present the core terms in a simple way and display a
good overview of what is considered information security also today. Guttman(1995) defines
17 | P a g e
computer security as the following:
NIST 800-12 discusses three perspectives involving management controls, operational and
technical controls. Where the management controls address security topics which can be
considered managerial as for example risk management. The operational control which are
executed by people as opposed to system but can include technical and management activities.
The technical perspective has focus on the security controls in regard to the computer system.
Guttman(1995) includes software and hardware as assets but not people in the definition of
computer security. ”The protection afforded to an automated information system in order to
attain the applicable objectives of preserving the integrity, availability and confidentiality of
information system resources (includes hardware, software, firmware, information/data, and
telecommunications).” (Guttman 1995, p. 17)
In this sense information/data can be stored in many ways for example in people minds which
therefore also includes people to preserve the security.
Guttman(1995) uses the concepts confidentiality, availability, integrity which means both data
integrity and system integrity. Integrity required that the data is timely, accurate, complete, and
consistent. Data integrity means that data is changed in a controlled manner. System integrity
means that a function is processed and “free from deliberate or inadvertent unauthorized
manipulation of the system”. (Guttman 1995, p.6)
Availability means that the service works fine and is available for the users. Confidentiality
means that the unauthorized users cannot see the confidential data.
De facto definition of information security
In this study the concepts and theory discussed by Meier(2011) represents the de facto software
industry definition of information security as the following:

Authentication

Authorization
18 | P a g e

Auditing

Confidentiality

Integrity

Availability
Authentication refers to the process when a user identifies himself during login. Authorization
refers to the process when the user’s rights to use system resources are verified. Auditing refers to
logging procedures where the users cannot deny for example an online buy. Confidentiality
refers to keeping the information private so it cannot be seen by someone unauthorized. Integrity
refers to keeping the information unchanged intentionally or unintentionally. Availability refers
to the degree the system is available for example 99.99% of time during a year. Meier argues that
information security issues can be explained by the relation and context the four concepts is
related (’asset’,’threat’,’vulnerability’,’attack’.)
” To summarize, a threat is a potential event that can adversely affect an asset, whereas a
Successful attack exploits vulnerabilities in your system.” (Meier et al. 2011)
A asset is something worth protecting, for example a credit card number or intellectual capital. A
threat can potentially destroy, hurt or steal an asset. Vulnerability refers to a weakness that makes
a threat possible, which can be a bad design or a mistake in the configuration. An attack means an
action that uses the weakness. This can be sending malicious code in some field with the
intention to destroy or steal.
Dhillon (2006; 2007) uses the same definition as Meier(2011) as base but then extends the
definition by introducing the concepts of responsibility, integrity, trust and ethicality. Where he
discusses actors that are driven by responsibilities and with integrity to avoid unauthorized to get
their hand on the company secrets. This can be done by the external actors using social
engineering where it’s about manipulating people to get the protected information. Big projects
can suffer from problems with keeping the intellectual capital safe when having a high degree of
external consultants moving in and out the company. It is not enough to keep control with a
written contract, it requires on site active management. Without proper management and
information security it’s possible to find ways to subvert controls that is only in the form of a
19 | P a g e
policy. Dhillon(2006; 2007) describes trust as an important part where actors must trust on each
other to make the collaboration to work smoothly. It is not possible to control everything and it
would increase cost which is contra productive. The writer uses ethicality as a concept where the
actor carries an inner compass which directs decisions and is a vital component in information
security. These definitions are the core of the informal security which cannot be implemented by
technical measures like antivirus, and firewall protection. “The major causes of loss due to an
organization's own employees are: errors and omissions, fraud, and actions by disgruntled
employees” (Guttman, Roback 1995)
Dhillon(2006; 2007) add the informal zone in information security where human relations and
social aspects is an component. The relational component has an economic value because we
choose to do business with people we know has a good history, or that we know somehow. It is
for example more difficult to cut out a relative from a deal because it has consequences thereby
the social relation has a value in business transactions. (Garnes 2009, Garnes 2004)
Dhillon(2006; 2007) discuss the informal aspects of information security and gives perspective
on efficient strategies for management. The writer describes the security situation where the
informal aspects are important because an informal aspect means company culture and
collaboration among people and group of peoples.
“The major causes of loss due to an organization's own employees are: errors and omissions,
fraud, and actions by disgruntled employees. One principal purpose of security awareness,
training, and education is to reduce errors and omissions. However, it can also reduce fraud and
unauthorized activity by disgruntled employees by increasing employees' knowledge of their
accountability and the penalties associated with such actions. Management sets the example for
behavior within an organization. If employees know that management does not care about
security, no training class teaching the importance of security and imparting valuable skills can
be truly effective. This "tone from the top" has myriad effects an organization's security
program.” (Guttman, Roback 1995)
If an employee leaves the company the intellectual property leaves with him. Surely we do not
want to have disgruntled employees because it might lead to the asset in the form of intellectual
capital leaves the company.
20 | P a g e
3.3 Meaning of social relations in project management
An important basis for business theory is the economic theories which Garnes(2004,2009)
discuss and highlights that human relations and trust has economic value in economic theories.
Social relations are essential in economic theories. Dhillon(2006,2007) links business theories to
information security and says that culture and communication is part of information security.
“Good business communication is essential to ensure integrity of operations, which in turn helps
in ensuring security” Dhillon(2006,2007, p.195)
Garnes(2004,2009) discuss the meaning of human relations in projects and displays two
statements”
1. Mennesker handlar med og mot sine omgivelser og samhandlar med og mot hverandre
2. Mennesker inngår i relasjoner med hverandre” Garnes(2004,2009, p.58)
These statements mean that humans interact with and against their environment and collaborate
with each other. Humans make relations with each other. Human relations are an important
parameter to be considered and therefore also affect aspects of information security. As
Dhillon(2006,2007) argues that the social aspect might be the best way to enforce control.
Garnes(2004,2009) states that human relations is an important part of the social aspects and
therefore we could learn from both writers how relations and trust can matter. This knowledge
can be used as an alternative theoretical explanation to understand information security incidents.
In other words an incident can have started due to misused trust or other relational aspects which
leads to security breaches.
“Familien og vennene våre inkluderas når vi vurderar nytteverdien” Garnes(2009, p.35) Which
means that the family and friends is included when we value the benefits. This statement from
Garnes(2009) highlights that the people value relations as one part of decisions they make.
Support from the upper management is crucial to succeed with projects
Garnes(2004,2009) discuss the importance of support from upper management in an early stage
to be able to succeed with projects and reduce risk in an early stage. This statement can also be
21 | P a g e
found in the NIST 800-55. “The foundation of strong upper-level management support is critical,
not only for the success of the information security program, but also for the program’s
implementation.” (Chew, Swanson & Stine 2008, p.3)
Chan(2011) estimates lack of support from upper-management as a high risk factor regarding the
success of the project.
22 | P a g e
4. Methodology
4.1 Research logic
During the study a deductive approach was chosen to construct and evaluate arguments.
Deductive means that the research constructed explanations and conclusions from a general
principle. The principle derived from the theoretical framework and data that did not match the
theoretical framework was identified.
4.1.1 Literature
Search engine tools were used for literature reviews: google scholar, scopus, proquest, web of
knowledge, Luleå University Library search engines for books, magazines and libris search
engine.
As one example a search with scholar.google.se on keywords: control,gates
,information,security,sdlc,case,study, governmental gave 168 results.
During investigation of the search results, it was found that control gates and
risk/impact/cost/benefit analysis were well explained in the literature but no case study discussing
control gates in the governmental sector similar to my research question was found. This gap in
knowledge was handled by studying control gates with context as well as possible at a real case
site and using earlier known research from information security, project management and control
gates ,risk/cost/benefit and impact analysis.
4.2 Research design
Yin(2009) suggest important components for covering the research design, which in this study
was interpreted to being the following:
- A Selection of research methodology
- B Define unit of analysis
- C Data collection
- D Interpretation of the data.
23 | P a g e
- E Logical decisions models for linking data to the research question.
As a help to choose research methodology Yin (2009) enlist the following matrices.
Method
Form of research
question
Requires control of
behavioural events?
How,why ?
Yes
Who, what, where,
No
how many, how
much?
Who what, where,
No
Archival analysis
how many, how
much?
How,why?
No
History
How,why?
No
Case study
Table 2. Yin (2009) p.8 when to use each method.
Experiment
Survey
Focuses on
contemporary events
?
Yes
Yes
Yes/no
No
yes
According this matrix the case study fitted this study because it required not control of
behavioural events and it focused on contemporary events. “ ’how’ and ‘why’ questions are more
explanatory… because such questions deal with operational links needing to be traced over time”
(Yin 2009, p.9)
Yin (2009) differs between explanatory(find explanations), descriptive(to describe), and
exploratory(to explore), and also single case or multiple case.
Type of
Explanatory
Descriptive
Exploratory
compositional
structure
Linear analytic
X
x
x
Comparative
x
x
x
Chronological
x
x
x
Theory-building
x
x
Suspense
x
Unsequenced
x
Table 3. Six structures and their application to different purposes of case studies. (Yin 2009)
p.176
According to Yin(2009) the un-sequenced structure fits to the descriptive study, where the
explanatory involves theory-building and suspense structure. “outcome is .. presented in the
24 | P a g e
initial chapter or section…descriptive case study has no especially important outcome” (Yin
2009, p.178)
Päivärinta(2011) argues for the use of a systematic framework to perform organizational learning.
Ideas from this framework were used to support this study by comparing the context that existed
during the development life cycle. Which methods were in use at the case study site. Were they
the same during the process, or did they differ? What were the reasons for the differences in
development context and what were the consequences of these differences? An example of
context can be that vendors are working using an agile scrum methodology but the customer
organization is using another methodology for system development. What was the consequence
of the difference? Päivärinta(2011) discuss the concept of rationale which this study used by
searching for an answer to what problem the organization solved with the gate meetings. What
was the functional reason that the organization choose to implement the gate meetings?
4.2.1 Selection of research methodology
During this study the descriptive/explanatory single case study approach was chosen. The
motivation to do so comes from that the case fitted well into the description that it was a
contemporary set of events which I have little or no control over. (Yin 2009) The motivation to
choose a descriptive/explanatory single case study approach comes from an understanding of
what was possible at the case site, where there was an option to have an approach to describe and
explain events at the single case site, and the need for descriptions of case examples regarding
control gates, with a case study that did not result in any special outcome, more than a description
of the current state, but with some options to build explanations to the outcome. It would not been
a good choice to choose multiple case sites because then the data collection would had suffered
and the explanations been more on the surface. The context was complex within a big
organization where it was possible for me to describe what I could see through observations and
read in emails and documentation and to report this in a scientific matter related to the master
course. Main focus was descriptive but conclusions rely on explanation building as method for
results. To have the angle with a more explanatory focus would make the study harder to perform
in this complex organization so I found the best choice to be a single case descriptive study with
explanation building to draw conclusions.
25 | P a g e
I’m newly employed at this site so I had a good opportunity to make observations on site and
collect data from company documentation. But I did not have very much control or power to
influence the organization or processes which motivates the decision not to choose an action
study. Still I was interested in understanding the case so surveys or an experiment wouldn’t give
the answers I’m looking for because people working at the work site would not have accepted
spending time on filling in long surveys or participating in experiments. Also it wouldn’t be
practical possible to set up a scientific experiment in this site to get answers because it would stop
production and interfere with the business.
Interviews
Interviews in this thesis has been based on six transcripts and six interviewed people which have
been summarized in a scorecard that has been discussed in group form with the key informants to
improve reliability. Interviewees have got the questions in good time before the meeting so they
had time to understand the questions and to think them through. Scorecards are a good way to
collect information like strategies and the visibility of the colour rating makes it easy to
understand and communicate to the team which gives a triangulating effect. Kaplan,
Norton(2002) highlights benefits with the balanced scorecard as an efficient tool to gather
informal data like strategies and to communicate them within the organization. At the site was a
good opportunity to gather key informants at one place at the same time and get valuable data.
“key informants are often critical to the success of a case study” (Yin 2009, p. 107) Members of
the group can interact and decide on the grade for the question. All questions were directly
related to the research question because they cover roles, responsibilities and risk/cost, benefit
issues within the product queue meetings and therefore also well-grounded in information
security theory and standards. Interviews have been done with a defined set of questions which
has been constructed in dialogue with the organization to get valuable information from the
product queue meetings. Execution of the interviews has been done in group form with key
informants from the developing team. Discussing the questions in group form gave a
triangulating effect, by giving key members opportunity to reflect and provoke discussions
around the questions, which gave me valuable feedback and increased reliability by the group
26 | P a g e
session. The group was able to review the information presented and reach consensus on the
group answers on the questions.
4.2.2 Define unit of analysis
The unit of analyse was the organization. Empirical data has been collected at the organization
integration centre, which was responsible for horizontal processes across the organization that
could span IT-projects. Building focus on the control gates was motivated in earlier studies. One
example was the stage gate model discussed by Cooper (2001) which defined stage gates which
were related to what was referred as control gates in this study.
4.2.3 Data collection
Data has been collected from multiple sources on the research single case site. Information was
collected using interviews, on site observations, and studying documentation. The collection
strategy was to find and identify the SSCG’s (Information Security System development life
cycle control gates) and to collect enough information from the unit of analysis to be able to
study them in this case. Informants were mapped to be able sort out key informants and decide
which informants were part of the SSCG’s. One way to start was to study the organizations
documentation on the development process and use this information when doing interviews to
verify which parts of the documented process were used and which parts were not used. And
then use the documentation to drill more into detailed questions about what were used and what
were unused related to documentation.
4.2.4 Collected data is interpreted using explanation building,
linking data to the question
During data collection the theory was used as a template to compare the empirical result of the
case study. (Yin 2009) Explanations were built as part of the analytic generalization.
27 | P a g e
Conclusions from the study were based on the built explanations from theory in relation to
collected data. Therefore data was linked to the research question using logical decision models
to interpret ate the empirical information and build explanations that were triangulated in relation
to the theory. Collected data were related to the NIST 800-64, 800-55, 800-12, 800-30 standards
to find correlation between the case and the standard. Relations in the data that matched, or did
not match were discussed and triangulated towards the theory and key informants at the case site.
4.2.5 Logical decision models for data processing
To be able to handle immediate decisions and interpretations during data collection the study
included 3 logical decision models for processing the collected data. The model ensured that the
collected data were linked to the research question at collection time, and therefore interference
due to time between collection and documentation was minimized. Interpretation was done at
collection time. ”You must be able to interpret ate the information as it is being collected” (Yin
2009, p.71)
Logical decision model I, Categorizing key informants
1. Is the informant part of SSCG? If not then find informant that is part of SSCG.
2. Is informant 100% clear over the process? If not document what informant is unclear over in
relation to company documentation? If yes, what is the content of the steps in the process in
relation to company documentation?
Logical decision model II, processing company documentation
1. Is the documentation possible to relate SSCG? If not, find documentation that is possible to
relate.
2. Does the documentation answer how the SSCG is done in a real case? If not, use material to
discuss with key informants.
Logical decision model III, processing observed data.
1. Is the observation possible to relate to a SSCG? If not, discard observation.
28 | P a g e
2. Is the actors in the observation part of a SSCG? If yes, prioritize the most important and use
model I.
3. Is there an action within the observation that can describe some content of the SSCG, if yes,
and then document the action.
4.3 Reliability and validity
Yin(2009) display the following matrix to test the quality of research design.
Tests
Case study tactic
Use multiple sources of
evidence.
Establish chain of evidence
Have key informants review
draft case study report
Do pattern matching
Internal validity
Do explanation building
Address rival explanations
Use logic models
Use theory in single case
External validity
studies
Use replication logic in
multiple-case studies
Use case study protocol
Reliability
Develop case study database
Table 3. Case study tactics for four design tests. (Yin 2009, p.41)
Construct validity
Phase of research in which
tactic occurs
Data collection
Data collection
Composition
Data analysis
Data analysis
Data analysis
Data analysis
Research design
Data collection
Data collection
The same matrix was used to verify how well the task was accomplished in this study.
Tests
Construct validity
Internal validity
Case study tactic
Use multiple sources of
evidence.
Establish chain of evidence
Have key informants review
draft case study report
Do pattern matching
Do explanation building
Address rival explanations
Use logic models
task accomplishment
YES
YES
YES, verified by CISO
YES
YES
YES
YES
29 | P a g e
Use theory in single case
studies
Use replication logic in
multiple-case studies
Use case study protocol
Reliability
Develop case study database
Table 4. Task accomplishment of validity and reliability
External validity
YES
YES
YES
4.3.1 Construct validity
Yin(2009) uses the following tests to verify construct validity:

Multiple sources of evidence.

Establish chain of evidence

Have key informants review draft case study report
In this study interviews, company documentation and direct observations were used as data
sources. A chain of evidence was maintained during the study using the research question and
linking the collected data both to the question and to this report. During the study key informants
have reviewed the interview material in the scorecard and discussed the questions and answers.
The Chief information officer has reviewed the draft of the empirical section. Yin(2009) discuss
the weakness of interviews. “reflexity-interviewee gives what interviewer wants to hear”
Yin(2009, p.102)
To ensure construct validity (Yin 2009) highlights the importance of having multiple sources of
evidence. In this study the data collection has covered many different sources of evidence using
interview of actors in the meetings, company documentation, on site observations. The study has
built chains of evidence and let key informants review the case study reports.
4.3.2 Internal validity
Yin(2009) uses the following tests to verify internal validity:
30 | P a g e

Do pattern matching

Do explanation building

Address rival explanations

Use logic models
During the study analysis data was matched according to pattern of answer and an explanation
was built on the pattern. Also a rival explanation was considered when the explanation was built.
During data collection a logical model was used to ensure collected data quality.
During data collection the study has elaborated explanations and compared with the template
theory in parallel with elaborating the rival explanations. Logical models have supported the
thoughts to ensure the chain of evidence to minimize the effect of interference. Yin(2009)
suggests an iterative manner to ensure the evidence which has been done during this study with
frequent revisions during constructions of explanations and rebuilding the chains of logic.
4.3.3 External validity
Yin(2009) uses the following tests to verify external validity:

Use theory in single case studies
Theory was used in this study.
Yin(2009) uses analytic generalization in this context and explains how a case study can
strengthen the external validity for a theory. This case study has contributed to strengthen used
theories. This case study is yet another example of the real world case of the theory.
4.3.4 Reliability
Yin(2009) uses the following tests to verify construct validity:

Use case study protocol

Develop case study database
31 | P a g e
In this study a case study protocol has been used and there has been a case study database
developed. This case study has kept records of collected data and the steps that lead to these
conclusions so it’s possible to trace and redo the conclusions at any time. The logical model has
helped understand how the decisions were made and using which data. The results has been
developed in an iterative/ triangulating manner so it was possible to trace the steps taken to the
final conclusion and to see that there is multiple sources to the result.
Weakness in this study
During interviews many factors can affect the result. Yin(2009) mention situations where the
interviewee gives what interviewer want to hear. This is a probable cause to subjective answers
where the interviewee answers according to what fits the situation best in regard to the employer,
or social factors. The final part of the interview was done in group form to conclude the scorecard
which was a situation where one must also consider the peer pressure.
As Yin(2009) states there can also occur weaknesses due to selectivity. In this study the data is
selected and reported which introduce many possible subjective selections.
Yin(2009) discusses weaknesses in relation to source of evidence where the author states that
reporting biases can occur. In the learning context of this master thesis there are clear reporting
requirements which help the students succeed within time. All information is filtered during the
master thesis seminars to get the preferred result when the course ends. The filters are the
researcher himself and the opponents, supervisors with both objective and subjective thoughts
about how research reports should look like and be done which influence all research in the
context therefore the results will be a result of the learning process participants including bias.
4.3.5 Review and approval
Information summarized in the empirical study (section 5) has been reviewed and approved by
the chief information security officer at the organization. Reviews and approvals of data with key
informants can increase reliability because the conclusions have been verified and can be redone
to the same result. This is also discussed by Yin(2009).
32 | P a g e
5 Empirical study
5.1 Case site description
The case study was performed within an IT-team in the governmental sector. The team consisted
of 7 people working internally and 15 people at the vendor. The department was new and was
being established and there were still many roles to be filled during the time of this study.
Responsibilities covered among other tasks systems integration and Service Oriented
Architecture and I’m part of this team as an employee. In this study the data was collected in
many forms like for example information from the company intranet and email dialogues.
Observations have been done on site and during internal courses, information sharing activities.
The team at integration centre has been interviewed by doing a scorecard rating of the product
queue meetings. According to the methodology (4.25) interpretation was done at data collection
time to reduce interference over time. Yin(2009) discusses the importance of collecting data and
linking interpretations at collection time and compares it to a scene where the evidence is ‘fresh’
and needs to be handled fast to reduce interference over time. Collection time interpretations
were marked with cursive text continuously in relation to the collected data. ”You must be able to
interpret the information as it is being collected” (Yin 2009, p.71)
5.2 Results
Results have been summarized and the collected data has been interpreted at collection time.
Result summary
1. Risk is not understood.
2. Business line is not part of the prioritization at the product queue meetings.
3. Vendor and customer have difficulties to perform prioritizations.
4. A cost/benefit analysis is not done as basis for the prioritizing.
5. Lack of criteria to make prioritizations.
6. Lack of roles and routines for the product queue meeting.
7. Detailed documentation of roles and responsibilities exists in the SecSDLC.
33 | P a g e
8. Mix methods are used. Agile, ITIL, and local adaptions.
9. Prioritizations of elements are solved with help from experienced individuals.
10. Vendor and customer are sharing virtual and physical working space, but the internal
business department is not actively sharing that information space.
11. Impact analysis is not done at all.
12. Incident and problem handling is working well and is prioritized by the business line.
Result one ‘risk is not understood’ came visible during the interviews of the team at the
integration centre where the team rated risk as not fully understood. Result two ‘Business line is
not part of the prioritization at the product queue meetings.’ came visible during the interviews
of the team at the integration centre where the team commented that the business department was
not part of the prioritizations at the product queue meetings. Result three ‘Vendor and customer
have difficulties to perform prioritizations’ is a result from email conversations between the
vendor and the customer where the vendor answers the customer that there is difficulties in this
direction. Result four ‘A cost/benefit analysis is not done as basis for the prioritizing.’ is a result
from the interview section where the team rated this as not done. Result five ‘Lack of criteria to
make prioritizations’ is taken from the company documentation which shows the criteria as nonexistent. Result six ‘Lack of roles and routines for the product queue meeting’ is taken from the
interviews where the team comments that there is a lack of roles and routines in this area. Result
seven ‘Detailed documentation of roles and responsibilities exists in the SecSDLC’ is a result
observed when comparing the NIST standard content with the content in the company
documentation. Result eight ‘Mix methods are used. Agile, ITIL, and local adaptions’ is
observed in the company documentation displaying mixed methods. Result nine ‘Prioritizations
of elements are solved with help from experienced individuals.’ Was a direct observation at
product queue meetings. Result ten ‘Vendor and customer are sharing virtual and physical
working space, but the internal business department is not actively sharing that information
space’ is a result from direct observation. Result eleven ‘Impact analysis is not done at all.’ Is a
result taken from the comments made during interviews by the team at integration centre. Result
twelve ‘Incident and problem handling is working well and is prioritized by the business line.’ is
a result from the interviews where the team answered that its working well on all perspectives.
34 | P a g e
5.3 Interpretations of data.
As Yin(2009) suggest interpretations of data at collection time. First the company documentation
is displayed and an interpretation was made during collection time. Next an email conversation is
highlighted to be followed by important concepts in the development process like delivery
planning and SDLC documentation from the department to be complemented with observations
on site.
5.3.1 Company documentation
When the administration discovered or got new requirements from the business department it was
handled as a change request which was processed in the weekly change council. Interpretation
at collection time: The weekly change council board was equal to the gate meetings described by
cooper(2001) because they had the option to make go/no go decisions for change requests. This
control gate meeting had both permanent participants and participants that were not permanent.
Among the roles participating were process leader change steering, process leader incident
steering, process leader problem steering, process coordinator, section manager, people from the
operation centre, service manager, system responsible, security manager, operation manager, area
managers. The policy for the gate meeting stated that the change coordinator or the problem
coordinator presents and explains the changes to be done for the council. These roles were filled
with people from the development teams, for example if the integration centre had suggested a
change/problem it could be the project manager and/or some individual from the integration
centre that presented these problems or changes which needs implementation in the coming
releases. Interpretation at collection time: If we assume it was the project manager from the
integration centre that was presenting the information for the board he could only have collected
the information about these issues from the product queue meetings at the integration centre,
because it is the product queue meetings which shares the core information about the
prioritization of the elements to be implemented. One part of the policy says that at the gate
meeting it’s required to present the cost/benefit, and risk/impact analysis. Interpretation at
collection time: Cost/benefit and risk/impact analysis is clearly defined in NIST800-64 standard
discussed by (Richard et al. 2008) as part of control gates in phase one , phase 2 which
strengthen the argument that this can be considered a gate.
5.3.2 Email conversation Company model for systems deliveries
35 | P a g e
The internal system delivery methodology has defined a pre meeting before gate meetings in the
process. The lowest level of gate meetings in the internal process was what’s internally called
‘product queue’ meetings.
Figure 5. Screenshot from intranet describing the existence of weekly gate meetings
Interpretation at collection time: In the screen shot is evidence of the weekly product queue
meetings and it’s clearly stated who was responsible for the meetings and what the agenda for
the meeting was. Still the team answers there is not anyone appointed responsible for the
meetings during the scorecard rating. (appendix A)
Clear roles and responsibilities are discussed in NIST800-64 control gates. (Richard et al. 2008)
The internal company documentation described a pre- gate meeting form that was done once a
week by key people in the team. Email conversation support that the customer used the vendor to
be the driver for the meetings and perform the cost/benefit analysis at the meetings together with
the customer, but the vendor experiences that they were not in the position to do so.
Figure 6. Screenshot from email conversation with the vendor. Vendors answers.
36 | P a g e
Interpretation at collection time: The main idea with this lowest level of gate meeting was to
have an updated queue of possible issues that can be included in a delivery. In other words a
preparation for a project with the purpose to have a list of issues that can easily be estimated in
time in order to sign agreements with vendors to implement the list of issues. This meeting form
cannot be directly mapped to the gate meeting as cooper defines it with go/no go decision for the
project; the product queue meeting is more like a pre meeting that serves for information sharing
and decision about the content and cost/benefit prioritizing for the later gate meetings. The
product queue meetings were the weekly meetings to collect the pieces that later can be puzzled
into a delivery project. It was here the team learned about the core of the project, and which was
later presented at the actual gate meetings for go/kill decisions.
As can be seen in the screenshot the criteria for prioritizing were not documented. Email
conversation supported, that there were, difficulties in defining these in the IT-department
because they needed to rely on business goals which were defined by another department in the
organization. Observations showed that people working with business goals were not part of the
product queue meetings. Email exchange supported that decision makers at the gate meetings
based their decision on the information presented by key people from the teams, that had an
understanding of the content of the product queue, and issues that were included. Observations
display that during the product queue meeting a list of ‘things’, in other word reported issues
were presented, and the idea was to arrange these in a prioritized order and include these in the
next delivery. These issues could be of any kind from identified bugs or questions for a new
architectural framework. Because the issues were yet not categorized in size of workhours they
could span small and big work tasks.
5.3.3 Concepts in the development process
Evidence shows that languages were mixed in the system development tools and then also in the
general method. Much of the English was kept with scrum terms like ‘sprints’ , ‘subtasks’ ,
‘stories’. Interpretation at collection time: It was obvious that vendors try to think agile
methodology during development, but also tried adapting to the customers concept.
37 | P a g e
Figure 7. Screenshoot of planning board displaying project sprints within SOA integrationcenter.
The term ‘virksomhetsko’ can best be translated to a business queue which has the purpose of
catching more general business requirements and not yet detailed functional descriptions to be
more processed.
Figure 8. Screenshot from vendors + customers mutual space for development process.
38 | P a g e
5.3.4 Delivery planning.
The IT-department followed predetermined dates for deliveries and the policy stated that the
reason for this, among other things, were to be able to evaluate the risk and coupling to other
systems.
Figure 9. Screenshot displays delivery planning for the IT-department.
5.3.5 Control gates in the SecSDLC
The organization had a SecSDLC document which described what and when security needed to
be assured in the SDLC process. The document enlisted that system development documentation
needed to be done and among other details it said “beskrivelse av gjennomført risikoanalyse”
which means that a description of the risk analysis is done.
Interpretation at collection time: This statement could be directly connected to the gate
meetings by the statement “Etatens metode for håndtering av endringer skal benyttes. Ref.
melding i oppdragsdatabasen ODA samt RFC-skjemaet som brukes av IKT-drift for å melde
39 | P a g e
endringsbehov (RFC: ”Request For Change”).” It means that to ensure security in the
administration the routines for the gate meetings earlier described at 5.1 needs to be followed.
The SecSDLC document said that the development process shall be followed and documented
and highlighted that information on the product queue elements internal priority needed to be in
place. Also there were roles with appointed responsibilities to ensure these processes and
documentation. The document states that these role needed to collaborate with roles within the
IT-department to ensure security in the process. During observation was noted that the role which
had responsibility to ensure the SecSDLC processes were part of the business department which
is located in another building. The SecSDLC document stated “Prosjektleder har et selvstendig
ansvar for at relevante krav blir identifisert, samt å legge fram for prosjekteier evt. mangler man
er kjent med.”
Interpretation at collection time: This statement proves the role of the ‘project owner’ which
was located at the business department and putted the project manager as responsible to give
roles higher up in the line correct information to make decisions. It meant the project manager
needs to retrieve this information from the developing team, and/or by attending at product queue
meetings. Roles and responsibilities is discussed in NIST800-64 Control gates by Richard(2008).
The IT- department had a special unit for ensuring security which according to the SecSDLC had
responsibility to performs audit of core documents like requirement specifications for the
security, and these had to be delivered and approved by the unit for information security. It was a
role within the business department who was responsible to create the information security
requirements.
Interpretation at collection time: This statement in the SecSDLC results in a big challenge for
this role located in the business department to ensure security due to limited option to
participation in the development process.
The SecSDLC stated “Det skal som del av arbeidet med utvikling av kravspesifikasjon for
sikkerhet gjennomføres minst en risikovurdering av den løsning som skal utvikles eller anskaffes.
40 | P a g e
For risikovurderingen må forutsettes at de sikkerhets- og personvernkrav som er identifisert blir
ivaretatt. Evt. supplerende tiltaksbehov må identifiseres. Tiltak som alt er identifisert som krav,
men som ikke virker ”sikkerhetsmessig lønnsomme” bør også identifiseres.” This means that a
risk analysis needed to follow the requirement specification with a cost/benefit evaluation.
Interpretation at collection time: In this context the policies highlighted the importance of the
cost/benefit and risk analysis but as earlier evidence show the developing teams were not
provided with any criteria how to perform this evaluation. Cost/benefit and risk analysis is
mentioned in the control gate NIST800-64 by Richard(2008).
The SecSDLC states “Ved bruk av SLEM eller andre iterative utviklingsmetoder skal produktet
fra hver iterasjon kontrolleres mot sikkerhetskravene knyttet til elementene som inngikk i
iterasjonen.“ with this statement the document links the vendor scrum sprints with the SecSDLC
and states that information security shall be ensured after every iteration.
Interpretation at collection time: The document was clear about who was responsible of
approving deliveries regarding security which aims responsibility to the business department.
The part describing required documentation was detailed with the different documents needed at
a delivery and the security testing process was well defined with included control gates to
approve testing requirements. Test cases is described in the phase2 control gate NIST800-64 by
Richard(2008).
The statement proves the mixed environment with linkage between agile iterations and the
stepwise ITIL process used in-house. This unclear mix must be a big challenge for all involved.
This security control gate required as a minimum to assure the security at the end of the last
iteration “Testing av sikkerhet skal minimum foretas etter siste iterasjon” The product and its
requirement specification needed to be assured after every iteration and checked towards the
security specification.
5.3.6 Relation between the SDLC and the SecSDLC
There was a part of the SecSDLC which described the relation to the SDLC. The document
stated “Ingen særskilt sikkerhetsoppfølging om sikkerhet ikke berøres i noen vesentlig grad av
41 | P a g e
leveransen” Interpretation at collection time: The statement means that a follow up on
information security is only done when so is needed. This part states “Likeså gjennomføres en
risikovurdering med det omfang som er relevant for leveransen.” Interpretation at collection
time: The statement means that a risk evaluation needs to follow the delivery. The document says
” I tilknytning til krav til sikkerhet må det påsees at evt. klassifisering av ressurser som er foretatt
blir dokumentert.” Interpretation at collection time: The statement meant that assets needed to
be classified. This statement highlighted the classification of resources to be concluded in the
cost/benefit or an impact analysis. Also testing of security was needed to be done with test cases
well documented and approved. In the beginning of this part the documents highlighted the
requirements for security which needed to be defined and approved.
Next in this study we will take a look at the product queue meetings which founded the base
information that the control gate council grounded their go/no go decisions on.
5.4 Interviews.
Scorecard rating of the product queue
The interview was performed as a group interview with six key people involved in the product
queue meetings for the integration centre. The limited set of questions were asked and the team
agreed on a rating how well the question was currently implemented. The team had the option to
rate the state to Red Not done, Yellow On the way, Green Done or Black no wish to do. The
meeting had the idea of arguing and deciding on one colour for every question to reach
consensus. The involved persons/roles at the interviews were project manager, team manager for
the SOA integration, test manager, 3 functional architects including myself. All questions were
delimited to the product queue because the product queue meetings founded the base information
for assessing risk/cost benefit before the gate meetings. Gate meetings are defined in information
security standards (Richard et al. 2008) and the product queue meetings in the organization could
be considered the atomic part of the gate meetings.
5.4.1 New elements that arrive and QA
42 | P a g e
The first section of questions (Appendix A) handled roles and responsibilities in the product
queue meetings. All questions were rated ‘Yellow’ by the team which meant that the team was
not satisfied and experienced roles and responsibilities unclear. Interpretation at collection
time: Comments from the team displayed explanations to be that there was a need for templates
displaying how the work should be done, and it was not yet clarified, who was responsible to do
the tasks. These issues could be partly explained by the fact that the team was new and not yet all
roles were hired. What couldn’t be explained by the theoretical framework is that the
organization has a well-defined process internally called SLEM for handling the product queue
but still it was not yet fully functional. According to NIST800-64 standard roles and
responsibilities need to be clear in the early phases of the project. (Richard et al. 2008)
5.4.2 Establishment of the product queue meetings
In the second part of interview questions (appendix B) more in depth questions about the product
queue were handled. On the question ‘Requirements are processed regularly?’ all interviewees
answered ‘Green’ which means the goal was fully implemented. Interpretation at collection
time: This evidence indicated that the internal process was implemented, and there was a
workflow which included regular QA like product queue meetings. Yet there was not a role
identified which had responsibility for the meetings.
This verified the internal model SLEM was implemented to introduce product queue meetings.
5.4.3 Impact evaluation
In the third part of interview questions (appendix C) impact analysis was handled. All questions
were answered ‘Red’ which means that impact analysis was not done in any way. Interpretation
at collection time: This evidence indicates that the organization was not working with impact
evaluations and the responses indicate that the reason was lack of structure. It seems people did
43 | P a g e
not know how do to this within the organization. The company documentation stated clearly that
the impact analysis was required before the control gate meeting. According to NIST800-64
standard impact analysis need to be done in the early phases of the project. (Richard et al. 2008)
5.4.4 Product elements are roughly estimated and prioritized
In the fourth part of interview questions (appendix D) risk assessments were handled. The team
experienced that risk was not fully understood and cost/benefit analysis was not used for
prioritizing elements in the product queue. The team answered that the business department was
not part of decisions and prioritizing within the product queue. Elements in the product queue
were regularly prioritized. Interpretation at collection time: This section displays, there was
lack of support from the business department regarding prioritizing elements within the product
queue. The reasons why the collaboration between the business department roles and the product
queue roles did not work is unclear. Observations support that the business department was
located in another building. There was a close collaboration between the vendors and the
customer which participate in the product queue meetings. Elements were regularly prioritized
but not with support from the business department, and not by using any defined criteria. The
evidence indicated that the prioritizing was not done with a predefined method. Risk/impact
analysis is clearly defined in NIST800-64 standard discussed by Richard (2008) as part of
control gates.
5.4.5 Incident and problem handling
Fifth part (appendix E) handles incidents and problem handling. This covered immediate errors
and bugs that must be fixed to maintain service and these problems were automatically prioritized
because a serious error can bring the service down. To all questions the team answers ‘green’
which means there were no issues with this area. Interpretation at collection time: Incidents
and problems were a separate lane in the organization because the business department has
defined it as a priority. This indicates that the problem which existed in relation to the product
queue prioritization was related to lack of support from the business department. The importance
44 | P a g e
of management's impact is discussed by Guttman(1995) NIST800-12. Management need to
support the prioritizations to be able to succeed.
5.5 Complement observations
The vendors had more than 15 people located on site and the activity was intense with
discussions and phone calls. Colleagues discuss about big amounts of email (up to 100 emails) in
just few days which indicates high activity. Observations shows frequent emails from the vendor
asking for correct permissions to be able to handle tasks within the organization, which in turn
lead to a meeting with the total plan manager, about how to solve the permissions for the vendors,
to be able to perform their tasks on site. This incident reveals that the reason for this high degree
of close collaboration was to solve practical issues more smoothly. Interpretation at collection
time: Sharing tool and virtual space removes many practical obstacles. Maybe it is so that the
mixture culture with agile and stepwise methods makes this necessary. Vendors shared the
management tool for incidents and error handling, in a wide degree with the customer working in
the same space, both physically and virtually. During an internal course at site for information
security certification I asked the question: “How is information security assured in the process?”.
In the room was a mixed audience from the organization with a teacher that was very experienced
and had been working as an architect since many years. The answer to the question was that the
SecSDLC process is not yet fully implemented and for now it’s ensured by educating the team
members. Interpretation: This answer indicates that a single individual have a hard time to get
a holistic perspective, even with years of experience. Will the SecSDLC ever be possible to fully
implement. Maybe it is so that the answer always will be the same.
45 | P a g e
6. Analysis
According to Methodology (4.2.3 to 4.2.5) the first step during data collection was to map the
informants which were part of SSCG and which were not. Next step was to relate data which was
documented in the company documentation, and which was unused. Methodology stated that
interpretations were done during collection time to reduce interference over time. Yin(2009)
suggests a triangulating approach to analyse multiple sources of evidence and build explanations.
To achieve a triangulating effect all sources of data was first analysed using the NIST800-64 and
NIST800-12,800-55,800-37,800-30 as lens and then explanations were built. All roles
interviewed were part of the SSCG.
6.1 Data in line with the NIST 800-64, 800-12, 800-55,800-30
Matching data using the NIST 800-64 control gate as ‘lens’.
NIST 800-64 Control Gate Criteria
The control gate in the standard states:
NIST 800-64 standard states in the control gate for the SDLC Phase:Initiation that the IT system
which is about to be developed has been anchored with the business department and concepts,
budget is assured. “A system concept review that verifies that the concept is viable, complete,
achievable, and in line with organizational mission objectives and budgetary constraints; “
(Richard et al. 2008, p.14)
The collected data reveals:
During interview(section 5.2.4) the team responded that the business department was not part of
the decisions for prioritizing but should have been involved because they are financing.
The control gate in the standard states:
NIST 800-64 standard states in the control gate for the SDLC Phase:Initiation that there is a risk
management document which includes categorization and impact levels.
“Included in this risk management review is a review of the information system security
categorization results, which include identified information types, resulting impact levels, and the
final system security categorization.” (Richard et al. 2008, p.14)
46 | P a g e
The collected data reveals:
During interview (section 5.2.4) the team rated that the risk was not fully understood but there
were rough estimates. Company documentation stated (section 5.3) that risk analysis was
required and the business department and project manager were responsible to ensure information
security requirements. During interview (section 5.2.3) the team rated that there was not done any
impact analysis.
NIST 800-30 standard states:
NIST800-30 displays a process to understand the risk involving criteria for prioritizing and
impact in case of incident occurrence.
Analysing data in relation NIST 800-30
NIST risk calculations require correct input data such as criteria for prioritization and impact
estimates. In the real case, there were not criteria for prioritization and no accurate estimate,
making it impossible for the integration center to calculate risk.
NIST 800-12, 800-55 states:
The standard NIST800-12, NIST800-55 highlights the importance of roles and responsibilities
which gives clear separation of duties. Also NIST800-55 relates the clear separation of duties as a
factor for a successful measurement program.
The collected data reveals:
The company documentation (5.3.3) had clearly defined roles and responsibilities but the
interview scored the roles and responsibilities yet to be unclear regarding work in the product
queue meetings.
6.2 Data not in line with the NIST 800-12,55,64
During interview the impact analysis was handled and the evidence (appendix C) indicates that
people at the organization did not work with impact analysis at all and the reason was that there
was no solid structure for it. NIST800-64 standard discusses impact levels in the control gate
47 | P a g e
phase one. “Included in this risk management review is a review of the information system
security categorization results, which include identified information types, resulting impact
levels, and the final system security categorization.” (Richard et al. 2008, p.14)
The company documentation (5.3.4) discusses the impact analysis in relation to the SecSDLC.
Despite this evidence the impact analysis was not done.
The explanations during interviews indicated there is no solid structure. What a ‘solid structure’
really means is unclear. At the same time there was lack of support from the business department
and lack of roles and responsibilities, unclear criteria for prioritization. This evidence indicates
that ‘no solid structure’ can be interpreted, that the integration Centre is unable to determine the
value and prioritization of assets due to lack of input from the business department. The NIST
standard requires well assessed input, but in the real case there was no well assessed input.
The empirical study (5.3) display that the customer wanted to use the vendor to help with
prioritizing elements in the product queue, where the vendor gave a response which indicated that
they were unable to do so. In this sense the product queue meetings were not fully successful
regarding the prioritizing, and cost/benefit/risk analysis, even if there was intense activity and
close collaboration between the vendor and the customer, by sharing of virtual (IT tools) and
physical(office landscape) spaces.
6.3 Analysing chain of evidence and building explanations
Yin(2009) highlights maintaining the chain of evidence to increase the reliability. The maintained
chain of evidence has been analysed linking evidence from the case study database to the report.
6.3.1 Analysing data
Confirming product queue meeting
Evidence from Interview
48 | P a g e
“Every week, but this was done also on different levels and the team members were involved
with many levels of queue meetings. The queue meetings for the team were one level, but there
are higher levels of queue meetings.” (appendix B)
Evidence from direct observation
“Weekly meetings were held, conducting prioritizing of elements.” (5.3 company documentation)
Evidence from company documentation
“documentation of weekly product queue meetings” (5.3 company documentation)
Explanation based on the evidence using theory
The organization had well defined work processes and meetings are kept regularly according to
policies.
According to company documentation the work elements should have been prioritized before the
gate meetings but the prioritization was not fully successful. The reason why was not fully clear,
but evidence was indicating that due to lack of criteria for prioritization, and input from the
business department like for example estimates valuing assets. Also there is a lack of roles which
has responsibility to work with the business department to refine the prioritizations.
Evidence from Interview
“roles for this need to be implemented” (appendix A)
Evidence from direct observation
“customer hires vendor to prioritize within the product queue which is unsuccessful” (5.3
company documentation)
Evidence from company documentation
“procedure for product queue meeting defined and there is clear roles for who was responsible
for the procuct queue” (5.3 company documentation)
Explanation based on the evidence using theory
49 | P a g e
As Siponen(2006) state The existence of information security standards does not ensure that the
work processes is successful. Evidence (5.3 company documentation) shows that the customer
tried to push the responsibility to the vendor which replies they are unable to perform the
prioritization. Explanation: The department could not handle the product queue well due to lack
of roles and responsibilities who worked towards the business department getting well assessed
input for prioritizations.
Lack of participation from business department
Evidence from Interview
“Business dep. was not part of the decisions for prioritizing. The business department should
have been involved because they was financing. ” (appendix D)
Evidence from direct observation
“Business department was not participating at product queue meetings and was located in
another building” (5.3 company documentation)
Evidence from company documentation
“no criteria for prioritization existent” (5.3 company documentation)
“SecSDLC states that prioritization needs to be in place” (5.3.3 company documentation)
Explanation based on the evidence using theory
Richard(2008) discusses cost and benefit analysis in an early phase. A cost/benefit assessment
has not been done according to the control gates. As Garnes(2004) discusses the importance of
support in an early stage from the upper management which in this case meant that the business
department must had supported the project in an early stage to reduce risk and secure success.
Impact analysis
Evidence from Interview
50 | P a g e
“Evidence indicated that the organization was not working with impact evaluations and the
responses indicate that the reason is lack of structure..” (appendix C)
Evidence from direct observation
“there were not impact evaluations documented“
Evidence from company documentation
“Impact analysis was discussed in relation to the SecSDLC.” (5.3.4 company documentation)
Explanation based on the evidence using theory
Impact evaluations were not done and the explanation given during interviews was lack of
structure. This could be interpreted using Siponen(2006) who discusses that even if the standards
exist it does not ensure quality. Whitman,Mattord(2009, p.164) suggests qualitative assessments
for organizations who are not able to use quantitative assessments. Political feasibility can be
part of the problem, when information security countermeasures and benefits isn’t well
communicated to decision makers as Whitman,Mattord (2009) elaborates. The impact assessment
is closely related to the risk calculation by assessing the value of the assets to be able to
determine the business impact. There was no collaboration with the business department
regarding prioritizations and therefore no assessments of the value of assets were available for
the integration center, thus was impact assessments impossible to perform at product queue
meeting level. The company documentation (5.1.6) which related SecSDLC to SDLC stated that
the assets needed to be classified, which was not successful at this case site.
6.3.2 Holistic view for the single case
The scorecard interviews showed that there were many roles and responsibilities yet to be filled
at the team and that could have been one explanation to some of the difficulties that the team
experienced. This fact could have lead to disgruntled employees if it took too long before the
roles were hired and responsibilities were made clear. If that explanation is correct the risk for
information security incidents were higher during this time because a high degree of security
breaches are performed by disgruntled employees according to theory in information security.
51 | P a g e
(Guttman, Roback 1995, Dhillon 2006; 2007) By looking at the data in overview perspective we
could see that there were difficulties in making cost/benefit/risk and impact analysis during the
product queue meetings. The data also displayed lack of participation from the business
department in the product queue meetings. The developing team did not perform impact analysis
at all due to lack of solid structure. The impact analysis seemed to be the most difficult to
perform because there was a consensus on ‘red’ answers throughout the interview regarding the
impact analysis. Despite this the prioritization and impact analysis was clearly stated in the
company documentation. Evidence show that the prioritization was performed despite that there
was no criteria for prioritizing and it was not supported from the business department. The
prioritization was performed by experienced individuals who made it using their own best
possible judgement. See overview appendix F.
The business department was located in another part of the town and roles from the business
department were not part of the product queue meetings. To be able to succeed in projects the
impact, risk/cost analysed needed to be firmly established in an early stage with the business
department. Garnes(2009) support that project risk is reduced if management is part of the
strategic meetings. The organization had SecSDLC documentation which clearly defined policies
for the information security process but evidence in the empirical study verified that the policies
were not fully implemented in a real case yet. This matched the discussion from Siponen(2006)
that states that the existence of standards does not mean they are implemented and ensure
success. The organization had its own method for working with system development which they
called SLEM. The method looked like a mixture between known methods and local adaptions to
the environment.
Rationale with gate and product queue meetings
By using the rationale concept discussed by Päivärinta(2011) we can look at the gate meeting, in
other words what was the rationale with using gate meetings at this organization. In the method
section we asked the question what was the functional reason that the organization choose to
implement the gate meetings? The answer was that the organization uses the gate meetings as a
forum for QA and sign off before launching any activities. In the empirical study we found
meetings that could be classified as gate meetings, but also lower level meetings that were called
product queue meeting. Also there was a document describing the content and purpose of the gate
52 | P a g e
meeting where one important part was the cost/benefit and risk/impact analyses, which should
have been evaluated. The purpose of the product queue meetings were to prioritize elements
according to cost/ benefit and risk analyses and to complete the elements with information, so
they could have been processed in the gate meetings. The product queue meetings were done
internally in the developing teams and portfolios, and the gate meetings were held horizontally
across the organization with key roles participating. Evidence from the empirical study revealed
that there were difficulties to perform the cost/ benefit and risk analysis in the product queue
meetings because of that no single individual was able to get a good overview and one probable
explanation could have been that it occurred due to complexity and lack of proper input from the
business department. The solution to this problem was to use one or more individuals at the
vendor to implement the product queue meetings, but also the vendor had the same difficulties.
Observations support that the process was continued by using experienced individuals at the
vendor to make the final prioritization of elements as a preparation to the gate meeting. Interview
questions related to the product queue meetings indicated that lack of dedicated roles for the
product queue tasks were one reason to the difficulties, but also that the department was under
establishment and many individuals were new to their roles.
Data not in line with theory from an alternative perspective
Evidence from the empirical study supported that during the gate meeting a team member from
the developing team, for example the project manager presented information with the changes to
the board at the control gate meeting using verbal presentation and written documents. There was
an interesting dialogue between the customer and the vendor in the empirical study, where the
vendor who had been assigned to make the cost/benefit analyses responded that they were unable
to do so and my conclusion to this evidence was that no single individual in this context was able
to do the prioritizing and cost/benefit decision or impact analysis with good enough quality alone.
Not the vendor and not the team at integration centre. This was visible also in the interview
section where the roles and responsibilities were undefined regarding cost/benefit and impact
analyses. This evidence can be looked at with theories from Dhillon(2006; 2007) where he
discusses the importance of responsibility, integrity, trust and ethicality. These factors will
become more important when it was impossible for one single individual to get an overview due
53 | P a g e
to complexity. The employees have to collaborate in a higher degree to be able to break down the
complexity. During the interviews there where some comments from the team that highlighted
improved communication between the business department and the IT-department and these
comments lead to new questions like if the collaboration between the IT-department and the
Business department was good enough? Do these departments understand each other well
enough? The integration centre needed to understand what the business department prioritized to
be able to conduct proper cost/benefit and risk analysis. The integration centre needed to
understand on which asset should be focus regarding the information security countermeasures
and on what could be accepted as a risk without countermeasures, also there was a need for the
motivation for those priorities, to be able to explain them at the gate meeting. Evidence indicated
that the organization included statements about risk assessments in policies. This evidence is
supported by Siponen(2006) who elaborates that the existence of the standards does not assure
that they are actually performed well in a real case. How well did the business department
understand details in the SecSDLC, SDLC processes? The background for the question was that
people sharing office space in the IT-department space, which worked all day with the SDLC
processes, had difficulties to categorize them and work according to them. How difficult would it
not be for an individual outside the department which did not work with the issues all day to
understand the details and the impact of them? In this light it was a reasonable assumption to say
that the role responsible for security in the Business department had limited opportunities to
understand the core information security issues and had to rely on second hand information
coming from the role responsible for security at the IT-department. The role responsible for
security at the IT-department had in turn rely on second hand information coming from the core
developing team. Also we can say that the gate meeting wanted answers from the core
developing team to describe what the core is all about, by getting information during the control
gate presentations, and so does the SecSDLC state in the company documentation, that the
process should work. This results in new difficulties because of the shared responsibilities with
limited understanding of the essential security processes, and complexity of the communication
steps in the business line from the business department down to the working team members. Yet
again this indicated the importance of good quality on the product queue meetings, but we could
conclude from the evidence that the product queue meetings did not have clear criteria for
making priorities of queue elements which meant that it was done based on individuals
54 | P a g e
perception of what were important and not by documented criteria. Observations on product
queue meetings supported that it were experienced individuals who made the final decisions of
the priorities in the product queue, which was not based on a defined criteria. It meant in the
worst case that the final product queue could have been prioritized by one experienced individual
in the core developing team without any interference from business goals because the complexity
gives the business department limited access to understand the impact of the system before its
online running and problems occur in reality. Looking at the SecSDLC it would only be possible
for the developing team to ensure security to a certain level because it was time consuming to
identify the risk/cost/benefit when criteria are nonexistent in the area of prioritizing queue
elements, that is if not the same individual that made the priorities for the queue also were part of
the information security assurance but that would yet be a new problem because due to separation
of duties. It is recommended to avoid security breaches by not give single individuals to much
control on crucial elements in the process. The site used sharing tools with the vendor which
meant that when the complexity increased in the big organization, there was a need to remove
practical obstacles to solve the tasks, and that was implemented by sharing space and giving a
high degree of permission to the vendors. To share office landscape meant that informal
knowledge was transferred more easily, also to handle complexity. This solution indicated that it
was crucial to be physically near the people to solve the tasks in the existent high complexity, or
at least the management believed it was. The SecSDLC was new to the organization and was still
not yet fully functional and this evidence indicates that the organization relied on the competence
of the developing team members to ensure security during development.
6.4 Summary analysis
The evidence from the data collection and analysis has been summarized to the following:

Integration centre holding the product queue meetings could not understand the risk due
to lack of necessary input like criteria, and business impact estimates.

Business department was not part of prioritizing at the product queue meetings.

Impact analysis was not done at all at the case site due to lack of proper input from the
business line.
55 | P a g e

Risk, benefit, cost, impact assessments are unsuccessful. There were difficulties with roles
and responsibilities. There was Lack of support from the business department. Lack of
criteria for prioritization. Risk was not fully understood.

Complexity was partly solved with close collaboration with the vendor, sharing virtual
and physical spaces.

Documentation of roles and responsibilities was clear in the SecSDLC.

Incident and problem handling was working well because incident and problem handling
had high priority from the business department.

Many methods were used in a mixture, agile methods and local adaptions of stepwise
methods. Vendor was working agile and customer with an internal method which is
stepwise.
At the product queue meetings the team at the integration centre was preparing work tasks for the
control gate meetings and conducting prioritizing. These prioritized lists were the information
needed to perform the control gate meetings with good quality and make the go/no go decisions
for the projects. The data indicate that prioritizations of elements and impact analysis were not
done and were not anchored towards the business department during product queue meetings.
Thus the integration centre was unable to calculate risk and make proper prioritizations of the
product queue elements related to risk and business impact.
56 | P a g e
7. Conclusions, discussions and further research
Even if the organization had well defined policies in place, which implemented quality assurance,
by having control gate meetings, the cost/benefit and risk/impact assessments at the level where
the integration center was working, were not fully successful. This contradictory result can have
an explanation from Siponen(2006) who says that even if the information security standards
exists, it does not secure a successful result. The evidence from the empirical study displayed that
the organization had detailed information security documentation, and well defined roles and
responsibilities, but they were not yet fully implemented in a real case. At the same time the
developing team was under establishment and there were many roles to be filled. The activity
was high and there was a close collaboration between the vendor and the customer but the
collaboration between the customer and its business department could have been improved. The
contribution of this case study was to give a practical case example describing control gates
implementation and experiences from cost/benefit, risk and impact analysis, during the early
phases of an organization IT-projects. The study described that information was prepared in the
product queue meetings before the gate meetings and described how the team at the integration
department experienced the situation. Evidence in this case study in relation to earlier research
theory extended the understanding of what it meant to be implementing information security
standards like control gates. This study is descriptive and has not presented a generalizable new
result. “descriptive case study has no especially important outcome” (Yin 2009, p.178) This
study has contributed with new understanding of a practical case within the governmental sector,
and some new aspects regarding quality of cost/benefit and risk/impact analysis. When the team
rated the impact analysis there was a consensus of ‘red’ which means ‘not done’ on all questions.
The evidence indicated that the impact was the hardest to perform due to complexity and lack of
proper input from the business department. The input lacking was criteria for prioritization, roles
that was responsible for refining the assessments and cost of impact. This difference can be
explained by the fact that cost analysis was related to economical prices which were possible to
assess, and benefits were easy to see, also risks there were always, but impact was more difficult
to predict. What if the prediction turned wrong, who was responsible for the future? It was
possible that impact analysis was loaded with politics which were preferably avoided.
57 | P a g e
Whitman,Mattord(2009) discuss the significance of political feasibility and highlight the even if
there is an economical value in information security countermeasures it is not sure they are well
communicated when decisions are made. “requires that arguments for information security
spending articulate the benefit of the expense for the whole organization, so that members of the
organizational communities of interest can understand its value” (Whitman,Mattord 2009, p.161)
7.1 Short simple answer to the research question
The research question was ‘How can control gates be implemented and what are the experiences
from when you try to use control gates for cost/benefit/risk and impact analysis?
The integration center was unable to secure prioritizing of product queue elements, in line with
the business needs, because it was impossible to do proper risk/cost/benefit and impact analysis,
due to complexity, lack of support and structure from the business line, with proper criteria for
prioritization, and good assessments regarding cost of impact. As Siponen(2006) elaborates the
existence of information security standards does not mean they are implemented in a way that
gives success for the organization. The NIST 800-55 states that a successful measurement
program can fail because of organizational dynamics, but the concept of organizational dynamics
is not elaborated in the NIST 800-55 document. The main experiences from implementing
control gates and trying to use cost/benefit/risk and impact analysis were that implementation can
fail due to complexity and organizational dynamics. “the information security measurement
program can fail when pressured by organizational dynamics” (Chew, Swanson & Stine 2008,
p.3)
Clearly the collaboration with the business department could have been improved because there
was no role from the business department contributing to the product queue meetings and assisted
on the cost/benefit and risk/impact analysis. The NIST 800-30 Guide for conducting risk
assessments describes formulas to calculate risk, and assumes well defined input to the formulas.
In the real case there was no well-defined input and the assessments were done by experienced
consultants who used their own methods to make the prioritizations.
New findings in the research
58 | P a g e
This paper describes the cause and effect within the governmental organization in terms
complexity and of lack of support from the business department towards integration center and
the effect it had on priorities, within product queue elements, and information needed for the
control gate meeting. The lack of proper input from the business department and overall
complexity of the systems, organization resulted in decreased quality during prioritization which
made the risk assessments to fail. It was known from earlier research that top-management
support is critical for success and this research reveals a real chain of events leading to failure of
risk/impact assessments due to complexity and lack of support from the top management.
“Support from top management, which also has the highest likelihood ratio of 7.299. For IS
protection measures, the support and coordination of top management are critical.” (Chan 2011,
p.633)
Also NIST800-30 discusses risk calculation but the chain of events in this study indicate that
similar risk formulas is toothless without proper input when it comes to really complex systems.
7.2 NIST 800-12 concepts
Guttman(1995) elaborates in NIST 800-12 an overview of the computer security area. The
overview lists elements of Computer security to be:
1. Computer security should support the mission of the organization
2. Computer security is an integral element of sounds management.
3. Computer security should be cost effective.
4. Computer security responsibilities and accountability should be made explicit.
5. System owners have computer security responsibilities outside their own
Organizations.
6. Computer security requires a comprehensive and integrated approach.
7. Computer security should be periodically reassessed.
8. Computer security is constrained by societal factors.
59 | P a g e
The standard 800-12 puts focus not only on technical measures but also on management and
social factors which means we can’t see look at computer security as something separate from the
business, but something which needs to be integrated in managing and planning the overall
project and its requirements. The overview content in NIST 800-12 is based on the OECD's
Guidelines for the Security of Information Systems. “draws upon the OECD's Guidelines for the
Security of Information Systems, Accountability ,Awareness, Ethics Multidisciplinary
,Proportionality, Integration ,Timeliness, Reassessment, Democracy” (Guttman, Roback 1995, p.
10) We can notice that this list isn’t fully matching to the list proposed by Meier(2011)

Authentication

Authorization

Auditing

Confidentiality

Integrity

Availability
The differences is the definition Guttman(1995) discussing Computer security versus information
security is the usage of the concepts proportionality and multidisciplinary where the definition
from Meier(2011) is more delimited.
Computer security should support the mission of the organization
In NIST 800-12 Guttman(1995) state that the purpose of information security is to protect the
organizations resources which can be hardware, software or information. “The purpose of
computer security is to protect an organization's valuable resources, such as
information, hardware, and software” (Guttman, Roback 1995, p. 9)
In this context Guttman(1995) emphasize that company value and profit comes first and security
secondary because information security does not exist for its own purpose but for keeping the
company safe and prosper to be able to stay that way also in the future. The main purpose is to be
competitive on the market and offer the customers best possible value and be as profitable as
possible in the task of doing so. “Security, then,ought to increase the firm's ability to make a
60 | P a g e
profit. In a public-sector agency, security is usually secondary to the agency's service provided to
citizens. Security, then, ought to help improve the service provided to the citizen.” (Guttman,
Roback 1995, p.9)
Computer security is an integral element of sounds management
Having a holistic view on information security is important because it cannot be looked as a
delimited element in the organization. It is common with usage of outsourcing and parts of the
system implemented by vendors which can also be members of the management. Guttman(1995)
discuss this in NIST800-12 and highlights that the security does not end and the border of the
organization but is extended outside the organization. “(1) know what general level or type of
security is employed on the external system(s) or (2) seek assurance that the external system
provides adequate security for the using organization's needs.” (Guttman, Roback 1995, p. 11)
This statement fitted very well to the described case in this study with frequent usage of vendors
on all levels in the organization, which also participated in the prioritization of work issues.
Computer security should be cost effective
The economical side must be considered when choosing information security countermeasures.
The first step is to understand the business and what brings economic value to the business and its
customers. It is not rational to make the information more safely protected than needed in relation
to its value if it’s lost. Guttman(1995) states in the NIST 800-12 standard that economic aspect of
protecting assets needs to be carefully considered. “Solutions to security problems should not be
chosen if they cost more, directly or indirectly, than simply tolerating the problem.” (Guttman,
Roback 1995, p. 11)
In this sense the cost/benefit/risk and impact analysis needed to be ensured in an early stage of
the project which is directly related to the product queue meetings in the case study.
Computer security responsibilities and accountability should be made explicit
Separation of duties is essential to ensure information security but also to get a workable division
of labor and to separate access and control of the crucial information. “The responsibilities and
61 | P a g e
accountability of owners, providers, and users of computer systems and other parties concerned
with the security of computer systems should be explicit” (Guttman, Roback 1995, p. 12)
In the case there was a degree of uncertainty about roles and responsibilities according to keyinformants at the integration center but at the same time the company documentation was clear
about responsibilities and roles.
System owners have computer security responsibilities outside their own organizations
In this case there was a usage of vendors with a close collaboration sharing physical and virtual
working spaces. How could information security be ensured when the vendors participate at
product queue meetings which are a strategic position? Guttman(1995) discuss security
responsibilities outside the organization and highlights the importance of coordination in this
sense. “In addition to sharing information about security, organization managers "should act in a
timely, coordinated manner to prevent and to respond to breaches of security" (Guttman, Roback
1995, p. 12)
This statement puts responsibility on the management to keep track of the knowledge moving in
and out of the organization.
Computer security requires a comprehensive and integrated approach
Guttman(1995) states that computer security must be seen in holistic perspective also
considering management and personnel. “Managers should recognize how computer security
relates to other areas of systems and organizational management.” (Guttman, Roback 1995, p. 13)
In these study theories from project management Garnes (2004) has been integrated to be able to
find explanations from a management perspective. The definition of control gate is connected to
Cooper(2001) and NIST800-64 standard which handles information security SecSDLC in
relation to control gate theory. As the focus is the organization and IT-projects which span
horizontally there is a need to find theories that can explain also the management perspective
including project risk.
62 | P a g e
Computer security should be periodically reassessed
Computer security is constantly moving forward because the technical progress drives the need
for new countermeasures. As Guttman(1995) states the research for new security
countermeasures can never rest. “System users and operators discover new ways to intentionally
or unintentionally bypass or subvert security.” (Guttman, Roback 1995, p. 13)
At the case site existed a special unit with chief information officer responsible for information
security. The integration center was a new unit which made a new challenge to the organization
to ensure security at the same time as the organization changed.
Computer security is constrained by societal factors
Guttman(1995) Gives an example of when the need for security conflict personal integrity and
the companies goals. “In some cases, privacy may be mandated by law” (Guttman, Roback 1995,
p. 14)
What the author means is that the company wants to log actions made using the computer but the
computer can also be seen as a tool to make private actions during work time for example
booking of an appointment to the hospital. In this sense there need to be a balance that fits the
local culture so the need for security do not intrude on privacy. At the case site the balance
between detailed control/security and privacy was difficult especially when there was an external
vendor involved which was related to another company. The many levels increase complexity.
The customer organization levels need for security in relation to the vendors company need for
security had to be coordinated which fast grew to an impossible task which made the security
situation uncontrollable at some level. As one example from the case site the role responsible for
the information security requirements were located in another building and had none or little
insight in what was really happening during development, but still was responsible to create
reasonable requirement regarding security.
7.3 Control gates NIST 800-64, 800-55
In NIST800-64 by Richard(2008) major security activities are discussed in relation to the control
gates. As this thesis is about the early stages of the SecSDLC the focus was at first phases and
63 | P a g e
control gates in the SecSDLC. In the first phases risk assessment is defined with the expected
output of a refined assessment of risk. This also includes design, project constraints and threats in
regard to business and IT components. NIST800-64 discusses impact analysis and states that the
impact to business needs to be considered.” Identify lines of business supported by this system
and how those lines of business will be impacted” (Richard et al. 2008, p.23)
Just as with the risk analysis the impact analysis was stated in the company documentation. In
the real case the team responded that there was no impact analysis and the risk was not fully
understood. This evidence indicated that there were issues with implementation of the control
gate assessments according to the standard and company documentation. The motivation stated in
NIST800-64 for creating risk assessments is to have control over the project. “Facilitation of
informed executive decision making through comprehensive risk management in a timely
manner” (Richard et al. 2008, p.1)
It was interpreted that the risk assessment was information needed for the managing the project.
In the case the business department wasn’t involved with the risk assessments in the product
queue which was contradictive to what is stated in the theory. The theory assumes that the
management needs to control the project, but what if the management did not care to control it
and thereby accepts the risk. Disgruntled employees can also exist among management which
should be considered a risk. In this sense the NIST standard is a top down ‘management control’
perspective. The NIST800-55 standard is discussed by Chew(2008) and states that upper
management must support the program to be able to succeed, this statement is similar stated by
Garnes(2004,2009) in regard to project management.
“The foundation of strong upper-level management support is critical, not only for the success of
the information security program, but also for the program’s implementation.” (Chew, Swanson
& Stine 2008, p.3)
In the case there was lack of support from upper management in regard to prioritization of the
product queue which could explain the difficulties experienced.
7.4 Informal aspects
64 | P a g e
The informal aspect which means social connection between people and feelings of trust,
ethicality and responsibility has become more important in new research.
Dhillon(2006,2007) links socio ethical controls with information security. “The notion of
obligation, accountability, and responsibility are indeed related and need to be considered in
conjunction with each other” (Dhillon 2007, P.208.)
Dhillon(2006,2007) says that even if there are policies and the employees and managers don’t
follow them, they don’t make any use. To compare with the opposite, an organization can work
perfectly without policies for everything if the employees have a strong culture based on ethical
behavior. These statements can extent the prior explanations to the theory in this case study
where there were documented policies but they weren’t followed. Maybe it’s so that the
socioethical perspectives are more vital than any other control. “the surest controls are likely to
be within informal systems” (Dhillon 2007, P.212.)
Similar to NIST800-30 standard Dhillon(2006,2007) discusses risk assessments and asset
identification ,feasibility analysis, identify and estimate the cost for an incident (business impact).
Risk can be assessed as the following:
R=pxc
R level of risk
P probability of an event probability of an negative event.
C cost
Plan and control is essential, an organization must create business disaster and business
continuity plans to ensure the business in case of a serious incident. Dhillon(2007) says all basic
controls that we implement must be related to the possible amount of money it means we can lose
in case of an incident. It is vital to have a working structure to ensure that everybody knows what
responsibilities is included in their role to be able to conduct the work and to reach progress. It’s
impossible to measure an employee’s performance without relating it to the role and
65 | P a g e
responsibility. Dhillon(2007, P.107) discuss three interrelated considerations “1. Organizational
considerations related to structures of responsabilites…”, “2.Ensuring organizational buy-in for
information system security..”, “3. Establishing security plans and policies…”
Dhillon(2006,2007) discusses Trust as distinct from Control and there must be a strong bond of
trust between employees with in a company to be efficient and succeed in the competition; it’s
not practically possible to have such rigorous control mechanism to control what everyone thinks
and how information is shared to 100%. Most projects have timeframes where creative and
innovative ideas is important and creative ideas flourish best in a uncontrolled environment
(Garnes 2004,2009) which means trust is essential for a company. It means that employees with
the right set of mind and a company where employees trust each other is the best kind of control.
Dhillon(2006,2007) discuss the RITE principles “responsibility, integrity, trust and ethicality”
Dhillon(2007, P.322.)
According to Dhillon(2007):
Responsibility: Today when most organizations become more horizontal it’s important for
employees to know their responsibilities to be able to fulfill each role responsibilities.
Integrity: As information is the companies most valuable asset it’s vital to keep it secure and this
is a tradeoff situation between giving access to third party and still keeping it secret.
Trust: Organizations are relaying partly on mutual trust for security. External controls are not
covering everything so to an portion collaboration must be based on trust.
Ethicality: Today short term employments are usual and people are moving in and out the
organization and therefore it’s impossible to have external controls to audit the information flow
fully. Due to this the only efficient way is to have internal controls and trust.
The informal aspects can give further understanding for the case where the business department
did not participate in the product queue and the lack of roles and responsibilities. Integrity, trust
and ethicality become important when no single individual is able to get overview and close
collaboration is necessary.
66 | P a g e
7.5 Management of projects and information security
Relations between people are an important factor to consider in project management and
therefore also in information security because information security is integrated with project
management as the SecSDLC is part of the SDLC. Garnes(2009) discusses classical economic
terminology and relational contracts. “Når man har samarbeidet over tid bygger det seg opp band
mellom personer og det blir gjort koblinger mellom organisasjoner. Partene har tillit til hverandra
og ser felles intresser i å opprettholde relasjonene” Garnes(2009, p.99)
Which means that when people have collaborated for a time there will be bounds between the
people and connections is made between organizations. People start to trust each other and see
mutual interests in keeping the relations. As Garnes(2009) argues this means that the relations has
an economical value and must be considered for all project management. As Dhillon(2006,2007)
explains the social factors has importance for information security it’s possible to see how well
the social factors are woven into organizational dynamics aspects which can give new perspective
on where to find explanations to failure of control gates that couldn’t be explained with theory. It
could be possible that collaboration between the business department and the control gate
meetings did fail due to similar factors. The integration center department was new and had not
yet build these bounds that Garnes(2009) highlights as important.
7.6 Recommendations
During this study evidence and conclusions were presented:

Key roles yet not implemented

Inability to perform cost/benefit and risk/impact analysis successfully

Lack of support from the line with proper criteria for prioritization.
In contradiction the incident and problem handling worked fine. The difference with the lane of
incident and problem handling was that this lane was fully supported from the business
department with clearly defined prioritizing. Bugs and error must be fixed to ensure service.
67 | P a g e
Richard et al. (2008) discusses cost and benefit analysis in an early phase. A cost/benefit
assessment was not been done according to the control gates. As Garnes(2004,2009) discusses
the importance of support in an early stage from the upper management which in this case means
the business department must support the project in an early stage to reduce risk and secure
success. This is also clear in NIST800-12 Guttman(1995) discusses the importance of
commitment from the organizational management.
“For a central computer security program to be effective, it should be an established part of
organization management. If system managers and applications owners do not need to
consistently interact with the security program, then it can become an empty token of upper
management's "commitment to security." (Guttman, Roback 1995, p.51)
In an ideal situation it would had been preferable to have a close collaboration with the business
department sharing physical and virtual spaces, of course there are issues that can occur in the
area of organizational dynamics which can make it impossible, still a better and closer
collaboration with the business department would had improved functions and reduced unclear
criteria when prioritizing. Roles and responsibilities needed to be implemented to reduce
workload on single individuals and would have made the separation of duties improved.
7.7 Further research
In this study reasons for difficulties with cost/benefit, risk/impact analysis were complexity and
lack of structure. This lead to some new questions, does the organization work with any
structured method for assessing risk? If not could a structured method like Delphi technique be of
help to give structure or would the difficulties be the same anyway? (Whitman,Mattord 2009,
p.165)
As Information security can be motivated to the top management by displaying the business cost
in case of impact and its likelihood to happen it is crucial that the assessments are proven to be
trustworthy. As it is now, it seems risk/impact theory is neither reliable nor easily understood
68 | P a g e
when complexity is high. In this light it is possible to understand the difficulty in securing
investment because top-management does not trust the outcome of the impact analysis. The cost
of business impact is one of the strongest incentives for information security investments. This
area requires further research. Is the reason lack of trustworthiness on the impact analysis that
information security isn’t a bigger issue in companies today?
As Chan(2011) discuss the reliability of Bayesian index method for risk modeling he states that
evidence is lacking. “The measurement of IS risk is the greatest concern of researchers Many
companies even use checklists or scoreboards to assess IS. They still worry about their validity
because they have no way of proving them, and some questions still remain: Will the
improvement measures reduce the risk? How can risk and security metrics be more closely tied to
the decision making?” (Chan 2011, p.635)
69 | P a g e
References
Ambartsoumian, V., Dhaliwal, J. & Meservy, T. 2011, "Implementing quality gates throughout
the enterprise IT production process
", Journal of Information Technology Management, vol. 22, no. 1.
Bishop, M.A. & Frincke, D. 2005, "A human endeavor – lessons from shakespeare
and beyond. IEEE Security & Privacy", vol. 4, no. 3, pp. 49-51.
Bredesen, I. 2005, Investering og finansiering, 3 utg edn, Gyldendal akademisk, Oslo.
Chan, C. 2011, "Information security risk modeling using Bayesian index", The computer
Journal, vol. 54, no. 4, pp. 628.
Chew, E., Swanson, M. & Stine, K. 2008, "Performance measurement guide NIST 800-55",
[Online], . Available from: http://csrc.nist.gov/publications/nistpubs/800-55-Rev1/SP80055-rev1.pdf.
Collis, J.M. & Messick, S. 2001, Intelligence and personality: bridging the gap in theory and
measurement, L. Erlbaum, Mahwah, N.J. ; London.
Cooper, R.G. 2001, Winning at new products : accelerating the process from idea to launch, 3rd
edn, Perseus, Cambridge.
Dhillon, G. 2006; 2007, Principles of information systems security: text and cases, Willey,
Danvers, Mass.
Gable, G. 1994, "Integrating case study and survey research methods: an example in information
systems. European Journal of Information Systems", European journal of information
systems, vol. 3, no. 2, pp. 112-126.
Gallagher, P. 2011, "NIST 800-30 Guide for conduction risk assessments", [Online], . Available
from: http://csrc.nist.gov/publications/drafts/800-30-rev1/SP800-30-Rev1-ipd.pdf.
70 | P a g e
Garnes, Å. 2009, Prosjektledelse og usikkerhet : et dialogisk perspektiv, Ny utg edn, Dixi omen,
Oslo.
Garnes, Å. 2004, Bedriftsforståelse og en fornemmelse av strategi : et dynamisk perspektiv på
bedrifter og strategivalg i historisk skapte forretningslandskap, Dixi omen, Oslo.
Ghemawat, P. 2006, Strategy and the business landscape, 2nd edn, Pearson Prentice Hall, Upper
Saddle River, N.J.
Glaser, T. & Pallas, F. 2007, Information Security and Knowledge Management: Solutions
Through Analogies?, http://ssrn.com/abstract=1014302, Technical University of Berlin Computer Science, Technical University of Berlin - Computer Science; KIT - Karlsruhe
Institute of Technology - Center for Applied Legal Sciences (ZAR).
Gordhamer, S. 2011, 5 New Paradigms for a Socially Engaged Company, mashable.com,
http://mashable.com/; http://mashable.com/2011/01/13/socially-engaged-company/.
Guttman, B. & Roback, E. 1995, An introduction to computer security : the NIST handbook, U.S.
Department of Commerce, Washington.
Henderson, B.D. 1979, Henderson on corporate strategy, Abt Books, Cambridge, Mass.
Kaplan, R.S. & Norton, D.P. 2002, The balanced scorecard, Tpb, Enskede.
Kendall, G.I., Rollins, S.C. & ebrary, I. 2003, Advanced project portfolio management and the
PMO, J. Ross, Conyers, GA.
Legrant, c., Kreitner, c., flynn, c. & paller, a. 10 jan 2005, "How to Achieve a Clear Definition of
Responsibilities for Information
Security. ", [Online], . Available from: http://net.educause.edu/ir/library/pdf/CSD3661.pdf.
[2012-04-24].
Mehrabian, A. 1981, Silent messages : implicit communication of emotions and attitudes, 2nd
edn, Wadsworth, Belmont, Calif.
71 | P a g e
Meier, J.D., Farre, C., Taylor, J., Prashant, B., Gregersen, S., Madhu, S. & Boucher, R. 2011, ,
Improving Web Services Security: Scenarios and Implementation Guidance for WCF
[Homepage of Microsoft], [Online]. Available: http://msdn.microsoft.com/enus/library/ff650794.aspx [2011, 09/28].
Mintzberg, H. 2000, The rise and fall of strategic planning, New edn, Prentice Hall, London.
Müller, R. 2009, Project governance, Gower, Farnham, Surrey, England ; Burlington, VT.
Nesse 2011, intellectual capital at statoil, Lecture edn, Nesse, Oslo.
Nielsen, D. 2008, 11 nov-last update, Conduction successful gate meetings [Homepage of
www.pmhut.com], [Online]. Available: http://www.pmhut.com/conducting-successful-gatemeetings [2011, 29 sep].
Päivärinta, S., Larsen 2011, "Towards a Framework for Building Theory from ISD Practices",
[Online], , pp. 2011-11-15.
Patel, R. & Davidson, B. 2003, Forskningsmetodikens grunder : att planera, genomföra och
rapportera en undersökning, 3, [uppdaterade] uppl edn, Studentlitteratur, Lund.
Porter, M.E. 2004, Competitive strategy : techniques for analyzing industries and competitors,
New edn, Free Press, New York.
Richard, K., Kevin, S., Matthew, S., Hart, R., Jim, F. & Jessica, G. 2008, Security Considerations
in the System Development Life Cycle.
Rosen, M. 2008, Applied SOA : service-oriented architecture and design strategies, Wiley Pub.,
Indianapolis, IN.
Siponen, M. 2006, "Information security standards focus on the existence of process, not its
content", Communications of the ACM, [Online], no. 8. [2011113].
solms, S.H. 2006, "Information Security Governance: A model based
on the Direct–Control Cycle", Computers security, [Online], vol. 25, .
72 | P a g e
Sverige. Statskontoret 2003, Ledningssystem för informationssäkerhet vid 24timmarsmyndigheter, Statskontoret, Stockholm.
Trott, P. 2008, Innovation management and new product development, 4th edn, Financial
Times/Prentice Hall, Harlow.
Walsham, G. 1993, Interpreting information systems in organizations, Wiley, Chichester.
Whitman, M.E. & Mattord, H.J. 2009, Principles of information security, 3rd edn, Thomson
Course Technology, Boston, Mass.
Yin, R.K. 2009, Case study research: design and methods, 4th edn, Sage, London.
73 | P a g e
Appendix
A. First part is questions about new elements that arrive and QA.
Question part I
It is identified who shall
register new elements to
the product queue?
Rating by the team
Yellow on the way.
It is decided what shall be Yellow on the way
registered in the product
queue and why?
It’s identified who shall
Yellow on the way.
assure quality of
requirements of new
product queue elements?
There are routines to
Yellow on the way
handle product queue
elements that is not
enough detailed?
Someone analyse if the
Yellow on the way
elements that arrives to the
product queue is detailed
enough?
Table 5. questions about new elements.
Comment from the team
Should be possible for
everyone in the team to
add new elements. But to
make sure it’s done, one
need to be responsible.
Templates need to be
developed.
Roles for this need to be
implemented.
Roles for this need to be
implemented.
Roles for this need to be
implemented.
B. Second part is questions about establishment of the product
queue meetings.
Question part II
Requirements are
processed regularly?
Rating by the team
Green done.
Comment from the team
Every week, but this is
done also on different
levels and the team
members are involved
with many levels of queue
meetings. The queue
meetings for the team are
one level, but there are
higher levels of queue
meetings.
74 | P a g e
One person is responsible
for the queue meeting?
Yellow on the way
The different levels of
queue meetings are
diffuse; maybe there is an
option to use fewer levels
of queue meetings.
It’s identified who should Green done.
be part of the queue
meetings?
Table 6. Questions about product queue meetings.
C. Third part is questions about impact evaluation.
Question part III
It’s identified who is
administrator of the
impact evaluations?
There are routines how to
do impact evaluations?
Rating by the team
Red Not done.
There are routines how to
handle incase impact
evaluations is not done?
Impact evaluations are
documented?
Red. Not done.
Red Not done.
Red. Not done.
Comment from the team
No solid structure. Is
performed from case to
case.
No solid structure. Is
performed from case to
case.
No solid structure. Is
performed from case to
case.
No solid structure. Is
performed from case to
case.
Table 7. Questions about impact evaluations.
D. Forth part is questions analysing if the product elements are
roughly estimated and prioritized.
Question part IV
The expected risk is well
understood from customer
as well as vendors?
The rough estimate is used
to calculate the cost?
A cost/benefit analysis is
done as basis for the
prioritizing?
Elements in the product
queue are regularly
Rating by the team
Yellow. On the way.
Comment from the team
Requirements for risk are
not documented and there
are only rough estimates.
Green. Done.
Black. Is not done.
Green. Done.
Is done as part of the
product queue meetings.
75 | P a g e
prioritized?
The customer (business
dep.) is part of the
prioritizing?
Yellow. On the way.
Business dep. Is not part
of the decisions for
prioritizing. Should be
involved because they are
financing.
Table 8. Questions about prioritizing.
E. Fifth part is questions related to incident and problem
handling.
Question part V
Rating by the team
Product errors(bugs) is
Green. Done
added as elements in the
product queue ?
Product errors is a result
Green done.
of the process of problem
handling and is
complemented with
estimates and suggestion
for solution ?
Product errors is
Green. Done.
prioritized in the same
queue as other issues like
changes and incidents?
Table 9. Questions about problem handling.
Comment from the team
F. Overview of data that can be related to control gates or
context in the lens of the theoretical framework.
Observations
Intense activity on site
Company documentation
Lack of criteria to make
prioritizations.
Sharing of virtual and physical
space. Office landscape and IT
tools.
Roles are separated by offices.
Business department is
located 15 minutes from the
Vendor and customer have
difficulties to perform
prioritizations.
If and when a QA of the steps
in the SecSDLC and SDLC
should be done is decided
Scorecard interviews
A cost/benefit analysis is not
done as basis for the
prioritizing.
Business department is not
part of the prioritization.
Lack of roles and routines for
the product queue meetings.
76 | P a g e
IT-department.
Product queue meetings are
handled by the developing
team.
Business department is not
participating in the product
queue meetings.
Product queue agenda is a list
of work issues to be
prioritized.
from case to case.
Project manager and role in
Risk is not understood.
the business
department(project owner) is
responsible to assure
SecSDLC during
development. Audits are made
from information security
department.
Clearly defined policies and
roles during gate meetings and
procedures.
Prioritizations needs to be
clear after product queue
meetings and presented at the
gate meetings.
risk analysis needs to follow
the requirement specification
with a cost/benefit evaluation
Experienced individuals are
used from the vendor to make
the final prioritization of
elements as a preparation and
no obvious criteria are used
for this.
Assurance of knowledge is
No uniform method for
done in interview form at
development. Mix of methods
meetings. For example a role
with agile, ITIL, and local
in lead position, or a board
adaptions.
listen to a project manager or
some from the developing
team to ‘judge’ maturity of the
issue to make decision for
go/no go. Measurement of
knowledge is done face to face
with documents which has
been prepared.
SecSDLC process is not yet
fully implemented
(observation/question asked
during lesson at company
course.)
Table 10. Three sources of evidence in the empirical study.
77 | P a g e