EaseSAF: A TOOL FOR MODELING SOFTWARE ASSURANCE
Nancy Talamantes
B.S, California State University, Sacramento, 1985
PROJECT
Submitted in partial satisfaction of
the requirements for the degree of
MASTER OF SCIENCE
in
SOFTWARE ENGINEERING
at
CALIFORNIA STATE UNIVERSITY, SACRAMENTO
SPRING
2014
EaseSAF: A TOOL FOR MODELING SOFTWARE ASSURANCE
A Project
by
NANCY TALAMANTES
Approved by:
__________________________________, Committee Chair
Dr. Ahmed Salem
____________________________
Date
__________________________________, Second Reader
Dr. Cui Zhang
____________________________
Date
ii
Student: Nancy Talamantes
I certify that this student has met the requirements for format contained in the University
format manual, and that this project is suitable for shelving in the Library and credit is to
be awarded for the Project.
__________________________, Graduate Coordinator
Dr. Nikrouz Faroughi
Department of Computer Science
iii
________________
Date
Abstract
of
EaseSAF: A TOOL FOR MODELING SOFTWARE ASSURANCE
by
Nancy Talamantes
Operational environments for today’s information technology solutions are complex and
varied. The majority are constructed with systems and systems-of-systems, often
including cloud computing and the internet. With the increase of web-centric
applications, it is important to understand how systems relate to each other both internally
and externally. People’s behavior in an organization is also for consideration in the
adoption and operation of assurance solutions. While a great deal of work has been done
to identify and catalog software assurance solutions, little information is available about
what is needed for an organization to successfully adopt and use them in operational
settings. The Software Engineering Institute (SEI) at Carnegie Melon, in tandem with
the United States Department of Defense (DOD) has developed a framework that models
an organization’s software assurance profile. In their paper, A Framework for Modeling
the Software Assurance Ecosystem: Insights from the Software Assurance Landscape
Project [1], the team from SEI describes their piloting of this framework to prove its
value and gain insights in the application of the model in an organizational setting.
This report presents the Assurance Modeling Framework methodology and the
development of the EaseSAF tool that semi-automates the nine steps of the methodology.
EaseSAF is a desktop tool that integrates the specification of assurance modeling
requirements using the formal methods based on SEI’s Assurance Modeling Framework.
It is an effort to develop an application that allows users to input and manage the
enormous amount of data and artifacts necessary to support this framework.
iv
_____________________________, Committee Chair
Dr. Ahmed Salem, Ph.D.
_______________________
Date
v
DEDICATION
This project is dedicated to my children, Daniel and Stephanie, who were my inspiration
and support through this seven and a half year journey. You were always there to listen
when I needed it most.
vi
ACKNOWLEDGMENTS
I would like to thank Dr. Salem for his support and guidance as my project advisor. His
assistance throughout the course of this project was invaluable.
I would also like to thank Dr. Cui Zhang for her time and understanding in agreeing to be
the second reader for this project. Her guidance and support have helped me enormously
over the years in my pursuit of this degree.
I would like to express my sincere thanks to the Computer Science faculty and staff that
went above-and-beyond more than once with their ready assistance, especially in the last
six months. It was not always a smooth ride.
Lastly, the instructors, fellow students and mentors from the Sacramento State Computer
Science department have inspired me throughout my graduate and undergraduate years.
Their skills and knowledge have guided me throughout my life and career. I am deeply
indebted to all of you.
vii
TABLE OF CONTENTS
Page
Dedication .......................................................................................................................... vi
Acknowledgments............................................................................................................. vii
List of Figures .................................................................................................................... ix
Chapter
1. INTRODUCTION ...........................................................................................................1
2. BACKGROUND AND RELATED WORK ...................................................................3
2.1 The Need for Software Assurance…………………………………………….3
2.2 The Assurance Modeling Framework and Pilot Project………………………7
2.3 The EaseSAF Assurance Modeling Framework Steps……………………….8
2.4 The EaseSAF Project………………………………………………………..10
3. EASESAF ANALYSIS, ARCHITECTURE AND DESIGN ........................................11
3.1 Functional and Non-functional Requirements……………………………….12
3.2 EaseSAF Architecture and Design…………………………………………...14
3.3 EaseSAF User Interface Design……………………………………………..18
3.4 EaseSAF Database Design…………………………………………………..38
4. EASESAF IMPLEMENTATION AND EXAMPLE ....................................................40
5. CONCLUSION AND FUTURE WORK ......................................................................55
References ..........................................................................................................................57
viii
LIST OF FIGURES
Page
Figure 1 EaseSAF Architecture .........................................................................................15
Figure 2 NetBeans IDE with EaseSAF Application ..........................................................19
Figure 3 EaseSAF Screen Flow Diagram ..........................................................................20
Figure 4 EaseSAF Welcome Form ....................................................................................21
Figure 5 Create New Profile Form.....................................................................................22
Figure 6 Open Existing Profile Form.................................................................................23
Figure 7 EaseSAF Nine Step Form ....................................................................................24
Figure 8 EaseSAF Profile Summary Form ........................................................................25
Figure 9 Principle Perspectives and Influences Form.......................................................26
Figure 10 Principle Perspectives and Influences Form – Organization Tab ....................27
Figure 11 Principle Perspectives and Influences Form – Supply Side Tab .......................27
Figure 12 Principle Perspectives and Influences Form – Demand Side Tab .....................28
Figure 13 Value Exchange Form .......................................................................................29
Figure 14 Potential Assurance Results Form .....................................................................30
Figure 15 Motivations Form ..............................................................................................31
Figure 16 Critical Behaviors Form ....................................................................................32
Figure 17 Adoption of Products Form ...............................................................................33
Figure 18 Future Drivers Form ..........................................................................................34
Figure 19 Inefficiencies Form ............................................................................................35
ix
Figure 20 Prioritized Improvements Form ........................................................................36
Figure 21 Upload Artifacts Form.......................................................................................37
Figure 22 EaseSAF Entity Relationship Diagram .............................................................39
Figure 23 Create a New Profile .........................................................................................41
Figure 24 Vulnerability Management Profile Description ................................................42
Figure 25 Vulnerability Management and CVE ................................................................43
Figure 26 Organization Tab for CVE ................................................................................44
Figure 27 Supply Side Tab for CVE ..................................................................................45
Figure 28 Demand Side Tab for CVE................................................................................46
Figure 29 Value Exchange Information for CVE ..............................................................47
Figure 30 Potential Assurance Results for CVE ................................................................48
Figure 31 Key Drivers of Risk for CVE ............................................................................49
Figure 32 Critical Usage Scenarios for CVE .....................................................................50
Figure 33 Operational Adoption and Usage of CVE ........................................................51
Figure 34 Future Impacts for CVE ....................................................................................52
Figure 35 List of Inefficiencies for CVE ...........................................................................53
Figure 36 List of Prioritized Improvements for CVE ........................................................54
x
1
Chapter 1
INTRODUCTION
The United States Department of Defense defines system assurance as the justified
confidence that a system functions as intended and is free of exploitable vulnerabilities,
either intentionally or unintentionally designed or inserted as part of the system at any
time during the life cycle [1].
In their technical paper, A Framework for Modeling the
Software Assurance Ecosystem: Insights from the Software Assurance Landscape
Project, the authors from the Software Engineering Institute (SEI) at Carnegie Mellon
University define an assurance solution as a policy, practice, or technology that
contributes to system assurance (i.e., to provide justified confidence that a system will
function as intended and is free of exploitable vulnerabilities) [1].
This paper presents the EaseSAF tool, a desktop application that provides semiautomated assistance to assurance analysts with managing the mounds of artifacts and
paperwork necessary to support this framework. Eliciting the information required for
assurance modeling involves a great deal of interviewing, analyzing and diagraming. The
artifacts such as the Value Maps and System Dynamic Diagrams are uploaded and
managed by EaseSAF. The analyst enters the framework question information into the
EaseSAF form for each of the nine steps in the modeling framework. The supporting
artifacts are uploaded to pre-designated windows folders for organized archiving. The
EaseSAF Profile Summary form summarizes the framework profile and framework
question information for the analyst in a concise report. The assurance analyst can dig
2
deeper into framework details by opening the archived supporting artifacts uploaded
during the modeling process. The completed profile entered into EaseSAF by the analyst
is a living collection of artifacts that the organization can update as assurance solutions
improve and assurance needs change.
The following paper describes the development of the EaseSAF desktop tool. Chapter 2
discusses the background and related work of this project. It will describe the Assurance
Modeling Framework developed by SEI and their pilot project to implement it in an
operational setting. Chapter 3 includes the description of the requirements, system
architecture and design of EaseSAF as a desktop tool. Chapter 4 documents the
implementation details used in building EaseSAF. Chapter 5 presents an example that
shows the benefits of using EaseSAF compared to the manual approach. The conclusion
of the report includes a summary and future work for EaseSAF.
3
Chapter 2
BACKGROUND AND RELATED WORK
2.1 The Need for Software Assurance
Many of today’s organizations rely on a complex set of systems, often involving cloud,
web and mobile connectivity. These systems can be supported by many platforms
including mainframe, web and desktop with elaborate middleware components. Systems
are built with custom source code often interfacing with Customized-Off-The-Shelf
(COTS) products that may contain proprietary, un-viewable and potentially insecure
internal code. Data is shared data through Service Oriented Architecture (SOA) services,
traditional file transfer, JDBC connections, the Cloud, web connectivity and mobile
devices. Data cleansing, data transformation and data transmission are accomplished
through busses, queues and other means often involving elaborate encryption and
decoding. Data stores are large and complex with new technologies such as “Big Data”
that allow the processing of huge amounts of data to appear seamless. Elaborate
Infrastructure to support this processing must be put in place. Extensive monitoring,
load-balancing and failover subsystems must be installed and faithfully maintained to
ensure that all system components and services are available 24/7. Capacity Planning
and system monitoring are an on-going challenge. Disaster recovery and infrastructure
also cannot be ignored. This complexity and flexibility comes at a cost. Systems are
constantly challenged to maintain security, performance and reliability in these everchanging environments.
4
For consideration as well are the soft factors of human involvement. Systems are built,
used, monitored, maintained, governed and financially supported by a diverse and everchanging group of participants that are a critical factor in the assurance ecosystem. In
their technical paper, Spotlight On: Programmers as Malicious Insiders–Updated and
Revised [5], the authors from SEI discovered that the majority, (over 25 percent), of
cyber security attacks are instigated by company insiders bent on revenge, mischief or
sabotage. The financial impact to an organization for these attacks can cost over a
million dollars. In their technical paper, Common Sense Guide to Mitigating Insider
Threats, 4th Edition [6], SEI’s Computer Emergency Response Team (CERT) analyzed
the characteristics of employees who are more susceptible to malicious behavior. Many
of the incidents identified by the CERT study were the result of an employee that had
recently been notified of termination or a change in status. System administrators,
network administrators, security administrators and developers have been documented as
the instigators of cyber-related revenge. The chief motivation is to create harm for the
responsible organization by causing a computer system disturbance that will either cost
the organization money, gain access to private customer information and/or damage
company reputation. Many of these individuals have previous criminal history and ties to
organizations intent on cyber sabotage.
Software is a key component of today’s systems and system-of-systems assurance
landscape. If application software is well written, thoroughly tested, well documented,
follows best practices and secure coding standards it will support and enhance an
organization’s operational setting. Poorly written source code, however, can sabotage a
5
functioning system with surprising speed. Security vulnerabilities built into the code
either intentionally or unintentionally, leave a system open to attack [3]. Under the
direction of Robert Seacord and the CERT team, The Secure Coding Initiative (2010)[3]
has developed secure coding standards for C, C++ and Java. According to the CERT
website, Seacord’s team has evaluated thousands of vulnerability reports and has
determined that most vulnerabilities stem from a relatively small number of common
programming errors. The CERT Oracle Secure Coding Standard for Java [3] states that
based on CERT’s research, 64 percent of the nearly 2,500 vulnerabilities in the National
Vulnerability Database in 2004 were caused by programming errors. Adopting secure
coding practices is one of many proactive approaches to improving software assurance.
As mentioned previously, the current trend among corporations and government agencies
to introduce commercial off-the-shelf (COTS) software products into systems in an
attempt to save development time and expense can be an assurance risk. The source code
in these COTS products are often proprietary and can leave an enterprise vulnerable to
malicious attack, unreliability and poor performance. The SAFECode project states in
their white paper, Software Assurance: An Overview of Current Industry Best Practices
[8] that software assurance is especially important for organizations critical to public
safety and economic and national security. These users require a high level of confidence
that commercial software is as secure as possible. Coders must follow best practices for
secure software development.
6
The newscasts have been full of reports involving large Information Technology projects
that have been released but are not functionally successfully. These systems are
unreliable, insecure and slow. The user experience for these systems is less than ideal.
While these failures are financially costly to an organization and damaging to its
reputation, mission-critical systems cannot afford such errors. In his article, Detecting
Architecture Traps and Pitfalls in Safety-critical Software [11] Julien Delange states that
safety-critical avionics, aerospace, medical, and automotive systems are becoming
increasingly reliant on software. Malfunctions in these systems can have significant
consequences including mission failure and loss of life. These mission critical systems
must be designed, verified, and validated carefully to ensure that they comply with
system specifications and requirements. The must also be error-free.
Brownsword, Woody, Alberts and Moore developed the Assurance Modeling Framework
with the hopes of solving these problems by methodically evaluating an organization’s
operational assurance. Their term assurance ecosystem [1] is used to describe the broad
range of interrelated elements that influence operational assurance, including
organizations, decision makers, policies, practices, technologies, and people . The
framework identifies and evaluates assurance properties such as security, reliability and
performance, which can often be competing forces. The framework also identifies all
stakeholders and organizations, including those from external sources, involved with
these assurance solutions, mapping the dependencies and interrelationships between
them. This is what the project team calls the assurance ecosystem. Each assurance
7
property consists of a set of assurance capability areas. For example, vulnerability
management is a capability area for the security assurance property. To prove the
framework’s validity, the team applied to the framework to two assurance solutions
related to vulnerability management: Common Vulnerabilities and Exposures (CVE®)
and Common Weakness Enumeration (CWE™). In the pilot project, the team performed
the analysis of each view by applying the prescribed methods to CVE and CWE and
recording their observations of each step . The technical paper, A Framework for
Modeling the Software Assurance Ecosystem: Insights from the Software Assurance
Landscape Project [1] is the compilation of the teams work with this pilot.
2.2 The Assurance Modeling Framework and Pilot Project
The purpose of the SEI Software Assurance Framework is to elicit, analyze and organize
organizational assurance information and determine the interrelationships among
participating individuals and assurance solutions. The framework provides a way to look
across the assurance ecosystem and examine the gaps, barriers, and incentives that affect
the formation, adoption, and usage of assurance solutions. The technical paper, A
Framework for Modeling the Software Assurance Ecosystem: Insights from the Software
Assurance Landscape Project [1] describes the current version of the Assurance
Modeling Framework and the results of its pilot application to the assurance capability
area; vulnerability management. The team’s previous paper, Value Mapping and
Modeling SoS Assurance Technologies and Supply Chain [9] describes their initial work
with the Assurance Modeling Framework. A second paper, The Landscape of Software
8
Assurance—Participating Organizations and Technologies [10] describes their approach
in developing the framework and the reasoning behind their choices in developing the
framework’s structure.
During their work on the pilot, it became clear to the project team that most organizations
are true ecosystems and designing an assurance modeling framework for their evaluation
would be challenging. An organization’s assurance ecosystem includes stakeholders,
decision makers, policies, practices and technologies that can be large and complex. All
of these can be competing forces and analyzing their interrelationships can be a daunting
task. The pilot team implemented the modeling activities by managing the artifacts
manually. The framework begins by identifying an organization’s mission and the
assurance properties that support this mission. It then uses an incremental approach to
evaluate one assurance capability area at a time, eventually creating a profile for an
assurance property such as Security. The profile is a set of views that collectively
describe an assurance ecosystem for the selected capability area. From the analysis of the
profile, inefficiencies and improvements for an organization’s operational assurance can
be identified.
2.3 The EaseSAF Assurance Modeling Framework Steps
The Assurance Modeling Framework can be broken down into ten discrete steps based
upon the nine framework views and a profile summary [1]. The following is a brief
description of each step:
9
1. Define Principal Perspectives and Influences - provides the “big picture” and
scope of the assurance capability area under evaluation . It also characterizes the
organization including critical stakeholders and primary relationships.
2. Define Value Exchanges - identifies organizational relationships and assurance
solutions and the high-level value exchanged among them.
3. Evaluate Potential Assurance Results - defines the ways that assurance
solutions align with what operational users do to achieve operational assurance
results. Identifies gaps and inefficiencies.
4. Define Motivations - evaluates drivers that are critical to achieving operational
assurance objectives.
5. Identify Critical Behaviors - evaluates the relationships between organizations
and assurance solutions to identify influences that drive critical behaviors.
6. Evaluate Maturity in Adoption of Products - evaluates the maturation and
adoption mechanisms used for collections of related assurance solutions.
Provides an understanding of the current state of the formation and
implementation of an assurance solution.
7. Determine Future Drivers - evaluates future trends, influences, and uncertainties
that may affect new operational demands and assurance solutions. Investigates
future factors such as business and mission needs, technologies, economics and
environments.
8. Identify Inefficiencies - evaluates patterns of possible inefficiencies or gaps in
assurance solutions and in their adoption and usage.
10
9. Prioritize Improvements - after evaluating the inefficiencies, a list of potential
improvements can be created for the organizations adoption and use of the
assurance solution being evaluated.
10. Profile Summary – summarizes all previous step information for review and
reference.
The steps are interrelated. Information, models and analysis associated with one step is
used in conjunction with other steps. It is important for assurance analysts to remember
that creating a profile of a selected assurance capability area is only based on current
observation. To remain useful, it must be kept current.
2.4 The EaseSAF Project
The purpose of this project is to build a semi-automated tool to assist the assurance
analyst with the substantial amount of artifacts needed to record and analyze software
assurance information elicited through interviews and investigation required by the
Assurance Modeling Framework. This tool will support all aspects of the modeling
framework including the views, methods and activity categories. It will build the
assurance profile from this gathered information and summarize it in the Assurance
Profile Summary. It will assist the assurance analyst with archiving and organizing the
artifacts and diagrams needed for each method and view. With the effective compilation
of assurance information, an organization can successfully develop and adopt an
assurance solution into an operational setting. The EaseSAF artifacts are uploaded and
11
archived in the application folders so the analyst can easily maintain and review the
assurance profile information as a living artifact for future assurance needs.
EaseSAF was developed as a desktop application using Java Swing with NetBeans
Matisse technology for the presentation and application layers. The persistence layer was
built using a Derby database that binds seamlessly with Java Swing and Java 7/JSF.
Standard Microsoft windows directory folders are used for uploading, printing and
archiving the supporting assurance framework artifacts.
EaseSAF users are required to have a full understanding of the Assurance Modeling
Framework as the tool was not built to educate or guide the user through the framework
steps. There is no online help at this time. EaseSAF is designed to be flexible and
provide a good user experience so that analysts can perform the steps in the order they
prefer without restrictions. Steps may be performed simultaneously as interviews and
investigations often reveal information related to several of the methods and views. The
analyst can update, edit and save documents as needed throughout the modeling process.
This flexibility is useful given the complexity of the Assurance Modeling Framework.
12
Chapter 3
EASESAF ANALYSIS, ARCHITECTURE AND DESIGN
3.1 Functional and Non-functional Requirements
Applying SEI’s Assurance Modeling Framework can involve a large quantity of artifacts
especially for complex organizations with many stakeholders. The greater the number of
systems and individuals involved, the more information the framework must take into
account. Each assurance capability area is evaluated for the assurance solution and the
interrelationships with stakeholders and other assurance solutions must be taken into
account. The full framework may require hundreds of interviews, artifacts and diagrams.
For this reason, the semi-automation of the Assurance Modeling Framework will make
the process easier and more manageable. The following are a list of the functional and
nonfunctional requirements for EaseSAF.
Functional requirements for the EaseSAF tool:

EaseSAF provides support for all ten steps of the Assurance Modeling
Framework.

EaseSAF provides flexibility in the order for implementing each step of the
modeling process.

The EaseSAF allows the user to open, edit and save any step information at any
time.

Several steps require supporting artifacts. EaseSAF automates the uploading
and archiving of these diagrams, as they are needed. This allows the user to
13
create the diagrams with the best tool available, (Visio, etc.) and upload it to be
archived with the other modeling artifacts on one package.

EaseSAF automates the uploading, archiving and printing of all interviewing and
investigation documents.

EaseSAF provides a Profile Summary of Framework results as a living artifact.
The EaseSAF tool can reduce the manual processes and assist with the artifact
organization needed to apply this framework. EaseSAF will record interview notes,
modeling artifacts and manage all supporting documentation in one complete package.
Supporting documentation can be archived edited and printed with the standard Microsoft
Windows functionality.
Non-functional requirements for the EaseSAF tool:

Usability – EaseSAF provides user interface that is easy to use and has consistent
forms.

Re-usability – Java Object-Oriented classes can be used in other projects.

Modifiability – EaseSAF supports modification by abstracting and separating
layers.

Portability – EaseSAF can be deployed to any desktop supporting JDK 7
technology.

Extensibility –EaseSAF is built to support rapid additions and changes to the
application.
14
EaseSAF is built with nonfunctional requirements in mind. Extensibility, modifiability
portability and usability were the primary focus as modifications and improvements to
the Assurance Modeling Framework and the original EaseSAF tool are always possible.
With the JAVA Swing/Model-View-Controller architecture, portability and modifiability
are supported. EaseSAF is very UI-centric and was developed with the user experience
in mind. The SEI researchers are still making changes to several methods in the
Assurance Modeling Framework. For this reason, EaseSAF was built to allow
extensibility and re-use.
3.2 EaseSAF Architecture and Design
Originally, the EaseSAF tool was to be developed as a web-based product. However the
deeper I got into the functional requirements, the more I realized that this tool was very
UI-centric and many changes were going to be occurring with the tool and with SEI’s
Assurance Modeling Framework. The need to redesign the User Interface (UI) quickly
and easily was a high priority. I also considered usability as an important nonfunctional
requirement for this tool. Java Swing with its elaborate component library and drag-anddrop technology quickly and easily met my needs during analysis and development of
this product. NetBeans Matisse IDE technology supports Java Swing and allows quick
and easy form composition. Each Swing component has properties that control look-andfeel as well as content. Swing components also have event handlers to control listeners
and subsequent actions such as when a button is pushed. Swing also has the ability to
bind each component to the data layer. All these features can be handled directly through
15
the source code or by easily manipulating properties in the component properties window
of the IDE. Optimally, the tool should be written as a web-based, thin client running on
an application server with mobile technology support. I would recommend using JDK
7/JSF with an open source component library such as PrimeFaces to enhance the user
experience, add Ajax capability and include a mobile UI kit to allow the tool to be used
as a mobile web application for handheld devices. PrimeFaces components support
responsive design to re-render for any device size. I will leave this for future developers.
Like most software systems either desktop or web-based, EaseSAF is a collection of
components that have relationships that interact with each other to accomplish a specific
functionality. In order to organize these components and effectively model their behavior,
a software system architecture is necessary. A multi-tier architecture with the ModelView-Controller design pattern proved optimal for EaseSAF.
Figure 1. EaseSAF Architecture
16
As shown in Figure 1 there are three basic components for the system:

View: This layer includes the Swing components and frames that compose the
forms displayed for users who enter input and/or send and receive commands
through this layer.

Controller: This layer contains the event logic of the application that controls
the view based on specified user actions such as click-events.

Model: This layer includes the data source, usually the database itself and the
database management system. Swing components bind with the data layer using
the built in binding properties. In EaseSAF, the database is Derby, which
seamlessly interfaces with the components in Java.
One of the advantages in a multi-tiered application is that users are able to work on the
application data without knowing at build time where the data is stored. To allow this
level of abstraction, the Swing components interact with the database and move data to
and from all EaseSAF forms. Another advantage is the modularity of the application’s
components with loose coupling and high cohesion characteristics, allowing this
architecture better modifiability and extensibility for the application in the future. Swing
is easy to understand, read and re-use. The multi-layer architecture also supports a robust
application. Changes made in one layer, do not affect other layers and can remain
independent.
17
Several technologies could have been selected for this application. Java Swing was
selected because it is very UI-centric and allowed easy manipulation of component
properties for data binding and event handling. The Java Swing framework is based upon
the Model-View-Contoller (MVC) design pattern that consists of the following:

The model, which stores the content

The view, which presents the content

The controller, which handles events and binding
The MVC pattern has the advantage of separating the presentation layer (view),
application layer (control) and data layer (model) so that changes to each layer can occur
independently without necessarily changing the other layers. A new presentation layer
can be written with a new UI application such as JSF, without affecting the application
layer or the data layer. This encourages extensibility and reuse. Swing developers can
reuse the code for their models and can even switch the look-and-feel in a running
program. MVC can have multiple views so that when one view is updated through the
controller, the other views automatically refresh themselves [17].
The MVC pattern is attractive to Swing developers because it allows them to implement
pluggable, look and feel, applications. The Swing framework is composed of UI
components and their properties. Each UI component consists of a wrapper class, such as
JButton or JTextField, that stores the model and the view.
Each component has three characteristics:
18

Its content or state, such as whether a button has been pressed or text is in a
textbox.

Its visual appearance, color or size, etc.

Its behavior, such as reactions to events.
The NetBeans IDE with Matisse technology enables the developer to construct a Java
Swing application quickly and easily. The IDE allows the developer to drag and drop the
components for the view from the palette shown on the upper right panel of Figure 2 and
set the controller events using the properties shown in the lower right panel of the IDE.
Binding to the data layer (model) occurs in the properties panel as well. The Swing
developer can switch to coding custom source directly by changing the view from Design
to Source in the middle pane. Business logic and additional event handling can be added
to the application layer by entering custom code in the source view. The middle pane
also has a History view that acts as a version control manager displaying each iteration of
the form. A developer can enter a brief description of the current version or recover a
previous version of an application by deleting the current version.
The data layer contains the information that the user enters in the view or presentation
layer. EaseSAF uses a derby database to support the data layer. Derby is a simple,
elegant database that seamlessly integrates with the Swing framework and the NetBeans
IDE. Once the database is built and connected to the application through the JDBC, the
data components in the Swing application can be bound to the database through the
properties window in the lower panel of the NetBeans IDE as shown in Figure 2.
19
Figure 2. NetBeans IDE with EaseSAF Application
3.3 EaseSAF User Interface Design
In this section the User Interface (UI) is described for the EaseSAF tool. Screen shots,
user instructions and a description of each form are provided. EaseSAF is composed of a
series of forms that guide the user through the Assurance Modeling Framework steps.
The forms consists of components that the user can enter information for the selected
step. The EaseSAF Screen Flow is shown in Figure 3 and the EaseSAF forms are shown
in Figures 4 – 19.
20
Save
Principle
Perspectives and
Influences Form
Nine Step/Cancel
Save
Nine Step/Cancel
Value Exchange
Form
Cancel
Open Existing
Profile Form
Cancel
Nine Step Cancel
Open
Save
Save
Nine Step/Cancel
Nine Step Form
Save
Welcome Form
Create
Potential
Assurance Results
Form
Nine Step Cancel
Create New Profile
Form
Save
Cancel
Summary
Cancel
Motivations Form
Nine Step/ Cancel
Save
Cancel
Profile Summary
Form
Nine Step/Cancel
Critical Behaviors
Form
Return
Open/Cancel
Nine Step/Cancel
Save
Upload
Nine Step/Cancel
Adoption of
Products Form
Save
Future Drivers
Form
Save
Inefficiencies Form
Save
Prioritized
Improvements
Form
Figure 3. EaseSAF Screen Flow Diagram
File Upload
21
3.3.1 EaseSAF Welcome Form
Figure 4. EaseSAF Welcome Form
Figure 4 shows the EaseSAF Welcome Form is the landing form for the user as they
bring up the application. It welcomes the user and displays the options to proceed with
the Assurance Modeling Framework. The user either creates a new profile or opens an
existing profile by clicking on the appropriate radio button and pressing Select. Pressing
Cancel exits the EaseSAF application. According to the framework, a profile is based
upon a selected assurance capability area that is to be evaluated.
22
3.3.2 EaseSAF Create New Profile Form
Figure 5. EaseSAF Create New Profile Form
Figure 5 shows the Create New Profile Form. The user enters the profile name and a
brief description of the profile. All subsequent work on this profile is consolidated with
the profile, similar to a project in other applications. To save the profile information the
user presses save and proceeds to the Nine Step Form. If the user presses cancel, the
profile information will not be saved and the user returns to the EaseSAF Welcome
screen.
23
3.3.3 EaseSAF Open Existing Profile Form
Figure 6. EaseSAF Open Existing Profile Form
Figure 6 shows the EaseSAF Open Existing Profile Form. The drop-down choices are
the existing profiles that are in the system, (in our example, there are no existing
profiles). The user selects the profile to open and edit and proceed to the Nine Step Form
. There is also an option to remove the profile. If a profile is removed and save is
pressed, there is not a way to recover the profile in the future. If the user presses cancel,
the profile selected is neither opened, nor removed and the user will return to the
EaseSAF Welcome Form.
24
3.3.4 EaseSAF Nine Step Form
Figure 7. EaseSAF Nine Step Form
The Nine Step Form is shown in Figure 7 and is the main driver of the Assurance
Modeling Framework Process. The user returns here after each step in the modeling
process and uses this form to select the next step. The user updates Stakeholder,
Assurance Property and Assurance Capability Area information at any time. The Profile
Name is static and comes from the Create/Open Profile form. Pressing the Save key
saves the entered information. Selecting the Framework Step takes the user to the next
step in the modeling process that was selected. Clicking the Profile Summary button
takes the user to the Profile Summary, even if all the steps do not have Framework
information entered.
25
3.3.5 Profile Summary Form
Figure 8. EaseSAF Profile Summary Form
Figure 8 shows an example of the Profile Summary Form that is a viewable report about
the Assurance Solution Profile. The report reflects the information entered on each of the
forms of the EaseSAF modeling framework.
26
3.3.6 Principle Perspectives and Influences Form
Figure 9. Principle Perspectives and Influences Form
Figure 9 shows the Principle Perspectives and Influences Form which is the first of the
nine steps in the Assurance Modeling process. The activity category Determine Context
and Scope, is associated with the Principal Perspectives and Influences view which
provides the big picture and general scope of the organization. The user arrives on this
form by selecting it on the Nine Step Form. The Principle Perspectives and Influences
Form is composed of three tabs; one for organizational information, one for supply-side
information and one for demand-side information.
27
Organization tab:
Figure 10. Principle Perspectives and Influences Form – Organization Tab
Figure 10 shows the Organizational tab. Answers to the view questions are entered and
clicking Save will save all information entered on each of the three tabs and keep the user
on the Principle Perspectives and Influences form. The user remains on this form or
presses the Return to Nine Step Form and returns to select the next step. Cancel will not
save any information on any of the three tabs and will return the user to the Nine Step
Form. The Upload Artifacts button brings up the upload artifacts form to upload any
supporting documentation to a windows folder pre-designated by the user.
Supply Side tab:
Figure 11. Principle Perspectives and Influences Form – Supply Side Tab
28
Figure 11 shows the Supply side tab where information that relates to individuals or
organizations that supply the assurance solution being evaluated and their motivations for
providing the assurance solution is entered. To save, cancel or upload artifacts, the user
must return to the Organization tab. The user enters supply-side information on this tab.
Demand Side tab:
Figure 12. Principle Perspectives and Influences Form – Demand Side Tab
Figure 12 shows the Demand Side Tab. The user enters demand-side information on this
tab. Demand side information relates to those individuals or organizations that depend on
suppliers for the assurance solution being evaluated and their motivation for wanting the
assurance solution. To save, cancel or upload artifacts, the user must return to the
Organization tab.
29
3.3.7 Value Exchange Form
Figure 13. Value Exchange Form
After the domain and stakeholders are identified in the Principle Perspectives and
Influences View, the next step is the Value Exchanged View. Figure 13 shows the Value
Exchanged view that provides a detailed understanding of participating organizations,
assurance solutions, and relationships. The user arrives at this form by selecting it on the
Nine Step Form. The Value Exchange diagrams are uploaded by clicking the Upload
Artifacts button. Additional View information is entered in the text fields shown on this
form. Clicking Save saves the information and keeps the user on this form. The user
may then press the Return to Nine Step Form button to return to the Nine Step Form and
30
select the next step. Pressing the Cancel button clears entered information and does not
save it.
3.3.8 Potential Assurance Results Form
Figure 14. Potential Assurance Results Form
Figure 14 shows the Potential Assurance Results Form. This view provides a detailed
understanding of the current state of participating organizations, assurance solutions, and
the relationships within the assurance ecosystem. The user arrives at this form by
selecting it on the Nine Step Form. The Summary of Roles and Responsibilities
Alignment Model and the the Sos Focus Analysis Alignment Model are uploaded by
clicking the Upload Artifacts button. Additional view information is entered in the text
fields shown on this form. Clicking Save saves the information and keeps the user on
31
this form. The user may then press the Return to Nine Step Form button to select the
next step. Pressing the Cancel button clears entered information and does not save it.
3.3.9 Motivations Form
Figure 15. Motivations Form
An essential step in evaluating operational assurance is to identify the motivations that
influence whether participants will support an assurance capability area. This view will
reveal what is currently working well, identify gaps and inefficiencies and improvements
by identifying drivers of success for a given assurance capability area. Figure 15 shows
the Motivations Form. The user arrives at this form by selecting it on the Nine Step
Form. The Driver Identification and Analysis Method artifacts are uploaded by clicking
the Upload Artifacts button. The user will also enter the additional view information in
32
the text fields shown on this form. Clicking Save saves the information and keeps the
user on this form. The user may then press the Return to Nine Step Form button to select
the next step. Pressing the Cancel button clears entered information and does not save it.
3.3.10 Critical Behaviors Form
Figure 16. Critical Behaviors Form
Figure 16 shows the Critical Behaviors form which provides a more detailed
understanding of an organization’s current state by understanding the critical behaviors of
participating groups that work together to ensure that assurance solutions are
implemented and used successfully. Once the critical behaviors are understood, patterns
of inefficiency can be identified. The user arrives at this form by selecting it on the Nine
Step Form. The System Dynamics models can be created using third party system
dynamics modelers such as Anylogic, Goldsim, Berkely Madonna, Sysdea, iThink®
33
,STELLA®
and SimGua.
These artifacts are uploaded by clicking the Upload Artifacts
button. Additional view information is entered in the text fields shown on this form.
Clicking Save saves the entered information. Cancel and Return to Nine Steps Form
buttons behave as described on previous forms.
3.3.11 Adoption of Products Form
Figure 17. Adoption of Products Form
Figure 17 shows the Adoption of Products Form that evaluates the current maturation of
the solution and the participants involved by identifying the maturation mechanisms used.
The user arrives at this form by selecting it on the Nine Step Form. The majority of
information in this step comes from Jolly Model artifacts. These artifacts are uploaded
by clicking the Upload Artifacts button. The user will enter the additional view
34
information in the text fields shown on this form. Clicking Save saves the information
and keeps the user on this form. The user may then press the Return to Nine Step Form
button to select the next step. Pressing the Cancel button clears entered information and
does not save it.
3.3.12 Future Drivers Form
Figure 18. Future Drivers Form
Figure 18 shows the Future Drivers form which provides an understanding of potential
future factors and their impact on assurance solutions and participating organizations by
identifying future trends and influences that impact technologies and assurance solutions.
35
The user arrives at this form by selecting it on the Nine Step Form. The majority of
information in this step comes from Strategic Alternatives Analysis artifacts. These
artifacts are uploaded by clicking the Upload Artifacts button. Additional view
information is entered in the text fields shown on this form. Clicking Save saves the
information. Pressing the Return to Nine Step Form button returns the user to the Nine
Step Form. Pressing the Cancel button clears entered information and does not save it.
3.3.13 Inefficiencies Form
Figure 19. Inefficiencies Form
Figure 19 shows the Inefficiencies Form. The final steps of the modeling framework
involve evaluating inefficiencies and potential improvements for the assurance solution
being evaluated. The form is used to enter inefficiencies that have been identified by the
36
analysts. The user arrives at this form by selecting it on the Nine Step Form. There are
no artifacts related to this method. The user will enter the view information in the text
fields shown on this form. Clicking Save saves the information and keeps the user on
this form. Pressing the Return to Nine Step Form button returns the user to the Nine Step
Form. Pressing the Cancel button clears entered information and does not save it.
3.3.14 Prioritized Improvements Form
Figure 20. Prioritized Improvements Form
Figure 20 shows the Prioritized Improvements Form. After evaluating the inefficiencies,
potential improvements can be identified for the organizations adoption and use of the
assurance solution being evaluated. It is important for assurance analysts to remember
that creating a profile of a selected assurance capability area is only based on current
37
observation. To remain useful, it must be kept current. The EaseSAF profile is a living
artifact that analysts can reevaluate and update over time. The user arrives at this form
by selecting it on the Nine Step Form. There are no artifacts related to this method. The
user will enter the view information in the text fields shown on this form. Clicking Save
saves the information. Pressing the Return to Nine Step Form button returns the user to
the Nine Step Form. Pressing the Cancel button clears entered information and does not
save it.
3.3.15 EaseSAF Upload Artifacts Form
Figure 21. Upload Artifacts Form
Figure 21 shows the Upload Artifacts Form. This form uploads artifacts from each of the
steps to be archived with the profile all in one package. The user arrives at this form by
pressing the Upload Artifacts button on any of the other forms. The user will browse and
38
select the windows file that contains the artifacts, and click Open. Clicking Cancel unselects the file and keeps the user on this form. The user may press the Return to button
to return to the previous step. The file is placed in the selected Windows directory folder
that the user has set up for this purpose. Pressing Return then returns the user to the
previous form. Artifacts can be edited, copied, printed and resaved with normal
Microsoft Windows technology.
3.4 EaseSAF Database Design
The database layer is the layer that holds the data entered by the users. For EaseSAF,
Derby was selected as the database as it is easy to build and binds seamlessly with both
Java Swing components and JSF. EaseSAF has many tables that support the
functionality of the tool. The Swing components are bound to the data tables and support
the exchange of data between the presentation and data layer. Binding of the Swing to
the data elements is done through the binding properties in the IDE. The only form
requiring a customized SQL query is the Profile Summary Form that joins information
from several tables to be displayed in the Profile Summary Form. Table and column
names in the database were selected to identify functionality. The primary keys are
represented with the PK symbol. There are no foreign keys at this time. Relational
cardinality is represented at the ends of the connecting lines. The database tables and
relations are represented in the EaseSAF Entity Relationship diagram shown in Figure
22.
39
1
profile
PK
pID
pName
pDesc
pStakeholder
pProperty
1
*
capability_area
PK
cID
pp_and_i
cChange
1
PK
value_exchange
PK
ppiDef
ppiParticipants
ppiProvide
ppiInvolve
ppiConstraints
ppiGoverns
ppiNeeds
ppiInvolved
ppiGroups
ppiDrive
vID
vOrgs
vElements
pa_results
PK
ppiID
paID
paCooperate
critical_behaviors
PK
motivations
PK
cbID
cbScenarios
mID
mMotivations
mNotAchieve
future_drivers
PK
adoption_of_products
PK
fID
fImpact
fEvolve
fList
aopID
aopMechanisms
aopContext
prioritized_improvements
inefficiencies
PK
PK
piID
iID
iPatterns
Figure 22. EaseSAF Entity Relationship Diagram
piCandidates
piImpact
40
Chapter 4
EASESAF IMPLEMENTATION AND EXAMPLE
EaseSAF was implemented using an iterative life cycle. Each iteration consisted of a
new set of requirements and was selected based upon its importance to the project and/or
its contribution to discovery. Having never used this application before, the more I
learned about Java Swing, the more functionality I was able to attempt. This approach
was successful as it helped with managing and planning smaller project goals. With each
iteration, I became more adept with Java Swing and I could focus on the actual
functionality that I was trying to achieve.
In 2010, Lisa Brownsword, Carol C. Woody, Ph.D, Christopher J. Alberts, Andrew P.
Moore piloted the Assurance Modeling Framework and documented their results in their
technical paper, A Framework for Modeling the Software Assurance Ecosystem: Insights
from the Software Assurance Landscape Project. They executed their pilot using the
Assurance Capability area, Vulnerability Management as an example for modeling.
Three assurance solutions related to vulnerability management: Common Vulnerabilities
and Exposures (CVE®) and Common Weakness Enumeration (CWE™) were also
included as examples along with CERT Secure Coding Standards. In this chapter, the
EaseSAF desktop tool is used to prove its efficiency in automating the Assurance
Modeling framework. All the data is taken from the
case study mentioned above and entered into EaseSAF to demonstrate the tools ability to
manage the nine step process and save the data in the database. The following pages will
41
describe using EaseSAF for the assurance modeling process with a detailed description of
how each step satisfied SEI’s framework requirements.
When EaseSAF is launched, the Welcome form in Figure 23 is displayed. For our
example, the radio button for Create New Profile is selected.
Figure 23. Create a New Profile
42
The next step is to enter the new profile information as seen in Figure 24. For our
example, Common Vulnerabilities and Exposures (CVE) profile information is entered.
The save button is then clicked and the profile information is perisisted in the data layer.
Figure 24. Vulnerability Management Profile Description
43
The Profile Name is populated from the Create New Profile Form shown in Figure 25.
The Stakeholders and the Assurance Property for CVE is entered and the Save button is
pressed. The user selects the Principle Perspectives and Influences View, which is the
first step in the Assurance Modeling Framework and presses Go to display the Principle
Perspectives and Influences form.
Figure 25. Vulnerability Management and CVE
44
The Principle Perspectives and Influences form is where Organization, Supply Side and
Demand Side information for CVE is entered by clicking on each tab and pressing Save
on the Organization tab. The user elicits this information using the Critical Context
Method and the interview template developed by SEI. This method describes the scope
and “big picture” of the organization and identifies key stakeholders that supply CVE
solutions and consume or use CVE solutions. The Upload Artifacts button is used to
upload and save these artifacts in pre-designated windows folders. When entering the
information for CVE is completed and saved on all three tabs, the Return to Nine Step
Form is clicked. Figure 26 shows the information entered on the Organization Tab for
CVE.
Figure 26. Organization Tab for CVE
45
Figure 27 shows how CVE supplier information is entered on the Supply Side tab.
Figure 27. Supply Side Tab for CVE
46
CVE consumer information is entered on the Demand Side tab as shown in Figure 28.
Figure 28. Demand Side Tab for CVE
47
The next step in the Assurance Modeling Framework is the Value Exchange View as
shown in Figure 29. This form is displayed when it is selected on the Nine Step Form.
The participants and items of value identified in the Value Map are listed here. This
information is elicited through the Value Mapping method developed by SEI. Value
Mapping provides a visual representation of the interactions between participating
organizations and CVE. The Value Maps and any other supporting documentation are
uploaded by clicking the Upload Artifacts button and saving to a pre-designated windows
folder.
Figure 29. Value Exchange information for CVE
48
The next step in the Assurance Modeling Framework is the Potential Assurance Results
View as shown in Figure 30. This form is displayed when it is selected on the Nine Step
Form. The information for this form is elicited through the SoS Focus Analysis method
developed by SEI. The resulting list of ways that participants and assurance solutions
work together to build, install and support CVE is entered on this form and saved. The
Summary of Roles and Responsibilities Alignment Model and the the Sos Focus
Analysis Alignment Model artifacts and any other supporting documentation are
uploaded by clicking the Upload Artifacts button and saving to a pre-designated windows
folder.
Figure 30. Potential Assurance Results information for CVE
49
Figure 31 shows the next step in the Assurance Modeling Framework, the Motivations
View. This form is displayed when it is selected on the Nine Step Form. This step allows
stakeholders to see what is currently working well, identify gaps, inefficiencies and
improvements for CVE. This information is elicited through the Driver Identification
and Analysis method developed by SEI. More detailed information about this method
can be found in A Framework for Categorizing Key Drivers of Risk[12]. The resulting
information about key drivers for CVE are entered on this form and saved. The list of
Driver Attributes and Candidate Drivers of Vulnerability Management list and any other
supporting documentation are uploaded by clicking the Upload Artifacts button and
saving to a pre-designated windows folder.
Figure 31. Key Drivers of Risk for CVE
50
The next step in the Assurance Modeling Framework is the Critical Behaviors View
shown in Figure 32. This form is displayed when it is selected on the Nine Step Form.
This step assists the analyst in understanding the critical behaviors of participating group
in their use of CVE. This information is elicited through the System Dynamics method.
The System Dynamics method helps analysts model and analyze critical behaviors
towards CVE as they evolve over time. More information about this method can be
found in Business Dynamics: Systems Thinking and Modeling for a Complex World [13]
and Thinking in Systems – A Primer [14]. The resulting information about the critical
usage scenarios for CVE are entered on this form and saved. The System Dynamics
diagrams and any other supporting documentation are uploaded by clicking the Upload
Artifacts button and saving to a pre-designated windows folder.
Figure 32. Critical Usage Scenarios for CVE
51
The next step in the Assurance Modeling Framework is the Adoption of Products View
shown in Figure 33. This form is displayed when it is selected on the Nine Step Form.
This step assists the analyst in understanding the current state of maturation and use of
collections of related assurance solutions within the organization. This information is
elicited through the Technology Development and Transition method still under
development by SEI. In the interim, SEI has selected the Jolly model for singletechnology maturation evaluation. The resulting information about the adoption and
usage of CVE are entered on this form and saved. The Jolly model diagrams and any
other supporting documentation are uploaded by clicking the Upload Artifacts button and
saving to a pre-designated windows folder.
Figure 33. Operational Adoption and Usage of CVE
52
The next step in the Assurance Modeling Framework is the Future Drivers View shown
in Figure 34. This form is displayed when it is selected on the Nine Step Form. This step
assists the analyst in understanding the potential future factors and their impact on
assurance solutions and participating organizations. This information is elicited through
the Strategic Alternatives Analysis method developed by SEI. More information about
this method can be found in Scenarios: The Art of Strategic Conversation [15] and
Powerful Times: Rising to the Challenge of Our Uncertain World [16]. The resulting
information about future impacts to CVE is entered on this form and saved. The Axes of
Uncertainty and any other supporting documentation are uploaded by clicking the Upload
Artifacts button and saving to a pre-designated windows folder.
Figure 34. Future Impacts for CVE
53
The next step in the Assurance Modeling Framework is the Inefficiencies View shown in
Figure 35. This form is displayed when it is selected on the Nine Step Form. This step
involves no formalized method. The information is elicited from review the preceding
steps in the framework and brainstorming inefficiencies in the organization’s adoption
and use of CVE. The resulting list of inefficiencies is entered on this form and saved.
There is no supporting documentation.
Figure 35. List of Inefficiencies for CVE
54
Once the inefficiencies for CVE have been identified, prioritized improvements can be
listed on the form shown in Figure 36. The Prioritized Improvements form is displayed
when it is selected on the Nine Step Form. This step involves no formalized method. The
information is elicited from review the preceding steps in the framework and
brainstorming improvements to the organization’s adoption and use of CVE. The
resulting list of improvements are prioritized and listed on this form and saved. There is
no supporting documentation. The resulting Profile Summary report for CVE can be
viewed in figure 7.
Figure 36. List of Prioritized Improvements for CVE
55
Chapter 5
CONCLUSION AND FUTURE WORK
Assurance solution analysts must have a firm understanding about an organization’s
assurance ecosystem to assist and guide those who fund assurance solutions. Analyzing
an organization’s ability to adopt assurance solutions is critical to this process.
Determining where resources should be applied and identifying critical gaps in assurance
solutions are key objectives for organizations. The EaseSAF project was a semiautomation of SEI’s Assurance Solution Framework built to assist assurance analysts
with the lengthy process of completing the framework requirements and managing the
many artifacts involved. Many improvements can be made to this tool.

EaseSAF should be web-based with a responsive design set of components that
can render for both mobile and desktop

The Adoption of Products step’s use of the Jolly Model method will need to be
changed when SEI has finished developing the Technology Development and
Transition Analysis method

EaseSAF should guide the analyst through the framework step, constraining steps
to be executed in the correct order.

The Profile Summary will need print functionality

Assurance Modeling Framework and EaseSAF Page level Help functionality
should be included for user assistance.
56
I encourage future developers to improve upon this tool to help achieve organizational
software and system assurance.
57
REFERENCES
[1] Lisa Brownsword, Carol C. Woody, Ph.D, Christopher J. Alberts, Andrew P. Moore.
A Framework for Modeling the Software Assurance Ecosystem: Insights from the
Software Assurance Landscape Project, Software Engineering Institute, Technical Report
CMU/SEI-2010-TR-028
(2010,
August).
[Online].
Available:
http://www.sei.cmu.edu/reports/10tr028.pdf
[2] J. Heffley and P. Meunier, Can Source Code Auditing Software Identify Common
Vulnerabilities and Be Used to Evaluate Software Security? Proceedings of the 37th
Annual Hawaii International Conference on System Sciences (HICSS–04), Track 9,
Volume 9, IEEE Computer Society, January 2004.
[3] The CERT Secure Coding Analysis Laboratory, SCALe (2014). SCALe home form.
Retrieved April 4, 2014, from CERT website:
https://www.cert.org/secure-coding/products-services/scale.cfm
[4] The Web Application Security Project, OWASP (2014). Category:OWASP Top Ten
Application Security Risks - 2013. Retrieved April 4, 2014, from OWASP Web site:
https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project#tab=OWASP_T
op_10_for_2013
58
[5] Matthew Collins, Dawn M. Cappelli, Tom Caron, Randall F. Trzeciak, Andrew P.
Moore. Spotlight On: Programmers as Malicious Insiders–Updated and Revised,
Software Engineering Institute, White Paper (2013, December).
[Online].
Available:
http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=72825
[6] Lisa George Silowash, Dawn Cappelli, Andrew Moore, Randall Trzeciak, Timothy J.
Shimeall, Lori Flynn. Common Sense Guide to Mitigating Insider Threats 4th Edition,
Software Engineering Institute, Technical Report CMU/SEI-2012-TR-012 (2012,
December).
[Online].
Available:
http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=34017
[7] The Web Application Security Consortium, WASC (2010, January 10). Web
Application Security Statistics . Retrieved April 8, 2011, from The Web Application
Security Consortium: http://projects.webappsec.org/w/page-revisions/13246989/WebApplication-Security-Statistics
[8] Software Assurance: An Overview of Current Industry Best Practices, SAFECode
Software Assurance Forum for Excellence in Code, Technical Paper, Executive Summary
(2008,
February).
[Online].
Available:
http://www.safecode.org/publications/SAFECode_BestPractices0208.pdf
[9] Siviy J.M., Alberts C., Moore A.P. , Carol C. Woody, Ph.D, Value Mapping and
Modeling SoS Assurance Technologies and Supply Chain, Systems Conference, 2009
59
3rd Annual IEEE , Technical Report 978-1-4244-3463-3 (March 2009).
[Online].
Available:
http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=34017
[10] Carol C. Woody, PhD, Lisa Brownsword, Christopher J. Alberts, Andrew P. Moore.
The Landscape of Software Assurance—Participating Organizations and Technologies, ,
Software Engineering Institute, Technical Report.
(2009, April).
[Online].
Available:
http://enu.kz/repository/2009/AIAA-2009-1919.pdf
[11] Julien Delange, Detecting Architecture Traps and Pitfalls in Safety-Critical
Software,
SEI Blog (December 2013).
[Online].
Available:
http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=34017
[12] Christopher J. Alberts, Audrey J. Dorofee, A Framework for Categorizing Key
Drivers of Risk, Software Engineering Institute, Technical Report.
(2009, April).
[Online].
Available:
http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=9093
[13] John D. Sternman, Business Dynamics: Systems Thinking and Modeling for a
Complex World , McGraw-Hill/Irwin (February 23, 2000)
60
[14] Donella H. Meadows, Thinking in Systems - A Primer
EarthScan (2008)
[Online].
Available:
http://www.ess.inpe.br/courses/lib/exe/fetch.php?media=wiki:user:andre.zopelari:thinkin
g_in_systems_a_primer.pdf
[15] Van der Heijden, Kees. Scenarios: The Art of Strategic Conversation, 2nd edition.
Wiley, 2005.
[16] Kelly, E. Powerful Times: Rising to the Challenge of Our Uncertain World. Wharton
School, 2005.
[17] Cay X. Horstmann, Gary Cornell , Core Java Volume I – Fundamentals, Ninth
Edition,
Prentise Hall, 2013.