PhD Research Proposal
PhD Research Proposal
Methodology Selection for the Engineering of Complex
Defence Systems
Mr. John Smith
School of Physics & Electrical Engineering
Systems Engineering & Evaluation Centre
Mawson Lakes Campus
University of South Australia
Mawson Lakes, SA – 5095
12th March 2013
Table of Contents
1.
2.
INTRODUCTION .............................................................................................................. - 3 Overview of Defence Systems Engineering ....................................................................... - 4 -
John Smith
-1-
12-Mar-2013
PhD Research Proposal
2.1.
What is Systems Engineering? ................................................................................... - 5 2.2.
A Typology for Systems Engineering ........................................................................ - 8 2.3.
Characteristics of Complexity in Defence Systems ................................................. - 10 2.4.
Existing Defence Systems Engineering Models ...................................................... - 11 2.5.
Multi-Methodological Approaches in Management Science ................................... - 13 2.6.
Problems in Modern Defence Systems Engineering ................................................ - 14 3. Research Problem ............................................................................................................. - 16 3.1.
ISO 15288 Life Cycle processes .............................................................................. - 18 3.2.
A Typology of Problems .......................................................................................... - 19 3.3.
A Typology of Systems............................................................................................ - 20 3.4.
Attributes for selection of systems (engineering) processes and practices .............. - 20 3.5.
Expected Contribution ............................................................................................. - 20 4. Research Methodology ..................................................................................................... - 22 4.1.
Literature Review..................................................................................................... - 22 4.2.
Methodology ............................................................................................................ - 22 4.3.
Case Studies ............................................................................................................. - 23 4.4.
Publications .............................................................................................................. - 23 5. Work Plan ......................................................................................................................... - 24 6. References......................................................................................................................... - 25 -
John Smith
-2-
12-Mar-2013
PhD Research Proposal
1.
INTRODUCTION
The nature of problems encountered in the system engineering domain for large and
novel Defence systems has over the past thirty years grown in complexity. The
integration of technology with human activity systems to produce complex capabilities
has required a continual change in our definition and application of systems engineering.
The acquisition of defence capability has transitioned from a mindset in the late 1960s of
equipment procurement to one associated with capability development. This change in
mindset has been driven partly by the growth of information technology and its impact on
technology based systems. Defence equipment has moved from being predominantly
hardware based to being multi-function, multi-role systems where the flexibility and
adaptability of the system is primarily derived from its software functionality.
Similarly, the global environment within which nations need to develop and project a
defence capability has also changed. Decades ago, nations would examine their short and
long term threat environments and based on such assessments, plan the defence force
structure most suited to meet such adversities. For Australia, the Defence White Paper
would set the direction of Australia’s defence force structure. It has been one of
protecting national interests, involvement in the local region and establishment and
maintenance of international alliances.
However, the threats facing defence forces are now quite different. Global terrorism,
support to Coalition forces and UN Peace keeping missions have dramatically changed
the demands placed on a defence capability to support these new missions and roles.
Today, defence forces may be called upon to be deployed in any part of the world with
little warning, undertaking missions that were not envisioned a few years ago and
consequently using defence capability that was not designed for such purposes.
Modern Defence systems need to be able to be integrated [in a “plug and play”
manner] into hybrid system of systems to address short term national and international
demands and on completion of such missions be disassembled to return to their prior
roles. These hybrid system of systems integrate human activity systems with advanced
technology [sociotechnical systems] to produce complex defence capabilities. More
recently, the move by Western countries towards embracing net centric warfare concepts
such as distributed functionality and composeability imply that system components that
may have been designed to be platform centric now need to operate in a network centric
manner – hence the metaphor of “plug and play”. This has introduced further complexity
in the development of defence systems.
This research aims to examine how does one design such complex systems to meet
demands that are not envisioned at the time of development and be able to integrate with
other capabilities not contemplated during development. Such is the challenge to modern
defence system engineering.
John Smith
-3-
12-Mar-2013
PhD Research Proposal
In this research proposal, we begin with an introduction to put in context the changing
demands on system engineering over the past few decades, a short description of the
status quo of systems engineering followed by a discussion of the research problem. Then
we discuss the approach used to undertake the research, followed by a work plan and
schedule.
2.
Overview of Defence Systems Engineering
Systems engineering gained recognition as a discipline following the Second World
War. The increasing cost and technical complexity of development and acquisition
programs in the 1950s and 1960s stimulated the need for a methodology to handle the
technical and managerial complexity of large projects. Some of this recognition was no
doubt due to large program failures that could have been avoided, or at least mitigated
through the use of systems engineering (M’Pherson, 1980; DSMC, 1983).
The development of defence programs such as the Polaris submarine and the Apollo
Space program in the 1960s saw the need to establish processes and procedures to assist
in the development of such programs. Standards began to be developed. Whilst the
problems encountered were amenable to such approaches, system engineering gained
acceptance as a valuable component in the success of large projects.
The 70’s saw the introduction of software as a major component of large defence
systems. Whilst software development in itself was to become a major challenge,
software changed the nature of equipment development. Prior to the introduction of
software based systems, equipment was designed for a set purpose and the hardware
manufactured to such an end goal. With the advent of software based systems, defence
systems could now exhibit multiple functions and roles – all reconfigurable through the
system software. System complexity grew. Software based defence systems encountered
significant difficulties in estimation and development control. Systems engineering’s
functionalist approach began to falter as developers struggled to grasp with problems that
were unprecedented and with ill-defined operational concepts leading to scope and cost
overruns and dissatisfied end users.
More recently, Defence has had to grapple with the need to response quickly to
situations on a global perspective with equipment not designed to meet such occurrences.
Modern day defence systems need to be able to be integrated into mission specific system
of systems, have network centric and not platform centric characteristics and display
robustness and agility within such an environment. Furthermore, these systems now
comprise advanced technology that is integrated with human activity systems
[organisations] to achieve complex capabilities as may be seen in modern C4ISR
systems.
Contemporary systems engineering, while cognisant of these issues, is still practiced
largely as a process-based, equipment-centric discipline based on traditional or classical
systems engineering process models. This traditional systems engineering approach
tends to be goal oriented and based on a belief system that the application of natural
John Smith
-4-
12-Mar-2013
PhD Research Proposal
scientific principles to the problem at hand will provide a solution. In line with this
concept, the problem solving method tends to be reductionist in that the belief is that the
problem can be decomposed to smaller components until each component is amenable to
scientific analysis and solution. Then the components are re-assembled to represent the
whole
However, today’s complex System of Systems (SOS) requires a multi-disciplinary
approach applied across their life cycle development and management. This implies that
a pluralist approach using multiple system methodologies, based on different social
models, is required to adequately address the scope of the problem. Mingers & Gill
(1997, p8-9) argue that real world problem situations are inevitably highly complex and
multidimensional. Different paradigms (or social models) each focus attention on
different aspects of the situation and so multimethodology is necessary to deal effectively
with the full richness of the real world.
What is needed is to apply systems engineering to the range of problems identified by
Cook (2004), in a framework that engages the many disciplines involved during the
different lifecycle phases of system of interest albeit simple equipment or the defence
force as a whole.
2.1. What is Systems Engineering?
It is worthwhile to examine some definitions of Systems Engineering to observe the
changing emphasis in the definitions over time to address the need to recognise multidisciplinary skills and that complex systems comprise more than just equipment. It is also
noteworthy that many definitions of systems engineering describe what it does and not
what it is. This is particularly prevalent in the earlier standards.
The definitions follow in chronological order:

A logical sequence of activities and decisions that transforms an operational
need into a description of system performance parameters and a preferred
system configuration. (MIL-STD-499A, Engineering Management, 1 May
1974. Now cancelled.)

An interdisciplinary, collaborative approach that derives, evolves, and verifies
a life-cycle balanced system solution which satisfies customer expectations
and meets public acceptability. (IEEE P1220, Standard for Application and
Management of the Systems Engineering Process, [Final Draft], 26 September
1994.)

An interdisciplinary approach encompassing the entire technical effort to
evolve and verify an integrated and life-cycle balanced set of system people,
product and process solutions that satisfy customer needs,– (Shishko, R, 1995,
(NASA Systems Engineering Handbook)).
John Smith
-5-
12-Mar-2013
PhD Research Proposal

The INCOSE SE Handbook (1998) defines system engineering as follows “An interdisciplinary approach and means to enable the realisation of
successful systems”.

The EIA 632 standard describes the “processes for engineering a system.”
EIA 632 provides a description of the typical system engineering processes
associated with the life cycle of a system. The processes for engineering a
system are grouped into the five categories as shown below in figures 1 and 2.
It does not provide a definition of systems engineering.
Figure 1. The five main groups of
processes associated with the EIA 632 System
Engineering model.
Figure 2. The EIA 632 System
Engineering Process model “Egg diagram”.
The more recently released ISO/IEC 15288, Systems Engineering – System Life
cycle processes, also does not provide a definition of systems engineering but rather
provides a “common process framework to improve communication and co-operation
among the parties that create, utilise and manage modern systems in order that they can
work in an integrated, coherent fashion. These processes extend beyond those previous
addressed in EIA 632 and address agreement, enterprise, project and technical processes
in co-operating organisations.
Table 1 depicts the typical lifecycle stages whilst figure 3 shows the 15288 system
life cycle processes – note the inclusion of enterprise processes.
John Smith
-6-
12-Mar-2013
PhD Research Proposal
Table 1. The 15288 example of typical life cycle
stages, their purpose and major decision gates.
Figure 3. The 15288 System Life Cycle
Model Processes.
Over the span of a thirty year period, the definition of systems engineering has
gradually developed to encompass the more complex systems needed to be engineered.
More recently, standards have become less prescriptive in terms of defining a systems
engineering process, but rather concentrate more on the establishment of a framework of
life cycle processes that can be integrated to support the development of a complex
system.
Figure 4: Heritage of Systems Engineering Standards
John Smith
-7-
12-Mar-2013
PhD Research Proposal
Figure 4 depicts the so-call “quagmire map” of the development of various system
and software standards to attempt to grapple with the increasing complexity of systems.
The latest standard IEC/ISO 15288 [5] lists 23 processes that cover the breadth of SE
and places them into four categories as depicted in figure 3. These indicate that SE has
expanded its breadth beyond that of a predominantly technical discipline. Indeed, SE can
be considered a meta-discipline that coordinates and interacts with other related
disciplines such as project management, development engineering, integrated logistic
support, test and evaluation, configuration management and software engineering. One
of the improvements that can be seen in IEC/ISO 15288 is that SE has become inherently
interwoven with the enterprise environment of the host organization
2.2. A Typology for Systems Engineering
Hitchins [6] proposes a five-layer model for systems engineering to try and
encompass the scope and diversity of activities that systems engineering embraces.
Level
Hitchin’s Five Layer Model
Level 1 – Product or The outcome of SE at this level is tangible products.
Subsystems engineering.
Level 2 - Project SE.
This is classic or traditional SE that leads to the creation of
complex artifacts such as aircraft, ships, and computer
networks.
Level 3 - Business SE.
At this level many projects combine to form a business or
enterprise. At this level additional functions appear such as
marketing, strategic management, human resource
management, etc. There is also the concept of ongoing
activity beyond the life of a single project. Continuous
process businesses such as chemicals, pharmaceuticals, and
smelting operate at this level. A military Service can be
thought to operate at this level.
Level 4 - Industrial SE.
This level is characterized by many businesses working
together to achieve large-scale outcomes such as vehicle
manufacture, telephone networks, national transport
systems, national health systems, and the defence force.
Level 5 - Socioeconomic This is the highest level and activities are normally
SE.
sociotechnical in nature and of national or global scale.
National security, of which defence is a component,
operates at this level.
Table 2:
Hitchin’s Five Layer Systems Model
Hitchins states that the layers form a “nesting” model, in that many products make a
project, many projects make a business, many businesses make an industry and many
John Smith
-8-
12-Mar-2013
PhD Research Proposal
industries make a socio-economic system. He goes on to say that these statements are
only approximate since a socioeconomic system has more in it than just industries and a
business comprises more than just projects, and so on. Nonetheless, Hitchins’ model is
useful because it:



Gives an appreciation of the scope of activities that fall within the term
systems engineering.
Illustrates how each activity fits within the layer above and as such
emphasizes both the open system view of the engineering of complex systems,
and the hierarchy of SE activities.
Indicates that the ISO/IEC 15288 processes can be applied to various levels of
complexity not just engineering projects at Level 2.
For the purposes of illustrating where certain activities fit within the scope of both
systems engineering and the system life cycle, we can map Hitchins’ model onto a twodimensional space defined by system level on the vertical axis, and life-cycle time on the
horizontal axis (Cook et al, 2003).
Layer 5 Socio-Economic Level
Strategic
Planning
System Scale
Layer 4 Supply Chain Level
Capability
Development
Layer 3 Business Level
Layer 2 System Level
Capability Acquisition
and Through-Life Support
Layer 1 Product Level
25 yrs
Into the future
10 yrs
Before EIS
Entry
into Service
Support
Disposal
Life-cycle temporal focus
Figure 5: A graphical depiction of Hitchins’ extended five-layer model showing the positioning of the
systems activities of interest.
The types of systems addressed during the 1950s tended to be level 2 – Project SE for
which Hitchins states that the traditional systems engineering is adequate. However as the
complexity of the systems increased, i.e. defence systems became level 3 and level 4, it
becomes clearer as to why the more traditional process oriented approaches fail to work.
Systems activities can now be mapped onto this space to indicate where they fit with
respect to these two dimensions as shown in figure 5. For example, strategic planning
John Smith
-9-
12-Mar-2013
PhD Research Proposal
primarily concerns issues that are 10-25 years in the future and operates at the socioeconomic and supply-chain (whole-of-defence) levels. The positioning of capability
development in the figure illustrates that this activity is centered at the front of the
business-layer lifecycle and remains involved until projects enter service. Capability
acquisition starts once capability development processes have identified a capability gap
to be filled. In Australia, acquisition and logistic functions have been combined within
the Defence Material Organization and thus project office functions continue through
until equipment disposal which may occur decades after introduction of a capability into
service. The containment of the acquisition and support functions into Level 2 indicates
that project offices tend to be rather insular in their concerns.
2.3. Characteristics of Complexity in Defence Systems
At this point it is worthwhile to state what the characteristics of complexity found in
modern systems [and in particular, defence systems]. The following are representative of
characteristics that lead to complex problems in modern day systems.
1. Poorly defined system boundaries
2. Size – i.e. the number of system components and their interactions that needs
to be addressed at anyone level
3. Multi-disciplinary nature – i.e. the system has many technologies involved
and interacting
4. System topology – the number and nature of the inter-relationships between
the system components
5. Ill-defined operational goals for the system – i.e. the end user has not been
able to clearly articulate how the system is to be utilised, but may have been
able to express the need for such a capability
6. Unprecedented nature – i.e. such a system has not been developed before,
hence little experiential base to draw upon to assist in the development of the
system
7. Nature of the system problem is changing – i.e. the problem under
consideration is continually changing and is dynamic [referred to as a wicked
problem by Rittel]
8. Human Activity Systems – systems where humans not only use the system
but represent functional components of the capability, hence the need to
define the interfaces between technology and organisational information
exchanges
9. Political differences between different organisational components of the
system
10. Conflicting operational and sociological views within the system
The causes of these characteristics may be due to incomplete information, the nature of
the problem [Rittel’s wicked problem scenario] or the nature of the environment within
which the system needs to operate. For Defence, it is the environment within which
defence systems need to operate that is a major influence on the complexity of the
John Smith
- 10 -
12-Mar-2013
PhD Research Proposal
problems that need to be managed if not solved.
2.4. Existing Defence Systems Engineering Models
The existing systems engineering models within Defence have tended to be based on
the traditional or “classic” systems engineering approach. This approach is goal oriented
and is based on a belief system that the application of natural scientific principles to the
problem at hand will provide a solution. In line with this concept, the problem solving
method tends to be reductionist in that the belief is that the problem can be decomposed
to smaller components until each component is amenable to scientific analysis and
solution. Then the components are re-assembled to represent the whole.
Such a process-oriented methodology is well suited to problems that are well defined
and are inherently technical by nature. Consequently, this approach to systems
engineering fitted reasonably well the type of systems being developed after the Second
World War.
As the system complexity grew from the 1950s to modern day, various process
models were proposed to cater for the difficulties found in handing some of the
characteristics of increasing complexity as stated above. Some of these process models
are briefly described below.
1. Classic Waterfall Model – a sequential process model beginning with a
requirements analysis phase and concluding with system Acceptance testing.
Figure 6 depicts such a process methodology. A key point in this process is
that to begin the process [namely the requirements analysis phase], a well
defined set of operational specifications or Concept of Operations [CONOPS]
is required. The process is goal oriented and hence requires well defined
inputs to process.
Secondly, a large percentage of the process lifecycle – ie approximately half is
associated with paper products – hence the process is documentation centric
and is so linked into its review cycle.
John Smith
- 11 -
12-Mar-2013
PhD Research Proposal
Sequential lifecycle
User
requirements
definition
Review
System
requirements
definition
Review
Change
Architectural
design
Review
Change
Component
development
Test
Change
Integration
& verification
System
test
Change
Installation
& validation
Acceptance
tests
Change
a3
Operation &
Support
Figure 6: Classic Waterfall Model for Systems Engineering
2. The V Model – still represents a similar process model to the waterfall method
but provides cross links between the requirements and design phase to the
production and test phases. Three critical elements that the “V” diagram
promotes well.
a. Definition: Represented in terms of a Specification Hierarchy that
reflects decomposition.
b. Build:
Life-cycles
for
component
implementation
and
manufacturing.
c. Verification:
Represented in terms of a Test Hierarchy that
reflects integration
3. Spiral or Helical Model – again a similar process model but is usually used
when the best solution to the problem is unclear at the outset. The idea is to
create a prototype of the solution, test it in an operational environment and
then by observing deficiencies in its performance, devise improvements to the
system. Each revolution of the spiral invokes the classic waterfall process
model. The cycle is repeated until an acceptable solution is reached. Figure 7
below depicts such a process model.
John Smith
- 12 -
12-Mar-2013
PhD Research Proposal
REFINE
USER
REQUIREMENT
REQUIREMENTS
REAPPRAISE
NEED &
RISK
DELIVERED
PRODUCT
VALIDATE
SYSTEM
PERFORMANCE
MODEL
PROBLEM
VERIFY &
INTEGRATE
COMPONENTS
ANALYSE
APPROACHES &
CONSTRAINTS
SYSTEM
REQUIREMENTS
Figure 7:
IMPLEMENT
COMPONENTS
DESIGN
ARCHITECTURE
DESIGN
COMPONENTS
DESIGNED
SYSTEM
Spiral model for Systems Engineering
4. Incremental Development Model – within this process model, the system level
requirements and architecture are developed and then the various system
components are developed and commissioned in a staggered and incremental
manner. The key point here is that the top level system structure is developed.
Each increment still uses the waterfall model
5. Evolutionary Development – This model attempts to manage the development
of large complex systems by initially developing a basic capability and then
through operational use as an understanding of the utility of the system grows,
further enhancements are developed. Again a similar process model to the
waterfall model is used for each development. A limitation is that the initial
baseline development needs to capture the system concept to allow further
enhancements to be integrable into the design. There is little value in the
approach if each enhancement requires large amounts of re-engineering of the
previous development.
All of these system engineering process oriented models have met with limited
success. Where the system type tends to fit into Hitchins Level 2 Project SE layer, the
traditional systems engineering methods and process models meet with reasonable
success. However, as the nature of the systems become more complex; i.e. moved up
Hitchins’ layer model, the inherent reductionist basis of the process models did not
adequately cater for the real world situation, consequently resulting in poor compliance to
end user needs and ineffective systems design.
2.5. Multi-Methodological Approaches in Management Science
Multi-methodological
John Smith
approaches
are
- 13 -
becoming
commonplace
in
systems
12-Mar-2013
PhD Research Proposal
intervention practice that management consultants conduct to ameliorate problems in
organisations. The theory is that management issues are too complex to understand
adequately with one model, one set of theories, and one world view. Jackson and Keys
(1984) proposed a system of systems methodology (SOSM) that presented different
methodologies as being appropriate for different types of problem contexts. Jackson
(1987) codified this approach by assigning systems approaches to management into
categories characterized by social theory and problem complexity. From this, Flood and
Jackson (1991) formulated a systemic meta-methodology entitled Total Systems
Intervention that guides practitioners through methodology choice in a systemic way
based on metaphors of the problem situation. This has been elaborated in subsequent
works (Jackson, 2000; Mingers 1999). The fundamental ideas that have emerged
concerning the need for pluralism in problem solving remain sound.
Flood and Jackson assigned systems engineering as a methodology to the simple,
unitary problem context. This implies that it is a useful methodology for problems where
the actors share an agreed set of objectives, the problem is clearly structured, and the
complexity level is low. Most systems engineers would take exception to this
categorisation because they would consider that acquirers and suppliers would have quite
different objectives (world views) and they would consider that the creation of, say, a
new aircraft carrier was reasonably complex!
In defence of the management science community, one could argue that building a
large system of this type is, in fact, a simpler and more unitary proposition than, say,
solving the problem of homeless youth or public health care. If one appreciates that TSI
is focused on social and organisational issues, then the stance of the leading authors is
reasonable. One also needs to appreciate that TSI and other management science
techniques are aimed principally at improving existing complex organisational situations
and not at creating unprecedented systems.
2.6. Problems in Modern Defence Systems Engineering
Whilst contemporary defence systems engineering may recognise the complexity of
the problems associated with complex defence systems, there does not exist any
methodology or meta-methodology akin to Flood and Jackson’s Total Systems
Intervention concept for the management sciences that assists a practitioner to investigate
the nature of the problem space and thereby develop a suitably rich picture from which an
appropriate set of methodologies based on different social belief systems may be selected
to tackle the development of the system under consideration.
Incorrect methodology selection does not imply that the system will not be developed
and commissioned; what it does imply is that the system built may not meet all the
requirements of the end user. Incorrect methodology selection simply implies that an
insufficient problem definition has occurred and consequently, an incorrect or inferior
model representation of the complex system has occurred.
John Smith
- 14 -
12-Mar-2013
PhD Research Proposal
This tends to be the case when methodologies appropriate for a level 2 type system
are used to develop type 4 or 5 level systems – i.e. systems that are sociological or
sociotechnical and consequently exhibit complex behaviour not strictly amenable to
scientific enquiry. Consequently, the inappropriate methodology solves the representation
of the complex system that is structured and well represented – ie it solves a simpler less
complex problem than that required to be addressed.
Symptoms of such occurrences may be observed when end users complain that the
system poorly meets their needs – usually a symptom of poor systems engineering
encountered in the problem definition or structuring phase.
John Smith
- 15 -
12-Mar-2013
PhD Research Proposal
3.
Research Problem
The final section in chapter 2 of this research proposal discussed the nature of
problems found in modern defence systems engineering. In particular, the problem to be
investigated is:
How does one select an appropriate set of methodologies to undertake
the systems engineering across the life cycle development of modern
defence systems?
In the previous section, we discussed the development of defence systems
engineering and the gradual change in the nature or type of system towards increased
complexity that Defence required to be engineered. This need to engineer systems with
increased complexity is driven by a number of factors, such as:
a.
b.
c.
Changes in the environment that Defence forces need to work in.
Changes in the Missions that Defence may undertake
Changes in the response times.
For Australia, this means the following:
a.
Australian forces may now be deployed anywhere in the world as part of a
Coalition force. Consequently, defence equipment needs to be
interoperable with that of a coalition force, and more so, Australian
equipment needs to be net centric, not platform centric so that force
attributes such as shared awareness, synchronization and agility may be
utilised across an instantiated coalition force. These are strong design
drivers for Defence systems when one considers that most systems will
not have been designed with this in mind or cognisant of the need to “Plug
& Play” in instantiated System of systems.
b.
The Australian Defence forces only a short period ago, before September
11, would predominantly focus on Defending Australia, Protecting
regional interests and establishing and maintaining international alliances.
Today, with the advent of Global Terrorism, Australian forces may
undertake missions in any part of the Globe with roles such as Support to
Coalition led Forces in Defence or Offensive positions in areas of the
Globe that Australia would normally not have been involved; e.g.
Afghanistan, Iraq, etc. Support to or Lead Coalition forces within the
Pacific Rim and Support to UN Peacekeeping forces. These new roles
require Australian Defence equipments to be interoperable, net centric and
“plug and play’ in the formation of mission specific System of systems.
c.
The responses times that Australian forces may have to meet may be very
short – days to weeks for some deployments. This is clearly not sufficient
time to design new specific defence systems to meet new and unexpected
roles and missions. Again, modern defence systems need to be adaptable,
John Smith
- 16 -
12-Mar-2013
PhD Research Proposal
agile; net centric and re-configurable [ie plug and play in system of
systems.].
If we analyse these new demands on Australian Defence systems, we find that these
modern defence systems have the following attributes:
1. Systems need to be interoperable with other systems that may not be defined
at the time of design
2. Need to undertake roles not envisaged at the time of design
3. Need to be net centric not platform centric
a. Need to be agile; i.e., system functions not tightly coupled together but
able to be connected to other functions from other systems
b. Need to be re-configurable – i.e. system can be re-configurable to
perform other functions than that designed for.
4. System needs to be able to be integrated into a larger system of systems
We can see the following:
1. The design of defence systems to meet such future possible roles means that
there is usually an ill defined set of requirements as to what operational
conditions the system will work within
2. The problem is ill structured as not all requirements can be forecast or are
known at the time of development and this needs to be catered for.
3. The ability to integrate into any systems of systems is not just at the technical
level but also at the systems, operational and organisational level. Hence, such
developments are inherently complex as these interfaces are not known at the
time of the system development but the systems needs to be flexible enough to
undertake such activity.
4. Net centric concepts such as shared awareness, synchronization and agility
also place demands on system design that can not be clearly predicted at
design time.
Defence communities in the United States, Europe and Australia are investigating a
number of approaches to the development of defence systems with such complex
capabilities. Approaches such as:
1. Threat based response options
2. Military Capability Packages
3. Capability Based planning.
The third option is currently being viewed as a viable means for the development of
such defence capability. Typical examples of such capability as rapid instantiated C4ISR
systems to meet either Joint [Australian] or Coalition deployed forces needs and the
development of a mobile and agile defence force capability.
These systems are clearly not at Hitchins’ level 2 Project Engineering layer but more
at level 3 and level 4. Hence it is not surprising to find that traditional systems
engineering struggles with these types of system development.
John Smith
- 17 -
12-Mar-2013
PhD Research Proposal
Whilst approaches such as Capability Based Planning gain flavour in the international
community for the planning of such complex defence systems, there is still little to no
guidance available as to how to engineer such systems. There is little to no system of
methods or Meta-methodology to assist the systems engineering practitioner in the
selection of an appropriate set of methodologies the engineer such complex defence
systems.
It is the development of such guidance to be able to select an appropriate mix of
methodologies that this research will focus on. The research will examine the following
areas.
1. Systems engineering and life cycle models and processes – in particular the
new standard ISO 15288 with a view to establishing a set of activities and
processes aligned with different life cycle phases of a system
2. Development of a Problem typology; primarily aimed at understanding the
attributes of problems, in particular those associated with complexity.
3. Development of a Systems typology – developing a military typology of
systems based initially on Hitchins’ 5 layer model.
4. Development of a set of attributes within an ISO 15288 framework that will
provide the necessary guidance towards the selection of problem solving
methodologies relevant to the activities associated with a particular life cycle
phase
5. Theoretical insight into the nature of the selection process and its utility to
modern defence systems
3.1. ISO 15288 Life Cycle processes
If we think of ISO 15288 as providing a framework for the development of modern
complex systems, i.e. the life cycle processes involved in the development of complex
systems; then the set of processes defined within 15288 can be related to the activities
that need to be undertaken throughout the lifecycle of the system.
It is useful to state that the systems lifecycle spans from conception to disposal and is
usually drawn as five or six phases and is designed to suit the problem domain, the level
of complexity, and nature of the system of interest. The phase to be tackled of the system
life cycle will determine the skill sets to be invoked and the detail of the methods to be
employed. For example, the operations phase requires quite different people, processes
and skills from conceptual design.
Therefore, if we think of these activities as the problems we need to solve to develop
the complex system, we can begin to link ISO 15288 to a set of problem solving
methodologies. Our thinking can be depicted as follows:
ISO15288  Lifecycle phase  Processes  Activities  Problems to solve  Problem Solving
Methodologies
John Smith
- 18 -
12-Mar-2013
PhD Research Proposal
3.2. A Typology of Problems
The systems engineering process is described in MIL-STD-499B as a comprehensive,
iterative problem solving process. In order to understand how best to select
methodologies to tackle systems engineering problems, it is helpful to understand what
constitutes a problem.
Jonassen (2003) explains that problems vary in at least four dimensions, namely
structuredness, complexity, dynamicity and domain specificity (or abstractness). The
definitions of these problem attributes have been taken from Jonassen (2003) with the
addition of the additional category, namely “wicked”, to structuredness.
1. Structuredness usually has two broad classifications, namely well-structured and illstructured. More recently, the term wicked has also been introduced in the literature
to define a special class of particularly difficult problems to solve. Some definitions
of these are:
 Well Structured – require the application of a limited and known concepts,
rules and principles within a restricted domain (or discipline), have a well
defined initial (or input) state, a known goal state or solution and a constrained
set of methods for solved the problems.
 Ill Structured – problems often have aspects that are not well understood, they
are interdisciplinary [i.e. can not be solved by applying concepts and
principles from a single domain]; they usually have poorly defined initial or
input conditions and may possess multiple solutions or solution methods or
often no solution at all.
 Wicked – this type of problem is similar to ill-structured problems except that
the problem parameters may change significantly within short time periods
requiring a constant re-appraisal of solution methods, domains and possible
end goals. It is due to the changing nature of the problem parameters that the
term “wicked” was coined.
2. Complexity is determined by the number of issues, functions or variables
involved in the problem, the degree of connectivity among these variables, the
nature of the functional relationships and the stability of these properties over
time. Hence complexity and structuredness overlap. Ill structured problems
tend to be complex.
3. Dynamicity is a measure of the stability of problems, moreso how the problem
parameters change over time (rapidly, slowly, etc). As the problem parameters change
so must the understanding of the problem to enable solutions to be developed, where
feasible. Wicked problems are an extreme example where the problem state changes
so quickly, that the formation of solutions needs to be continually revised.
Consequently, solution-oriented methods may not be feasible.
4. Domain context refers to the observation that problems within a domain rely on
cognitive operations that are specific to that domain. Problem solving activities
therefore are dependent on the nature of the context or domain knowledge.
It is proposed to develop a problem typology for defence systems that will provide
John Smith
- 19 -
12-Mar-2013
PhD Research Proposal
insight to the nature of the problem to be tackled and thereby to the relevant problem
solving methods.
3.3. A Typology of Systems
Hitchins provides a five layer model of systems in terms of increasing complexity. It
is proposed to develop a similar typology for defence based systems for use in
understanding at what level a defence system sits. The combination of proper
identification of system type along with a problem typology for the nature of problems
found within the various life cycle activities of the system provide a basis for selection of
problem solving methodologies
3.4. Attributes for selection of systems (engineering) processes
and practices
The previous sections have described the generic processes of systems engineering,
systems engineering life cycles phases, Hitchins’ 5-Layer model mapped to a Defence
model, and the dimensions of problem solving. Each of these can contribute to a set of
problem attributes such as:







Structuredness – can be represented on a Likart scale
Complexity – a suitable range – e.g. TSI’s simple and complex
Context – Nature of the problem
Level – associated with Hitchin’s 5-layer model.
Phase – associated with the ISO 15288 life cycle processes
Problem of interest – mapped to the most logical 15288 process(es)
Dynamicity – is the problem static or changing with time, how quickly?
It is postulated that the selection of an appropriate set of problem solving
methodologies is a complex cogitative process that is akin to parsing the tree structures
associated with complex decision making problems. This will be explored further in the
research. A key point here is that the processes need not be (nor usually are) sequential,
the process can be quite non-linear with the individual applying different weighting
criteria based on experiential judgement.
It is this key activity that is at the heart of methodology selection.
3.5. Expected Contribution
Contemporary systems engineering practice as applied to modern defence systems
still tends to apply traditional systems engineering processes and practices. As discussed
earlier, modern defence systems tend to be found at level 3 and 4 of Hitchins’ system
typology model and consequently implies that traditional systems engineering practice is
John Smith
- 20 -
12-Mar-2013
PhD Research Proposal
poorly suited for these class of systems.
Whilst the Defence community is exploring alternative methods to assist in the
planning and development of Defence capability (the most recent being Capability Based
Planning), little thought has gone towards the selection of appropriate problem solving
methodologies for the life cycle development of these systems.
Currently there does not exist any theoretically based method for the selection of
appropriate problem solving methodologies across the life cycle phases of a modern
defence system.
The outcomes of this research combines the ideas of systems engineering and
cognitive problem solving to propose a list of attributes that can be used to characterise
systems engineering problems at various phases within a systems life cycle and thereby
provide guidance as to the appropriate mix of problem solving methodologies.
The outcome will use an ISO 15288 framework within which system activities can be
related to problem solving methodologies in a robust theoretical manner. This framework
will permit the development of a rich picture of the problem types to be encountered and
thereby facilitate an understanding of the mix of methods from different social models
that may be necessary to solve or manage the problems.
This will result in a more rigorous application of systems engineering processes and
practices to complex systems and assist in better aligning user expectations with system
operation and performance.
As a minimum, such knowledge would assist novices to get started and avoid
inappropriate applications of systems engineering. In an environment, such as a research
laboratory where systems engineering is not considered a core skill, this knowledge
would be very useful for those who are tasked to produce a concept demonstrator, assist a
development project, or provide advice to capability development.
More appropriately, it would be a valuable tool for the informed systems practitioner
to facilitate and develop the framework of considerations necessary to ensure that an
appropriate mix of methods is chosen for any particular task.
John Smith
- 21 -
12-Mar-2013
PhD Research Proposal
4.
Research Methodology
4.1. Literature Review
A literature review will be undertaken to study the various system engineering processes and
practices currently being applied in Defence systems as well the nature of complexity in modern
systems, approaches towards engineering complex systems and their relevance to military
defence systems. A further study will be undertaken into multimethodology practices as
employed in the management sciences and their relevance to defence oriented systems.
Finally, a study will be undertaken to discover within the literature, the current state of the art
with respect to theoretical approaches to systems engineering of complex systems and their
relevance to defence systems.
4.2. Methodology
A two fold approach will be taken in the system of methods applied to this research problem.
a.
An FMA Approach
Checkland and Holwell (1998), state that there are three elements necessary to describe any piece
of research:



The Area of Concern (A), which might be a particular problem in a discipline (area of
study), a real-world problem situation, or a system of interest.
A particular linked framework of ideas (F) in which the knowledge about the area of
concern is expressed. It includes current theories, bodies of knowledge, heuristics, etc as
documented in the literature as well as tacit knowledge.
The methodology (M) in which the framework is embodied that incorporates methods,
tools, and techniques in a manner appropriate to the discipline that uses them to
investigate the area of concern.
Figure 8 extracted from Checkland and Holwell (1998), illustrates the relationship between these
three elements and how undertaking the methodology creates new knowledge about all three
elements.
Figure 8:
John Smith
Elements relevant to any piece of research (Checkland and Holwell, 1998: p 13).
- 22 -
12-Mar-2013
PhD Research Proposal
In the context of this research problem,
1. Area of concern is how does one select appropriate problem solving methodologies
associated with the application of systems engineering in the various life cycle phases of a
defence system.
2. Framework of ideas will comprise a number of items such as the ISO 15288 framework,
Hitchin’s work on systems engineering, a number of approaches on problem typologies,
concepts from Flood and Jackson in terms of their Total Systems intervention concept,
Mingers and Gill’s work on Multi-methodologies, heuristical knowledge from Rechtin and
Maier plus an assortment of concepts and ideas from the broader literature
3. The methodology or in this case the system of methods to be used will comprise a systems
analysis of the source material, and a postulate of the outcome to be examined for theoretical
correctness and practical soundness.
b. Practical Assessment of Method
The second component to the methodology is in support of the last statement in point a – namely
“postulate of the outcome to be examined for theoretical correctness and practical soundness.” the focus being on “practical soundness.
It is proposed to review a number of defence systems that have been developed by the author over
the course of this research program and assess how well the concepts proposed with the research
work align with practical experience. These studies will take the form of case studies. Their use is
to apply some guidance and insight into the practical utility and theoretical soundness of the
approach.
4.3. Case Studies
It is proposed to undertake a number of case studies of the systems engineering methods applied
to a selection of defence programs. The case studies will be selected across a range of system
types and problem complexity.
4.4. Publications
A number of reports and conference publications will be produced and submitted to international
conferences and journals.
John Smith
- 23 -
12-Mar-2013
PhD Research Proposal
5.
Work Plan
This work is being undertaken part time as the researcher is employed on a full time
basis within his own systems engineering consultancy. It is anticipated that the work will
be completed in December 2006.
Timeframe
1999/2000
2001/2002
2003/2004
2005
2006
John Smith
Activities
a. Enrolment
b. Draft Research proposal
c. Review of classical system engineering techniques for
concept technology demonstrators (CTD)
d. Review of literature on CTD development
e. SETE 99 paper
a. Review of literature on Soft Systems Techniques
b. Checkland, Senge, Flood and Jackson’s TSI
c. Kline’s work on multi disciplinary thinking
d. Work on Defence Capability and effects based domain
partitioning
e. Reports on Systemic behaviour of Defence Capability
domain structures
f. INCOSE 2002 paper
a. Extension of research into enterprise level systems
b. Hitchin’s and Rechtin and Maier’s work
c. Report on Organisational Intervention – 2003
d. DSTO reports
e. INCOSE 2004 paper
f. SETE 2004 paper
a. Update to PhD research Proposal
b. Research into attributes for ISO 15288 activities
c. Problem typology to be developed
d. Modified Hitchin’s Typology
e. Finalise theory and process
f. Begin writing sections of PhD
g. Proposed INCOSE 2005 paper (accepted)
h. Proposed SETE 2005 paper
i. DSTO Research reports
a. Case studies – review and test theory and process
b. Continue to finalise draft thesis
c. Mid Year review of draft thesis
d. End of year – completion of Thesis
- 24 -
12-Mar-2013
PhD Research Proposal
6.
References
Arnold S., and Cook S., (2002), “Developing a Coalition Systems Engineering Process”,
INCOSE Insight Vol 5, No 3 pp 7-9.
Avison, D. E. and Fitzgerald, D., (2003) Information Systems Development:
Methodologies, Techniques and Tools, McGraw Hill Book Company Europe. Third
edition.
Cook, S.C., (2003) Systems Engineering for Complex Problem Solving, Course Lecture
Notes, University of South Australia.
Cook, S.C., (2004), “The Rise of Systems Engineering within the Australian Defence
Organisation”, in Proceedings of the 2004 IEEE International Engineering Management
Conference, Singapore, 18 to 21 October 2004, pp 75 to 79.
Cook, S. C., Kasser, J. E. and Ferris, T. L. J., (2003) “Elements of a Framework for the
Engineering of Complex Systems”, Proc. of the 9th ANZSYS Conference, Systems in
Action, 18-20 November 2003, Melbourne, Australia, Paper No. 3000079.
DoD (2002), Capability System Life Cycle Management Manual, Department of Defence,
Australia.
DSMC, Systems Engineering Management Guide, Defence Sysrtems Management
College, Fort Belvoir, Virginia, USA, 1993
Hitchins D. K., (2003) Advanced Systems Thinking, Engineering, and Management,
Artec House, ISBN 1-58053-619-0.
Hodge, R.J. (1998)“Defence Capability Development -Learning from the Future”, in
Proceedings of SE98, Canberra, Australia, 4-6 November 1998, pp. 43-56.
Hodge, R. and Walpole, G., (1999), “A Systems Approach to Defence Capability
Planning – A Work in Progress,” in Proceedings of SETE99, Adelaide, Australia, 20-22
October 1999, pp. 21-32.
INCOSE. Systems Engineering Handbook (v 1.0), International Council on Systems
Engineering, 1998.
ISO/IEC 15288, System engineering – System life cycle processes, ISO/IEC.
John Smith
- 25 -
12-Mar-2013
PhD Research Proposal
Jackson, M.C. and Keys, P. (1984). Toward a systems of systems methodologies. Journal
of the Operational Research Society, 35, 473-486
Jackson M. C., (2000) Systems Approaches to Management, Kluwer Academic/Plenum
Publishers, ISBN 0-306-46506-X.
Jonassen, D.H. (2003). Learning to Solve Problems: An Instructional Design Guide,
ISBN: 0-7879-6437-9 Hardcover 256 pages December 2003, Pfeiffer.
M’Pherson P. K., “Systems Engineering: an approach to whole system design,” The
Radio and Electronic Engineer, 1980, Vol. 50, No 11/12, p545-558
MIL-STD-499B, (1994), Systems Engineering, US Department of Defence.
Mingers, J. and Gill, A. (1997), Multimethodology, The Theory and Practice of
Combining Management Science Methodologies, Wiley, ISBN 0471974900.
Sheard S. A. & Lake J. G., Systems Engineering Models and Standards Compared, Software
Productivity Consortium, Online Accessed, October 1998.
http://www.software.org/pub/papers/9804-2.html
John Smith
- 26 -
12-Mar-2013