SoS Architecting Working Group – CSSE Convocation
Introductions
I. Those Present
Name
A Winsor Brown
Dave Dornebos
Elliot Axelband
Azad Madhni
Paul Robitaille
Scott Jackson
Gerry Nadler
Rod Robertson
Ricardo Valerdi
Stan Settles
Thomas Baehren
Barry Boehm
Ed Colbert
Vu Nguyen
Jesal Bhuta
Thomas Tran
Organization
USC CSSE
Motorola
Rand/USC
ISTI
Lockheed Martin &
INCOSE
USC
USC
Boeing
MIT
USC
Robert Bosch GmbH
USC
USC
USC
USC
NGC
Email
AWBrown@CSSE.USC.edu
Dave.Dorenbos@Motorola.com
Elliot_Axelband@Rand.org
Amadni@IntelSysTech.com
Paul.Robitaille@LMCO.com
Jackessone@COX.net
Nadler@USC.edu
Rodney.Robertson@Boeing.com
Rvalerdi@mit.edu
Settles@USC.edu
Thomas.Baehren@de.Bosch.com
Boehm@USC.edu
EColbert@USC.edu
Nguyen@USC.edu
Jesal@usc.edu
Thomas.Tran@NGC.com
II. Ground Rules for discussion/brainstorming
A.
B.
C.
D.
One person at a time
Positive statements only, please
Follow through to a definition
What constitutes an item for the brainstorming
1. SoS Architecting
2. Value: research topic (add to SoA and chance for funding)
E. Time limit?
612934202 – 1 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
Items/topics proposed
I. System of Systems Resilience -- Scott Jackson
A. Resilience=
1. Scott's informal definition: (based on Resilience Engineering website) "Resilience is the
attribute of a system that makes it less likely to experience failure. Resilience engineering
considers the process, disciplines, infrastructure, and cultural attributes that need to be in
place to make failure less likely."
2. Human factors over the entire spectrum: Design, test, production, operation, etc.
B. Discussion
1. SoS
a. Are more complex
b. Have more modes of failure
2. Architecture Dimension
a. Product and its attributes/capabilities
b. Environment
3. How would this be "researched"
a. Cultural impacts in failures have been evaluated, but little positive suggestion
4. What should be the Engineering Method for Architecting Systems
a. What is the right thinking process
b. How to change the engineers misguided thinking
c. Meld cultures
C. Program startup difficulties because disconnects on how to represent the architecture
1. Research
a. Organizational theory
b. What/How to model, including process (what is a
2. Need for multiple views
3. Why not an AADL (SAE) or a combination of SysML and
4. How to get a group to ask questions that coalesces them toward making progress
II. Illustrations of successes of SoS
A. What's a valid SoS
1. What were goals?
2. Was it architected?
B. What constitutes success
1. Contract vs. WWW
C. Identify cases
1. book
a. exemplars (analogs)
1) "Why Decisions Fail" (last several pages summarize
a) hat are
2) "Managing the Unexpected"
612934202 – 2 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
2. Carry over lessons learned
a. DoD Funding for capturing experience(s) and their contexts
b. Risk: look at as an opportunity, not a loss
c. Take
3. Modeling based to reason about successes and failure
4. Dimensions of SoS architect
a. BB 1994 AF Architecture Study General's goal: an architecture; but world does not stand
still so need to look at process/methods
1) Engg
2) Stakeholder
b. Noun Vs Verb
1) Architecture as a noun:
2) Architecting as a verb:
5. Efficiency vs. Effectiveness
6. Evolution is always a factor of success of System of System
D. Research topics
1. What are the views and what models are needed
2. What methods for analysis and tradeoff
3. Not just an aggregate
III. System or SoS ambiguity
A. Traditional Sys Eng'g looks at …; SoS research has to address the others
1. e.g., Interoperability
2. What
B. Attributes of SoS and how to quantify and measure them
1. ilities (including those that do not end in ility)
a. How to characterize
b. How to
2. Dynamic change of structure (participants)
3. example A0: operational availability
4. How to discuss tradeoffs
a. one at a time (e.g., safety only focus) vs.
1) concurrent?
2) collectively (aggregation)
b. How to "prove" a SoS has any ility
c. Lack of trust if not verified and validated (on a batch)
5. Dynamics of change
a. if predictable, methods may exist to analyze (pre-analyze)
b. if dynamic (size, type and number of elements), then there are no needs
1) evolution envelope
2) environment and
612934202 – 3 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
IV. Process for how to "Specify SoS" in a standardized/consistent/repeatable way
A. Processes to remove variability
1. Today's are under specified which
2. basis:
a. Performance
b. Logistics
3. Model Driven Architecting to systematize SoS RFPs and evaluations of proposals
4.
B. Examples
1. FCS
2. 1st Telephone switching system
C. Characteristics
1. What are States and Modes, including invalid ones (to be avoided)
a. How to identify them
b. Are there classes of states
1) what are their characteristics
2) can they be "modeled" and reasoned about
3) what are minimal capabilities within a state
2. Visualizability of configurations/structures
D. Model driven approaches
1. Text (one thing said, ten things understood); model(s) as a solution for one thing said, one
thing understood?
2. Vision: Model -> prose -> specification
3. Simulation based Acquisition (nee Design at Lockheed Martin)
a. Models to be evaluated
b. Simulation based Design elevated to Acquisition, but hyped because of lack of
tools/methods to "implement"
c. Model driven architecture
1) Platform Independent -> Platform Specific
2) Value added of human in the loop
4. Challenge is in the fidelity of the models
a. Simulation is not the same as self satisfaction, but if you do it for a long time simulation
can be confused with satisfaction.
b. Stimulation component of simulation
c. Simulation communities
1) Test and training
2) Engineering (architecture and
d. Engineering needs of fidelity different fidelity than the training/testing needs
5. DODAF approach does not lend itself to modeling
a. But simulations can be mapped into models
E. Continue to bridge the simulations/models
1. Between/across "layers"
2. Within layers (across other dimensions such as time, space …)
612934202 – 4 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
V. Human limits to handling complexity
A. CAOC
1. Human as user/operator
2. Human as architecting agent
3. Human as reactor to
B. IKWSI (not knowing how SoS results will be used
C. Area for Research: Team cognitive task analysis
1. How do successful teams work?
2. How to get players to come to the "field of dreams" that already exist:
3. How to get individuals to change, beyond
a. Working in a team
b. Building trust
4. Flattening of the World changing the landscape
5. People adapting to change?
D. Are the ways people collaborate and interact (in the SoS Architecting domain) so
different from the way they do in the System Engineering world
VI. Inherent fragility of software and information technology to accommodate netcentricity
A. Net Centric examples
1. warfare
2. …
B. Concerns
1. Tampering
2. Identity "corruption"/theft
C. How to make the SoS
1. More Robust
a. Backup modes
b. Training
c. Alternate modes of
2. Robust ~= Resilient?
3.
D. Software protection Technology
E. Defense in depth (biological metaphor: ward off, surround, absorb, … die [so as not to
affect others])
F. Homeland security as a source of funding? Multi-biometrics and
G. Research Topic "Investigate the inherent vulnerability of SoS" and "How to make SoS
more robust in the presence of the vulnerabilities".
612934202 – 5 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
VII.
A.
B.
C.
D.
E.
F.
G.
H.
I.
J.
VIII.
Evolution -- Scott
Planned ways or un-planned bad ways?
Friendly: Open Architectures or Open Source
Emergence impact?
Emergence is a negative property of "designed/architected" system; but must be "coped
with" in SoS.
Planned/controlled vs. "steered" via constraints and incentives: What mechanisms can
be applied to achieve the later
Contractual aspects: A guideline for how to write 6.2/6.3 contracts with un-priced parts;
without add-on contract approach. Capability vs. spec'd deliverable.
1. Stick with same team over large variations
2. Adaptive tactics?
Evolution as a process in which different entities (creatures) attempt to "survive and
thrive".
Approaches to adapting to un-anticipated change
1. Agile = dynamic adaptation
2. Critical Chain Theory = buffers along the chains to absorb impacts
Hard to funding things that have no explicit result
Genetic algorithms ?
"Guided" Emergence (SoS Architecting for friendly/positive emergence)
A. Add "Guided" (can't control) to make it a research topic
1. Guiding behavior without controlling
2. Achieved through learning algorithms?
B. Behaviors emerge in teams
1. Bad: fall apart because of lack of trust
2. Good: Boston Celtics
C. Communities of practice vs. Communities of interest
1. COP tend to be stove piped
2. Members of COIs don't "practice"
3. How to get COPs to cooperate (as a n
D. Adaptive vs. Adaptable System
1. Avoid unnecessary expense
E. Twiki as an example?
F. How did "Charity" or "Philanthropy" evolve?
G. What are the canonical behaviors that can emerge? How to make it have value?
612934202 – 6 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
IX. No single owner SoS -- Thomas and Gerry
A. e.g.,
1. Automotive transportation
2. WWW
3. Education
4. Health care
5. Mortgage and finance
6. Government
B. What about funding in the non-DoD business model funding?
1. Business (automotive)
2. GOs
a. NSF (for Multi-mission systems)
b. LA County?
c.
3. NGOs
4. Philanthropic orgs for any of the "e.g."s
NOTE: The material below is what was presented and corrected on the fly
612934202 – 7 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
Notes on Difficulty/Value Trade Space
I. USC CSSE is a trusted organization
II. Research to be performed by USC Ph D student or Post Doc.
(or possibly team with other U)
III. Difficulty of Research
A. Different dimensions (especially for emerging fields of engineering)
1. Intrinsic difficulty of the topic (tractability)
2. Lack of Resources
a. Data
b. Equipment/Facilities
c. Tools
3. Funding
a. Exists, but with stiff competition
b. Unlikely (barren land)
B. Lesson learned:
1. Use the one of highest importance to the evaluator (strongest feeling)
2. Identify the rationale reason on the score sheet
612934202 – 8 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
Research Topics
I. System of Systems Resilience -- Scott Jackson
A. Resilience=
1. Scott's informal definition: (based on Resilience Engineering website) "Resilience is the
attribute of a system that makes it less likely to experience failure. Resilience engineering
considers the process, disciplines, infrastructure, and cultural attributes that need to be in
place to make failure less likely."
2. Human factors over the entire spectrum: Design, test, production, operation, etc.
B. How would this be “researched”?
1. Determine the architectural aspects of the end product that will make it less vulnerable to
catastrophic failure in a system of systems environment.
2. Determine the architectural aspects of the supporting infrastructure, to include the
developer, customer, suppliers, operators, and maintainers, to make the resulting end
product less vulnerable to catastrophic failure in a system of systems environment.
3. Identify methodologies to create a culture that will enable system resilience in a system of
systems environment.
4. Develop statistical methods of showing when a system is vulnerable to catastrophic failure
based on defects at various phases of the life cycle including architecting, development and
operation.
C. What are the cultural issues?
1. Cultural impacts in failures have been evaluated but little positive suggestion
2. What is the right thinking process?
3. How can mis-guided thinking be changed?
4. What is the effect of melding cultures?
5. Risk denial is a major cultural issue.
D. Discussion
1. SoS
a. Are more complex
b. Have more modes of failure
2. Architecture Dimension
a. Product and its attributes/capabilities
b. Environment
3. How would this be "researched"
a. Cultural impacts in failures have been evaluated, but little positive suggestion
4. What should be the Engineering Method for Architecting Systems
a. What is the right thinking process
b. How to change the engineers misguided thinking
c. Meld cultures
5. Program startup difficulties because disconnects on how to represent the architecture
6. Research
a. Organizational theory
b. What/How to model, including process (what is a
7. Need for multiple views
8. Why not an AADL (SAE) or a combination of SysML and
9. How to get a group to ask questions that coalesces them toward making progress
612934202 – 9 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
II. Illustrations of successes of SoS -- Gerry Nadler
A. What's a valid SoS
1. What were goals?
2. Was it architected?
B. What constitutes success
1. Contract vs. WWW
C. Identify cases
1. book exemplars (analogs)
1) "Why Decisions Fail" (last several pages summarize what the fundamental reasons are)
2) "Managing the Unexpected"
2. Carry over lessons learned
a. DoD Funding for capturing experience(s) and their contexts
b. Risk: look at as an opportunity, not a loss
c. Take
3. Modeling based to reason about successes and failure
4. Dimensions of SoS architect
a. BB 1994 AF Architecture Study General's goal: an architecture; but world does not stand
still so need to look at process/methods
1) Engg
2) Stakeholder
b. Noun Vs Verb
1) Architecture as a noun:
2) Architecting as a verb:
5. Efficiency vs. Effectiveness
6. Evolution is always a factor of success of System of System
D. Research topics
1. What are the views and what models are needed
2. What methods for analysis and tradeoff
3. Not just an aggregate
612934202 – 10 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
III. System or SoS Attributes (and term ambiguity) -- Ricardo Valerdi
A. Traditional Sys Eng'g looks at …; SoS research has to address the others
1. e.g., Interoperability [of Systems]
2. What
B. Attributes of SoS and how to quantify and measure them
1. ilities (including those that do not end in ility)
a. How to characterize
b. How to
2. Dynamic change of structure (participants)
3. example A0: operational availability
4. How to discuss tradeoffs
a. one at a time (e.g., safety only focus) vs.
1) concurrent?
2) collectively (aggregation)
b. How to "prove" a SoS has any ility
c. Lack of trust if not verified and validated (on a batch)
5. Dynamics of change
a. if predictable, methods may exist to analyze (pre-analyze)
b. if dynamic (size, type and number of elements), then there are no needs
1) evolution envelope
2) environment an d
612934202 – 11 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
IV. Model Driven approaches -- Azad Madni & Paul Robitaille
A. Model Driven Architecting to systematize SoS RFPs and evaluations of proposals
1. Text (one thing said, ten things understood); model(s) as a solution for one thing said, one
thing understood?
2. Vision: Model -> prose -> specification
3. Simulation based Acquisition (nee Design at Lockheed Martin)
a. Models to be evaluated
b. Simulation based Design elevated to Acquisition, but hyped because of lack of
tools/methods to "implement"
c. Model driven architecture
1) Platform Independent -> Platform Specific
2) Value added of human in the loop
4. Challenge is in the fidelity of the models
a. Simulation is not the same as self satisfaction, but if you do it for a long time simulation
can be confused with satisfaction.
b. Stimulation component of simulation
c. Simulation communities
1) Test and training
2) Engineering (architecture and
d. Engineering needs of fidelity different fidelity than the training/testing needs
5. DODAF approach does not lend itself to modeling
a. But simulations can be mapped into models
B. Continue to bridge the simulations/models
1. Between/across "layers"
2. Within layers (across other dimensions such as time, space …)
612934202 – 12 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
V. SoS Views (nee characteristics) -- Barry Boehm
A. What are States and Modes, including invalid ones (to be avoided)
1. How to identify them
2. Are there classes of states
a. what are their characteristics
b. can they be "modeled" and reasoned about
c. what are minimal capabilities within a state
B. Visualizability of configurations/structures
C. Other views for characteristics with context
D. Domain specificity may apply
612934202 – 13 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
VI. Human attributes in handling complexity and change
-- Gerry Nadler & Azad Madni
A. Combined Air Operations Center (CAOC)
1. Human as user/operator
2. Human as architecting agent
3. Human as reactor to
B. IKIWISI (not knowing how SoS results will be used)
C. THWADI (That's How We've Always Done It)
D. Area for Research: Team cognitive task analysis
1. How do successful teams work?
2. How to get players to come to the "field of dreams" that already exist:
3. How to get individuals to change, beyond
a. Working in a team
b. Building trust
4. Flattening of the World changing the landscape
5. People adapting to change?
E. Are the ways people collaborate and interact (in the SoS Architecting domain) so
different from the way they do in the System Engineering world
612934202 – 14 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
VII. Inherent fragility of software and information technology to accommodate netcentricity -- Elliot Axelband
A. Net Centricity application area examples
1. warfare
2. commerce
3. emergency response
B. Concerns
1. Tampering
2. Identity "corruption"/theft
C. How to make the SoS
1. More Robust
a. Backup modes
b. Training
c. Alternate modes of
2. Robust ~= Resilient?
D. Software protection Technology
E. Defense in depth (biological metaphor: ward off, surround, absorb, … die [so as not to
affect others])
F. Homeland security as a source of funding? Multi-biometrics and
G. Research Topic "Investigate the inherent vulnerability of SoS" and "How to make SoS
more robust in the presence of the vulnerabilities".
612934202 – 15 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
VIII.
A.
B.
C.
D.
E.
F.
G.
H.
I.
J.
Evolution -- Scott Jackson
Planned ways or un-planned bad ways?
Friendly: Open Architectures or Open Source
Emergence impact?
Emergence is a negative property of "designed/architected" system; but must be "coped
with" in SoS.
Planned/controlled vs. "steered" via constraints and incentives: What mechanisms can
be applied to achieve the later
Contractual aspects: A guideline for how to write 6.2/6.3 contracts with un-priced parts;
without add-on contract approach. Capability vs. spec'd deliverable.
1. Stick with same team over large variations
2. Adaptive tactics?
Evolution as a process in which different entities (creatures) attempt to "survive and
thrive".
Approaches to adapting to un-anticipated change
1. Agile = dynamic adaptation
2. Critical Chain Theory = buffers along the chains to absorb impacts
Hard to funding things that have no explicit result
Genetic algorithms ?
612934202 – 16 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
IX. "Guided" Emergence (SoS Architecting for friendly/positive emergence)
-- Azad Madni
A. Add "Guided" (can't control) to make it a research topic
1. Guiding behavior without controlling
2. Achieved through learning algorithms?
B. Behaviors emerge in teams
1. Bad: fall apart because of lack of trust
2. Good: Old Boston Celtics (or Lakers vs. Pistons)
C. Communities of Practice vs. Communities of Interest
1. COP tend to be stove piped
2. Members of COIs don't "practice"
3. How to get COPs to cooperate.
D. Adaptive vs. Adaptable System
1. Avoid unnecessary expense
E. Twiki as an example?
F. How did "Charity" or "Philanthropy" evolve?
G. What are the canonical behaviors that can emerge? How to make it have value?
612934202 – 17 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
X. No single owner SoS -- Thomas Baehren and Gerry Nadler
A. e.g.,
1. Automotive transportation
2. WWW
3. Education
4. Health care
5. Mortgage and finance
6. Government
7. IR look-ahead for automotive market
B. What about funding in the non-DoD business model funding?
1. Business (automotive; consummer products)
2. GOs
a. NSF (for Multi-mission systems)
b. LA County?
c. Other?
3. NGOs
4. Philanthropic orgs for any of the "e.g."s (Gates Foundation for 3rd World Health Care?)
612934202 – 18 of 19
v 0.3 06/27/16
SoS Architecting Working Group – CSSE Convocation
Value X Difficulty (Averages)
612934202 – 19 of 19
v 0.3 06/27/16