Day 3 EVENT: CASE Wrap-Up
(Did not get headcount)
CASE Co-Chair Allen Arrington, Sierra Lobo, Inc., opened the session with the
question, “Where do we go from here?” He then introduced the Track chairs. They
were:
 Track 1 - Laura McGill, Raytheon Company: Complex Systems Development
 Track 2 - David Dress, NASA / Langley Research Center: Integration, Test, and
Verification of Complex Systems
 Track 3 - Sophia Bright, The Boeing Company: Program Management to Achieve
Robust and Resilient Systems
Theme: The purpose of this session was to summarize the outcomes of the various
sessions within the three tracks, and to lay out plans for upcoming events.
General comments on Track 2: Track chairs and members for the audience then
offered general comments on ideas developed during the proceedings. These were
separate from the Track summaries presented later in this section It was desirable to
present some of the inputs in different order than that in which they were offered at the
meeting, in order to group them with similar comments or with the comments they were
intended to expand upon.









David Dress: We need theoretical constructs that can be tested experimentally.
David Dress reminded the group of Dr. Felder’s challenge to thought: “Are we
building separate buildings of a common foundation?” (referring to the terms of
reference, processes and procedures of systems engineering of complex
systems).
David Dress: We should be personally respectful and intellectually disrespectful.
o Caution: Some people identify so closely with their technical areas they
take technical challenges as personal attacks.
Laura McGill: How do you achieve such an environment for personal respect and
intellectual disrespect?
David Dress: If we are not allowed to fail, we’ll never find the boundaries.
Steve D’Urso: Referred to the Malcolm Baldridge Award. “First-time quality” may
prevent you from finding the edge of the envelope.
Laura McGill: Develop case studies around our biggest flight test failures.
Laura McGill: A lot of what we do in aerospace is long-term work that goes on
and on.
Laura McGill: Champions (the leadership) vs adapters (who are they?)
1












Sophia Bright: How do we train the workforce that is going to be involved in
developing these complex systems?
Jimmie McEver: We can address this at the graduate level.
o How do other engineering schools / communities do it?
o Let’s give our engineers the tools to enable them to deal with other team
members
Unidentified: We can train students in systems engineering for complex systems
using current curricula. Don’t need to add any courses, just alter the way we
present current material.
Anna McGowan: Who needs to “get it” and what do they need to “get?”
o Having one person at the top who gets it isn’t sufficient.
o We need a vocabulary that extends beyond systems engineers.
Anna McGowan: One role of systems engineering is to facilitate conversations
and interactions across various disciplines.
Laura McGill: Consider having a real workshop to develop workforce skills at the
next CASE event.
o Need to talk with Tom Irvine about doing this at Sci-Tech in January.
Laura McGill: We’ve assigned a cultural architect to one of our teams as an
experiment.
Sophia Bright: Referred to the Complex Systems Primer, and said we want to do
this workshop and put pen to paper. We need to develop actionable responses
from these discussions.
David Dress: Lots of talk; what is the goal?
o A goal could be set at a very high level: What could we do to make the
development of a complex system successful?
David Dress: Identify all the attributes of a systems engineer at the workshop.
Bryan Herdlick: It is often the system engineer’s job to interface with program
managers and stakeholders. Need to be able to speak a similar language.
Jimmie McEver: The set of things we can do by getting together a couple of
times a year in the aerospace industry is limited. It is important to meet two days
twice a year, but should CASE meet in small working groups more often geared
toward producing useful products?



o This can be done as a working group, technical committee, or whatever.
David Dress: According to Adam Stetzler in his Mars Science Lab presentation
on Monday, in life you need stressors. Don’t become too comfortable.
David Dress: Post-traumatic growth: We’re doing CASE because we’re in
trauma. That trauma is causing us to have these meetings. This is an opportunity
for us to grow.
2







Steve D’Urso: Things don’t get done because they’re important. The get done
when they are important AND urgent.
o We must elevate things we want done to important and urgent.
David Dress: Recalled Kelly Johnson’s statement: “I will take the task but if you
interfere I will give it back.”
Bryan Herdlick: A systems engineer generally has one or more engineering
disciplines from which they have emerged.
Sophia Bright: A program manager looks at systems engineering from a
marketing perspective. What is the return on investment?
o The systems engineer should be involved in that marketing process. Do
they have the education to understand their economic benefit to the
company or enterprise?
Unidentified: Some (systems engineers) may not want to consider that economic
input, but they should have the opportunity to do so.
o Perhaps they should see the business case for their involvement.
Unidentified: Management needs to translate the executive vision into reality.
Unidentified: Leaders should not “permit” new ideas; they should “enable” new
ideas.
Track 2 Session Summaries:
The following are highlights from the Track 2 activities.
Session 1, Development of Testable Requirements at the SoS / Capability Level:
 Panel focused on capability to perform a mission, not a requirement.
o Grade systems on whether they do what we want them to do vs. whether
they meet “requirements.”
 If we suspect the system isn’t going to work the first time, we often change
requirements.
 For systems of systems, corporate citizenship is required.
o Capability must be focused on the desired effect (e.g., making the enemy
capitulate).
 Pilots aren’t going to provide the system requirements. They can provide the
requirements for the operational scenario.
 Characterization and test are different.
o Characterization is a series of experiments to understand the system’s
behavior.
 Hardware in the loop (HWIL) is often difficult to predict.
 Enemies that are not well-trained; they are difficult to predict.
 Get feedback from stakeholders on systems’ performance in the field.
 Some things aren’t fully testable.
3
o Humans react differently under real battlefield conditions.
Session 2, Cost Optimization and Risk Mitigation through Strategic Early Verification:
 What are we really trying to do?
o Have we done this in the past?
 What can we do more upstream in the development cycle (to move V&V to the
left)?
 Where is testing done today?
o Where could we do it?
o Where should we do it?
 Example of a complex problem: Use of composites in Boeing 787
o Hard to generalize methodology
o Had to make FAA part of the process in getting certification
 Think in terms of CFD and test, not CFD vs. test.
 Benefits of early testing in a program should be documented and marketed.
Session 3, Workforce Skills Development:
This was a joint session with Track 3, Program Management to Achieve Robust and
Resilient Systems, chaired by Sophia Bright, The Boeing Company. The session had no
presentations and consisted mainly of unstructured discussion among the panelists and
audience. For a summary of that discussion, refer to the report on Track 2 / Track 3,
Session 3.
Session 4, Integration of Experimental Ground Test (GT) Capability and Computational
Methods (Part 1)
 Where are we in the relationship between GT and CFD?
o Where are we relative to other countries?
o How did we get there?
o Where do we need to be?
o How do we get there?
 Must refine both GT and CFD for the future.
 Must understand the physics in the wind tunnel vs flight test.
 Turbulence modeling in CFD is still black magic.
 Experimental testing and CFD are now about 50-50.
o Will probably move toward CFD.
o Experimental testing is reliable.
 Limited capability to measure all parameters.
o CFD yields uncertainty at the margins.
 The role of wind tunnels is changing from developmental to characterization.
4


Need the right models and the right tools to deal with the new configurations
being developed as we go forward.
o We will need wind tunnels for the foreseeable future.
o CFD won’t be available to replace wind tunnels.
The zero tolerance – zero failure mindset flies in the face of failures that occur in
testing.
o Needs to be left behind us.
General Closing Statements:
Laura McGill gave the final comments for the CASE conference.
 The objective of CASE is to bring people together who are responsible for the
development of aerospace systems that fly.
 The “C” (complex) in CASE has taken on a life of its own.
o Let’s get back to systems integration and engage more industry
practitioners.
 Maybe we should invite international participation.
o Ms. Bellman suggested progress in Germany that might be illuminating.
 At Sci-Tech we want to have a similar session and a virtual panel, with more
emphasis on solutions and best practices.
 Have CASE located with the AIAA Propulsion and Energy Forum and Exposition
2014 in Cleveland, Ohio next year.
 Looking for leadership for CASE in 2014.
 Response to suggestion for webcast sessions.
o People may be less frank.
o Such sessions are expensive.
 Bryan Herdlick suggested building a topic map with comments here, and using
that to distinguish ourselves.
 There was a reminder of the Complex Systems Design & Management (CSD&M)
2013 Conference in Paris, France in December, 2013
5