Stevens-USC SERC:
A DoD University Affiliated Research Center (UARC)
Barry Boehm, USC
USC-CSSE Annual Research Review
March 16, 2009
Outline
• Nature of UARCs
• SERC Overview
– SERC organization and vision
– SERC research strategy
• Initial tasks
– SysE Effectiveness Measures Assessment (EM)
– SysE Methods, Processes, Tools Evaluation (MPT)
• Workshop objectives and approach
3/16/2009
2
What is a UARC?
1.
University Affiliated Research Centers are “not-for-profit, private sector
organizations affiliated with, or part of, universities or colleges that
maintain essential research, development and engineering capabilities
needed by sponsoring DoD components.”
2.
They maintain long-term, strategic relationships with sponsoring DoD
components in specific core areas and operate in the public interest, free
from real or perceived conflicts of interest.
3.
UARCs are financed through long-term, non-competitive contracts
awarded by sponsoring DoD components for specific core work.
3/16/2009
3
Several Existing UARCs
1.
Johns Hopkins University APL; 4,300 people; annual funding $680M
2.
UC Santa Cruz with NASA Ames – information technology, biotechnology, nanotechnology,
computer science, aerospace operations, astrobiology, and fundamental biology. Has a
Systems Teaching Institute with San Jose State University and UCSC to teach through hands
on experience on research projects.
3.
Penn State University Applied Research Laboratory for the Navy with focus on undersea
missions and related areas; strategic partner with NAVSEA and ONR; established 1945; has
>1000 faculty and staff
4.
University of Washington APL – acoustic and oceanographic studies ; established in 1943
5.
UC Santa Barbara Institute for Collaborative Biotechnology – Army Research Office;
partnered with MIT and Cal Tech – focus on biologically-derived and biologically-inspired
materials, sensors, and information processing …
6.
University of Texas UARC started in 1945 focuses on sonar, acoustics, software system
research, satellite geodesy, active sonar, …; now has 600 people on staff
7.
USC Institute for Creative Technology – US Army; focus on virtual reality multimedia
applications for training, C4ISR
3/16/2009
4
SERC Organization
Lead organizations
Members
•
•
•
•
•
•
•
Auburn University
Air Force Institute of Technology
Carnegie Mellon University
Fraunhofer Center at UMD
Massachusetts Institute of
Technology
Missouri University of Science and
Technology (S&T)
Naval Postgraduate School
•
•
•
•
•
•
•
•
•
•
Pennsylvania State University
Southern Methodist University
Texas A&M University
Texas Tech University
University of Alabama in Huntsville
University of California at San Diego
University of Maryland
University of Massachusetts
University of Virginia
Wayne State University
As the DoD Systems Engineering Research-University Affiliated Research Center, SERC will be
responsible for systems engineering research that supports the development, integration,
testing and sustainability of complex defense systems, enterprises and services. Its members
are located in 11 states, near many DoD facilities and all DAU campuses.
3/16/2009
5
SERC Organization - II
Dr. Dinesh Verma
Executive Director
Dr. Art Pyster
Deputy Executive Director
Julie Norris (acting)
Director of Operations
Dr. Barry Boehm
Director of Research
Pool of more than 140 Senior Researchers and hundreds of
research faculty and graduate students from across members
Stevens’ School of Systems and Enterprises will host the SERC at Stevens’ Hoboken, NJ, campus. Stevens’ faculty
engagement will be complemented by a critical mass of systems engineering faculty at USC. A fundamental tenet
of SERC is its virtual nature – each of its 18 members will be a nexus of research activities. All research projects
will be staffed by the best available researchers and graduate students from across the members.
3/16/2009
6
Rough Financial Model
• Minimum of $2M/year
– Can add more funded tasks within contract
• First year: two specified tasks
– 1. SysE Effectiveness Measurement: EM (USC lead; $500K)
– 2. SysE Methods, Processes, Tools Assessment : MPT (Stevens lead;
$350K)
• Further tasks proposals being submitted
– In response to SERC Research Strategy Thrusts
• Other Govt. sponsors can sole-source through UARC
– Procedures being worked out
– Industry, FFRDC collaboration modes being worked out
3/16/2009
7
SERC Vision and Perspective
Vision
DoD and IC systems achieving mission outcomes – enabled by
research leading to transformational SE methods, processes,
and tools.
Perspective
The SERC will be the primary engine for defense and intelligence
community SE basic research. In doing so, the SERC will:
1.Transform SE practice throughout the DoD and IC communities by
creating innovative methods, processes, and tools that address critical
challenges to meeting mission outcomes (what),
2.Become the catalyst for community growth among SE researchers by
enabling collaboration among many SE research organizations (who),
3.Accelerate SE competency development through rapid transfer of its
research to educators and practitioners (how).
3/16/2009
8
SERC Research Strategy Thrusts
1. Enterprise Responsiveness
2. Systems Science and Complexity
3. Human Capital
4. Program and Systems Engineering Integration
5. Life Cycle Systems Engineering Processes
3/16/2009
9
1. Enterprise Responsiveness Thrust
– Collaboration. Research innovative collaborative
technologies to dramatically improve systems engineering
is performed, especially for teams that are geographically,
culturally, and linguistically diverse.
– Modeling and Simulation. Research how to more rapidly
and easily develop modeling and simulation tools that help
validate concepts of operation, architectures, and other
key systems engineering artifacts.
– Resilient Program. Research how to architect and
implement a program that is resilient in the face of a wide
range of potentially harmful changes, such as budget cuts,
requirements turbulence, failure of needed technology
maturation, and program redirection
3/16/2009
10
1. Enterprise Responsiveness Thrust - II
– Resilient System. Research how to architect and
implement a system that is resilient in the face of a wide
range of threats, both cyber and physical.
– Producibility. Research how to dramatically decrease the
amount of effort and schedule required for implementing
new software-intensive systems.
– Parsimony. Research how to reduce the amount and size
of SE artifacts, e.g., determining minimum-essential
architectural views number, detail, rigor, and capture time.
– Strategic Assessment. Research improved process models
and decision criteria for better SE support of strategic and
enterprise decisions.
3/16/2009
11
2. Systems Science and Complexity
– Composition. Research how to determine key properties
of a system (assurance, scalability, availability,
producibility, interoperability, lethality, resilience, …) when
(a) the properties of its subsystems are known, (b) the
system architecture that links those subsystems is known,
and (c) the concept of operations is known.
– Emergence. Research how to manage emergence in
requirements, technology, and system usage in a way that
is beneficial or benign rather than disruptive to
development and acquisition programs; and how to
anticipate and mitigate emergent behaviors.
– System Conceptualization. Research approaches to better
visualize and define system concepts that enable
collaboration among multiple stakeholders.
3/16/2009
12
2. Systems Science and Complexity - II
– Holistic SE. Investigate systems thinking methodology and
its application to DoD enterprise and capabilities planning.
– Validation. Research how to improve the early and
continual validation of complex, expensive systems.
– Transformation. Take systems engineering and systems
thinking MPTs that are applied to critical contemporary
issues outside the DoD domain (such as creating a more
cost effective health care system for the U.S. general
population) and transform them so as to be directly
applicable to problems in the DoD domain.
3/16/2009
13
3. Human Capital
– Collaboration and Education. Extends Enterprise
Collaboration thrust element to include education.
– Acceleration. Research how to accelerate the growth of
the systems engineering workforce for both DoD and its
contractors.
– Dispersion. Research how to rapidly and effectively grow
SE competencies in project managers, contracting officers,
and other acquisition personnel besides SEs for whom SE
competency could enhance their performance.
– Staffing. Research how to staff a program with the right
mix of systems engineers as a function of program type,
size, complexity, risk profile, and other system and system
acquisition characteristics.
3/16/2009
14
4. Program and Systems Engineering Integration
– SE Effectiveness. Research SE program execution
performance, return on investment, and metrics to
demonstrate and improve value of SE.
– Value. Determine how much SE is appropriate in specific
circumstances to help ensure that the right amount of
resources is available with the right skills.
– Economics. Research how to better estimate systems
engineering cost and schedule for complex programs and
complex systems.
– Assessment. Research improved process models and
decision criteria for better SE support of management
decisions across all life cycle phases (currently funded, in
part, as the EM TTO)
3/16/2009
15
4. Program and Systems Engineering Integration - II
– Maturity and Readiness. Research how to quantitatively assess the
maturity of SE artifacts. Be able to inform technical, management,
governance, and investment decisions using that assessment.
– Teambuilding. Research how to better stimulate and support effective
distributed team performance among SEs and other stakeholders.
– System of Systems, Networks, and Enterprises. Research how to manage
the acquisition and development of a SoS, a network-centric system, or
an enterprise system where there is typically no single “owner” for all the
component systems, components emerge and disappear dynamically,
and components often have conflicting acquisition strategies.
– Services. Research how to effectively manage the acquisition and
development of a service as a function of type, scope and complexity,
uncertainty, risk, and other characteristics.
3/16/2009
16
5. Life Cycle Systems Engineering Processes
– Life cycle models. Research the best life cycle models for
the acquisition and development of a SoS, a networkcentric system, a service, or an enterprise system as in
Thrust 4.
– Balance. Research life cycle models for the acquisition and
development of a system which begins as a quick response
built using agile methods and then evolves into a system
that is deployed on a larger-scale with radically different
maintainability, availability, and similar attributes.
3/16/2009
17
5. Life Cycle Systems Engineering Processes - II
– Architecting. Research how to create architectures that
demonstrably and quantifiably support key system
properties such as assurance, scalability, availability,
producibility, interoperability, lethality, and resilience, and
how to derive architectural component characteristics and
constraints from desired system properties.
– Landscape. Create a landscape of SE MPTs with respect to
key characteristics such as their maturity, the types of
systems to which they are best applied (e.g., networkcentric vs. embedded or enterprise vs. platform), and their
value in actual application. Use insights from this
landscape to help guide where SE MPT research is needed.
(currently funded, in part, as the MPT TTO)
3/16/2009
18
SERC Research Initiation Strategy
FY09 Focus
SE external
Factors/Context
Mission
Drivers
Baseline SE
MPTs
Research to
Determine
Validate
Research
to
Determine
Validate
Address Specific
Modified MPTs
MPT
Research
to
Determine
Validate
Address
Specific To Modified
MPTs
MPT
SE issues
address
gaps
Effectiveness
Research
to
Determine
Validate
Address
Specific To Modified
MPTs
MPT
SE issues
address gaps
Effectiveness
Address
Specific To Modified
MPTs
MPT
SE issues
address gaps
Effectiveness
SE issues
To address gaps
Effectiveness
Determine SE
Effectiveness
And Value
Measures
Early focus on a solid baseline and quantifiable, observable
Measures to enable future demonstration of improvement
3/16/2009
19
Outline
• Nature of UARCs
• SERC Overview
– SERC organization and vision
– SERC research strategy
• Initial tasks
– SysE Effectiveness Measures Assessment (EM)
– SysE Methods, Processes, Tools Evaluation (MPT)
• Workshop objectives and approach
3/16/2009
20
First Two Task Orders
1.
2.
“Assessing Systems Engineering Effectiveness in Major Defense
Acquisition Programs (MDAPs)”
–
Government lead: OUSD(AT&L)/SSE
–
Barry Boehm (USC) task lead, with support from Stevens, Fraunhofer
Center, University of Alabama at Huntsville
“Evaluation of Systems Engineering Methods, Processes, and Tools
(MPT) on Department of Defense and Intelligence Community
Programs”
–
Government lead: DoD
–
Mike Pennotti and Rich Turner (Stevens) task leads, with support
from USC, University of Alabama at Huntsville, USC, Fraunhofer
3/16/2009
21
Coordinated approach
•
Synergetic contribution to Sponsor’s
SysE effectiveness goals
•
– Common collaborators, battle rhythm with
regular e-meetings, shared workshops
– Research Integrity Team
– Best practices + progress monitoring
and improvement
•
Common context
– Domains of interest and levels of
organization
3/16/2009
Coordinated management
•
Coordinated Technical Approaches
– Definitions, evaluation criteria and
methods
– Cross-feed/peer review ongoing results
22
Workshop Objectives and Approach
• Review, improve on EM and MPT task objectives
– Utility of results to DoD constituencies
– Identification of needs-capabilities gaps
– Identification of promising research directions
• Review, improve on EM and MPT task approaches
– Survey and evaluation approaches, criteria, and instruments
– Involvement of key stakeholders
• Contractors, program/portfolio managers, oversight organizations
– Coverage of DoD application domains
• Initial EM priority: Weapons platforms
• Initial MPT priority: Net-centric services
– Test and evaluation of results
• Capture participants’ relevant experience
3/16/2009
23
SERC Research Methodology
3/16/2009
24
Example: Mission and Stakeholder
Requirements and Expectations
Can we develop a transformative, interactive,
and graphical environment to bring
stakeholders (warfighters and analysts)
together with SEs to develop a graphical/visual
conops in an extremely agile manner?
Every study on failed
projects, refers to inadequate
requirements, and
understanding of the “real”
problem: large projects and
small projects; defense or
commercial
3/16/2009
Understand and synthesize
advances in multi-media
technologies and interactive
graphics; gaming
technologies; real options
theory
25