Reflections on Lockheed Martin`s 20 year affiliation with USC CSSE


Reflections on Lockheed Martin’s

20 year affiliation with USC CSSE

Copyright 2013 Lockheed Martin

Historical Perspective

Software Development in military systems reached a

“crisis” in the 80’s

Size and Complexity were exponentially increasing

There were a plethora of software languages

DoD Programs were costing twice as much and taking 50% longer than planned





















F/A 18CD









F-16C/50 T3




1970 1980 1990 2000

*Source: Rob Gold, OSD/DDR&E, NDIA Strategic Software Summit, October 2006


Percentage of Functionality Implemented in Software












F-4 (1960) A-7 (1964) F-111 (1970) F-15 (1975) F-16 (1982)

Program (Year)

B-2 (1990) F/A-22 (2000) F-35 (2006)

Copyright 2013 Lockheed Martin

What Did We Do About It?

• We invested in Software Technology

Research (SW Tech Center w/ Dr. Win Royce)

• We invested in Software Producibility

Initiatives (Reuse, Autocoders, Rational …)

• We brought together Subject Matter Experts

(SW Task Force and Working Groups)

• We created and joined consortiums and associations






– Etc.

Copyright 2013 Lockheed Martin

In June 1993, Lockheed became a charter affiliate member of CSE

• AT&T Bell Laboratories

• Electronic Data Systems Corporation

• Hughes Aircraft Company

• Institute for Defense Analyses

• Litton Data Systems

• Lockheed Corporation

• Northrop Corporation

• Science Applications International


• Software Productivity Consortium


• Teledyne, Inc.

• TRW, Inc.

• U.S. Air Force Rome Laboratory

• U.S. Army Research Laboratory

• And its affiliate via a Technology cooperation Agreement:

CMU Software Engineering Institute

With this breadth of collaborative opportunities and knowledge sharing, everyone comes out ahead

Copyright 2013 Lockheed Martin

Initial Benefits of CSE

• As an affiliate of CSE we also became an affiliate of COCOMO®

• We provided data points (it wasn’t easy to solve proprietary issues)

• We collaborated with our own work

• Walt Johnson on COTS SW estimation

• George Bazoki on SW size estimation

• We participated in workshops

• We gained insight into better software development estimation

Copyright 2013 Lockheed Martin

Other Research Topics of Interest

In 1996 and 1997 CSE brought in many of the thought leaders in Systems and Software Architecture

• Eb Rechtin, Mark Maier, Bob Balzar, Dave Wile and others

• This is a critical area of expertise needed by Lockheed

Martin on many large scale programs

In 1998 CSE brought a theme of Rapid Application


• This was a precursor to Agile Development

• Concepts were consistent with Lockheed Martin Skunk

Works and Kelly Johnson’s 14 rules

In 1999 and 2000 CSE ventured into Model Based

Architecting and Software Engineering

• The best thing about architectural modeling standards is that there are so many to choose from

• Modeling (and simulation) means different things to different people based on the context that it is being applied

Copyright 2013 Lockheed Martin

“Be Quick, be quiet, be on time.”

“The Skunk Works is a concentration of a few good people solving problems far in advance – and at a fraction of the cost – by applying the simplest, most straight-forward methods possible to develop and produce new products.” - Kelly Johnson

Copyright 2013 Lockheed Martin

Kelly’s 14 Rules

1. The Skunk Works manager must be delegated practically complete control of his program in all aspects. He should report to a division president or higher.

2. Strong but small project offices must be provided both by the military and industry.

3. The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people (10% to 25% compared to the so-called normal systems).

4. A very simple drawing and drawing release system with great flexibility for making changes must be provided.

5. There must be a minimum number of reports required, but important work must be recorded thoroughly.

6. There must be a monthly cost review covering not only what has been spent and committed but also projected costs to the conclusion of the program. Don't have the books ninety days late and don't surprise the customer with sudden overruns.


Copyright 2013 Lockheed Martin

Kelly’s 14 Rules (cont.)

7. The contractor must be delegated and must assume more than normal responsibility to get good vendor bids for subcontract on the project. Commercial bid procedures are very often better than military ones.

8. The inspection system as currently used by the Skunk Works, which has been approved by both the Air Force and Navy, meets the intent of existing military requirements and should be used on new projects. Push more basic inspection responsibility back to subcontractors and vendors. Don't duplicate so much inspection .

9. The contractor must be delegated the authority to test his final product in flight. He can and must test it in the initial stages . If he doesn't, he rapidly loses his competency to design other vehicles.

10. The specifications applying to the hardware must be agreed to well in advance of contracting . The Skunk Works practice of having a specification section stating clearly which important military specification items will not knowingly be complied with and reasons therefore is highly recommended.


Funding a program must be timely so that the contractor doesn't have to keep running to the bank to support government projects.

Copyright 2013 Lockheed Martin

Other Research Topics of Interest (cont.)

In 2000 CSE hosted an Executive Workshop on Evolutionary

Acquisition and Spiral Development

• “They keep using that word - I don’t think it means what they think it means”*

• Government, Industry and Academia collaborated to understand the benefits and drawbacks

1. Objectives,

Alternatives, and



4. Planning and


In 2000 the concept of a Systems Engineering Cost Model emerged at the 15 th annual COCOMO Forum (COSYSMO)

• Gary Thomas presented the concept

• Ricardo Valerdi took it on as his PhD research project

• Garry Roedler served as focal point for Lockheed Martin

In 2002 and 2003 CSE conducted Executive Workshops on

Agile Development

• This had the largest impact on Lockheed Martin of any other research topic at CSE

• Hybrid application of Agile Practices have been widely used within Lockheed Martin ever since this introduction

*Adapted from a line in the movie The Princess Bride

Copyright 2013 Lockheed Martin

2. Risk

Analysis and


3. Product


Integrated Systems and Software Engineering

In 2002-2005, much of the aerospace industry was moving towards integrated

Systems and Software Engineering Processes

• SEI came out with CMMI

• ISO was harmonizing ISO/IEC 15288 and 12207

• DoD AT&L Systems Engineering added Software to the Organization

• NDIA added Software to the System Engineering Division

• STC changed to Systems and Software Technology Conference

• Software Productivity Consortium became Systems and Software Consortium

At the suggestion of the Affiliates, USC merged CSE and SAE as CSSE

Copyright 2013 Lockheed Martin

Continuing Benefits of CSSE to Lockheed Martin

• Engagement of the Systems Engineering Community

• Establishment of a Systems and Software Architect Development Program

• Certificate Program

• Masters Degree

• UARC – Systems Engineering Research Center (w/Stevens Inst. And others)

• Ongoing research for our primary customer – DoD

• Understanding and Insight into the Incremental Commitment Model

Through Collaboration, USC CSSE And Its Affiliates Have

Truly Developed a Win-Win Relationship

Copyright 2013 Lockheed Martin

Personal Reflection

What have I personally gained from my affiliation with USC CSSE?

• Affirmation

• Knowledge

• Insights

• Friendships

• Good Times

• Satisfaction

• And Great Food!

Contact info:

Gary Hafen

System Engineer Principal

Lockheed Martin Aeronautics Company

(661) 572-7178 (Office)

(661) 878-4175 (Cell)

Copyright 2013 Lockheed Martin