IIS Approach to
Process Right-Sizing
Lawrence Goldstein
Southern California SPIN
February 6, 2004
Agenda
• Northrop Grumman Internal Information Services
– What is it?
– Geographic scope
– Some historic data on software process improvement
at IIS
• IIS Process Right-Sizing
– General approach
– Process size
– Size-a-matic™
– Tailoring options
– Waivers
– Special cases
– Maintenance
– Web / RAD
– Summary
2
Who is Northrop Grumman IT IIS?
•
Northrop Grumman is a $25 billion global
defense company
–
–
–
•
Operating in 50 states and 25 countries
Approximately 120,000 employees
Defense product focus
–
–
–
–
Advanced aircraft, shipbuilding and space
technology
Systems integration
Defense electronics
Information technology
Northrop Grumman’s Internal Information
Services (IIS) is the business unit
responsible for providing the IT
infrastructure to the rest of the corporation
– Our only customer is the rest of Northrop
–
Grumman
Our mission is to provide strategic
information solutions and technologies which
contribute to Northrop Grumman’s
competitiveness
3
IIS Geographic Scope
IIS Locations – North America
Enfield, NS
Toronto, ON
Amherst, NY
Reading, MA
Norwalk, CT
Bethpage, NY
Troy, MI
Salt Lake City, UT
Clearfield, UT
Rolling Meadows, IL
Aurora, CO
Baltimore, MD
Greenbelt, MD
Bellevue, NE
Kettering, OH
Sunnyvale, CA
Palmdale, CA
Pt. Mugu, CA
Century City, CA
El Segundo, CA
Hawthorne, CA
Redondo Beach
Torrance, CA
San Pedro, CA
Carson, CA
Azusa, CA
San Bernardino, CA
San Diego, CA
San Jose, CA
Gaithersburg, MD
Columbia, MD
Bethesda, MD
Colorado Springs, CO
College Park, MD
Goleta, CA
Woodland Hills, CA
Northridge, CA
GA
Huntsville, AL
Tempe, AZ
Albuquerque
Sierra Vista, AZ
Herndon, VA
Newport News, VA
Reston, VA
Fairfax, VA
Chantilly, VA
Charlottesville, VA
Wheeling, WV
San Angelo, TX
Garland, TX
Dallas, TX
Fort Hood, TX
Austin, TX
San Antonio, TX
Pascagoula, MS
Avondale, LA
Ocean Springs, MS
Lake Charles, LA
St. Augustine, FL
Apopka, FL
Melbourne, FL
Stuart, FL
Over 3,700 Employees
4
IIS Process Improvement Path
1992
1994
1996
1998
2000
2002
2004
2006
TRW*
Vought*
Litton*
Grumman*
NISC*
L2
L2
CBA-IPI
L2
CBA-IPICBA-IPI
L3
CBA-IPI
CMMI
L3
Westinghouse*
Rolling Meadows*
Newport News*
* heritage organizations
IIS Infrastructure
(Operations, N/W, hardware, etc.)
5
IIS Process Right-Sizing
General Approach
6
Number of Projects
The Problem
0
0
Project Labor hours
100,000+
7
Number of Projects
The Problem
New Development
Lawson
SAP
PeopleSoft
Web
PC
0
0
Project Labor hours
100,000+
8
Number of Projects
The Problem
New Development
Lawson
SAP
PeopleSoft
Web
PC
0
0
Project Labor hours
100,000+
9
IIS Process Tailoring Philosophy
• The main purpose behind the process is to
protect the project from failure
– Deliver the product on-time, agreed to cost,
agreed to quality
• If less work is needed to create the product, then
less process should be needed to protect the
project
– Less to go wrong
– Consequences of failure are lower
– High dollar value projects with little labor are
treated as special cases
But …
• The process also is needed for organizational
purposes
– Some process is always required regardless of
amount of work to create the product
10
The Devil is in the Details
11
IIS General Approach
The project team determines whether the project is
Minor, Small, Medium, or Large.
Minor:
<151
Small:
151-1800
Medium:
1801-7200
Large:
>7200
Adjust Size
Size
12
Determine Project Size 1 of 2
• Determine and describe the effort
estimating approach and data to be used
and how it will be used
– Others should be able to recreate
estimate from data provided
• Estimate size of software work products
• Calculate estimated effort
– Convert work product size to labor hours
to produce the products
– Size any other project activities and
convert them into labor hours
13
Determine Project Size 2 of 2
• Use Size-a-matic™
– Effort Estimate
– Key-in total project effort
estimate (hours)
– Size Adjustments
– Evaluate risk levels for 6
key areas
– Describe rationale for
levels selected
– Resolves the issue of a
“large Minor” vs. a
“small Medium”
14
Tailoring Building Blocks
• Regardless of development lifecycle chosen by a project,
•
•
•
•
the same general sets of activities always have to be
done
The differences usually revolve around the ORDERING of
those activities and the scale of those activities
All lifecycles have to
– Define requirements
– Design solutions
– Construct code
– Discover defects
– Etc.
So why not define plug-and-play processes , tailorable
by size, for the different project lifecycles to use?
Then lifecycle definition becomes a case of pre-defined
process ordering with minimal process development
required
15
Tailor, Tailor, Tailor
Project size is used to identify the specific activities
and deliverables required for each CMMISM Process
Area involved in the project.
RM
Minor
Small
Medium
Large
PP
PMC
M&A PPQA CM
•Do
•Do
•Do
•Do
•Do
•Do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Do
•Do
•Do
•Do
•Do
•Do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Do
•Do
•Do
•Do
•Do
•Do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Do
•Do
•Do
•Do
•Do
•Do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
•Don’t do
…
16
And Then There Are Waivers
• Waivers provide the option for tailoring “outside of
process boundaries”
– Waive compliance with required process
– Based on a substantiated business case which
requires deviation from the standard
– Approved by someone high enough in the
management chain to understand the implications
• If you define your process set correctly, waivers
should rarely be needed
– All the common tailoring should be pre-defined,
balanced, with the trade-offs well understood and
accepted
– Waivers should definitely be the exception rather than
the rule
17
IIS Process Right-Sizing
Special Cases
18
Special Cases
• Special circumstances require special methods
– Maintenance of production systems
– Web / Rapid Application Development (RAD)
• A separate lifecycle methodology is applied
– Pre-tailored to address the special circumstances
with just the right amount of process in all the right
places
– Project sizing is not used to determine tailoring
because the lifecycle is pre-tailored
– Less project-by-project options means simpler to plan
Process Costs
Risk of Failure
19
2003 Distribution by Project Size
WADM
9%
MINOR
7%
SMALL
19%
UMBRELLA
46%
MEDIUM
9%
LARGE
10%
20
Maintenance Umbrella Projects
1 of 4
• Covers “level of effort” maintenance work
•
•
•
efforts
– Application systems in production
– Supported by pre-allocated labor pool of
experts
– Budgeted on an annual basis
Each new customer request is either
– New or changed requirement
– Reported system defect
– Service request (data load, table change,
etc.)
Groups minor-sized work efforts occurring
within a specified period of time, range of
hours, and/or cost amount
– Annual renewal
– Pre-determined fixed cost (e.g. 500 hours)
– No upper limit on aggregate work covered,
only on each discrete work effort
Special maintenance umbrella project plan
template
21
Maintenance Umbrella Projects
2 of 4
• Addresses a collection of minor work efforts on
production systems which are related in one or
more ways
– Common customer
– Common system
– Common hardware
– Common support organization
22
Maintenance Umbrella Projects
3 of 4
• Allows for the one-time definition of a standard
approach pertaining to all of the related work efforts
– Common roles and responsibilities
– Common PPQA approach
– Common CM approach
– Common RM approach
– Common RSKM approach
– Etc.
• Requires more process than any single minor work
effort, but less than if all work performed were
treated as individual minor projects
– Recognizes that more process is needed to protect
production software than is required for minor work
efforts
– Better protection for less cost
23
Maintenance Umbrella Projects
4 of 4
• Work requests which size as larger than Minor are
spun off as separate projects
– Separate project plan
– Separate funding
– Separate QA
– But can still use common CM, RSKM, etc.
Not Minor
Customer Request
Minor
Sizing Process
24
Eat at Joe’s
Reg.
Double
1/3 lb
Hamburger
$1.25
$1.75
$2.50
Soy-burger
$1.05
$1.50
$2.00
Medium
Large
Small
Soda (coke, orange, root beer, sprite)
$ .75
$ 1.25
$ 1.85
Milkshake (chocolate, vanilla)
$ 1.75
$ 2.25
$ 3.00
Malted (chocolate, vanilla, peanut butter)
$ 2.25
$ 2.75
$ 3.50
25
Jo’s Estimating Table
Simple
Average
Complex
DB Segment Change
10 hrs
15 hrs
25 hrs
DB Segment Add
15 hrs
20 hrs
30 hrs
DB Segment Delete
10 hrs
12 hrs
15 hrs
< 5 elmts < 10 elmts < 20 elmts
Screen Change
20 hrs
30 hrs
50 hrs
Screen Add
30 hrs
45 hrs
60 hrs
Screen Delete
10 hrs
15 hrs
20 hrs
26
Umbrella Special Tailoring Example
Standard process
– Must perform a Post-Implementation Evaluation (PIE)
n days following implementation
Umbrella process
– Allows for the “bundling” of multiple small changes
into quarterly PIEs instead
– Major changes are still be evaluated separately
27
Web Application Development
Unique set of problems
and opportunities
Q: How do you marry
structured process to
the agile philosophy?
A: Very carefully!!
28
Basic Terminology
• WADM™ – Web Application Development Methodology
– A lightweight, rapid application development process for use
in
SM
•
•
•
•
web application development, which is SW CMM / CMMI level
3 compliant
Lightweight
– Minimal amount of intermediate “stuff”
– Steps
– Artifacts
– Reviews
Rapid application development
– A series of short, incremental development cycles
Web application
– An application, possibly linked to one or more databases, which
uses the web as the user interface
SW CMM / CMMISM level 3 compliant software development
methodology
– A structured set of processes and procedures, which has the
goal of reliably producing high quality software and other work
products, on schedule, and within cost.
29
WADM™ Overview
1 of 2
• The WADM™ is a full lifecycle process for web development
•
•
projects, and for non-web development projects having similar
characteristics
It calls for a high level of customer involvement, short
development cycles, and rapid development.
General criteria for using the lifecycle
– A high degree of customer involvement
– Sufficient size and work scope to allow functionality to be
allocated to progressive production releases
– Simple design
– A requirement for rapid delivery of functioning releases (get
something up and running quickly)
– The overall program can be compartmentalized into stand-alone
projects with small teams of developers
30
WADM™ Overview
Read the label!
2 of 2
• The WADM™ is not
appropriate if the project
– Has high algorithmic
complexity
– Involves integrated
database design
– Is solely maintenance
– Has low customer
involvement
– Involves conversion or
platform migration
– Must be implemented as a
whole, at one time
– Cannot be divided into
coherent functional units
that can be implemented in
successive releases
– Plans to subcontract all or a
portion of the work
31
WADM™ Key Elements
•
•
•
•
Customer Focus
– Customer has ownership of, and a high degree of visibility into, the
project.
– High level of customer commitment and involvement in the
development team.
– Customer can add, delete, modify and/or reprioritize functions as the
application is being developed.
– Meets customer’s real demand due to the customer’s degree of
influence and control over the development effort.
Short Time Span
– The development cycle is divided into successive, short-duration builds.
– The first build and each successive build are usable.
Low Cost
– No unnecessary functions are allowed. Functions that are not required
are not developed.
– Project documentation and reviews are minimal.
– Works with virtual teams.
Produces High Quality Product
– Product has minimum number of defects.
– Testing and defect prevention are integral to the process.
32
WADM™ Structure
Proposal
Phase
Startup
Phase
Proposal
•SOW
• XXX
• XXX
•Budget
• XXX
• XXX
•Schedule
• X----• X-----
Kick Off Activities
•Project Plan
•Form Team Activities
•Kick Off Meeting
Startup Meeting
• Elicit Initial Set of
Use Cases and
Constraints
• Initial Release Plan
Development Cycles
1 of 5
Releases of
Usable Software
Development
Cycle 1
Release 1.0
Development
Cycle 2
Release 2.0
Additional Development Cycles
Development
Cycle n-1
Release n-1
Development
Cycle n
Release n
33
WADM™ Structure
Proposal
Phase
Startup
Activities
Development
Cycle
Design
Completed
Cycle
Build
Acceptance
Testing
2 of 5
The
Planning
Game
Standup
Meeting
Construction
Cycle
Acceptance
Test
Development
Construction
Design Cycle
Planning
Meeting
34
WADM™ Structure
3 of 5
• Three nested cycles of iterative activities:
– The Development Cycle
– The Design Cycle
– Enclosed within the Development Cycle
– The Construction Cycle
Proposal
Phase
Startup
Activities
Development
Cycle
Design
Completed
Cycle
Build
Acceptance
Testing
– Enclosed within the Design Cycle
• Every cycle in the WADM™ begins with a
•
•
The
Planning
Game
Standup
Meeting
Construction
Cycle
Acceptance
Test
Development
Construction
Design Cycle
Planning
Meeting
planning / management meeting followed by
the work activities of the cycle.
WADM™ projects are typically composed of
several sequential Development Cycles
Project Management, Risk Management, Issue
Management, Requirements Management,
Configuration Management, and
Measurements & Analysis activities occur
throughout
35
WADM™ Structure
•
Development Cycle
– Objectives
– To ensure that the
1 Cycle
~Every
Month
–
–
customer needs are
understood by all parties
(identify customer
requirements)
To ensure that the
customer needs are
prioritized by the
customer
To ensure that the
customer needs are met
(validate and verify the
requirements)
• Design Cycle
– Objectives
4 of 5
1 Cycle
~Every
Week
– To design an application
capability to meet a
selected subset of the
customer requirements
– Duration: ~1 Week
– Made Up of:
– 1 Design Cycle Planning
–
Meeting
4-5 Construction Cycles
– Duration: ~ 1 Month
– no Development Cycle
should exceed six weeks
in duration
– Made Up of:
– 1 Planning Game
– 3-5 Design Cycles
36
WADM™ Structure
5 of 5
• Construction Cycle
– Objective
1 Cycle
Per
Day
– Code and Unit Test
Modules of the Application
of the Design Cycle
– Duration: Daily
– Made Up of:
– Daily standup meetings
– Daily construction activities
– Continuous integration
37
Planning Game’s 5 Standard Moves
Project
Constraints
Complexity
1
Project
Goals
Risks
Identify
Use Cases
Estimate
Use Cases
3
Prioritize
Use Cases
Business
Team Members
(The Customer)
Development
Team Members
(The Developer)
Loading
Factor
Business
Needs
2
Business
Team Members
(The Customer)
4
Plan
Iterations
5
Commit to
Release
Plan
Release
Plan
Development
Team Members
(The Developer)
38
WADM™ Plow-Back
Proposal
Phase
Startup
Activities
Development
Cycle
Design
Completed
Cycle
Build
Acceptance
Testing
The
Planning
Game
Standup
Meeting
Construction
Cycle
Acceptance
Test
Development
Construction
New and Deferred
Use Cases
Design Cycle
Planning
Meeting
39
WADM™ QA
QA
Project Plan
& Schedule
Review
QA
Consultation
Proposal
Phase
SQA
SQAof
End
SQA
End
QAof
Cycle
End
of
Cycle
End
of
Review
Cycle
Review
Cycle
Review
Reviews
Startup
Activities
Development
Cycle
Design
Completed
Build
Cycle
Acceptance
Testing
The
Planning
Game
Standup
Meeting
Construction
Cycle
Acceptance
Test
Development
Construction
Design Cycle
Planning
Meeting
40
WADM™
CMMISM
AGILE
WADM™ Defect Management
• defect avoidance
– A practice where the focus is to minimize the number
of defects created during the construction of a work
product.
– It avoids the creation of the defect in the first place.
• defect detection and resolution
– A post-construction activity to identify and disposition
the defects which were created during construction.
– It identifies and resolves defects that were created
despite the project’s defect avoidance practices.
• defect management approach
– Defines both the defect avoidance and the detection
& resolution processes to be followed by the project
team.
– It also defines the criteria for deciding which
processes are to be applied for each work task.
41
WADM™ Minimum Deliverables
Phase
Minimum Required Deliverable/Work Product
Feasibility
(Proposal)
High Level Statement of Work
High Level Estimate
Start Up
Activities
Project Plan*, including
Statement of Work
Estimated Cost
High Level Schedule
Design and Code Standards
Team Operating Rules
Initial Use Case-Based Requirements Document*
Initial Use Cases
Design Constraint Based Requirements
Initial Release Plan (found in Use Case Based Requirements Document*)
Detailed Loaded Schedule
Committed to by Business Team Members and Developers
Initial List of Risks*
Initial List of Issues*
1 of 3
* WADM™ Template required
42
WADM™ Minimum Deliverables
2 of 3
Phase
Minimum Required Deliverable/Work Product
Development
Cycle:
Planning Game
Updated Risks*, Issues*, Action Items*
Use Case Based Requirements Document*
Use Cases, Estimates, and Design Constraint Based Requirements
Updated Release Plan
Acceptance Test Scenarios (Inspection, Procedural, & Automated)
Use Case Estimate and Rationale Document*
Development
Cycle:
Design and
Construction
Sub-Cycles
Updated Risks*, Issues*, Action Items*
Design
CRC Cards
UML Graphics
Unit Test Plans
Unit Test results
Configuration Management
Code migrated from Working to Released to Submitted for
Acceptance Testing
* WADM™ Template required
43
WADM™ Minimum Deliverables
Phase
Minimum Required Deliverable/Work Product
Development
Cycle:
Test
Updated Risks*, Issues*, Action Items*
Test Class Updates
Test Procedure Document Updates
Use Case Test Scenario Result Summary Log*
End of
Development
Cycle
Approved Software
Configuration Management
Code migrated from Submitted to Approved (in production)
Updated Use Case Based Requirements Document*, if changed
Updated Release Plan, if changed
3 of 3
* WADM™ Template required
44
WADM™ Process Tailoring
1 of 2
• WADM™ is a pre-tailored version of the IIS process set
– Specific to the rapid development / agile environment
– All team members need to be trained in this methodology
• Normal processes apply . . . except where superceded
•
explicitly by WADM™ processes
– Project planning checklist does not apply
– Size-a-matic™ tool does not apply
Prescribed templates
– Boilerplate text must be followed as is
– Required WADM™ Templates
– Project Plan
– Use Case Based Requirements Document
– Risk Management Log
– Issue Management Log
– Use Case Estimate & Rationale Document
– Use Case Test Scenario Results Summary Log
– Action Tracking Log
45
WADM™ Process Tailoring
2 of 2
• There is very little additional tailoring that the individual
•
project can perform
– Decision on the use of CRC cards, UML, or both
– Choice of defect avoidance alternatives
– Choice of defect detection approaches
– Identification of any project specific standards and tools
– Production implementation approach and deployment strategy
The WADM™ is agile, but not flexible
– Most flexibility has been tailored out of the WADM™
– To decrease documentation needs
– To ensure SW-CMM / CMMISM conformance
– The WADM™ must be used as is
46
IIS Process Right-Sizing
Summary
47
Experiment
48
Summary
1 of 6
• One size does not fit all
– It never has, and never will!
– And even if it did, different situations call for different
dress …
49
Summary
2 of 6
50
Summary
3 of 6
• Look for the Rosetta Stone to your environment
– Analyze your project types
– Maybe effort is not the driver for your process
tailoring decisions, but something is …
– Develop varied and flexible approaches that match
your business needs
51
Summary
4 of 6
Tame that SW-CMM /
CMMISM monster
–Turn it into
something that your
programmers and
engineers will relate
to
52
Summary
5 of 6
• Apply the intelligence principle to your process
definitions
– What is the purpose of all this?
– You don’t have to follow the SW-CMM or the CMMI
slavishly at the sub-practice level … meet the intent
– If you have families of projects, customize your
process definitions to make planning them easier
• Allow your projects to apply the intelligence
principle as well
– Keep your eyes on the prize: better run and more
predictable projects that meet your customers needs
– Avoid process for process sake
53
Summary
6 of 6
Trust in the Force …
but make it easy to comply!
54