The Architecture and Design of a
Corporate Engineering Data Repository
Dr. Gan Wang
Ms. Lori Saleski
27th International COCOMO Forum
16-18 October 2012
Software Engineering Institute, Pittsburgh, PA
Copyright © 2012 by Gan Wang, Lori Saleski and BAE Systems. Published and used by INCOSE with permission.
Outline






What is the Engineering Data Repository?
CONOPS
Architectural and Design Considerations
User Interfaces
Data Collection Process
Lessons Learned
2
What’s the Engineering Data Repository?

A suite of tools, comprised of
– Database of current and historical program data
– Data search and query portal
– Estimating tools and process
– Data collection and process
– Analysis and calibration tools
EPB Process Flow – Proposal Phase
Data Search
Portal
Data Collection Tool
Entry Criteria
• Customer’s Proposal Materials
• Approved Phase 1B Package
• Approved PTW
• Proposal Shared Repository
Independent Cost Estimate
Run Concurrent
C
H
Establish the
Programmatic
Baseline
D
Engineering
Program Metrics
Database
A
B
Prepare and
Hold
Proposal
Kick-Off
Meeting
Define Top Level
Cost Target
Allocations for
NRE and RE
Efforts
F
Review
Program- matic
Baseline
Pass
J
Prepare and
Hold Estimate
Kick-Off
Meeting
Perform Cost
Estimation and
Schedule
Assessment
May be combined
Establish the
Technical
Solution /
Baseline
G
Review
Tech. Solution
/ Baseline
I
Pass
Execute
Independent Design
Review
TSR
K
Execute the
Technical Bid
Review
Pass
2A DR
Pass
TBR
E
Perform Quick
Look Cost
Estimate
L
Execute
Phase 2A Phase
Review
M
Pass
Execute
Request for Bid
Approval
2A
RBA
Exit Criteria
•
•
•
•
•
•
•
RBA Briefing and Certificate
TBR Package and Certificate
Phase 2A Package and Certificate
2A IDR Package and IDR Certificate
Risk & Opportunity Registers & Plans
Assumptions
CBB Provisioning Plan and MR/TC Generation
Plan
• Engineering Assessment Scorecard
• Proposal Shared Repository
Key:
Activity
Review
Optional
Review
Optional
Flow
Identify historical programs
•
•
Quality program
Establish program technical and financial
POCs
Conduct initial data collection training
session
•
•
Learn the Data Collection Template
Learn metrics definitions
Collect raw data
•
•
Financial database
Technical data records
Interview and data review sessions
•
•
Estimating Tools
Recollect questionable data
Additional interview sessions
Conduct analyses
•
•
Normalization
Statistical analysis
3
What’s in the Repository?
Historical Data:
Tools and Processes:



Engineering (parametric) estimating tools
Data collection and analysis tools


Procedures, templates, checklists
Training materials, videos

References
Project metadata
– Project and program level descriptions
– Attributes and characteristics

Engineering financial actuals
– Actual (historical) efforts
– Estimated (in-execution) efforts

System-level design metrics, e.g.,
–
–
–
–

Requirements
Interfaces
Algorithms
System complexities
Subsystem/component-level metrics,
e.g.,
–
–
–
–
SLOC counts
SWaP
Drawing counts
Supportability
4
What’s Motivation?

The dilemma we ran into in the past…
DESIGN.
Adapted from Dilbert, © Scott Adams, Inc.

Provide enterprise-wide, central and easy-to-use data store for all
engineers
– Consistent use of historical data in system design and business winning
– Supporting entire project life cycle
– Improved estimation accuracy, confidence and credibility
– Save cost and reduce cycle time
– Strategic competitive discriminator
5
Lifecycle Operational Data Needs
Use of historical data essential in system design and program execution
Design Reviews
SRR
CDR
SDR/PDR
Prove Op.
Capability
VALIDATION
In-Field
Support
Production,
Installation and
Commissioning
6
7
8
Decommissioning
Procedures
DEFINE/
REFINE
5
Validate, Build &
Clear for Op.
Trials
Design
System
Integration
of System
Product
Mfacturing
Definition
4
Disposal
Support
Product and
Service
SPECIFICATION
System
Qualification
3B
Freeze
Engineering
Definition
Agree System
Specification
3A
Produce
System
Specification
Contract
Accepted
Mobilize
2C
In support of:
ECP Development,
ROMs & Life Cycle
estimates
Certify, Build,
& Product
Operational
Clearance
SELECTION
2B
Initial Design
Deploy
Technology
Prepare
Proposal
2A
Detail
Remaining
Options
1B
Down Select
Options
Assess Each
Option
Develop
Option
Definitions
Identify
Options
INNOVATION
Prepare Proposal
Proposal
Accepted
Prepare Bid/No Bid
Information
Phase Reviews
In support of:
Cost estimating for Project baseline and rebaseline
(budget estimates), ETCs / EACs / ECP development,
product design trades, risk & opportunity, engineering
productivity measurement
In support of:
Firm Cost Estimates /
Independent Cost
Evaluations (ICE), LCCE,
Engineering bid estimation
In support of:
ROM & Life Cycle
estimates, “Price to
Win“ and feasibility
analysis
High Level &
Detailed cost
history, metric
data, calibrated
estimating models
Detailed cost history and metric data by
development phase, calibrated
estimating models
Detailed cost history,
metric data, calibrated
estimating models
Project/Product
level data for
ROMs/Quick Looks
PROD.
SUPPORT
DISPOSAL
PRR
TRR
6
Typical Use Cases
+
Data Repository
Everyone
Query
Single Data
Point
Ingest
Single Data
Point
+
Query
SingleAttribute
Statistics
+
Data
Collector
Query
Group of
Data Points
Data
Analyst
7
Three Interconnecting Logical Databases
Use Past Results to Predict Future Performance
Engineering Estimates
Project Engineering EACs
Project Engineering Actuals
Project Life Cycle
Project Post Mortem
Cost
Proposals
Project Bid / Proposal
Time
Estimate
Database
In-Progress
Program
Database
Post Mortem
Database
Business Capture
Program Execution
Process Improvement
8
Cost Element Structure
Project/Organization
Specific Structures
Mapping
Rules
1.0 – System/Project
1.1 – Integrated Project Management
1.2 – Systems Engineering
1.3 – Prime Mission Product (PMP)
1.4 – Platform Integration
1.5 – System Test & Evaluation (ST&E)
1.6 – Training
1.7 – Data Management
1.8 – Peculiar Support Equipment
1.9 – Common Support Equipment
1.10 – Operational / Site Activation
1.11 – Industrial Facilities
MIL-STD 881 Compliant
9
Dynamic Data Structure Design
Data Records
Dynamic linked
lists used to
support
different data
types and
periodic
collections
throughout
project and
system life
cycles
Data Instances
Project 1
Project 2
Project 3
…
…
Project N
Data Record
Closed
Instance
In-Edit
Instance
Record
Closed
10
Database Workflow Supporting Collection of
Multiple Data Instances of Same Program Data
Access Control by Roles
Built-in Data Ingestion and Review Functions to Support Role-based Interactions
11
Data Query Portal – Search by Program Attributes
Search
Criteria
Accessible
from Corp.
Intranet
Search
Relevance
Search
Dashboard
12
Detail Data Queries
Calculated
Statistics
Detail
Project
Data
13
Consistent Process is a Cornerstone
Identify historical programs
Qualification of Data Point
•
•
• Is project
complete?
• Clear start and
finish?
• Clear funding
line?
• Is it relevant to
future bids?
• Project points of
contact found?
• Project financial,
technical and
other supporting
data available?
Quality program
Establish program technical and
financial POCs
Conduct initial data collection
training session
•
•
Learn the Data Collection Template
Learn metrics definitions
Collect raw data
•
•
Financial database
Technical data records
Interview and data review
sessions
•
•
Recollect questionable data
Additional interview sessions
Conduct analyses
•
•
Normalization
Statistical analysis
14
Practical Lessons Learned

Data quality
– Consistency, consistency, consistency!
– “Apples-to-oranges” has little to no benefit

Plan for additional time
– Assumptions/definitions of data
– Analysis in aggregate



Problem solving exercise
Corporate expertise
Management championship
15
In Closing

Enterprise-level Engineering Data Repository
– Database of current and historical program data
– Data search and query portal
– Estimating tools and process
– Data collection and process
– Analysis and calibration tools


Used throughout project and system design life cycles
Proven effective
– Easy-to-use, popular with engineers
– Improves accuracy, confidence and credibility
– Saves cost and reduce cycle time
– Enhances business competitiveness
16
Questions & Comments
Dr. Gan Wang
gan.wang@baesystems.com
703-668-4259
Ms. Lori Saleski
lori.saleski@baesystems.com
603-885-6353
17