DSES-6620 Simulation Modeling and Analysis
Navy Supply Chain Model
Verification and Validation
Rich Sewersky
May 2, 2002
Background
•
•
•
•
New business for Aerospace: Provide spare parts at fixed cost per flight hour
called Performance Based Logistics (PBL)
Key measure and risk is %Fill Rate = on time shipments/total requisitions
Goal of model is to predict fill rate and other measures such as inventory to be
able to complete business tradeoffs/contingency planning
Model prepared in TaylorED by researcher, Sewersky was Subject Matter
Expert
– 14 repairable components (Main Gearbox Module chosen for project)
– One year run period with specified starting position (contract terms)
•
•
Project goal was to verify and validate model so it can be used for business
Key steps
–
–
–
–
•
Research commercial/military V&V guidelines
Review model logic and code and research report
Explore model sensitivity to key inputs and output variability
Calibrate output to known commercial simulation package
Terms:
– Ready for Issue (RFI) - Components in working order
– Carcasses - Broken components
TaylorED Modeling Paradigm (connect the dots)
PoolInfoTb
InfoTb
BkOrderTb
Uniform (1, flowtime)
Uniform (1, deltime)
Negexp (3.121)
Results - Verification
• Research found DOD has Defense Modeling and Simulation Office
– has web site with guidance and educational materials, bibliography
• www.dmso.mil/public/transition/vva/
– several working groups including formal V&V and Accreditation
– key issue is model interoperability and contractor sell-off
– many of these models support “war games”
• Developed logic diagram based on model structure and code flow
– (found one code error and corrected it - minor performance effect)
• Checked effect of key variables on output measures
–
–
–
–
–
Variation turned off
Demand interarrival rates
Transport times
Processing times
Distribution fitting
• Model reactions made sense
Results - Validation
• Researcher had checked model outputs against OPUS
spares modeling tool - OPUS 91% Fill vs. Model 96%
• Sewersky baseline runs averaged 93% with fairly high
variation (stdev 9.2). (note that the run with all variation
turned off was 90.6%)
• Input data is also highly variable
– Navy demand histories
– Navy transportation times (returned “carcasses”)
– Shop flow times (part shortage driven)
Results Summary
Baseline Run (with WIP distribution error)
Baseline Run (with WIP distribution error corrected)
Run with all variations removed
Run with demand variation only (transport and
processing at worst case fixed level)
Run with demand variation only (transport and
processing at average fixed level)
"Wartime" Scenario (Worst Case Everything)
Avg
StDev
Avg
StDev
Avg
StDev
Avg
StDev
Avg
StDev
Avg
StDev
Total BK Total
Fill
Order
Order# Rate(%)
7.9
119.8
93.7
9.8
8.0
7.8
8.4
117.5
93.3
11.5
9.2
9.2
11
116
90.5
n/a
n/a
n/a
15.8
110.3
86.6
14.5
12.1
11.1
2.4
121.6
98.1
3.2
344.2
20.1
6.7
352.2
20.1
2.5
2.3
0.1
Avg
WIP
16.8
1.1
18.6
1.5
20.7
n/a
20.2
1.8
11.7
0.6
57.0
3.1
Backorders
Avg RFI Count Average Minimum Maximum
in WH
Duration Duration Duration
10.6
13.2
4.0
0.1
18.3
1.7
9.5
3.1
0.6
5.8
9.2
21.0
6.6
0.1
23.3
2.4
6.6
3.9
0.3
8.2
7.2
11
16.3
0.7
31.9
n/a
n/a
n/a
n/a
n/a
8.3
15.8
13.1
0.1
50.3
2.9
14.5
5.6
1.7
10.7
21.7
3.0
4.1
0.8
11.9
0.9
0.1
0.1
Most orders remained unprocessed so these
statistics are misleading (only for those actually processed)
3.4
344.2
20.1
2.1
50.9
3.1
1.6
30.5
2.7
4.5
79.6
2.1