Overview
Early Detection of Reliability Problems Using
Information From Warranty Databases
• Background and motivation
• Example: automobile “VIN” and “Warranty Report” databases.
Huaiqing Wu
• Statistical formulation of the detection problem.
William Q. Meeker
• Allocation of false alarm probabilities.
over time.
Department of Statistics
Iowa State University
Balance power
• Critical values for the early detection monitoring procedure.
Ames, IA 50011
www.public.iastate.edu/˜wqmeeker
• Average run length.
• Summarization of monitoring procedure behavior.
August 16, 2001
• Concluding remarks.
1
2
Previous Work
Background and Motivation
• General warranty topics:
1996)
• Serious, unanticipated reliability problems arise
• Early detection can save large amounts of money and
customer good will
• Statistical detection rules in a warranty database can
identify or warn of serious reliability problems long before they would otherwise be discovered
• Monitoring existing warranty data bases will add little additional costs
Blischke and Murthy (1994,
• General reviews of statistical methods: Robinson and McDonald (1991), Lawless and Kalbfleisch (1992), and Lawless (1998)
• Dealing with messy data problems: Suzuki (1985a, b),
Kalbfleisch and Lawless (1988), Lawless, Hu, and Cao (1995),
Hu, Lawless, and Suzuki (1998), Karim, Yamamoto, and
Suzuki (2001a), and Wang and Suzuki (2001a, b)
• Detecting a change point from marginal count warranty
data: Karim, Yamamoto, and Suzuki (2001b)
3
4
Fraction of Automobiles with Warranty Reports
in the First Four Months of Service Versus
Production Month for 12 Different Labor Codes
Automobile “VIN” and “Warranty Report” Databases
• VIN (production) database
5
1996 1996 1997
Apr
Dec
Aug
Build Month
1998
Apr
1996 1996 1997
Apr
Dec
Aug
Build Month
1998
Apr
0.012
0.008
0.004
0.0
Fraction With Reports After 4 Months
0.003
0.001
1994
Dec
1995
Aug
1996 1996 1997
Apr
Dec
Aug
Build Month
1996 1996 1997
Apr
Dec
Aug
Build Month
1998
Apr
1998
Apr
0.003
0.002
0.001
1994
Dec
1995
Aug
1996 1996 1997
Apr
Dec
Aug
Build Month
1998
Apr
Labor Code H1240 [1048]
Fraction With Reports After 4 Months
0.004
1995
Aug
1996 1996 1997
Apr
Dec
Aug
Build Month
0.0
1998
Apr
Labor Code E9995 [985]
1994
Dec
1995
Aug
Labor Code D3044 [777]
Fraction With Reports After 4 Months
0.04
0.03
0.02
0.01
1994
Dec
Fraction With Reports After 4 Months
0.004
0.002
1995
Aug
1998
Apr
0.0
1998
Apr
Labor Code E0432 [823]
1994
Dec
1996 1996 1997
Apr
Dec
Aug
Build Month
0.0020
1996 1996 1997
Apr
Dec
Aug
Build Month
1995
Aug
Labor Code C6881 [631]
0.0010
1995
Aug
Labor Code B7876 [359]
0.0
1994
Dec
0.0
0.015
1995
Aug
1994
Dec
Fraction With Reports After 4 Months
0.010
0.004
0.0
1998
Apr
Labor Code D3088 [784]
1994
Dec
1998
Apr
0.002
1996 1996 1997
Apr
Dec
Aug
Build Month
1996 1996 1997
Apr
Dec
Aug
Build Month
Labor Code C3348 [535]
Fraction With Reports After 4 Months
0.004
0.002
Fraction With Reports After 4 Months
0.0
1995
Aug
1995
Aug
0.0
Fraction With Reports After 4 Months
0.0004
0.0
1994
Dec
Fraction With Reports After 4 Months
Also contains cost, dealership code, and other information
1994
Dec
0.005
1,350,675 records with 1,908 unique labor codes
1998
Apr
0.0
Warranty reports received between January 1995 and
November 1998
1996 1996 1997
Apr
Dec
Aug
Build Month
Labor Code C3301 [530]
Fraction With Reports After 4 Months
• Warranty report database
1995
Aug
Labor Code B0608 [197]
0.0
1994
Dec
Provides date of sale and other production information
Labor Code A5580 [149]
Fraction With Reports After 4 Months
0.0020
0.0010
0.0
566,406 unique VINs
Labor Code A0357 [14]
Fraction With Reports After 4 Months
One record for each automobile for a particular “platform” manufactured between January 1995 and August 1998
1994
Dec
1995
Aug
1996 1996 1997
Apr
Dec
Aug
Build Month
1998
Apr
6
Retrospective View Giving the Fraction of Automobiles
with Labor Code C0140 Warranty Reports
in the First Four Months of Service
versus Production Month
General Formulation of the Detection Problem
• Generalization of Shewhart process monitoring schemes
0.004
0.005
• One month used as the production/ time in service period
in examples
0.002
0.003
• Data for a given production period accrue sequentially
(due to variability in sale date and tracking by time in
service)
0.001
• Nonparametric approach (based on report counts) most
appropriate for detection
• Use a parametric model (e.g., Weibull or Lognormal) for
prediction of future failures
0.0
Fraction With Reports After 4 Months
0.006
• Stratify by production period, time in service, and labor
code
1994
Dec
1995
Apr
1995
Aug
1995
Dec
1996
Apr
1996
1996
Aug
Dec
Production Month
1997
Apr
1997
Aug
1997
Dec
1998
Apr
1998
Aug
7
Notation
8
Formal Detection Rule Framework
• ni units produced in period i
• Multiple-parameter hypothesis
• nij units produced in period i and sold j periods after
manufacture
• Rijk number of warranty reports kth period in service for
units manufactured in period i and sold in period i+(j−1)
j
• Sijk = =1 Rik is the cumulative number of reports for
units manufactured in period i and sold in period i+(j−1)
• Cijk critical limit for Sijk
0
0
H0 : λ1 ≤ λ0
1 , λ2 ≤ λ2, . . . , λM ≤ λM
versus
0
0
Ha : λ1 > λ0
1 or λ2 > λ2 or . . . or λM > λM ,
• λ0
k values are from historical information or reliability targets
• αk is the nominal false alarm probability for testing λk
• The overall false alarm probability for testing the hypothesis H0 versus Ha in is
• λk intensity for service period k
• M number of future periods to monitor report intensities
• fj fraction sold j months after production;
M
j=1 fj ≤ 1
• Superscript 0 (e.g., fj0) denotes historical or baseline
value.
α∗ = 1 −
M
(1 − α∗k ) ≤ 1 −
k=1
M
(1 − αk ) = α,
k=1
• Control and allocation of false alarm probabilities should
be labor code specific.
9
2nd month in service
12
8
+ +
4
+
0
1 2 3 4
3rd month in service
12
8
+
4
+
0
1 2 3 4
12
8
4
0
12
8
4
0
12
8
4
0
- - -
JUN 1997
+
-+ - +
2
3
4
+
+
1
2
3
4
+
2
3
4
-+
1
2
3
4
12
8
4
0
+
1
- -
2
3
4
12
8
4
0
-+
1
- -+
1
12
8
4
0
- -
- - -
2
3
4
4th month in service
12
8
4
+
0
1 2 3 4
-
Cumulative Number of Reports (+) and Critical Values (-)
MAY 1997
1st month in service
12
+ 8
+
4
+
+
0
1 2 3 4
1
JUL 1997
Production Month
APR 1997
Sequential Test Monitoring Charts
Labor Code C0140 Warranty Reports
+ Indicates Cumulative Number of Reports Sijk
− Indicates Corresponding Critical Limit Cijk
10
Control and Allocation of False Alarm Probabilities
• Limit M , the number of periods monitored
• Simple rule: αk to be proportional to the information
available for testing H0k versus Hak :
0
0
αk = C(f10 + · · · + fM
−k+1)λk ,
• Must also allocate power for the sequential accumulation
of data. We use the error spending approach developed
Slud and Wei (1982) for sequential clinical trials.
Sale Month Since Production
11
12
2nd month in service
0.0
Values of αjk(i)
0
1
2
3
4
0
1
2
3
4
(a)
(b)
3rd month in service
4th month in service
0.0004
0.0004
0.0002
0.0002
0.0
0
1
2
3
4
0
1
(c)
2
3
0.0
0.0
0.003
0.0002
0.0
0.002
0.0004
0.0002
0.001
0.0004
Fraction With Reports After 1 Months of Service
1st month in service
Nonparametric Estimate of Fraction With Reports
After One Month of Service as a Function of
Production Month for Data Available in July 1997.
0.004
Typical Spending Functions for Production Month
May 1997 for Labor Code C0140 With α = .1% and
M = 4, and ρ = .5, —; ρ = 1, · · · ; and ρ = 2, - - -.
4
(d)
1994
Dec
Sale Month Since Production (MAY 1997)
1995
Mar
1995
Jun
1995
Sep
1995
Dec
1996
Mar
1996
Jun
1996
Sep
1996
Dec
1997
Mar
1997
Jun
Production Month
13
Weibull Probability Plot and Weibull ML Estimate for
Production Month May 1997 Based on Data Available
in July 1997 (after two months in service)
14
Weibull Probability Plot for Production Month May
1997 Based on Data Available in November 1998
.02
.02
.01
-
.01
Proportion with Reports
Proportion with Reports
.005
.003
.005
.003
-
-
-
-
-
-
-
-
-
- - - - - -
-
-
.001
.001
^
η
= 721153
-
^
β
= 0.502
.0005
.0005
1
1
2
5
10
2
5
10
20
20
Months in Service
Months in Service
15
16
Concluding Remarks and Extensions
Average Run Length
• All 1,908 labor codes were investigated.
• Probability of no alarm is triggered during the first ( − 1)
periods of monitoring
ARL = E(N ) =
∞
Pr(N ≥ )
• Efficient monitoring should provide economical early detection, especially with modern computing/data storage
capabilities
=1
• Under H0
Pr(N ≥ ) =
−1
i=1
• Methods implemented on 48 interesting labor codes. Results summarized in the paper.
• Modeling past data is useful for setting the baseline report
rates
γi,−i ≥ (1 − α)−1.
• Simple expression for ARL if ni is constant and fij =
fj0, j = 1, . . . , M
• Can also compute under specified Ha
• Should monitor different production lines/shifts separately
• Runs rules useful in some settings
• Detection also provides input to warranty cost forecasting
algorithm
17
18