A Methodology for Predicting Probability of Detection
for Ultrasonic Testing
W. Q. Meeker
V. Chan
C-P. Chiou
R. B. Thompson
Iowa State University
Motivation
FAA Engine Titanium Consortium goal to improve and quantify POD inspection for detecting hard alpha
Empirical determination of POD for new situations is not always practical due to lack of time, test specimens (especially
data on real aws), and other limited resources.
Physical models of ultrasonic signals provide a useful alternative to empirical determination but may not account for
all factors and sources of variability, particularly for real aws
(e.g., hard alpha inclusions).
27th Annual Review of Progress in QNDE
Ames, Iowa
July 19, 2000
Acknowledgments to:
FAA Engine Titanium Consortium POD working group
Combining physical models of ultrasonic signals with an empirical adjustment has the potential to provide a workable
methodology
2
1
Overview
Probability Model for a UT Flaw Signal
Factors and sources of variability in UT inspection.
Comparison of physical model-based distribution and empirical distribution.
Probability integral transform.
Adjustment to account for the dierences between the model
and the empirical distributions.
Example based on synthetic hard alpha aws in titanium.
Probability distribution for a aw signal Y is
Pr(Y y; x) = FY (y; x)
where x = (xNDE; xPART; xFLAW ).
I xNDE contains NDE inspection system factors like probe
characteristics, focus depth, scan increment, pulse rate,
etc.
I xPART contains PART factors like part geometry, type of
material being inspected, surface roughness, etc.
I xFLAW contains aw factors like size, depth, density, shape,
composition, and degree of voiding/cracking.
3
Sources of Variability in Nondestructive Evaluation.
Systematic Dierences Between the Model-Predicted
Signal pdf fY (y; x) and the actual pdf fY (y; x)
for a given x.
Measurement System Variability
Probability Density
0.0
0.5
1.0
1.5
2.0
2.5
3.0
1.0
0.0
0.5
1.0
1.5
2.0
2.5
Materials Effects Variability
Flaw Morphology Variability
Probability Density
Maximum Peak-to-Peak Voltage in Gate
Probability Density
Maximum Peak-to-Peak Voltage in Gate
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Maximum Peak-to-Peak Voltage in Gate
0.0
0.5
1.0
1.5
2.0
2.5
Actual signal distribution
Model prediction
3.0
Relative Frequency
Probability Density
Single Flaw/Ideal Measurement
4
3.0
Maximum Peak-to-Peak Voltage in Gate
0.5
0.0
0.02
0.10
0.50
2.00
10.00
50.00
Mon Jul 17 20:33:05 CDT 2000
Signal Y
5
6
Comparison Between Actual FY (y; x) and
Model-Predicted cdf FY (y; x).
Points Represent Empirical Estimate of FY (y; x)
from Simulated Data
Probability Integral Transform
Suppose that the random variable Y has a cumulative distribution function FY (y; x).
1.0
Probability
Then
= FY (Y ; x)
has a Uniform distribution between 0 and 1.
U
0.5
This is a well-known result in probability and mathematical
statistics.
Actual signal distribution
Model prediction
0.0
0
10
20
30
40
Mon Jul 17 20:33:13 CDT 2000
Signal Y
8
7
Illustration of the
Probability Integral Transform with Two x's
0.6
0.4
0.0
0.0
0.2
0.2
0.4
Probability
0.6
0.8
0.8
1.0
1.0
Illustration of the
Probability Integral Transform with a Single x
70
80
90
100
110
120
130
70
Signal Strength
80
90
100
110
120
130
Signal Strength
9
Empirical Probability Integral Transform
10
FY (y)
Suppose we have signal values y1; y2; : : : ; yn (along with the
corresponding, possibly dierent, x1; x2; : : : ; xn values). Then
compute u1; u2; : : : ; un from UT physical model as ui = FY (yi; xi).
11
1.0
Probability
If the UT physical model ts well, then the u1; u2; : : : ; un
values will lie close to the diagonal.
If the u1; u2; : : : ; un values do not lie close to the diagonal, the
systematic deviation suggests how to make an adjustment to
the UT physical model.
The empirical probability integral transform provides
I A means of assessing model goodness of t.
I An method of model adjustment when the model does not
t.
and Data Probability-Integral Transformed with
the Actual FY (y) Distribution
0.5
0.0
0.0
0.5
1.0
Tue Jul 18 21:58:13 CDT 2000
Probability-Transformed Signal
12
FY (y)
and Data Probability-Integral Transformed with
an Inadequate UT Physical Model
Suppose that the random variable Y has a cumulative distribution function FY (y; x).
Then
U = FY (Y ; x)
has a Uniform distribution between 0 and 1.
The transformation
Z = ;1(U ) = ;1[FY (Y ; x)]
has a standard normal (Gaussian) distribution with mean =
0 and standard deviation = 1.
1.0
Probability
Extension of the Probability Integral Transform
0.5
0.0
0.0
0.5
1.0
Tue Jul 18 21:58:34 CDT 2000
Probability-Transformed Signal
14
13
Measured UT Response from Synthetic Hard Alpha
Flaws from Chiou et al. (1996, 1997) .
Steps in the Analysis
Compute
zi = ;1(ui) = ;1[FY (yi; xi)]; i = 1; : : : ; n
UT responses from SHA flaws
500
If the UT physical model FY (y; x) ts the data well, then the
zi; i = 1; : : : ; n should look like a standard normal (Gaussian)
distribution with parameters = 0 and = 1.
200
mV
100
Otherwise obtain an adjusted distribution by estimating and from zi; i = 1; : : : ; n and using
" ;1
#
Pr(Y y; x) = Fy (y; x) = (FY (^y; x)) ; ^
as the predicted signal-distribution (input to POD computations).
50
20
Depth= 1 inches
Depth= 0.5 inches
Depth= 1.25 inches
10
2.0
2.2
2.6
3.0
3.4
3.8
4.2
4.8
Mon Jul 17 21:27:55 CDT 2000
Size
15
16
Physical Model Probability-Integral-Transformed
Synthetic Hard Alpha Data.
Physical Signal-Prediction Model
1.0
M=2, mean=0.57, sd=1.61
0.6
Probability
0.4
0.2
0.0
where x = (a; d) reects aw size and depth (e.g., from the
model of Yalda, Margetan, and Thompson (1998)) based on
Rician distributions to predict the distributions of ultrasonic
aw signals in the presence of backscattered noise).
0.8
The physical model approximation cumulative distribution
F Y (y ; x ) = : : :
•
•••
•••••
•
0.0
•••
•
•
•
0.2
••
•• •
•• •
•• •
•• •
0.4
•• •
••
•••
• ••
• ••
•• •
•••
••
•••
•••
•
•••
•••
•• •
•• •
0.6
••••
•••
•••
•••
•
•
•
••••
•
•••
•••
•••
•• •
•••
•
•
•
•• •
• ••
• ••
•• •
•• •
•••
0.8
1.0
Probability-Transformed Signal
17
18
Adjusted-Model Probability-Integral-Transformed
Synthetic Hard Alpha Data.
The adjusted probability distribution model for aw signal
Y is:
" ;1
#
Pr(Y y; x) = Fy (y; x) = (FY (y; x)) ; 0.6
0.4
0.2
0.0
••
•• •
•••
••
• •
•••
0.0
• ••
• ••
•••
•••
• ••
••
•
• ••
•••
•••
•••
•
•
• ••
0.2
••
•••
•••
•• •
•• •
0.4
• ••
• ••
••••
• • •
• ••
•
•
•
•••
• ••
•••
•••
When = 0 and = 1, Pr(Y y; x) =
same as the UT physical.
0.6
0.8
FY (yi; xi)
is the
Basic POD can be computed as
POD(a; x) = Pr(Y > ythresh)
= 1 ; Pr(Y y; x)
1.0
Probability-Transformed Signal
20
19
Model-Predicted POD as a Function of Size for
Several Depths.
Concluding Remarks
Physical models provide an eective methods for quantifying
POD in situations when data on real aws is limited.
1.0
Statistical methods can be used to quantify and adjust for
systematic biases and sources of variability that are not accounted for in the model.
0.8
0.6
POD
Probability
0.8
1.0
M=2, mean=0.57, sd=1.61
• ••
•••
•••
•••
•• •
•
•
•
•••
•••
• ••
•••
•••
Computation of POD(a)
0.4
0.2
Improved computing capabilities and continued eorts in model
development will increase the needs/opportunities for the use
of models to quantify NDE inspection capability.
Depth= 0.5 inches
Depth= 1 inches
Depth= 1.3 inches
0.0
0
1
2
3
4
5
6
7
Mon Jul 17 21:28:17 CDT 2000
Size
21
22