AVERAGE OF DELTA – A NEW
CONCEPT IN QUALITY CONTROL
GRD Jones
Department of Chemical Pathology,
St Vincents Hospital, Sydney, Australia
APCCB 2004 1
Background
• The Average of Normals (AON) is an accepted QC process
for clinical laboratories.
• AON is the average of a set number of patient results,
usually within set limits (eg normal range).
• The AON rule “fires” when the function exceeds a pre-set
limit (eg 2.5 x analytical CV).
• Delta checks are the comparison of a result with a previous
result from the same patient.
• Delta checks are used to detect blunders or other errors
• I combine these concepts to produce the Average of Delta
(AOD) a new QC tool for clinical laboratories.
APCCB 2004 2
Terminology
• A Delta Value is a recent patient result minus the preceding
result for that patient.
• An AOD function is the average of a series of delta values.
• AODN is an AOD function averaging N delta values.
• NAOD is the number of samples included in an AOD
function.
• N90 is the number of samples with valid previous result,
required to detect a change in assay bias with 90%
probability using an AOD function.
• CVwi is the within-individual Biological Variation.
• CVa is the analytical variation expressed as a CV.
• SDAOD is the SD of the AOD function
APCCB 2004 3
Methods
• AOD functions were modelled in a spreadsheet
application using Microsoft Excel.
• Variations in CVa and CVwi were modelled using the
random number generator with a Normal distribution.
• Models were based on 100 data sets, each of 110 delta
values.
• Factors adjusted in the model were:
– the ratio of Cva/Cvwi
– NAOD
– Bias changes in assay performance.
APCCB 2004 4
Modelling Equations
• Data sets were generated for various values of CVa and CVwi
with the variation (CVdata set) in results described as follows
CVdata set = SQRT(CVa2 + CVwi2)
• Second data sets were independently generated using the
same values for CVa and CVwi
• Delta Values were were obtained by subtracting the data
points from the second data set from those in the first to
produce a series of delta values.
• Changes in bias were modelled by addition of fixed amounts
to the delta values at a fixed point in the data set.
• AOD functions were set to trigger if a data point fell outside
limits defines by +/- 2.5 SDAOD.
APCCB 2004 5
AOD Functions
0.6
0.4
AOD value
Figure 1
• AOD functions for various
values of NAOD.
CVa = 0.1, CVwi = 0.2
• As the value of N
increases, the scatter of
the AOD function
decreases.
• The decrease in SD with
increasing N is equal to
dividing by the square
root of N. (data not shown)
0.2
0
-0.2
-0.4
-0.6
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61
Sample Number
Purple: n=2 SD = 0.22
Red: n=10 SD = 0.10
Blue: n=50 SD = 0.045
APCCB 2004 6
Effects of Bias on AOD Functions
• A fixed bias was added after delta value 10 in each data set.
• The AOD function followed the change in bias with the
following features:
– With smaller values of NAOD, the response occurred more
rapidly, but was smaller relative to the scatter of the AOD
function
– With higher values of NAOD, the response was slower, but
was larger relative to the scatter of the AOD function.
• Examples are shown in figure 2.
APCCB 2004 7
0.9
0.7
0.5
0.3
0.1
-0.1
-0.3
-0.5
Figure 2.
AOD(2)
AOD2
14
1
0.9
0.7
0.5
0.3
0.1
-0.1
-0.3
-0.5
11
36
21
31
41
51
41
51
AOD(10)
AOD
10
10
1
0.9
0.7
0.5
0.3
0.1
-0.1
-0.3
-0.5
11
20
21
31
AOD(50)
AOD
50
18
1
11
21
30
31
41
Sample Number
51
• AOD functions for NAOD
of 2, 10 and 50.
• CVa = 0.1, CVwi = 0.2
61 • A fixed bias of 0.3 was
introduced at sample 10.
• The red lines show the 2.5
x SDAOD. For AOD2 the
value is 0.56; for AOD10,
0.24; for AOD 50, 0.11.
61
• Five example AOD
functions are shown in
each graph ( ).
• Blue arrows show N90.
• Orange arrows show
average first rule firing.
61
APCCB 2004 8
Error Detection with AOD
• Error detection of bias can be measured as the number of
delta values (samples with previous results) required to
detect changes in bias with specified certainty.
• N90 and average first detection for a range of values for
Cva/Cvwi and NAOD are shown in figure 3.
• The following conclusions can be drawn:
– Earlier error detection occurs with lower values of
Cva/Cvwi
– Error detection generally varies with NAOD in a “U-shape”
with an optimal range of values depending on CVa/CVwi .
• Examples of actual values for CVa/CVwi are in the table.
APCCB 2004 9
st Firing
Average11st
Average
Firing
50
45
40
35
30
25
20
15
10
5
0
A
fifty
twenty
ten
five
two
0
90
NN90
NAOD
1
2
CVwi/CVa
3
CVwi/CVa
50
45
40
35
30
25
20
15
10
5
0
B
NAOD
fifty
twenty
ten
five
two
0
1
2
CVwi/CVa
CVwi/CVa
Figure 3
• Number of samples
required for average 1st
firing (A) and N90 (B) for
detection of a shift of 2.5 x
CVa for various values of
CVwi/CVa and NAOD.
• Earlier error detection with
lower CVwi/CVa
• Optimal error detection
with NAOD 5-20
3
APCCB 2004 10
Table.
• Examples of Cvwi / CVa .
• CVwi from Westgard
Website
(www.westgard.com)
• CVa from SydPath
Laboratory (Olympus
AU2700)
APCCB 2004 11
Discussion
• The limitation of AON is the ratio of group biological
variation to analytical CV.
• AOD may outperform AON if:
– CVwi is small compared to between-person biological
variation (a low Index of Individuality).
– The frequency of samples with previous results is high.
– Clinics or weekends affect AON results. Note than AOD
should not be affected by change in patient mix as it uses
patients as their own control.
• AON may complement standard QC if it can:
– Detect smaller errors than standard QC
– Detect errors before standard QC
– Allow less frequent use of standard QC
APCCB 2004 12
Conclusions
• Average of Delta may allow improved error detection
without additional QC testing.
• The process would most suit tests as follows:
– A low within-individual biological variation compared to
the analytical variation.
– A high frequency of repeat testing.
• Software programs must be written to further evaluate and
this tool and allow for use in the routine environment.
APCCB 2004 13