Dynamic Calibration and
Validation Testing
The world leader in serving science
Why is Dynamic Calibration Required
 Factory calibration calibrates each detector so it is free of interelement correlations and determines the factors in the A3 model
that enhances the performance of Thermo Gamma-Metrics coal
analyzers.
 But the factory can not exactly match the field conditions;
therefore fine tuning of the calibration is required in the field.
• Sources used in the field are different from what was used in the factory
• Coal stream is continuous through the unit in the field while it is a finite
length (equal to the calibration standard length) in the factory.
• The range of ash and sulfur are narrower in the field therefore the analyzer
can be tuned to be more accurate over the narrower range.
• The analyzer must be calibrated to the reference sample system.
2
Precision and Accuracy and Catch 22
Neither
Precise nor
Accurate
Precise
but not
Accurate
Precise
and
Accurate
Bias Adjustment Needed
The Catch 22 is we don’t know what the location of the bull’s eye is. We are all
measuring it with a number of systems each with their variances. Typically an
analyzer customer thinks his sample system and lab have no error and the
analysis results from the sample system/lab is the bull’s eye for an analyzer.
3
The objective
 During the field calibration and validation process, the objective
is to calibrate the analyzer to match the results of the
customer’s reference system, usually his sample system and
lab.
 The analyzer measures the quality of all the coal that passes
through it.
 The lab analyzes a very small subset of the material on the belt
to determine the quality of a large lot (usually one hour) of coal
flow.
 It is critical that the sample analyzed in the lab is “representative”
of the coal on the belt seen by the analyzer.
 It is critical that the lab prepare and analyze samples with
precise adherence to standard procedures,. with well maintained
equipment and well trained people.
4
Sample collection
 Collection of each gross sample is controlled by ASTM
Standards D2234 (Collection of a Gross Sample of Coal) and
D2013 (Preparing Coal Samples for Analysis) in the USA.
• D2234 covers collection of a gross sample without any crushing.
• D2013 covers the reduction and division of the gross sample
collected according to D2234 with crushing.
 To calibrate an analyzer generally requires 30 gross samples
covering a range of coal quality.
 At least 20 gross samples are required to validate the analyzer
calibration.
5
Minute-by-minute sulfur and ash data for a unit train
loading.
Loading from High Sulfur Pile
Dozer Pushing from
High Ash Pile
D 2234 defines how many increments are
required to accurately represent the quality of
this train.
6
The highlights of ASTM D 2234 collection of a gross
sample.
 Number of increment samples required to make up a single gross
sample?
• For coal with consistent quality (clean coal from a single seam) 15
increments up to a 1000 ton lot size.
• For inconsistent coal (raw coal, partial wash coal, any blend coal) 35
increments up to a 1000 ton lot size.
• For lot sizes greater than 1000 tons, the number of increments required is;
N 2  N1
total lot size (tons)
1000tons
N1 = minimum increments shown above
N2 = number of increments required for lot size
7
Example of number of increments required
 Loading a unit train
 Blended Coal dozed from different stacking tube piles; N1 = 35
 The unit train capacity is 10,000 tons
 Therefore
10,000
N 2  35
 110.68  111
1000
 111 individual increments are needed to represent the coal on the belt.
8
Impact of number of increments on the repeatability and how
well the sample represents the total lot.
As these graphs show, it is critical that enough increments are taken to accurately
represent the quality of the lot of coal. If too few increments are taken, the random
error in the sample collection process will negatively impact the calibration of the
analyzer.
9
The highlights of ASTM D 2234 collection of a gross
sample, continued.
 When a sample increment is taken the sampling device must have an
opening at least 2-1/2 to 3 times the top size of the coal, but never
smaller than 1.25-inches. This applies to stop belt sampling, falling
stream cutters or sweep samplers. Falling material must strike the
sample device at right angles to the opening.
 In addition the cutter speed is specified (18inches/second) and the
minimum amount of sample that has to be taken, based on the material
top size is also specified.
Top size
5/8 inch
16-mm
2 inch
50-mm
6 inch
150-mm
2-lbs
1-kg
6-lbs
3- kg
15-lbs
7-kg
Minimum increment weight
10
The highlights of ASTM D 2234 collection of a gross
sample, continued.
 If a very large sample increment is collected in the primary
sample, it is often required that the primary sample be reduced in
size with secondary sampling before a crushing step is
undertaken.
• At least six secondary increments are required for each primary
increment.
• All the rules concerning minimum increment weight, cutter speed
and cutter opening apply to this secondary sampler.
 Once the sample has passed through a crusher, ASTM D 2013
defines the sampling procedures.
11
ASTM D 2013 Preparing coal samples for analysis
 D 2013 covers both the manual and the mechanical reduction
and devision of samples collected according to D 2234.
• Applies to sample reduction after crushing in the sample system
• Applies to the lab manual reduction of the sample collected by a
sample system
 At least 60 increments are required from the sample taken by D
2234. Good practice recommends that at least one tertiary cut
be taken for every primary cut if this is a larger number than 60.
 The minimum sample size required is determined by how fine
the coal is crushed and whether the coal is 100% clean coal from
one seam or is it a variable coal quality (raw coal or blends).
12
ASTM D 2013 Preparing coal samples for analysis,
continued
Preparation of Laboratory Samples
Crush to 95% passing
through sieve
Divide to a minimum weight of, Grams
Group A
Group B
No.4 (4.75 mm)
2000
4000
No. 8 (2.36 mm)
500
1000
No. 20 (850 μm)
250
500
No. 60 (250 μm)
50
50
Table shows the minimum weight of the final sample depending
on the material size following crushing.
Group A = “coals that have been cleaned in all sizes”
Group B = “All other coals including unknown”
13
ASTM D 2013 Preparing coal samples for analysis,
continued
 Good practice in a sample system
• Cutter opening after crushing 1-¼ inch minimum opening
• Maximum cutter speed 18 inches/sec.
• Final sample weight collected by the sample system about 40
pounds.
 ASTM D 2013 manual sample reduction procedures must be
rigorously followed in the lab sample preparation process to
insure the sample is not biased or contaminated in the process.
 Errors made in sample collection or preparation will invalidate a
sample.
14
Dynamic Calibration of the Analyzer
 Dynamic calibration of the analyzer involves calibrating the analyzer so
it matches the reference system (the sample system and the lab).
 The Moisture, Sulfur, Ash Oxides and the Ash are calibrated by plotting
the analyzer results against the reference system.
Sulfur Analyzer versus Lab Results
1.2
1.1
1
Lab Sulfur
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.2
0.4
0.6
0.8
Analyzer Sulfur
15
1
1.2
1.4
Dynamic Calibration of the Analyzer
 As the X-Y plot of the sulfur calibration data showed, PGNAA
responses are linear; therefore, the field calibration of the
analyzer will be the equation of the linear regression fitting the
analyzer raw results to the reference system results.
 Before the linear regression is performed, the data should be
screened for outliers (data that does not match the trend seen
with the rest of the data).
 Outliers need to be addressed before the analyzer is calibrated.
16
Handling Outliers in the Data
 Errors can happen in the collection of samples, problems in the lab, improperly
recording values, etc. therefore it is not unexpected to see outliers in a data set.
 If an outlier is seen, an effort needs to be made to determine is there an
attributable reason. In the data below the sample weight for all samples except
the outlier were 40-pounds ± 4. The sample weight of the outlier was 8-pounds.
Something must have happened to the sample system. Therefore this point
was discarded and not used in the calibration.
Sulfur Analyzer versus Lab Results
1.2
Outlier data point discarded
from the data set because it
was determined the collected
sample weight was too small
to be representative of the
tons sampled.
1.1
1
Lab Sulfur
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.2
0.4
0.6
0.8
1
1.2
Analyzer Sulfur
17
1.4
Dynamic calibration
Sulfur Analyzer Versus Lab Results
1.20
Lab % Sulfur
1.00
y = 0.8108x + 0.0642
R2 = 0.9534
0.80
0.60
0.40
0.20
0.00
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
Analyzer % Sulfur
The dynamic calibration is completed by applying the gain (in this case 0.8108) and
offset (+0.0642) determined to the analyzer calibration. This is done for the ash and
moisture too, completing the dynamic calibration of the analyzer.
18
Validation Test
 Once the analyzer has been dynamically calibrated the validation test
can be performed.
 The validation test ideally consists of the collection of 30 samples,
collected and analyzed in exactly the same manner as the dynamic
calibration samples.
• As the normal guarantees are only for ash, sulfur and often moisture, the
ash oxides only need to be run if the customer is specifically interested in
them.
 The Root Mean Square Deviation (RMSD) of the analyzer to the lab
data is calculated.
n
RMSD 
2


analyzer

lab

i
i
i 1
n = number of data sets
n
19
RMSD example
Analyzer Ash
Lab Ash
Difference
Difference2
8.25
8.40
-0.15
0.0225
7.88
8.20
-0.32
0.1024
9.01
8.73
0.28
0.0784
8.65
8.82
-0.17
0.0289
7.50
7.22
0.28
0.0784
8.02
7.71
0.31
0.0961
Sum of differences
squared =
Number of samples =
RMSD =
20
0.4067
6
0.26
Acceptance of Analyzer Performance
 The analytical precision of the analyzer is guaranteed by Thermo.
 RMSD measures the combined error of the analyzer and the sample
system/lab reference system.
 If the assumption is made that the analyzer and the sample system/lab
have approximately the same precision (an assumption that has been
proven to be often true). Then the precision of the analyzer can be
estimated as:
RMSD
2
21
Acceptance continued
 If the measured RMSD of the analyzer is equal to or less than
the precision guarantee, the analyzer has passed the
acceptance test.
 If the RMSD is larger than the guarantee, the analyzer precision
may still satisfy the guarantee.
• If the lab and analyzer precision assumed to be equal, an RMSD of
0.707 would indicate an analyzer precision of 0.50.
 If the precision of the analyzer is in question, then ASTM D 6543
“Evaluation of Measurements Made by On-Line Coal Analyzers”,
details the procedure to conduct a Grubbs Estimator
measurement of the precision of the analyzer.
22
Grubbs Estimator
 Requires dual ASTM sampling of the coal stream at two different
locations.
 Requires the collection of 60 samples from each sample location
 Requires the samples to be processed completely by independent labs.
 The result is an estimate of the precision of the three analysis systems,
the coal analyzer, sample collection/lab one and sample collection/lab
two.
 The Grubbs estimate of the precision of the analyzer is what is
compared to the guarantee performance.
 Thermo’s guarantee is a one sigma guarantee (68% of the results will
fall within plus or minus the guarantee value), and sukbject to statistical
significance testing.
23
Precision of Sample Collection
 How well does the sample collected represent the true quality of
the lot?
• What is the precision of the sample collected by the sample system?
• For the example in slide 9 the measured ash had a precision of ±0.60 and the sulfur
precision was ±0.036 at the ASTM recommended increment.
• This number will be less for coal streams with less variability and more if the variability is
greater.
• The sample collected by the sample system goes back to the lab
and is air dried, crushed and manually reduced to 500 grams that
goes into the lab.
• The lab then runs very small subsets of the prepared sample in a
number of instruments.
24
Reproducibility Lab to Lab
 Repeatability is the ability of a single lab to duplicate the same
results when repeating the same analysis of the same sample.
 Reproducibility is how well the results from two different labs
should compare for the same sample.
 As the coal analyzer is one “lab” we should look at the
reproducibility of results on analysis system to another 95% of
the time (2 sigma).
• D 3173 Moisture Reproducibility - 0.3% for coals with less than 5%
moisture and 0.5% for coals with more than 5% moisture.
• D 3174 Ash Reproducibility – 0.32% on a dry basis
• D 4239 Sulfur Reproducibility – 0.02 + 0.09(average sulfur between
labs)
25