Shewhart or Levey- Jennings control charts
 In a stable test environment the distribution of the results of the same sample analyzed a number of times is a Gaussian distribution
 Deviation indicates a systematic error
 The levey-Jennings control chart is derived from the Gaussian distribution indicating the mean and the one, two, and three standard deviation ranges on both sides of the mean
 The chart illustrates the relationship between the levey-
Jennings control chart and the Gaussian distribution from which it is derived
 This figure also shows the expected percentage of results that should fall within each of the standard deviation ranges

Shewhart or Levey- Jennings control charts
Run Number/ Date

Shewhart or Levey- Jennings control charts
 On the control chart used to evaluate the result of the control runs o the dates or number of analyses are listed along the X-axis o and the values of the control along the Y- axis
 The mean and the 1,2 and 3 standard deviation (s) limits of the control analysis to date are marked
 As data is obtained it is plotted one point at a time along the chart

Shewhart or Levey- Jennings control charts o In a random distribution and in a correctly operating test system approximately: o o o
65% of the values will be between the
± 1s ranges and will be evenly distributed on either side of the mean
95% of the values should fall between the ± 2s ranges and 99% between the ± 3s limits
 This means that one data point in 20 should be placed between either of the 2s and 3s limits
 One data point placed outside of the 3s limits will occur once in
100 analyses

Shewhart or Levey- Jennings control charts
Run Number/ Date



The ± 2s limits are considered as warning limits
A value between the 2s and 3s limit indicates the analysis should be repeated
The ± 3s limits are rejection limits. Analysis should stop, patient results held and the test system investigated

 Inspecting the pattern of plotted points can lead to the detection of increased random error and shifts and/or trends in calibration or in the analysis process
 Example of a shift in calibration can be failure in recalibration when lot numbers of reagents changed during analytical run




A trend can start on one side of the mean and move across it or it can occur entirely on one side of the mean
Trends can be caused by the deterioration of reagents, pump tubing or light sources in instruments
Occurrence of shifts and trends is the result of either proportional or constant systematic error

 Two levels of control at different concentrations will be more efficient in monitoring the method when evaluated statistically
 By running and evaluating the results of two controls together, trends and shifts can be detected much earlier
 Westgard and associates have formulated a series of so called “multi-rules” to help evaluate results from
Gaussian distribution both within a level of control and between control levels
 Violation of any of these rules individually or in any combination indicates an error
9

10

• 1
2S rule
• 1
3S rule
• 2
2S rule
• R
4S rule
• 4
1S rule
• 10
X rule
11

2S


One of two control results falls outside ±2s
Warning Rule
– does not cause rejection of a run
 Alerts technologist to possible method or instrument malfunction
12

3s
 When the control limits are set as the mean ± 3s
 A run is rejected when a single control measurement exceeds the mean plus 3s or the mean minus 3s control limit
13

2S
 2
2s
- reject when 2 consecutive control measurements exceed the same mean plus 2s or the same mean minus 2s control limit
Level 1
Level 2
14

4S
 R4s- reject when 1 control measurement in a group exceeds the mean plus 2s and another exceeds the mean minus 2s
Level 1
Level 2
15

1S
 41s- reject when 4 consecutive control measurements exceed the same mean plus 1s or the same mean minus 1s control limit
Level 1
Level 2
16

x
 10x- reject when 10 consecutive control measurements fall on one side of the mean
Level 1
Level 2
17

18


3S
4S

2S
1S
X

Level 1 Glucose
Level 2 Glucose
3SD
2SD
1SD
M
-1SD
-2SD
-3SD
3SD
2SD
1SD
M
-1SD
-2SD
-3SD

Use of Patient Samples as Controls
 Using of patient samples for quality control is acceptable because:
1.
The physical and chemical differences between control material and patient samples are absent
2.
The data being generated by the analysis of the patient samples require no extra effort to obtain for use as quality control data
3.
Inexpensive method of quality control
21

 Values that are beyond reference intervals or are not consistent with the survival of life.
 Results should be checked by repeat analysis on a fresh aliquot of the same sample or by an alternate method.
 The values are usually the result of a random error or mistake such as short sampling, clot formation.
 Comparison of absurd values to the clinical picture of the patient provides validity of the result.
22

 In a proficiency survey, a laboratory is compared to other laboratories with the same or similar methodology by the analysis of identical samples sent to each of the participants.
 Upon receiving the survey specimen, the participant prepares it according to the provided instructions and analyzes it as a routine sample
 The results and information concerning the method of analysis are recorded on a provided report form and mailed back to the supplier of the survey.
24

 For the evaluation, results for each sample are grouped by analyte and methodology
 The performance of the group is determined either by calculating a mean result and standard deviation for each specimen or by determining the consensus or majority opinion
 The performance of the individual laboratories that comprise the group is evaluated by comparison to the group performance
 The mean or consensus result for a survey specimen is considered the accurate and true value for that sample
 The closer the individual laboratory result is to the survey mean or consensus, the more accurate it is
25

 CAP and similar surveys are designed to accomplish the following goals:
1.
2.
3.
4.
To assess the current state of the art of laboratory medicine
To provide information to help laboratory directors select the best methods and reagents
To satisfy the accreditation requirements
To provide a voluntary educational peer comparison program that allows a laboratory to compare its performance with its peers
26

 The proficiency survey samples sent to participating laboratories are very different from patient samples
 The rigors of shipping materials great distances and the need for a large batch of sample to send to all of the participants does not permit unstabilized or unfixed specimens to be used
 Proficiency samples include: o lyophilized serum, plasma, urine, spinal fluid, stabilized blood cells, ampules of whole blood for blood gas analysis, lyophilized cultures, photomicrograph transparencies, and stained blood films for identification
 Therefore, survey specimens may be different physically and chemically from patient samples, and are not subjected to collection and processing procedures
27




These differences prevent proficiency surveys from accurately reflecting the overall performance of the laboratory.
The preparation of survey material by the laboratory is critical
The specimens must be prepared according to instructions to reduce interlaboratory variation
 Exact details for the reconstitution of lyophilized samples are given, and sometimes the diluent is provided
 Once the sample is ready for analysis it should be treated as a routine sample
 The questionnaire or report form must also be filled out correctly
 Mistakes made in filling out these forms would be interpreted as analytical errors and may lead to time wasted in trying to investigate a bad result
28

B- Evaluation of Data from Quantitative
Procedures
 Quantitative procedures are evaluated statistically by comparing the individual laboratory's results to its peer group
 A peer group is defined as a minimum of 20 laboratories having the same or similar methodology or instrument in common
 When fewer than 20 laboratories are in a peer group, the group is evaluated against a chosen comparative method
 A comparative method is not always a reference method but one that has historically performed well and has good agreement with most of the common methods.
29

B- Evaluation of Data from Quantitative
Procedures
 The mean and standard deviation(s) of the peer group is calculated
 Outliers beyond three standard deviations are excluded and the mean and standard deviation is calculated a second time
 The second mean and standard deviation is used to calculate the Standard Deviation Index
 Mistakes such as reporting the wrong values for the wrong samples and the misplacing of decimal points are common transcription errors that can exert undue influence on the calculation of the mean and standard deviation statistics
 Their elimination will result in a more accurate picture of the peer group.
30

B- Evaluation of Data from Quantitative
Procedures
 The Standard Deviation Index (SDI) describes how far from the mean an individual laboratory's result falls in a peer group survey
 The closer to the mean or the smaller an individual laboratory's SDI, the more accurate the result
 The mean is accepted as the true value of the specimen
 All of the results for a survey are grouped by sample, analyte and methodology
 The mean and standard deviation is calculated for each peer group.
31

B- Evaluation of Data from Quantitative
Procedures
 The SDI is calculated as follows:
SDI =
Individual Lab Result — Group mean
Group Standard Deviation
 A chart showing the SDI of successive survey specimens will graphically display the laboratory's performance over several separate shipments or survey samples
 A cumulative SDI chart similar to the one illustrated in
Figures 8-1 and 8-2 demonstrates random and systematic analytical error by the pattern of the SDI values for successive surveys
32

B- Evaluation of Data from Quantitative
Procedures
 If a laboratory is consistently below the mean, the SDI will be a series of negative numbers
 If the laboratory is consistently above the mean, the SDI will be a series of positive values
 All SDIs beyond the ±2 SDI are unacceptable and need to be investigated
 Values falling in the ± 1.5 to ±2.0 SDI range, though acceptable, are flagged as a warning of the beginning of a possible trend or shift.
33

34

35

 In 1964 Dr. W. J. Youden proposed a method of evaluating precision as well as accuracy in chemistry and hematology analyzers and manual analytical methods using paired survey samples
 The technique involves sending two samples to participating laboratories
 One sample would have a low concentration, the other a high concentration
 The results are grouped by analyte and methodology and a mean and standard deviation is calculated for each sample
 A control chart is prepared so that one sample is on the x-axis and the other on y-axis
 Each axis is scaled so that 1 standard deviation is the same size on each coordinate and the mean value for each sample is at the midpoint
36

 The ± 3s (from the mean) limit for each sample is marked
 A line perpendicular to the axis is drawn from the mean for each sample
 A circle with a radius = 3s is drawn where the two perpendicular lines meet
 A 45 o line is drawn from the X-Y intercept, through the center of the circle, and beyond (Figure 8.3)
 After the Youden Plot is constructed, the individual laboratories are plotted, using the results of the paired samples
 This shows the distribution of the participants of that survey
37

38

 The closer an individual lies to the junction of the two sample means, the more accurate its performance
 Interpretation of the plot is as follows: o Laboratories falling near the intersection of the means and within the circle are considered accurate. The closer to the center, the more accurate the result o Laboratories that fall either high or low along the 45 o line or between lines parallel to it and tangent to both sides of the circle are indicative of systematic error, that is, the method is precise but not necessarily accurate o Laboratories outside of the 45 o parallel tangent lines are of random error
39

 An easier to construct, modified form of the Youden Plot can be made by using uniform scales on both axes and drawing rectangle that encompasses the area within the 3s limits of both samples (Figure 8.4 and 8.5)
 The 45 o line is drawn in but not the parallel tangent lines.
Interpretation is similar to the traditional form
40

42

43

 For qualitative procedures a "consensus method" or referees are used for evaluating individual results.
 A consensus method is one that has been chosen by either the majority of the participants or the survey committee as the best standardized and easily performed method for the analysis.
 The use of referees is most common in the identification of blood cells, urine sediment, and microorganisms.
 These individuals or laboratories receive samples, as do the other participants, and submit their results.
 The consensus of a group of referees is designated as the acceptable identification.
 Accurate and acceptable performance is in agreement with the referees or the consensus method.
44

E- Survey Evaluation by the Participants
 The survey summaries are returned to the participants for evaluation of their performance.
 The laboratory director or his designee, preferably the section head, should review the survey report and initiate an investigation of all unacceptable results.
 The findings of the investigation are to be documented, signed by the laboratory director, and retained with the survey report as part of a permanent record of the laboratory's participation
45

Possible causes of unacceptable proficiency survey performance
1.
2.
3.
4.
5.
6.
7.
8.
9.
Transcription errors made when filling out the report form.
Improper or incomplete reconstitution of lyophilized survey samples.
Inaccurate identification of cells, microorganisms, or formed urine sediment or transparencies or stained slides.
Inaccurately calibrated analyzer.
Analytical test procedure error, i.e., short sampling, timing error, loss of precision due to sample clot, or poorly maintained instrument.
Failure to provide a suitable environment for the growth of microorganism.
The use of reagents and solutions that have lost reactivity and sensitivity.
Mixing up of paired or multiple samples during analysis.
Damage to survey material during shipping, i.e.. exposure to excessive heat, or rough handling resulting in physical damage
46

F- Regional Quality Control Programs

 A laboratory buys sufficient control product to last for a year or longer.
 Each month the laboratory submits to the manufacturer's regional computer center either the results of all of its control runs grouped by control lot number and analyte or a summary of the control runs consisting of the monthly mean, standard deviation and the number of control runs for each analyte.

The manufacturers of commercial control material offer a form of external quality control through the use of regional quality control programs.
The computer center calculates a monthly mean, standard deviation, and coefficient of variation (CV) for the peer group that consists of all laboratories with the same method and lot number of control for each analyte.
47

Regional Quality Control Programs
 Comparison among the members of the peer group may be either by
SDI or Youden Plots.
 The SDI or the Youden Plots indicate where the laboratory stands with respect to the other members of the group and the group mean.
 Assuming that the mean value of each group is the accurate value for that control material the accuracy of each individual laboratory can be inferred by its position with respect to the mean.
 For laboratories who submit all of their data to the computer center a monthly mean s, and CV of each analyte is calculated.
 All participating laboratories receive the lot-to-date cumulative mean, s, and CV for each analyte in that lot of control. This summary data is returned to the participating laboratory for evaluation of its precision by the laboratory director.
48

Regional Quality Control Programs
 Regional quality control programs do not have the number of participants that the larger CAP and other survey programs are able to attract
 The vendors of commercial controls offer regional programs and statistical evaluation as an enhancement to purchase their product.
 Because of fewer participants each group may or may not be classified by methodology or instrumentation.
 An advantage of the regional programs is that the data is evaluated more frequently, on a monthly basis, and not quarterly, as with most surveys.
49

Fallacies Of External Quality Control
 Good performance on proficiency surveys and in regional quality control programs does not necessarily indicate a proficient laboratory
 If proficiency survey samples receive special treatment, a true picture of the laboratory's accuracy and analytical error is not shown
 Nor do these samples reflect any problems with collection, sample processing, instrument maintenance programs, transcription errors, or the rapport the laboratory has with the users of its services.
 The use of the results of the external quality control samples should be placed in proper perspective as an important but partial picture of the laboratory's performance.
50