”Improving cost effectiveness in the assessment of
laboratories”
Copenhagen, September 10, 1997
Joint EAL/EUROLAB Workshop in cooperation with EURACHEM
No 2/97
Foreword
EAL, European co-operation for Accreditation of Laboratories, (now merged with EAC to EA) and
EUROLAB, European Federation of National Associations of Measurement, Testing and Analytical Laboratories, have jointly arranged biannually a workshop since 1992. Issues of interest to both parties have been
treated on these workshops. In many cases there have also been different opinions and positions and therefore the discussions have been very interesting. Often are the discussions the most interesting part of these
workshop. It has therefore been the intention to document the discussions in the proceedings from the workshops. It is however difficult to in a practical way reflect the exact wordings of the discussions. In these proceedings the discussions are extracted from the notes written by the EUROLAB Secretary during the workshop and the written discussions are therefore not reproduced in the exact wordings used during the workshop and there is even the possibility that there may be misunderstandings. If so, the EUROLAB Secretary
is responsible for such misunderstandings.
533576078
Introduction by the EUROLAB President, the Secretary General of EAL and
the Vice-Chairman of EURACHEM
Claes Bankvall, President EUROLAB, SP
The President of EUROLAB, Mr Bankvall, opened the workshop and thanked the Force Institute represented by Mr Larsen and Mrs Jürgenssen, for the hosting and the arrangement of the workshop. He also welcomed the participants and especially the participants from EURACHEM since it was the first time the
members of the EURACHEM Executive Committee had the opportunity to participate in this type of workshops. In the future it might be possible to activate also other sectorial groups.
This is the third of the joint workshops. They are considered by EUROLAB to be very important and they
are valuable tools in the co-operation between the laboratories and the accreditors. Another very important
tool in this co-operation is the Permanent Liaison Group between EAL and EUROLAB.
The President pointed out that the activity of the participants in workshop is the key to the success of it.
Peter van de Leemput, Secretary General of EAL, RvA
The Secretary General of EAL, Mr van de Leemput was the representative of EAL. Mr van de Leemput
presented the apologies of Mr Kaarls who did not had the possibility to participate.
This is the last workshop to be arranged in co-operation between EAL EUROLAB and EURACHEM. This
is not due to any dissatisfaction but is due to the merger between EAL and EAC to EA.
The workshop is a good opportunity for fruitful discussions and the workshop has a historical background
since it has been arranged several other times and the understanding between the accreditors and laboratories has improved greatly during the years.
The costs for accreditation have been high during the last ten years, mostly due to the development of quality systems and of the conformity assessment system. The costs have now been stabilised and the new version of the ISO Guide 25 may introduce possibilities to lower the costs for accreditation, but it is important
to remember that the main purpose of accreditation is to satisfactorily assure the confidence in the competence of laboratories to their customers and this leads to high demands on the assessors which in turn leads
to rather high costs.
Veikko Komppa, Vice-Chairman of EURACHEM, VTT
The Vice-Chairman of EURACHEM Mr Komppa welcomed the participants and gave the regards from Marie Walsh who was unable to attend the workshop. This is the first time EURACHEM is participating in this
sort of workshops. Earlier, EURACHEM has been represented by single laboratories who have been members of EUROLAB.
The organisation of EURACHEM is very similar to the organisation of EUROLAB, but it has a different
role than EUROLAB. EURACHEM is organising analytical laboratories and has members in 25 European
countries. The work in the joint EEE-groups is by EURACHEM considered as valuable tools for quality assurance in Europe.
533576078
Cost effectiveness optimisation of the
accreditation process
533576078
Internal Audit and review: a way to
minimize surveillance
Harry Gundlach
Raad voor Accreditatie
The Netherlands
533576078
Discussion on ”Internal Audit and review: a way to minimize surveillance”
Mr Bryden asked about the qualification criteria and especially how to demonstrate costumer satisfaction.
The laboratories are already performing investigations of the satisfaction of their customers, e.g. by sending
out questionnaire to their customers. Are these sorts of investigations enough to show customer satisfaction
or are there additional requirements?
Mr Gundlach could not exactly answer the question at the moment since it is still under discussion. A customer satisfaction questionnaire will most surely be a part of the way to show the customer satisfaction.
A question concerning the meaning of independent internal audits was raised by the audience. Mr Gundlach
answered that the audits shall be performed in accordance with ISO 10011, have a qualified lead auditor,
cover at least all requirements of EN 45001. The audit team shall be independent from the activity to be audited and finally audit reports shall form a basis for the management report.
Mr Gundlach was asked how this leads to a reduction in costs. He answered that since the accreditors are using the internal audits which are already performed by the laboratories instead of performing their own audits with assessors the costs will be lowered. There will of course be traditional audits but they will be fewer.
Mr Steck asked if Mr Gundlach could assure that the national accreditation bodies will accept all internal
audits? Mr Gundlach could not assure that all national accreditation bodies will accept all internal audits but
he assured that the main goal for the accreditors is to have a good accreditation and not to earn money.
Mr Andersson raised several questions and comments in connection with Mr Gundlach presentation. He was
wondering what the requirement that internal audits should form the basis for accreditation reviews may
lead to if they are formalised to the equivalent in extent to third party assessment. He also stressed that additional requirements to EN45000 should not be introduced and that the approach of internal audits and review as a way to minimise surveillance might be just a way to move costs within the laboratories. The most
important matter is to have a level playing field in Europe.
Mr Gundlach stated that it is not the accreditors’ intention to introduce new requirements to EN45001 but
clarify the already existing requirements
Mr Tsimillis pointed out that the picture shown by Mr Gundlach was not correct since there is requirements
for internal audits even if the accreditation is performed in the more traditional manner. Mr Gundlach
agreed.
Mr Larsen stressed that the laboratories already perform internal audits and that EAL wants to increase the
frequency of the internal audits. He asked Mr Gundlach if he was prepared to work against such an increase.
Mr Gundlach promised that RvA should take up the matter for discussion in EAL. But he also stressed that a
high frequency of internal audits leads to a lower frequency of the surveillance audits which in turn leads to
lower costs for the laboratories.
Mr Eberhardt pointed out that demands for an increase of the frequency of internal audits in turn leading to
fewer surveillance audits just is a shift of costs and in fact may lead to an increase of the total costs for accreditation. He also mentioned that the majority of customer complaints concerns cost for the test.
Mr Gundlach stated that customer complaints concerning the cost for the test are not the issue for the accreditors and that the added value is gained since the laboratories already are performing internal audits.
With this approach these audits will be used in the accreditation process.
533576078
Mr Eggimann agreed with Mr Andersson. He especially stressed that the customer satisfaction is measured
by the laboratories acceptance on the market. He also repeated that no additional standards or requirements
should be added. He also saw a great danger in the audit reports forming the basis for the management review, with such a system the laboratories will form an internal police, which Mr Eggimann is strongly
against. All levels in a laboratory should be responsible for the corrective actions etc. mentioned in the internal audits reports.
Mr Gundlach answered that the management should not deal with all the internal audit reports but only with
summaries of them and therefore there should be no internal police. Good auditors are meeting the requirements in ISO10011. Mr Gundlach stressed that good industry are usually not satisfied with only complying
with the lowest requirements but is aiming at a higher level of quality and that the words ”at least” not is
equivalent with the word ”more”.
533576078
The cost effectiveness of accreditation A Laboratory perspective
Richard Worswick
Laboratory of the Government Chemist, LGC
United Kingdom
533576078
Discussion on ”The cost effectiveness of accreditation - A Laboratory perspective”
Mr Steck described the situation at his own laboratory. They have four different quality management system
in use and one common assessment for several different systems would be very valuable for his laboratory.
He was however not in favour of too much harmonised standards since different standards are used for different activities in the laboratory. Mr Steck also mentioned that in his view the new revised ISO Guide 25
will be negative for smaller laboratories while it will be beneficial for larger laboratories.
Mr Ots was surprised of the lack of correlation between good results in interlaboratory tests and accreditation of the laboratories. Earlier investigations have shown different results and in fact some of the proficiency testing schemes were started to find out if accreditation lead to better performance.
Mr Worswick was also surprised by the results. He is in favour of accreditation but would have preferred a
better outcome for the accredited laboratories. It seems however as the accredited laboratories are better in
making corrective actions and also to improve from the proficiency tests. Maybe the situation is becoming
similar to the one in the ISO 9000 community where the performance of an organisation is not at all affected
by an ISO 9000 certification.
Mr Solumsmoen supported Mr Worswick’s view that customer complaints very seldom concern the paperwork in the quality system.
Mr van de Leemput pointed out that it seems like the personnel in the laboratories are becoming the most
important factor for the quality in laboratories. He was wondering if accreditation of people will be an important issue for the laboratory community in the near future.
Mr Worswick agreed on the great importance of the laboratory personnel and mentioned that he had noticed
several problems in the recent transformation of his organisation LGC. Changes of an organisation is always
problematic. The culture of the organisation is extremely important and such things as rewards are becoming important tools to build up such a culture. Mr Worswick is however hesitating to accredit people.
533576078
Assessment of laboratories competence
533576078
The effective and efficient assessment of the competence
of testing laboratories applying for accreditation
Hans-Peter Ischi
Swiss Accreditation Service, SAS
Switzerland
533576078
Discussion on ”The effective and efficient assessment of the competence of testing laboratories applying for accreditation”
Mr Bryden asked if the concept of the 3 categories of laboratories is formally expressed in the accreditation
certificate and if it is in agreement with ISO Guide 25. Mr Ischi answered that the Swiss accreditation board
has applied the concept of the three categories since the start of laboratory accreditation and the category of
the laboratory is mentioned in the extended annexes to the scope of accreditation. Both the ISO Guide 25
and the document EAL P10 is of great help when using the three category approach.
Mr Albutt liked the idea of having different levels. It seems however as if there are different procedures
around Europe and there is not a level playing field in Europe. According to Mr Ischi there is not a big difference between the way to apply the scope of accreditation in different countries. All accreditation bodies
have the same goal even if there are different procedures. All members in EAL have agreed upon EAL P10
and therefore there should not be big differences. Mr Albutt responded that in his opinion EAL P10 was accepted around Europe but not implemented.
Mr Forstén discussed how far it is possible to go with accreditation and if it is possible to accredit personal
competence and how far. It is not possible to go much further in the accreditation of quality systems documentation. In the revised ISO Guide 25 there are more issues concerning the customer and how to deal with
customers. This is the borderline of the laboratories and accreditation must understand that in many cases
they are not asking for the same things as the customers, e.g. laboratories are very seldom discussing the
measurement uncertainty with their customers, but it is a never ending story in the discussions with the accreditors. To the accreditors there should be more important things than the paperwork to focus on.
Mr Ischi argued that a good assessor can feel if there is a good culture and a good management in the laboratory just after a short visit. When a type C laboratory is accredited the focus is put on the validation process
and the fitness for purpose approach is used and the most important question is ”what are the needs of the
customers?”.
Mr Solumsmoen stressed that the contract review is important both to type B &C laboratories. The part of
EN 45000 dealing with these matters are not used today.
Mr Ischi agreed that Mr Solumsmoen’s comment is correct. In EN 45001 it is written that the laboratories
shall understand the demands of the clients.
Mr Worswick expressed doubts in the accreditors competence to assess personal competence among laboratory personnel.
533576078
Relevant elements of laboratory competence
Hans Andersson
SP - The Swedish National Testing
and Research Institute
Sweden
533576078
Discussion on ”Relevant elements of laboratory competence”
Mr Andersson stated that complaints from the customers of the laboratories very often are related to ”bad”
contract reviews.
Mr Steffen would like to add costumer co-operation to the agenda of meetings between the accreditors and
their stakeholders and also to the agenda for the future Advisory Committee. Mr van de Leemput also agreed
especially in two points, the competence and the feedback. But one should remember that there are different
definitions of the word competence. A commonly agreed definition should be used. He also noticed that legislation is a one-way system with very little feedback. The laboratories has an automatic feedback due to
their contacts with their customers.
533576078
Intercomparisons/proficiency testing
533576078
Comparison to independent reference values as a tool
for creating confidence
Paul de Bièvre
European Commission
Institute for Reference
Materials and Measurement
Belgium
533576078
Discussion on ”Comparison to independent reference values as a tool for creating confidence”
Mr Golze pointed out that uncertainty should be used in connection to the procedures used and what is technical possible to achieve. Mr van de Leemput agreed and pointed out that feedback from the customers
about what is needed concerning uncertainty is very valuable. He also believe that what is needed is too
much affected by what is possible.
533576078
EAL Interlaboratory comparisons: a prerequisite for
establishing mutual recognition agreements between accreditation bodies
Klaus Brinkmann
Physikalisch-Technische Bundesanstalt
Germany
533576078
Discussion on ”EAL Interlaboratory Comparisons: A prerequisite for establishing and maintaining mutual recognition agreements between accreditation bodies”
Mr Steck was a little bit shocked by the fact that there are demands on the laboratories for mandatory participation in proficiency test schemes. Mr Steck supports the idea of proficiency tests, but he questions if all
laboratories have to participate in all proficiency test schemes.
Mr Brinkmann stressed that for the MRAs it is very important to assure confidence across the borders and
the proficiency test schemes are important tools to assure this confidence. It is not necessary for all laboratories in one country to participate in all international proficiency test schemes, just a few laboratories in each
country have to participate in each PT scheme.
Mr Forstén pointed out that the ISO Guide on proficiency tests was influenced early by calibration and there
are many areas in testing where it is impossible to apply the thinking from calibration. Mr Brinkmann
agreed.
Mr Steffen pointed to the work in the joint EAL-EURACHEM-EUROLAB group on the use of proficiency
test in the accreditation procedures which is important. It is also important to avoid starting new schemes but
to use old ones already in function. Mr Brinkmann agreed.
Mr Larsen mentioned that to be able to make corrective actions in time the laboratories need to read the report from the proficiency tests soon after the test has been performed. Therefore he asked how long time it
takes from the test results are reported to the report is distributed in the EAL proficiency test schemes. Mr
Brinkmann answered that it takes four weeks.
533576078
Improving cost effectiveness in the assessment
of laboratories
Alan Bryden, LNE
533576078
Discussion on ”Improving cost effectiveness in the assessment of laboratories ”
Mr Tsimillis was surprised by the lack of consistency in the results of the interlaboratory test reported by Mr
Bryden especially since the 44 participating laboratories all were notified bodies. Mr Bryden agreed but
could not explain the lack of consistency.
Mr Steffen asked for a list of the participating laboratories to be sent to the joint EAL-EURACHEMEUROLAB group on the use of proficiency test in the accreditation procedures. Mr Bryden will do so.
Mr Worswick mentioned that a possible solution is to exclude the laboratories which are performing badly.
But he is hesitating to use PT schemes to exclude the badly performing laboratories, instead PT schemes
should be used to improve the performance of the laboratories.
Mr Steffen believes that accreditors should look more to the corrective actions that to the actual results of
the PT schemes.
Mr Worswick pointed to the fact that there is a difference between calibration and testing in the way PT
schemes may be used. He also wonders if the accreditors have the competence to check corrective actions.
He also mentioned that the PT schemes are expensive and they can not be used in all cases. As an example
he mentioned that LGC is accredited for over 250 methods but only have the possibility to participate in 14
PT schemes.
Mr van de Leemput saw a possible way out, if the quality on the laboratory floor needs to be improved a
”target concept” with a target value could be used
533576078
Customers expectations
533576078
Will the accreditors ever be able to satisfy the
expectations of their customers?
Heikki Ots
SWEDAC
Sweden
533576078
Discussion on ”Will the accreditors ever be able to satisfy the expectations of
their customers?”
Mr Holcombe asked if the questionnaires gave a hint of the wish of the laboratories to have harmonised
standards, e.g. GLP and EN45001. According to Mr Ots that is the situation in Sweden. Mr Bryden stressed
that for him an EN 45001 which is harmonised with ISO 9000 is very important and therefore should be
aimed at.
Mr Bryden suggested that this sort of questionnaires should be distributed all over Europe regularly and
have a common format which could be discussed in the PLG. Mr Ots mentioned that SWEDAC is making
the enquiry every 3rd year. A matter to be discussed is to whom the questionnaire should be sent. Should e.g.
authorities and the accreditation bodies themselves be allowed to answer the questionnaire?
Mr Worswick asked if exactly the same questions were asked the second time and if SWEDAC used help
from professionals when putting the questionnaire together? Mr Ots answered that not exactly the same
questions were asked and this was due to the fact that the experience from the first questionnaire was used
when the questions in the seconded was constructed. Mr Larsen mentioned that EUROLAB Danmark was
asked by DANAK to help formulate the questions in the Danish questionnaire.
Mr Albutt stated that the consumers look at the safety and reability of a product and that the product gives
good value for the money. These are the important things for the customers. Not if the laboratory has a good
quality system.
Mr Saed raised the question of how accreditation is promoted and if so if it is promoted enough. Mr Albutt
stressed that accreditation has to be promoted much more, especially to authorities, governments and industry. Mr Ots agreed.
Mr Ots once more stressed the importance of promoting accreditation and to influence the society.
533576078
Customer expectations
Ken Albutt
AMTAC Laboratories
United Kingdom
533576078
Discussion on ”Customer expectations”
The discussion concerning this point was moved to and included in the final discussion.
533576078
Final discussions on lecture topics and discussion on the revision of ISO Guide
25
Mr Steck mentioned that he was very close to an approval of the revised ISO Guide 25. There are three different part of the revised guide about which Mr Steck has questions:
1) Is it necessary to have a written permission from the customer when the laboratory is using a non routine
in-house test method?
2) What is meant by the word regularly?
3) Is it necessary to validate standard methods?
Mr van de Leemput said that it is not the intention of the draft group that standardised test methods should
be validated. Neither should written permission from the customer be asked for when using non routine inhouse methods.
Mr van de Leemput described the future plans concerning the work with the revised guide. At the moment
the comments are collected and they will then be categorised. The comments will be discussed in the drafting group on October 23rd and 24th. A new draft will be ready in February. It has been decided that the new
version will be published as a standard and not as a guide. USA, France and Australia were against this decision and they would have preferred publication as a guide.
Mr Albutt asked if there will be a transition period when the standard is approved? Mr van de Leemput did
not think that the matter of a transition period is so important since even though the new standard will be
different from the old guide e.g. with new criteria, the differences on the laboratory floor level will not be so
big. The question of having a world-wide harmonisation is at the moment discussed in an ILAC working
group.
Mr Forstén stressed that it is important that there exists a good draft of a document describing how to assess
according to the revised ISO Guide 25 before the work with the ISO Guide is finished. A draft assessment
document should be ready at the latest in February. Mr van de Leemput agreed and stressed that the continued work with the revised ISO Guide 25 must be closely discussed with the laboratories.
Mr Steffen believes that the outcome of this workshop will set a trend for future accreditation. Important
ideas which has been introduced are: a reduction of the accreditors’ audits and more weight on internal audits performed by the laboratories themselves, more importance is put on proficiency tests and inter comparisons, there should also be a common European questionnaire on customer satisfaction.
Mr Eberhardt noted that accreditation grows but the number of laboratories does not grow. Mr van de
Leemput answered that it is the accreditation of laboratories that is growing. But if the need for accreditation is decreasing in the future the accreditors are prepared to reduce staff. Mr Holcombe noted that there
are about 400-500 accredited laboratories in United Kingdom while there are 20000 laboratories which are
possible costumers for the accreditors. There are also totally new areas for accreditation e.g. the medical area.
Mr Erp Taalman Kip discussed the correlation between ISO Guide 25 and ISO 9000. There is a difference in
culture between the actors in the two systems which may cause problems. LGC seems to have a good solution where the whole laboratory is certified according to ISO 9000 and some parts are accredited according
to EN 45001. Mr van de Leemput agreed. Mr Saed stressed the importance of the possibility to have simultaneous implementation of OECD GLP and EN 45001. Mr van de Leemput pointed to the fact that there are
some ”political” problems with simultaneous implementation of GLP and EN 45001. This issue will be dis-
533576078
cussed in the EAL committee as well as in the EAL Executive Committee and the EUROLAB Board of
Administrators.
Mr Jarl Forstén put forward that from the laboratories’ point of views there are different requirements in the
different standards and it is a fact that most of the laboratories’ customers do not ask for accreditation partly
since accreditation is not well known by industry. Some of the customers ask for ISO 9000 certification or
GLP, also different sorts of classification are sometimes asked for. In some cases the requirements in the
different quality management systems are contradictory. The laboratories want to operate with one quality
system and have fewer audits.
Mr Tsimillis brought up the question of co-ordination between different operators of PT schemes. Mr van de
Leemput was convinced that there will be more co-ordination between the operators of PT schemes when
EAC and EAL will merge.
Mr Saed pointed to problems in connection with uncertainty. First, there is a gap of knowledge between
those active in the testing community and those active in the university world where the concept of uncertainty is not used much in the education. There is also a need for a pragmatic approach from the accreditors
when dealing with uncertainty statement in test reports. The customer and his/hers use of the test results
should be considered.
Mr de Bievre was pessimistic concerning the use of the word ”uncertainty”. In his opinion it will take at
least ten years before it is functioning. He suggested the introduction of the word ”reability” which he suggested should be used instead of ”uncertainty”. Mr Steffen stressed that it is very important to continue the
work to reach consensus concerning uncertainty.
533576078
Summary
The EUROLAB President summarised the workshop. The workshop had been dominated by four issues
which could be called the four C:s, namely:
Culture
Competence
Cost effectiveness
Customers
In particular the workshop has underlined the following issues:
1) Use QA integrated in management systems
2) Prioritise personal responsibility, personal attitudes and personal competence
3) Use and weigh the essential elements for synergy (e.g. from ISO-Guide 25, example of synergy: proficiency test/internal audit/ technical assessor)
4) Set up an intercomparison program. The EEE-group on proficiency testing could be executive body for
this task.
5) Identify areas where accreditation will support ”fitness for use”, to identify future developments
6) As many laboratories are working in a commercially competitive market costs including accreditation
costs must be analysed with the aim to be cost-effective. The accreditation bodies have in general a monopoly in their country. Therefore it is very important that they operate cost-effectively and to a level needed.
7) Minimise the number of extra guidance documents. There is a need for harmonised standards, the proliferation of requirements should however be avoided and the amplification of sectorial solutions in explanatory documents should be kept low.
8) A new assessment document or procedure for accreditation according the revised ISO Guide 25/EN
45001 seems to be needed as not all requirements (e.g. cost-effectiveness, customer service) easily fits into
the accreditation surveillance.
9) Continue and enhance co-operation between customers, laboratories and assessors as well as promotion
of accreditation
10) The PLG will put together a questionnaire for investigation of customers satisfaction to be sent to the
accreditors customer.
533576078