”Improving cost effectiveness in the assessment of laboratories” Copenhagen, September 10, 1997 Joint EAL/EUROLAB Workshop in cooperation with EURACHEM No 2/97 Foreword EAL, European co-operation for Accreditation of Laboratories, (now merged with EAC to EA) and EUROLAB, European Federation of National Associations of Measurement, Testing and Analytical Laboratories, have jointly arranged biannually a workshop since 1992. Issues of interest to both parties have been treated on these workshops. In many cases there have also been different opinions and positions and therefore the discussions have been very interesting. Often are the discussions the most interesting part of these workshop. It has therefore been the intention to document the discussions in the proceedings from the workshops. It is however difficult to in a practical way reflect the exact wordings of the discussions. In these proceedings the discussions are extracted from the notes written by the EUROLAB Secretary during the workshop and the written discussions are therefore not reproduced in the exact wordings used during the workshop and there is even the possibility that there may be misunderstandings. If so, the EUROLAB Secretary is responsible for such misunderstandings. 533576078 Introduction by the EUROLAB President, the Secretary General of EAL and the Vice-Chairman of EURACHEM Claes Bankvall, President EUROLAB, SP The President of EUROLAB, Mr Bankvall, opened the workshop and thanked the Force Institute represented by Mr Larsen and Mrs Jürgenssen, for the hosting and the arrangement of the workshop. He also welcomed the participants and especially the participants from EURACHEM since it was the first time the members of the EURACHEM Executive Committee had the opportunity to participate in this type of workshops. In the future it might be possible to activate also other sectorial groups. This is the third of the joint workshops. They are considered by EUROLAB to be very important and they are valuable tools in the co-operation between the laboratories and the accreditors. Another very important tool in this co-operation is the Permanent Liaison Group between EAL and EUROLAB. The President pointed out that the activity of the participants in workshop is the key to the success of it. Peter van de Leemput, Secretary General of EAL, RvA The Secretary General of EAL, Mr van de Leemput was the representative of EAL. Mr van de Leemput presented the apologies of Mr Kaarls who did not had the possibility to participate. This is the last workshop to be arranged in co-operation between EAL EUROLAB and EURACHEM. This is not due to any dissatisfaction but is due to the merger between EAL and EAC to EA. The workshop is a good opportunity for fruitful discussions and the workshop has a historical background since it has been arranged several other times and the understanding between the accreditors and laboratories has improved greatly during the years. The costs for accreditation have been high during the last ten years, mostly due to the development of quality systems and of the conformity assessment system. The costs have now been stabilised and the new version of the ISO Guide 25 may introduce possibilities to lower the costs for accreditation, but it is important to remember that the main purpose of accreditation is to satisfactorily assure the confidence in the competence of laboratories to their customers and this leads to high demands on the assessors which in turn leads to rather high costs. Veikko Komppa, Vice-Chairman of EURACHEM, VTT The Vice-Chairman of EURACHEM Mr Komppa welcomed the participants and gave the regards from Marie Walsh who was unable to attend the workshop. This is the first time EURACHEM is participating in this sort of workshops. Earlier, EURACHEM has been represented by single laboratories who have been members of EUROLAB. The organisation of EURACHEM is very similar to the organisation of EUROLAB, but it has a different role than EUROLAB. EURACHEM is organising analytical laboratories and has members in 25 European countries. The work in the joint EEE-groups is by EURACHEM considered as valuable tools for quality assurance in Europe. 533576078 Cost effectiveness optimisation of the accreditation process 533576078 Internal Audit and review: a way to minimize surveillance Harry Gundlach Raad voor Accreditatie The Netherlands 533576078 Discussion on ”Internal Audit and review: a way to minimize surveillance” Mr Bryden asked about the qualification criteria and especially how to demonstrate costumer satisfaction. The laboratories are already performing investigations of the satisfaction of their customers, e.g. by sending out questionnaire to their customers. Are these sorts of investigations enough to show customer satisfaction or are there additional requirements? Mr Gundlach could not exactly answer the question at the moment since it is still under discussion. A customer satisfaction questionnaire will most surely be a part of the way to show the customer satisfaction. A question concerning the meaning of independent internal audits was raised by the audience. Mr Gundlach answered that the audits shall be performed in accordance with ISO 10011, have a qualified lead auditor, cover at least all requirements of EN 45001. The audit team shall be independent from the activity to be audited and finally audit reports shall form a basis for the management report. Mr Gundlach was asked how this leads to a reduction in costs. He answered that since the accreditors are using the internal audits which are already performed by the laboratories instead of performing their own audits with assessors the costs will be lowered. There will of course be traditional audits but they will be fewer. Mr Steck asked if Mr Gundlach could assure that the national accreditation bodies will accept all internal audits? Mr Gundlach could not assure that all national accreditation bodies will accept all internal audits but he assured that the main goal for the accreditors is to have a good accreditation and not to earn money. Mr Andersson raised several questions and comments in connection with Mr Gundlach presentation. He was wondering what the requirement that internal audits should form the basis for accreditation reviews may lead to if they are formalised to the equivalent in extent to third party assessment. He also stressed that additional requirements to EN45000 should not be introduced and that the approach of internal audits and review as a way to minimise surveillance might be just a way to move costs within the laboratories. The most important matter is to have a level playing field in Europe. Mr Gundlach stated that it is not the accreditors’ intention to introduce new requirements to EN45001 but clarify the already existing requirements Mr Tsimillis pointed out that the picture shown by Mr Gundlach was not correct since there is requirements for internal audits even if the accreditation is performed in the more traditional manner. Mr Gundlach agreed. Mr Larsen stressed that the laboratories already perform internal audits and that EAL wants to increase the frequency of the internal audits. He asked Mr Gundlach if he was prepared to work against such an increase. Mr Gundlach promised that RvA should take up the matter for discussion in EAL. But he also stressed that a high frequency of internal audits leads to a lower frequency of the surveillance audits which in turn leads to lower costs for the laboratories. Mr Eberhardt pointed out that demands for an increase of the frequency of internal audits in turn leading to fewer surveillance audits just is a shift of costs and in fact may lead to an increase of the total costs for accreditation. He also mentioned that the majority of customer complaints concerns cost for the test. Mr Gundlach stated that customer complaints concerning the cost for the test are not the issue for the accreditors and that the added value is gained since the laboratories already are performing internal audits. With this approach these audits will be used in the accreditation process. 533576078 Mr Eggimann agreed with Mr Andersson. He especially stressed that the customer satisfaction is measured by the laboratories acceptance on the market. He also repeated that no additional standards or requirements should be added. He also saw a great danger in the audit reports forming the basis for the management review, with such a system the laboratories will form an internal police, which Mr Eggimann is strongly against. All levels in a laboratory should be responsible for the corrective actions etc. mentioned in the internal audits reports. Mr Gundlach answered that the management should not deal with all the internal audit reports but only with summaries of them and therefore there should be no internal police. Good auditors are meeting the requirements in ISO10011. Mr Gundlach stressed that good industry are usually not satisfied with only complying with the lowest requirements but is aiming at a higher level of quality and that the words ”at least” not is equivalent with the word ”more”. 533576078 The cost effectiveness of accreditation A Laboratory perspective Richard Worswick Laboratory of the Government Chemist, LGC United Kingdom 533576078 Discussion on ”The cost effectiveness of accreditation - A Laboratory perspective” Mr Steck described the situation at his own laboratory. They have four different quality management system in use and one common assessment for several different systems would be very valuable for his laboratory. He was however not in favour of too much harmonised standards since different standards are used for different activities in the laboratory. Mr Steck also mentioned that in his view the new revised ISO Guide 25 will be negative for smaller laboratories while it will be beneficial for larger laboratories. Mr Ots was surprised of the lack of correlation between good results in interlaboratory tests and accreditation of the laboratories. Earlier investigations have shown different results and in fact some of the proficiency testing schemes were started to find out if accreditation lead to better performance. Mr Worswick was also surprised by the results. He is in favour of accreditation but would have preferred a better outcome for the accredited laboratories. It seems however as the accredited laboratories are better in making corrective actions and also to improve from the proficiency tests. Maybe the situation is becoming similar to the one in the ISO 9000 community where the performance of an organisation is not at all affected by an ISO 9000 certification. Mr Solumsmoen supported Mr Worswick’s view that customer complaints very seldom concern the paperwork in the quality system. Mr van de Leemput pointed out that it seems like the personnel in the laboratories are becoming the most important factor for the quality in laboratories. He was wondering if accreditation of people will be an important issue for the laboratory community in the near future. Mr Worswick agreed on the great importance of the laboratory personnel and mentioned that he had noticed several problems in the recent transformation of his organisation LGC. Changes of an organisation is always problematic. The culture of the organisation is extremely important and such things as rewards are becoming important tools to build up such a culture. Mr Worswick is however hesitating to accredit people. 533576078 Assessment of laboratories competence 533576078 The effective and efficient assessment of the competence of testing laboratories applying for accreditation Hans-Peter Ischi Swiss Accreditation Service, SAS Switzerland 533576078 Discussion on ”The effective and efficient assessment of the competence of testing laboratories applying for accreditation” Mr Bryden asked if the concept of the 3 categories of laboratories is formally expressed in the accreditation certificate and if it is in agreement with ISO Guide 25. Mr Ischi answered that the Swiss accreditation board has applied the concept of the three categories since the start of laboratory accreditation and the category of the laboratory is mentioned in the extended annexes to the scope of accreditation. Both the ISO Guide 25 and the document EAL P10 is of great help when using the three category approach. Mr Albutt liked the idea of having different levels. It seems however as if there are different procedures around Europe and there is not a level playing field in Europe. According to Mr Ischi there is not a big difference between the way to apply the scope of accreditation in different countries. All accreditation bodies have the same goal even if there are different procedures. All members in EAL have agreed upon EAL P10 and therefore there should not be big differences. Mr Albutt responded that in his opinion EAL P10 was accepted around Europe but not implemented. Mr Forstén discussed how far it is possible to go with accreditation and if it is possible to accredit personal competence and how far. It is not possible to go much further in the accreditation of quality systems documentation. In the revised ISO Guide 25 there are more issues concerning the customer and how to deal with customers. This is the borderline of the laboratories and accreditation must understand that in many cases they are not asking for the same things as the customers, e.g. laboratories are very seldom discussing the measurement uncertainty with their customers, but it is a never ending story in the discussions with the accreditors. To the accreditors there should be more important things than the paperwork to focus on. Mr Ischi argued that a good assessor can feel if there is a good culture and a good management in the laboratory just after a short visit. When a type C laboratory is accredited the focus is put on the validation process and the fitness for purpose approach is used and the most important question is ”what are the needs of the customers?”. Mr Solumsmoen stressed that the contract review is important both to type B &C laboratories. The part of EN 45000 dealing with these matters are not used today. Mr Ischi agreed that Mr Solumsmoen’s comment is correct. In EN 45001 it is written that the laboratories shall understand the demands of the clients. Mr Worswick expressed doubts in the accreditors competence to assess personal competence among laboratory personnel. 533576078 Relevant elements of laboratory competence Hans Andersson SP - The Swedish National Testing and Research Institute Sweden 533576078 Discussion on ”Relevant elements of laboratory competence” Mr Andersson stated that complaints from the customers of the laboratories very often are related to ”bad” contract reviews. Mr Steffen would like to add costumer co-operation to the agenda of meetings between the accreditors and their stakeholders and also to the agenda for the future Advisory Committee. Mr van de Leemput also agreed especially in two points, the competence and the feedback. But one should remember that there are different definitions of the word competence. A commonly agreed definition should be used. He also noticed that legislation is a one-way system with very little feedback. The laboratories has an automatic feedback due to their contacts with their customers. 533576078 Intercomparisons/proficiency testing 533576078 Comparison to independent reference values as a tool for creating confidence Paul de Bièvre European Commission Institute for Reference Materials and Measurement Belgium 533576078 Discussion on ”Comparison to independent reference values as a tool for creating confidence” Mr Golze pointed out that uncertainty should be used in connection to the procedures used and what is technical possible to achieve. Mr van de Leemput agreed and pointed out that feedback from the customers about what is needed concerning uncertainty is very valuable. He also believe that what is needed is too much affected by what is possible. 533576078 EAL Interlaboratory comparisons: a prerequisite for establishing mutual recognition agreements between accreditation bodies Klaus Brinkmann Physikalisch-Technische Bundesanstalt Germany 533576078 Discussion on ”EAL Interlaboratory Comparisons: A prerequisite for establishing and maintaining mutual recognition agreements between accreditation bodies” Mr Steck was a little bit shocked by the fact that there are demands on the laboratories for mandatory participation in proficiency test schemes. Mr Steck supports the idea of proficiency tests, but he questions if all laboratories have to participate in all proficiency test schemes. Mr Brinkmann stressed that for the MRAs it is very important to assure confidence across the borders and the proficiency test schemes are important tools to assure this confidence. It is not necessary for all laboratories in one country to participate in all international proficiency test schemes, just a few laboratories in each country have to participate in each PT scheme. Mr Forstén pointed out that the ISO Guide on proficiency tests was influenced early by calibration and there are many areas in testing where it is impossible to apply the thinking from calibration. Mr Brinkmann agreed. Mr Steffen pointed to the work in the joint EAL-EURACHEM-EUROLAB group on the use of proficiency test in the accreditation procedures which is important. It is also important to avoid starting new schemes but to use old ones already in function. Mr Brinkmann agreed. Mr Larsen mentioned that to be able to make corrective actions in time the laboratories need to read the report from the proficiency tests soon after the test has been performed. Therefore he asked how long time it takes from the test results are reported to the report is distributed in the EAL proficiency test schemes. Mr Brinkmann answered that it takes four weeks. 533576078 Improving cost effectiveness in the assessment of laboratories Alan Bryden, LNE 533576078 Discussion on ”Improving cost effectiveness in the assessment of laboratories ” Mr Tsimillis was surprised by the lack of consistency in the results of the interlaboratory test reported by Mr Bryden especially since the 44 participating laboratories all were notified bodies. Mr Bryden agreed but could not explain the lack of consistency. Mr Steffen asked for a list of the participating laboratories to be sent to the joint EAL-EURACHEMEUROLAB group on the use of proficiency test in the accreditation procedures. Mr Bryden will do so. Mr Worswick mentioned that a possible solution is to exclude the laboratories which are performing badly. But he is hesitating to use PT schemes to exclude the badly performing laboratories, instead PT schemes should be used to improve the performance of the laboratories. Mr Steffen believes that accreditors should look more to the corrective actions that to the actual results of the PT schemes. Mr Worswick pointed to the fact that there is a difference between calibration and testing in the way PT schemes may be used. He also wonders if the accreditors have the competence to check corrective actions. He also mentioned that the PT schemes are expensive and they can not be used in all cases. As an example he mentioned that LGC is accredited for over 250 methods but only have the possibility to participate in 14 PT schemes. Mr van de Leemput saw a possible way out, if the quality on the laboratory floor needs to be improved a ”target concept” with a target value could be used 533576078 Customers expectations 533576078 Will the accreditors ever be able to satisfy the expectations of their customers? Heikki Ots SWEDAC Sweden 533576078 Discussion on ”Will the accreditors ever be able to satisfy the expectations of their customers?” Mr Holcombe asked if the questionnaires gave a hint of the wish of the laboratories to have harmonised standards, e.g. GLP and EN45001. According to Mr Ots that is the situation in Sweden. Mr Bryden stressed that for him an EN 45001 which is harmonised with ISO 9000 is very important and therefore should be aimed at. Mr Bryden suggested that this sort of questionnaires should be distributed all over Europe regularly and have a common format which could be discussed in the PLG. Mr Ots mentioned that SWEDAC is making the enquiry every 3rd year. A matter to be discussed is to whom the questionnaire should be sent. Should e.g. authorities and the accreditation bodies themselves be allowed to answer the questionnaire? Mr Worswick asked if exactly the same questions were asked the second time and if SWEDAC used help from professionals when putting the questionnaire together? Mr Ots answered that not exactly the same questions were asked and this was due to the fact that the experience from the first questionnaire was used when the questions in the seconded was constructed. Mr Larsen mentioned that EUROLAB Danmark was asked by DANAK to help formulate the questions in the Danish questionnaire. Mr Albutt stated that the consumers look at the safety and reability of a product and that the product gives good value for the money. These are the important things for the customers. Not if the laboratory has a good quality system. Mr Saed raised the question of how accreditation is promoted and if so if it is promoted enough. Mr Albutt stressed that accreditation has to be promoted much more, especially to authorities, governments and industry. Mr Ots agreed. Mr Ots once more stressed the importance of promoting accreditation and to influence the society. 533576078 Customer expectations Ken Albutt AMTAC Laboratories United Kingdom 533576078 Discussion on ”Customer expectations” The discussion concerning this point was moved to and included in the final discussion. 533576078 Final discussions on lecture topics and discussion on the revision of ISO Guide 25 Mr Steck mentioned that he was very close to an approval of the revised ISO Guide 25. There are three different part of the revised guide about which Mr Steck has questions: 1) Is it necessary to have a written permission from the customer when the laboratory is using a non routine in-house test method? 2) What is meant by the word regularly? 3) Is it necessary to validate standard methods? Mr van de Leemput said that it is not the intention of the draft group that standardised test methods should be validated. Neither should written permission from the customer be asked for when using non routine inhouse methods. Mr van de Leemput described the future plans concerning the work with the revised guide. At the moment the comments are collected and they will then be categorised. The comments will be discussed in the drafting group on October 23rd and 24th. A new draft will be ready in February. It has been decided that the new version will be published as a standard and not as a guide. USA, France and Australia were against this decision and they would have preferred publication as a guide. Mr Albutt asked if there will be a transition period when the standard is approved? Mr van de Leemput did not think that the matter of a transition period is so important since even though the new standard will be different from the old guide e.g. with new criteria, the differences on the laboratory floor level will not be so big. The question of having a world-wide harmonisation is at the moment discussed in an ILAC working group. Mr Forstén stressed that it is important that there exists a good draft of a document describing how to assess according to the revised ISO Guide 25 before the work with the ISO Guide is finished. A draft assessment document should be ready at the latest in February. Mr van de Leemput agreed and stressed that the continued work with the revised ISO Guide 25 must be closely discussed with the laboratories. Mr Steffen believes that the outcome of this workshop will set a trend for future accreditation. Important ideas which has been introduced are: a reduction of the accreditors’ audits and more weight on internal audits performed by the laboratories themselves, more importance is put on proficiency tests and inter comparisons, there should also be a common European questionnaire on customer satisfaction. Mr Eberhardt noted that accreditation grows but the number of laboratories does not grow. Mr van de Leemput answered that it is the accreditation of laboratories that is growing. But if the need for accreditation is decreasing in the future the accreditors are prepared to reduce staff. Mr Holcombe noted that there are about 400-500 accredited laboratories in United Kingdom while there are 20000 laboratories which are possible costumers for the accreditors. There are also totally new areas for accreditation e.g. the medical area. Mr Erp Taalman Kip discussed the correlation between ISO Guide 25 and ISO 9000. There is a difference in culture between the actors in the two systems which may cause problems. LGC seems to have a good solution where the whole laboratory is certified according to ISO 9000 and some parts are accredited according to EN 45001. Mr van de Leemput agreed. Mr Saed stressed the importance of the possibility to have simultaneous implementation of OECD GLP and EN 45001. Mr van de Leemput pointed to the fact that there are some ”political” problems with simultaneous implementation of GLP and EN 45001. This issue will be dis- 533576078 cussed in the EAL committee as well as in the EAL Executive Committee and the EUROLAB Board of Administrators. Mr Jarl Forstén put forward that from the laboratories’ point of views there are different requirements in the different standards and it is a fact that most of the laboratories’ customers do not ask for accreditation partly since accreditation is not well known by industry. Some of the customers ask for ISO 9000 certification or GLP, also different sorts of classification are sometimes asked for. In some cases the requirements in the different quality management systems are contradictory. The laboratories want to operate with one quality system and have fewer audits. Mr Tsimillis brought up the question of co-ordination between different operators of PT schemes. Mr van de Leemput was convinced that there will be more co-ordination between the operators of PT schemes when EAC and EAL will merge. Mr Saed pointed to problems in connection with uncertainty. First, there is a gap of knowledge between those active in the testing community and those active in the university world where the concept of uncertainty is not used much in the education. There is also a need for a pragmatic approach from the accreditors when dealing with uncertainty statement in test reports. The customer and his/hers use of the test results should be considered. Mr de Bievre was pessimistic concerning the use of the word ”uncertainty”. In his opinion it will take at least ten years before it is functioning. He suggested the introduction of the word ”reability” which he suggested should be used instead of ”uncertainty”. Mr Steffen stressed that it is very important to continue the work to reach consensus concerning uncertainty. 533576078 Summary The EUROLAB President summarised the workshop. The workshop had been dominated by four issues which could be called the four C:s, namely: Culture Competence Cost effectiveness Customers In particular the workshop has underlined the following issues: 1) Use QA integrated in management systems 2) Prioritise personal responsibility, personal attitudes and personal competence 3) Use and weigh the essential elements for synergy (e.g. from ISO-Guide 25, example of synergy: proficiency test/internal audit/ technical assessor) 4) Set up an intercomparison program. The EEE-group on proficiency testing could be executive body for this task. 5) Identify areas where accreditation will support ”fitness for use”, to identify future developments 6) As many laboratories are working in a commercially competitive market costs including accreditation costs must be analysed with the aim to be cost-effective. The accreditation bodies have in general a monopoly in their country. Therefore it is very important that they operate cost-effectively and to a level needed. 7) Minimise the number of extra guidance documents. There is a need for harmonised standards, the proliferation of requirements should however be avoided and the amplification of sectorial solutions in explanatory documents should be kept low. 8) A new assessment document or procedure for accreditation according the revised ISO Guide 25/EN 45001 seems to be needed as not all requirements (e.g. cost-effectiveness, customer service) easily fits into the accreditation surveillance. 9) Continue and enhance co-operation between customers, laboratories and assessors as well as promotion of accreditation 10) The PLG will put together a questionnaire for investigation of customers satisfaction to be sent to the accreditors customer. 533576078