Laboratory capacity and verification
testing report
Prepared for the
Vietnam Energy Efficiency Standards and Labelling (VEESL)
Program
Produced by
Chris Evans, Kevin Lane
399 Silbury Boulevard, Milton Keynes
MK9 2AH, United Kingdom
Email: enquiries@S2E4.com
Telephone: +44 (0)1908 303 600
www.S2E4.com
June 2014
Table of Contents
1
INTRODUCTION...................................................................................................................................... 7
2
EXISTING ACCREDITATION AND APPROVAL OF TESTING ORGANISATIONS ............................................ 9
2.1 ACCREDITATION BY THE BOA ..............................................................................................................................9
2.2 APPROVAL BY MOIT.........................................................................................................................................9
3
PROVISION OF EXPERT TRAINING ........................................................................................................ 10
3.1 ELECTRIC FANS ...............................................................................................................................................11
3.1.1 Report back to VEESL and MOIT from the electric fans expert ...........................................................11
3.2 RICE COOKERS ...............................................................................................................................................13
3.2.1 Report back to VEESL and MOIT from the rice cookers expert ...........................................................13
3.3 COMPACT FLUORESCENT LAMPS ........................................................................................................................14
3.3.1 Report back to VEESL and MOIT from the CFLs expert .......................................................................15
4
INTER-LABORATORY COMPARISON TESTS ........................................................................................... 17
4.1 RESULTS OF WITNESS TESTING ...........................................................................................................................17
4.2 RESULTS OF TESTING ELECTRIC FANS ...................................................................................................................18
4.2.1 Discussion of results of testing electric fans .......................................................................................21
4.3 RESULTS OF TESTING RICE COOKERS ....................................................................................................................23
4.3.1 Discussion of results of testing rice cookers .......................................................................................26
4.4 RESULTS OF TESTING CFLS ...............................................................................................................................28
4.4.1 Discussion of results of testing CFLs ...................................................................................................30
5
CONCLUSIONS AND RECOMMENDATIONS ........................................................................................... 33
5.1 PRODUCT TESTING RELATED ADVICE FOR EACH OF THE QUATEST LABORATORIES .......................................................33
5.2 RECOMMENDATIONS APPLICABLE TO ALL APPROVED TESTING ORGANISATIONS IN VIETNAM...........................................34
5.3 THE ACCREDITATION OPERATED BY THE VIETNAMESE BOARD OF ACCREDITATION (BOA) ..............................................34
5.4 SUPPORT FOR THE MOIT APPROVAL PROCESS ......................................................................................................34
5.5 RECASTING THRESHOLD FOR ENERGY LABEL BANDS ................................................................................................35
APPENDIX 1
EXAMPLE OF TEST RESULTS SHEET FOR ONE MODEL OF ELECTRIC FAN ............................... 36
APPENDIX 2
EXAMPLE OF TEST RESULTS SHEET FOR ONE MODEL OF RICE COOKER ............................... 37
APPENDIX 3
EXAMPLE OF TEST RESULTS SHEET FOR ONE MODEL OF CFL ............................................... 38
2
List of Figures
Figure 1: Testing of electric fans with support from the expert ...............................................11
Figure 2: Testing of rice cookers with support from the expert ...............................................13
Figure 3: Testing of CFLs with support of the expert .............................................................15
Figure 4: Electric fan manufacturer label and the energy label ...............................................19
Figure 5: Star rating of the 20 fans as measured by the three laboratories ..............................20
Figure 6: Maximum-to-minimum star rating of the 20 fans as measured by the three laboratories
.....................................................................................................................................20
Figure 7: Reported minimum-to-maximum efficiency of the 20 fans, by three laboratories ........22
Figure 8: Rice cooker and its energy label ...........................................................................23
Figure 9: Star rating of the 20 rice cookers as measured by the three laboratories ...................25
Figure 10: Maximum-to-minimum star rating of the 20 rice cookers as measured by the three
laboratories .....................................................................................................................25
Figure 11: Reported average cooking efficiency (%) of the 20 models of rice cookers ...............26
Figure 12: Example of CFL label .........................................................................................29
Figure 13: Average efficacy (lm/W) measured for the 20 models of CFLs ................................30
Figure 14: Average measured power (W) of the 20 CFL models .............................................31
Figure 15: Measured/reported initial luminous flux (lm) of the 20 CFL models .........................32
List of Tables
Table 1: Summary of findings by electric fans testing expert .................................................12
Table 2: Summary of findings by rice cookers testing expert .................................................14
Table 3: Summary of findings by CFLs testing expert ...........................................................16
Table 4: Brand/model list of electric fans tested ...................................................................18
Table 5: Brand/model list of rice cookers tested ...................................................................24
Table 6: Energy efficiency bands given in TCVN 8252:2009 ...................................................27
Table 7: Brand/model/size list of CFLs tested ......................................................................28
Table 8: Difference between the average efficacies of the tested CFL lamps ............................30
Table 9: Example of test results sheet for one model of electric fan ........................................36
Table 10: Example of test results sheet for one model of rice cooker ......................................37
Table 11: Example of test results sheet for one model of CFL ................................................38
Disclaimer
The views and opinions expressed in this article are those of the authors and do not necessarily
reflect the official policy or position of any agency of the Australian Government.
The authors have made their best endeavours to ensure the accuracy and reliability of the data
used herein, however make no warranties as to the accuracy of data herein nor accept any
liability for any action taken or decision made based on the contents of this report.
3
Executive summary
The Australian Government Department of Industry (DoI) is providing assistance to Vietnam
through the Vietnam Energy Efficiency Standards and Labelling (VEESL) Program, funded by the
Australian Agency for International Development (AusAID).
This report has been compiled by consultants with extensive experience of product testing who are
providing direct assistance and advice to the VESSL Program. It provides results for that part of the
Program intended to help build laboratory testing capacity in three state-owned QUATEST
laboratories.
The work being reported was undertaken in partnership with Vietnam’s Ministry of Industry and
Trade (MOIT) and comprised of two main components:


Visits to the test laboratories, QUATEST 1, 2, 3, by experts in the energy efficiency
performance testing of electric fans, rice cookers and compact fluorescent lamps (CFLs).
The purpose of the visits being to provide supplementary training to the already
experienced laboratory testing staff;
The inter-laboratory testing of 20 models of electric fans, rice cookers and CFLs (which
followed on from the visits by the experts).
Achieving accurate and consistent test results by these three laboratories is crucial for effective
standards and labelling in Vietnam. Methods for measuring declared values for energy
consumption and performance characteristics must be of sufficient accuracy to provide confidence
to governments, consumers and manufacturers. Any significant variations in the reproducibility1 of
results achieved by test laboratories could reduce the effectiveness of the implementation of the
MEPS and labelling regulations. It must be recognised that some variation in measurement is
unavoidable. This can be due to the variance of the product itself and to the measurement method
applied. While the first one is the responsibility of the manufacturer, the second one is the
responsibility of either or both the test laboratory(s) and the organisation publishing the test
standard.
The part of the Program reported here began with the supplementary training visits made to the
three QUATEST laboratories. In each case, the experts provided advice focussed on making
improvements to testing procedures - the precise application of testing methods, the use of
calibrated measuring equipment and the routine assessment of sources of errors of measurement.
1
reproducibility conditions
conditions where test results are obtained with the same method on identical test items in different laboratories with
different operators using different equipment
[IEC 61923:1997, 3.9]
4
The results of the inter-laboratory testing that then followed were mixed. Those of multiple
samples of the same models of CFLs showed, with the exception of consistently lower results from
one of the QUATESTs, good reproducibility – as should be expected.
The results of the testing of electric fans and rice cookers were much less reproducible with large
and inconsistent variations in performance measured for the same samples of some models. The
experts identified a number of reasons, which could have contributed to these variations, which
were the responsibility of the test laboratories:

Some inconsistencies in the precise application of techniques between the laboratories;

Some equipment not being as specified in the test standard;

Some equipment not being calibrated.
Additionally, in the case of the rice cookers, the expert identified that the published test method
needed amending as it could not be undertaken in ways that avoided significant errors of
measurement.
The challenges faced by laboratories when conducting performance tests on products should not
be underestimated. Relatively small changes in procedures or in the accuracy of measuring
instruments can have a significant impact on the energy efficiency ratings that are subsequently
calculated from the laboratory test results. Implementation of the advice that the experts gave to
each of the QUATEST laboratories was intended to have a positive impact on improving the
consistency of test results.
The authors of this report recommend further actions are taken. A multi-agency approach, in which
the Vietnam Board of Accreditation (BoA) takes part, should implement a programme to assist the
test laboratories to improve the suitability and calibration of their measuring equipment. In
addition, estimating and eliminating sources of measurement errors are also important actions to
be taken.
Some specific recommendations:


2
Amend the measurement standard TCVN 8252:2009 for rice cookers in order to improve its
repeatability2;
Assist the BoA to implement best practice when accrediting test laboratories to
ISO/IEC17025;
repeatability conditions
conditions where independent test results are obtained with the same method on identical test items in the same laboratory by the
same operator using the same equipment within short intervals of time
[IEC 61923:1997, 3.6]
5




Introduce a requirement that inter-laboratory trials are to be undertaken for each new test
standard to identify and resolve any inconsistencies in application of the test methodology
and so ensure that reproducible test results are consistently obtained;
Each QUATEST to implement a procedure under which uncertainties of measurement3 are
estimated for all the test programs they undertake.
Further and specific refinements to laboratory procedures should be made at each
QUATEST. The guidance provided by the testing experts and the comments presented in
this report are relevant to this.
Request the BoA to then undertake a re-accreditation program (which would normally be
undertaken annually) for the QUATEST laboratories;
3
The expression “uncertainties of measurement”, which is used frequently in this report, deals with assessing the variations in results
that can occur when making a measurement. Testing a product to measure its performance will always be affected by errors due to
instrumentation, methodology, presence of confounding effects and so on. Consequently, experimental uncertainty estimates need to
be undertaken by test laboratories to identify (and, where possible, reduce) the different sources of errors and to provide an assessment
of the confidence in the results.
It is a requirement of the international standard used for accrediting test laboratories, ISO/IEC 17025 General requirements for the
competence of testing and calibration laboratories, that testing laboratories have and apply procedures for estimating uncertainty of
measurement.
6
1
Introduction
The Government of Vietnam has introduced legislation to implement Minimum Energy
Performance Standards (MEPS) for appliances and equipment alongside a program of Energy
Labelling. The Australian Government is supporting the Vietnam Energy Efficiency Standards and
Labelling (VEESL) Program, using funds supplied by the Australian Agency for International
Development (AusAID) and the Australian Department of Industry (DoI). These funds have been
used to employ consultants with extensive experience in this area who are providing direct
assistance and advice to the Australian and Vietnamese Government officials managing VESSL.
The aim of the VEESL Program is to improve prosperity in Vietnam while increasing energy security
and reducing carbon emissions. This will be achieved through a range of activities designed to build
sustainable capacity within Vietnam to set and enforce appliance and equipment energy standards,
and monitor and evaluate the MEPS and Energy Labelling Program. Two of the main activities,
within the scope of this report, are to:

Support the monitoring of compliance through verification testing;

Support the development of local test capacity and technical and commercial skills in
laboratories through training, and through inter-laboratory (round robin) testing.
The legislation to implement MEPS and Energy Labelling for appliances and equipment became
effective during 2013. This has required new products to meet MEPS levels and, in many cases,
being required to display an energy label at the point of sale. All affected products must be
compliant with the legally imposed performance requirements.
Though the responsibility for this lies with the product supplier, the Vietnamese market
surveillance authorities also have a responsibility for checking that these products are in
compliance with these requirements. Some checks, such as whether a label is being correctly
displayed, can be made at the point of sale4 but the technical performance of the product can only
be checked through tests conducted by a suitably skilled and equipped testing organisation.
Since the results of these check tests may be used as evidence for enforcement action, it is vital
that tests are conducted accurately according to the specified test methodology, and that the
results are sufficiently repeatable and reproducible. Much of the current responsibility for
conducting compliance tests is likely to be with the state testing organisations, Quality Assurance
and Testing Centre (QUATEST) 1, 2 & 3, and a program was developed on behalf of VEESL to
support these organisations.
The VEESL program reported here was undertaken in partnership with the Vietnam Ministry of
Industry and Trade (MOIT). It consisted of the following components:
4
Survey of Energy Efficiency Labelling of Appliances in Vietnam was published by VEESL in April 2014
7

Providing supplementary training for laboratory staff undertaking the testing of electric
fans, rice cookers and compact fluorescent lamps (CFLs);

Witnessing of the testing of these products by the experts that had provided the training;

Production of testing methodological guides for electric fans and rice cookers;

Inter-laboratory testing of 20 models of electric fans, rice cookers and CFLs.
Products were purchased from the Vietnamese market for testing. The results of testing were
intended to enable the program to achieve the following:

An exploration of whether the standard test procedures, their regulated performance
levels and the testing being conducted were sufficient to deliver the performance
standards required by the Vietnamese authorities; and, if so,

A check on whether these products were compliant with their regulatory requirements and
performed as claimed on their energy labels.
8
2
Existing accreditation and approval of testing organisations
There are two official accreditation or approval programs that are applicable to testing
organisations in Vietnam. Both have been applied to the QUATEST laboratories. The first is based
on the accreditation process managed by the Bureau of Accreditation Vietnam (BoA). The second,
approval to undertake the tests required by the standards and labelling regulations, is undertaken
by MOIT.
2.1 Accreditation by the BoA
The BoA is the official body in Vietnam responsible for accrediting certification bodies under the
ISO/IEC Guide 65 and specifically for accrediting test laboratories in accordance with ISO/IEC 17025.
The BoA has international recognition as the Vietnamese member of the International
Accreditation Forum (IAF), Asia Pacific Laboratory Accreditation Cooperation (APLAC), International
Laboratory Accreditation Cooperation (ILAC) and the Pacific Accreditation Cooperation (PAC).
QUATEST laboratories 1, 2 & 3 each have a generic accreditation certificate for ISO/IEC 17025
issued by the BoA. These certificates do not state which specific TCVN standards that the
laboratories have been accredited for (accreditation under ISO/IEC 17025 is specific to identified
standards; it cannot be applied generically to the laboratory). Examination of the content of the
BoA’s website, in which detailed schedules of accredited tests are displayed for each organisation,
reveals that the detailed lists they publish are not up to date. Enquiries are on-going, but currently
it has not been possible to confirm whether each QUATEST has been specifically accredited to
ISO/IEC 17025 for testing the energy efficiency of electric fans, rice cookers and CFLs.
2.2 Approval by MOIT
An approval process of testing organisations is undertaken by MOIT. This is undertaken separately
to the ISO/IEC based accreditation process undertaken by the BoA.
The VEESL program has previously provided guidance to MOIT that was intended to assist them to
conduct audits of testing organisations. This guidance was prepared to ensure the quality systems
operated by approved testing organisations extended beyond the minimum requirements required
by ISO/IEC standards.
9
3
Provision of expert training
The QUATEST laboratories were already able to test electric fans, rice cookers and CFLs so were
seeking expert guidance to help them further refine their expertise. International experts for each
product group were commissioned to support the VEESL program. The role the experts were asked
to adopt was one of providing support and training rather than one of critique and supervision.
To do this, each expert separately spent three days in November 2013 at each QUATEST and
undertook a similar program based on the following framework:
•
Inspect the current test arrangements and compare with those required by the applicable
TCVN standard. Make note of any deficiencies and, where practicable, assist the staff to
remedy them. Where possible, assist the staff to improve these to best practice standards.
•
Check the samples to be witnessed tested by the laboratories. These should have been
selected in advance by MOIT and consist of a minimum of three different models taken
from the retail market.
•
Assist the staff to conduct tests and record results, providing guidance and training to
correct any deficient practice wherever possible.
•
Formally witness the testing (i.e. post-training) of the three models. Note any deficiencies
in the tests carried out and take a record of the results obtained.
•
Throughout the period spent at each laboratory, note any other aspects of laboratory
management that fall short of best practice and which may in some way prevent the
laboratory achieving consistent results in the testing it does on the other product
categories. Provide briefings to laboratory management where improvements could be
made.
•
At the completion of testing at the first laboratory, supervise the packing of the three
models tested5 and ensure arrangements have been made to immediately ship these
samples to the next laboratory to be visited.
•
Provide report backs to the laboratory management using a Laboratory Report Template.
This report must also be provided in a verbal briefing to senior laboratory management at
the end of each training visit.
•
Provide a training mission report back to VEESL and MOIT using a Training Mission Report
Template.
•
Use, where appropriate, materials from the laboratories’ existing testing guidance manuals,
to produce a test guidance manual for the product under review. This document is
intended to replace the individual guides currently being used and so ensure that all
laboratory staff followed the same guidance in future. (This requirement applied to electric
fans and rice cookers only.)
5
For rice cookers and electric fans; three separate batches of the same model were provided to each
laboratory for CFLs
10
3.1 Electric fans
The expert engaged to provide support and training was:
Johnny Chow
Technical Manager
Intertek-Consumer & Commercial Electrical
Hong Kong
Training was focussed on the requirements of the applicable Vietnamese national standards TCVN
7826, and TCVN 7827.
Figure 1: Testing of electric fans with support from the expert
3.1.1 Report back to VEESL and MOIT from the electric fans expert
In addition to supplying an individual verbal and written report back to the managers of each
QUATEST laboratory, Mr Chow submitted a written report of his key findings to VEESL. A summary
of the content of each of these reports is provided in Table 1. These summaries cover whether the
conditions for undertaking the test were as required by the relevant test standard, whether the
measuring equipment had been calibrated (i.e. checked for accuracy), whether the staff were
evaluating and estimating the applicable uncertainties of measurement and whether the test was
being conducted exactly as required. All of these must be undertaken correctly in order to achieve
repeatable and reproducible test measurements.
11
Table 1: Summary of findings by electric fans testing expert
QUERIES
QUATEST1
QUATEST2
QUATEST3
Were all the
No. Significant defects
No. Significant defects
No. Significant defects
conditions specified in
were found
were found
were found
No
Yes
Yes
No
No
No
Were significant
Yes. Most
Yes. Most
Yes. Most
improvements in
improvements that
improvements that
improvements that
undertaking the test
were possible to make
were possible to make
were possible to make
procedures achieved
in the time available
in the time available
in the time available
during the visit of the
were to improve the
were to improve the
were to improve the
expert?
alignment of the test
alignment of the test
alignment of the test
fan with the
fan with the
fan with the
positioning of the
positioning of the
positioning of the
measuring instrument.
measuring instrument.
measuring instrument.
the test method for
undertaking the tests
found to be
satisfactory when
expert arrived?
Had all measuring
equipment been
calibrated?
Were uncertainties of
measurement being
estimated?
What are the main

further actions
recommended for the
Test rooms must be draught-free and of the correct dimensions; all
substantial gaps in the wall fabric need to be eliminated.

laboratories to take?
The accuracy of the test result is dependent on the alignment of the
test fan with the positioning of the measuring instrument. The
current inaccurate alignment marking on the floors of the chambers
need to be removed and laser pointers used to obtain and constantly
check the necessary precision of alignment.

The wind speed meters used must meet the requirements given in
the test standard.

The laboratories must apply measurement uncertainties procedures
and so identify and, where possible, reduce sources of measurement
error. This is a requirement of the accreditation standard
ISO/IEC17025.
12
3.2 Rice cookers
The expert engaged to provide support and training was:
Michael Cheng
Technical Manager
Intertek-Consumer & Commercial Electrical
Hong Kong
Training was focussed on the requirements of the applicable Vietnamese national standard TCVN
8252.
Figure 2: Testing of rice cookers with support from the expert
3.2.1 Report back to VEESL and MOIT from the rice cookers expert
In addition to supplying an individual verbal and written report back to the managers of each
QUATEST laboratory, Mr Cheng submitted a written report of his key findings to VEESL. A summary
of the content of each of these reports is provided in Table 2. These summaries cover whether the
conditions for undertaking the test were as required by the relevant test standard, whether the
measuring equipment had been calibrated (i.e. checked for accuracy), whether the staff were
evaluating and estimating the applicable uncertainties of measurement and whether the test was
being conducted exactly as required. All of these must be undertaken correctly in order to achieve
repeatable and reproducible test measurements.
13
Table 2: Summary of findings by rice cookers testing expert
QUERIES
QUATEST1
QUATEST2
QUATEST3
Were all the
No. Significant defects
No. Significant defects
Yes
conditions specified in
were found.
were found.
No
No
Yes
No
No
No
Were significant
Yes.
Yes.
Yes.
improvements in test
Primarily to the
Primarily to the
Primarily to the
procedures achieved
detailed application of
detailed application of
detailed application of
during the visit of the
test method.
test method.
test method.
the test method for
undertaking the tests
found to be
satisfactory when
expert arrived?
Had all measuring
equipment been
calibrated?
Were uncertainties of
measurement being
estimated?
expert?
What are the main

The accuracy of the test result is dependent on temperature
further actions
measurement and the way in which stirring (of the contents of the
recommended for the
rice cooker) needs to be undertaken to achieve a uniform
laboratories to take?
temperature. The expert has developed test guidance for all the
laboratories to follow in order to achieve this in a more consistent
way.

The testing needs to take place in substantially draught-free areas.

The weighing equipment needs to be upgraded.

The laboratories must apply measurement uncertainties procedures
and so identify and, where possible, reduce sources of measurement
error. This is a requirement of the accreditation standard
ISO/IEC17025.
3.3 Compact fluorescent lamps
The expert engaged to provide support and training was:
Shuming Hua
Chief Executive
Global Efficient Lighting Centre
Beijing
14
Training was focussed on the requirements of the applicable Vietnamese national standard TCVN
7896.
Figure 3: Testing of CFLs with support of the expert
3.3.1 Report back to VEESL and MOIT from the CFLs expert
In addition to supplying an individual verbal and written report back to the managers of each
QUATEST, Mr Hua submitted a written report of his key findings to VEESL. A summary of the
content of his report is provided in
15
Table 3. These summaries cover whether the conditions for undertaking the test were as required
by the relevant test standard, whether the measuring equipment had been calibrated (i.e. checked
for accuracy), whether the staff were evaluating and estimating the applicable uncertainties of
measurement and whether the test was being conducted exactly as required. All of these must be
undertaken correctly in order to achieve repeatable and reproducible test measurements.
16
Table 3: Summary of findings by CFLs testing expert
QUERIES
QUATEST1
QUATEST2
QUATEST3
Were all the
No. Significant defects
No. Significant defects
No. Significant defects
conditions specified in
were found.
were found.
were found.
No
No
No
No
No
No
Were significant
Yes.
Yes.
Yes.
improvements in test
Primarily because the
Primarily because the
Primarily because the
procedures achieved
testers had not
testers had not
testers had not
during the visit of the
previously received
previously received
previously received
expert?
sufficient training for
sufficient training for
sufficient training for
testing lamps.
testing lamps.
testing lamps.
the test method for
undertaking the tests
found to be
satisfactory when
expert arrived?
Had all measuring
equipment been
calibrated?
Were uncertainties of
measurement being
estimated?
What are the main

Staff to receive more training for testing lamps and (in some
further actions
laboratories) to conduct testing more frequently in order to build
recommended for the
experience and maintain skills.
laboratories to take?

Test equipment needs to be managed better, it needs to be
calibrated and included within a test instrument quality system.

Some of the test equipment should be updated.

The laboratories must apply measurement uncertainties procedures
and so identify and, where possible, reduce sources of measurement
error. This is a requirement of the accreditation standard
ISO/IEC17025.
17
4
Inter-laboratory comparison tests
The three QUATEST laboratories undertook inter-laboratory comparisons. Each tested the same
samples of 20 different models of electric fans and rice cookers. Testing of CFLs was undertaken on
30 samples of 20 different models requiring a total of 600 CFLs to be obtained. For each of the 20
models, the 30 samples were subdivided into three batches of 10 samples; one batch went to each
QUATEST laboratory.
The main purposes of these trials were to provide:

A measure of the extent of the variation and uncertainty of results;

An understanding of the cause of these variations in order to…

Be able to advise MOIT and the laboratories on steps that could be taken to minimise these
variations and so achieve repeatable and reproducible results.
The first, of two, phases of the inter-laboratory comparison tests was initiated when the expert
trainers were present at the QUATEST laboratories. Part of their role was to witness the initial tests
being undertaken. These tests were performed after the experts had provided some
training/guidance and had assisted the testing staff to achieve the best possible set-up using the
test equipment available at the time.
The second phase of the inter-laboratory comparison testing took place after the experts had
departed. The laboratory staff then tested all the products using their existing knowledge
supplemented by the expert training.
4.1 Results of witness testing
Each expert was required to formally observe (“witness”) the testing of a selection of products that
they had trained the laboratory staff to test. This had two purposes:

To enable the experts to review how well the tests were being conducted and so note
whether the advice and training they had been given previously had improved the ways in
which the tests were performed; and

To establish the organisation of the inter-laboratory comparison tests that would continue
to take place after their departure.
Each expert submitted detailed report test sheets to VEESL of the test results he witnessed being
measured. An example of one of the result sheets for each of the three product types is given in
Appendices 1-3.
The content of the experts’ reports, which mainly deal with sources of measurement uncertainty
i.e. the reasons for variations in results obtained from reproduced tests of the same product, are
discussed later in this report. The measurements obtained in the tests that were witnessed have
not been reported here since they have been superseded by those that were subsequently
repeated by the three QUATEST laboratories following them making further improvements to their
testing procedures.
18
4.2 Results of testing electric fans
Twenty different models of electric fan were tested, after the expert training had been provided.
The brand and model identifications are listed on Table 4.
Table 4: Brand/model list of electric fans tested
Reference number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Fan name
ĐIỆN CƠ TICO: B300
CHING HAI: BF168
LIOA: QB-409K
CHING HAI: HS802
SHARP: PJT1621V
ĐIỆN CƠ TICO: B400
VINAWIND: QĐ400-MS
LIOA: QT-409KLG
VINA ASIA: L16003
ĐIỆN CƠ HD: HĐ-450 (treo)
SANAKY: SK-16C
ĐIỆN CƠ 91: QB-450 (dung)
VINAWIND: QĐ450-ĐM
DAIICHI: DC-SF525
DAIICHI: DC-SF535
HOTOFAN: QĐC400
DAIICHI: DC-SF650
LIOA, MODEL: Q-409K
ĐIỆN CƠ TICO: B300
ĐIỆN CƠ 91: QĐ-CN450P
They were a mixture of table top and pedestal fans; there was one wall fan. Blade diameters
ranged from 300-450mm; input powers were in the range 35-65W (example shown in Figure 4).
19
Figure 4: Electric fan manufacturer label and the energy label
Each sample was measured in accordance with TCVN 7827:2007, the official test method for
measuring the airflow performance of these types of product.
The results obtained were then used in combination with TCVN 7826:2007, which provides the
levels of energy efficiency of the fan (k) and the corresponding energy label star rating. Figure 5
gives the reported level of energy efficiency or star rating (based on R6) of the 20 fans by the three
laboratories.
6
TCVN 7826 and TCVN 7827 use “k” and “R” to mean the same thing (ratio of measured efficiency to minimum requirement), which
some users may find confusing.
20
Figure 5: Star rating of the 20 fans as measured by the three laboratories
There is clearly a difference between the star ratings measured by the laboratories for some of the
fans. This may be more clearly seen if the measured star range for each fan is only plotted as a line.
Figure 6: Maximum-to-minimum star rating of the 20 fans as measured by the three laboratories
21
4.2.1 Discussion of results of testing electric fans
The variation in results for the star rating for the label, reported in Figures 5 & 6 is very large for
some of the models tested. Since the tests were conducted on the same samples, using the same
test procedure and the same guidance manual was available to follow, the following may have
contributed to the variation:
1. Large measurement uncertainties being caused by the test method not being reproduced
in exactly the same way by each of the three QUATESTs;
2. Large measurement uncertainties being caused by the test method because it is not
possible to repeat it in a consistent way;
3. The threshold values for the energy label levels, the star values, are too compressed with
the consequence that small differences in performance measurements are increased
through being multiplied in the labelling calculation, which results in having the effect of
exaggerating the real difference in energy efficiency.
Examining each of these in turn:
Large measurement uncertainties being caused by the test method not being reproduced in exactly
the same way by each of the three QUATESTs
The training expert identified this as the most significant factor at the time when the very first tests
were being conducted. The following causes of measurement uncertainties were reported:

The wind speed measuring instrument was not being accurately aligned with the wind flow
from the fan under test;

None of the test chambers at the three QUATESTs were fully correct as they did not
completely eliminate draughts from external sources and, in one case, the chamber was of
incorrect dimensions;

The wind speed test instruments used by two of the QUATESTs did not meet the
specification given in TCVN 7827;

Some equipment had not been calibrated;

Some wind flow readings had to be taken from the wind speed measuring instrument
manually which meant that test personnel had to be in the vicinity of the wind flow being
measured, a factor that could interfere with the wind flow being measured.
All of these factors, which were possibly contributing to increasing the level of measurement
uncertainties, were fully identified to the QUATEST staff prior to them conducting the performance
tests that are reported in Figure 5 & 6.
At the time of writing this report, it was not known to what extent remedial actions had been
undertaken by any of the three QUATESTs in the period following the departure of the training
expert and prior to them conducting the main set of tests.
22
Large measurement uncertainties being caused by the test method because it is not possible to
repeat it in a consistent way
The opinion of the training expert was that this was not a significant factor. However, the wind flow
from the sample under test has to be meticulously measured under precisely maintained
conditions. The laboratory testing staff must understand that this test procedure has to be
undertaken using methods requiring a high level of precision. The only enhancement
recommended for the basic test methodology was for a guidance note to be added to TCVN 7827
to draw the attention of testing staff to the importance of accurately aligning the wind flow
measuring instrument with the fan being tested and the importance of ensuring nobody is standing
in the vicinity of the air flow when it is being measured. Given the sensitivity of the measurement
process, an additional precaution would be to require the test to be conducted under repeatability
conditions to ensure that the results being obtained are consistent.
The banding scales for levels of energy efficiency are too compressed with the consequence that
small differences in performance measurements are accounting for large (exaggerated) differences
in the levels of energy efficiency
The average difference in the k values between the minimum and maximum values for each fan is
0.4 (Figure 7). Since the current labelling scale has increments of 0.1 i.e. a 10% increase in efficiency
at each level, the variation in test results from the three laboratories is, on average, spanning four
levels of efficiency. It is clear from this variation, that the results of inter-laboratory tests were not
sufficiently reproducible to support the current energy label.
Figure 7: Reported minimum-to-maximum efficiency of the 20 fans, by three laboratories
23
Furthermore, most of fans comfortably exceed the best performance requirements required in the
energy labelling regulations as can be seen from the test results of the samples displayed in Figure
7. An examination of the MOIT registration database for electric fans confirms that almost all the
fans listed have 5 stars.
It is clear that the thresholds need to be recast if they are to provide any meaningful signals to the
market, and be useful to policy makers. The re-grading of the energy label should be done using
other available evidence, e.g. examination of the latest registration database, examination of other
countries regulations on electric fans, and finding best-practice products. Ideally, this re-grading
process would be done after the laboratories have succeeded in improving the level of
reproducibility and so further reduce the level of test result differences between themselves.
4.3 Results of testing rice cookers
Twenty different models of rice cookers were tested. All were table top designs with capacities
ranging from 1 – 3 litres; input powers ranging from 350-1000W (an example is shown in Figure 8).
Figure 8: Rice cooker and its energy label
The brand and model identifications of the twenty models tested are listed in Table 5.
24
Table 5: Brand/model list of rice cookers tested
Reference number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Rice cooker name
SHARP KSH-211
HIỆP HƯNG: HH105
SUPOR: CFXB50YB5B-70
SUPO VIET: SP-30R
GOLDSUN: ARC-G18CP1
ELECPOR: SR-P1C
HONEY’S: H0901-M22D
SUPO VIET: SP-22R
GOLDSUN: ARC-GDF18C2
DAEWOO: DFC-5018
COMET: CM8018
SHARP: KSH-219V
AIDI: MR-SM07J
SUPO VIET: RC-18SB
DAELUX: DXR50J-G
CARIBBEAN: CRC-182
SHARP: KSH-222
KANGAROO: KG376
SUPO VIET: SP-12R
MIDEA: MR-CM18SD
Each sample was measured in accordance with TCVN 8252:2009, the official test method for
measuring the energy efficiency performance (K) of these types of product.
Figure 9 gives the reported energy efficiency grade or star rating (based on the Energy Efficiency
Index, K) of the 20 rice cookers by the three laboratories.
25
Figure 9: Star rating of the 20 rice cookers as measured by the three laboratories
There is clearly a difference between the laboratories for some of the rice cookers. This may be
more clearly seen if the measured star range for each rice cooker is only plotted as a line (Figure
10).
Figure 10: Maximum-to-minimum star rating of the 20 rice cookers as measured by the three
laboratories
26
The large variation in the star rating is mainly due to the large variation in the measured cooking
efficiency.
NOTE: the efficiency reported by QUATEST2 as exceeding 100% is assumed to be an error.
Figure 11: Reported average cooking efficiency (%) of the 20 models of rice cookers
4.3.1 Discussion of results of testing rice cookers
The variation in results for the star rating for the label, reported in Figure 9 and Figure 10, is very
large for some of the models tested. Since the tests were conducted on the same samples using the
same test procedure and the same guidance manual was available to follow, the causes for the
variation are expected to be one or some of the following:
1. Large measurement uncertainties being caused by the test method not being reproduced
in exactly the same way by each of the three QUATESTs;
2. Large measurement uncertainties being caused by the test method because it is not
possible to repeat it in a consistent way;
3. The threshold values for the energy label levels, the star values, are too compressed with
the consequence that small differences in performance measurements are increased
through being multiplied in the labelling calculation, which results in having the effect of
exaggerating the real difference in energy efficiency.
Examining each of these in turn:
27
Large measurement uncertainties being caused by the test method not being reproduced in exactly
the same way by each of the three QUATESTs;
The training expert identified this as one of the two most significant factors at the time when the
very first tests were being conducted. The main cause of the measurement uncertainties being that
the accuracy of the test result is dependent on temperature measurement. The stirring of the
contents of the rice cooker in a thorough and consistent way in order to achieve a uniform
temperature distribution is crucial to this and, perhaps not surprisingly, different laboratory staff
undertake this manual activity in slightly different ways.
Large measurement uncertainties being caused by the test method because it is not possible to
repeat it in a consistent way
The training expert confirmed that this was the other significant factor. For example, the
measurement uncertainty of 0.5oC permitted by the existing standard already accounts for one
label grade. Furthermore, the standard needs to be changed to make it less sensitive to
temperature measurement. This could be achieved by using a longer heating time in order to make
the temperature rise larger. This would ensure that the label grade calculation becomes less
sensitive to the temperatures being measured. Alternatively, the test method could be changed to
measuring the energy required to raise the water temperature in the rice cooker for a larger fixed
value, say 50K.
The banding scales for the energy efficiency grades are too compressed with the consequence that
small differences in performance measurements are accounting for large (exaggerated) differences
in the final energy efficiency grade
The current grade performance thresholds given in TCVN 8252 appear very compressed, see Table
6 below. The impact of this is significant since a change in measured temperature as small as 0.5oC
results in a change of label grade.
Table 6: Energy efficiency bands given in TCVN 8252:2009
Grade
Energy Efficiency Index
K
1
≥ 1,00
2
≥ 1,05
3
≥ 1,10
4
≥ 1,15
5
≥ 1,20
However, examination of the MOIT registration database for rice cookers shows a wide spread of
ratings from 1 through to 5 stars i.e. most rice cookers do not exceed the best performance
requirements required in the energy labelling regulations.
28
Consequently, from a marketplace perspective, there is no requirement yet to recast the
thresholds and this leads to the conclusion that expanding the banding scales would not provide a
practicable solution for dealing with the spread of performance results reported by the three
QUATEST laboratories.
4.4 Results of testing CFLs
Twenty different models of CFLs were tested, with each of the three laboratories having 10 of each
model to test from the same purchased batch of 30 samples. The lamps were not transferred
between three QUATEST laboratories. Brand and model identifications are listed on Table 7. Rated
powers were in the range 5-23W. Some models claimed to have high efficiency performance and so
were qualified to display the endorsement energy label.
Table 7: Brand/model/size list of CFLs tested
Reference number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
CFL name
Rạng Đông 5W
Rạng Đông 11W
Rạng Đông 15W
Rạng Đông 20W
Rạng Đông 40W
PHILIPS, GENIE 5W
PHILIPS, GENIE 11W
PHILIPS, GENIE 14W
PHILIPS, Essential 18W
PHILIPS, Essential 23W
Điện quang 5W
Điện quang 8W
Điện quang 14W
Điện quang 18W
Điện quang 26W
OSRAM DULUXSTAR COMPACT 8W
OSRAM DULUXSTAR COMPACT 11W
OSRAM DULUXSTAR COMPACT 14W
OSRAM DULUXSTAR COMPACT 20W
OSRAM DULUXSTAR COMPACT 23W
An example of one of the CFLs tested with its energy label marking is shown in Figure 12.
29
Figure 12: Example of CFL label
Each sample was measured in accordance with TCVN7896:2008, the official test method for
measuring the energy efficiency performance of these types of product.
The initial luminous flux (lm) and the power (W) were measured, and the ratio reported (lm/W).
The results for this (as the average of the 10 samples tested for each model) from each of the three
QUATEST laboratories are shown in Figure 13.
30
Figure 13: Average efficacy (lm/W) measured for the 20 models of CFLs
4.4.1 Discussion of results of testing CFLs
These results from the three QUATEST laboratories are, when compared to those obtained for
electric fans and rice cookers much more consistent. All models appeared to be compliant in
respect of efficacy (lm/W), which was the only compliance parameter monitored in this testing
program.
The results reported by QUATEST3 showed consistently lower values, a few per cent lower than the
other two laboratories on average – see Table 8, below.
Table 8: Difference between the average efficacies of the tested CFL lamps
Declared*
Q1
Q2
Q3
(lm/W)
56.24
56.97
56.87
54.50
Q3=100%
103%
105%
104%
100%
Average efficacy
* average of 17 lamps
The cause of this variation could be due to variations in the measurement of input power or
variations in measuring the light output. A further examination of the input power recorded by
each laboratory for each was made to determine whether this was the factor responsible.
The average power for each model by each laboratory is given in Figure 14.
31
Figure 14: Average measured power (W) of the 20 CFL models
These data show that the measured input powers are very close to each so are not causing the
variation in reported efficacy. The variation is therefore coming from the measurement of light
output (Figure 15). An examination of the remarks made by the expert trainer suggests that the
failure to calibrate the standard (calibration) lamps used by each of the three QUATEST laboratories
to the same national (reference) standard lamp is most likely to be the cause of the consistent
variation in initial luminous flux measurements.
The following figures shows the reported initial luminous flux, which clearly shows that QUATEST 3
is reporting lower figures than the other two laboratories, especially for the larger CFLs.
32
Figure 15: Measured/reported initial luminous flux (lm) of the 20 CFL models
NOTE: In the figure above, the display of most of the results from QUATEST 1 are hidden by the nearly identical results
from QUATEST 2.
33
5
Conclusions and recommendations
Vietnam has a number of testing laboratories that have the ability and potential to undertake the
testing required to support the appliance MEPS and energy labelling testing regime. The state
owned QUATEST laboratories, for which this program from VEESL provided support, were already
able to test electric fans, rice cookers and CFLs so were seeking expert guidance to help them
further refine their expertise.
The results obtained by the inter-laboratory testing of electric fans and rice cookers will need
careful examination by MOIT in order to identify whether any can be used for enforcement
purposes. Many, though, appear too variable to be used for this purpose.
The results for CFLs are much more consistent and could have been used for enforcement purposes
if any had performed below the minimum permitted performance level. None did.
The main finding of this report is that the variability in test results for two of the three product
sectors (rice cookers and electric fans) is very large for some models. Examination of the test
methodologies used, the procedures followed and the data recorded by the test laboratories has
identified that the following are likely to be the main reasons for the variability:

Not all measurements are being reproduced in a consistent way by each of the three
laboratories;

The specification and calibration of measuring instrumentation needs to be improved in
some cases;

The published test method for rice cookers needs to be amended to enable it to be
consistently reproduced by different test laboratories;

Thresholds for energy label bands for electric fans are too narrowly defined.
These reasons are covered in the discussions below. Actions have been recommended that should
enable the results of laboratory testing to become sufficiently accurate to provide confidence to
the Government, consumers and manufacturers.
5.1 Product testing related advice for each of the QUATEST laboratories
The deployment of testing experts to provide training and guidance to testing staff at the QUATEST
laboratories has been beneficial since they have identified where further refinements can be made.
The results of the inter-laboratory testing show that, if not done already, the following needs to be
done at each laboratory:

Upgrade and calibrate testing equipment as already recommended by the experts directly
to each of the QUATEST laboratories;
34

Implement a program to apply measurement uncertainties procedures and so identify and,
where possible, reduce sources of measurement error;

Implement the use of the guidance manuals for testing electric fans and rice cookers.
Note: the use of these and the routine estimation of uncertainties of measurement should
support the continuing development of a performance testing culture at the QUATEST
laboratories – one in which performance testing is always undertaken in a meticulous way;
5.2 Recommendations applicable to all approved testing organisations in Vietnam
Poor reproducibility of test results between testing organisations is not uncommon. It has
previously been found in similar programs conducted in other countries. As a result of these, a
number of processes have been developed to improve the repeatability and reproducibility of
testing at different locations, and these are recommended by the authors of this report:

Use of common testing manuals (or shared interpretation sheets) for all products impacted
by the standards and labelling regulations;

Undertaking of regular inter-laboratory trials to check that the test results being obtained
are reproduced consistently and, where not, to have the test laboratories to work
cooperatively together to establish and eliminate the causes of poor reproducibility;

Feed the experiences of using the test methodology standards by the laboratories back to
the standards development body (VSQI, for Vietnam) in order to support the continuing
development (to improve the repeatability and, hence, the reproducibility) of those test
methods.
5.3 The accreditation operated by the Vietnamese Board of Accreditation (BoA)
The reports from each of the experts suggest that the QUATEST laboratories do not fully match the
minimum standards that are required of laboratories that have been accredited to the
requirements of ISO/IEC 17025. For example, uncalibrated or incorrect measuring equipment was
sometimes used and none of the laboratories were routinely estimating uncertainties of
measurement.
This is a matter that the BoA can assist the QUATEST laboratories with. The BoA can advise the
QUATEST laboratories of the detailed requirements in the ISO/IEC17025 standard and can make a
detailed examination of these at their next accreditation visits to the laboratories.
This is particularly important as the accreditation process is there to provide the assurance to all
stakeholders that testing is being carried out properly, and with sufficient accuracy.
5.4 Support for the MOIT approval process
Currently, MOIT manage a separate approval process to the accreditation process operated by the
BoA. Such approval processes are not unusual as many regimes recognise that the formal
accreditation process has some limitations and so wish to impose further conditions.
35
MOIT may wish to review the procedures it follows in its approval process once it is satisfied that
repeat accreditation visits by the BoA have resulted in further improvements being implemented
by the QUATEST laboratories.
5.5 Recasting threshold for energy label bands
The evaluation of results reported in section 4.2.1 suggests that the thresholds for the energy
label bands for electric fans would benefit from being re-cast. The evaluation identified that the
market is already saturated with 5 star products and most of the fans tested easily exceeded the
energy efficiency requirements for 5 stars.
NOTE: further work relating to this topic of recasting thresholds is covered by another part of the
VEESL Program.
36
APPENDIX 1 Example of test results sheet for one model of electric fan
Table 9: Example of test results sheet for one model of electric fan
Measurement Radius
Power
point
consumption
(at the
highest
Setting with
Osc)
(I/D)
( r)
(P)
Wind speed
(at the
highest
Setting)
Average
of Wind
speed
Left
V = (Vl +
Vr )/2
Right
Check
point
Cross sec. of
donuts
Wind flow
blowing
through the
donuts
S=
Qn = V.S =
2*(PI)*r*b*10- (m2)*(m/min)
6
1
2
3
4
5
6
7
8
9
10
11
(mm)
(W)
20
60
100
140
180
220
260
300
340
380
420
47.80
47.80
47.80
47.80
47.80
47.80
47.80
47.80
47.80
47.80
47.80
(m/min)
162.6
197.4
205.2
184.8
156.0
142.8
111.0
99.6
80.4
60.6
40.2
202.2
208.2
212.4
167.4
153.0
108.6
99.6
56.4
26.4
0.0
0.0
(m/min)
182.40
202.80
208.80
176.10
154.50
125.70
105.30
78.00
53.40
30.30
20.10
Go
Go
Go
Go
Go
Go
Go
Go
Go
Go
Stop
(m2)
(m3/min)
0.005
0.015
0.025
0.035
0.045
0.055
0.065
0.075
0.085
0.096
0.106
0.917
3.058
5.248
6.196
6.989
6.950
6.881
5.881
4.563
2.894
2.122
Total
wind
flow
blowing
through
the
donuts
Energy
Efficiency
Coefficient
Level of
Energy
Efficiency
of Fans
(stable,
stand,
wall)
Q = sum
(Q1
…Qn)
EER do =
Q/P
R = EER do
/ EER min
Level
(m3/min) (m3/min.W)
0.917
3.975
9.223
15.419
22.408
29.359
36.239
42.120
46.684
49.577
51.699
0.019
0.083
0.193
0.323
0.469
0.614
0.758
0.881
0.977
1.037
1.082
0.02
0.10
0.24
0.40
0.59
0.77
0.95
1.10
1.22
1.30
1.35
0
0
0
0
0
0
0
2
3
3
4
APPENDIX 2 Example of test results sheet for one model of rice cooker
Table 10: Example of test results sheet for one model of rice cooker
Witness test results for Energy Efficiency test of Rice cookers TCVN8252:2009
Sample 4
Product description:
Rice Cooker Model #4
Model No.:
Ratings
220V 50Hz
Actual Power
Cooker weight:
Aluminium, W g1 (g)
304 Staninless steel, W g2 (g)
CP1 (cal/g K)
0.22 CP2 (cal/g K)
700W
Test staff:
666W
0
0.12
Formal witness test
Ambient temperature 20.6 ~ 21.8°C
Relative humidity
52% ~ 72%
Test date:
22 Nov & 25 Nov-2013
Test voltage
219.8V
Frequency
50Hz
Cooker mode:
Cook
Test
T1 (ºC)
T2 (ºC)
ΔT (ºC)
Water volume (g)
Energy consumption, Ec (Wh)
Initial test
19.2
31.0
11.8
2648
44.9
Trail 1
19.2
30.8
11.6
2648
44.9
Trail 2
19.4
31.2
11.8
2648
44.9
Trail 3
19.4
31.2
11.8
2648
44.9
Average of trails 1,2,3
19.3
31.1
11.7
2648.0
44.9
Rd%
Energy Efficiency Index, K
{ΔT x [Q + (W g1 x CP1) + (W g2 x CP2)] x 100} / (0,24 x Ec x 3600)
Rd/Rmin
1.03
82.11910478
label grade 1
38
APPENDIX 3 Example of test results sheet for one model of CFL
Table 11: Example of test results sheet for one model of CFL
Witness test results for Energy Efficiency test of CFL TCVN 7896
Lamp model Information
Identifier
VEESL-DR11-22
VEESL-DR11-25
VEESL-DR11-26
VEESL-DR11-28
VEESL-DR11-29
Average
non-uniformity factor
Brand A, 11W
Voltage (V)
Current (A)
220.0
220.0
220.0
220.0
220.0
220.0
/
0.074
0.074
0.076
0.075
0.074
0.075
2.68%
Power
(W)
9.6
9.6
9.9
9.8
9.7
9.7
3.09%
PF
0.593
0.591
0.591
0.592
0.592
0.592
0.002
Flux (lm)
Efficacy
(lm/W)
529.0
531.7
547.0
552.4
532.2
538.5
4.35%
55.10
55.39
55.25
56.37
54.87
55.40
2.71%
Colour coordinate
x
y
0.3164
0.3469
0.3160
0.3443
0.3173
0.3476
0.3155
0.3450
0.3170
0.3473
0.3164
0.3462
0.0018
0.0033
CCT
6199
6231
6157
6252
6170
6202
95
CRI
81.9
81.5
82.0
81.4
82.0
81.8
0.6
39