A PPENDIX 8
TESTS
1
Testing of the system
Testing of the system shall consist of an acceptance test and a service level test in accordance
with clause 12 of the contract and the present Appendix.
Testing of options to be delivered after the acceptance date shall be conducted as described in
Appendix 13.
2
Common rules for testing
2.1
Conduct of test
The date for conduct of the test appears from the time schedule, cf. Appendix 1, or any other
agreement made between the parties.
The test shall be conducted under conditions that, to the widest extent possible, correspond to a
normal operational situation.
2.2
Report on test conduct
When the acceptance test has been completed, the supplier shall immediately prepare a report on
the test conduct, specifying any observed defects in a list of defects. The customer shall prepare a
similar report when conducting the service level test.
2.3
Approval of test
A test has been passed when the requirements for the test result have been fulfilled, and an
approval shall then be made in accordance with clause 12, 2nd paragraph, of the contract.
If the test cannot be approved by the customer, the customer shall without undue delay notify the
supplier thereof in writing, stating the reason for non-approval.
2.4
Remedying the observed defects
In case the customer approves the acceptance test with observed defects, these shall be specified
in a list of defects. The customer shall only be under an obligation to approve the service level test
when such defects have been remedied in all material respects, cf. clause 12.1 of the contract.
Defects observed after the acceptance date, including defects observed in connection with the
service level test, shall be remedied in accordance with Appendix 7.
3
Acceptance test
3.1
Purpose
The purpose of the acceptance test is primarily to determine whether the agreed functionality and
documentation have been delivered. However, at the acceptance test it may also be determined
whether service levels have been fulfilled and associated services have been delivered in
accordance with the details agreed.
3.2
Test schedule
As part of its tender, the supplier has prepared a draft test schedule for the acceptance test,
included as a sub-appendix to the present Appendix 8.
In the supplier's draft test schedule, the character, extent and time of the customer's participation
have been specified, including what data the customer shall make available in order to duly
perform the test.
Following completion of the clarification phase, the supplier shall in the test schedule incorporate
all changes proposed by the customer, unless the supplier is able to substantiate that such
proposals will imply activities beyond what may reasonably be expected.
The customer's proposals for changes in the test schedule shall reach the supplie r not later than
[..] working days before the test starts, after which the supplier shall prepare a revised test
schedule. The revised test schedule shall have been received by the customer not later than [..]
working days after the date on which the supplier receives the customer's proposal for changes.
3.3
Arrangement
The supplier shall arrange the acceptance test as a functional test to check if the agreed
functionality, including integration to other systems and documentation, has been delivered. The
supplier's draft test schedule must include testing of essential functions and the documentation.
Other functions and conditions shall be tested in accordance with the test schedule.
If a separate indication is desired as to whether the system fulfils the ser vice levels specified in
Appendix 10 when the acceptance test is conducted, the customer shall notify the supplier thereof
not later than the date on which the clarification phase is completed.
3.4
Conduct of test
The acceptance test shall be conducted by the supplier with the customer's active participation, cf.
clause 12.1, 1st paragraph, of the contract. The customer's participation is described in further
detail in the supplier's draft test schedule, cf. clause 3.2 above.
In connection with the conduct of the acceptance test, a number of fault situations shall be
provoked, to which the system must respond with fault messages that make it possible for the user
to continue on a well-defined basis.
3.5
Approval criteria
The requirements for acceptance test results are fulfilled if the test is conducted without observing
one or more qualified defects.
A qualified defect is present if:

the majority of users cannot use the system or if the functionality is reduced to such a
degree that the system must be regarded as out of operation, or

the usability of the system is reduced or impeded to a significant extent, or

the usability of the system is reduced to a not inconsiderable extent, unless the defect only
affects a few of the relevant users or the defect can be bypassed without a significant extra
effort on the part of the customer.

any one of the service levels indicated in Appendix 10 has deteriorated by more than […]%,
provided that it has been agreed that the service level should be tested during the
acceptance test.

significant defects are observed in associated services.

[…]
It shall not be considered as a qualified defect if:

only one specific function is not working, unless the function in question is of key
significance to the customer's use of the system, or

only […] users out of a larger group of users are affected by the defect, and the other users
in the group can still use the system, or

the failure to work is concerned with only one function, unless the function in question is of
key significance to the customer's use of the system.

[…]
In case the number and content of non-qualified defects are of such an extent that the customer's
use of the system is affected thereby as if a qualified defect had been in evidence, these shall be
regarded collectively as a qualified defect. The number of non-qualified defects may in no
circumstances exceed […].
3.5.1
Defects in third-party software
Notwithstanding clause 3.5, the customer shall not be entitled to refuse approval of the acceptance
test because of defects in third-party software in case there are less than […] non-significant
defects in third-party software for which the supplier has indicated reasonable bypassing, thus
making it possible for the customer to use the system without a significan t extra effort.
Any indicated bypassing shall not deprive the customer of the right to have the defect remedied
where this is remedied by a third party.
4
Service level test
4.1
Purpose
The purpose of the service level test is to determine whether the deliver y complies with the
specified service levels, cf. Appendix 10, cf. clause 12.2, 1st paragraph, of the contract.
4.2
Test schedule
The supplier shall prepare a test schedule describing the technical and resource -dependent
conditions on which the conduct of the test is based. The test schedule is to ensure that the
service level test gives a correct representation as to whether the specified service levels have
been met.
4.3
Conduct of test
Before the service level test is commenced, the acceptance test shall hav e been approved by the
customer. Furthermore, the service level test shall only be commenced when the customer has
notified the supplier thereof, but not later than 20 working days after the acceptance date, cf.
clause 12.2, 2nd paragraph, of the contract. In accordance with clause 12.2, 1st paragraph, of the
contract, the service level test shall be conducted by the customer assisted by the supplier, and
shall be carried out for the system as a whole.
During the service level test the supplier shall be e ntitled and liable to optimize performance of the
system on a current basis to the extent that this might be necessary, and to remedy any defects
observed. Furthermore, the supplier shall assist the customer in answering the customer's
questions regarding use of the system, including provision of hotline service etc.
The service level test shall comprise at least […] working days in succession, during which the
system has been in service with normal functions. The time limit for completing the service leve l
test appears from the time schedule, cf. Appendix 1.
4.4
Approval criteria
The requirements for the result of the service level test have been fulfilled when the service levels
for the last […] working days of the service level test period have been fulfill ed for the system as a
whole.
If the customer has approved the acceptance test with observed defects as specified in the list of
defects, cf. clause 2.4 above, then the customer shall only be under an obligation to approve the
service level test when such defects have been remedied in all essentials, cf. clause 12.1, 3rd
paragraph, of the contract. This shall mean that the only remaining defects fall within categories D
and E, cf. Appendix 7. However, the customer shall not be under an obligation to approv e the
service level test in case there are more than […] such defects. The remaining defects shall be
remedied in accordance with Appendix 7.
4.5
Measurement of operating efficiency
See Appendix 10 for measurement of operating efficiency.
4.6
Measurement of response times
See Appendix 10 for measurement of response times.