Common Assessment Project (CAP)
Software Testing
THE FOLLOWING DOCUMENT IS INTENDED TO SUPPORT ORGANIZATIONS
PARTICIPATING IN THE COMMON ASSESSMENT IMPLEMENTATION. THE DECISION TO
USE THIS DOCUMENT IS AT THE SOLE DISCRETION OF THE PARTICIPATING
ORGANIZATION. ANY DECISIONS MADE BY ORGANIZATIONS SHOULD BE CONSIDERED IN
THE CONTEXT OF ORGANIZATION PROCUREMENT POLICIES AND SUPPORTING
PROVINCIAL GUIDELINES AND LAWS, WHERE APPLICABLE. ITEMS CONTAINED IN THIS
DOCUMENT SHOULD BE DISCUSSED WITH THE PARTICIPATING ORGANIZATION’S LEGAL
COUNSEL WHERE APPROPRIATE. THE TRANSFER PAYMENT AGENCY REPRESENTING THE
PROJECT TEAM HOLDS NO RESPONSIBILITY OR LEGAL LIABILITY FOR THE ITEMS AND
WORDING CONTAINED WITHIN IT IN WHOLE OR IN PART.
Local Health Integrated Networks (LHINS) and Health Service Providers (HSPs) may wish
to include some of the activities identified in this document as deliverables, with
assigned deliverable dates, in either a Vendor Statement of Work (SOW) or a Vendor
Agreement. If a LHIN has decided on a vendor strategy, depending on that strategy and
the software model, the nature of these activities and those responsible for completing
the activities may be different, but in general the activities are the same.
Vendors and LHINs/HSPs are responsible for ensuring that solutions are sufficiently
tested prior to going live and entering assessments into the system. There are different
types of testing. In general, and at a minimum there are three types of testing that must
occur with any software implementation. The three types of testing are:
Unit Testing – Unit testing involves the testing of individual software components or
modules. This testing is typically done by the vendor’s programmers/developers. It
requires detailed knowledge of the internal program design and code. The vendor is
responsible for conducting unit testing. This is done without the involvement of, or
notice to, the LHIN/HSP. After all unit testing is complete, system testing can
commence.
System Testing – System testing involves testing compliance of complete, integrated
system with software requirements specifications. This testing involves the testing of
functionality and does not require knowledge of the internal program design or
code. The vendor is responsible for conducting system testing. This is done without
the involvement of the LHIN/HSP. The vendor may notify the LHIN/HSP when
system testing is complete as it is a prerequisite to commencing with user
acceptance testing.
ccim.on.ca
Page 1 of 3
User Acceptance Testing (UAT) - This type of testing is typically done to verify a
system meets customer requirements. The ‘user’ or customer (i.e., LHIN/HSP) does
the testing to determine whether to accept the software solution. The LHIN/HSP is
responsible for ensuring the solution meets requirements. While the LHIN/HSP
performs UAT, materials and direction are typically provided by the vendor as part of
quality assurance activities. At a minimum, two (2) cycles of UAT should be planned.
This is to accommodate the fixing of defects found after the first cycle of testing.
Quality assurance activities may be led by the vendor or the LHIN/HSP. The
LHIN/HSP may want to coordinate and lead quality assurance activities in lieu of the
vendor. With UAT successfully complete, the vendor and LHIN/HSP are ready to
install the software solution into the ‘live’ or ‘production’ environment.
Testing is conducted in a test environment (i.e., not the production or live environment
where users are actively using existing software applications). This is particularly
important where there are systems in place that are being actively used and these
systems will be replaced, or upgraded, with the new solution. Testing activities must be
conducted outside of live environments to avoid any risk to the live system or data.
To support testing activities, there should be a test plan and a schedule. The test plan
includes, but is not limited to: the types of testing that will happen, entry and exit
criteria for moving to the next type of testing, who is involved in the testing, and the
criteria for testing completion before going live with the solution.
Testing includes creating and running test scripts and test scenarios. These scenarios
and scripts test system functionality. The results of the test scripts and scenarios are
compared against expected results to indicate whether the test was successful.
When the predetermined success rate for testing is achieved, testing is complete.
The roles and responsibilities of test participants should be defined for testing activities
and should also list or identify the criteria for test type completion. Some activities to
consider in the development of a test plan and schedule are:
1. Identify testing process for UAT
Establish end user support procedures. This includes creating an error
tracking/ issues log, identifying how bugs will be reported and managed, and
establishing a timeline for the entire testing process.
2. Complete testing documentation
This includes test scenarios and checklists
3. Identify when unit testing will be complete
4. Identify when system testing will be complete
ccim.on.ca
Page 2 of 3
5. Identify when UAT will commence
This date will be the earliest date when LHIN/HSP can begin testing
6. Identify when patches will be installed to address bugs identified through UAT
7. Identify when UAT will be complete
After testing activities are successfully completed, vendors and LHINs/HSPs can proceed
with validating the implementation. Implementation validation must be performed by
both the vendor and the LHIN/HSP. The vendor validates first, followed by the
LHIN/HSP. The validation is performed against the test environment which must be the
exact same solution (i.e., software code) as the ‘live’ environment.
Implementation validation is addressed in the ‘Implementation Validation Guide’ on the
CCIM website.
ccim.on.ca
Page 3 of 3