Common Assessment Project (CAP) Software Testing THE FOLLOWING DOCUMENT IS INTENDED TO SUPPORT ORGANIZATIONS PARTICIPATING IN THE COMMON ASSESSMENT IMPLEMENTATION. THE DECISION TO USE THIS DOCUMENT IS AT THE SOLE DISCRETION OF THE PARTICIPATING ORGANIZATION. ANY DECISIONS MADE BY ORGANIZATIONS SHOULD BE CONSIDERED IN THE CONTEXT OF ORGANIZATION PROCUREMENT POLICIES AND SUPPORTING PROVINCIAL GUIDELINES AND LAWS, WHERE APPLICABLE. ITEMS CONTAINED IN THIS DOCUMENT SHOULD BE DISCUSSED WITH THE PARTICIPATING ORGANIZATION’S LEGAL COUNSEL WHERE APPROPRIATE. THE TRANSFER PAYMENT AGENCY REPRESENTING THE PROJECT TEAM HOLDS NO RESPONSIBILITY OR LEGAL LIABILITY FOR THE ITEMS AND WORDING CONTAINED WITHIN IT IN WHOLE OR IN PART. Local Health Integrated Networks (LHINS) and Health Service Providers (HSPs) may wish to include some of the activities identified in this document as deliverables, with assigned deliverable dates, in either a Vendor Statement of Work (SOW) or a Vendor Agreement. If a LHIN has decided on a vendor strategy, depending on that strategy and the software model, the nature of these activities and those responsible for completing the activities may be different, but in general the activities are the same. Vendors and LHINs/HSPs are responsible for ensuring that solutions are sufficiently tested prior to going live and entering assessments into the system. There are different types of testing. In general, and at a minimum there are three types of testing that must occur with any software implementation. The three types of testing are: Unit Testing – Unit testing involves the testing of individual software components or modules. This testing is typically done by the vendor’s programmers/developers. It requires detailed knowledge of the internal program design and code. The vendor is responsible for conducting unit testing. This is done without the involvement of, or notice to, the LHIN/HSP. After all unit testing is complete, system testing can commence. System Testing – System testing involves testing compliance of complete, integrated system with software requirements specifications. This testing involves the testing of functionality and does not require knowledge of the internal program design or code. The vendor is responsible for conducting system testing. This is done without the involvement of the LHIN/HSP. The vendor may notify the LHIN/HSP when system testing is complete as it is a prerequisite to commencing with user acceptance testing. ccim.on.ca Page 1 of 3 User Acceptance Testing (UAT) - This type of testing is typically done to verify a system meets customer requirements. The ‘user’ or customer (i.e., LHIN/HSP) does the testing to determine whether to accept the software solution. The LHIN/HSP is responsible for ensuring the solution meets requirements. While the LHIN/HSP performs UAT, materials and direction are typically provided by the vendor as part of quality assurance activities. At a minimum, two (2) cycles of UAT should be planned. This is to accommodate the fixing of defects found after the first cycle of testing. Quality assurance activities may be led by the vendor or the LHIN/HSP. The LHIN/HSP may want to coordinate and lead quality assurance activities in lieu of the vendor. With UAT successfully complete, the vendor and LHIN/HSP are ready to install the software solution into the ‘live’ or ‘production’ environment. Testing is conducted in a test environment (i.e., not the production or live environment where users are actively using existing software applications). This is particularly important where there are systems in place that are being actively used and these systems will be replaced, or upgraded, with the new solution. Testing activities must be conducted outside of live environments to avoid any risk to the live system or data. To support testing activities, there should be a test plan and a schedule. The test plan includes, but is not limited to: the types of testing that will happen, entry and exit criteria for moving to the next type of testing, who is involved in the testing, and the criteria for testing completion before going live with the solution. Testing includes creating and running test scripts and test scenarios. These scenarios and scripts test system functionality. The results of the test scripts and scenarios are compared against expected results to indicate whether the test was successful. When the predetermined success rate for testing is achieved, testing is complete. The roles and responsibilities of test participants should be defined for testing activities and should also list or identify the criteria for test type completion. Some activities to consider in the development of a test plan and schedule are: 1. Identify testing process for UAT Establish end user support procedures. This includes creating an error tracking/ issues log, identifying how bugs will be reported and managed, and establishing a timeline for the entire testing process. 2. Complete testing documentation This includes test scenarios and checklists 3. Identify when unit testing will be complete 4. Identify when system testing will be complete ccim.on.ca Page 2 of 3 5. Identify when UAT will commence This date will be the earliest date when LHIN/HSP can begin testing 6. Identify when patches will be installed to address bugs identified through UAT 7. Identify when UAT will be complete After testing activities are successfully completed, vendors and LHINs/HSPs can proceed with validating the implementation. Implementation validation must be performed by both the vendor and the LHIN/HSP. The vendor validates first, followed by the LHIN/HSP. The validation is performed against the test environment which must be the exact same solution (i.e., software code) as the ‘live’ environment. Implementation validation is addressed in the ‘Implementation Validation Guide’ on the CCIM website. ccim.on.ca Page 3 of 3