Inshore Vessel Monitoring System (I-VMS) Approval Programme Approval Policy & Technical Validation 13th October 2014 1 Notices DOCUMENT SUBJECT TO CHANGE This document and the technical information it contains is subject to change by the UK Fisheries Department at any time. NO WARRANTY OR LICENSE ALL INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”, WITHOUT ANY EXPRESS OR IMPLIED WARRANTY, INCLUDING BUT NOT LIMITED TO A WARRANTY THAT IT IS ACCURATE OR COMPLETE, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR A WARRANTY AGAINST INFRINGEMENT. THIS DOCUMENT GRANTS NO RIGHT OR OTHER LICENSE, WHETHER EXPRESSLY OR BY IMPLICATION. Author(s): Katie Hill (Marine Management Organisation) Andy Wills (Marine Management Organisation) Revision History: Revision KH Date 23/08/2013 KH KH KH KH 11/09/14 09/10/2014 10/10/2014 10/10/2014 Comments Corrections to v.1 draft doc Wording consistency Add process map 2 MC and AW comments Final formatting Contents 2 1. Introduction and Scope .................................................................................... 4 2. Approval Programme Policies .......................................................................... 4 2.1. Introduction ................................................................................................... 4 2.2. Award of ‘Approved Product’ ......................................................................... 5 2.2.1 Result Description .................................................................................. 5 2.4 Confidentiality................................................................................................ 6 2.5 Freedom of Information ................................................................................. 6 2.6. Approval Programme Data Retention Policy ................................................. 6 2.7 Fees and Costs .................................................................................................. 6 2.7.1 Technical Test Costs...................................................................................... 6 2.7.2 Postage Costs ....................................................................................... 7 2.7.3 Product Re-test....................................................................................... 7 3. Technical Validation Procedures ..................................................................... 7 3.1. Submission Review Procedures................................................................... 7 3.2. Technical Review Procedures ...................................................................... 8 4 Test failures and product retesting ..................................................................... 9 4.1 Problem Reporting Process .......................................................................... 9 4.1.1 Resolution of Problem Reports ............................................................... 9 4.2 Device Failure ................................................................................................. 10 5. Post Approval Policy ......................................................................................... 11 5 5.1. Approved Product Register ......................................................................... 11 5.2 Marketing & Press Release Policy .............................................................. 11 5.3 Device upgrade or change .......................................................................... 12 5.4 Removal of Approved Product Status ......................................................... 13 4.5. Approved Product Re-validation Requirements .......................................... 14 Appeals Procedure .......................................................................................... 14 6. References ....................................................................................................... 14 6.1. Normative References ............................................................................... 14 6.2 Definitions .................................................................................................. 15 7. Appendix A: Process Map 2 (Device Test Failures) ...................................... 16 8. Appendix B: I-VMS Approval Programme Problem Report Template ........ 17 9. Appendix C: I-VMS Programme Change Analysis Template ...................... 18 3 1. Introduction and Scope This document defines the Approval Programme policies and validation procedures that govern the operation of the Inshore Vessel Monitoring System (I-VMS) Approval Programme [hereafter: Approval Programme]. This document defines what can be approved and what it means to be approved. This document is intended primarily for a product supplier who would like to submit their product for approval against the I-VMS specification of requirements and should be read in conjunction with the I-VMS Approval Programme Guide The goal of the Approval Programme is to ensure that software suppliers providing IVMS products to UK fishing vessels fishing in English waters implement the I-VMS specification of requirements correctly and consistently. This document describes the Approval Programme policies and validation procedures which apply to the device suppliers product in order to assess if the product meets the approval criteria. To become an Approved Product, a product supplier must submit their product to independent validation by the approval panel made up of MMO and IFCA representatives. When approval has been confirmed, products are listed by the MMO as an ‘MMO and IFCA IVMS Approved Product’ on the approved products register which can be found at: https://www.gov.uk/government/collections/inshore-vessel-monitoring-system-project 2. Approval Programme Policies 2.1. Introduction The section provides reference to the policies governing the Approval Programme. These policies do not provide an exhaustive set of rules. A product supplier wishing to submit a product to the Approval Programme who is not able to fulfil all the guidelines, requirements and objectives described in this document, needs to raise the issue to the approval panel for clarification prior to beginning the Approval Programme process. The operation of the Approval Programme is based on the following principles: The Approval Programme is operated in an open and transparent manner, avoiding conflicts of interest or the award of approval other than for relevant technical considerations; The Approval Programme shall not practice any form of discrimination such as hidden discrimination by speeding up or delaying the processing of product applications; The Approval Programme will not be used to fix prices, lessen competition or have the effect of boycotting or excluding competitors; The Approval Programme will provide equal due process to all parties interested in or affected by the programme, including the conduct of timely 4 hearings, with prompt decisions on claims respecting the denial or discontinuation of approval. 2.2. Award of ‘Approved Product’ The Approval Program validation procedures require that a product is assessed against the approval criteria defined by the MMO and IFCA I-VMS specification of requirements and Section 3 of this document. There are two possible results of participating in the Approval Programme: 2.2.1 Result Description The product is assessed to pass the approval criteria as validated by the approval panel. The product has met the approval criteria for the I-VMS specification of requirements. The product is assessed to fail the approval criteria as validated by the approval panel. The product has failed to meet the approval criteria for the I-VMS specification of requirements. Note that the product supplier is permitted, within specific guidelines and with the approval panel approval to re-submit their product in subsequent test windows. Using the supplier declaration form the product supplier is required to describe how the mandatory requirements are to be met by the product, and specify any additional functionality. The approval criteria as validated by the approval panel are defined as follows: Product is validated to support all mandatory aspects of the I-VMS specification of requirements; Approval shall not be granted until all validation criteria are met. Test and audit failures which raise any doubt as to the compliance of a suppliers product must be corrected and the correction verified by the approval panel before approval is granted. The test failures and their resolution are documented by the approval panel 2.3. Approval Programme Administration 5 This Approval Programme shall be administered by the approval panel, made up of MMO and IFCA representatives. Product suppliers can participate in the Approval Programme privately and beyond the awareness of other product suppliers. The approval panels role is to uphold the policies and procedures defined in this document. 2.4 Confidentiality All information relating to a product supplier and the product to be approved is held confidentially during the approval process, that is, prior to the successful completion of the Approval Programme. If products are not successful all information will remain confidential. All technical and commercially sensitive information relating to a product supplier and their product submission details always remain confidential within the Approval Programme with the exception of the product and product supplier details that are published on the approved product register. This includes information related to the product submission information, supplier declaration form, technical audit reports and problem reports. The MMO has contracted an external organisation which will carry out a number of technical tests on the device in line with the specification of requirements. They will be bound by the same confidentiality requirements. 2.5 Freedom of Information Product suppliers should be aware that the MMO and IFCAs are subject to the requirements of the UK Freedom of Information Act 2000 and the Environmental Information Regulations 2004 and any other applicable legislation governing access to information (“FOI Legislation”). The MMO and IFCAs shall be responsible for determining in its absolute discretion whether the commercially sensitive information and/or any other information is exempt from disclosure in accordance with the provisions of the Secretary of State for Constitutional Affairs' Code of Practice on the Discharge of Functions of Public Authorities under Part I of the Freedom of Information Act 2000 or the Environmental Information Regulations. 2.6. Approval Programme Data Retention Policy All product submissions and the validation reports for the Approval Programme will be archived by the MMO. The archives are kept for a minimum of 6 years to provide an audit trail for Approved Products. 2.7 Fees and Costs 2.7.1 Technical Test Costs The product supplier will be required to meet any costs associated with: 6 provision of 2 devices and associated components integration with the EU VMS hub, transmission of positional reports and status codes throughout the Approval Programme installation of device B on the IFCA vessel Costs to repair/replace devices if test failures are identified. There will be no costs for testing when a device is submitted for the first time. If test failures are identified and cannot be rectified within a reasonable time (see s4.2 for more details) the approval panel may close the test programme. If the product supplier wishes, the device can be re-submitted in the next test window however in this instance all testing fees will be met by the product supplier. The fee covers the costs associated with formal independent validation of the product. 2.7.2 Postage Costs The product supplier will cover the following postage costs: Submission of 2 devices to the MMO All postage costs associated with rectifying test failures (if device cannot be fixed remotely) 2.7.3 Product Re-test If an approved device requires re-testing as a result of a major change to the software or version of the device it may have to be re-tested (see s.5.3 for more details) All testing costs will be met by the product supplier. The approval panel may re-test approved devices to ensure that they still meet the specification requirements. Product suppliers will be expected to meet the costs of this quality assurance practice 3. Technical Validation Procedures For a product to be approved, a product supplier must submit their device to an independent validation by the approval panel .This section defines the technical validation procedures to be performed by the approval panel. 3.1. Submission Review Procedures Prior to testing any device suppliers will be asked to provide a number of submissions. All submissions to the Approval Programme are reviewed as a paper based exercise by the approval panel based on the following review criteria: Test Item 1 Audit Step Audit Actions Conditions of Participation and Confidentiality agreement Validate that a signed conditions of participation and a confidentiality agreement are in place between the approval panel and the I-VMS product supplier before the formal 7 Supplier Declaration Form (SDF) technical audit of the device. Note that both documents must be signed (the confidentiality agreement is an annex to the conditions of participation) Review and verify that all mandatory fields required have been completed correctly. 3 Product Identification and Commercial Description. Ensure the SDF confirms the device meets all mandatory features of the I-VMS specification of requirements. Verify that information has been completed by the submitter 4 Statement of Conformance 6 Approval panel fee 2 Verify that all information has been included and that there is either an electronic signature or physical signature. Where applicable. This will only be required where a device is re-submitted in a new test window after failing a prior approval process. 3.2. Technical Review Procedures The technical review consists of a system examination by the approval panel of the supplier’s I-VMS product. This involves testing of the product with the objective to verify that the features of the product defined in the supplier declaration form have been implemented as stated in the I-VMS specification of requirements. A technical audit report is written by the approval panel detailing the objective evidence collected and reviewed during the submission and technical validation supporting the product supplier’s compliance claim. This report will detail if a device has passed or failed and the supplier will be sent an email notification. The product supplier should facilitate the technical validation as follows: provide a technical contact to assist with the smooth running of the testing process able to answer queries and respond to issues. Ensure an engineer is available to install a device onto a vessel at a given location within the timescales provided (the installation location may be anywhere in England) perform a dry-run of the geofence tests, as provided in the supplier declaration form, to ensure the product can fully demonstrate the compliance prior to submission of the device for final testing. The technical validation is conducted by the approval panel and consists of the following audit actions: Test Item 1 Validation Step Two Devices submitted for testing within test window Validation Actions Visual external check of both devices 8 2 3 timescales MMO and IFCA technical testing of product Installation and configuration procedures for the supplier product by witnessing a clean install and tamper test of the product by an approved engineer within given timescales. Witness and document the configuration of product to connect to the EU VMS Test hub. Witnesses the generation of the I-VMS reports to the EU VMS test hub Validation of I-VMS data transmission functions through at-sea testing and test hub tests ensuring conformity to the I-VMS specification of requirements. External organisation Tests, carried out by the external testing technical testing of product organisation, demonstrate the following device functionality is in line with the specification of requirements: GNSS augmentation and positional accuracy GNSS roaming capabilities and multi constellation GNSS performance GNSS blackout testing Tampering prevention Power management 4 Test failures and Product Re-testing 4.1 Problem Reporting Process The MMO and IFCAs recognise that during the approval process, issues with the audit and validation procedures or interpretation of the I-VMS specification of requirements may occur. If a product supplier encounters a problem that inhibits or will inhibit the approval of their product an applicant may raise a problem report (PR). The problem report is used specifically for errors which are inhibiting the approval effort. A problem report template can be found in appendix B of this document. And can be submitted to the approval panel at: IVMS@marinemanagement.org.uk 4.1.1 Resolution of Problem Reports Each PR is assigned a unique number so that tracking is possible. Sufficient information related to the issue should be included with the submission of the PR in order for the approval panel to be able to analyse and review the issue. Failing to do so may lead to rejection of the PR. 9 The approval panel is responsible for reviewing each problem report submitted by a product supplier. An initial response provided to the submitter within 5 working days of submission to allow simple problems to be dealt with quickly. A formal review by the approval panel will be undertaken for issues that are more complicated or when the preliminary review does not resolve the issue. Final resolution will be provided within an additional 10 working days. Non-rejected PRs will be actioned and where necessary proper corrections will be inserted into revisions of I-VMS specification of requirements and the Approval Programme documents. If the product supplier is not satisfied with the final resolution of the PR, they may invoke the appeals process. 4.2 Device Failure The process map in Annex A: Process Map 2 (Device Failures) details actions to be taken in the case of a device test failure. The intention of the formal testing is to allow a product supplier to demonstrate compliance with the I-VMS functional requirement specification. Formal testing is not intended to be part of the I-VMS product supplier’s development process or debugging process. As a result only minor test failures will be allowed to be rectified and continue within the test window The approval panel will consider all incidents of test failure to decide if the testing can continue within the window. Length of time required to complete testing, impacts of problem resolution on tests already completed, severity of test failures, number of previous test failures and any other issues the approval panel consider relevant will be taken into account in making this determination. If a test failure is identified the initial 6 week testing process may be suspended (although, where possible, tests against other specification elements will be carried out) but will resume if the failure can be resolved within 30 calendar days. If possible issues should be rectified remotely but if this is not possible suppliers are responsible for arranging for the collection of the device and subsequent return to the appropriate location once the issue has been resolved. Testing should not exceed a 10 week period in total or continue more than 6 weeks after the window has closed. If the failure cannot be rectified the test programme will close and the supplier can, if they wish, resubmit their device in the next test window. Timescales should act as a guide only and the approval panel reserve the right to close the test programme at any time. If product supplier is required to and wishes to resubmit the device in the following window and undergo the full technical validation the full costs will be met by the supplier. If the approval panel close the test programme the device is deemed to have failed the Approval Programme. The decision can be reviewed through the appeals 10 process if a product supplier disagrees with the grounds for denying approval. The product supplier must inform the approval panel within 10 working days if they wish to appeal the decision. 5. Post Approval Policy This section provides reference to the policies governing post approval requirements 5.1. Approved Product Register The MMO will maintain an approved product register consisting of a record of all approved products on the MMO website (IFCA’s will also be able to display a link to the register on their website). The approved product register contains the name of the supplier, name and version of the product, I-VMS products that have been successfully validated against the I-VMS specification are listed will be listed on the following website: https://www.gov.uk/government/collections/inshore-vessel-monitoring-system-project IFCAs may display a link to this page on their own website. The first publication of the Approval Product Register will be on completion of the first 6 week test period1. Suppliers will be notified by email within 5 working days if their product has passed the approval process. Products which are approved in subsequent testing windows will be added to the Approval Products Register and suppliers notified by email within 10 working days of the device receiving approval. 5.2 Marketing & Press Release Policy On successful completion of the Approval Programme, a product supplier may issue a press release for their product. Companies are required to send their announcement to media@marinemanagement.org.uk for review. The MMO will return the press release with comments and edits to the company within 5 working days from when press release was received. It is the responsibility of the MMO to review each press release using the following criteria: The press release is reviewed to ensure that it does not contain untruths pertaining to the activities of the Approval Programme or functionality of the product or product supplier. 1 Any devices submitted in the November 2014 test window between 10th November and 24th December will be subject to a 7 WEEK test window to allow for the Christmas holiday period 11 The press release is reviewed for accuracy of any statements and copyrights pertaining to the Approval Programme. The press release is reviewed to ensure that it references only the submitter's company and product and does not add or use comparisons to other participants or products that were involved in the Approval Programme. The press release is reviewed to ensure that it does not breach the NDA signed by all participants of the Approval Programme 5.3 Device upgrade or change When a product goes through its cycle of upgrades, fixes, marketing changes and major revisions, a product supplier may need to re-test and re-submit the product as an Approved Product. The policies for re-testing / re-certification are as follows: Category Major change in software or version1 with changes to mandatory requirements as detailed in the I-VMS specification of requirements Description and requirement Product is required to be submitted for retesting All submission documents must also be resubmitted. The product details are listed as a new entry in the approved product register. 1 Versioning is from 1.x to 3.x. A minor change is a point increase, e.g. 1.n to 1.m. Minor change in the software or version with no changes mandatory requirements as detailed in the I-VMS specification of requirements Changes in Product Details If there are no changes in the I-VMS specification of requirements features then no formal re-testing or re-submission is required. A completed I-VMS programme product change analysis form must be submitted to the approval panel detailing the changes between the current listed version(s) and the new software version(s) to be listed. A new supplier declaration form may be requested by the approval panel. The product details in the approved product Register on the MMO website are updated as appropriate based on the details in the Product Change Analysis document. If there are no changes in software or operating environment (e.g. change in product name only), then no formal re-testing or resubmission is required. A completed I-VMS Programme Product Change Analysis document must be submitted to the approval panel detailing the changes in product details. 12 All other situations The product details in the Approved Product Register on the MMO web site are updated as appropriate based on the details in the Product Change Analysis document. Complete product change analysis document to the approval panel detailing change in situation. On review of the change analysis the panel will make a decision if a full formal validation is required Product details are listed as a new entry in the approved product register on the MMO web site. The product change analysis form can be found in appendix C of this document. All product change analysis forms or any queries regarding product changes should be submitted by to the approval panel using the following email address: IVMS@marinemanagement.org.uk 5.4 Removal of Approved Product Status If an approved product is found by any means to no longer conform with the I-VMS specification of requirements, the approval panel will notify the I-VMS product supplier who shall do one of the following: Within 20 working days rectify the non-conformity and satisfy the approval panel of the integrity of the rectification. Within 20 working days satisfy the approval panel that there is no nonconformity in the Approved Product. Within 10 working days invoke the appeals process as described in this document (see s. 5). Only products meeting the Approval Programme validation procedures are included on the approved product register. The MMO will remove the I-VMS product from the approved product register within 20 working days of finding that an approved device is no longer a compliant product and informing the product supplier. Once a product ceases to be an approved product, all rights to refer to the product as an ‘MMO and IFCA I-VMS Approved’ ends and the product supplier must remove all references to the device as ‘Approved’ within 10 working days A product supplier MUST inform the approval panel immediately if a product listed on the approval product register is to be removed (for example if the product is no longer commercially available or obsolete) or upgraded to a new version. 13 5.5. Approved Product Re-validation Requirements When a product has met the Approval Programme validation and test procedures, details are posted on the MMO website on the I-VMS approved product register and remain there until the expiry of the Approval Programme. An I-VMS product supplier must submit a completed product change analysis form to the approval panel for all subsequent versions of their validation product that are deployed on ‘live’ vessels The approval panel reserves the right to require a product be submitted for reapproval. 6 Appeals Procedure If the product supplier’s complaint meets one of the following situations, then the decision can be reviewed through the appeals process: The product supplier disagrees with the grounds for denying approval. The product supplier disagrees with the resolution of a problem report The product supplier disagrees with a formal notification to rectify nonconformance (that is, a product supplier is representing a product as being an approved product that no longer meets the audit and validation criteria for approval) The supplier should notify the approval panel of their intention to appeal within 10 working days of being informed of a decision by emailing: IVMS@marinemanagement.org.uk An independent panel will deal with appeals: Within 10 working days of receiving a product suppliers grounds of appeal the appeal panel will indicate whether further information is required and/or expert evidence is required. Within 6 weeks of receiving the product suppliers grounds of appeal the appeals panel will meet to consider the appeal. Identifying information is removed and the appeals panel meets formally to review the request including any additional material either provided by the supplier or requested by the committee. Experts may be invited as needed. The decision of the panel is final. 6. References 6.1. Normative References 1. I-VMS specification of requirements 2. Supplier declaration form. 3. I-VMS Approval Programme. Programme Guide. 14 4. Product change analysis form 5. Problem report form 6.2 Definitions Approval Panel Made up of MMO and IFCA representatives appointed to conduct validation of the submitted product against the audit and test procedures. External testers and facilities have been employed for some aspects of the approval process procedures. Approved Product Register A web-based record of all approved products, which is maintained by the MMO. External Testing Organisation A organisation contracted by the MMO to provide expertise and a robust testing method for the technical elements of the specification of requirements which cannot be carried out in an ‘atsea’ environment Geofence A geofence is a defined spatial perimeter for realworld geographic areas Supplier Declaration A statement made by the supplier of an implementation or system claimed to conform to the I-VMS specification, stating which capabilities have been implemented, including which optional features are supported. Technical Validation The operation of independently validating the product submitted for approval through auditing and testing. Working day Monday to Friday inclusive, excluding English public and bank holidays 6.2.3 Acronym Definition I-VMS MMO IFCA SDF PR Inshore Vessel Monitoring System Marine Management Organisation Inshore Fisheries and Conservation Authority Supplier Declaration Form Problem Report 15 Appendix A: Process Map 2 (Device Test Failures) I-VMS Approval Process: Map 2 (Device Test Failures) Please refer to IVMS Approval Process: Map 1 Yes Has device passed tests? No MMO to contact device supplier Is this 1st incident of non conformance? No Close test programme. Supplier informed. Device can be resubmitted in next window Yes Can the problem be rectified within a reasonable time frame?1 MMO will consider on a case by case basis (in collaboration with supplier and external tester) to determine if testing can continue3 Testing Resumes No Supplier to rectify issue Yes No Yes Is there still reasonable time within the test window once the test failure is rectified? Testing to continue where possible Yes Yes On resubmission will complete or significant retest be required? No Can the problem be fixed remotely? No Device returned to supplier2 1 As a guide; if the failure can be resolved within 30 calendar days of the test failure being identified, then the Contractor shall resume testing with no loss of time to the six week test programme. If there are continuing issues each test failure will be considered on a case by case basis and the testing should not exceed a 10week period or continue more than 6weeks after the window has closed. 2. Postage is at the suppliers cost. Fix, return and delivery must be done within 30 calendar days of the test failure being identified. Supplier can request tester return the device. (Contractor can invoice MMO for postage costs) 3 Factors considered include. Length of time required to complete testing, time already spent testing device, severity of non conformance, number of previous non conformances and any other issues the MMO consider relevant. The intention of the formal testing is to allow a product supplier to demonstrate compliance with the I-VMS functional requirement specification. Formal testing is not intended to be part of the I-VMS product supplier’s development process or debugging process. At this stage only minor non conformances will be allowed to be rectified and continue within this test window. Purple Boxes = supplier action 16 8. Appendix B: I-VMS Approval Programme Problem Report Template Problem Report No: Date: Summary: Detailed Description: XXXXX <YYYY-MM-DD> <Provide a one line summary of the Problem> <Provide a detailed description of the Problem> Specification Reference: <Specification - {Version|Section|Page|Line}> Process Reference :<Process Document - {Version|Section|Page|Line} Additional > Information/Attachments: <Yes>||<No> Submitters Details (will be removed for confidentiality) Company Name: Contact Name: Contact E-mail address: Contact Telephone: Device relating to Problem Report: Internal Use Only Responder Response Days Open: Date Resolved: <##> <YYYY-MM-DD> Notes on submitting a Problem Report: The submitter should only complete the areas of the proforma that are not greyed out. The greyed out areas will be completed in response to the Problem Report. The submitter's details will be removed by the Problem Report Handler to maintain confidentiality of the issue. The Problem Report Handler will assign a Problem Report number. The submitter should provide a one-line summary of the issue and a more detailed description of the issue. If the submitter wishes to provide additional information, they should indicate that information has been submitted with the Problem Report, and include the additional information as an e-mail attachment. The Problem Report Handler will keep all vendor specific material confidential 17 9. Appendix C: I-VMS Programme Change Analysis Template This template is to be used for minor changes where there is no impact to I-VMS functional requirements. For major changes the product must be re-submitted for a new formal validation. Change Analysis Submission Date Supplier Name: (as currently listed) Supplier E-mail address: Supplier Telephone: Name of Product: (as currently listed) Versions: (as currently listed) Date Approved: (as currently listed) Version of Approval process applied: (as currently listed) Proposed Version(s) for Listing (relating to this Change Analysis): High level description of changes between listed version(s) and proposed version(s): <YYYY-MM-DD> Description of changes in the proposed version(s): Do changes impact I-VMS functionality? Details of any I-VMS functional requirements impacted by the changes : Yes [ ] No [ ] Description of testing performed by vendor to validate that I-VMS functional requirements are not negatively impacted: Description of how the changes fits into vendor’s versioning methodology, including how this version number indicates that this is a “minor” change: Any additional supporting information relating to the changes: Notes on submitting the change analysis: This change analysis form should be completed and submitted to the Approval panel detailing the changes between the current listed version(s) and the new software version to be listed. This change analysis form should be submitted in conjunction with an updated Supplier Declaration Form 18 19