User Manual AKKA – Networking Test Suite 5.22.0 Version Date: 2021-06-10 Document version: 6.4.0 Status: approved File name: UserManual.pdf Department: CoE Digital Validation & Products Department Products Page 1 of 44 Information security class: open List of Content 1 Overview of the AKKA – Networking Test Suite ........................................................................ 6 1.1 Scope of delivery............................................................................................................................. 6 2 Important instructions .................................................................................................................. 7 2.1 Intended Use ................................................................................................................................... 7 2.2 Responsibility of the operator ......................................................................................................... 7 3 Used hardware- and software components ............................................................................... 8 3.1.1 Vector relay box VH1150 / VH1160 ................................................................................................ 8 3.1.2 Supported Vector network interfaces .............................................................................................. 9 3.1.3 Restrictions when using VN89xx interfaces ..................................................................................10 3.1.4 Minimum required hardware driver version ..................................................................................10 3.2 Used software components ..........................................................................................................11 3.2.1 Vector CANoe ...............................................................................................................................11 3.2.2 DAG Security Package .................................................................................................................11 3.2.3 ZenZefi ..........................................................................................................................................12 3.3 Used Open Source Software ........................................................................................................12 4 Installation ...................................................................................................................................13 4.1 General Requirements ..................................................................................................................13 4.2 System Requirements ...................................................................................................................13 4.3 Installation Procedure ...................................................................................................................13 4.3.1 License File ...................................................................................................................................15 5 Test preparation and execution .................................................................................................16 5.1 Preparing and connecting of the Device under Test (DUT) ..........................................................16 5.1.1 Ethernet .........................................................................................................................................16 5.1.2 LIN .................................................................................................................................................16 5.2 Initiating and configuring of the NTS .............................................................................................17 5.3 Install and use diagnosis certificates ............................................................................................18 5.4 Configuration of diagnostic channel and bus system ...................................................................19 6 Noteworthy features ...................................................................................................................22 6.1 NTS Headless Mode .....................................................................................................................22 6.2 CANoe register/unregister ............................................................................................................26 6.3 AddOn-Feature DWT (Dauerwecktest) – License extension required .........................................27 6.3.1 Introduction ...................................................................................................................................27 6.3.2 System Requirements ...................................................................................................................27 6.3.3 Test preparation and execution ....................................................................................................27 6.3.4 Step by step explanation of the Dauerwecktest ............................................................................28 6.3.5 Test result and test result verdict ..................................................................................................28 6.3.6 Excel evaluation ............................................................................................................................29 6.3.7 Headless Mode .............................................................................................................................29 Page 2 of 44 Information security class: open 7 Ethernet testing with the NTS and different CANoe versions ................................................30 8 Overview of specification documents according to bus system ...........................................32 9 Maintenance test environment ..................................................................................................34 10 Maintenance NTS and support issues ......................................................................................35 11 Technical specifications ............................................................................................................36 11.1 Operating conditions .....................................................................................................................36 12 Frequently asked questions ......................................................................................................37 12.1 No connection to test hardware VH1150 / VH1160 ......................................................................37 12.2 CAPL DLL is invalid ......................................................................................................................38 12.3 Invalid DUT file ..............................................................................................................................39 12.4 Invalid channel assignment for network interface VN89xx ...........................................................39 12.5 Wrong version of ZenZefiNodeLayer.dll .......................................................................................39 12.6 Vector VH11xx connected to the computer is not recognized by the NTS ...................................40 13 Appendix ......................................................................................................................................41 13.1 Abbreviations ................................................................................................................................41 13.2 Glossary ........................................................................................................................................42 14 Legal notices ...............................................................................................................................44 14.1 Copyright notice ............................................................................................................................44 14.1.1 General legal notice ......................................................................................................................44 Page 3 of 44 Information security class: open Index of Figures Figure 1: NTS Setup Wizard – Start of installation ..................................................................................... 13 Figure 2: NTS Setup Wizard – Confirm installation .................................................................................... 14 Figure 3: NTS Setup Wizard – Complete installation ................................................................................. 14 Figure 4: License file overview ................................................................................................................... 15 Figure 5: Select diagnostic certificate for ECU ........................................................................................... 18 Figure 6: Configuration of the diagnostic channel ...................................................................................... 19 Figure 7: Configuration of the diagnostic channel (with required FrTp file) ............................................... 20 Figure 8: Selection of the bus to be tested and start of the test run .......................................................... 21 Figure 9: Selection of the bus interfaces to be stored in the configuration file ........................................... 22 Figure 10: Register/unregister CANoe ....................................................................................................... 26 Figure 11: Error message in the CANoe Write window – No connection to VH1150 / VH1160 ................ 37 Figure 12: Network hardware – Configuration of Date and Time ............................................................... 37 Figure 13: Vector CANoe – CAPL DLL ...................................................................................................... 38 Figure 14: Network hardware – Update CANoe Software ......................................................................... 38 Figure 15: Invalid DUT file .......................................................................................................................... 39 Figure 16: Invalid channel assignment for bus interface VN89xx .............................................................. 39 Page 4 of 44 Information security class: open Index of Tables Table 1: Supported Vector network interfaces for NTS and DWT ............................................................... 9 Table 2: Minimum required driver version .................................................................................................. 11 Table 3: Open source software .................................................................................................................. 12 Table 4: Voltage supply variants ................................................................................................................ 16 Table 5: Terminal settings in the NTS ........................................................................................................ 17 Table 6: DWT test report structure ............................................................................................................. 28 Table 7: DWT test verdict possible scenarios ............................................................................................ 29 Table 8: Access mode overview for Vector Ethernet interfaces................................................................. 30 Table 9: NTS handling for Ethernet access mode configuration ................................................................ 31 Table 10: Abbreviations .............................................................................................................................. 41 Page 5 of 44 Information security class: open 1 Overview of the AKKA – Networking Test Suite The AKKA – Networking Test Suite, briefly NTS, is a software tool, developed for testing individual network interfaces of ECUs. NTS focuses on electronic control units designed for OEM vehicle networks. The tests included in the NTS are developed according to the latest released OEM networking specifications. The following topologies and bus systems are supported: Daimler Cars: - STAR2-Topology: LIN, CAN 2.0, FlexRay - STAR3-Topology: LIN, CAN 2.0, CAN FD, FlexRay, Ethernet (100Base-T1 / 1000Base-T1) - STAR3.5-Topology: LIN, CAN 2.0, CAN FD, FlexRay, Ethernet (100Base-T1 / 1000Base-T1) Daimler Trucks and Busses: - SLP9-Topology: CAN2.0 (only OSEK-NM) - SLP10-Topology: CAN2.0 (OSEK-NM and AUTOSAR-NM) - SLP11-Topology: CAN2.0 (OSEK-NM and AUTOSAR-NM), CANFD, Ethernet (100Base-T1) All network interfaces are connected separately with the corresponding test hardware channels and do not influence each other. Main focus of the testing phase is: testing of the basic networking implementation and proper startup- and shutdown-behaviour as well as reaction on disturbances and compliance of defined cycle times. AKKA – Networking Test Suite labels the total package which is provided by AKKA for the test of the ECU. This includes on the one hand a graphical user interface for the configuration of the test system and on the other hand the test cases which are specified by the OEM, including the execution logic and evaluation. 1.1 Scope of delivery The AKKA scope of delivery contains the NTS software and a license file. The NTS software is available via a specific, secure download platform. Access to the download platform is granted on request. The license key will be sent by e-mail. The software CANoe required for carrying out the test and the related hardware network interfaces can be obtained from Vector Informatik GmbH. The ECU Extracts, relevant for the test configuration, are provided by the OEM as well as the test or requirements specification. Page 6 of 44 Information security class: open 2 Important instructions 2.1 Intended Use The NTS shall only be used to test the functionality of the automotive network interfaces at network management level. The field of application is limited to the development of control devices in the automotive field. The bus systems, as they are mentioned in chapter 1, are supported. Testing of functional ECU software, as it is the case, for example, with HIL or SIL tests are not part of the NTS. 2.2 Responsibility of the operator The NTS provided by AKKA is only a software product. When using the other hardware and software necessary for operation, the corresponding manuals must be observed. In order to ensure trouble-free operation, the SW versions specified in the Feature List or in this document must be used. The NTS Feature List is available upon request or can be reached via the NTS menu “Help”. For access to the download platform please contact GT-NTS@akka.eu. Page 7 of 44 Information security class: open 3 Used hardware- and software components To use the AKKA – Networking Test Suite only the CANoe test hardware VH1150 or VH1160 as well as the network interface provided by Vector Informatik GmbH are necessary. The DUT, hence the ECU to be tested, is connected by banana plugs to the switched power supply at the VH1150 or VH1160 and its network interface by D-Sub 9-connector. If a CAN- or LIN-network interface shall be tested, the bus cables are connected to the VH1150 or VH1160. In case a FlexRay- or Ethernet-network interface shall be tested, the network interface is directly connected to the Vector bus-interface. In case of an Ethernet test, the wakeup-line of the ECU (only clamp 30) must be triggered. For more information see chapter 5.1. The supply voltage pins are always connected to the VH1150 / VH1160 which provides voltage for the DUT at the right time and if necessary wakes it up. 3.1.1 Vector relay box VH1150 / VH1160 The Vector relay box VH1150 / VH1160 switches the supply voltage of the DUT. Between two test cases the supply voltage is automatically switched off and on again to ensure a defined state of the DUT and to exclude influences of one test case to the following test case. Certain DUTs (eg. clamp-15-DUTs) have to be woken up by a switched voltage on clamp 15, if they cannot be woken up by the bus. This clamp-15voltage signal is created by the VH1150 / VH1160, too. Due to fact that the VH1150 / VH1160 does not enable any bus access, a Vector network interface is necessary in addition. The bus cables are connected through the VH1150 / VH1160 in case of CAN and LIN. Therefore, it is possible to interrupt the bus lines or to short-circuit them against each other, against GND or LIN against the supply voltage. These possibilities are not available for FlexRay and Ethernet, thus the DUT is connected directly to the network interface. However, the supply voltage is still switched by the VH1150 / VH1160. The installation of the required VH1150 / VH1160-driver is described in the user manual. This refers to the also required FTDI-driver and their reference sources. For proper usage it is required to have the VH1150 / VH1160 with hardware revision 5 or higher. Details can also be found in the FAQ section of this document. Attention: Please make sure to adhere to given voltage and current limitations. Maximal supply voltage VH1150 / VH1160: 28 V Maximal current VH1150 / VH1160: 2.5 A For further information, please refer to the VH1150 / VH1160-user manual. Page 8 of 44 Information security class: open 3.1.2 Supported Vector network interfaces As a network interface several interfaces purchased by Vector Informatik GmbH are available, which only have to support the bus system to be tested. It is not necessary to choose one network interface that supports all kind of bus systems if only one bus system shall be tested. In general, all Vector network interfaces should be compatible to the AKKA – Networking Test Suite as long as the Hardware Driver Version meets the minimum required version mentioned in chapter 3.1.4. However, due to the multiplicity of available network interfaces, the compatibility of all network interfaces cannot be assured. Table 1 shows, which Vector network interfaces are supported by the AKKA – Networking Test Suite, since they have been used with the NTS. This overview shall only represent a selection guide and makes no claim on completeness or correctness. Further information on the products is available at Vector Informatik GmbH. CAN 2.0 CAN FD LIN FlexRay Ethernet VN1610A yes yes - - - VN1611 yes yes limited1 - - VN1630(A) yes yes yes - - VN1640(A) yes yes yes - - VN5610(A) yes2 yes2 - - yes3 VN5620 - - - - yes3,8 VN5640 yes yes - - yes3 VN7600 - - - yes - VN7610 yes yes - yes - VN7640 yes yes yes5,6,7 yes6 - VN8910(A) with VN89704 yes yes yes yes - VN8911 with VN89704, 9 yes yes yes yes - Table 1: Supported Vector network interfaces for NTS and DWT 1) The VN1611 does not support LIN2.1 compliance-tests (LIN Conformance Tests). 2) The VN5610(A) needs to be configured in Network-based mode in order to use a new firmware that fully supports the CAN interface and the required features for the NTS. 3) Please note that the NTS changes the configuration of the device. 4) In this case, the VH11xx needs to be connected to the computer instead of to the VN89xx. Please refer to chapter 3.1.3 for further details. 5) Using device driver versions ≤ 10.8.20 leads to crashes while running an LCT. Page 9 of 44 Information security class: open This issue is currently in progress at Vector Hardware Informatik. VN7640 is not supported for DWT when testing LIN 6) Using device driver versions ≤ 10.8.20 leads to crashes while running a DCT with FlexRay backbone bus. This issue is currently in progress at Vector Hardware Informatik. 7) DWT does not support LIN on the VN7640. Please use another LIN network interface. 8) This hardware can only be used with Vector CANoe 12 SP4 or higher in Network-based mode. 9) Has not been tested yet with DWT. Other Ethernet interfaces, such as VN5430 have not been checked yet regarding the compatibility with the NTS or DWT. In case you plan to purchase further interfaces, which are not listed here, do not hesitate to contact our support. This way, we can ensure, that the AKKA – Networking Test Suite covers your test hardware to give you access to all offered functions. 3.1.3 Restrictions when using VN89xx interfaces When using VN89xx with the NTS or DWT please make sure that the device is set to “interface mode”, using the file “can.ini”. The parameter EnableUSBAutoConnectToRuntimeKernel is needed for this action and is only available until CANoe 11 (C:\ProgramData\Vector\CANoe). Please disable (=0) this parameter in order to use the VN89xx as a standard USB interface. “Extended Real Time” (ERT) functionality is not supported by NTS or DWT. Please note that VN8972 has not been tested yet regarding the compatibility with the NTS or DWT and that additional Vector network interfaces connected to the VN89xx via USB or RJ45 are not supported by the NTS! 3.1.4 Minimum required hardware driver version To execute all available test cases the driver version of the used network interface shall be greater than or equal to the version number mentioned in Table 2. If the necessary driver is not available for the desired network interface another network interface shall be used or some test cases cannot be executed. Page 10 of 44 Information security class: open Bus system Minimum required driver version CAN 2.0 / CAN FD 10.9.12 Ethernet 11.2.16 FlexRay no minimum required version LIN no minimum required version Table 2: Minimum required driver version 3.2 Used software components Besides the hardware components Vector VH1150 / VH1160, the Vector network interface and the DUT, the software component Vector CANoe and the actual AKKA – Networking Test Suite are necessary to perform the test. 3.2.1 Vector CANoe The software tool CANoe offered by Vector Informatik GmbH is used as basic software. Using CANoe, the access to the bus data as well as the control of the network interface and the VH1150 / VH1160 is possible. For the use of CANoe a software-license with appropriate bus system option is required. The CANoe-license is available at Vector Informatik GmbH and thus not included in the AKKA – Networking Test Suite-License. The NTS requires Vector CANoe v11.0.81 (SP4), CANoe v12.0.216 (SP5), CANoe v13 (SP3), CANoe v14.3 or CANoe v15.1. CANoe 15 or higher has not been tested yet regarding compatibility with DWT. CANoe pex is not supported by the NTS or DWT. 3.2.2 DAG Security Package Vector Informatik GmbH provides an OEM-specific function extension for CANoe, the DAG Security Package. Without the appropriate “VectorSecurityPackage_DAG_2.6.1.0 (Version: 2.6.1.0)” or newer versions of the VectorSecurityPackage, STAR3 tests cannot be executed. The DAG Security Package can be found on the Daimler Diagnostics Portal. Please carefully read through the readme file provided with the Vector Security Package as manual steps might be necessary (copying dll-files is needed). Please use at least CANoe 11 SP4 if Vector Security Package DAG 2.6.1.0 or higher is used. Page 11 of 44 Information security class: open 3.2.3 ZenZefi Since diagnostic rights management will be activated STAR3 ECU and partially STAR2 ECU tests can only be executed with installed ZenZefi tool. The ZenZefi tool can be found on the Daimler Diagnostics Portal. The required minimum version starting from March 24th 2021 is 1.80.0.10. A diagnostics certificate will be needed, see chapter 5.3. With the version 1.80.0.4 the authentication is done via OIDC. Therefore, from the PC where ZenZefi is running the server sso.daimler.com must be reachable via port 443. If required, please adjust your firewall settings. With the version 1.80.0.10 a required additional parameter for getting refresh tokens from the OIDC-Server is implemented. After the 23rd of March 2021 your previous ZenZefi-versions will not get refreshtokens from the OIDC server without this parameter. 3.3 Used Open Source Software The following Table 3 shows the corresponding open source software, which is used as part of the AKKA - Networking Test Suite. The full list of open source software can be found in the “Info” section of the tool. SW-Name File-Name Version License type Website SharpCompress SharpCompress.dll 0.24.0 MIT License (MIT) https://github.com/ adamhathcock/sha rpcompress MahApps.Metro MahApps.Metro.dll 1.6.5.1 MIT License (MIT) http://mahapps.co m/ ControlzEx ControlzEx.dll 3.0.2.4 MIT License (MIT) https://github.com/ ControlzEx/Control zEx Ninject Ninject.dll 3.3.4 Apache License Version 2.0 http://www.ninject. org/ Markdig Markdig.dll 0.17.1 BSD 2-Clause "Simplified" License https://github.com/l unet-io/markdig Markdig.Wpf Markdig.Wpf.dll 0.3.0 MIT License (MIT) https://github.com/ KryptosFR/markdig.wpf Table 3: Open source software Page 12 of 44 Information security class: open 4 Installation This chapter describes the installation procedure of the AKKA – Networking Test Suite. It also describes the requirements that need to be considered before using the tool. 4.1 General Requirements The software you are about to install requires an installed Vector CANoe (pro) version. The required version is specified in the Feature List or in chapter 3.2.1 of this user manual. Make sure, that the additional CANoe option (e. g. CANoe .FlexRay option) for the respective bus system is available. The AKKA – Networking Test Suite only runs with Microsoft .NET Framework 4.8, which has to be installed before using the tool. 4.2 System Requirements Processor: Intel Core i5 | 3.0 GHz (minimum: Intel Core 2 Duo | 2.6 GHz) Memory (RAM): 8 GB (minimum 2 GB) Hard disk space: 500 MB Operating System: Windows 7 / Windows 10 Display resolution: 1920 x 1080 (or higher) 4.3 Installation Procedure 1) Run the NTS-MSI-Installer-file to start the installation procedure. A window will appear with further instructions. These will guide you through the whole installation process. No admin rights are required. Figure 1: NTS Setup Wizard – Start of installation Page 13 of 44 Information security class: open 2) Confirm installation. Figure 2: NTS Setup Wizard – Confirm installation 3) After a successful installation, the following window appears (Figure 3). Before running the tests, a valid NTS license file must be loaded. Figure 3: NTS Setup Wizard – Complete installation Page 14 of 44 Information security class: open 4.3.1 License File To run tests on the AKKA – Networking Test Suite a valid license file is necessary. It can be obtained for one or for multiple ECUs, bus systems, vehicle lines and topologies. After the start of the tool, before running the tests, the license file has to be loaded. Switch to the license overview window (Figure 4) and import the available license file. It is possible to include several license files. Each license is listed with its validity period and additional information (e. g. ECUs, Licensee, Maintenance Expiry Date, Supported Versions, …). For maintenance prolongation you will need an updated license file. Figure 4: License file overview Licenses, which were provided before July 1st 2019, are no longer compatible with NTS versions v.3.27.0 or later. Page 15 of 44 Information security class: open 5 Test preparation and execution To be able to perform the tests, first the DUT has to be connected to the test software and the NTS has to be configured. 5.1 Preparing and connecting of the Device under Test (DUT) Each ECU interface which is about to be tested has to be connected with its voltage supply pins and its bus lines to the test hardware. The appropriate wiring harness is not part of the AKKA – Networking Test Suite. For each ECU to be tested, clamp 31 (ground) and clamp 15 (battery+ from ignition switch) and/or clamp 30 (battery+) have to be connected. Type of DUT clamp 15 clamp 30 clamp 31 Clamp-15-DUT yes - yes Clamp-30-DUT, wakeup via CAN - yes yes Clamp-30-DUT, wakeup via Clamp-15-pulse yes yes yes Table 4: Voltage supply variants Additionally, for every test run exactly one network interface, so to say one CAN-High- and one CANLow-cable, terminated with 120 Ω, has to be connected. In case of FlexRay FR-BP and FR-BM, terminated with 100 Ω shall be used. The network interface used for diagnostic services has to be connected, if it is not the same as the network interface which is about to be tested. The network interface for the wakeup has to be connected, if the DUT is not wake able over the bus which is about to be tested. In some cases, a secondary wakeup bus has to be connected. Afterwards, the configuration of the tests can be continued. 5.1.1 Ethernet To wake up a clamp-30-Ethernet-ECU a valid wake up pulse must be generated. For this purpose the Device I/O pins of the VH1150 / VH1160 are used. Please connect the VH1150 / VH1160 Device I/O via a D-SUB9 connector with the wake-up line of the ECU. The following pins are used (VH1150 / VH1160): - 5.1.2 Pin 3 = Ground (clamp 31) → connect to clamp 31 on the front side of the VH1150 / VH1160. Pin 9 = Analog output → connect to wake-up line of the ECU. LIN When testing LIN is necessary to use a ground offset LIN cable. For details please refer to the VH1150 / VH1160 user manual (chapter 3.5). The following pins are needed for the D-Sub 9 connector (male): - Pin 3 = Ground (clamp 31) → connect to clamp 31 on the front side of the VH1150 / VH1160. Pin 7 = LIN Signal → connect with LIN signal of the DUT. Pin 9 = VB+ (clamp 30) → connect to clamp 30 on the front side of the VH1150 / VH1160. Page 16 of 44 Information security class: open 5.2 Initiating and configuring of the NTS After the start of NTS.exe the user has to load an ECU Extract for the DUT which is about to be tested via the menu File → Open or via Drag and Drop. Then, the user can apply some fundamental settings in the tab General, e.g. specify in which folder the test reports shall be stored. Afterwards, he chooses the bus, which has to be tested, and subsequently chooses the hardware to which the bus is connected to. Moreover, a few test configurations need to be checked and adjusted, if necessary. - The Result Directory describes the path where the test results (e.g. test reports, trace files, …) are stored. The path is set by default but can be changed to a preferred place. It has to be chosen e.g. if the DUT is a clamp-30-DUT with network management support or a clamp-15-DUT without network management support. Terminal setting within the NTS Physical connection Test behaviour Terminal 30 with NM Connect clamp 30 and 31 from DUT to VH11xx Wakeup and keep awake are done via a bus specific procedure. Terminal 15 with NM Connect clamp 30, 31 and 15 from DUT to VH11xx Wakeup is done as described above. Keep awake is done via clamp 15 supply in addition to clamp 30 and 31. Terminal 15 without NM Connect clamp 30, 31 and 15 from DUT to VH11xx Wakeup is done via clamp 15 pulse. Keep awake is done via clamp 15 supply in addition to clamp 30 and 31. Table 5: Terminal settings in the NTS - - If the DUT is diagnosable, the diagnostic-frame-pair has to be checked. If several pairs were found in the ECU Extract, it cannot be distinguished automatically which pair is relevant for this network interface. The predefined values for the Powerdown Current Threshold (maximum current flow in the Powerdown-Mode) as well as the Powerdown Timeout (maximum time until the DUT must have switched into the Powerdown-Mode), shall be changed only in exceptional cases. After a click on the start button, a DUT-File is generated, which includes relevant information about the DUT extracted from the ECU-Extract. Subsequently, CANoe is started, the relevant test cases are preselected, and the test is started. The execution of the tests may take some time, but the progress can be seen in CANoe. Finally, a test report is generated automatically and is copied into the Result Directory folder, which can be set by the user. Thereby, a test run is completed and a new network interface of the DUT can be connected and tested. Page 17 of 44 Information security class: open 5.3 Install and use diagnosis certificates To authenticate against an ECU, you might need to own the corresponding diagnosis certificate. Please follow the steps below to get access to the certificate and to use it the correct way. 1) 2) 3) 4) 5) 6) Please log into the Daimler supplier portal. Request the required certificate in ACDC (please see glossary for explanation). After your request has been processed, log into ZenZefi to check your certificates. As final step, you need to open the AKKA – Networking Test Suite. Select the certificate as shown in Figure 5: Select diagnostic certificate for ECU. In case the NTS GUI was opened while using ZenZefi, you might need to use the refresh button. Figure 5: Select diagnostic certificate for ECU Page 18 of 44 Information security class: open 5.4 Configuration of diagnostic channel and bus system Depending on the ECU, diagnostic data such as the Hardware Part number can be read out from the ECU. The prerequisite is that the connected ECU unit supports diagnostics services. Then the NTS can be configured with the following steps. Selection of supported diagnostic channel: 1) 2) Select supported diagnostic channel and diagnostic request /- response. Select available Vector Network interface (the corresponding Vector Network interface has to be connected to the computer). Figure 6: Configuration of the diagnostic channel Page 19 of 44 Information security class: open Selection of supported diagnostic channel (with required FrTp file): In case of ECU Extracts with the AUTOSAR 3.1.4 format are used, it is necessary to load a FrTp (FlexRay-Transport Protocol) file. This file includes the diagnostic information of the ECU. 1) 2) 3) Select FrTp file, if it is not loaded automatically. Select supported diagnostic channel and diagnostic request /- response. Select available Vector Network interface (the corresponding Vector Network interface has to be connected to the computer). Figure 7: Configuration of the diagnostic channel (with required FrTp file) Page 20 of 44 Information security class: open Selection of the bus to be tested: 1) 2) 3) Select the bus, which is about to be tested. Select available Vector Network interface (the corresponding Vector Network interface has to be connected to the computer). Start the test with the configured settings. Figure 8: Selection of the bus to be tested and start of the test run Page 21 of 44 Information security class: open 6 Noteworthy features 6.1 NTS Headless Mode The NTS provides a headless mode option to enable the NTS usage within a continuous or automated testing environment. The headless mode consists of two parts: 1) Create a configuration file using the NTS UI 2) Perform the tests using a batch file providing the created configuration file The generation is done by the tool itself (user interface) as following: 1) Open NTS.exe: UI will appear. 2) Load the desired ECU Extract and configure the options as you wish: timeout, hardware, diagnostic certificate, secondary files (if necessary) ... 3) The application will check automatically if the configuration is suitable for the test. 4) Check the bus system that you want to test and press "alt + Start": a similar dialog as the example above will be shown. It is now possible to select the bus interfaces to be tested. A proper configuration should be selected for every bus interface that later is going to be tested. Figure 9: Selection of the bus interfaces to be stored in the configuration file 5) If you press OK button, next a File Dialog will appear in order to save the configuration file. Default name of this file is {EcuName}_Configuration.nts 6) Once confirmed, the configuration file will be generated and stored in the desired location. If there has been a failure during this process, it will be shown in the Log Viewer of NTS. The configuration file can be used for automated testing, see the following description. Page 22 of 44 Information security class: open Load the configuration into the application The main purpose of the configuration file is to contain all the user input, so no further user interaction is required to start a test. However, it is also possible to load the configuration file into the user interface. This will perform: 1) Load the corresponding database specified in the configuration file and load the options that you specified in the configured file. 2) Return the UI to the bus interface view in which the configuration file was generated. This can be useful if you just want to perform minor modifications on your configuration. Start headless mode: The headless mode is basically a console application which has the functionality of NTS. The backend code is exactly the same as the code contained in the user interface. This means that with the same inputs, the same CANoe Configuration and the same test is going to the performed. However, the executable has a different name: ➔ Setup Versions: %userprofile%\AppData\Local\AKKA DSW GmbH\AKKA - Networking Test Suite\NTS.Cli.exe The headless mode can be opened from the command window (cmd.exe and change directory where the executable is). Also, it is necessary to provide the file path of the configuration file. To do this, locate the configuration file using the file explorer, and then shift + right click and select "Copy as Path". This copies the whole path. To test, it is best to create a .bat file like the following: echo on "%userprofile%\AppData\Local\AKKA DSW GmbH\AKKA - Networking Test Suite\NTS.Cli.exe" run “%userprofile%\Documents\CLI\ECU_Configuration.nts" pause run command: run <filepath> [-reset] [-log <directory>] [-zuser <ZenZefiUserName> -zpassword <ZenZefiPassword>] [zlogout] Starts a NTS test run with the settings from the specified configuration file (.nts). The file path to the configuration file needs to be given for <filepath>. Options available: 1. - log: create a log file at the specified directory. 2. - zuser <ZenZefiUserName> - zpassword <ZenZefiPassword>: log in a custom ZenZefi user. 3. - zlogout: log out a custom ZenZefi user after the test run ended. 4. - reset: resets user settings, e.g. the recently opened databases. Page 23 of 44 Information security class: open Example: NTS.Cli.exe run %userprofile%\ECU_Configuration.nts -log %userprofile%\NTS-Log -zuser dummyUser -zpassword myZenZefiPw registercanoe command: registercanoe <canoeversion> Registers the specified CANoe version and unregisters all others. Admin rights are needed. A list of all installed versions can be obtained by executing the command “info -canoelist”. <canoeversion> must match exactly one entry of the obtained list of installed versions. Specify <canoeversion> in quotation marks. Example: NTS.Cli.exe registercanoe “Vector CANoe 12.0” info command: info [-canoelist] [-license] Shows content of the NTS Info View if no option is specified Options available: 1. - canoelist: Shows all CANoe versions installed. 2. - licenses: Shows all NTS licenses available. Example: NTS.Cli.exe info help command: help [-<command>] Shows all commands available. If a command is specified as option, help for this command is shown. Options available: 1. -<command>: Shows all options available for the specified command Example: NTS.Cli.exe help -run Page 24 of 44 Information security class: open Limitations: 1) The Headless Mode does not support DBC files at the moment. 2) The following limitation is relevant if you want to test more than one CAN, CANFD or LIN Master interface without manual user interaction. a. The NTS test setup uses the Vector relay box VH1150 or VH1160. For fault injection test cases the communication of the bus interface has to be routed via the relay box (see User Manual chapter 3.1.1). Therefore, the results for fault injection test cases (in case of Daimler test group "Stress Tests") are only valid for the bus interface which has been connected to the relay box. For further bus interfaces the results are not valid. b. Therefore, as the test results are not complete, they may not be used for distribution to the OEM (or only if the OEM gives his acceptance). Tests may be automated one by one always checking the hardware connections or manually. Page 25 of 44 Information security class: open 6.2 CANoe register/unregister The NTS offers the possibility to change the registration of the CANoe version to be used for testing. Admin rights on the computer may be needed for this action. To do this, navigate to “Tools” → “Vector CANoe Interop”, select the version to be used and click on “Register Component”. Figure 10: Register/unregister CANoe All versions except the one selected will be unregistered in the Windows registry. A deinstallation or whatsoever will not be triggered. After successful registration the NTS must be restarted for the new settings to take effect. Please check chapter 12.5 of this document for possible manual steps depending on the CANoe version currently registered for testing. Page 26 of 44 Information security class: open AddOn-Feature DWT (Dauerwecktest) – License extension required 6.3 6.3.1 Introduction Permanent wakeup tests (English term for Dauerwecktest, abbreviated by DWT) can be referred to as stress tests for sleep and wake up behaviour of electronic control units (in the following referred to as ECU). These electronic control units are specifically awoken after sending the last NM-Message with successively increased delay. This aims at guaranteeing a reliable bus wake up. Furthermore, the correct startup of ECUs from bus sleep mode is checked against defined time constraints. By focussing on the power down moment, bus communication blockers can be identified. The term bus blocker includes electronic control units which hinders bus sleep for unacceptable long duration. After testing single control units, a test report will summarize errors and time constraints deviations at a glance. In order to perform a permanent wakeup test please make sure that your license supports this feature. After importing the database for the ECU to be tested you can configure the test similar to a usual networking test, despite of different parameters. These parameters are explained on the user interface itself. 6.3.2 System Requirements Please see chapter 3 of this document for the system requirements. 6.3.3 Test preparation and execution After successfully loading a database file, you may apply a few general settings for the DWT which are described in the UI. These settings include: - Minimum wakeup delay: The lower end of the NM timer. - Maximum wakeup delay: The upper end of the NM timer. - Wakeup delay increment: The NM timer is incremented with this value for each test cycle. - NM Timer: Marks the time after a bus specific event before waking up the DUT - Sleep timeout: The DUT shall stop sending NM messages within this time. - Wakeup timeout: The DUT shall at least send one message after the tester’s wakeup within this time. When selecting the bus interface to be tested, simply select the Vector hardware to be used for testing and click on “Start”. The test parameters such as TNetworkStartup or NM payload are derived from the OEM specification and shall only be changed in justified cases. Please note that depending on the test settings, the test execution may take a lot of time. Page 27 of 44 Information security class: open 6.3.4 Step by step explanation of the Dauerwecktest The DWT follows a process which is more or less the same for each bus system: 1) ECU wakeup by a bus specific initial wakeup event → For CAN(FD): Send NM frames→ For FlexRay: Send Wakeup Pattern (WUP / WUDOP)→ For Ethernet: Trigger wakeup line→ For LIN Master: Wakeup event depending on the LIN backbone→ For LIN Slave: Send wakeup pulse 2) Wait for ECU response 3) a) For CAN(FD), FlexRay and Ethernet: The wakeup is considered successful if: → The initial wakeup event has been triggered AND → At least one NM frame was captured during the wakeup process (Tx or Rx) AND → The ECU replied to the initial wakeup event b) For LIN Master: The wakeup is considered successful if: → The initial wakeup event has been triggered on the backbone bus AND → The ECU started scheduling on the LIN bus under test c) For LIN Slave: The wakeup is considered successful if: → The initial wakeup event has been triggered on bus AND→ The ECU replied to the initial wakeup event AND → The ECU replied to the amount of headers sent d) For all bus systems: The wakeup is stopped if: → The wakeup timeout timer has expired (eg. if the ECU does not reply to the wakeup event). 6.3.5 Test result and test result verdict The test results can be found in the selected result directory. The result folder contains of the following files: - folder "traces" - DWT.log - [Test report].html - [Test report].xml The test result of the Permanent Wakeup Test is self-explanatory in general but in the following, a few details on how to read the test results can be found. The test result consists of the following sections: Timestamp Test step Description Result Timestamp of the entry which can be found in the trace file Test step such as precondition or the corresponding test cycle Detailed description of the result Verdict of the test step, such as pass, fail, warning Table 6: DWT test report structure Page 28 of 44 Information security class: open The combinations leading to a test verdict can be found in the following table: 0 <= TStartup measured <= TStartup max. TStartup max. < TStartup measured <∞ TStartup measured = ∞ (no wakeup) ECU did go to sleep (did stop sending NM messages and thus, resetting the delay timer) OK NOK NOK ECU did not go to sleep (did not stop sending NM messages and thus, resetting the delay timer) NOK NOK NOK Table 7: DWT test verdict possible scenarios 6.3.6 Excel evaluation Using “File” → “Generate DWT Excel evaluation”, you may select a test report xml-file found in the test result folder and an export path in order to generate an Excel evaluation. As a result, the Excel file will include the following data: • Delay time for each cycle [ms] • TStartup time for each cycle [ms] • Max. TStartup time [ms] • Did keep awake [0..1]: Shows if the ECU did go to sleep or if it didn't 6.3.7 Headless Mode You may use the DWT for headless testing, please follow the steps described in chapter 6.1 of this document. Page 29 of 44 Information security class: open 7 Ethernet testing with the NTS and different CANoe versions With CANoe 12 SP4 and Vector Driver >= 11.2 changes regarding Automotive Ethernet Testing and setup of VN device supporting Ethernet have been implemented as the new network-based mode has been added to the previously existing channel-based mode. Therefore, we want to give an overview in order to ensure the best experience when testing Ethernet ECU interfaces with the Networking Test Suite (NTS). What are the changes when using the NTS? To give you a good user experience and to limit manual steps we want to automize as much as possible. The NTS therefore configures the Vector Ethernet device and the CANoe configuration depending on the database file provided and on the software you have installed on your computer used for testing. Which software and hardware variants are compatible? The following table gives an overview on the Vector Ethernet devices currently and on the access mode support. The network-based Ethernet mode requires CANoe >= 12 SP4 and Vector Driver >=11.2. Vector Ethernet Interface VN5610 Access Mode support Channel-based mode Network-based mode VN5610A Channel-based mode Network-based mode VN5640 Channel-based mode Network-based mode VN5620 Channel-based mode Network-based mode ➔ Compatibility has not been tested yet with the NTS! VN5430 Channel-based mode Network-based mode ➔ Compatibility has not been tested yet with the NTS! VN8914 together with VN5620/VN5430 Not supported by the NTS, see chapter 3.1.3 Table 8: Access mode overview for Vector Ethernet interfaces Page 30 of 44 Information security class: open The following table gives an overview on the NTS handling with the possible software combinations: CANoe version >= 12 SP4 >= 12 SP4 >= 12 SP4 < 12 SP4 < 12 SP4 < 12 SP4 Driver setup package version >= 11.2 >= 11.2 < 11.2 >= 11.2 >= 11.2 < 11.2 Currently configured access mode of the VN Ethernet device networkbased channelbased channelbased networkbased channelbased channelbased Result NTS configures the VN Ethernet device using EDC. NTS changes the device to network mode and configures it using EDC. NTS cannot start test as CANoe >= 12 SP4 with networkbased mode is required. User needs to update the driver or switch to an earlier CANoe version. User needs to manually switch to channelbased mode or update CANoe to 12 SP4. NTS configures device using .vaset file. NTS configures device using .vaset file. → Automatic configuration → Automatic configuration → Automatic configuration → Automatic configuration → No automatic configuration possible → No automatic configuration possible Table 9: NTS handling for Ethernet access mode configuration Page 31 of 44 Information security class: open 8 Overview of specification documents according to bus system This list shows test and requirements specification documents according to bus systems. The tests are based on the following Daimler Specifications: - MBN 10415 (LIN 2.1 Networking Requirements) - MSS 10416 (CAN Networking Performance Specification) - MSS 10417 (FlexRay Networking Performance Specification) - MSS 10796 (Standard Security Specification) - MSS-T 10796 (Standard Security Architecture for Daimler Trucks & Busses) - MSS 10815 (Standard Security Architecture – Implementation Specification) - MSS 20200 (General Networking Performance Specification) - MSS-T 20200 (General Networking Performance Specification for Daimler Trucks & Busses) - MSS 20202 (CAN Networking Performance Specification) - MSS-T 20202 (CAN Networking Performance Specification for Daimler Trucks & Busses) - MSS 20204 (LIN Networking Performance Specification) - MSS-T 20204 (LIN Networking Performance Specification for Daimler Trucks & Busses) - MSS 20206 (FlexRay Networking Performance Specification) - MSS 20208 (Ethernet Networking Performance Specification) - MSS-T 20208 (Ethernet Networking Performance Specification for Daimler Trucks & Busses) - MSS 20220 (Networking Test Suite Specification) - MSS-T 20220 (Networking Test Suite Specification for Daimler Trucks & Busses) - MSS 20222 (CAN Networking Test Suite Specification) - MSS-T 20222 (CAN Networking Test Suite Specification for Daimler Trucks & Busses) - MSS-T 20222 (CAN SLP9 Networking Test Suite Specification for Daimler Trucks & Busses) - MSS 20224 (LIN Networking Test Suite Specification) - MSS 20226 (FlexRay Networking Test Suite Specification) - MSS 20228 (Ethernet Networking Test Suite Specification) - MSS-T 20228 (Ethernet Networking Test Suite Specification for Daimler Trucks & Busses) - QEV111AES5MSS (End-to-End Communication Protection) - LIN Specification Package Revision 2.2A (not part of Daimler Specifications) Page 32 of 44 Information security class: open The test results of the Networking Test Suite contain links to the corresponding Networking Test Suite Specification document. Further coverage, e.g. between requirements and test specification documents are managed by Daimler AG. The specification documents mentioned above can be retrieved from Daimler AG. Page 33 of 44 Information security class: open 9 Maintenance test environment For the realization of the tests the NTS uses functionality of CANoe. As part of the maintenance and enhancement of the test environment, the versions of the used software must be updated from time to time. The ability to run the NTS depends heavily on the operating environment. Therefore, only the SW versions specified in the Feature List by AKKA must be used. Otherwise, fault-free operation is not guaranteed. Page 34 of 44 Information security class: open 10 Maintenance NTS and support issues In order to give support, service and tool updates, a valid maintenance contract is needed. The NTS support can be contacted via GT-NTS@akka.eu. The validity of your maintenance contract can be checked in the “License” tab of NTS, see chapter 4.3.1. If you have any technical issues you can contact us at GT-NTS@akka.eu. When you do so, please provide the following information in order to avoid delays: In case of questions regarding test cases or CANoe: - Test report of the DUT → (html report) Trace file of the questioned test case(s) → blf file(s) NTS logfile → can be found in %userprofile%\NTS\Logs for the corresponding NTS version Operating system used In case of questions regarding the NTS tool: - NTS license used (eg. ECUName_vehicleLine_100xxx_2020.lic.xml) NTS logfile → can be found in %userprofile%\NTS\Logs for the corresponding NTS version Operating system used Page 35 of 44 Information security class: open 11 Technical specifications 11.1 Operating conditions The NTS is intended for the development environment in the laboratory. Use in vehicles is not provided. Page 36 of 44 Information security class: open 12 Frequently asked questions 12.1 No connection to test hardware VH1150 / VH1160 Question: During start of a test the message “Could not set up connection to test hardware VH1150.” (See Figure 11) appears although the VH1150 / VH1160 is connected properly to the VN8910A. Answer: Unplug the VH1150 / VH1160 from VN8910A, update Date and Time at the VN8910A (see Figure 12) and plug VH1150 / VH1160 again, when the update is finished. Ensure that the VH1150 / VH1160 is not connected to the VN8910A when updating driver, date, time, IP address or image of the VN8910A in future. Figure 11: Error message in the CANoe Write window – No connection to VH1150 / VH1160 Figure 12: Network hardware – Configuration of Date and Time Page 37 of 44 Information security class: open 12.2 CAPL DLL is invalid Question: During start of a test the message “Requested CAPL DLL ‘…’ is invalid.” (see Figure 13) appears. Answer: Unplug the VH1150 / VH1160 from VN8910A, update the CANoe Software at the VN8910A (see Figure 14), plug in the VN8910A again. Figure 13: Vector CANoe – CAPL DLL Figure 14: Network hardware – Update CANoe Software Page 38 of 44 Information security class: open 12.3 Invalid DUT file Question: During start of a test the message “Invalid DUT file!” appears. Answer: At least one parameter in the DUT file was changed and the test will not be executed. Close CANoe and restart the test execution from of the NTS. Figure 15: Invalid DUT file 12.4 Invalid channel assignment for network interface VN89xx Question: During start of a test the message “Measurement start was aborted due to an invalid channel assignment”. Answer: The usage of a VN89xx in conjunction with a further network interface (e.g. VN1630) is not supported by CANoe. Please connect the interface or interfaces of the DUT only to the VN89xx network interface. Depending on the supported interfaces of the VN89xx network interface the FlexRay-, CAN- and LINNetwork can be tested with this network interface. Figure 16: Invalid channel assignment for bus interface VN89xx 12.5 Wrong version of ZenZefiNodeLayer.dll Question: I have issues with different versions of ZenzefiNodeLayer.dll. Answer: With the VectorSecurityPackage v2.0.13 or later, there are different versions of ZenzefiNodeLayer.dll available. Please read carefully the instructions in the Readme.txt and follow the steps depending on the CANoe version used. The latest VectorSecurityPackage can be obtained via Daimler Diagnostic Portal. Page 39 of 44 Information security class: open 12.6 Vector VH11xx connected to the computer is not recognized by the NTS Question: My Vector VH11xx is connected to my computer, but it is not recognized by the NTS or the FTDI driver min. version is not met. Answer: You are likely missing the FTDI driver. The FTDI should be automatically installed on your PC when connecting the Vector VH1150 / VH1160 via USB (must be powered on). If this does not work, please check the following Vector website and download / install the FTDI driver. Please restart your computer after successful installation. The following steps could be also performed. Driver Check: - - - When a VH11xx Hardware is connected to the PC a COM/LPT Device "USB Serial Port (COMx)" must be visible in the device manager of windows. Using right click → Properties → Details to get the current driver version installed on the PC. The relevant dynamic link library can be found under the path "%windir%\SysWOW64\ftd2xx.dll" (32-Bit is used by the CANoe Runtime) or "%windir%\system32\ftd2xx.dll" (used by other tools). If the DLL exists, the Product Version (right click "Properties" on the file) of the DLL contains the desired information. Driver Update/Installation: Depending on the Windows default installation it can be necessary to install the FTDI driver manually. The driver can be downloaded on https://www.ftdichip.com/Drivers/VCP.htm. 1) Download the setup package using the link which can be found in the comment column of the table ("Available as a setup executable"). 2) Disconnect the USB Connection between the VH11xx Hardware and the PC. 3) Follow the setup wizard through the installation. 4) Restart your computer after successful installation. 5) Reconnect the VH11xx Hardware again. 6) (Additional) It may take a while until the driver installation is done after reconnection, especially when drivers are searched using windows update. 7) Use the NTS to verify the installation. Alternatively, use the Driver Check described above to verify the installation. Change COM-Assignment: If the steps above did not solve the issue please assign the VH11xx to a different COM-port using the Windows device manager. Page 40 of 44 Information security class: open 13 Appendix 13.1 Abbreviations Abbreviation Explanation BRS-bit Bit Rate Switch-bit, if set switching to a higher data rate CAN Controller Area Network CAN FD Controller Area Network Flexible Data Rate CAN_H CAN-High CAN_L CAN-Low DUT Device Under Test – ECU DWT Dauerwecktest ECU Electronic Control Unit FrTp FlexRay-Transport Protocol HW Hardware LIN Local Interconnect Network NM Network Management NTS Networking Test Suite PDU Protocol Data Unit – data item, summary of signals SW Software UDS Unified Diagnostic Services – diagnosis service Table 10: Abbreviations Page 41 of 44 Information security class: open 13.2 Glossary Term Explanation ACDC Authorization Center for Diagnostics and Coding Application-PDU PDU which contains application data. Clamp 15 Clamp name in the vehicle: switched plus of the ignition-starter switch. Clamp 30 Positive cable direct from the battery. CYCLIC Transfer-type cyclical, independent of signal values. CYCLIC_ON_CHANGE Transfer-type cyclical, if signal values are unchanged in comparison to previous transfer; immediately, if signal values have changed in comparison to previous transfer. Diagnostic Request Request of a diagnosis service to a DUT. Diagnostic Response Response of the DUT to a Diagnostic Request. DUAL_CYCLE Transfer-type slowly cyclical, if signal values correspond to the default value; quick cyclical, if signal values don´t correspond to the default value. ECU Extract Description of all networks connected to a DUT from the view of a DUT, also described as database. Errorframe Special frame of the CAN-BUS to signalize incorrect transfer. Hard Reset Diagnostic request, to restart the DUT immediately. Last-Is-Best Collection-semantics, a PDU is allowed to exist in one PDUContainer at most once (older PDU is rejected). NM PDU PDU which contains network management data. Power down-Modus Energy-efficient mode of the DUT in which no communication takes place. QUEUED Collection-semantics, a PDU is allowed to exist multiple times in a Container-PDU. VH1150 / VH1160 Relay box of Vector Informatik with the possibility to switch the supply voltage of the DUT, to measure the current flow and short-circuit the CAN-cables and interrupt them. VN8900 Modular FlexRay/CAN Network-Interface of Vector Informatik which enables the access to the buses which have to be tested in connection with VN8910A and VN8970. Page 42 of 44 Information security class: open VN8910A Basic device to VN8900 with Intel ATOM Processor. VN8970 Plug-in module to VN8900, which enables the access to CAN FD as well as the generation of Error frames. Page 43 of 44 Information security class: open 14 Legal notices 14.1 Copyright notice (1) This document and the contents are protected by copyright. Without written consent by AKKA Industry Consulting GmbH the following are not permitted: changes, abbreviations, extensions or additions; and/or publications or translations; and/or commercial use, even for educational purposes. (2) Reproduction is permitted for personal use only and only under the condition that this copyright notice is retained on the reproduced document itself. (3) The following restrictions apply for the software product described in this user documentation or the respective offer. Unless expressly agreed upon otherwise in writing, - there is no entitlement to the publication of the source code. - It is not permissible to decompile, disassemble, or reverse engineer the software product. - It is not permissible to publish, distribute, modify, or alter the software product or produce derivative software products in part or whole in a different way. 14.1.1 General legal notice (1) The use of the software products described in this documentation and the respective offer may lead to risks and is to be undertaken with the necessary precaution. (2) The software product described in this documentation enables the control and influence of mechanical and/or electric/electronic systems. The software product is released only for the express purpose stated in the respective offer or this documentation. In particular, it is not suitable for uses where there is a risk of damage to persons or property. In addition, it is also not suitable for systems or hardware or software environments other than those described in this documentation or the respective offer. For this reason, the software product may be used only by persons who are specially trained in the use of the software product and who have understood the need for the necessary precautions for the use of the software product. Page 44 of 44 Information security class: open