Data Mining with CANape 9.0 Analysis of Measurement Data © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V 1.15 2011-04-01 Offline Evaluation and Data Mining Motivation: To look for a needle in a haystack Problem: A huge amount of measurement files A certain search criterion Goal: Find the subset of measurement files in which the search criterion is fulfilled and at which points in time the criterion are fulfilled Example: In which files has the motor temperature risen above a certain level? © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 2 Offline Evaluation and Data Mining Manual Analyzing CANape offers a wide range of possibilities to analyze measurement data manually. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 3 Offline Evaluation and Data Mining Analyzing with mathematical functions CANape offers a wide range of functions in the internal library to analyze measurement data automatically User can add own function libraries, written in C or generated with Real-Time Workshop out of Simulink models to do the data mining CANape analyzes a lot of different measurement formats CANape / CANgraph* Measurement data Function Library MDF, DAT, BLF, ASC, CLF, CSV, ATF, XLG, … * CANgraph is a subset of CANape and contains the same offline analyzing possibilities C-Functions and Simulink-models DLLs © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 4 Offline Evaluation and Data Mining Data Mining Workflow Open one measurement file in a graphic window Select the appropriate functions and define the hit condition Analyze one measurement file exemplarily in a graphic window Analysis completed? Select amount of measurement files Run the analysis steps over all measurement files Start Data Mining Control hit list report Hit list report allows the convenient manual analysis of the affected files © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 5 CANgraph vs. CANape CANgraph is a subset of CANape with the use case: Analyze measurement data! CANgraph has the same features like CANape All features described in this presentation are available in CANgraph, too. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 6 Agenda Offline Evaluation and Data Mining 2 CANgraph vs. CANape 6 > Manual Offline Analysis CANape/CANgraph Functions 29 Manual Offline Analysis with Functions 36 Automatic Offline Analysis 45 © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 8 7 Manual Offline Analysis MDF Shell Extension © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 8 Display MDF file comments in the Windows Explorer Manual Offline Analysis Open measurement files Three different ways to open measurement files Open measurement file and visualize the content directly in a new Graphic Window Open measurement file and select signal and windows individually Open measurement file and visualize the content in the existing configuration © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 9 Manual Offline Analysis Open measurement files Open measurement file and visualize the content in a new Graphic Window Out of the Windows Explorer per Drag & Drop into the open CANape Out of the Analysis menu © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 10 Manual Offline Analysis Open measurement files Open measurement file and select signal and windows individually Out of the Symbol Explorer with submenu Select signals and use Drag & Drop to add them into existing or new windows © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 11 Manual Offline Analysis Open measurement files Open measurement file and visualize the content in the existing configuration © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 12 Manual Offline Analysis Open measurement files Replace loaded measurement files © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 13 Manual Offline Analysis Manage measurement files Manage the measurement files in CANape © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 14 Manual Offline Analysis The ”Select data file”-Window shows you the comments of the measurement documents (in the tool tip and in the preview section) to simplify the file selection. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 15 Manual Offline Analysis Select the signals you are interested in © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 16 Manual Offline Analysis The selected signals are visualized in a graphic window © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 17 Manual Offline Analysis Open the Symbol Explorer for a comfortable access to loaded measurement files, function definitions, virtual signals, and data mining results, etc. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 18 Manual Offline Analysis Use the Symbol Explorer for accessing signals from measurement files Find the signals with the search functionality Use the columns to visualize additional information of the signals Select the signals with Drag&Drop and put them into new or existing windows © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 19 Manual Offline Analysis Multiple Graphic Window © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 20 Manual Offline Analysis Color functions Color functions can be used in measurement windows to display signals, curves and maps © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 21 Manual Offline Analysis Offline user comments Various offline comment settings allow to mark critical measurement points with a certain color and display relevant information, including macros for: date and time value differences time differences gradient © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 22 Manual Offline Analysis Add columns to the legend With a double click, the user may add information columns to the Graphic Window E.g., the values and the gradients are shown regarding the position of the measurement cursor (much more information is available) © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 23 Manual Offline Analysis Visualize envelopes in the Graphic Window Speed up the loading time of a measurement data file and see outliers directly (even in measurement data that lasts for hours) 5 h test drive © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 24 Manual Offline Analysis Move signals with mouse in Graphic Window With the right mouse button, all signals can be moved inside the graphic window to shift the signals to your convenient position Horizontal: Insert time offset Vertical: Change Y position © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 25 Manual Offline Analysis Change scaling of X-, Y-, and XY-axes by mouse wheel Change the scaling of the axes with mouse wheel scrolling (depending on the X-, Y-, XY-mode) Changes at the X-axis (time) are applied to every signal in the window Changes at the Y-axis are only applied to the selected signals Change the mode of the window by pressing the buttons X, Y or Z for the XY mode or use these icons © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 26 The red signal is selected Manual Offline Analysis Measurement File Manager List of all loaded measurement files within the configuration Features: Clear view of measurement file contents: Signal list including detailed information Replace loaded measurement files (serial analysis) Edit measurement file - comments (offline) © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 51 27 Agenda Offline Evaluation and Data Mining 2 CANgraph vs. CANape 6 Manual Offline Analysis 8 > CANape/CANgraph Functions Manual Offline Analysis with Functions 36 Automatic Offline Analysis 45 © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 29 28 CANape/CANgraph Functions Editor Overview Debugger compile List view Editor window Tree view Message window © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 30 CANape/CANgraph Functions Internal functions The popup menu of the editor window offers an extensive library: Control structures Mathematical functions Evaluation functions Diagnostics functions File functions … Online help via function key F1 © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 31 CANape/CANgraph Functions Creating a new function function Function_1 () { // Enter the code here return ; } Function name Parameter list function My_Function (signal1, signal2) { write( " Signalvalue_1= %.2f Signalvalue_2= %.2f“ ,signal1,signal2); return signal1 + signal2; } C language elements can be used within functions, for example ifelse control structures, arithmetic operators, etc. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 32 CANape/CANgraph Functions Variable types Local variables: Only valid within the function For functions of type ‘Static‘ (created only one time) function My_Function_1 ( ) { double a=0, z=0; // Static Var z= a; if (z > gMaximumValue)//Global Var { z=0; a=0;} a=a+1; return z; Adjustment in function properties dialog possible Global variables: Interface between functions and scripts Definition in the function editor Initialization via script Initialization with a calibration window } gMaximumValue=20; © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 33 CANape/CANgraph Functions Function Library With CANape / CANgraph a growing number of functions is available in a kind of library Just use and change them according to your needs Add your own functions to the library © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 34 Agenda Offline Evaluation and Data Mining 2 CANgraph vs. CANape 6 Manual Offline Analysis 8 CANape/CANgraph Functions > Manual Offline Analysis with Functions Automatic Offline Analysis © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 35 29 36 45 Manual Offline Analysis with Functions Insert virtual file channel based on MATLAB/Simulink DLL Map file signals to DLL ports © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 37 Manual Offline Analysis with Functions Insert virtual file channel based on defined functions In the Symbol Explorer is a new topic “Function Definitions” The Library functions are part of CANape The Project functions are individually generated functions of the project © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 38 Manual Offline Analysis with Functions Use of a Library or Project Function Select a function with the mouse and use Drag & Drop to put the function on top of a signal in the legend A Submenu informs you about the possible input ports of the function. In this example, there is only one input port Select the input port The function “Check_greater_3000” will calculate a virtual signal with the “EngineRPM” as the input signal © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 39 Manual Offline Analysis with Functions The virtual signal with the name of the function is calculated and visualized like any other signal in the window In this example, the function searches for a threshold of 3000 of the input signal “EngineRPM”. The virtual signal shows the finding in this measurement file. If the value of the input signal ≥ 3000 the value of the virtual function is ‘1’. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 40 Manual Offline Analysis with Functions Adjust local parameters in the function properties dialog Show initialization values of local static variables of functions, allow to change these values for a function instance Functions can be used multiple times in the same configuration with individual local parameters Comment syntax: //! © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 41 Manual Offline Analysis with Functions Use functions in a calculation chain Calculated results of functions can directly be used as an input for the next function Channel 1 -> Function “LowPassFilter” -> Function “MeanAverage” -> Function “Derivation” Use Drag&Drop as well inside the window © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 42 Manual Offline Analysis with Functions With the new columns in the graphic windows the “Input parameters” and the “Output parameters” of functions and Simulink DLLs can be shown. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 43 Agenda Offline Evaluation and Data Mining 2 CANgraph vs. CANape 6 Manual Offline Analysis 8 CANape/CANgraph Functions 29 Manual Offline Analysis with Functions 36 > Automatic Offline Analysis © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 44 45 Automatic Offline Analysis Data Mining Workflow Open one measurement file in a graphic window Select the appropriate functions and define the hit condition Analyze one measurement file exemplarily in a graphic window Analysis completed? Select amount of measurement files Run the analysis steps over all measurement files Start Data Mining Control Hit List Report Hit-List report allows the convenient manual analysis of the affected files © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 45 Automatic Offline Analysis Overview Up to now: data visualization of one measurement file Measurement data CANape / CANgraph MDF, DAT, BLF, ASC, CLF, CSV, ATF, XLG C-Functions and Simulink-models Since CANape 8.0 / CANgraph 8.2: automated analysis of several measurement files Tons of measurement files CANape / CANgraph System of Rules Function Library MDF, DAT, BLF, ASC, MDF, DAT, CLF, CSV, BLF, ASC, MDF, DAT, ATF, XLG CLF, CSV, BLF, ASC, MDF, DAT, ATF, XLG CLF, CSV, BLF, ASC, ATF, XLG CLF, CSV, ATF, XLG Report & Results C-Functions and Simulink-models © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 46 Automatic Offline Analysis Up to now: visualization of one virtual file channel Example: Search for a special constellation of signals © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 47 Automatic Offline Analysis Since CANape 8.0: automated analysis Definition of the MDF-Files for the analysis Selection of the CANape function or Matlab.DLL Test condition of the return value Filename for the result Data Mining results Evaluation via the Symbol Explorer © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 48 Automatic Offline Analysis Configuration of the analysis function Link to the input objects, exemplarily for all measurement files which have to be checked © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 49 Automatic Offline Analysis Configuration of the Condition (Return value) Definition of the test criterion Threshold value of the return value of a CANape function Threshold value of the output variable of a Matlab.Dll © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 50 Offline Evaluation and Data Mining Boolean Conditions Boolean conditions provide a convenient way to define a complex evaluation method without the need to write a evaluation function. Enter the Boolean Condition with a “pocket calculator” © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 51 Offline Evaluation and Data Mining Boolean Conditions Define the condition like: (Condition_1 && Condition_2 || Condition_3) || testdword0 > 100 Test the conditions © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 52 Offline Evaluation and Data Mining New Data Mining Options with Scripts Select files by date of creation and use additional scripts Define add. scripts that should be executed before/after analyzing file etc. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 53 Offline Evaluation and Data Mining New Data Mining Options with Scripts Control the data mining with scripts. E.g.: Find file and the point of time with the maximum value of a signal Stop the method if there are invalid values Stop the analyzing of the file, if a signal is missing © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 54 Automatic Offline Analysis Set Options Set Options, define scripts that should run before, after, ... © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 55 Automatic Offline Analysis Results Start Analysis Results list with *.dmr – report files Listing of the measurement files containing hits © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 56 Automatic Offline Analysis Verification of the results using the Symbol Explorer The methods are offered as virtual file channels in the Symbol Explorer Display them with Drag&Drop With a double-click on the HIT the according file with the time position will be shown. © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 57 Thank you for your attention. For detailed information about Vector and our products please have a look at: www.vector.com Authors: Sophia.Stahl@vector.com Andreas.Patzer@vector.com Vector Informatik GmbH Ingersheimer Str. 24 70499 Stuttgart © 2011. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. Slide: 58