SAP AG
E2E BI Housekeeping
Document Version 1.4 - August 2014
5/16/2013
Introduction:
A central housekeeping process (Report E2E_HK_CONTROLLER) is running on a daily basis on SAP
Solution Manager ABAP1.
Per default, the E2E BI Housekeeping includes the following tasks:
-
Aggregation of data, i.e. data is transferred from info cubes with hourly resolution to those with
only daily resolution2
Deletion of data, i.e. data from the higher volume cubes (H-Cubes) is deleted if it is older than a
specified period of days (default is 91 days)
Update Statistics3
Compress Infocube4
Delete Requests5
With ST710 a Twin Cube concept is introduced, e.g. for statistical records data and other performance
data, as well as for the Monitoring and Alerting infrastructure. This Twin Cube concept is not handled by
E2E BI Housekeeping. However, data in the “old” E2E Diagnostics Infocubes are still handled.
Logs of E2E BI Housekeeping can be found in the spool log of report E2E_HK_CONTROLLER as well as in
the Application Log (TA SLG1) in the BI Client.6
Deletion of Data:
The life time (storage time) of metrics stored into the E2E Diagnostic Infocubes is configurable7.
Currently, the following info cubes are supported:
Info Cube
0CCMARCH
0CCMARSH
0CCMSAEXE
0CCMSATPH
0SMD_ADCH
0SMD_ADCD
0SMD_EA1H
0SMD_EA1D
0SMD_EA2H
0SMD_EA2D
0SMD_EA3H
0SMD_EA3D
0SMD_EA4H
1
SAP Solution Manager 7.0 EHP1 (ST400)
X
X
X
X
X
X
X
X
X
X
X
X
X
SAP Solution Manager 7.1 (ST710)
X
X
X
X
X
X
X
X
X
X
X
X
X
For proper functionality, please make sure that SAP notes 1266915, 1254977, 1510178 and 1510411 are applied.
With ST400, aggregation of data to an hourly resolution is only supported for 0SMD_* infocubes
3
Available with ST-BCO 7.10 (ST710)
4
Available with ST-BCO 7.10 (ST710)
5
Available with ST-BCO 7.10 (ST710)
6
Available with ST400 SP24. Object/Subobject is BW_PROCESS/REPORT (ext. number 'E2E_HK_CONTROLLER')
7
Please be aware that the configuration might be overwritten in case that a new SP is deployed to SolMan.
2
0SMD_EA4D
0SMD_EA5H
0SMD_EA5D
0SMD_KACH
0SMD_KACD
0SMD_PE2H
0SMD_PE2D
0SMD_PI2H
0SMD_PI2D
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Configuration of Data Deletion is done via the Configuration Table E2E_BI_DELETE. There you can specify
multiple periodic “Deletors”. A Deletor is a set of selection criteria which are the basis of a deletion.
The following examples show the recommended default configuration settings for info cubes
0SMD_PE2H and 0CCMARCH8.
TARGET_CUBE:
The relevant Info Cube from which the data shall be deleted.
BIDESTINATION:
The RFC Destination to the BI Client (used by the Extractor Framework); only
required for a remote BI or if BI client and SolMan client differ.
DELETOR_ID:
Entries with the same DELETOR_ID (and the same target cube / BI Destination)
will be combined (with “AND”) to a set of selection criteria and therefore
represent one “Deletor”.
INFOOBJECT:
The characteristic which is used to select the relevant data for deletion.
KEYFL:
Used internally. Need to bet set to ‘X’.
Via the selection criteria (SIGN, SELOPTION, LOW, HIGH), the select option for the maintained info object
is to be specified. Thereby, any select option can be applied to any info object contained in the relevant
info cube.
Please note, that there is a default runtime substitution for characteristic 0CALDAY. The Placeholder
$DELTA=N$ will replace at runtime the selection criterion so that the parameter LOW is calculated to be
“<Current Day> - N” Thus, $DELTA=N$ selects any data older than N days.
In the example above, data older than 91 days will be deleted from the info cube 0SMD_PE2H, while
data older than 61 days will be deleted from info cube 0CCMARCH9.
8
Please note, that BIDESTINATION needs to be adopted.
Selective Deletion: How-To
Prerequisites:
-
Check cubes that show excessive growth
-
Consider if any particular selection can be used to delete data
In the following Example only data for a particular system are to be deleted from an info cube.
Procedure:
-
Start Transaction SE16, Select Table E2E_BI_DELETE, Select the cube you want to delete data from.
-
Select one entry as a template by clicking on the line  Menu “Table Entry”  “Create with
Template”
-
Choose a Deletor ID that has not yet been assigned (e.g. 2). This will be the ID of the new Deletor you
are going to create.
9
Please note that for info cubes 0CCM* a Deletor needs to be specified via 0CALDAY and 0CCMPTYP (I EQ DAY).
Both selection criteria need to have the same Deletor ID.
-
The first selection statement in the new Deletor should be for 0CALDAY. For the new Deletor ID,
select the allowed lifetime by reducing the value in $DELTA=N$
Example: Assume you want to delete data for a specific system after 30 days  set the Parameter in
LOW to $DELTA=30$
-
Save the first entry of your new Deletor.
-
Create a second entry for the new deletor for the second characteristic you want to select
In this example, for choosing the system, enter 0SMD_LSID as characteristic under INFOOBJECT.
-
Make sure that you use the same Deletor ID you used in the previous step
-
Enter the required selection criteria. For selecting one particular system, enter the System Name as
LOW Parameter and choose SIGN and SELOPTION appropriately, i.e. the Select Option should be
“EQ” to include the value under “LOW” into the selection. Select the SIGN I for including the
selection statement. The value helps (F4) of the fields SIGN and SELOPTION will tell you more about
the available parameters and options.
-
Save the second line of the new Deletor.
Verify that for your new deletor, the entries in the table E2E_BI_DELETE look like this10:
Selective Deletion: D-Cubes (0SMD_*D)
If you notice, that some or all of the E2E Diagnostic Day-Cubes (0SMD_*D) exhibit excessive growth, you
can follow the steps above and create a Deletor which, e.g., deletes all data older than one year:
10
Eventually BIDESTINATION needs to be adapted.
Recommended settings for SAP Solution Manager 7.0 EHP1 (ST400)1112
Housekeeping of BI Hashtable
Since there is a length restriction of 60 characters in BI for values stored in a characteristic, a hash
algorithm is implemented, when e.g. storing data for the E2E Exception Analysis. The Hash Table in use is
SMD_HASH_TABLE (in BI client).
In some cases, IDs or timestamps cannot be removed from an error message text (see also SAP note
1333571). This leads to unique hash entries, which can result in a strongly growing hash table, e.g., as a
consequence of some massive java based error occurrence in the managed systems.
There are two different ways of housekeeping possible.
Option 1: General Housekeeping (Recommended)
Housekeeping is enabled via a Deletor in table E2E_BI_DELETE. For Hash entry deletion, please specify
the relevant be (field TARGET_CUBE) and leave the fields INFOOBJECT, SIGN, SELOPTION, LOW, HIGH,
SIDLOW, SIDHIGH empty, see screen shot.
Please note, that in contrast to the storage time configuration described above, the ranges here keep
empty. The allowed lifetime for Hash entries is controlled via the column ALIFETIME13. ACOUNT is to be
set to a very high value.
11
D-Cubes can be included according to the procedure shown above.
BIDESTINATION needs to be adapted.
13
For ALIFETIME, the same logic as above applies, i.e. via the DELTA-parameter the maximum lifetime is to be
specified.
12
Please note, that hash entry deletion will also affect aggregated data. As a result, e.g. message details
will not be available in application E2E Exception Analysis for data older than the here specified value.
Option 2: Delete only the most prominent entries
In order to identify, the most prominent entries for a too large grown Hash Table only, please use
Transaction TAANA14 and follow the following procedure:
-
Start Table Analysis for table SMD_HASH_TABLE and select the value help for Analysis Variant to
generate an Ad-Hoc Variant:
-
Select the following parameter for Analysis:
14
in BI Client in case of remote BI
-
Execute the Variant in online mode:
-
Identify which combination of METRIC_TYPE / METRIC_NAME has the largest number of entries with
LCOUNT < 315
15
In SMD_HASH_TABLE, the parameter LCOUNT is the reuse counter. The lower LCOUNT, the more unique the
hash entry is. In addition, please note, that here Metric Type contains the Target Cube, while Metric Name contains
the Infoobject.
Enable Hash Table Housekeeping for the parameters detected via a Deletor in table E2E_BI_DELETE. For
Hash entry deletion, please specify the relevant infocube (displayed under METRIC_TYPE) and the
relevant infoobject (displayed under METRIC_NAME).
Please note, that in contrast to the storage time configuration described above, the ranges here keep
empty. The allowed lifetime for Hash entries is controlled via the column ALIFETIME16. ACOUNT is
equivalent to LCOUNT in the analysis above. Usually you focus on values 1 ≤ LCOUNT ≤ 3.
Warning: Depending on the data volume, choosing this option might result in potential performance
issues during housekeeping.
Compress Infocube, Delete Requests17, Update Statistics18
This Housekeeping tasks can be enabled via entries in table E2E_BI_HOUSEKEEP. There you can specify
the relevant Infoprovider, the BI Destination as well as the requested task.
16
For ALIFETIME, the same logic as above applies, i.e. via the DELTA-parameter the maximum lifetime is to be
specified.
17
Please refer to SAP note 1178655 before using this feature.
18
FM ‘RSDU_ANALYZE_INFOCUBE’ is executed for the selected infocube.
MS SQL and Oracle and Sybase ASE: In case of long query runtimes a compression of the relevant
infocubes might be necessary. Furthermore, a compression of the requests of the E2E cubes is
recommended on a regular basis. Please refer to SAP note 1178655 for details. In addition, please note
that the maximum number of partitions on SQL Server 2005 is 1,000.19
The “compress cube” is only evaluated for the source InfoCubes of the pair of infocubes that is part of
the aggregation (Table=E2E_BI_AGGREGATE records that fit to Aggregator=E2E_HK_AGGREGATE) . The
names of these infocubes follow the naming convention 0SMD_*H. The algorithm tries to compress all
requests, except those which are not yet aggregated.
For all other infocubes the housekeeper will not trigger the compression.
Since selective deletion done via configuration in DB table E2E_BI_DELETE might take a long time, an
additional request based deletion of data is offered. When using this feature, all requests of the relevant
infocube which are older than the specified number of days in field “REQUEST_INTERVAL” are deleted.
Please note that deletion will be done independently on whether data is already aggregated or not.
During deletion the load behavior of the infocube is switched.
Since the compress functionality removes the request information that is needed for the delete request
operation, you can use either the “compress cube” or the “delete requests” functionality.
19
See SAP note 1010854 for details.