Best Practices to Administrate, Operate, and Monitor an SAP HANA System Dr. Bjarne Berg COMERIT Produced by Wellesley Information Services, LLC, publisher of SAPinsider. © 2015 Wellesley Information Services. All rights reserved. What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 1 Introduction – Dr. Berg 2 Key Responsibilities for HANA Admin and Installs Area Task Hardware installation and health check Linux OS installation HANA platform installation Initial Setup Data source connectivity Adding DB instances (MCOS) SMD agent installation HANA DB admin Third party software installations HANA system monitoring Operations HANA DB monitoring Backup and recovery "Bare metal" recovery Firmware patching Linux OS upgrades and patching Maintenance Peripheral components patching HANA platform components updates & patching Support Issue resolution process Hardware Customer SAP vendor x x x x x x x x x x x x (x)* x (x)* x x x (x)* x x * depending on support contract 3 Updating the SAP HANA Appliance • SAP hardware partners ship SAP HANA pre-configured and with the most recent support package stack at the time when the SAP HANA appliance is shipped • The customer is responsible for the subsequent implementation of SAP HANA patches, revisions, or support packages, and support package stacks • Systems that were installed with the SAP HANA Unified Installer can use the automated update procedure • This requires a backup be completed, data replication to be suspended, and the business made aware of the planned outage It is recommended that you do this on a quarterly basis, or when other systems are scheduled for maintenance at the same time (i.e., holidays) 4 Updating the SAP HANA Appliance and SUM • The Software Update Manager (SUM) for SAP HANA Support Package Stack (SPS) can execute automatic updates of the Lifecycle Management perspective as part of self-update • Because it is not part of the base SAP HANA install, unless the hardware partner installed SUM as part of the install, you have to first install it from the SAP Marketplace before you can use it • During install, it is important to note that all archives, including SUMFORHANA, must be located in the same directory as the stack.xml file • After the SUM is installed, you can choose to apply either SPSs that contain larger upgrades, or individual support packages based on your needs or upgrade schedule Additional details can be found in SAP note: 1545815 5 Updating SAP HANA Studio • • You can choose to update the software automatically based on periodic updates with SUM or execute the software update manually The Lifecycle Management perspective of the SAP HANA Studio is updated when you update SAP HANA Studio To update the SAP HANA studio, choose Help > Check for Updates 6 Updating SAP HANA Studio • • • • You can enter the settings for the automated update of SAP HANA Studio under Windows> Preferences> Install/Update The site you add should be in the following format: file:////update_server/hdbstudio/repository/ or http://<host_name>:<port_number>/tools/hdb. studio.update There are also numerous options regarding Automatic Updates in HANA Studio Make sure your SAP HANA Studio contains the Lifecycle Management perspective 7 Licensing • • The two types of license keys for SAP HANA are: Temporary keys (typically 90 days) Permanent keys To check your type of license keys and expiration dates, right-click on a system in the Navigator pane in Studio, select Properties, and choose Licenses More information on monitoring of license keys is found in SAP Note 1704499 8 Licensing • License Keys can be enforced or unenforced. This means that if you have enforced keys, the SAP HANA system will shut down if you try to use the system for more memory than you are licensed for (SAP grants a little extra memory consumption in before shutting down) • If the system is shut down due to a license key violation, you cannot access the system via queries nor can the system be backed up. To see if your keys are enforced or not, take a look inside the license file. If you see “SWPRODUCTNAME=SAP-HANA,” your keys are not enforced. If you see “SWPRODUCTNAME=SAP-HANA-ENF,” your license keys are enforced. • Changes to license keys can only be made by administrator with the system privilege LICENSE ADMIN in the security role. 9 License Audit by SAP • If you have installed permanent SAP HANA licensing keys, SAP may periodically request a license audit • If requested, you simply open SAP HANA Studio, click on your database and select PROPERTIES on the context menu. • From here you click on the EXPORT SYSTEM MEASUREMENTS button and save the XML file. You will email this XML file to SAP as part of the license audit. You need to have the system privilege LICENSE ADMIN assigned to have access to this function 10 What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 11 HANA Editions and Components • While HANA is sold as an appliance, there are many internal components and the edition you buy may contain different licenses to these components Area Lifecycle Management Component ID Component Name BC-DB-HDB SAP HANA database BC-DB-HDB-ENG SAP HANA database engine BC-DB-HDB-PER SAP HANA database persistence BC-DB-HDB-SYS SAP HANA database interface BC-DB-HDB-DBA SAP HANA database/DBA cockpit BC-DB-HDB-POR SAP HANA DB Porting BC-DB-HDB-BAC SAP HANA Backup and Recovery BC-CCM-HAG SAP Host agent BC-DB-HDB-CCM SAP HANA CCMS BC-DB-HDB-CLI SAP HANA Clients (JDBC/ODBC) BC-DB-HDB-R SAP HANA Integration with R BC-DB-HDB-SCR SAP HANA SQL scripts BC-DB-HDB-MDX MDX engine: Microsoft Excel client BC-HAN-MOD SAP HANA Studio - Information Modeler BC-HAN-3DM Information Composer SAP Landscape Transformation (SLT): trigger-based BC-HAN-SRC SAP HANA UI toolkit Sybase Replication Server: log-based BC-DB-HDB-TXT SAP HANA Text and Search features BI-BIP-CMC, BI-BIP BI Platform BC-DB-HDB-DXC SAP HANA Direct extraction connector BI-RA-WBI Web Intelligence BC-DB-HDB-SEC SAP HANA Security and User Mgmt BI-RA-XL Dashboard Designer BC-DB-HDB-XS SAP HANA Application Services BI-RA-CR, BI-BIP-CRS SAP Crystal reports BC-DB-HDB-AFL SAP HANA Advanced functions library BI-RA-EXP SAP BusinessObjects Explorer BC-DB-HDB-AFL-PAL SAP HANA Predictive analysis library BI-BIP-IDT Information Design Tool (for universes) BC-DB-HDB-AFL-SOP SAP HANA Sales & Operations Planning BI-RA-AO-XLA Microsoft Excel add-in BC-DB-HDB-PLE Component ID Component Name BC-HAN-SL-STP SAP HANA unified installer BC-HAN-UPD Software Update Manager BC-DB-HDB-INS SAP HANA database installation BC-DB-HDB-UPG SAP HANA database upgrade BC-HAN-DXC SAP HANA Direct Extractor Connection Enterprise Edition EIM-DS BC-HAN-LOA (also have platform edition components) BC-HAN-LTR BC-HAN-REP End User Clients Area SAP Data Services: ETL-based SAP HANA Load Controller: log-based Platform Edition SAP HANA Planning Engine 12 HANA Release Strategy and Names • • As of 2015, SAP introduced the idea of “production verified revisions” to provide in-depth testing of all services packs for SAP HANA Based on the planned releases over the next 12 months, customers should adjust their plans for service packs accordingly 13 Sizing a BW system for HANA • Using the BW Automated Sizing tool in the Migration Cockpit 14 SAP BW on HANA Sizing Tool for Existing BW Implementations • SAP has released an updated tool that generates a report for sizing SAP BW. • This program takes into consideration existing database, table types, and includes the effects of non-active data on the HANA system To increase speed, you can suppress analysis tables with less than 1 MB size The higher precision you run the estimate at, the longer the program is going to run This program is also referenced in SAP Notes 1909597 and 1736976 on the Service Marketplace 15 The Sizing Result Since timeouts are common when running the sizing program, you can temporarily change the parameter in rdisp/max_wprun_time to 0 in BW transaction RZ11. Finally, you estimate the growth for the system as a percentage or as absolute growth. The output is stored in the file you specified and the file can now be emailed to hardware vendors for sizing input and hardware selection 16 Sizing for BusinessSuite on HANA • SAP also have programs to size the system for BusinessSuite on HANA • In this example from July 2015, we see that a system of 1.992 GB is required to migrate the ECC 6 box to HANA 17 Main Hardware Options • It is worth noting that IBM is also working on certification by SAP for their Power severs (POWER 8 and E870) and have posted some great performance benchmarks on SCN together with SAP. • IBM may therefore, depending on project timing, also be a viable candidate for hardware. 18 What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 19 Key Resources for Monitoring HANA System Area Tool HANA Administration guide System Admin Purpose - How to use the HANA cockpit and HANA studio for system admin. - Core functions of high-availability, disaster recovery & scalability tinyurl.com/AdminHana - Security administration - How to manage and monitor applications for data provisioning and custom applications built in the extended services (XS) framework. Tool to manage system landscape connections and central management of DB configurations HANA Troubleshooting and How to trouble shoot and fix DB performance issues and guidance Performance Analysis Guide on general optimization. How to monitor, setup and manage systems that have HANA Multitenant DB Guide multitenant DBs Landscape Technical Operations Manual How to operate and administrate a HANA landscape. Admin SAP DB Control Center (DCC) Guide on how to use DCC to monitor HANA and other databases DBA Cockpit for HANA Web Resource tinyurl.com/DBACockpit tinyurl.com/TroubleGuide tinyurl.com/HanaDBs tinyurl.com/TechOperations tinyurl.com/databaseCC 20 Monitoring with Admin Console in HANA Studio 21 System Landscape • The Landscape tab verifies that the system is running and displays the status of the relevant services: • For each server within the HANA system, the following services should be running: • • • nameserver indexserver preprocessor • • • statisticsserver sapstartsrv xsengine If a distributed system is being used, this can be configured under the Configuration subtab 22 Monitoring with HANA Cockpit in Fiori 23 Monitoring with The DBA Cockpit 24 Monitoring with SAP Landscape and Virtualization Manager (LVM) 25 Monitoring with Alerts • The statistics server is the tool used for monitoring in SAP HANA and offers real-time system resource alerts on vital information. There are 74 Alerts available. • Server crashes or stoppages • Hard disk reaching critical capacity • CPU at risk of experiencing bottlenecks or high stress 26 Configuring Alerts • Customer Alerts can be created to assist in monitoring system performance in the Administrator Editor under the Alerts tab • Creating an administrative e-mail account is recommended in order to isolate system monitoring information • The recipients of alerts can be optionally modified to inform those who should receive alert notifications instead of targeting different alerts to specific email addresses • Each alert has three specific thresholds for when the alert can be executed: High, Medium, Low The values for these thresholds can be defined as percentages. The scheduled times for when the alerts should be triggered can also be set, the default is every six hours once a day. 27 Monitoring Availability with Alerts Check Type Availability ID Time Description 0 Intra-day Identifies internal statistics server problem. 3 Intra-day Identifies inactive services. 4 Intra-day Restarted Services- services that have restarted since the last time of the check. 21 Daily 22 Intra-day 23 Intra-day 24 Intra-day 31 Daily 41 Daily 70 Periodic 78 Daily 80 As needed Identifies internal DB events. SAP Recommended Admin Action Resolve the problem. For more information, see the trace files. You may need to activate tracing first. Investigate why the service is inactive, for example, by checking the service's trace files. Investigate why the service had to restart or be restarted, for example, by checking service's trace files. Resolve the event and then mark it as resolved by executing the SQL statement ALTER SYSTEM SET EVENT HANDLED '<host>:<port>' <id>. Notification of all alerts- if any alerts since the last check is triggered Notification of medium and high priority alerts- since the last check is triggered Notification of high priority alerts- since the last check is triggered License expiry-If the disks to which data and log files are written are full. A disk-full event causes DB to stop These alerts can trigger email blasts to specified recipients. Investigate the alerts. Obtain a valid license and install it. For the expiration date, see the monitoring view M_LICENSE. In-memory DataStore activation- If a problem with the activation of an inmemory DataStore object exists Consistency of internal system components after system upgrade For more information, see the table Connection between systems in system replication setup- closed connections between primary/ secondary system. If connections are closed, the primary system is no longer being replicated. Investigate why connections are closed (i.e., network problem) and resolve the issue. Availability of asynchronous table replication- Monitors error messages related to asynch table replication. Determine which tables encountered the table replication error using system view M_ASYNCHRONOUS_TABLE_REPLICAS, and check the corresponding indexserver alert traces. _SYS_STATISTICS.GLOBAL_DEC_EXTRACTOR_STATUS and SAP Note 1665553. Contact SAP support. 28 Monitoring Backups with Alerts Check Type Back-up ID Time 28 Periodic 32 Periodic 33 Periodic 35 36 Daily Daily 37 Daily 38 Daily 54 Periodic Savepoint duration- Identifies long-running savepoint operations. Check disk I/O performance. 65 As needed Runtime of the log backups currently running- If the most recent log backup terminates in the given time. Investigate why the log backup runs for too long, and resolve the issue. 66 As needed 69 Periodic Storage snapshot is prepared- if the period, during the DB is prepared for a storage snapshot, exceeds threshold. Enablement of automatic log backup- if automatic log backup is enabled. Investigate why the storage snapshot was not confirmed or abandoned, and resolve the issue. Enable automatic log backup. For more details please see SAP HANA Administration Guide. Number of log segments- segments in the log volume of each service Check for number of log segments. Make sure that log backups are being auto created and that there is enough space Check whether the system has been frequently and unusually restarting services. If it has, then resolve the root cause of this issue and create log backups as soon as possible. 72 Daily Description SAP Recommended Admin Action Most recent savepoint operation- How long ago the last savepoint Investigate why there was a delay defining the last savepoint and consider triggering was defined, that is, how long ago a complete, consistent image of the operation manually by executing the SQL statement ALTER SYSTEM SAVEPOINT. the DB was persisted to disk. If you need point-in-time recovery, reconfigure the log mode of your system to "normal". In the "persistence" section of the global.ini configuration file, set the Log mode LEGACY- If the DB is running in log mode "legacy". Log mode "legacy" does not support point-in-recovery and is not parameter "log_mode" to "normal" for the System layer. When you change the log recommended for productive systems. mode, you must restart the DB system to activate the changes. It is also recommended that you perform a full data backup. Log mode OVERWRITE- If the DB is running in log mode "overwrite". Log mode "overwrite" does not support point-inInvestigate why the service had to restart or be restarted, for example, by checking recovery (only recovery to data backup) and is not recommended service's trace files. for prod systems. Existence of data backup Perform a data backup as soon as possible. Status of most recent data backup Investigate why failed, resolve the problem, and perform a new data backup as soon as possible. Age of most recent successful data backup Status of most recent log backups- If the most recent log backups Perform a data backup as soon as possible. Investigate why the log backup failed and resolve the problem. for services and volumes were successful. 29 Monitoring Configuration and CPU with Alerts Check Type ID Time 3 As needed Discrepancy between host server times- discrepancies in a scale-out system. 10 Periodic Delta merge (mergdog) configuration- If the 'active' parameter in the 'mergedog' section of system configuration file(s) is 'yes'. 16 Periodic Lock wait timeout configuration- if 'lock_waittimeout' parameter in 'transaction' section of indexserver.ini file is between 100,000 and 7,200,000. 26 Periodic Unassigned volumes- Identifies volumes that are not assigned a service. 34 Daily 79 Periodic 5 Intra-day Configur ation CPU Description If all volumes are available. Configuration consistency of systems in system replication setup- Identifies configuration parameters that do not have the same value on the primary system and a secondary system. Host CPU Usage- Determines the % CPU idle time on the host and therefore if CPU resources are running low. SAP Recommended Admin Action Check operating system time settings. mergedog is the system process that periodically checks column tables to determine if a delta merge operation needs to be executed. Change in SYSTEM layer the parameter active in section(s) mergedog to yes In the 'transaction' section of the indexserver.ini file, set the 'lock_wait_timeout' parameter to a value between 100,000 and 7,200,000 for the System layer. Investigate why the service had to restart or be restarted, for example, by checking service's trace files. Investigate why the volume is not assigned a service. I.e.., assigned service is not active, the removal of a host failed, or the service removal was performed incorrectly. Investigate why the volume is not available. The identified configuration parameter(s) should have the same value in both systems, adjust the configuration. If different values are acceptable, add the parameter(s) as an exception in global.ini/[inifile_checker]. Investigate CPU usage 30 Monitoring Files and Disk Usage with Alerts Check Type ID Time 46 As needed 50 Periodic 51 SAP Recommended Admin Action RTEdump files- Identifies new runtime dump files (*rtedump*) have been generated in the trace directory. These files These contain information about, for example, build, loaded modules, running threads, CPU, etc..Check contents of the dump files. Number of diagnosis files- written by the system (excluding zip-files). A large number of files can indicate a problem with the DB (i.e., problem with trace file rotation or a high number of crashes). Investigate the diagnosis files. Daily Size of diagnosis files- very large file sizes can indicate a problem with DB. Check the diagnosis files in the SAP HANA studio for details. 52 Daily Crashdump files- new files that have been generated in the trace directory 53 Daily Pagedump files- new files that have been generated in the trace directory 56 Periodic Python trace activity- If trace is active and for how long. Trace affects performance. If no longer required, deactivate the python trace in the relevant configuration file. 2 Intra-day Disk Usage- Determines what % of each disk containing data, log, and trace files is used. This includes space used by non-SAP HANA files. Investigate disk usage of processes. Increase disk space, for example by shrinking volumes, deleting diagnosis files, or adding additional storage. Resolve the disk-full event: In the Admin Editor on the Overview tab, choose the \"Disk Full Events\" link and mark the event as handled. Alternatively, execute the SQL statements ALTER SYSTEM SET EVENT ACKNOWLEDGED '<host>:<port>' <id> and ALTER SYSTEM SET EVENT HANDLED '<host>:<port>'<id>. Diag-nosis Files Description Check the contents of the dump files. Disk 30 Intra-day Check internal disk full event- If the disks to which data and log files are written are full. A disk-full event causes your DB to stop and must be resolved. 60 Periodic Sync/Async read ratio- Identifies a bad trigger asynchronous read ratio. 61 Periodic Sync/Async write ratio- Identifies a bad trigger asynchronous write ratio. This means that asynchronous reads are blocking and behave almost like synchronous reads. This might have negative impact on SAP HANA I/O performance in certain scenarios. Note 1930979. 77 Intra-day DB disk usage- The total used disk space of the DB. All data, logs, traces and backups are considered. Investigate the disk usage of the DB. See system view M_DISK_USAGE for more details. 31 Monitoring Memory Usage • Memory in SAP HANA is consumed for a variety of purposes: • The operating systems and support files • Proprietary code and stack of program files • Column and row stores where data is stored • Working space where computations occur, temporary results are stored, and shared user memory consumption occurs • SAP HANA tracks memory from the perspective of the host. The most important aspects are the following: Physical memory – The max amount of physical (system) memory available on the host Allocated memory – The memory pool reserved by HANA from the operating system Used memory – The amount of memory from th4 pool that is actually used by HANA DB 32 Monitoring Memory Usage • • • • The physical memory on most SAP HANA hosts is from 256 GB - 2 TB This is used to run the Linux OS, SAP HANA, and any additional programs that run on the host SQL statements can be used to obtain or edit memory information. There is a set of predefined SQL statements provided by SAP that are available for use Used memory serves the following purposes: • Program code and stack Working space and data tables (heap and shared memory) The program code area houses the SAP HANA database while it is active. Various parts of SAP HANA can share a common program code. The stack is required to complete actual computations 33 Monitoring Memory with Alerts Check Type Memory ID Time Description 1 Intra-day 3 Periodic 12 Intra-day 17 Periodic 20 Periodic Table growth rate of non-partitioned column-store table 27 Periodic Record count of column-store table partitions 29 Periodic Size of delta storage of column-store tables Investigate the delta merge history in the monitoring view M_DELTA_MERGE_STATISTICS. Consider merging the table delta manually. 40 Daily Total memory usage of column-store tables- The % of the effective alloc limit being consumed by individual column-store tables as a whole This is the cumulative size of all of a table's columns and internal structures. Consider partitioning or repartitioning the table. 43 Daily Memory usage of services- % of effective alloc limit a service is using. Check for services that consume a lot of memory. 44 Periodic Licensed memory usage- % used. Increase licensed amount of main memory. See the peak memory allocation since installation in the system view M_LICENSE, column PRODUCT_USAGE 45 Periodic Memory usage of main storage of column-store tables- % of effective alloc limit consumed by column-store tables. Consider partitioning or repartitioning the table. 55 Periodic Columnstore unloads- # of columns that have been unloaded from memory. Can indicate performance issues. Check sizing with respect to data distribution. 58 As needed 67 Periodic Table growth of rowstore tables Increase the size of the plan cache. In the 'sql' section of the indexserver.ini file, increase the value of the 'plan_cache_size' parameter. Reduce the size by removing unused data 68 Periodic Total memory usage of row store used by a service Investigate memory usage by row store tables and consider cleanup of unused data 73 Periodic Overflow ratio of rowstore version space. 74 Periodic Overflow ratio of metadata version space. 75 Periodic Rowstore version space skew- if rowstore version chain is too long. 81 Periodic Cached view size- how much memory is occupied by cached view Host physical memory usage- The % of total physical memory available on the host Row store fragmentation Memory usage of name server- Determines what % of allocated shared memory is being used by the name server on a host. Record count of non-partitioned column-store tables- Current table size is not critical. Plan cache size- if the plan cache is too small. SAP Recommended Admin Action All processes consuming memory are considered, including non-SAP HANA processes. Investigate memory usage of processes. Implement SAP Note 1813245. Increase the shared memory size of the name server. In the 'topology' section of the nameserver.ini file, increase the value of the 'size' parameter. Partitioning need only be considered if tables are expected to grow rapidly. A non-partitioned table cannot contain more than 2,000,000,000 (2 billion) rows). Consider partitioning the table only if you expect it to grow rapidly. Identify the connection or transaction that is blocking version garbage collection. You can do this in the SAP HANA studio by executing the "MVCC Blocker Connection" and "MVCC Blocker Transaction" statements available on the System Information tab of the Administration editor. If possible, kill the blocking connection or transaction. Increase size of the cached view. In the "view_cache" section of the indexserver.ini file, increase the value of the "total_size" parameter. 34 Monitoring Security, Sessions and Transactions with Alerts Check Type ID Time 57 Daily Session & Transactions System Secure store file system (SSFS) consistency regarding the DB User passwords- Identifies DB users whose password is due to expire with the PW policy. If it expires, the user will be locked. This may impact application availability. Granting of SAP_INTERNAL_HANA_SUPPORT role- if the internal support role is currently granted to any DB users. Total memory usage of table-based audit log- % of the effective allocation limit is being consumed by the DB table used for table-based audit logging. SAP Recommended Admin Action Check and make sure that the secure storage file system (SSFS) is accessible and consistent regarding the DB. Change password of the DB user. It is recommended that you disable the password lifetime check of technical users so that their password never expires (ALTER USER <username> DISABLE PASSWORD LIFETIME). 62 Daily 63 Daily 64 Periodic 25 Daily Open connections- % of the max number of permitted SQL connections The max number of permitted connections is configured in the "session" section of the open. indexserver.ini file.Investigate why max number is being approached. 39 Daily Long-running SQL statements 42 As needed Security Sessions Description Check if the corresponding users still need the role. If not, revoke the role from them. Consider exporting the content of the table and then truncating the table. Investigate the statement. For more info, see table _SYS_STATISTICS.HOST_LONG_RUNNING_STATEMENTS. Long-idling cursors 47 Periodic Long-running serializable transactions 48 Periodic Long-running uncommitted write transactions 49 Periodic 59 Daily 83 Daily Long-running blocking situations Percentage of blocked transactions Table consistency- the number of table consistency errors and affected tables Close cursor, uncommitted transaction, or the serializable transaction in the application, kill connection, or by executing the SQL statement ALTER SYSTEM DISCONNECT SESSION <LOGICAL_CONNECTION_ID>. For more information, see the tables HOST_LONG_IDLE_CURSOR, HOST_LONG_SERIALIZABLE_TRANSACTION and HOST_UNCOMMITTED_WRITE_TRANSACTION (_SYS_STATISTICS). Investigate the blocking and blocked transactions and if appropriate cancel one of them. Contact SAP support 35 More System Information in HANA Studio 36 Server Performance Information • It is possible to monitor more detailed aspects of system performance on the Performance tab in order to detect and fix performance issues. • In the Thread view you can end the operation of a specific thread Since multiple threads run together in one session and in one transaction, the operations of all subsequent threads belonging to that session/transaction will also be terminated. 37 Managing Large Tables with Partitioning • When column tables grow containing high data volumes, it would be advantageous to split them “horizontally” into smaller partitions • SAP HANA automatically manages the partitions in the background which simplifies the access and frontend development and gives the administrator a key tool to manage disks, memory, and large column stores • In a distributed (scale-out) SAP HANA system, it is possible to place the partitions on different nodes and thereby increase performance exponentially due to more processors being available for the users • In a partitioned schema, it is possible to have 2 billion rows per partition with virtually no limit on how many partitions can be added • As a result, this becomes a matter of hardware and landscape architecture as opposed to a question of database limitation 38 Managing Large Tables with Partitioning • • There are three different ways of creating partitions from an administration standpoint in SAP HANA: By ranges By hash By round-robin While more complex schemas are possible with multilevel partitioning, these three options cover the basics used in the higher level options. In addition to these options, you application layer may offer additional software options depending on the application you are running on top of HANA 39 Partitioning Column Tables by Range • If data familiarity is acute, data can be partitioned by any range in a table • The most common partition is by date, though it is possible to use material numbers, postal codes, customer numbers, or anything else • Partitioning by date increases query speed and limits data to a single node • The maintenance of range partitions is somewhat higher than the other options since new partitions must be constantly added as data outside the existing partitions emerge, as is the case with time sensitive data Example of partitioning by SQL: CREATE COLUMN TABLE SALES (sales_order INT, customer_number INT, quantity INT, PRIMARY KEY (sales_order)) PARTITION BY RANGE (sales_order) (PARTITION 1 <=values < 100000000, PARTITION 100000000 <== values <200000000, PARTITION OTHERS) 40 Partitioning Column Tables by Hash • • • • Partitioning column stores by the hash does not require an in-depth knowledge of the data Instead, partitions are created by an internal algorithm applied to one or more fields in the database by the system itself. This is known as a hash The records are then assigned to the required partitions based on this internal hash number The partitions can be created in SQL with defined rules such as the following: If the table has a primary key, it must be included in the hash If more than one column is added, and the table has a primary key, all fields used to partition on must be part of the primary key If the number of partitions is not defined, the system will determine the optimal number of partitions based on the configuration. As a result, this is the recommended setting for most hash partitions Example of partitioning by SQL: CREATE COLUMN TABLE SALES (sales_order INT, customer_number INT, quantity INT, PRIMARY KEY (sales_order, customer_number)) PARTITION BY HASH(sales_order, customer_number) PARTITIONS 6 41 Partitioning Column Tables by Round-Robin • • • In a round-robin partition, the system assigns records to the partitions on a rotating basis While it makes for efficient assignments and requires no data familiarity, it also means that removing partitions in the future will be more challenging as both new and old data will be present in the same partitions The following syntax can be used in SQL to create the partitions: CREATE COLUMN TABLE SALES (sales order INT, customer number INT, quantity INT) PARTITION BY ROUNDROBIN PARTITIONS 6 In this example, six partitions are being created and records are assigned on a rotating basis. If the last statement is changed to PARTITIONS GET_NUM_SERVERS(), the system will assign the optimal number of partitions based on the system landscape. The only requirement is that the table does not contain a primary key. 42 Moving Files and Partitions for Load Balancing • Periodically moving files and file partitions allow column tables to achieve better load balancing across hosts and are useful for adding or removing a node from the system, creating new partitions, and load balancing existing ones that have grown very large • Before initiating this process, save the current distributions using the RESOURCE ADMIN system privilege for recovery later in the event of an error • From the Table Distribution Editor the catalog, schemas, and tables can be viewed • A table can be moved to another location by right-clicking it and selecting Move Table. A similar process can be used for moving partitions to consolidating partitions to single hosts • If a “disk full” event is triggered it will be display on alerts and will suspend the use of the database. You can find information in Volumes tab, and if it is full due to other temporary files being stores, they may be deleted. The event is then marked as “handled” in the Overview tab ceasing the suspension of the database 43 What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 44 Security Authentication • • • SAP HANA has two forms for authentication security Internal Authentication Users are created in SAP HANA database only Authentication is handled by SAP HANA database via username/password External User Repositories Kerberos or Security Assertion Markup Language (SAML) Once authenticated, users are then check for authorization privileges Database users can have the following types of privilege: Direct Privileges Inherited Privileges When Kerberos is used, the users in the key distribution center should be mapped to the database users in SAP HANA by making user’s principal name the external ID. 45 Overview of Privilege Types • • Package privilege Package privileges allow access to and the ability to work in packages in the repository of the SAP HANA DB Packages contain design time versions of various objects, such as analytic views, attribute views, calculation views, and analytic privileges Application privilege Developers of SAP HANA XS applications can create application privileges to authorize user and client access to their application. Application privileges are granted and revoked through the procedures GRANT_APPLICATION_PRIVILEGE and REVOKE_APPLICATION_PRIVILEGE procedure in the _SYS_REPO schema Application privileges can be granted directly to users or roles in run time in the SAP HANA studio. It is recommended that you grant application privileges to roles created in the repository in design time 46 Privileges on users Privileges on users are SQL privileges that users can grant on their user. ATTACH DEBUGGER is the only privilege that can be granted on a user For example, User A can grant User B the privilege ATTTACH DEBUGGER to allow User B debug SQLScript code in User A's session. User A is only user who can grant this privilege 47 Roles Management • • Adding Roles Go to the NAVIGATOR pane in Studio, and select the system you want to grant access to Select the CATALOG folder, and then the AUTHORIZATION folder Right-click on the ROLES folder, and select NEW ROLE Deleting Roles Go to the NAVIGATOR pane in Studio, and select the system you want to grant access to Select the CATALOG folder, and then the AUTHORIZATION folder Expand the Roles folder and right-click on the ROLE and select DELETE 48 Standard Roles • CONTENT_ADMIN This role contains all the privileges required for using the information modeler in the SAP HANA studio, as well the additional authorization to grant these privileges to other users. It also contains system privileges for working with imported objects in the SAP HANA repository • MODELING This role contains all the privileges required for the information modeler in SAP HANA studio It therefore provides a modeler with the database authorization required to create all kinds of views and analytic privileges The MODELING role contains the standard analytic privilege _SYS_BI_CP_ALL. This analytic privilege potentially allows a user to access all the data in all activated views, regardless of any other analytic privileges that apply. The CONTENT_ADMIN role is very privileged and should not be granted to users, particularly in production systems. The CONTENT_ADMIN role should only be used as a template. 49 Standard Roles • MONITORING This role contains privileges for full read-only access to all metadata, the current system status in system and monitoring views, and the data collected by the statistics server • RESTRICTED_USER_ODBC_ACCESS This role contains the privileges required by restricted database users to connect to SAP HANA through the ODBC client interface This role is intended to be used in conjunction with application-specific roles It is recommended that the privileges required to use an application are encapsulated within an application-specific role, which is then granted to restricted database users. 50 Standard Roles • PUBLIC This role contains privileges for filtered read-only access to the system views. Only objects for which the users have access rights are visible. By default, this role is granted to every user, except restricted users • SAP_INTERNAL_HANA_SUPPORT This role contains system privileges and object privileges that allow access to certain low-level internal system views needed by SAP HANA development support in support situations. All access is read only This role does not allow access to any customer data. 51 Users Management Adding Users To add users, go to the NAVIGATOR pane in Studio, and select the system you want to grant access to Select the CATALOG folder, and the AUTHORIZATION folder Right-click on the USERS folder, and select NEW USER Deleting Users To delete users, go to the NAVIGATOR pane in Studio, and select the system impacted Select the CATALOG folder, and select AUTHORIZATION folder Choose the USERS folder, and select the user to be deleted 52 Users Management Deactivating Users To deactivate users, go to the NAVIGATOR pane in Studio, and select the system impacted Select the CATALOG folder, and then select the AUTHORIZATION folder Choose the USERS folder, and select the user to be deactivated Activating Users To activate users, go to the NAVIGATOR pane in Studio, and select the system impacted Select the CATALOG folder, and then select the AUTHORIZATION folder Choose the USERS folder, and select the user to be activated 53 Users Management • Emergency User IF the SYSTEM user is deactivated and can no longer connect to the SAP HANA database You can verify that this is the case in the USERS system view. For user SYSTEM, check the values in the columns USER_DEACTIVATED, DEACTIVATION_TIME, and LAST_SUCCESSFUL_CONNECT You can still use the SYSTEM user as an emergency user even if it has been deactivated. Any user with the system privilege USER ADMIN can reactivate SYSTEM with the statement ALTER USER SYSTEM ACTIVATE USER NOW. To ensure that an administrator does not do this casualy, we recommended that you create an audit policy monitoring ALTER USER statements. 54 Security Password Policy • You can also set your own password policy for SAP HANA which includes the different password rules: minimum password length use of characters max number of log-on attempts blacklisted passwords password expiration notifications 55 Changing Password Policy To change a password policy, right-click on the SAP HANA system in the NAVIGATOR pane and select OPEN SECURITY Under the PASSWORD POLICY tab you can change all the settings to conform to your company’s password rules 56 What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 57 Backup and Standby • Supports synchronous backup between production system and backup storage Alerts can be setup to monitor backups and two primary backup methods exists: • There are 4 basepath options for traditional file backups in HANA Studio: • Traditional File BACKINT API for third party vendors Basepath data backup – Standard backups to external mount point Basepath data volumes – Permanent location for data volumes Basepath log backup – External mount point for logs segment to be copied every 15 minutes Basepath log volumes – Permanent location for log volumes IBM offers a backup management solution called Tivoli Storage Manager and SAP provides a script in SAP Note 1651055 to help clean up log files If log files become too large, longer backup times may result 58 SAP HANA designed with High Availability Supports recovery measures ranging from faults and software errors to disasters that decommission an entire data center Provides the ability to rapidly resume operations after a system outage with minimal business loss (fault resilience) Offers a service auto-restart functionality which automatically detects the failure and restarts the stopped service process Allows the assignment of up to 3 master servers as the name server in case the active master name server fails, the system can restore itself to the available standby master The number of standby servers defined during installation cannot subsequently be reduced without major work. However, standby servers can be added after installation. 59 High Availability and Fault Tolerance • High Availability configuration N active servers in one cluster M standby server(s) in one cluster Shared file system for all servers • Failover Server X fails Server N+1 reads indexes from shared storage and connects to logical connection of server X 60 Scale out – Standby Server Configuration SAP HANA cold standby host Standby host is kept ready for the event that a failover situation occurs during production operation Standby host is not used for database processing All the database processes run on the standby host, but they are idle and do not allow SQL connections 61 What We’ll Cover • • • • • • Licensing and Update Maintenance Hardware Options and Sizing Performance Monitoring and Load Balancing Managing Roles, Privileges, and Security High Availability, Disaster Recovery and Backup Planning Wrap up 62 Where to Find More Information • www.sap-press.com/sap-hana_3687/ • www.amazon.com/SAP-BW-HANA-Migration-Handbook/dp/150852761X/ • SAP’s main page for all SAP HANA-related information www.saphana.com/community/try • Bjarne Berg, Rob Frye and Joe Darlak: BW to HANA migration handbook www.saphana.com/welcome • Bjarne Berg and Penny Silvia, SAP HANA: An introduction (SAP PRESS, 3rd Edition). SAP HANA Marketplace http://scn.sap.com/community/bw-hana SAP BW powered by SAP HANA on SCN 63 7 Key Points to Take Home • The Software Update Manager (SUM) for SAP HANA Support Package Stack (SPS) can execute automatic updates of the Lifecycle management perspective as part of self-update • Make sure that you know the type of key the system is using to insure that the SAP HANA system will not shut down • There is a System Monitoring option within HANA that provides useful overview information to help prevent potential problems • Managing user roles within HANA system can be done through a simple process • SAP HANA supports synchronous backup between production system and backup storage • An system admin can set up password policy within HANA • SAP HANA is designed with Support for High Availability 64 Your Turn! How to contact me: Dr. Berg bberg@comerit.com 65 Disclaimer SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP SE. 66