25 Critical Lessons for Teams That Are Running Global SAP NetWeaver BI Projects Dr. Bjarne Berg © 2007 Wellesley Information Services. All rights reserved. What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 2 Introduction • • • • SAP NetWeaver BI (BI) has matured substantially over the last decade In this session we will explore the lessons learned from a variety of companies, looking at real examples We will pay particular attention to design principles, performance enhancements, and explore what you can learn from other’s mistakes We will also look at BI trends and what others are doing in the SAP NetWeaver BI space 3 BI Has “Grown Up” and Is Now a Core Infrastructure Requirement • 59% of companies now have an enterprise data warehouse • This number will grow to 86% by 2009 However, only 39% of companies have a BI competency center and only 54% have an enterprise information strategy BI competency center BI governance program No plans Within 2009 Currently have Enterprise Info. strategy Information quality program Enterprise Data Warehouse 0% 10% 20% 30% 40% 50% 60% Source: Business Week Research, Sept. 2006 Where will your company be in three years? 4 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 5 Global Master Data Integration and SAP NetWeaver BI • When an organization has multiple SAP R/3 systems, the SAP NetWeaver BI master data integration can be cumbersome 1. Determine what local master data needs to be maintained in the BI system and load it as separate master data 2. Determine what global master data needs to be maintained in the BI system and load it to the shared areas To do this you will need master data mapping rules or tables 3. If you have conflicting master data, consider loading the local masterdata to DSOs (make it reportable) and merge the global master data to be used by the InfoCubes Alternatively, you can load versions of master data into custom master data objects in SAP NetWeaver BI and use the merged master data as the “0 objects” If you have multiple regional transaction systems, plan on spending 5 - 10% of your global project time on master data integration, design, and testing. 6 Global Master Data Integration and SAP NetWeaver BI (cont.) • You can also decide to use the SAP NetWeaver Master Data Management (MDM) tool to manage and merge the global master data in a central location Source: SAP NetWeaver Magazine, Alan Joch If an MDM implementation is warranted, make it a separate project and don’t include this as part of the global BI project. The BI system simply extracts the new merged and integrated master data. 7 Privacy and Security Concerns • • European and many Asian countries have strict privacy rules European Union Commission’s Directive on Data Protection protects personal information from Europe to countries whose privacy practices are not deemed “adequate” • • All European Union (EU) member countries are bound by the European Commission’s finding of adequacy Some European Restrictions 1. Personal information cannot be collected without consumers’ permission. They have the right to review the data and correct inaccuracies. 2. Data processing companies must register their activities with the government. 3. Employers cannot read, copy, or transfer workers’ private email. Your company can become entangled in many local laws, or seek participation in the 4. Personal information cannot be “safe harbor” agreements by the Department shared by companies or across of Commerce (costs up to $500) borders without express • The safe harbor eliminates the need for prior approval to begin data transfers, and makes approval from the appropriate EU member countries automatic permission from the data subject. Safe harbor agreements can reduce exposure to such legal issues. 8 Tip 1: Capacity and Scalability Is the Top Concern for Your CxO • Don’t under size your global BI system Spend adequate funding on hardware, memory, processing power and disk space A survey of 353 top C-level officers in large companies, reported that the top BI concern was the scalability of their solutions. Overall System Performance and capacity 7.9 Server & Desktop processing capacity 7.88 Determining ROI 7.65 Security 7.62 Data Integration 7.51 Too many Business Intelligence tools in use 7.31 Danger of distributing outdated data 7.21 Difficult for end users to learn or use Source: Intel, SAP & Business Week "Seizing the BI Opportunity" 2006. 7.03 7 7.25 7.5 7.75 8 9 Tip 1: Capacity and Scalability — Global Hardware Example Example of a large BI system Production 12 CPUs 96 GB RAM Sandbox 4 CPUs 32 GB RAM Development 4 CPUs 32 GB RAM Test 4 CPUs 32 GB RAM Key numbers: 3.1 Terabytes of data 2810 named users About 620 active users • • • This company reallocated their Sun6900 box (on Oracle) and have three app servers on the production box More memory allows them to take better advantage of the parallel load of the BI system and to cache many of the frequently run queries (BEx Broadcaster) The hardware also allows them to compare performance between the boxes Global systems must consider network connections (speed), data encryption, and distributions of application and Web servers for load balancing (due to 24hour operations). 10 Tip 2: Pick a Formal Methodology — You Have Many Choices • • Accelerated SAP (ASAP) methodology is not your only choice Even though harder to manage on a global project due to long communication lines, consider RAD, JAD, or EP based on the time to delivery and impact of failure When to Select Different Methodologies High Joint Application Design (JAD) System development Life-Cycle based methodologies (SDLC) Time to Delivery Extreme Programming (EP) Rapid Application Development (RAD) Low Low High Impact of Failure Look at what your global organizational partners have done. You may know more about RAD than you think! Source: Bjarne Berg, DM Review 2004 11 Tip 3: Report Dispositioning: Don’t Throw Every Report into BI • Many tools exist that can report on SAP R/3 data • Make cost-effective decisions • Just because the report is not in SAP NetWeaver BI does not mean it cannot be added to a Portal or viewed on the Web Not all reports belong in SAP NetWeaver BI • You might have static reports that belong in SAP R/3, which would not be cost effective to develop in SAP NetWeaver BI Avoid using BI as a “dumping group” You need to make conscious decisions on what reporting needs you are going to meet, and how you will accomplish this You need a formal report dispositioning process and need to evaluate every requirement as BI vs. transaction reporting. 12 Tip 4: SAP Solutions Manager — EarlyWatch Reports Are Great! • EarlyWatch reports provide a simple way to confirm how your system is running and to catch problems • • A “goldmine” for system recommendations Run them periodically and read the details A real EarlyWatch report from a mid-sized company that has been running SAP BW for the last three to four years On a large global project, system issues can be hard to pin-down without access to EarlyWatch reports. The monitoring reports allows you to tune the system before the user community gets access and complaints arise. 13 Tip 4: SAP Solutions Manager — EarlyWatch Performance Info 1 Performance Indicators The following table shows the relevant performance indicators in various system areas. Area System Performance Hardware Capacity Database Space Management Query Performance Indicators Active Users Max. CPU Utilization on DB Server Max. CPU Utilization on Appl. Server DB Size Last Month DB Growth Avg. Total Runtime of the BW Queries Avg. Database Runtime of the BW Queries Value 18 74 % 74 % 355.52 GB 118.63- GB 11.5 s 8.0 s Trend down steady steady steady steady down steady In a 24-hour operational systems due to time-zones, The performance of your system was analyzed with respect to the average response times and total workload. We did not detect any major problems that could affect the performance of your system. you will have less time to react and fix issues. Therefore, early detection of system issues are critical to the success of a The following table shows the average response times for various task types: global project. 1 Performance Overview Task type DIALOG + RFC UPDATE UPDATE2 BATCH HTTP Dialog Steps 195240 Avg. Resp. Time in ms 3253.3 Avg. CPU Time in ms 728.7 Avg. Wait Time in ms 1.8 Avg. Load Time in ms 2.5 5 48 59288 257762 984.2 133.2 11599.3 693.5 28.2 17.1 2091.2 183.7 26.0 0.7 0.6 4.4 15.2 3.3 8.5 2.2 Avg. DB Avg. GUI Time in ms Time in ms 1110.9 6.3 585.4 80.8 5772.6 405.0 1.1 Current Workload The following table lists the number of current users (measured from our workload analysis) in your system. Users Measured in System Low Activity 98 Medium Activity 11 High Activity 7 Total Users 116 14 Tip 5: Organizing the Global Team — Six Ways to Balance the Global BI Development Effort Option 1 Single site 2 Distributed analysis 3 Distributed analysis and design 4 Co-located analysis and design 5 Multiple co-located analysis and design 6 Fully distributed development Benefits Risks The more distributed the BI development effort becomes, the more difficult it is to maintain communication and get cohesive requirements. 15 Tip 5: Organizing the Global Team — Small Technical Team Roles • These are roles not positions Sometimes one team member can fill more than one role Project Sponsor Many companies fail to formally assign roles and responsibilities. As a result, they have many “jack of all trades” and “masters of none.” Project Manager Business Team Business Analyst Presentation Developer Technical Team BI Architect ETL Developer BI Basis and functional SAP R/3 support Four to five team members and normally three to six months duration on each go-live depending on scope ETL = Extract, transform and load 16 Tip 5: Organizing the Global Team — Large Functional Teams Project Sponsor/ Steering Committee Project Manager BI Architect In a global BI organization, you simply need to create functional teams (instead of the previous technical team models). Portal Developer(s) Sales Team Finance Team Business Analyst/(sub-team lead) BI Developer Presentation Developer(s) ETL Developer Business Analyst/(sub-team lead) BI Developer Presentation Developer(s) ETL Developer Material Mgmt. Team Business Analyst/(sub-team lead) BI Developer Presentation Developer(s) ETL Developer BI Basis and functional SAP R/3 support 15-30 team members and normally 6-18 months duration between each go-live 17 Tip 5: Organizing the Global Team — Localized BI Training Reference Title Audience Language Class size (max) Note All Local 25 Bring in house Query developers Local 15 Bring in house BW-310 Intro to SAP BI BW-305 BI Reporting & Analysis BW-350 BI Data Acquisition ETL developers English 10-12 SAP facility BW-365 BW Authorizations System admin English 1-3 SAP facility SAP-330 BW Modeling BI developers English 10-20 SAP facility • • Training for end-users and the local query developers should be completed in their own language to assure understanding and encourage participation Developer training should be in the project language (e.g., English, German, French) Don’t under estimate the value and cost savings of in-house training. 18 Tip 6: You Have to Plan for Global Cockpits 58% of all companies reported that they were already using BI dashboards and 25% more planned to do it in 2007 – 2008 • Source: Business Week Research Sept. 2006, survey Dashboards are no longer cutting-edge • Purpose Usage Updates Data Measures Context Source Expected by senior management Dashboard Displays performance Scorecard Displays progress Cockpits Displays status and events Performance monitoring Performance management Performance management Real-time feeds Monthly snapshots Daily snapshots Events Summaries Summaries and events Metrics KPIs Metrics & KPIs Exceptions/alerts Targets and thresholds Trends Linked to systems Linked to plans Dashboards and cockpits should be in your longterm global BI strategy (three to five years). Linked to BI systems Sources: Wayne Eckerson, 2005; Bjarne Berg 2006 19 Tip 7: Pick a Project Language and Stick with It! • If you don’t enforce a global project language, BI project documentation becomes fragmented • • The project team will quickly disintegrate into groups based on the language with which they are most comfortable Enforce a project language and require that all emails are written in it and all notes are taken in the same language Don’t allow “side bars” in languages that others don’t understand Make sure the project language is clear, and that pertinent documents are translated in a timely fashion. 20 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 21 Tip 8: In the Blueprinting Phase: Model Your Global Solution 1. Create a model based on pre-delivered SAP NetWeaver BI content 2. Map your data requirements to the delivered content, and identify gaps 3. Identify where the data gaps are going to be sourced from Unit Material Logistics Material number Material entered Material group Item category Product hierarchy EAN/UPC Storage Requirements Plant Shipping/receiving point Billing Customer + Currency Key Unit of Measure Base unit of measure Sales unit of measure Volume unit of measure Weight unit of measure Sold-to Ship-to Bill-to Payer Customer class Customer group ~ Customer country ~ Customer region ~ Customer postal code ~ Customer industry code 1 End user Number of billing documents Number biling line items Billed item quantity Net weight Subtotal 1 Subtotal 2 Subtotal 3 Subtotal 4 Subtotal 5 Subtotal 6 Subtotal A Net value Cost Tax amount Volume Organization Standard Content Map functional requirements to the standard content before you make enhancements Company code Division Distribution channel Sales organization Sales group Personnel Sales rep number Accounting Cost center Profit center Controlling area Account assignment group Billing information Billing document Billing item Billing type Billing category Billing date Creation date Cancel indicator Output medium ~ Batch billing indicator Debit/credit reason code Biling category Reference document Payment terms Cancelled billing document Divison for the order header Pricing procedure Document details Sales order document type Sales deal Sales docuement Time Calendar Calendar Calendar Calendar year month week day Storage Objects LEGEND Delivered in standard extractors Delivered in LO extractor Not in delivered Content -but in R-3 22 Tip 9: Accept Cultural Differences — No Culture Is Dominant! • • • • • Cultural differences should not be tolerated, but embraced Europe has longer vacations (four to six weeks are common, not exceptions) Family time is important — don’t plan 12-hour workdays for four months Not everyone is equally interested in hearing how we do things in the US Many cultures find it offensive to talk about salaries, and money • Talk about value and deliverables instead Consider a co-project manager 23 Tip 10: Meet Local and Global Requirements • In most global organizations there are varying product hierarchies, financial reporting requirements, mapping of accounts, data integration, and tool requirements • Try not to solve only the enterprise reporting needs of the corporation, but also the local reporting and analytical needs • This way you will get local support and ensure the system will actually be used A successful global BI project has more that one constituent If you ignore local requirements, they will simply reinvent your effort and find work-arounds and other tools. 24 Tips 11: Determine Where to Start All functional areas are not equally supported by strong standard BI business content • Some areas have much you can leverage, others will require significant enhancement to meet your requirements The differences are often due to customization in SAP R/3 by companies and/or industry solutions Focus on an area that solves a problem instead of becoming a “replacement” project. Approximate Usage of Standard Content BW 3.5 (percentage of overall development effort) 100 80 Gradually, using a priority-phased approach, solve other business problems. 60 40 20 to ry In ve n M * SC E m gm t lit y Qu a rib ut io n es Di st Sa l * CR M AP O g ur in CO an uf ac t M FI 0 * Rapidly improving content A good way to think of a BI rollout is in terms of business problems. 25 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 26 Tip 12: InfoCube Design — Making the Right Design Decisions • Dimensions • BI allows you to create up to 16 dimensions in a single InfoCube (13 are free) However, using all 16 on a first implementation limits any future extensions without major redesign of the system Line item dimensions increase query performance These are physically stored in the fact table and therefore have fewer table joins Key figures While no limitations are imposed by BI for the number of key figures, typical implementations contain 1 - 25 While a higher number may be required (i.e. CO-PA), you’ll notice tradeoffs of load performance when many records are loaded While more than 45 key figures are not necessarily wrong, it might be considered unusual — you should perform an impact study of the extract on SAP R/3 27 Tip 12: InfoCube Design — General Guidelines • Navigational attributes • Lends flexibility to the way users can access data Common configuration consists of 1 - 35 attributes While technically not incorrect you should review InfoCubes that do not contain any navigational attributes Review any InfoCube that contains more than 60 This may be an indicator that too much information is being placed in a single InfoCube Hierarchies Hierarchies are ways for users to “drill-down” into the data for analysis purposes Typical configurations tend to have 1 - 8 hierarchies Review any InfoCube with no hierarchy (or with more than 8) to validate the design with end-user navigation For global projects you need to consider global and local currencies as well as various units of measures (e.g., lbs, kilo, tons, gallons, liters). Once decided upon, these units of measures should be available in all relevant InfoProviders. 28 Tip 12: InfoCube Design — Evaluating Designs (Real Example) Name Billing documents condition values Customer Delivery service Invoice summary Order summary Sales order condition value Sales overview Profitability analysis Inventory mgmt plant summary Material stock/movements Plant & periodic plant stocks Ad-hoc query order line details Conditions order & billing document Order and Invoice summary SD Pricing order & billing docs Campaign management Commissions Daily management Disposition summary Inquiry summary Matrix Monthly management report Program summary Type Tech_nm Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube MC MC MC MC Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube ZSD_C15 0SD_C01 0SD_C04 ZSD_C06 ZSD_C03 0SD_C15 0SD_C03 Z_COPA_X MRP_MATL 0IC_C03 0IC_C01 ZSD_M01 OSD_MC01 ZSD_C04 ZSD_M02 ZDM_C006 ZDM_C003 ZSD_C01 ZDM_C001 ZDC_C005 ZDM_C002 ZDM_C005 ZDM_C004 Cubes with many red or yellow codes should be examined • Dims* Characteristics Largest dim # (all) total of Char 10 47 11 8 14 5 9 23 5 16 55 11 16 62 13 10 41 10 11 34 7 15 56 14 6 9 3 9 18 4 8 15 5 7 14 3 4 66 56 16 72 20 12 53 12 15 56 19 14 34 14 9 31 8 9 16 4 15 22 4 10 16 3 7 9 3 16 23 4 KF 3 16 15 19 23 2 17 85 22 24 21 6 2 24 3 29 8 10 1 1 141 6 12 # info. Nav Sources attributes 1 107 1 5 2 10 4 116 5 96 1 0 7 16 1 70 1 16 5 15 0 0 88 94 9 109 4 0 6 6 1 0 1 29 1 0 1 0 8 1 The observation relates a company’s current BI system to normally observed configuration parameters, which serve as benchmarks to what is commonly seen at other implementations KF = Key figures 29 Tip 12: InfoCube Design — Evaluating Designs (Another Example) 30 Tip 13: Partitioned InfoCubes That Are No Longer the Same • Often when InfoCubes are physically partitioned, changes occur as new development and fixes are applied After a while there is a risk that some of the physically partitioned InfoCubes no longer are identical This can cause many issues (i.e., If archiving is used, you must ensure copies of these older datastores are maintained to be able to restore data) A real example InfoCubes Nam e Technical nam e All Largest Characteri Key Nav. dim ensions dim ension stics Figures Attrib. Sales Order: ACD 2007 ZCORD_A07 15 5 40 9 59 Sales Order: LPD 2006 ZCORD_L06 14 4 40 9 59 Sales Order: LPD 2007 ZCORD_L07 15 5 40 9 59 Sales Order History: ACD 2006 ZCODI_A06 15 13 58 39 66 Sales Order History: ACD 2007 ZCODI_A07 15 13 60 9 66 Sales Order History: LPD 2006 ZCODI_L06 15 13 58 39 66 Sales Order History: LPD 2007 ZCODI_L07 15 13 60 9 66 NOTES Added 'Created by' as a Dimension(instead of field in Date dim) Added 'Created by' as a Dimension(instead of field in Date dim) 1) Removed 30 Key Figures, 2) Added field "Date for inv/bill index and print out" to Date dimension 3) added field "Customer purchase order type" to business reason dimension 1) Removed 30 Key Figures, 2) Added field "Date for inv/bill index and print out" to Date dimension 3) added field "Customer purchase order type" to business reason dimension 31 Tip 14: Naming Conventions Should Be Followed InfoCubes should be named: 0ABC_C01 or ZABC_C02 ODSs should be named: 0ABC_O01 or ZABC_O02 MultiProviders should be named: 0ABC_M01 or ZABC_M02 ODS = Operational Data Store InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube Supply Chain Mgmt - Inventory Mgmt. Supply Chain Mgmt - Inventory Mgmt. Industry Sectors - Oil & Gas - Exchanges Industry Sectors - Retail Non SAP Area - Marine Services Non SAP Area - Marine Services Non SAP Area - Marine Services Non SAP Area - Marine Services Non SAP Area - SFIO Non SAP Area - SFIO Non SAP Area - SFIO Non SAP Area - SFIO Non SAP Area - EPM (Ent. project mgmt) Non SAP Area - EPM (Ent. project mgmt) PIW - Profit Improvement Warehouse Monthly Operation Planning 3 Monthly Operation Planning 3 Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. & Controlling - Profit Center Acct. Cust.Mgmt Rel.MgmtCRM AnalyticsCross-Scenario Analyses-Case Mgmt Analysis Cust. Rel.Mgmt- CRM Analytics- Cross-Scenario Analyses-Activity ERP Analytics- Sales & Distribution Analyses SAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsSAP R/3 SD Sales & Distribution Analyses ERP AnalyticsCommercial Excellence Strategic Enterprise Mgmt - BPS - Capital Investment Planning Cube for Risk Management Material stocks/movements (as of 3.0B) Exchange balance Retail Competitor Pricing PEDCO DTHEAD_DTTAIL PEDCO LOGSUM PEDCO MTHEAD_MTTAIL PEDCO RCHEAD_RCTAIL SFIO Movement Position SFIO Movement Position History SFIO OIPR SFIO OIPR History EPM Cube II Enterprise Project Mgmt (EPM) PROFIT IMPROVEMENT WAREHOUSE MOP3 Pricing Marker Monthly Operation Planning 3 Margin Analysis PCA: Summary 1 PCA: Summary 1 History PCA: Transaction data Profit Center Accounting - Customer Profit Center Accounting - Other Profit Center Accounting - Periodic Balance Profit Center Accounting - Planning Items Profit Center Accounting - Summary Profit Center Accounting - Vendor CRM Case Management Analysis Activities Billing Cube Billing Cube - History Billing Summary Billing: Condition Data Cube Billing: Tax Conditions Cube Daily Lift Report Delivery Cube Delivery Cube - History PAWS: Pricing Analysis PAWS: Pricing Analysis (1) PAWS: Pricing Analysis Archive CE Pricing Log Capital Investment Planning ZRISK 0IC_C03 0OI_EXC01 ZRTL_C01 ZPDCO_C01 ZPDCO_C05 ZPDCO_C03 ZPDCO_C02 ZSDMVMPOS ZSDMVMHIS ZSD_OIPR ZSD_OIPHS ZEPM_C02 ZEPM_C01 ZPIW_P001 ZMOP_CMKR ZMOP_P01 ZPCA_C04 ZPCA_C03 ZPCA_C03H 0PCA_C01 ZPCA_C09 ZPCA_C10 ZPCA_C05 ZPCA_C06 ZPCA_C07 ZPCA_C08 ZCRM_CASE 0CSAL_C01 ZSD_CVF0 ZSD_CVFH1 ZSD_C17 ZSD_C06 ZSD_C06C ZSD_ZS561 ZSD_CVL0 ZSD_CVLH1 ZPAWS ZPAWS1 ZPAWS2 32 ZCE_C01 ZBPS_P05 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 33 Global Performance Issues • Global projects have many performance challenges • Hardware must be optimized for 24/7 operations and network capabilities BI system must also have optimal backup and disaster recovery windows and rapid data load processing Since most of the query rendering time is spent on the database, it is important that global projects spend serious time optimizing the database reads by leveraging tools such as the BI Accelerator and the classical SAP NetWeaver BI aggregates The correct definition of global aggregates can dramatically reduce the time spent on database reads by the BI queries, thereby delivering faster query results to the users in various geographical locations. 34 Tip 15: Performance Enhancements Are Available — Use Them • Check indexes periodically • Check database statistics to route queries faster • Under RSA1 Manage Performance At this company, 50% of the InfoCubes had outdated database statistics that should be updated For large InfoCubes, or cubes with many users, the percentage used to build the database statistics can be increased to 15 - 20% • May yield improved query routing Nam e COPA(US) : P&L L'Oreal R110: 2006 Technical nam e Indexes Aggregate Stats DB index build Stats YCPAPL_1 10% COPA(US) : P&L L'Oreal R110: ACD 2007 ZCPAPLA07 10% COPA(US) : P&L L'Oreal R110: LPD 2007 ZCPAPLL07 10% YCZO_1 10% ZCARHI_1 10% FIAR (CS) : Cube - Hist. indicators Zoom YCZOHI_1 10% Agreement Cancellation and rejection Carry Over Consolidated Open Orders Consolidated Open Orders Delivery Historical Invoice LPD Invoice RGA Data Sales Order History Sales order Service rate Invoice: ACD 2004 Invoice: ACD 2005 Invoice: ACD 2006 Invoice: LPD 2004 Invoice: LPD 2005 Invoice: LPD 2006 Sales Order: ACD 2006 Sales Order: ACD 2007 Sales Order: LPD 2006 Sales Order: LPD 2007 Sales Order History: ACD 2006 Sales Order History: ACD 2007 Sales Order History: LPD 2006 Sales Order History: LPD 2007 Invoice: ACD 2007 Invoice: LPD 2007 Delivery: ACD 2007 Delivery: LPD 2007 Service Rate: ACD 2007 Service Rate: LPD 2007 YC13_AGR 10% YC11_CR 10% ZCSD_CROV 10% ZC_OO 10% FI-AR Accounts Receiv. Line Item IC FI-AR Historical Indicators US YC_OO 10% YC12_DEL 10% ZCHSTLI 10% YC13_INV 10% ZC_RGADTL 10% ZCORDINV 10% YC11_ORD 10% YC11_SR 10% ZCINVA04 10% ZCINVA05 10% ZCINVA06 10% ZCINVL04 10% ZCINVL05 10% ZCINVL06 10% ZCORD_A06 10% ZCORD_A07 10% ZCORD_L06 10% ZCORD_L07 10% ZCODI_A06 10% ZCODI_A07 10% ZCODI_L06 10% ZCODI_L07 10% ZCINVA07 10% ZCINVL07 10% ZCDELA07 10% ZCDELL07 10% ZCSRIA07 10% ZCSRIL07 10% 35 Tip 15: Performance Enhancements — Aggregates Are Often Incorrectly Built (Real Example) • • Several cubes have no aggregates, while others can benefit from generating new proposals A score above 30% for average aggregate valuations should be a target for a data store 36 Tip 15: Performance Enhancements — Correct Aggregates Are Easy to Build • This example shows the benefits of aggregates by using system statistics to generate proposals This very large InfoCube had over 160 queries attached to it and only one aggregate • • Select the run time of queries to be analyzed (e.g., 20 sec) Select time period to be analyzed • Only those queries executed in this time period will be reviewed to create the proposal High value aggregate proposal (users who had queries that ran over 20 seconds during the last six months, would have benefited from 438 times — each query execution) 37 Tip 16: Memory Cache Is Often Set Too Low • The cache settings at many companies are too low • Review the settings with the Basis team and look at the available hardware • For example, at one large company the cache was set at 100 MB for local and 200 MB for global cache, which is too low for a system with thousands of users and 4 TB of data During the review, use the transaction code RSCUSTV14 in SAP NetWeaver BI to increase the cache if needed Focus particularly on the global cache To monitor the usage of the cache, use transaction code RSRCACHE and also periodically review the analysis of load distribution using ST03N – Expert Mode Example: At one company over 61% of all 35,644 navigation steps in a month accessed the database instead of the cache, even after the query was executed. 38 Tip 17: Use the BEx Broadcaster to Pre-Fill the Cache Distribution Types By broadcasting the query result of commonly used queries to the cache, your users do not need to execute the query from the database. Instead the result is already in the system memory (much faster). 39 Tip 18: Focus Performance Enhancements on Large InfoCubes Type InfoCube InfoCube InfoCube InfoCube InfoCube InfoCube InfoArea Non SAP Area - SFIO Non SAP Area - SFIO Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. Fin. Mgmt & Controlling - Profit Center Acct. • SFIO OIPR History SFIO OIPR Profit Center Accounting - Customer Profit Center Accounting - Vendor Profit Center Accounting - Other PCA: Summary 1 Tech Name ZSD_OIPHS ZSD_OIPR ZPCA_C09 ZPCA_C08 ZPCA_C10 ZPCA_C03 Index x x x x x x Stats x x x x x x No of Rows 157,218,110 157,131,010 119,969,043 116,068,014 116,012,946 108,784,493 Typically 20% of the cubes and queries will see 80% of the usage • Name Focus on these, when performance tuning, or it can become overwhelming InfoCubes with over 100 million rows should be analyzed for performance enhancements, such as broadcasting of queries to cache, aggregates, indexes, database stats, etc. Avoid building local queries that are unique to the various geographies or business units. Focus instead on the reusability of global queries that are saved as local views (favorites). 40 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 41 Tip 19: Avoid “Swiss Army Knife” MultiProviders: It Will Be Slow! • There is a temptation to use only a few MultiProviders to build all queries This may dramatically slow down queries Hint: This design may prevent queries from being executed in parallel Avoid attempting to build a single MultiProvider to do all functions of the data warehouse 42 Tip 19: “Swiss Army Knife” MultiProviders and Parallel Processing • To avoid an overflow of the memory, parallel processing is cancelled as soon as the collected result contains 30,000 rows or more and there is at least one incomplete subprocess • The MultiProvider query is then restarted automatically and processed sequentially What appears to be parallel processing corresponds to sequential processing plus the preceding phase of parallel processing up to the termination Generally, it’s recommended that you keep the number of InfoProviders of a MultiProvider to no more than 10 Avoid creating InfoCubes that are specific to one country or business unit. If you do this, you will have to later combine them in large MultiProviders that may cause slow performance. 43 Tip 19: “Swiss Army Knife” MultiProviders and Parallel Processing (cont.) • Consider deactivating parallel processing for those queries that are MultiProvider queries and have large result sets • With SAP BW 3.0B SP14 (SAP BW 3.1 SP8), you can change the default value of 30,000 rows — refer to SAP Notes 629541, 622841, 607164, and 630500 A larger number of base InfoProviders is likely to result in a scenario where there are many more base InfoProviders than available dialog processes, resulting in limited parallel processing and many pipelined sub-queries 44 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 45 Tip 20: Clean Up Old Objects That Are No Longer Used • • After a few years of running SAP NetWeaver BI, companies often have many objects that are no longer used Cleaning them up makes for a simpler development environment, which is easier to navigate and has a positive impact on analysis on when designing new objects Keep your environments clean of obsolete junk. 46 Tip 21: Don’t Replicate Legacy Data Logic in SAP NetWeaver BI • • It’s tempting to replicate the SAP transaction system in the BI environment You can spot this when you find many ODSs that serve as lookup tables (easy to recognize, since these systems have many ODSs and each of them have few fields) Number of fields per ODS 14 12 Number of ODSs Don't allow the BI system to become a replication of the transaction system, merely because that is what your developers know. 10 8 6 4 2 0 0-5 6-10 11-15 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 90+ 47 Tip 22: Avoid Direct Querying of ODS • • • • • To quickly solve local reporting requirements, it is tempting to bypass the global InfoCubes and query the data stores (ODSs) directly — this is a mistake ODSs are for central staging of historical data, detailed analysis, lookups, and data mining — not for local reporting If you allow the users to bypass the global InfoCube logic, you will find yourself supporting slow local queries that do not take full advantage of the inherent performance of the OLAP processor Many local detailed queries will consume system resources, causing other global users to suffer If you follow this path, you will be asked to create InfoSet queries and eventually have minimal advantages of standard content, which will lead you to the road to build a traditional legacy data warehouse Restrict the number of users that can access the ODSs. 48 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 49 Tip 23: Be Aware of Divergent Country Accounting Rules • Different accounting standards must be dealt with on global projects • For example, pension obligations are considered long-term debt in most European countries and long-term liabilities in the US (unsecured) Accounting rules for depreciation, amortization, depletions, and allocations vary from country to country For example, the US allows for accelerated depreciation in certain instances Plan on involving accountants and mapping tables if you are consolidating reporting for both local and global purposes. Make sure you have an audit trail of transformations. 50 Tip 24: Standardize Currency and Units of Measures • • A global BI requires standardized currencies and Units of Measure (UOM) for aggregate reporting Conversions from pounds to kilos, or from liters to gallons are easy, but you have to decide how to handle currencies Possible solution A: Some companies use a pro-forma currency translation rate for reporting during an interim period and a real currency translation rate when the financial books are closed or funds are transferred Possible solution B: Some companies use previous month’s currency translation rate for reporting during an interim period and a real currency translation rate when the financial books are closed or funds are transferred Sites like www.xe.com can provide you pro-forma currency translation daily, but you have to make a decision. 51 Tip 25: Determining Hierarchies — Global vs. Local • On a global project you can create local and global hierarchies • For example, a large European telecom company created a five-level sales material hierarchy in BI that reflected the Dutch reporting view, a seven-level hierarchy that reflected the German reporting view, and a four-level hierarchy for corporate reporting This allowed reporting for each subsidiary and a new corporate reporting hierarchy that was not available in any local system Lessons learned: You don’t have to select a single hierarchy; local needs can also be accommodated Trick: MultiProviders can be used to mask the complexity for the query developers. 52 What We’ll Cover … • • • • • • • • • Introduction Scoping the SAP NetWeaver BI project Getting the project started Designing InfoCubes — the good, the bad, and the ugly Improving Performance — what to do, and how to do it Understanding the curse of MultiProviders Cleaning up and ODS management Considering global integration points Wrap-up 53 Resources • Steve McConnell, Rapid Development (Microsoft Press 1996, ISBN: 1556159005) • Jeremy Kadlec, Start to Finish Guide to IT Project Management (NetImpress 2003, ISBN: B0000W86H2, Digital, 109 pages) • COMERIT.NET (presentations, tools and acccellerators) www.Comerit.net 54 Resources (cont.) • European privacy laws • www.msnbc.msn.com/id/15221111 European Commission’s Directive on Data Protection www.export.gov/safeharbor Where to FIND it 55 7 Key Points to Take Home • • • • • • • Be sensitive to other cultures and don’t enforce US working methods More than one methodology is available for you to use Long-term environment sizing and planning is critical Performance tuning is not an afterthought, but a project task An SAP NetWeaver BI architecture should be formulated before a project starts Plan for BI cockpits and dashboards — they are not nice-tohaves, but a critical part of BI Get the global team active and develop the SAP NetWeaver BI solutions for local and global needs 56 Your Turn! How to contact me: Dr. Bjarne Berg bberg@comerit.net 57