Unleash SAP using the Microsoft Platform: Using BizTalk Server to Bring Two Worlds Together Published: February 2010 Authors: Chris Kabat and Naresh Koka, Principals at MPS Partners Applies to: Microsoft BizTalk Server and SAP Summary: This document describes how to integrate, automate, and simplify business processes using Microsoft BizTalk Server and SAP. Business scenarios are presented involving SAP interoperability and technical patterns for how their solutions can be implemented with an integration platform like Microsoft BizTalk Server. 1 Copyright The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. © 2010 Microsoft Corporation. All rights reserved. Microsoft, BizTalk, and SharePoint are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. 2 Contents Overview.......................................................................................................................................... 4 Integrate, Automate, Simplify Your Business Processes ............................................................. 4 Providing Business Value ............................................................................................................ 6 Better Together ........................................................................................................................... 7 Business Rules, Calculations, Workflows – Where do they live? .................................................... 9 SAP Integration Scenarios ............................................................................................................... 9 Integration Scenario – Universal Connectivity for SAP ............................................................... 9 Integration Scenario – Composite applications – Consolidation of disparate SAP instances... 11 Integration Scenario – Self Service Applications – Process Automations ................................. 11 Integration Scenario – Data Synchronization – MDM .............................................................. 12 SAP Integration Mechanisms ........................................................................................................ 12 BizTalk Server and the BizTalk Adapter Pack................................................................................. 14 SAP Interoperability in Action ....................................................................................................... 15 Building the Order Submission Process ......................................................................................... 18 Extending the Order Submission Process ...................................................................................... 24 Adding Real-time Visibility into Orders ......................................................................................... 26 Conclusion ..................................................................................................................................... 29 3 Overview Organizations today must contend with fast moving and ever-changing business requirements to maintain their competitive edge. A one-stop-shop for enterprise applications is becoming less possible and certainly less practical to IT organizations. It is very rare that a single Enterprise Resource Planning (ERP) solution can effectively meet all the business requirements of an enterprise. To meet the need of their business, organizations often invest in new applications that offer expanded functionality. When these applications are not integrated, business units often fail to leverage the new functionality available to realize their objectives and work towards the organization’s mission. Many organizations choose SAP as a suite of modules to run their daily business. Large organizations especially use this popular ERP to move themselves towards standard business processes in areas such as Sales and Distribution, Material Resource Planning, or Human Resources. Organizations that run their businesses on SAP face a number of different challenges. The well-defined robust business process that they purchased SAP for usually comes at a cost. It is very common that SAP must communicate in real time to several other systems and platforms in real time to achieve the productivity and agility goals. This process is certainly not straight forward, and can cost organization real dollars in implementation, training, and long term operational costs. Using the business process provided by SAP alone will not generally provide the business value and productivity gains an organization needs to strategically differentiate themselves from their competition. Organizations need to integrate and interact with those processes in many different ways to achieve the greatest value. This document will address how we can integrate, automate, and simplify these processes using Microsoft® BizTalk® Server and the Microsoft application platform. We will discuss the business value of such an approach within the context of a fictitious outdoor outfitter called Adventure Works. We will describe some business scenarios involving SAP interoperability and technical patterns for how their solutions can be implemented with an integration platform like BizTalk Server. Integrate, Automate, Simplify Your Business Processes Organizations have been both successfully and unsuccessfully integrating with business processes for many years. Typically this type of integration has been done with vendor APIs, flat files, and more recently collections of Web services. However, these patterns of integration generally end up in tightly-coupled, point-to-point set of interfaces between systems and partners. The interfaces tend to be very rigid and inflexible. Back-end system changes can cause a cascading rewrite of interfaces that can take months. Imagine an SAP ERP system upgrade that will already require months and many resources. This upgrades timeline can be extended even further due to the plethora of system interfaces that need to be addressed as part of the project. BizTalk Server is an integration platform that integrates heterogeneous applications both inside and outside an organization’s firewalls. It can help insulate us from changes in either source or destination systems by serving as an integration broker. Along with basic integration capabilities, BizTalk Server is a platform that can provide us with much 4 more. BizTalk Server can help integrate, automate, and simplify your business processes: Integrate Applications: Many organizations struggle with inconsistent information across the enterprise. Most often in SAP scenarios, SAP is considered the source of truth system for many pieces of data such as orders. If an organization chooses to receive orders outside of SAP, for example a shopping cart application, the data must be reentered into the ERP system for fulfillment. BizTalk Server and its adapter for SAP allows organizations to enter that information in only one system and have it synchronized throughout the organization in real-time. This can often provide better data integrity and reduce manual labor and overall operation costs. Automate Processes: Often, business processes such as an order approval process can touch many parts of the organization. Often the initial order may be initially reviewed by customer service using a customer service or Customer Relationship Management (CRM) system. Based on the size of the order, it may then be sent to a manager for review via email or a workflow based system such as Microsoft Office SharePoint®. Once approved, it may be sent to the ERP for final processing. Although each of these steps have their own automation, the entire process is typically not automated because each individual system cannot talk to each other. Using BizTalk Server and its robust set of adapters, long-running business process orchestrations can be created to allow the entire process to be conducted in an efficient way. It can automate the hand off between systems, removing the chance of human error. Furthermore, it provides business activity monitoring to track information about the process along the way. Using this information, an organization can verify service levels across the entire organization are being adhered to. Simplify Operations: Often it just makes sense to put the data in the hands of the users that need it. Not all users of an organization may have direct access to the back end ERP system. Often this requires many hours of training and can be quite cost prohibitive, especially for the occasional user. In organizations running SAP for HR, it is a common request for employees to interact with their personal data from within a Microsoft based portal such as SharePoint Server. It is much easier for employees to interact with tools they work with on a regular basis. This can be done by using Microsoft BizTalk Server as a platform for Service Oriented Architecture (SOA). Function from the SAP system can be exposed as composable Web services that can be accessed from front end applications. IT can provide front end composite applications that allow users to interact with the data from SAP. Better yet, the same services can be exposed to “power” information workers and business users so they can build the applications they need without going through long IT cycles. A well planned out service oriented architecture can greatly simplify interactions across the entire IT landscape. 5 The implementation of an advanced business process and integration server such as BizTalk Server can truly help an organization achieve business value over time. By using the rich adapter set for systems across the enterprise, we can bring data to its lowest common denominator and manage the semantic translations that allow one system to talk to another. By doing this, we truly integrate, automate, and simplify our business processes and give our organization a competitive advantage. Providing Business Value Often in discussions of integration, business value gets lost in discussions of advanced architectures and complex mapping. It is very important that the effort put into the integration of systems have the goal of providing business value in mind. Here are some of the key business values provided by BizTalk Server: Extract value from existing IT investments: Specifically within an SAP environment, this is huge. Often this has been worded as “Do more with what you already have.” By creating a service layer on top of an existing system, we can extend the reach of that system. Using the adapters, we can create finegrained or rich composite services that expose business function from our back end systems. With these services we can then provide that business function to consumers such as SharePoint Server, Office business applications, or custom .NET applications. Drive business efficiency through automation: By automating complex business processes across multiple systems we can increase the overall operational efficiencies of our processes. For example, in our order approval process mentioned previously, a manager may have to approve orders over a certain amount. In the past, that process may have been paper or email based. If a manager was unavailable, that could delay the process for days. Using a business process orchestration, we can define rules to help us handle the exceptional case where an order has not been approved within a certain timeframe. Not only will that provide customers quicker service, but it also saves manual labor by the customer service rep as that person will not have to track down approvals for an important customer order. 6 Increase business agility: Two things are true in modern enterprises: First, business users have become more technical savvy. Second, business needs are ever changing to allow an enterprise to keep up with the competition. Business processes must be more agile and flexible than ever before. By providing some well-designed, loose-coupled services, business processes can be reorganized and changed more rapidly to meet the current business needs. Whether the processes are modified by “power” business users or IT users, having a platform for service oriented architecture and integration can greatly increase this flexibility. BizTalk Server provides many benefits within an organization. It can streamline operations and provide consistency and visibility across an enterprise’s business processes. It can help provide a quick ROI and the business value an organization needs in an integration platform. Next we will address how that business value is delivered in an SAP environment. Better Together SAP provides its customers a tremendous amount of value through core business processes in many different functional areas. Sometimes these processes can be used as is and sometimes they must be customized to match up with how an organization does business. Either way, having a strong starting point based on industry best practices saves organizations time and money over building these processes from scratch. Organizations invest a lot of time, resources, and real dollars to achieve this vision with the SAP platform. However, how a user interacts with these SAP business processes can be challenging. The user interface delivered by SAP has to cover the least common denominator for all businesses using this process. This often means the screens are very difficult to use because of the amount of data required. Without a large amount of customization, the screens can be somewhat daunting. What one would consider a single task requiring just 7 a few pieces of information, may require navigation to multiple screens. This may be fine for power users, but can be more difficult for an occasional user or someone who is not familiar with the SAP GUI at all. Even as SAP extends their user interface to be more flexible and feature rich, tailoring each screen to specific user scenarios will continue to be difficult. A majority of the occasional users discussed above are generally familiar with the Office suite or are familiar with SharePoint Server. Why not use the Microsoft platform to create applications in the tools they use every day? For example, what if a customer service representative receives an email and needs to modify or place an order on behalf of a customer. Instead of flipping back and forth between Outlook and the SAP GUI, we can provide a task pane application to modify the order right in Outlook. Whenever data from both SAP and non-SAP systems must be combined, a composite based solution using the Microsoft platform is typically the most robust. This type of solution lets us both simplify and automate the process. In our previous example, there are two approaches to this problem of integrating SAP with Office. For standard scenarios in which this type of integration is desired, Duet is a joint initiative between Microsoft and SAP that can provide an answer. Duet provides the ability to interact with SAP business process in Office. It is an off the shelf product and it is very specific to SAP and Office. Duet will not provide us a platform for heterogeneous integration beyond SAP and Office. In our real world example, however, we want a richer and more custom solution that can integrate process across many different systems. From an architecture perspective, we may want to provide a service to modify orders in SAP. This service can then be called from multiple applications, not just limited to our one scenario. In this case, BizTalk Server and the BizTalk Adapter Pack provide the capabilities we need to interact with the SAP APIs as needed to implement our scenario. By using BizTalk Server we can provide a robust and scalable architecture for front ending requests to SAP in a way that other applications using the Microsoft platform can understand. BizTalk Server as we proposed here is a middleware tool that is ideal for implementing service oriented solutions that talk to SAP. SAP has a competing product called PI (Process Integration, formerly known as XI) that provides a similar toolset with a specific slant towards SAP to SAP integration. SAP provides out-of-box modules in PI that allow modules within SAP to communicate. BizTalk Server comes with over 70 adapters that allow it to talk to the Microsoft platform and beyond (including SAP). Time to market is typically faster on the BizTalk platform because of its tight integration with Visual Studio. This is especially true when integrating with other applications on the Microsoft platform or other applications that have BizTalk Adapters. We normally see the following middleware patterns in organizations that use pieces of both the Microsoft and SAP platform: 8 Use BizTalk to facilitate communication between SAP and the Microsoft platform or other applications outside of SAP that have BizTalk adapters. Uses PI to facilitate SAP to SAP inter-module communication. Use both platforms to provide robust solutions where SAP manages composition of SAP modules and BizTalk Server manages composition of SAP and non-SAP applications. Working together with SAP, the Microsoft platform can provide more robust solutions and interfaces, greater flexibility, and easier deployment in tools and platforms that business users are used to. It can extend the reach of SAP to larger groups of people without exposing them directly to the SAP toolset and reduce the overall cost of end user training. SAP Integration Scenarios There are a number of different scenarios in which SAP is the target of integration. This section of the document will describe those scenarios and the business drivers behind them. Later in the document, we will show concrete examples of how some of these scenarios may be implemented using the Microsoft platform and BizTalk Server. Integration Scenario – Universal Connectivity for SAP There are many common scenarios in which we need to connect external partners, other systems, mainframes, and even physical events to SAP. This can be done through a pure EAI type architecture or through something more service oriented. In these cases, the true business value is to integrate Business Rules, Calculations, Workflows – Where do they live? Beyond the technology platform decision, there is usually also a functional decision that needs to be made. This decision is to define where business rules, business calculations, and workflows live. The Microsoft platform, and specifically BizTalk Server with its business rules engine (BRE), can provide a robust platform for rule development and processing. However, SAP business rules are typically the single-source of truth in an SAP centric organization because they are very tied to the business processes they support. For example, envision the calculation of sales tax for an order. SAP has an infinite number of ways to implement these tax rules for different types of customers (e.g. international or domestic, type of partner, etc.). It would not make sense to re-implement these rules on the Microsoft platform, but it may make sense to expose that rule through a service for consumption by the Microsoft platform. However, when creating functionality on the Microsoft side of the equation, it makes most sense to keep the business rules on the Microsoft side. For example, imagine we provided an expense reporting process using SharePoint Server and Forms services. We may want that expense reporting to go through some routing through non-SAP users before being handed to SAP for approval by accounting. We may route to different levels of management based on expense report amount or restricted items. It is very possible that these approvers never interact with SAP and it would require a lot of training to expose them to the SAP GUI. In this example, it makes sense for the workflows and supporting business rules to live within the Microsoft platform. After the expense report goes through the approval process, a service can be invoked to pass the expense report to SAP for further processing. The decision of where to put business function and rules is a common struggle in a combined SAP and Microsoft environment. The best approach is to realize that the best of both worlds exist. As long as guiding principles defining a standard way to leverage these platforms are defined and followed, organizations can have great success with the platforms working together. 9 with existing processes. Some examples are: B2B/EDI: Often organizations need to trade information with partners via standard interfaces like X12/EDIFACT EDI. BizTalk Server has a robust platform for EDI that can integrate directly with inbound or outbound iDocs. BizTalk Server can also integrate EDI data with RFC or Enterprise Service interfaces. Although iDocs and the EDI standards are very close, it most often makes sense to have an intermediate, canonical schema in the middle tier that data is transformed to. This sets up an architecture for future growth. For example, an ORDERS05 Idoc is sent from SAP and mapped to an Orders schema within BizTalk Server and published. This can then be subscribed to multiple processes, one of which may transmit an X12 850 document to partners. Mainframe: Because BizTalk Server has adapters for both SAP and Host Systems, it is very possible for data to be integrated from one system to another. To allow each system to maintain its own business rules, integration points can be made back and forth. Specifically, the BizTalk Adapters for Host Systems allow calls initiated from BizTalk or initiated from the Mainframe. This way, a legacy customer service application on the mainframe might call a BizTalk hosted service (through a COBOL program) that then invokes a business process to be started within SAP or retrieve information from SAP in real time. RFID: SAP has extensive business processes in sales, distribution, and material management. These processes often involve human interactions and documents to be passed back and forth along with shipments. BizTalk Server has RFID capabilities that can use physical events to automate this process, removing the human element where possible, thus increasing the efficiency of the process. For example, as a shipment leaves a doc, a goods issue transaction may be issued in SAP. At the same time, an EDI 856 (Advance Ship Notice) may be sent as well. ESB: An enterprise service bus is a very dynamic architecture for advanced message processing. The idea is that any consumer may plug itself in or unplug itself from the bus at any given time. As it does this, the appropriate messages should be routed to it over time. It is very easy to envision SAP as the consumer of ESB messages. For example, an Order message placed on the bus may be routed to SAP for fulfillment. However, with the BizTalk Adapter Pack, SAP may also be a provider of messages. As events occur within SAP, messages can be sent to the bus for processing. This is done by BizTalk Server acting like an RFC endpoint/onramp to the bus. SharePoint: As discussed previously, SharePoint is becoming a platform for composite applications. As such, it often needs to consume SAP data to give information workers the data they need. This can be done by exposing and composing SAP function into BizTalk services. These services can then be consumed by custom Web parts or by the Business Data Catalog (BDC)/Business Connectivity Services. On another front, SharePoint can be an entry point for electronic content such as invoice documents. When these document management scenarios are implemented, metadata from the document can be used to enter data within SharePoint (i.e. in our example as the invoice is uploaded, 10 metadata can be captured and an invoice document can be created in SAP using standard BAPIs or Enterprise Services) Using BizTalk Server and the BizTalk Adapter Pack provide many different connection options with SAP. There are many more scenarios that are more advanced that will be discussed as well. Integration Scenario – Composite applications – Consolidation of disparate SAP instances For global companies, it has become increasingly important to harmonize IT systems and processes worldwide. Companies’ customers are expanding across national boundaries and expect to have their requirements managed consistently from region to region. Managing global supply chains is more difficult when systems differ from country to country. BizTalk Server provides a cost-effective platform for companies to create a composite solution environment whereby information and processes from multiple SAP instances can be combined to provide a unified view for the end user. In global enterprises, SAP instances may be different from country to country or region to region. Sometimes this is done for security reasons, and sometimes it is done to satisfy different national laws. In these cases with many disparate SAP systems across the organization, is often the case that SAP business processes can be used as is, without customization. Just as often, these processes must be customized to match up with how an organization does business globally. How a user interacts with these SAP business processes can be challenging and vary by nth degree from country to country. For example a tax module configured for the US will be completely different from that configured for India. Getting business processes to work across these instances or pulling data from these multiple instances to get a single Enterprise view of data can be challenging. Using BizTalk Server we can pull that data into one place or integrate that data across instances to simplify the processes and provide the desired information to the users that need it. Integration Scenario – Self Service Applications – Process Automations As continuing cost and efficiency pressures force businesses in every sector to cut expenses and streamline operations, companies are finding that technology that replaces costly and labor-intensive functions can play a key role in making operations more cost effective and efficient, allowing them to devote more attention to pressing management issues. For example, human resources executives spend many hours each week answering routine questions from employees about such things as how to determine medical benefits, how to change the investment allocation on a 401k, what a company´s family leave policy is and how to value stock options or fulfilling employee re-quests such as changing personal demographic data, leave request etc. However, a solution based on BizTalk Server can allow companies to aggregate dozens of routine human resources functions and provide employees self-service access. 11 Today companies need to strive to empower their vendors and customers and increase efficiency. To solve this more and more companies are looking to leverage their existing investments made in the Microsoft and SAP platforms. The decisions facing many of these companies involves whether to use the software licenses they already own, have deployed and their users are familiar with, against purchasing new licenses that will require new deployments and training. Rather than have external customers or vendors call an enterprises’ back-office, they are now setting up portals where customers and vendors can get answers to commonly asked questions like, PO status, where is the corrected invoice?, can I see the supporting documents for this PO, etc. BizTalk Server with its robust connectivity solutions and integration engine provide a plat form to enable these types of self-service applications. Integration Scenario – Data Synchronization – MDM Every enterprise is arguably made up of the same two things: information and process. These days with disparate software solutions managing different parts of the enterprise, data synchronization is key factor to maintain consistency. MDM – Master Data Management has become much more about "the discipline for ensuring the consistency of an enterprises’ data" than it is about "enabling a single view of product or customer data." And success with MDM is at least as much about the data, or information, and the processes governing its life and use, as it is about any other business processes -- broken or not. BizTalk Server provides a cost-effective platform for companies to create a data synchronization solution using its robust messaging platform and out-of-the-box connectivity options to a myriad of ERP packages and database platforms. SAP Integration Mechanisms Before we discuss further the implementation of the SAP integration scenarios, it makes sense to discuss SAP integration mechanisms from a high level. There are several ways in which integration and interoperability can occur between SAP and the Microsoft platform. Some of these mechanisms are: Category Office Integration Mechanism Microsoft Duet Portal Integration iView WebParts Data Integration .NET Data Provider for BW .NET Data Provider for SAP Description Used when organizations have standard SAP business process scenarios that need to be surfaced in Office. Used when an enterprise uses MOSS and SAP Enterprise Portal. Allows pieces of SAP Enterprise Portal to be surfaced in MOSS. Allows SSIS/.NET access to SAP BW data normally for business intelligence and reporting needs. Allows SSIS/.NET access to SAP ERP data. 12 Category Service Oriented/Application Integration Mechanism (BizTalk Adapter Pack) SAP Enterprise Services RFC/BAPI (BizTalk Adapter Pack) Description Provides some composition on SAP side, called as standard Web services. Direct access to SAP RFCs as both a consumer and service. This mechanism includes transfer of iDOCs (standard flat file format used by SAP). Based on the scenarios previously laid out in this document, we will focus mainly on the service oriented/application integration mechanisms. Using these mechanisms, we can build a set of services on top of SAP that allow us to build applications on the Microsoft platform. By providing these services in a standard way, they can be consumed by: Custom .NET Applications: o SmartClient or Web Applications o Office applications o Mobile Applications o Rich Internet Applications based on Silverlight MOSS Artifacts: o Business Data Catalog o Custom Web Parts o InfoPath forms/workflows o Enterprise Search Trusted partners with the ability to consume services When discussing this type of integration, it makes sense to discuss quickly some benefits of a service oriented architecture and an appropriate platform. Obviously there are many different ways to interact with SAP. Solutions can be built to talk from a .NET application directly to an SAP RFC or even to an SAP Enterprise Service. The problem with this type of architecture is that the application will be tightly coupled to the SAP API. This can cause problems during system upgrades and can cause the application to have to work with complex data structures (a lot of SAPs parameters are a combination of German and English). It also can proliferate a number of point to point interfaces that may require changes as business function changes. A more desired architecture would be to front end the integration points to SAP with a well-defined service layer. This can abstract consumers from back end interface changes as well as provide advanced processing to occur within the service tier. For example, a single service call can bring in data from systems other than SAP, aggregate the results and return to the consumer. It also provides the opportunity to use more dynamic architectures such as an Enterprise Service Bus (ESB) architecture and advanced SOA 13 mechanisms such as service virtualization. An ideal platform for this type of architecture is BizTalk Server. BizTalk Server and the BizTalk Adapter Pack Since the advent of BizTalk Server 2006 R2, integration with SAP has been done through the BizTalk Adapter Pack. The BizTalk Adapter Pack adapter for SAP is a WCF Line of Business adapter. This means it uses the standard WCF adapters to communicate with SAP. Specifically, this adapter uses the RFC client libraries to talk to SAP. It is no longer based on the SAP .NET Connector architecture that the original SAP adapter was built upon. The BizTalk Adapter Pack allows us to communicate with SAP in the following ways: Mechanism RFC/BAPIs (BizTalk as a consumer) RFC (BizTalk as a server) iDocs .NET Data Provider Description Allows consumer to invoke RFC on the SAP system. Allows BizTalk Server to emulate an RFC endpoint as if it were another system. Allows BizTalk Server to send and receive iDocs. The underlying mechanism is through standard SAP RFCs for iDoc communication. Allows SSIS/.NET to access the SAP ERP tables directly. This mechanism requires that some custom ABAP programs are deployed on the SAP system. When specifically discussing RFC communication, the adapter pack itself can be hosted in a number of different ways, including outside of BizTalk Server. In fact it can be hosted in the following ways: Within a .NET application Hosted in IIS using the WCF Service Model Hosted in/Consumed using BizTalk Server There are pros and cons to each approach. Normally the following is suggested: Mechanism Situation RFC/BAPI from .NET Use for .NET applications that need to access SAP. Use for creating fine-grained, low-latency services with a well-defined contract. 14 RFC/BAPI hosted in IIS RFC/BAPI in BizTalk Server Use for very fine-grained, low-latency services where just exposing the underlying method is fine. Normally used to expose methods to MOSS using BDC. Use for complex, composite services. Use as a host when serving as RFC endpoint . SAP Interoperability in Action Adventure Works has been evaluating their strategy for accepting orders in different ways across their organization. They have many different channels that they accept orders from their customers and partners: Adventure Works has an on premise POS that allows orders to be placed in the retail stores. Adventure Works allows their customers to place orders online via their eCommerce website. Adventure Works lets their business partners place orders via electronic data interchange (EDI). There are also custom Web services and file formats they support for one-off scenarios. No matter how the order is accepted, the order must be sent to SAP because that is Adventure Work’s system of record for sales and distribution information. Currently, each channel has its own way of communicating with SAP. There is a lot of cost involved in maintaining these different mechanisms. When the business requires new data to be supplied by each system it requires changes that cascade across all systems. This greatly reduces the agility of the organization and usually kills strategic business initiatives before they even start. Adventure Works has an on premise POS is a.NET application that uses a 3rd party product to send order information to SAP. It runs on a nightly basis and relies on flat files being set via FTP to a central server for further processing. From an eCommerce standpoint, Adventure Works currently runs SAP Internet Transaction Server (ITS) and only do online sales with their closest partners. They have very complicated pricing rules based on negotiated prices and the volume of parts being ordered (among many other variables). AdventureWork’s marketing department has asked the IT department for a much more dynamic interface and the ability to run their own marketing promotions and maintain some product information. They feel that they can begin selling direct to consumers and bring more business value to their organization by selling their products in a different way. They have decided to implement Microsoft Commerce Server 2009 to provide the features they need and decouple the site from 15 SAP. However, integration between Commerce Server and SAP is very critical to their business and they need a solution for this. Adventure Works does very little EDI currently. In fact, when an EDI order is received, it is reentered into the SAP system by customer service representatives. They would like to automate this process in the future to reduce workload and guarantee better accuracy. Because each of these channels are processed by different processes, business rules are accessed and executed in many different ways. Customer service representatives must go through extensive training to handle all of the exception scenarios and how they need to be handled (usually requiring direct changes in the SAP GUI). Another challenge with the diverse set of order processing rules is the ability to gain visibility across all of the channels. There is really no way to pull the data together across all of the channels to gain the business intelligence their executives need to make decisions about product lines and methods of distribution. The business began to realize that the lack of business agility was making them less responsive to their customers. One of their customers noticed that pricing was not consistent across their EDI and Web orders. Not only was this an embarrassing situation but it cost Adventure Works real dollars as they had to refund the difference over several months. Executive management decided it was time to solve these problems and asked their enterprise architecture team to frame up a solution that met the following goals: SAP is still the source of record for orders Business and pricing rules must be 100% consistent across all channels A flexible solution that was adaptable to change must be created, and the change should be isolated to a single process to avoid inconsistency The management team wanted real time information on the ordering process to give them immediate visibility into their customer’s orders Adventure Works enterprise architecture team began discussing ways to handle these requirements. The team saw 2 large pending challenges in their discussions: 1. What was the best way to integrate data between SAP and each of the channels receiving orders. 2. How could the team avoid building tightly-coupled point to point interfaces between the systems involved now and ones that will most likely be added in the future. The team decided that an integration server was the best approach to solve these challenges and that BizTalk Server would be the tool of choice. They knew that it had adapters for Commerce Server, SAP, and many more for future integrations. They also knew that because of their development’s team knowledge of .NET and Visual Studio, that this would be the most rapid development platform for the task at hand. 16 The team quickly realized there was some inherent complexity in working with SAP. SAP contained most of the pricing and tax rules and was the source of truth for inventory, products, and orders. There was also information that was specifically e-Commerce related that didn’t fit nicely in SAP. Adventure Works decided on the following guiding principles for their master data: Customers: Customers must reside in SAP. For anonymous/unknown customers, a generic SAP customer will be used. Products: All products must reside in SAP. However, product ids will be integrated over on a regular basis. Additional information will be available in the POS and eCommerce system and stored in the in each respective systems catalog. Inventory: Inventory information remains in SAP and needed to be accessed in real time to avoid conflicts Orders: Orders needed to be verified and processed in SAP. Because the customer service group could place orders in SAP as well, all interactions with orders needed to happen with real time information from SAP. The following is a representation of the order confirmation page of the Adventure Works online store, this is the page that is displayed after orders are submitted to SAP: The following is an example of an order being placed via EDI: 17 ISA*00* *00* *ZZ*MPS *091217*0847*U*00405*000000371*0*T*:~ GS*PO*ZZ*1234*091217*0847*371*T*00405~ ST*850*0371~ BEG*00*SA*4e97c28e-5956-414b-9**20100115~ PER*IC*John Smith*TE*312-555-1212~ N1*ST*Johns Sporting Goods*92*40000~ N3*1234 Main~ N4*Atlanta*GA*66666~ PO1*1447*1*EA*400~ PO1*1449*1*EA*400~ SE*9*0371~ GE*1*371~ IEA*1*000000371~ *ZZ*AWC It is beyond the scope of this document to discuss the integration of all the data entities listed above, so we will focus specifically on the order process and how that data is submitted to SAP from each system. Building the Order Submission Process As stated previously, BizTalk Server has adapters for both Commerce Server and SAP. It also provides the ability to handle EDI documents from Adventure Works’ partners. However, each has their own format and schema for the Order process. For the greatest flexibility, we have defined an Adventure Works canonical order model. The canonical model allows us to define an order schema that meets Adventure Works’s needs in the vocabulary that Adventure Works uses. It is often tempting to just use mapping to translate from the source system straight to the SAP schema. This would create a pointto-point interface and not leverage the most robust features of the integration platform. Using BizTalk Server we can create a much more loosely coupled solution around our canonical schema. In fact, we will soon show how by using this model we can extend our order solution to other systems. The following diagram shows a logical representation of our intended integration between the systems. 18 So the first step is to define our canonical order schema. This schema contains all information that Adventure Works requires to represent an order. We created the schema using Visual Studio with the BizTalk Schema Designer as represented below: We then have to prepare our schemas that represent messages coming from our source systems. The POS system will be using a WCF call to pass orders to BizTalk Server. 19 We can “publish” our canonical schema as a WCF service. Commerce Server has a command line tool to build the schema for us (OrderMapping.exe). For the B2B transactions, BizTalk Server provides the schema out of the box. The Commerce Server and EDI schemas are pictured below: Commerce Server Order EDI 850 (Purchase Order) Each system will pass its message to BizTalk Server in its own format. In our case we then need to create a map that converts these different formats to our canonical. We can put this map on the receive port so the when the message is submitted it is transformed to our canonical before it ever makes it to BizTalk Server. The following is a sample of a map from the EDI format to the canonical order: 20 As far as processing the order message, we have decided to create an orchestration to send the message to SAP. We have done this so that we can check the status from SAP and potentially kick off a workflow if the order processing fails. The first step however is to generate the SAP Schemas. We do this by going into Visual Studio and use the “Add Generated Items | Consume Adapter Service” wizard. This then prompts us for a URI that represents a connection to the SAP system. After this is filled out we can browse for the RFC or BAPI we would like to execute. In our case, we are going to navigate to BAPI->Sales and Distribution->Sales->Sales Order. We select only the operations that we need. 21 When we click the OK button, it generates the following schema for use by BizTalk Server: We then can create a map to map from our canonical to the request schema defined by the BAPI_ORDER_CREATEFROMDAT2. The map will look something like this: 22 Now we can plug this information into our orchestration and execute the process when we receive our canonical order message. We chose to do implement this as an orchestration because there are multiple SAP calls that must be executed to place the order in SAP. Specifically, a BAPI_COMMIT_WORK call must be made to commit the transaction within SAP. 23 Keep in mind, because we are taking this canonical approach, there are many other ways we could receive orders and still use the same orchestration to process the message. For example, we could process a flat file and map the records to our canonical and process them all in the same way. This really shows the flexibility of BizTalk Server. Extending the Order Submission Process Adventure Works has implemented the different interfaces to accomplish the integration between their different ordering channels and SAP. Adventure Works has also implemented Dynamics CRM to allow their sales force to get insight into the touchpoints with their customers. The idea has been suggested that as orders are placed that they be forwarded to the CRM system so the sales force can see those orders without going into SAP. Because of the loose coupled architecture we built with BizTalk Server, and the fact that there is an adapter for Dynamics CRM, the solution can be created nicely. 24 We need to generate the schema for the insert of Orders into Dynamics CRM. In CRM Orders and Order Lines are separate entities. We are going to focus on just the order header at this time. We must use the “Add Adapter Meta Data” wizard to create our schemas. We go through the following screens to determine which entities to process: We can now map our canonical order schema to our order header insert request. We then 25 create an orchestration to insert the order. This orchestration has a similar filter as the one to SAP. This means when an order comes in from Commerce Server, both systems will be updated. Now once everything is enlisted and started, the orders will flow from Commerce Server into SAP and Dynamics CRM. Adding Real-time Visibility into Orders Now that Adventure Works has implemented the integration of order information across the enterprise, management has requested that they get visibility into the number of orders coming into the system for customers in different regions or states. In normal situations, this would require another interface be built to send the data to a data warehouse system. Because we are using BizTalk Server, we can use the business activity monitoring (BAM) features to quickly provide that data. This solution can be created without redeveloping or redeploying the original solution. First we need to create our activity definition. This defines what information we want to track. In our case we are going to track: Customer, State, Order Amount, Order Date, and Channel. 26 We then define how we want to display the data by creating a view. We define the dimensions and how we want to aggregate the data. We are using a real time aggregation to allow our users to see the data as soon as it is updated. 27 We then deploy this information to BizTalk Server using a command-line tool (bm.exe). This creates the infrastructure required for our data to be tracked. This is referred to as the observation model. Now that the observation model is deployed, we need to tell BizTalk Server how to track the data. We do this using the Tracking Profile Editor Tool. We are going to tell BizTalk Server we want to track the order data within our orchestration. We do this by choosing a shape and schema and then map the data to the appropriate data item in our activity. Now that that is all configured and applied, data will begin to be tracked. Remember this did not involve any redevelopment of our solution. Now that data is flowing through the system, our managers can view the data using the BAM Portal or we could create SSRS reports to deliver the information to them. 28 Conclusion The Microsoft platform, and specifically BizTalk Server, can provide a robust platform for extending the reach of the SAP Platform. By taking a loosely-coupled, service oriented approach, BizTalk Server can facilitate flexible communication between Microsoft-centric applications and SAP. It can offer the most productive, cost effective and easy to manage automation & integration solutions for SAP customers providing the following advantages: Comprehensive automation capabilities: Orchestration, Messaging, Business Rules and SAP Connectivity (iDoc, RFC, iView, etc). This allows organizations to synchronize SAP information/data with other applications and automate crossapplication and cross-functional processes with ease. Business Activity Monitoring help customers gain real-time operational insight into those processes. Productive and rapid development with Visual Studio: Development teams have to learn only one development tool and one programming framework/interface (.NET) Cost Effective Solutions: BizTalk Server with its unified development and management environment and integrated capabilities and wide availability of partners and .Net skills makes it very cost effective automation and integration solution for SAP customers and allow them to achieve business objectives in the most rapid time possible. 29 Using the Microsoft platform and the SAP platform together can provide advanced solutions in almost any organization and give the best of both worlds. The Microsoft platform can work right along-side the SAP platform and extend the reach beyond the boundaries of SAP, providing business users the solutions they need in the tools they use every day. About the Authors Chris Kabat and Naresh Koka are both principals for MPS Partners, a Chicago based services firm that focuses on turning vision into value through people, empowered by technology. Chris serves as a Principal Architect and is the Vice President of MPS Partners’ Connected Business Systems division. He is also a member of the Microsoft BizTalk Server Virtual Technology specialist team. Chris has been working with the integration of SAP systems for over 10 years. Naresh Koka serves as a Principal Architect and is the Vice President of MPS Partners’ Industry Group with a specific emphasis on SAP Integration. He is a member of the Microsoft SAP Alliance virtual team and has also worked with the integration of SAP systems for over 10 years. 30