ERP ON THE WIRE Enterprise Resource Planning Applications and Their Effects on the Network www.GanymedeSoftware.com CONTENTS Introduction * ERP Architecture * Two-tier Implementations * Three-tier Client/Server Implementations Transaction Flows and Volumes * Looking at the Numbers * Deploying Throughout the Enterprise * Local Area Network Performance * * Wide Area Network Performance – T1 * Wide Area Network Performance – 56Kb * Wide Area Network Performance – 28.8Kb * Putting the Results in Perspective * Conclusions * How Ganymede Can Help * Introduction "In the beginning God created the heavens and the ERP." – ORACLE 1:1 Enterprise Resource Planning (ERP) applications might not have had divine origins, but there is little doubt that somewhere, right now, a business or technology professional like yourself is praying for a successful ERP deployment. As the name implies, ERP software is a set of applications that ultimately are relied upon to automate and integrate many or all functions of an enterprise. Common components include Finance modules, Human Resources software, as well as Manufacturing and Logistics applications. After a successful roll out, ERP applications offer a variety of returns on the initial investment. Benefits might include more efficient business processes, tighter cost controls, and better customer service. However, these gains do not happen overnight. ERP applications are notoriously complex and usually require extensive customization. Recognizing these gains in your enterprise will require significant investments in time and money. Business processes may need to change, and users will need time to become efficient in using the new system. In the middle of planning for and making these changes, there is one item that often gets overlooked: The network. "Systems" people spend a good deal of timing sizing servers and figuring out how many users they can support. "Desktop" folks plan for the software distribution on the client side. Department heads discuss when and how their people will be trained to use the new application. In many cases it is simply assumed that the network will handle this application just like it handles every other application. Is this a fair assumption? Do ERP applications behave like other network applications? The answer to both questions is - No! In a 1999 Computerworld cover story article entitled: ERP Pioneers, Keith Bearden, CIO of A-dec Inc., described some of the challenges he faced when he was tasked with addressing the performance issues associated with an ERP application which A-dec had recently purchased and deployed. It took about six months to fix the performance issues during which time they changed databases and performed server and network upgrades. Another six months was spent redesigning business processes and training end users – doubling the cost of the project. Mr. Bearden was quoted as saying: "We lost a lot of business, because [workers] didn’t understand it, and the performance was so bad." In this White Paper we will discuss the behavior of ERP transactions and the crucial role that the network plays in delivering acceptable levels of performance to the end users. We will detail the potentially dramatic performance impact associated with running these applications across slower, higher-latency links and demonstrate why an up-front analysis of network performance is critical. ERP Architecture ERP applications are most commonly deployed in a distributed and often widely dispersed manner. While the servers may be centralized, the clients are usually spread to multiple locations throughout the enterprise. Generally speaking, there are three functional areas of responsibility that is distributed among the servers and the clients. First, there is the database component – the central repository for all of the data that is transferred to and from the clients. Then, of course, the clients – here raw data gets inputted, requests for information are submitted, and the data satisfying these requests is presented. Lastly, we have the application component that acts as the intermediary between the client and the database. Where these components physically reside and how the processes get distributed will vary somewhat from one implementation to the next. The two most commonly implemented architectures are outlined below. Two-tier Implementations In a typical two-tier architecture, the server handles both application and database duties. The clients are responsible for presenting the data and passing user input back to the server. While there may be multiple servers and the clients may be distributed across several types of local and wide area links, this distribution of processing responsibilities remains the same. Figure 1-1 below provides a simple illustration of a two-tier implementation. Three-tier Client/Server Implementations In three-tier architectures, the database and application functions are separated. This is very typical of large production ERP deployments. In this scenario, satisfying client requests requires two or more network connections. Initially, the client establishes communications with the application server. The application server then creates a second connection to the database server. Figure 1-2 illustrates this type of implementation. Transaction Flows and Volumes So far, this all looks pretty unremarkable, right? We have clients talking to servers, servers talking to other servers, no big deal… After all, in the vast majority of cases, there are several other applications already in place that are doing similar things in terms of transactions flows long before the ERP application enters the picture. There are PC, Mac, or UNIX clients talking to web servers, sending and receiving email, transferring and printing files, running legacy host-based applications, etc. etc. Pay attention now, we’re getting to the good part! It isn’t the pattern of the transaction flows that we are worried about here. It is the shear volume of constant back-and-forth transactions traveling between the client and the server(s). On a high-speed, low-latency link, this may not present a problem. In fact this is why performance problems may not surface until after the application roll out is well underway. During the initial development and pilot testing of the system, the clients and the servers are often sitting on an isolated network or a single subnet. In this environment, the end-users may experience no performance problems whatsoever. But what happens when the application gets rolled out to other buildings on the campus, branch offices, or remote users? Looking at the Numbers So how much client to server interaction are we talking about here? Well, keep in mind that most ERP applications are customized for each individual customer, so the results you see will in all likelihood be unique to your organization. But let’s try to put things in perspective. In building Application Scripts that emulate real application transaction flows, Ganymede Software has done extensive analysis of how various applications perform "on the wire." Here are some of the results we have observed. As a baseline, let’s start with a typical network transaction such as sending an email message using SMTP (Simple Mail Transfer Protocol). From the network perspective, this type of transaction will result in about a dozen "sends" of data to and from the email client. By "send" we mean a transmission of data that is initiated by either the client or the server. So how does this compare to a typical ERP transaction? Here are some sample transactions from the top five ERP applications and the number of sends associated with those transactions: Transaction Client-to-Server Sends Server-to-Server Sends Total Sends Add employee – Two tier 242 N/A 242 Add customer – Three tier 83 411 494 View customer – Three tier 105 455 560 Journal entry – Three tier 179 521 700 Journal entry – Two tier 715 N/A 715 Maintain P.O. – Two tier 768 N/A 768 Maintain Sales Order – Two tier 858 N/A 858 Add Inventory Item – Two tier 988 N/A 988 What’s going on here? Why are there so many exchanges taking place over the wire? Keep in mind that ERP applications and the individual transactions themselves can be quite complex. A single transaction such as adding an inventory item may require the end-user to fill out several forms of information. As each form in completed, data is transmitted back and forth across the network. Each of these forms has several fields that must be filled in. This is done by typing in text or perhaps selecting an item from a drop-down menu. In some cases we have observed that simply displaying these menus or even tabbing from one field to the next results in data being transmitted between client and server. Note the difference in the traffic flows between two-tier and three-tier environments. Three-tier environments provide unique performance management challenges due to the number of variables in the end-to-end equation. When end-users complain of poor "application" performance, isolating the problem to a particular server or network connection can be a difficult task. You are likely to find that the behavior of these transactions on the network is quite unlike anything else you have ever seen. If you were under the assumption that ERP applications behave like any other application on your network, you can throw those assumptions out the door! Keep in mind that the numbers listed above are representative of a single transaction being executed by a single user. Imagine what will happen as the application scales and dozens or hundreds of users are executing a mix of these transactions simultaneously across an already busy network. Armed with this information, let’s take a stab at answering the question that was raised previously: What happens when the application gets rolled out to other buildings on the campus, branch offices, or remote users? Deploying Throughout the Enterprise The test results in this section were generated using Chariot from Ganymede Software Inc. and The Cloud from Shunra Software Ltd. Chariot was used to emulate the network flows and volumes associated with a typical ERP transaction. The Cloud was used to emulate several different WAN environments. In each of the tests below, we used Chariot to emulate the "Add Employee" transaction listed in the above table. This transaction is comprised of 242 sends between the client and the server. We executed this transaction 10 times during each test. From an overall performance perspective, the Add Employee transaction could be described in the following manner: Total transaction time = Client data entry time + Server processing time + Network transmission time If you take the total amount of time needed to complete this transaction and remove the server time and the client time from the equation, you are left with the network transmission or response time. Regardless of how fast the server can run the application and look up data or how fast the user can type, the network component will remain fairly constant unless the makeup of the network itself or the traffic volumes change. Since the subject of this paper is the network component, our tests focus on the network response time. Local Area Network Performance For our first test, we used Chariot to emulate the Add Employee transaction over a 100Megabit LAN connection. Here are the results: Average Response Time Minimum Response Time Maximum Response Time Total Measured Time 6.20390 seconds 5.80800 seconds 6.51000 seconds 62.039 seconds Wide Area Network Performance – T1 Next, we used The Cloud to emulate a T1 link between the client and the server. The same test yielded the following results: Average Response Time Minimum Response Time Maximum Response Time Total Measured Time 10.46050 seconds 10.01400 seconds 10.71600 seconds 104.605 seconds So we jumped from 6.2 seconds on our LAN test to 10.5 seconds on the T1. At first glance, that doesn’t look too bad. But let’s examine this scenario in more detail. In this test, we are simulating a user who had an "empty" T1 all to himself – no other traffic and no other users. In reality, this is a shared resource and each user will only get a small piece of the total available pipe. So while 10.5 second response times are theoretically possible, perhaps during off-peak times, this is not a realistic expectation for day-to-day production transactions. Wide Area Network Performance – 56Kb For our third test, we executed the same transaction over a simulated 56Kb link: Average Response Time Minimum Response Time Maximum Response Time Total Measured Time 38.95420 seconds 38.56100 seconds 39.55800 seconds 389.542 seconds Once again, this test assumes that the pipe is empty and that a single user has access to the full resource. In reality, this starts to give you a little better feel for what you might expect as a user on a shared T1 line. Our 6.2 seconds of network time for each transaction has now climbed to close to 40 seconds. Wide Area Network Performance – 28.8Kb To satisfy our curiosity, for our final test we simulated a 28.8Kb dial-up link: Average Response Time Minimum Response Time Maximum Response Time Total Measured Time 78.13370 seconds 76.51800 seconds 80.40900 seconds 781.337 seconds This test demonstrates the type of response times that a user could expect if they were accessing the application over a dial-up modem link. Our maximum response times are now over 80 seconds. A user on a busy T1 might expect to see similar results. Putting the Results in Perspective Let’s think about this from the perspective of the end-user of the application sitting at the other end of that 56Kb link. They call the help desk and complain about performance. Help desk personnel PING the server and PING the enduser. In both cases they get response times that are well within reason, perhaps 100ms. The conclusion is that this isn’t a network problem and the trouble ticket gets handed over to the "Systems guys." A single PING simply does not tell the story here; in fact it can be very misleading. The reality of the situation is that for each transaction, the network is introducing at least 40 seconds of delay. Another key point: In our tests, we were assuming "clean" Wide Area Network connections. In the real world there are any number of anomalies that might occur throughout the course of the business day: Congestion, packet loss, out-oforder packets, duplicates, fragments, bit errors, etc. Any of these conditions will adversely impact the results that we have shown here. Lastly, let’s not overlook the fact that the transaction that we tested with had only 242 sends. Some of the transactions that we have analyzed have over four times this number. Conclusions 1. ERP applications and transactions are probably unlike anything else that you have running on your network. It is not safe to assume that the network will handle them well. 2. Every ERP deployment is different. The application modules are customized and the architecture of the system and distribution of computing resources will be unique to your environment. 3. It is critical to understand how your application performs on the network before beginning enterprise-wide deployment. 4. Test your network in advance to understand how these applications will perform and what impact they will have on existing applications. Pay particular attention to WAN links and heavily utilized LANs. 5. The location of computing resources may have a significant impact on how the application performs, particularly in three-tier environments. Understand the traffic flows between the client and the server as well as the server-to-server communication flows. How Ganymede Can Help Ganymede Software’s family of performance monitoring and management solutions can take much of the guesswork out of ERP deployments and help eliminate surprises once the application begins to scale. Using Chariot, you can now answer pre-deployment questions such as: "How well will this application run on my network?" or "What will this application do to the response time of existing applications running on my network?" Chariot provides a simple, yet scaleable way to emulate a single user, or hundreds of users, running a variety of applications and transactions to and from any point on your network. When a performance problem occurs, Chariot allows you to immediately isolate whether or not the problem is network related. This can dramatically reduce the time, effort and expenses associated with troubleshooting problems and provide an even bigger payoff in terms of user productivity gains. Since ERP application performance is critical, many corporations have invested in traffic queuing technology and prioritization tools in an attempt to provide endusers with a guaranteed Quality of Service (QoS). Chariot provides the means to verify that these technologies are working correctly in your environment and the performance data you need to "tune" end-to-end performance. Since every ERP application is unique, Ganymede has developed a tool that can provide Application Scripts that are customized to match your environment. Application Scanner captures application transaction flows and automatically produces Application Scripts that will precisely emulate the way your application performs. ERP applications can help your organization improve efficiency, reduce costs, and realize dramatic improvements in customer service. With proper up-front planning your implementation will be a success. Good luck! For more information on Ganymede’s Enterprise Performance Management solutions, please visit our web page: http://www.GanymedeSoftware.com.