Troubleshooting BizTalk Server Soap Adapter Microsoft Corporation Published: November 2008 Author: Shailesh Agre, Larry Franks, Dwaine Gilmer, Larry Meadows, Everett Yang Summary Microsoft® BizTalk® Server uses the SOAP adapter and orchestrations to receive and send Web service requests. The SOAP adapter enables orchestrations to be published as Web services and consume external Web services. Because configuring and/or using Web services can be complex, BizTalk Server handles many of these complexities on behalf of the user. Even so, there are still issues arising from time to time. This white paper is designed to highlight common issues and to provide guidance for how to resolve them. Copyright Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, BizTalk, and Visual Studio are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners. Contents Troubleshooting BizTalk Server SOAP Adapter .............................................................................. 5 IIS, ASP.NET and Web–Related Issues ...................................................................................... 5 Application Pool Recycling ....................................................................................................... 6 Orchestration Application Domain Unloading ........................................................................... 6 SecondsIdleBeforeShutdown ................................................................................................ 6 SecondsEmptyBeforeShutdown ............................................................................................ 7 BizTalk Server Upgrades and .NET Versioning ....................................................................... 7 IIS Security Issues .................................................................................................................... 7 SOAP Message Timeouts ........................................................................................................ 8 Response Message Timeout ................................................................................................. 8 SOAP.ClientConnectionTimeout ........................................................................................... 8 Request Message Timeout ................................................................................................... 8 ASP.NET Execution Timeout ................................................................................................ 9 executionTimeout .................................................................................................................. 9 HTTP Preauthentication ........................................................................................................... 9 Redirection ................................................................................................................................ 9 Performance ........................................................................................................................... 10 Threading Issues ................................................................................................................. 10 Avoid Thread Starvation .................................................................................................. 10 ASP .NET Settings Related to HTTP, SOAP, or WSE Adapter Performance .................... 12 Maxconnection Attribute .................................................................................................. 12 maxconnection ................................................................................................................. 13 maxWorkerThreads, minWorkerThreads, maxIoThreads, minIoThreads ....................... 13 maxWorkerThreads.......................................................................................................... 14 minWorkerThreads........................................................................................................... 14 maxIoThreads .................................................................................................................. 15 minFreeThreads and minLocalRequestFreeThreads Properties .................................... 15 Antivirus Software................................................................................................................ 17 File Locks ......................................................................................................................... 17 Application Domain Restarting ......................................................................................... 17 Latency ................................................................................................................................ 17 Web Services .......................................................................................................................... 18 Exposing an Orchestration as a Web Service ..................................................................... 18 Publishing an Orchestration as a Web Service: An Example ............................................. 18 Using Client Certificates with BizTalk Server ...................................................................... 22 Client Certificate Installation ............................................................................................ 22 Organization Security Restrictions ................................................................................... 24 Consuming Web Service Using a Send Port in Messaging–Only Scenarios...................... 25 Simple messages ............................................................................................................. 25 Multipart messages .......................................................................................................... 26 Consuming Web Services with complex Web Service Definition Language (WSDL) ........ 27 Accessing SOAP Headers from an Orchestration .............................................................. 28 Unknown SOAP Headers .................................................................................................... 29 RPC–Style Web Services .................................................................................................... 29 Tools ....................................................................................................................................... 29 Troubleshooting BizTalk Server SOAP Adapter Note To download a copy of this guide, go to http://go.microsoft.com/fwlink/?LinkId=128814. The SOAP adapter is a powerful integration mechanism for Microsoft® BizTalk® Server. It enables BizTalk Server to connect to Web services and the Windows Communication Foundation (WCF) framework by receiving and sending Web service requests. On the receive side, the SOAP receive adapter creates a BizTalk message object, and promotes the associated properties to the message context. On the send side, the SOAP send adapter calls into a Web service. The SOAP send adapter then reads the message context on the BizTalk message object to get the proxy name and then calls the associated external Web service proxy. The following illustration shows how the SOAP Receive and Send adapters work with BizTalk Server 2006 and BizTalk 2006 R2. Note This paper applies to BizTalk Server 2006 and BizTalk Server 2006 R2. The topics in this paper apply to both versions unless specifically noted. When BizTalk Server processes a message, many components work together, such as Internet Information Services (IIS), Active Server Pages .NET (ASP .NET), SOAP adapter, orchestrations, remote Web services, and so on. Because of the variety of technologies, performance considerations, and business scenarios, configuring the SOAP adapter appropriately can be complex. This white paper is explains these common issues and provides guidance on solutions. IIS, ASP.NET and Web–Related Issues By default, the receive SOAP adapter loads into the IIS process space (also called an isolated host) to use Hypertext Transfer Protocol (HTTP). This tightly integrates the performance and configuration of IIS to the SOAP adapter. The following topics are covered in this section: Application pool cycling Orchestration application domain unloading BizTalk Server upgrades 5 IIS security issues SOAP message time outs ASP .NET execution time outs HTTP pre-authentication HTTP redirection Application Pool Recycling The receive SOAP adapter uses IIS to establish an HTTP session with a remote Web client. When the HTTP session is closed or broken, the failure is handled appropriately in the SOAP adapter. This type of failure can occur for several reasons, including the following: IIS was recycled. This closes the HTTP session The client application HTTP request has timed out A network failure closed the HTTP receive session The SOAP Adapter’s application pool configuration can cause problems if is set to cycle. The application pool for the SOAP adapter should not be configured to recycle during processing and the Idle Timeout setting should be configured to “off.” For more information about how to configure the IIS application pool, see “How to modify Application Pool Recycling events in IIS 6.0” at http://go.microsoft.com/fwlink/?LinkId=115054. For more information about how to configure the Idle Timeout setting, see “Recycling Application Pool Settings” at http://go.microsoft.com/fwlink/?LinkID=93510 and “Recycling Worker Processes” at http://go.microsoft.com/fwlink/?LinkId=122780. Orchestration Application Domain Unloading The SOAP adapter is often used with an orchestration. The performance of the orchestration is part of the overall performance of the solution. When the application domain is not loaded into memory, this can cause timeouts to occur. The default behavior of the application domain for an orchestration is to shut down if idle more than 30 minutes (SecondsIdleBeforeShutdown) or if the application domain is empty more than 20 minutes (SecondsEmptyBeforeShutdown). In some situations it might be beneficial to keep an orchestration application domain in memory. These settings apply only to orchestrations. By default, BizTalk Server has the following two shutdown settings: Note To override the Default attribute, enter a value for the Max and/or Min value(s). SecondsIdleBeforeShutdown SecondsIdleBeforeShutdown is the number of seconds that an application domain is idle (that is, it contains only dehydratable orchestrations) before being unloaded. Specify a Max value of “-1” to signal that an application domain should never unload when idle but not empty. When an idle but non-empty domain is shut down, all of the contained instances are dehydrated first. 6 Location: BTSNTSvc.exe.config Enumeration: milliseconds Default: 1800 Max: -1 (infinite) Min: 0 Example: SecondsIdleBeforeShutdown = 1800 SecondsEmptyBeforeShutdown Location: BTSNTSvc.exe.config Enumeration: milliseconds Default: 1200 Max: -1 (infinite) Min: 0 Example: SecondsEmptyBeforeShutdown = 1200 To keep the application domain loaded in memory, configure SecondsIdleBeforeShutdown and SecondsEmptyBeforeShutdown to -1 (this value is equivalent to 9,223,372,036,854,775,807 ticks). However, the trade off is overall memory usage will increase. For more information about orchestration application domain unloading, see “Orchestration Engine Configuration” at http://go.microsoft.com/fwlink/?LinkId=104421. BizTalk Server Upgrades and .NET Versioning If you upgrade from BizTalk Server 2004 to BizTalk 2006, you may encounter issues with .NET versions. Because BizTalk Server 2004 used .NET Framework version 1.1 and BizTalk Server 2006 uses a newer version, a mismatch of the load application pool can occur. For example, an orchestration exposed as a Web service encounters a failure due to the shutdown of the SOAP adapter application pool. This occurs when the IIS virtual directory is configured to use the ASP.NET 1.1. BizTalk Server 2006 requires ASP.NET 2.0. To fix this problem, change the default ASP.NET version settings. For more information, see “Locations Tab, ASP.NET Configuration Settings Dialog Box” at http://go.microsoft.com/fwlink/?LinkId=115051. IIS Security Issues Security settings in IIS are another area with where you may experience configuration issues. When you set up the SOAP adapter, follow the BizTalk Server 2006 documentation guidelines to minimize these issues. For more information, see SOAP Adapter Security Recommendations at http://go.microsoft.com/fwlink/?LinkId=118344. Specifically follow the security recommendations for securing IIS. If IIS 6.0 is installed, follow the recommendations for configuring application isolation. For more information see “Isolating Web Sites and Applications” at http://go.microsoft.com/fwlink/?LinkId=25222. If IIS 5.X is installed, see “From Blueprint to Fortress: A Guide to Securing IIS 5.0” at http://go.microsoft.com/fwlink/?LinkId=24776. 7 SOAP Message Timeouts The following section explains the various types of SOAP timeouts that can occur and how to address them. Response Message Timeout In some situations, response message timeouts can occur with the SOAP adapter. When using a request-response SOAP port, a Web service may timeout before the orchestration completes. This can occur when an orchestration takes longer than the time-out settings on the Web client. This can also occur on a solicit-response when a Web service takes too long to send a response back to the SOAP adapter. If this is the case, the following message appears in the Event Log: The SOAP.ClientConnectionTimeout can be used on a Web service taking a long time to return a response. Only the SOAP.ClientConnectionTimeout property can be changed for all port binding types. To prevent this timeout on a Send port, set the time-out period for the client connection to value to a greater than expected completion time. SOAP.ClientConnectionTimeout Note To override the Default attribute, enter a value for the Max and/or Min value(s). Location: message context property Enumeration: milliseconds Default: 900000 Max: Min: Example: MessageResponse(SOAP.ClientConnectionTimeout) = 900000; The event log error from a SOAP Request Response failure is as follows: This error occurs because the Web Service closed the connection before the response was delivered. To fix this issue, extend the time that the Web service waits for a response before timing out. On an IIS Server, make this change in the IIS Web config file. You can also increase the execution timeout value to the number of seconds that ASP.NET will wait before closing the connection. For more information, see “httpRuntime Element (ASP.NET Settings Schema)” at http://go.microsoft.com/fwlink/?LinkID=122828. Request Message Timeout When calling a Web service with slow response, the timeout can be adjusted by using the context property as shown in the following example: WSRegistrationRequest(SOAP.ClientConnectionTimeout) = 900000; 8 ASP.NET Execution Timeout The third timeout value depends on the ASP.NET executionTimeout property timeout settings. When you increase the overall transaction timeout, adjust the executionTimeout value to receive all the expected responses. For more information about configuring the timeout value, see “executionTimeout” at http://go.microsoft.com/fwlink/?LinkId=122121. executionTimeout Location: machine.config file orweb.config fileEnumeration: TimeSpan attribute Default: 110 Sample: <configuration> <system.web> <httpRuntime executionTimeout=”00:05:00”> </system.web> </configuration> HTTP Preauthentication Communication between the SOAP adapter and an HTTP client can fail if HTTP preauthentication is configured incorrectly. IIS supports many authentication options, of these, Basic and Kerberos support the concept of preauthentication. The basic concept of authentication is when a client requests access to a protected resource on the server, the server challenges the client to provide valid credentials. Once the client provides the credentials, the server grants the client access to the resource. For more information, see “HttpWebRequest.PreAuthenticate Property” at http://go.microsoft.com/fwlink/?LinkId=122829. Web Services and Web servers may not always support pre-authentication. For example, it may be disabled on the Web server. When using the SOAP adapter on a Web server configured to support preauthentication, the first request must contain a valid set of user credentials. On occasion, some Web services will return a status code of 500 Internal Server Error instead of the expected response of 401. To prevent issues with the SOAP adapter and unexpected responses from Web services, in an orchestration, create a custom component and call it from an Expression shape. For more information, see the sample code by Feroze Daud that forces credentials to be sent on the first request at http://go.microsoft.com/fwlink/?LinkId=115052. Redirection When a SOAP adapter issues a request to a Web service, it is possible for the HTTP request to be redirected to another server. The initial HTTP Request results in a HTTP redirection class response of a status code of 3XX. In IIS, these response status codes can result in a status code 9 of 401, unauthorized access returned to the SOAP adapter. When this error occurs, BizTalk Server will not attempt to navigate to the redirected URI in the HTTP 3XX status code response. To verify if the request was redirected, reconfigure the SOAP adapter to post directly to the URI indicated in the redirect response and see if the request succeeds. There are two possible solutions: Use a dynamic send port in an orchestration and consume the Web service. When Web service returns the redirect request with 302/307 status, it comes as an HTTP exception to the SOAP adapter. The orchestration can catch the general exception and extract the new URI. Then use the dynamic send port to resend the request. Create a .NET component to consume the Web service in an orchestration. Design the component to handle the exception and to resend the request to the new URI. Performance The topics below describe some factors that may impact the performance of SOAP, HTTP, or WSE Adapters. Threading Issues A thread is the basic unit for an operating system to allocate processing time. A process—such as a BizTalk host instance—represents an executing program and contains one or more threads in its context. Thread starvation can cause slow performance with the SOAP, HTTP, or WSE adapters. A large volume of messages (floodgate) can trigger performance degradation with these adapters. This is especially true if a large number of messages arrive immediately after an adapter was initialized. The root cause is often the system needs to be tuned to handle the load instead of a hardware limitation. Within an instance of a BizTalk host where SOAP, HTTP, or WSE Adapters are running, worker threads handle queued work items, and I/O threads dedicated callback threads associated with I/O completion ports for completed asynchronous I/O requests. Performance problems occur when there are not enough free threads in the thread pools to handle the number of messages (SOAP, HTTP, or WSE requests). When a BizTalk host starts, the default thread pool sizes are small and cannot fulfill the requests in a floodgate scenario. As a result, the adapter must add more worker or I/O threads to the thread pool. This process can be time consuming. More worker or I/O threads are added until the requests can be fulfilled or the maximum thread limit is reached. If the maximum thread pool limit is not large enough to handle the sustained work load, messages must wait until a thread becomes available before they can be processed. Avoid Thread Starvation Set minimum worker and I/O thread pool size to a level appropriate to the initial and sustained work load. Maximal worker thread pool size should be set to accommodate possible peak burst load. Sustained and burst load should be defined by the business requirements and validated during your stress testing. We recommend that you do not define excessive thread pool sizes. 10 While large thread pools may resolve thread starvation, they can increase context switching, a situation where Windows switch from one running thread to the other. Excessive context switching can offset the performance gained Add CLR Hosting key to the previous listed registry path if the key does not exist already. Note To override the Default attribute, enter a value for the Max and/or Min value(s). MaxIOThreads Location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$Hostname Enumeration: count Default: 20 Max: Min: Key: DWORD MaxWorkerThreads Location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$Hostname Enumeration: count Default: 25 Max: Min: Key: DWORD MinIOThreads Location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$Hostname Enumeration: count Default: 1 Max: Min: Key: DWORD MinWorkerThreads Location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$Hostname Enumeration: count Default: 1 Max: Min: 11 Key: DWORD Calculate the value of the MinWorkerThreads DWORD entry, using the following formula: For example, if 50 messages are delivered to the SOAP adapter as soon as the adapter is initialized, set the value of MinWorkerThreads to 55. Note the following: MaxIOThreads and MinIOThreads should have the same values as MaxWorkerThreads and MinWorkerThreads. If MaxWorkerThreads or MinWorkerThreads are configured, but not MaxIOThreads or MinIOThreads, BizTalk Server sets MaxIOThreads and MinIOThreads to the values of MaxWorkerThreads or MinWorkerThreads. The values specified for the thread pools are per CPU. For example, setting MaxWorkerThreads to 100 has an effective value of 400 on a 4 CPU computer. These numbers are recommended starting points. Perform appropriate testing to determine the optimal setting for your business requirements. Also note the .NET thread pools BizTalk host instance can be shared by the orchestration engine, in addition to the managed adapters. To apply the thread pool changes only to a specific adapter, move the adapter’s handlers to a separate host. A larger thread pool can put more pressure on your Biztalk Server CPUs and the SQL Server hosting the BizTalk databases. Increasing the MaxWorkerThreads value beyond 100 can have an adverse effect on the performance of SQL Server. Make sure the computer(s) installed with SQL Server can handle the additional stress before increasing the thread pool size greater than 100. For more information about threading issues, see “FIX: Slow performance on startup when you process a high volume of messages through the SOAP adapter in BizTalk Server 2006 or in BizTalk Server 2004” at http://go.microsoft.com/fwlink/?LinkId=115057. ASP .NET Settings Related to HTTP, SOAP, or WSE Adapter Performance In addition to thread starvation, there are several ASP.NET settings affecting SOAP, HTTP, or WSE adapter performance. These settings are made in the web.config or machine.config files. Because these adapters communicate with external Web sites or Web services, consider tuning these external components as part of overall performance review. The settings described in this section are specific to BizTalk Server. The tunable attributes include maxconnection, maxWorkerThreads, minWorkerThreads, maxIOThreads, and minIOThreads. The properties that can be tuned include minFreeThreads and minLocalRequestFreeThreads. Maxconnection Attribute The maxconnection attribute in the machine.config file defines the maximum number of connections to a server or group of servers. The default value is 2. While this may be adequate 12 for a desktop application that makes a few concurrent requests, is can be a bottleneck if BizTalk Server makes a large number of HTTP, SOAP, or WSE requests. Note To override the Default attribute, enter a value for the Max and/or Min value(s). maxconnection Location: machine.config Enumeration: count Default: 2 Max: Min: Sample: <configuration> <system.net> <connectionManagement> <add address="*" maxconnection="2"/> </connectionManagement> … </system.net> </configuration> In most situations, the recommended value for maxconnection attribute is (12 * the number of CPUs). Test results have shown this setting works well in a variety of scenarios. Like any performance related setting, however, validate the setting for the specific scenario. Note Local requests are not limited by this setting (BizTalk Server calling a local Web service). Increasing maxconnection attribute increases thread pool and CPU utilization. Additionally, the thread pool settings described in the previous section may not be optimal. Test larger thread pool sizes to determine if adjustment is necessary. maxWorkerThreads, minWorkerThreads, maxIoThreads, minIoThreads These settings affect a Web application or Web service similar to a BizTalk host instance. Bottlenecks can occur when ASP.NET runs out of worker and I/O threads to process incoming requests or perform I/O work. To identify this bottleneck, monitor the following performance counters: ASP.NET\Requests Queued 13 Process\%Processor Time (select aspnet_wp.exe and/or w3wp.exe). If requests are queued while process utilization is slow, there is thread pool starvation. Note To override the Default attribute, enter a value for the Max and/or Min value(s). maxWorkerThreads Location: machine.config Enumeration: count Default: 20 Max: Min: Sample: <configuration> <system.net> <connectionManagement> <add address="*" maxWorkerThreads ="2"/> </connectionManagement> … </system.net> </configuration> minWorkerThreads Location: machine.config Enumeration: count Default: 1 Max: Min: Sample: <configuration> <system.net> <connectionManagement> <add address="*" minWorkerThreads ="2"/> </connectionManagement> … </system.net> 14 </configuration> maxIoThreads Location: machine.config Enumeration: count Default: 20 Max: Min: Sample: <configuration> <system.net> <connectionManagement> <add address="*" maxIoThreads ="2"/> </connectionManagement> … </system.net> </configuration> minFreeThreads and minLocalRequestFreeThreads Properties The minFreeThreads property specifies the number of free threads in the worker thread pool. This property ensures there are free threads available if an existing request has work requiring additional threads to process. Incoming requests are queued if the number of free threads in the worker threads pool falls below this property. Note To override the Default attribute, enter a value for the Max and/or Min value(s). minFreeThreads Location: machine.config Enumeration: count Default: 8 Max: Min: Sample: <configuration> <system.net> 15 <connectionManagement> <add address="*" minFreeThreads ="2"/> </connectionManagement> … </system.net> </configuration> The recommended value is 88 * number of CPUs. This allows for 12 concurrent requests per CPU with maxWorkerThread set to 100. Make sure that maxWorkerThreads and maxIOThreads settings from the previous section are greater than or equal to your minFreeThreads setting. minLocalRequestFreeThreads Location: machine.config Enumeration: count Default: 4 Max: Min: Sample: <configuration> <system.net> <connectionManagement> <add address="*" minLocalRequestFreeThreads ="2"/> </connectionManagement> … </system.net> </configuration> The minLocalRequestFreeThreads is similar to minFreeThreads, except it deals specifically with local requests (requests from local host). The default value for minLocalRequestFreeThreads is 4. For more information about prescriptive architecture, see “Chapter 10 — Improving Web Services Performance” at http://go.microsoft.com/fwlink/?LinkId=115056. For more information about tuning ASP.NET, see “How To: Tune ASP.NET” at http://go.microsoft.com/fwlink/?LinkId=115055. For more information about contention, performance, and deadlocks, see “Contention, poor performance, and deadlocks when you make Web service requests from ASP.NET applications” at http://go.microsoft.com/fwlink/?LinkID=66483. 16 Antivirus Software Virus scans on a BizTalk Server is a very important operational consideration. While the value of performing scans for known viruses is well documented, the affect on BizTalk Server is less well known. Sometimes, third-party programs run as a program or as filter drivers (driver/program/module inserted into the existing driver stack to perform some specific function) may interfere with the responsiveness of the BizTalk Server service. Two issues caused by antivirus software are file locks and the unloading of the application domain during virus scans. File Locks File lock errors may occur if an antivirus software program gains a file lock to the file BizTalk Server is attempting to use for processing. This error may occur if the antivirus software scans the virtual Web Service folders. The results are not always predictable and can vary from slow performance to access denied on certain files. To solve this issue, identify the file improperly locked and if the antivirus software allows, exclude the file from the scan. Application Domain Restarting XML Web service files are saved under the .ASMX extension. Like .ASPX files. These are compiled by the ASP.NET runtime when there is a request to the SOAP adapter URI. The ASP.NET application can restart frequently when antivirus software scans the files of the Web service endpoint on IIS. More specifically, restarts occur because antivirus software scans the Web.config file in the root of the application, the Machine.config file, the Bin folder, or the Global.asax file. For more information, see “Why is my ASP.NET application restarting?” at http://go.microsoft.com/fwlink/?LinkId=122883. Latency By default, BizTalk Server is configured to optimize throughput instead of response. The time BizTalk takes to respond is referred to as “latency.” In many cases it is important to minimize latency. BizTalk Server may be optimized for this. Keep in mind optimizing for latency can negatively impact throughput and memory utilization. It can also impact other BizTalk processing. There are several good sources of information regarding latency. Professional BizTalk Server 2006 (Jefford, Smith, & Fairweather) devotes an entire chapter on the topic. For more information, see “BizTalk Server 2004: Performance Tuning for Low Latency Messaging” at http://go.microsoft.com/fwlink/?LinkID=108777. Often overlooked in discussions of latency is the time it takes BizTalk Server to load orchestrations. Additionally, if the virus scanning includes the virtual folder of the low latency Web services each startup is delayed to allow for the just in time (JIT) process to complete. For more information, see the sections titled “Orchestration Application Domain Unloading” and “Orchestration Engine Configuration.” 17 Web Services BizTalk Server provides powerful support for exposing orchestrations as Web services and consuming existing Web services. The following information explains how to take advantage of these features and some of the fine points of dealing with the topic of Web services. Exposing an Orchestration as a Web Service The Web Service Publishing Wizard is used to publish an orchestration. It creates a Web service proxy hosted in IIS to handle the interface between the Web service interface and the BizTalk Messaging Engine. This proxy uses the operation name specified for the orchestration port as the Web method name. During receipt by the proxy, the Web method name is promoted into the MethodName property and used for routing the message to the orchestration. By default, any orchestration exposed as a Web service has a subscription expression similar to the following: Think of this as two subscriptions. The first subscription matches any document typed as this document and is not using the SOAP transport. The second subscription accepts any incoming message as long as it is stamped with the matching MethodName. Both are limited to documents received by the specified receive port. What this provides is a subscription for the Web service proxy. If the method name exposed by the proxy matches the operation name in the orchestration, a non-SOAP receive location can be added to the receive port for testing. The method name is promoted automatically from the receive handler, not the pipeline, so even using a “passthrough” pipeline the message is correctly routed. This is why it is important to sync the method name with the operation name when creating the Web service separate from the orchestration. There are several methods of publishing an orchestration as a Web service, and several issues may be encountered during and after this process. This section covers the steps for using the Web Services Publishing Wizard, solutions to errors, and describes how receiving the document and routing it to the orchestration process works. Publishing an Orchestration as a Web Service: An Example The following steps outline how to create and publish an orchestration as a Web service accepting incoming requests and returns a response document. 18 To create the orchestration 1. Open Visual Studio®. 2. In Visual Studio, on the File menu, point to New, and then click Project. 3. In the New Project dialog box, under Project types, select BizTalk Projects and then in Templates, click Empty BizTalk Server Project. In the Name of this project box, type WSOrchExample, and then click OK. 4. In Solution Explorer, right-click the project name, point to Add, and then click New Item. 5. In the Add New Item dialog box, in Categories, select Schema Files, and in Templates, select Schema. Name the schema schemaInputSchema.xsd, and then click Add. 6. In the design view for InputSchema, in the left pane, right-click the Root element and select Insert Schema Node, Child Field Element. Name the new field InputData, and then click the Save icon. 7. In Solution Explorer, right-click the project name, point to Add, and then click New Item. 8. In the Add New Item dialog box, in Categories, select Schema Files, and in Templates select Schema. Name the schema OutputSchema.xsd and then click Add. 9. In the design view for OutputSchema, in the left pane, right-click the Root element and select Insert Schema Node, Child Field Element. Name the new field OutputData and then click the Save icon. 10. In Solution Explorer, right-click the project name, point to Add, and then click New Item. 11. In the Add New Item dialog box, in Categories, select Orchestration Files, and in Templates, select BizTalk Orchestration. Name the orchestration WSOrchestration and then click Add. 12. In Orchestration View, right-click Messages and select New Message. In properties for the new message, set the Identifier as Message_Input and the Message Type to WSOrchExample.InputSchema. 13. Repeat the previous two steps, substituting the Identifier of Message_Output and the schema name of WSOrchExample.OutputSchema. To configure the Receive shape in the orchestration 1. In Visual Studio, open the toolbox window and drag the following shapes to the Orchestration Design Surface: Receive Transform Send 2. Select the Receive shape. In the Properties box, set the Activate property to True. Set the Message to Message_Input. 3. Select the ConstructMessage_1 shape generated around the Transform shape. In the 19 Properties dialog box, set Messages Constructed to Message_Output. 4. Double-click the Transform shape. In the Transform Configuration dialog box, set the Fully Qualified Map name to WSOrchExample.Mapping. 5. Under Transform, select Source. In the Source Transform list, add a new row with Message_Input as the Variable Name. 6. Under Transform, select Destination. In the Destination Transform list, add a new row with Message_Output as the Variable Name, and then click OK. 7. In the Map Editor, expand the Root element for the Source and Destination schemas. Drag the InputData element to OutputData. This generates a link between these two elements. Click Save. To configure the Send shape in the orchestration 1. Switch back to the Orchestration Designer for WCFOrchestration. Select the Send shape. In Properties box, set the Message to Message_Output. 2. Right-click the Port Surface and select New Configured Port. Click Next until you get to the Select a Port Type page. Set the Communication Pattern to Request-Response and the Access Restrictions to Public. Click Next. 3. Set the Port Direction of Communication to I’ll be receiving a request and sending a response. Click Next, and then click Finish. 4. In the Port shape Port surface, click Operation_1. In the Properties box, change the Identifier to WSPort. The Operation Identifier will be used as the Web Method when the orchestration is published as a Web service. 5. From the Port entry, drag the Request arrow to the Receive shape, and then drag the Response arrow to the Send shape. This associates the Receive and Send shapes with this port. Build and deploy this project. To publish the orchestration as a Web service 1. Click Start, point to Microsoft BizTalk Server and click BizTalk Web Services Publishing Wizard. 2. On the Welcome page, click Next. 3. On the Create Web Service page, select Publish BizTalk orchestrations as Web services. Click Next. 4. Click Browse and select the assembly containing the BizTalk project you created in the previous procedure, then click Next. 5. In the Orchestration and Ports dialog box, note the ports selection tree. If the orchestration contains multiple ports, select only the ones to be exposed. Because you created only one port, click Next to accept the defaults. 6. On the Web Service Properties page, click Next to accept the defaults. 7. On the Web Service Project page, enter the location where the Web service will be 20 created. For this example, use the default http://localhost/WSOrchExample_Proxy. Select Allow anonymous access to the Web service and Create BizTalk receive locations in the following application. In the drop-down list, select BizTalk Application 1, and then click Next. 8. On the Summary page, review the settings selected for the Web service, and then click Create. To configure the Application Pool 1. Click Start, point to Administrative tools, and then click Internet Information Services Manager. 2. Expand the Internet Information Services (IIS) Manager. Right-click Application Pools and click New Application Pool. In the Add New Application Pool dialog box, set the Application Pool ID to WSOrchPool, and then click OK. 3. Expand the Application Pools folder, then right-click WSOrchPool and select appropriate properties. On the Identity tab, set the Application Pool Identity to the BizTalk service account. Click OK. 4. In Internet Information Services (IIS) Manager, expand the Web Sites folder, then the Default Web Site entry. Right-click the location you just created and click Properties. On the Directory tab, set the Application pool to WSOrchPool. This sets the Web service to run as the BizTalk service account so incoming requests are written to the appropriate BizTalk database. To configure bindings 1. Click Start, point to Microsoft BizTalk Server 2006, and then click BizTalk Administration. 2. Expand BizTalk Server 2006 Administration, BizTalk Group, Applications, and BizTalk Application 1. Select the Orchestrations folder. 3. Right-click WSOrchExample.WS.Orchestration and click Properties. 4. In the left pane, select Bindings. In the Bindings pane, set the Host to BizTalkServerApplication, and in Bindings set the Receive Port for Port_1 to WebPort_WSOrchExample_Proxy/WSOrchExample_WSOrchestration_Port_1. This associates the physical receive port created by the Web Services Publishing Wizard to the logical port in the orchestration. Click OK. 5. Right-click WSOrchExample.WSOrchestration and click Start. 6. Select Receive Locations and right-click the WebService_WSOrchExample_Proxy/WSOrchExample_WSOrchestration_Port_1 receive location and click Enable. The orchestration starts and will process incoming requests from the Web service generated by the Web Service Publishing Wizard. 21 The configuration steps described previously are suitable for a simple solution. More complex scenarios may require you to create a Web service from schemas and then bind an orchestration to the receive port at a later date. Another option is to use the wizard to create the virtual directory and Web service proxy, and then create the receive ports by using the Specify now option. Important Regardless of the exact steps used to publish an orchestration as a Web service, it is important to ensure the Web method matches the operation of the receive port in the orchestration. If it does not, the subscriptions will not work. Using Client Certificates with BizTalk Server Two issues you might encounter when using client certificates with BizTalk Server are ensuring the certificates are installed correctly and organizational security restrictions. Client Certificate Installation Using client certificates while consuming a Web service over HTTPS (a secure Web channel) ensures the certificates are correctly installed. Use the following guidelines: The client must have the private key for the certificate used. When opening the certificate in the Certificate dialog box, on the General tab, ensure the certificate includes the following text: You have a private key that corresponds to this certificate. 22 The certificate should be stored in the Personal store for the BizTalk service host account. Use the following procedure to verify that the certificate is stored in the appropriate location. To check a certificate location 1. Log on to the BizTalk Server using credentials for the service host account. 2. Open the Certificates Administration Console for the current user account, and expand the Personal store. 3. Expand the Certificates – Current User and Personal nodes. 4. Make sure the client certificate is listed in the Certificates node. The following illustration shows an example. 23 Note If the certificate is not listed in the Certificates node, it can be imported. To do this, right-click the Personal node, point to All Tasks, and then click Import. For more information about configuring certificates, see “WinHttpCertCfg.exe, a Certificate Configuration Tool” at http://go.microsoft.com/fwlink/?LinkID=99076. In the Microsoft BizTalk Server Operations Guide, see "Managing Certificates" from at http://go.microsoft.com/fwlink/?LinkId=117120. Organization Security Restrictions Each organization may have restrictions on using client certificates for security reasons. One such restriction is when a user requests a client certificate a password prompt is displayed. A client certificate can be used only if the correct password is provided. Because BizTalk Server uses services and services cannot interact with dialog boxes so do not use client certificates requiring password validation. To prevent this issue, configure the policy so no passwords are prompted when a certificate is used. This setting is enforced by the Group Policy Object (GPO) System Cryptography: Force Strong Key protection for user keys stored on the computer. If setting this policy, then the value should be set to “User input is not required when new keys are stored and used” as indicated in the following illustration: 24 Consuming Web Service Using a Send Port in Messaging–Only Scenarios BizTalk Server 2006 included a new feature where users can consume a Web service directly in the send port without using an orchestration. Simple messages Consuming a Web service in a send port is relatively simple when the Web service exposes complex types for its parameters. The task becomes more complex when the consumed Web service uses primitive .NET data types such as strings or integers. For example, assume the client is a .NET application. This application calls a Web service (published from schema using BizTalk Server) using a request-response SOAP receive port. A solicit-response send port subscribes to the request-response SOAP port. The send port consumes the Web service and returns a response to SOAP receive port. In turn, a response will be sent back to the client application. 25 If complex in/out type parameters are used for a back end Web service adding a reference to this Web service to the BizTalk project creates Reference.xsd under Web reference section in the project explorer. To address this issue, use the following guidelines: 1. Create a simple BizTalk project. 2. Add reference to the Web service. 3. Create an input message type schema. 4. Create an output message type schema. 5. Create a map (for example, map1) which maps the input request to the Web service request schema from Reference.xsd. 6. Create another map (for example, map2) which maps the Web service response schema from Reference.xsd to output schema. (You may not require a map for the output message). 7. Deploy the BizTalk project. 8. Publish schema as Web service for the BizTalk project and publish as follows: Input message schema - for request Output message schema - for response This creates a request-response receive port. 9. For this receive port, apply the first map (for example, map1) for inbound maps and apply the second map (for example, map2) for outbound maps. 10. Create a solicit-response send port configured as follows: Use SOAP transport with the following configuration: i. Specify the Web service URL. ii. Specify the BizTalk assembly previously created and the appropriate method. In the Filters section for the send port, add a filter on the receive port name and specify the receive port name created previously. 11. Create a .NET client application. Add a Web reference for BizTalk schemas published as Web service and test the solution. If using standard .NET types for in/out parameters for Web services, when adding a reference to this Web service to the BizTalk project, Reference.xsd is not created. (Reference.xsd contains message schema formats used to map the message being sent to the web message format.) Multipart messages Multiple Web parameters are mapped as a multipart message through the SOAP adapter. If receiving messages from adapters not directly support receiving/sending multipart messages, use a pipeline component to create a multipart message before sending the request to the target Web service. If receiving messages using adapters able to handle multipart messages such as SOAP receive adapter, a custom pipeline component is not necessary. 26 The following steps outline how to consume a Web service taking multiple parameters without using an orchestration. 1. Create a simple orchestration receiving a multipart message with two parts (Number1 and Number2) both are of type integer (int) and return an integer (int). 2. Publish the orchestration as a Web service. Note Another method is to publish a schema as a Web service to create this front-end Web service. However, because of the native types (int) publish the orchestration as a Web service. 3. Create a two-way SOAP receive port that subscribes to this Web service just created. 4. Create a two-way SOAP send port subscribing to the SOAP receive port created in step 3 by name. 5. Configure the SOAP send port to call the method of the back end Web service accepting two integer parameters. For more information and to use the sample project for this scenario, see http://go.microsoft.com/fwlink/?LinkId=134833. Consuming Web Services with complex Web Service Definition Language (WSDL) When consuming a Web service, issues with WSDL’s may surface. In general, there are absolute size and number of import levels limits for schemas in BizTalk Server. If errors occur when using complex or large messages try using a less complex or smaller message. Importing other WSDL’s or schemas into the Web service WSDL is not supported by the SOAP adapter. This does not mean it will not work in all cases. Testing is the only way to know for a particular case. WSDL importing is supported with the WSE 2.0 add-on. This requires a hot fix, see “FIX: Error message when you use the Add Generated Items wizard to generate a schema for BizTalk Adapter for Web Services Enhancements 2.0: "Specified argument was out of the range of valid values"” at http://support.microsoft.com/kb/931980. The following is a suggested workaround for schema imports using Typed DataSets as parameters to Web methods. 1. Add the Web reference to a C# project and then generate the proxy. 2. Create a SOAP send port and specify the proxy on the send port and choose the method. 3. In the orchestration, define a late bound port and define the message types. For most cases where no property promotion or distinguished field access is needed, the type can be defined as XMLDocument. Select PassThrough pipelines with this type. 4. In the BizTalk Server Administration console, on the Web Service tab in the SOAP Transport Properties dialog box of the SOAP send port, specify the proxy to use along with the specify assembly, type, and method. For more information, see the SOAP Transport Properties Dialog Box, Web Service Tab topic in the BizTalk Server Help. 27 Accessing SOAP Headers from an Orchestration SOAP header information is available inside the orchestration. To add SOAP headers, use a special variation of the property schema. For more information, see Andy Babiec’s blog at http://go.microsoft.com/fwlink/?LinkId=129021. 1. Add/update the Web reference. 2. Open the Reference.xsd and note the root node for the SOAP header and close. 3. Add a property schema to the project and change the target namespace to “http://schemas.microsoft.com/BizTalk/2003/SOAPHeader”. 4. Rename the root node to the value from step 2. 5. Locate the Property Schema Base property drop-down list. Set the value to MessageContextPropertyBase. 6. Save the file as SoapHeader.xsd and close. Note The name is not important. However, we recommend that you do not name the Type name property the same as the root node because this causes compilation error. 7. Add a Construct Message shape to build the message to call the Web service. 8. Add a Message Assignment shape if not present after the Transform shape in the orchestration. 9. Edit the Message Assignment shape using the following example: SampleWS_Request_Msg(MyBizTalkProject.AuthenticationHeader) = "<ns0:AuthenticationHeader xmlns:ns0=\"http://www.acme.com/WebService/\"> <ns0:UserName>MyName</ns0:UserName> <ns0:Password>NotSoSecretPassword</ns0:Password> </ns0:AuthenticationHeader>"; Where: SampleWS_Request_Msg is the message variable name in the orchestration for the external Web service. MyBizTalkProject is the project name. AuthenticationHeader is the name of the property schema. Note The Intellisense drop-down list appears for the message and the property schema should be available. Note An alternative solution is to use an existing SOAP header from another message. 28 Unknown SOAP Headers Normally, inbound message header values are context variables of the inbound message. However, unexpected header values can be included. The Web service can be configured to make these available to an orchestration. This is sometimes done to include information not originally part of the Web service. Newer versions of an application may know how to use the data while older versions will continue to work. No error occurs if an unexpected SOAP header is received by the Web service. By default, the header is ignored. These values are available as the “unknown” header context variable if configured during the deployment wizard. For more information, see the BizTalk Web Services Publishing Wizard topic in the BizTalk Server Help. RPC–Style Web Services Microsoft tools are designed by default to work with the “document literal” style of Web services. BizTalk Server does not support remote procedure call (RPC)-style SOAP messages. However, some WSDL’s can be made to work. The difference is the creation of multipart messages to populate the parameters instead of the usual nodes for document WSDL’s. This could lead to a lot of extra coding if mapping is not an option. Here’s an extract from Tomas Restrepo’s blog Commonality. “This is important because now you have to manipulate the messages manually, using Expression Shapes in your orchestrations. Using tools like maps is out of the question because you can't directly manipulate the entire message as a single unit (since it is multipart). Another common scenario with RPC/Encoded services is that at least one of the parameters to an operation is a string type used to carry an encoded XML fragment. To manipulate those, you'll want to manually define schemas for the fragments, and then load them at runtime into separate messages using an intermediate XmlDocument instance (there are many examples of this out there, so I won't cover it in more detail here). If you need to assign a value to one of these parameters, you'll probably want to use a simple component that can extract a string from a regular BizTalk message you can assign to it (you can certainly manipulate them directly as strings, but that might be harder).” Tools When working with Web services, it is useful to see the traffic between BizTalk Server and the Web server. Network tracing tools provide this functionality. During development and testing use HTTP until everything works. Using SSL (HTTPS) encrypts the data and this limits the usefulness of tracing. Microsoft’s NetMon is available at no cost from the Microsoft Download Center. A lesser known utility is Fiddler. Most tracing tools do not capture calls to a Web service on the BizTalk Server computer. Determining how the Web service is called, its methods and parameters, sometimes requires extra effort. In most cases simply consuming the Web service provides the required information. BizTalk Server takes care of these details behind the scenes. Message types are created for 29 communication using an orchestration. When dealing with complex WSDL’s or RPC style Web services (see previous section) more work is required. In this case, the first step is to get the WSDL and see what can be determined from a review. Open the WSDL in Internet Explorer or other editor. Look for import of schemas or other WSDL’s. Check the various types it defines. The WSDL should be provided by the partner or available for review from the Web service. If the previous recommendations are not enough, create a Visual Studio project to consume the WSDL. Either point the Visual Studio “Add Service Reference” wizard at the Web service or use the WSDL file directly (WSDL.exe) to create the proxy class. Use the project to call the Web Service using the proxy objects. Visual Studio provides more extensive debugging tools for reviewing the behavior of the code than inside BizTalk Server. Another option is to use a third-party tool, such as soapUI. The soapUI tool consumes the WSDL and graphically shows the methods and parameters. It enables calls to the Web service by typing values into the parameters and clicking a button. Web service results can be examined without a single line of code. For more information about the soapUI tool, see http://www.soapui.org. Note The soapUI tool is not supported by Microsoft, and Microsoft makes no guarantees about the suitability of these programs. Use of these programs is entirely at your own risk. 30