Module 5: Routing BizTalk Messages Time estimated: 135 minutes Module objective: In this module, you will learn the purpose and scope of the Microsoft BizTalk Server publish and subscribe messaging models and how to configure BizTalk message routing. Overview Microsoft BizTalk Server provides a mechanism for routing, processing, and sending messages between business processes. Content-based routing (CBR) is a method of routing messages based on the context and/or content of an incoming message. BizTalk provides two methods of monitoring and tracking message activity, which enables developers, administrators, and business users to diagnose messaging problems. This module shows you how to enable message routing and use BizTalk tools to monitor live and archived message activity. Lesson 1: Introduction to Message Routing Lesson objective: Explain how BizTalk Server 2010 routes and processes messages. Overview The BizTalk messaging system consists of components such as receive locations, receive and send ports, pipelines, and the MessageBox database. The component that is responsible for the actual message routing process is referred to as the messaging engine. This lesson provides an overview of how the BizTalk messaging engine works to process messages and how message routing is enabled. The Publish and Subscribe Architecture Describe the publish and subscribe routing model used by BizTalk Server for routing and processing documents. The Publish and Subscribe Architecture BizTalk Server uses a publish and subscribe (or pub/sub) messaging model. The pub/sub model is highly scalable at both the database and processing levels. This pub/sub routing mechanism can manage large volumes of messages and large individual messages, and can interact with a wide variety of back-end systems. This design also makes adding and changing subscribing components on-the-fly extremely easy. Publication Publication is the process by which messages are placed in the MessageBox database. The principal publishing component in BizTalk Server is the receive port, which receives and processes messages into the database. Orchestrations also publish messages as they are sent from the orchestration to the MessageBox database for further routing. Properties defining which messages will be routed can be promoted as the message is being processed through receive pipelines and can also be specified in the business process (orchestration) itself. Each of these processes will be detailed in upcoming modules. Subscriptions Subscriptions are criteria on which message routing is determined. Orchestrations, send ports, and send port groups can each subscribe to messages. Each subscription allows the subscriber to initiate or continue the processing of a message. Subscriptions are managed by the MessageBox database. When a message meets the specifications of a subscription, the message is passed from the MessageBox database to the subscriber. If multiple subscribers exist for a given message, each gets a copy of the message. The preceding illustration outlines the steps in the publish/subscribe model, as follows: 1. Subscriptions are created for each subscriber (send port, send port group, or orchestration) by configuring a filter expression. For example, this expression could be for all purchase order messages with totals over $10,000. 2. A message is received, processed, and published to the BizTalk MessageBox database. Messages can be received through a variety of different adapters, including File, FTP, and HTTP. 3. Subscriptions, which are maintained by the MessageBox database, are evaluated to determine which subscriber(s) should receive a copy of the message. 4. Each subscriber will receive a copy of the message (with a unique message ID). The message stays in the MessageBox database until all subscribers have received and successfully processed the message. At each stage in the processing, data necessary for tracking messages is written to a separate database (the tracking database), where it resides until deleted by administrative action. What Is the MessageBox Database? Explain the purpose of the MessageBox database and how message subscriptions work. MessageBox Database The MessageBox database stores messages and message properties, and it maintains message subscriptions. The messaging engine delivers the messages from the database to subscribers. The MessageBox databases also store the queues and state tables for each BizTalk host. Messages are immutable once they arrive at the MessageBox database, but beforehand, messages can be manipulated in a variety of ways, including validating the message against a known schema, promoting specific nodes to be used in message routing, decrypting a message, or transforming a message by using a BizTalk map. Each of these processes is addressed in other modules in this course. Adding MessageBox Databases MessageBox databases are the basis of work-item load balancing across multiple servers that perform cooperative processing. With all of the messages and processes that the MessageBox database is responsible for, it can rapidly become a bottleneck in a BizTalk environment. To increase the number of messages that your system can process, you may need to create additional MessageBox databases. With the Enterprise edition of BizTalk Server 2010, you can use the BizTalk Administration Console to create multiple MessageBox databases. When multiple MessageBox databases are in use within a BizTalk Server Group, the master MessageBox (the first one created) acts only as a router and routes all new messages to the other message boxes for processing. A degradation of performance can occur when there are only two MessageBox databases because the messages are still being processed by only one database since the master database performs only the routing of messages. Therefore, it is highly recommended that when using multiple MessageBox databases you have at least three MessageBox databases. A single computer running Microsoft SQL Server™ can host several BizTalk Server MessageBox databases before the server becomes overloaded. Determine the performance impact of the computer running SQL Server when implementing multiple MessageBox databases. You can designate local and remote databases as MessageBox databases. Verify that the SQL Server Agent is running on all computers running SQL Server before adding a new MessageBox database so that database maintenance tasks are performed. Note: Refer to the “How to Add a New MessageBox Database” and “Creating a Highly Available BizTalk Server Environment” topics in the Microsoft BizTalk Server 2010 documentation for more information about adding MessageBox databases. Disabling New Message Publication As noted, several separate MessageBox databases can receive and store (publish) messages at any time. If it is necessary to delete or move a MessageBox database, you should first disable new message publication. Also, if you have multiple MessageBox databases, the master database may experience an improvement in performance if new message publication is disabled on the master MessageBox database. Disabling new message publication does not affect existing messages because any existing messages stay in the original MessageBox databases. You can use either the BizTalk Administration Console or Microsoft Windows Management Instrumentation (WMI) to disable new message publication. Deleting MessageBox Databases Situations may arise in which you want to delete a MessageBox database from a BizTalk group. For example, you might want to delete a MessageBox used for a series of tests now concluded. You can use either the BizTalk Administration console or WMI to delete a MessageBox database. Deleting a MessageBox database from a BizTalk Group does not actually remove the database from SQL Server. To remove the MessageBox database completely and permanently, you must remove it by using SQL Server Enterprise Manager after it is removed from the BizTalk Group. What Is Message Routing? Describe the concept of message routing. Message Routing A typical BizTalk Server business process involves receiving, processing, and sending messages. At times, you may receive messages (such as partner-to-partner correspondence) that do not require intensive processing in an orchestration, and that could therefore benefit from a simpler solution. The BizTalk content-based message routing model provides great flexibility in the way that messages are handled. For example, two incoming messages of the same type can be processed differently based on a customer name or a total order amount. Scenario One common scenario that requires message routing is when a receive port is configured to receive a single message type, as in the case of a purchase order (PO). Assume that you have created a send port to an ERP system and that you have configured a map that needs to receive a copy of all POs. You can configure the filter on the send port to process all messages received through the specified receive port. Additional send ports and filter expressions could be added to route messages based on other properties. Examples The preceding illustration shows three send ports, each with a different filter for content-based routing. All PO messages in which the CustomerName field contains Contoso are to be routed to send port A. All messages with the Price field greater than 1000 are routed to send port B. All messages with a quantity greater then 500 and in which the Price is less then 1000 (which may represent an error condition) are routed to send port C. A message is routed to all send ports for which it meets the filter criteria. Therefore, if a PO is received from Contoso, in which the Price is over 1000, a copy of the message is routed to both send ports A and B but not send port C. What Is a Port? Describe how BizTalk Server uses receive ports and locations to receive messages and how it uses send ports to send messages. Ports Ports specify how messages are sent and received inside BizTalk Server, as well as between BizTalk Server and external processes. Each port has a direction, a transport type, and a binding, which together determine the direction of communication (one-way or two-way), the pattern of communication (one-way or request-response), the location to or from which the message is sent or received, and how the communication takes place. A port’s binding may refer to the physical location of a read or write, the pipeline used for message processing, or an orchestration. A port must be associated with: A Universal Resource Identifier (URI—a physical location). A transport (examples are File, HTTP, and BizTalk Message Queuing). A port may be associated with: A send pipeline, to prepare a message for sending. For example, pipelines can be used for assembling, encrypting, compressing, or performing some other action on a message. A receive pipeline, to prepare a received message for processing. For example, receive pipelines can be used for disassembling, decrypting, or decompressing a message. A map, to transform messages from one type to another. Note: Pipelines are covered in greater detail in Module 6, “Creating Pipelines.” BizTalk Message Flow Describe how messages are processed through BizTalk messaging components. Overview The two main functions of BizTalk Server 2010—BizTalk Orchestration and BizTalk Messaging— form the underlying architecture that enables you to integrate and exchange messages with the many types of external systems and applications that exist in your organization and in those of your trading partners. The BizTalk Messaging engine is responsible for performing the following tasks: Receiving inbound documents through receive ports, which may include the use of a map for message translation or transformation. Parsing inbound documents to determine their specific format in a pipeline. Receive pipelines can also perform data validation to ensure that the message being received is valid compared with a known schema. Extracting key identifiers and identifying applicable routing rules. Filters can be placed on send ports to route documents based on the contents of a message. Delivering documents to their respective destinations. Some of the options available to route documents include file drops, databases, or other business processes. Tracking documents. The BizTalk Group Hub allows developers, IT professionals, and business analysts to monitor message activity and troubleshoot errors as they arise. What Is Property Promotion? Describe how promoted properties work to enable content-based routing. What is Property Promotion? In BizTalk Server, promoted properties enable various BizTalk components to access key items of data contained in a message. Promoting a property makes the data within that node of the schema accessible to messaging processes. In addition to the properties that you may choose to promote, various adapters promote contextual properties for routing purposes. The information about the promoted properties is contained in a specially formatted XML schema known as a property schema. Properties from a single message can be promoted to different schemas depending upon your needs. For example, if you need to route all invoices with a total dollar amount over $500, you would promote the amount node in the source schema. The amount node would then be available for use in a filter expression to create subscriptions. Performing a quick promotion on a schema node promotes the node to the default property schema defined for the XML schema in question. If a property is quick promoted and no default schema yet exists, Microsoft Visual Studio® will offer to automatically create the schema for you. Quick promotion is done by right-clicking the node and then clicking Quick Promote. By using the option to view promotions in Visual Studio, you can manage the promoted properties (including removing the promotion if no longer needed). This view also includes the ability to distinguish specific nodes. Distinguished fields are used in orchestrations and will be discussed later in Module 10, “Creating Transactional Business Processes.” Visual Studio provides a property schema template for creating your own property schemas. Promoted Properties Property promotion enables BizTalk messaging services to route messages based on the context and content of the message. You want to promote properties only when required for message routing, message correlation (a special form of message routing, which will be covered in Module 9, “Automating Business Processes”), or property-tracking. Excessive property promotion can adversely affect BizTalk Server performance. Demonstration: Promoting a Property Learn how to promote a schema property by using the Quick Promotion option in the BizTalk Schema Editor. Promote Properties 1. In Windows Explorer, navigate to C:\AllFiles\DemoCode\Module5\Demo, and then double-click Demo.sln. The following demonstration is not dependent upon completion of the previous demonstrations. This solution provides artifacts and file paths that differ from those used in the previous demonstrations. 2. In Solution Explorer, double-click PurchaseOrder.xsd to open the schema. 3. Right-click the <Schema> node, and then click Expand Schema Node. 4. Right-click State, point to Promote, and then click Quick Promotion. 5. In the Microsoft Visual Studio dialog box, click OK. This is a notice that Visual Studio will create a property schema that will contain this promoted property. In Solution Explorer, notice that the schema named PropertySchema has been added to the project. 6. PropertySchema.xsd will open in the Schema editor window, and a new Microsoft Visual Studio dialog box will appear, asking you if you want to reload PropertySchema.xsd. Click Yes. Notice that the node you promoted is part of this schema. 7. Right-click Property1, and then click Delete, and then click Yes. The Property1 node serves no purpose and can be safely deleted. 8. On the File menu, click Save All. 9. In PurchaseOrder.xsd, right-click OrderTotal, point to Promote, and then click Quick Promotion. 10. Another Microsoft Visual Studio dialog box will appear, asking you if you want to reload PropertySchema.xsd. Click Yes. 11. In Solution Explorer, right-click the Messaging project, and then click Deploy. The next demonstration requires that this project be deployed. 12. Pause the bt10d-demos virtual machine. Lesson 2: Configuring Message Routing Lesson objective: Configure BizTalk Server message routing. Overview BizTalk Server enables you to route messages directly to a send port or to an orchestration based on the contents of the message. This feature is very powerful because it can increase flexibility of business processes by eliminating the need to deterministically configure the flow of messages. Steps for Enabling Messaging Routing Identify the steps for enabling the routing of messages in BizTalk. Enabling Message Routing To enable message routing, you must perform the following steps. 1. Promote the nodes in the source schema that will be used for content-based routing of the message. 2. Create and configure both a receive port and a receive location for incoming messages. Receive ports and receive locations are used to receive messages from internal applications and from external trading partners. 3. Create and configure any send ports or send port groups that will be used for the routing of the message. 4. Configure the filter expressions on the send ports or send port groups to create the message subscriptions used to route messages to subscribers. 5. Enlist and/or start the send ports. Enlisting a send port creates the subscription in the MessageBox database but will not begin processing messages until it is started. 6. Enable receive locations, which will begin processing messages from the endpoint. Send ports should (at the least) be enlisted before receive locations are enabled. If messages are received into the MessageBox and there are no subscriptions, the messages will be suspended. Configuring a Receive Port and Receive Location Configure a receive port and a receive location. Receive Ports BizTalk Server uses receive ports to receive messages from internal systems and trading partners. Receive ports can be configured to use one or more maps for incoming message transformation. If more than one map is specified, only the map that matches the inbound message will be applied. You can also configure the port to track message bodies or properties. Although message body tracking provides more information about each message, it can adversely affect the performance of BizTalk Server. Additionally, receive ports can function as either one-way ports or two-way ports. A one-way port only receives messages, whereas a two-way port, otherwise known as a request-response port, can both receive and send messages. This module focuses on one-way ports because they are used more often than two-way ports. Receive ports and receive locations can be created and configured by using BizTalk Explorer as shown in the preceding illustration. Both receive ports and receive locations can also be created and configured by using the BizTalk Administration Console. Receive Locations A receive port can have multiple receive locations associated with it. This is useful when you have messages being received from multiple physical locations, such as from a file folder and an FTP site, but you want all the messages received through any of the locations to be processed in the same manner. Receive locations are configured with a URI, an address from which inbound messages arrive, a receive handler, and a receive pipeline. The URI is specific to the transport or adapter type (FILE, FTP, HTTP, Microsoft SharePoint® Services, and others). The receive handler is the BizTalk host under which the receive location will process messages. The receive pipeline processes the messages between being picked up and being written to the MessageBox database. BizTalk pipelines will be covered in more detail in Module 6: “Creating Pipelines”. For more information on BizTalk hosts, refer to the BizTalk Server 2010 documentation on “Managing BizTalk Hosts and Host Instances.” Receive locations can be in one of three possible states: Enabled. When enabled, receive locations process messages. Disabled. Disable a receive location if you want to prevent the receive location from receiving messages or if you want to delete the receive location. Suspended. If a service window has been configured for the receive location, the location will indicate that it is suspended while the service window is inactive. If an error occurs while a receive location is listening for messages, the receive location will retry the number of times specified in the Retry Count property spaced at the Retry Interval (minutes) property. If it continues to fail, the receive location may be disabled (this is a property of the individual adapter). Demonstration: Configuring a Receive Port and a File Receive Location Learn how to create and configure a receive port and a receive location, and how to enable a receive location by using the BizTalk Server Administration Console. Create and Configure a Receive Port 1. Resume the bt10d-demos virtual machine. 2. Start the BizTalk Server Administration Console, and expand BizTalk Server Administration, BizTalk Group, Applications, and then Demos. 3. Right-click Receive Ports, point to New, and then click One-way Receive Port. 4. In the Receive Port Properties dialog box, in the Name box, type PurchaseOrders, and then click OK. Create and Configure a Receive Location 1. In the BizTalk Administration Console, in the left pane, right-click Receive Locations, point to New, and then click One-way Receive Location. 2. In the Select a Receive Port dialog box, double-click PurchaseOrders. 3. In the Receive Location Properties dialog box, in the Name box, type PurchaseOrdersFILE. It is helpful to include the Transport Type in the name of the receive location. 4. In the Type list, click FILE, and then click Configure. 5. In the FILE Transport Properties dialog box, click Browse. 6. Expand Computer, Local Disk (C:), AllFiles, DemoCode, and Messages, and then click Make New Folder. 7. Rename the new folder IN, and then click OK. Notice that the file mask is *.xml. This means that all XML files in this folder will be processed. 8. In the FILE Transport Properties dialog box, click OK. 9. In the Receive Location Properties dialog box, in the Receive pipeline list, click XMLReceive, and then click OK. Enable the Receive Location 1. In the BizTalk Administration Console, in the left pane, click Receive Locations. 2. Right-click PurchaseOrdersFILE, and then click Enable. The receive location is ready to begin processing messages. 3. Pause the bt10d-demos virtual machine. Creating and Configuring a Send Port Configure a send port. Overview Send ports are the locations to which BizTalk Server sends messages. A send port can function as either a one-way port or a two-way port. As specified earlier, one-way ports send messages, and two-way ports (also known as solicit-response ports) can both receive and send messages. However, not all adapters support two-way ports. The FILE, MSMQT, and SMTP adapters are supported only as one-way ports. There are three general types of send ports: Static send ports. These send ports contain a fixed destination address. When you create a static send port, as in the preceding illustration, you specify the adapter type, destination address (URI), and pipeline used for the static send port. Dynamic send ports. Dynamic send ports do not contain a fixed destination address. Instead, the destination address is set within the orchestration. Direct send ports. In the case of the static and dynamic send ports, the message is written to the MessageBox database before being processed into the send port. In some situations, it may be advantageous to have orchestrations communicate directly. In these cases, direct send ports can be used. Send ports can be created using either BizTalk Explorer or the BizTalk Administration Console. Some of the properties you can configure for a send port include: Send port name. Since you will likely have many send ports in your application, you should always assign a descriptive name to a send port to help identify which messages or processes are associated with a specific send port. For example, POAck_to_Vendor is a much better name then Port_1. Also, descriptive names can help when moving from one environment to another. Primary and secondary transport. Examples of transport types include FILE, FTP, SQL Server, and HTTP. Send ports contain both a primary and a secondary transport type so that if the primary transport is unavailable for some reason, messages can still be sent using the secondary transport type. For example, your primary transport type may be an FTP site. However, if the FTP site is unavailable, you can set the secondary transport to FILE and still successfully process messages through the send port. Later when the FTP site is back online, you can manually transfer received messages from the file location to the FTP site. Address (URI). The address or URI is the physical location where messages are sent when processed by a send port. The address is dependent upon the transport type. Retry count. The retry count is the number of times BizTalk will try to resend a message if the transmission fails. The default value is 3, which is satisfactory for many situations, but the allowed range is 0 through 1000, which allows flexibility in use. For example, you may not want retries to occur at all when communicating with a Web service, whereas you may want to try continuously until successful when sending to a SQL database. Service window enabled/disabled. You can enable the service window if you want to configure a port to send only at specified times of the day. By default, the service window is disabled, meaning that the send port will send messages at any time. If the service window is enabled, the Start Time and Stop Time properties must be specified. You can enable the service window, for example, if you want to send messages to a busy HTTP site only between midnight and 08:00 to prevent network delays during normal business hours. The messages will actually be delivered to the send port and be queued in the spool table in the MessageBox database until the appropriate time. Other properties that must be specified for a send port include the BizTalk host and the send pipeline. Both of these properties are configured on the Send tab of the Send Port Properties dialog box. BizTalk hosts and send pipelines are covered in more detail in Module 6: “Creating Pipelines”. Configuring Send Port Filters Configure send port filters. Filter Expressions Filter expressions are used to create subscriptions. Subscriptions determine which messages will be delivered to a specific port. You can use both the promoted properties from the inbound message schema and the BizTalk global properties to create the filter expressions on a send port. This is known as content-based routing because the actual contents of the message are used to determine how the message should be routed. For example, if you need to route all messages with an order amount over $500 to a specific folder for managerial approval, you can promote the amount node in the source schema and then create a filter expression for a send port to subscribe to messages only with an amount greater than or equal to $500. Subscribing to Failed Messages BizTalk Server 2010 provides the ability to create a subscription for failed messages. When failed message routing is enabled on a receive port, a filter expression on a send port can be used to send the failed message to an orchestration or some other endpoint, such as a SharePoint site, to fix the error and resubmit the message. A sample of an error handling solution is provided in the BizTalk Server 2010 SDK (software development kit). Using a Send Port Group Identify the purpose of a send port group and configure a send port group for message routing. Send Port Group A send port group is a collection of send ports that you can use to send a message through multiple send ports by using a single configuration. This is useful when message processing is complete and a message needs to be sent to multiple destinations. Send port groups are useful, for example, when you need to send a message from a specific business process to three different physical locations. For example, you might need to submit a confirmation message of an invoice that was over $10,000 to an FTP site, an HTTP directory, and a file drop for auditing purposes. If you create three separate send ports—one for each transport type—and then create a send port group consisting of these three send ports, you can create a single filter that will send any messages that meet the filter criteria to all three physical send ports. The filter for a send port group is configured in the same way as a filter for a single send port. You can create send port groups by using either BizTalk Explorer or the BizTalk Administration Console. Enlisting and Starting a Send Port Describe the various states of a send port group and enlist and start a send port for message processing. Overview Send ports and send port groups always have a state. The state of the send port determines whether it accepts messages, processes messages, or does neither. The state of the send port group can be changed by actions invoked on a send port it contains. You can change the state of a send port by using either BizTalk Explorer or the BizTalk Administration Console. Send Port States Send ports are always in one of three states: Bound, Started, or Stopped. Bound. Ports that are fully configured but do not have any subscriptions are in the bound state. When send ports are in the bound state, they are ready to be enlisted and started. Fully configured ports are bound to a pipeline, associated with a particular BizTalk host, and can optionally specify a map. Started. When a send port is in the started state, the subscription for this send port or send port group is active and messages are being delivered and processed. Stopped. If a send port has been enlisted or has previously been started and is now stopped, the send port or send port group will not currently be running. When the send port or send port group is in this state, all new messages routed to this send port or send port group are sent to the suspended queue of the send handler. If the port is subsequently started, any queued messages will be processed. If a port that is stopped is then disabled, its state will change to Bound, and any messages that have been queued will be removed from the queue and will not be processed by the port. However, the message remains in the MessageBox database. Enlisting The actions that can be invoked to change the state of a send port are Enlist, Start, Stop, and Unenlist. When send port groups are first created, they are in the Bound state. To change the send port group to the Started state, the send port group must be either started or enlisted and then started because starting a send port implies enlisting. When a send port or send port group is enlisted, it is associated with a BizTalk Host, and the subscriptions for the send port or send port group are created in the MessageBox database. However, if a send port has been enlisted but not started, it is still in the Stopped state. Stopping a send port moves the send port state to Stopped but does not delete the subscription in the MessageBox database. Unenlisting a send port changes the send port state to Bound and deletes the subscription to that send port in the MessageBox database. A query can be performed through the BizTalk Administration Console hub page for all subscriptions. A preconfigured query is accessible at C:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Utilities\BTSSubscriptionViewer.btq. Demonstration: Configuring a Send Port and a Send Port Group Learn how to create and configure send ports and send port groups, and how to define filters. Configure a Send Port with a Filter for a Standard BizTalk Property 1. Resume the bt10d-demos virtual machine. 2. In the BizTalk Server Administration Console, right-click Send Ports, point to New, and then click Static One-way Send Port. 3. In the Send Port Properties dialog box, in the Name box, type ProcessedOrdersFILE. 4. In the Type list, click FILE, and then click Configure. 5. In the FILE Transport Properties dialog box, click Browse. 6. Expand Computer, Local Disk (C:), AllFiles, DemoCode, click Messages, and then click Make New Folder. 7. Rename the new folder ProcessedOrders, and then click OK. 8. In the FILE Transport Properties dialog box, in the File name box, type Order%MessageID%.xml, and then click OK. 9. In the Send Port Properties dialog box, in the Send pipeline list, ensure that PassThruTransmit is selected. 10. In the left pane of the Send Port Properties window, click Filters. 11. In the Property list, click BTS.ReceivePortName. 12. Verify that the Operator is set to ==. 13. In the Value box, type PurchaseOrders, and then click OK. This send port will subscribe to all messages received through the PurchaseOrders receive port. Create Two Send Ports with No Filters 1. In the BizTalk Server Administration Console, right-click Send Ports, point to New, and then click Static One-way Send Port. 2. In the Send Port Properties dialog box, rename the port to ToManagerFILE. 3. In the Type list, click FILE, and then click Configure. 4. In the FILE Transport Properties dialog box, in the Destination folder box, type C:\AllFiles\DemoCode\Messages\OUT\. 5. In the FILE Transport Properties dialog box, in the File name box, type ToManager%MessageID%.xml, and then click OK. 6. In the Send Port Properties dialog box, in the Send pipeline list, ensure that PassThruTransmit is selected, and then click OK. 7. In the BizTalk Server Administration Console, right-click Send Ports, point to New, and then click Static One-way Send Port. 8. In the Send Port Properties dialog box, rename the port to ToAccountingFILE. 9. In the Type list, click FILE, and then click Configure. 10. In the FILE Transport Properties dialog box, in the Destination folder box, type C:\AllFiles\DemoCode\Messages\OUT\. 11. In the FILE Transport Properties dialog box, in the File name box, type ToAccounting%MessageID%.xml, and then click OK. 12. In the Send Port Properties dialog box, in the Send pipeline list, ensure that PassThruTransmit is selected, and then click OK. Create a Send Port Group 1. In the BizTalk Server Administration Console, right-click Send Port Groups, point to New, and then click Send Port Group. 2. In the Send Port Group Properties dialog box, rename the send port to SendForAudit. 3. In the Send ports section, in the Name list, click ToAccountingFILE. 4. On the second line, in the Name list, click ToManagerFILE. 5. In the left pane, click Filters. 6. In the Property list, click Demo.Messaging.PropertySchema.State. 7. Verify that the Operator is set to ==. 8. In the Value box, type WA, and then click OK. Start the BizTalk Host Instance 1. In the BizTalk Server Administration Console, in the left pane, expand Platform Settings, and then click Host Instances. 2. Right-click BizTalkServerApplication, and then click Start. The BizTalk services may not have started automatically if the virtual machine took too much time to boot up. If you encounter any problems, you can double-click C:\AllFiles\startBtServices.cmd to ensure that they start. Start the Demos Application 1. In the BizTalk Server Administration Console, right-click Demos, and then click Start. 2. In the Start ‘Demos’ Application dialog box, click Options. Examine the Application artifacts and Hosts options. 3. In the Start ‘Demos’ Application dialog box, click Start. Test the Ports 1. In Windows Explorer, navigate to C:\AllFiles\DemoCode\Messages. 2. Open LargePO-WA.xml and SmallPO-WA.xml and then verify that the State field is WA. 3. In Windows Explorer, double-click LargePO-OR.xml and SmallPO-OR.xml, and then verify that the State field is OR. 4. Close Microsoft Internet Explorer. 5. Copy the four XML messages to the IN folder. Be sure to copy the message to the IN folder. If you move the message, it will be permanently modified by the BizTalk process. 6. Navigate to the ProcessedOrders folder, and notice the four messages there. 7. Navigate to the OUT folder and then notice that there are four messages: two that begin with ToManager and two that begin with ToAccounting. Only the messages with WA in the State field are sent using the send port group. 8. Close all open windows. 9. Pause the bt10d-demos virtual machine. Lesson 3: Monitoring Orchestrations and Messages Lesson objective: Track and monitor BizTalk messaging activity. Overview Microsoft BizTalk Server provides tools that can be used to troubleshoot messaging and orchestration processes. The BizTalk Administration console offers the BizTalk Group Hub page (the Hub), which can be used for viewing live and completed BizTalk service instances. Live instances are those that are currently executing, whether in an active, dehydrated, or suspended state. Dehydrated instances are those whose state information has been persisted and the resources in use released. These tools provide powerful search and summary capabilities to assist in identifying the overall health of a BizTalk system. What Is the BizTalk Group Hub Page? Describe the capabilities of the BizTalk Group Hub page and use the Hub to monitor and track live data. The Group Hub The BizTalk Server Group Hub provides IT professionals with an overview of the health of a BizTalk group by displaying live data relating to orchestrations, suspended service instances, host instances, and adapters. When working in the Hub, you can click any of the links to execute a query that returns details in a new page. You can also create and save your own queries. The Hub is divided into three sections that provide an overview of the health of your BizTalk Server system: Configuration Overview. This section, located at the top of the Group Hub page, indicates the overall health of the BizTalk group by displaying the number and state of Applications, Host Instances, and Adapter Handlers within the BizTalk Server Group. This section shows a red light or green light view of the health of these items. When initially accessed, the Hub displays a collection of links. However, you may need to refresh the page by pressing F5 before displaying the overall health of the group. Work in Progress and Suspended Items. This section displays the summary links for viewing dehydrated orchestrations, retrying and idle ports, and running, ready, scheduled, and suspended service instances. Grouped Suspended Service Instances. This section, the bottom section of the Hub page, displays summary links for viewing suspended service instances grouped in various ways. This grouping assists in managing suspended instances in a batch fashion. For instance, if several messages are all generating the same error, you will see them here grouped by the Error Code. Once the error has been resolved, all instances can be resumed (or terminated) at the same time. You can also use the Group Hub page to turn on tracking options for a message instance and view actual message data. You can create run and save custom queries in the Group Hub to troubleshoot problems with a particular message. Users must have BizTalk Administrative privileges to configure tracking for BizTalk Server. Running a Query in the Group Hub Use the BizTalk Hub page to query and view specific instance details. Viewing Instance Details When any link in the Hub is clicked, a new query page is opened with the Query Expression at the top of the window. This query can be modified to further restrict the results of the query. The Query Results section displays a count of the instances that match the query, with a detailed list of the resulting service instances below. Right-clicking any service instance allows you to perform the following actions related to that instance. Service Details. The Service Details option will show the status of the instance, for example, the instance ID, start time, or any errors that might have occurred. The Messages page shows all of the messages associated with this instance to date. On the Messages page, you can double-click any message and proceed to its details, where the message context and even the message itself can be viewed. Show Messages. This initiates a new query that returns the messages associated with this instance, which is the same list you would see if you navigate through Service Details. Show Instance Subscriptions. If the instance is a message related to an orchestration, instance subscriptions will be shown with this query. Terminate and Suspend Links. Use to suspend or terminate an individual instance. Rightclicking the summary in the results section allows you to suspend or terminate all of the related instances at once. Message Tracking Using the Tracked Message Event and Tracked Service Instance queries to monitor and track live and historical data. Overview A BizTalk Server messaging system can process thousands of messages in a day. Administrators, developers, and sometimes even business analysts require the ability to track messages and to view, monitor, and query the data related to messages being processed. The BizTalk Hub Tracked Message Event and Tracked Service Instance queries give users the ability to view the tracking details related to service instances whether these relate to message routing or orchestrations. Benefits Some of the capabilities of the Tracking queries include the ability to: Track when a service begins or ends or when a message is sent or received. The Tracking queries can reveal the start time and the end time for every message processed by BizTalk Server. The Tracking queries also show the state of any messages processed through BizTalk Server and the context properties of each of the messages. Message states include Active, Suspended, Completed, and Terminated. Monitor the health of operations, and create reports to analyze the current state of business processes. Every message goes through a series of contiguous processing steps; you can access this message flow by querying for messages and then right-clicking the message you want to track. After you click Message Flow from the context menu, you can see the processing steps for the message, and you can also see if any errors occurred while the message was being processed by BizTalk Server. Gather information about the state of a running process to be able to implement appropriate changes to the business process when needed. You can modify which data you want to track without affecting the rest of the BizTalk environment. Redeployment of assemblies is not necessary when changing tracking options. Changes to which properties are tracked is managed through the BizTalk Administration Console. Monitoring live data also enables you to monitor your system so you can fix problems in the development or staging environment. Make it possible for business users to track message activity when a problem arises at the business level so that the administrators or developers can investigate the problem further. A power user may determine that all messages in a particular format are getting hung up within an orchestration and save the problematic messages to disk to pass this information on to a developer who can remedy the problem. Analyze archived information. The BizTalk hub queries allow the user to access both live data and archived data. Analyzing archived data enables you to examine trends both in your business and in your system because you can look as far back as is necessary to troubleshoot a problem in a particular business process. Debug orchestrations. The Orchestration Debugger enables you to track the activity of a single orchestration instance on a shape-by-shape basis. It displays a rendered view of the orchestration created in the Orchestration Designer. You access the Orchestration Debugger through a shortcut menu by right-clicking any service or message instance associated with an orchestration type. You can switch back and forth between the Orchestration Debugger and the Message Flow view. You can configure exactly what information gets tracked, but note that tracking can have a negative impact on BizTalk message-processing performance. This is because at each data collection point, information has to be written to the tracking database. Care should be taken in enabling message tracking for this reason. The effects of tracking should be anticipated and included in the testing process to identify the overall impact. Identifying Types of Events and Data That Can Be Tracked Identify the types of events and data that can be tracked by BizTalk Server. Event Tracking BizTalk Server tracks data as events based on tracking filters you have set. You can configure BizTalk to track events such as: The starting or stopping of a service. The sending or receiving of messages. The beginning or ending of a pipeline. The beginning or ending of an orchestration. The execution of each shape in an orchestration. Data Tracking You can turn on tracking for any information in the message, including promoted properties, routing information, and partner data. The BizTalk Hub also allows you to see suspended orchestrations and pipeline information. The BizTalk Hub can be used to track: Promoted properties. This is useful when you have implemented content-based routing that uses promoted properties, and you want to locate messages that were processed. After you have turned on tracking for a promoted property in a schema, you can find messages in which the promoted property is a specific value. For example, if you want to see only messages that have a Quantity of less than 100, you can query the tracking database based on the Quantity promoted property. However, only promoted properties for which you have turned on tracking will show up in the Property drop-down list. Tracking is configured in the BizTalk Group Hub. Routing information. Because the BizTalk Hub allows you to trace the path of a message as it is routed through BizTalk Server, the BizTalk Hub can be very useful for troubleshooting errors. The BizTalk Hub can display error codes and routing states for a message so that you can troubleshoot errors in real time. Schema information. The BizTalk Hub allows you to find messages based on the schema type. This can be useful when you want to see only messages associated with a particular schema. Also, if you have promoted any properties in the schema that you are tracking, you can select the specific property you want to track from the Property drop-down list and narrow your search even further. Viewing and Tracking Message Activity Identify the types of events and data that can be tracked by using the BizTalk Hub Tracking Queries. Overview You can view both live and archived data by using various BizTalk Hub queries. You must have message body tracking turned on in order to save messages after processing of service instances is complete. Also, make sure the SQL Server Agent service is running on all MessageBox databases. This makes message bodies available to the BizTalk Group Hub and WMI, and it enables you to perform cleanup in the MessageBox database. Message body tracking is configured by using the BizTalk Group Hub page and is covered later in this module. Capabilities of the BizTalk Hub The BizTalk Hub enables you to: Track both live and archived data. Select schemas to investigate and filter messages. Messages can be filtered based on dates, the send or receive port that was used, trading partners (if available), and promoted properties. Display and track services such as orchestrations and messaging. The BizTalk Hub displays the state of each service instance returned from the query. Common service instance states include completed and terminated. View message instances including metadata and message body. If a message did have problems, you can easily find error messages to help troubleshoot the problem and then resolve it. Debug orchestrations in real time. The ability to debug a running business process in real time is a powerful feature of the BizTalk Hub that can be used to track the flow of a message through an orchestration. Demonstration: Tracking and Viewing Message Activity Learn how to use the BizTalk Group Hub to track message activity and examine message flow. Create the Message to Be Tracked In Windows Explorer, navigate to C:\AllFiles\DemoCode\Messages. 1. Resume the bt10d-demos virtual machine. 2. Copy LargePO-WA.xml to the IN folder. Be sure to copy the message to the IN folder. If you move the message, it will be permanently modified by the BizTalk process. Use BizTalk Tracking Queries to Track Completed Service Instances 1. In the BizTalk Administration Console, expand the BizTalk Server Administration node, and then click on BizTalk Group. The BizTalk Group Hub provides you with the ability to track and view archived message activity. The Group Hub page should be used to track, debug, and view running service instances. 2. On the New Query tab, select Tracked service instances from the list, then click Run Query . This query displays the most recently completed message instances. 3. Right-click the most recent Microsoft.BizTalk.DefaultPipelines.XMLReceive service instance, and then click Message Flow. 4. Maximize the Message Flow window. The top section has information about this service instance, such as the type, start and end times, and the host used. The bottom section has information about how the message came in and went out of this service. This is an instance of the XMLReceive default pipeline; it is associated with the PurchaseOrders port. When the XMLReceive pipeline finished processing the message, the message was sent to three different instances of the PassThruTransmit pipeline. Note: The order in which the message is sent by the send ports is non-deterministic. This means that the order that the message was sent through the ports in your instance may be different from the following steps. 5. Click the first Microsoft.BizTalk.DefaultPipelines.PassThruTransmit link under the PurchaseOrders port. Notice the name of the port and the URL. They will match one of the three send ports that you created. The ordering of delivery to the three send ports is notdeterministic, due to the publish/subscribe architecture. 6. Click the Microsoft.BizTalk.DefaultPipelines.XMLReceive link to go back to the XMLReceive pipeline instance. 7. Click the second Microsoft.BizTalk.DefaultPipelines.PassThruTransmit link under the PurchaseOrders port. Notice the URL and port used by this instance of the pipeline. It will be another one of the send ports created earlier. 8. Click the Microsoft.BizTalk.DefaultPipelines.XMLReceive link to go back to the XMLReceive pipeline instance. 9. Click the third Microsoft.BizTalk.DefaultPipelines.PassThruTransmit link under the PurchaseOrders port. This will display the name and URL for the third send port. 10. Close all windows and shut down the virtual machine. Lab: Routing BizTalk Messages Time estimate: 45 Minutes Scenario The BizTalk messaging engine allows you to route messages based on message context. In this lab, you will add existing BizTalk artifacts the messaging project. Next, you will promote properties in an XML schema to be used to selectively route messages. You will then configure BizTalk messaging ports to receive and route sales order messages, based on message context. Finally, you will test the port configuration by submitting test messages. Start the Virtual Machine Procedure List 1. If the Server Manager window is not already open, click on the Server Manager icon located in the task bar next to the Start button. 2. Expand Roles, Hyper-V, Hyper-V Manager. The last node to appear displays the machine name. Click on it to see the list of virtual machines available. 3. Double-click the virtual machine bt10d-05 to open a Virtual Machine Connection window. 4. Click on the Action menu in the Virtual Machine Connection window and choose Start. 5. Once the virtual machine starts, press CTRL+ALT+END. 6. Log on using the user name Administrator and the password pass@word1. 7. At the Windows Activation prompt, click Ask Me Later, and then click OK. Ensure that the BizTalk Services are started Procedure List 1. In Windows Explorer, navigate to C:\AllFiles. 2. Double click on startBtServices.cmd. 3. When prompted, press any key to close the command-line window. Exercise 1: Adding an Existing Schema and Map to the Project Overview Another BizTalk developer has given you BizTalk artifacts to use in your project. In this exercise, you will import an existing schema and map into the Messaging project. Add an Existing Schema and Map to the Project Procedure List 1. In Windows Explorer, navigate to C:\AllFiles\LabFiles\Lab5\AdvWorks and open the AdvWorks.sln file. 2. In Solution Explorer, right-click Messaging, point to Add, and then click Existing Item. 3. In the Add Existing Item – Messaging dialog box, navigate to C:\AllFiles\LabFiles\Lab5\AdvWorks\Artifacts. 4. While pressing the CTRL key, click LoanApplication.xsd and SalesOrder_To_LoanApp.btm, and then click Add. 5. In Solution Explorer, double-click the LoanApplication.xsd schema. 6. In the Schema Editor, right-click <Schema>, and then click Expand Schema Node. Examine the structure of the LoanApplication. 7. Close the LoanApplication.xsd schema window. 8. In Solution Explorer, double-click the SalesOrder_To_LoanApp.btm map. Examine the mapping of the nodes within the map. 9. Close the SalesOrder_To_LoanApp.btm map. Exercise 2: Promoting Schema Properties Overview Promoting a property makes specific data in a message accessible to the BizTalk messaging engine so messages can be routed based on their contents. In this exercise, you will promote several properties in the SalesOrder schema. Promote Properties Procedure List 1. In Solution Explorer, double-click SalesOrder.xsd. 2. In the Schema Editor, right-click <Schema>, and then click Expand Schema Node. 3. Under the Detail node, right-click OrderType, point to Promote, and then click Quick Promotion. 4. In the Microsoft Visual Studio message box, read the message, and then click OK. If a message box appears asking if you want to reload PropertySchema.xsd, click Yes. Notice in Solution Explorer that the PropertySchema.xsd artifact has been created. This is the default schema created to hold promoted properties. Also notice that the OrderType node now has an icon to indicate that the node has been promoted. 5. Ensure that PropertySchema.xsd is open in the schema editor. If it is not, double-click PropertySchema.xsd in Solution Explorer. Notice that the OrderType node is listed in the PropertySchema schema. The Property1 node is created by default and can be safely deleted. 6. Right-click Property1, and then click Delete. 7. In the BizTalk Editor dialog box, click Yes to confirm the deletion. 8. In Solution Explorer, right-click the Messaging project, and then click Deploy. Verify in the status bar that the deployment succeeded. Exercise 3: Creating a Receive Port and a Receive Location Overview You can use the BizTalk Server Administration Console to manage the BizTalk configuration database. BizTalk Server uses receive ports and receive locations to process inbound messages. In this exercise, you will create a receive port and receive location by using BizTalk Explorer. Create the SalesOrder Receive Port Procedure List 1. On the Start menu, click All Programs, then click Microsoft BizTalk Server 2010, and then click BizTalk Server Administration. 2. In BizTalk Administration Console, expand BizTalk Server Administration, BizTalk Group, Applications and BizTalk Application 1 3. Right-click Receive Ports, point to New, and click One-Way Receive Port. 4. In the Receive Port Properties dialog box, type SalesOrder in the Name box, then click OK. Create the SalesOrderFILE Receive Location Procedure List 1. In Windows Explorer, navigate to C:\AllFiles\LabFiles\Lab5. 2. Create a new folder, and name it SalesOrderIN. 3. In the BizTalk Administration Console, click the Receive Ports icon under BizTalk Application 1, then right-click the SalesOrder receive port, point to New, and click Receive Location. 4. In the Receive Location Properties dialog box, type SalesOrderFILE in the Name box. 5. Choose FILE from the Type drop-down list, and then click the Configure button. 6. In the FILE Transport Properties dialog box, click Browse. 7. In the Browse For Folder dialog box, navigate to C:\AllFiles\LabFiles\Lab5\SalesOrderIN, and then click OK twice. 8. In the Receive Pipeline drop-down list, choose XMLReceive, and then click OK to close the Receive Location Properties dialog box. Exercise 4: Creating a Send Port for All Orders Overview In this exercise, you will create a send port and configure it to subscribe to all messages received by the SalesOrder receive port. Create the SalesOrderFILE Send Port Procedure List 1. In the BizTalk Administration Console, right-click the Send Ports icon under BizTalk Application 1, point to New, and click Static One-Way Send Port. 2. In the Send Port Properties dialog box, in the Name box, type SalesOrderFILE. 3. Choose FILE from the Type drop-down list, and click the Configure button. 4. In the FILE Transport Properties dialog box, click Browse, browse to C:\AllFiles\LabFiles\Lab5, create a new folder named AllSalesOrders, and then click OK. 5. In the File name box, type Processed%MessageID%.xml, and then click OK. 6. In the Send Pipeline list, click PassThruTransmit. 7. In the left pane of the Send Port Properties dialog box, click Filters. 8. In the right pane, click the top box in the Property column, and choose BTS.ReceivePortName from the drop-down list. 9. In the Value box, type SalesOrder, and then click OK. Exercise 5: Creating a Send Port for Cash Orders Overview Subscriptions can be created based on multiple message properties. In this exercise, you will create a send port and configure it to subscribe to all messages that have the word CASH in the OrderType promoted property field, and are received by the SalesOrder port . Create the CashOrdersFILE Send Port Procedure List 1. Under BizTalk Application 1, right-click Send Ports, point to New, and then click Static One-Way Send Port. 2. In the Send Port Properties dialog box, in the Name box type CashOrdersFILE. 3. In the Type list, click FILE, and then click Configure. 4. In the FILE Transport Properties dialog box, click Browse, browse to C:\AllFiles\LabFiles\Lab5, create a new folder named CashOrders, and then click OK. 5. In the File name box, type Cash%MessageID%.xml, and then click OK. 6. In the Send pipeline list, click PassThruTransmit. 7. In the left pane of the Send Port Properties dialog box, click Filters. 8. In the right pane, click the top box in the Property column, and choose BTS.ReceivePortName from the drop-down list. 9. In the Value box, type SalesOrder. 10. On the second line, click AdvWorks.Messaging.PropertySchema.OrderType in the Property list. 11. In the Value box, type CASH, and then click OK. Exercise 6: Creating a Send Port for Credit Orders Overview You want all credit orders to be sent to a different port than the cash orders. In this exercise, you will create a send port and configure its filter to subscribe to all messages that contain the word CRED in the OrderType promoted property field, and are received by the SalesOrder port. Create the CreditOrdersFILE Send Port Procedure List 1. In the BizTalk Server Administration Console, under BizTalk Application 1, right-click Send Ports, point to New, and then click Static One-Way Send Port. 2. In the Send Port Properties dialog box, in the Name box, type CreditOrdersFILE. 3. In the Type list, click FILE, and then click Configure. 4. In the FILE Transport Properties dialog box, click Browse, browse to C:\AllFiles\LabFiles\Lab5, create a new folder named CreditOrders, and then click OK. 5. In the File name box type Credit%MessageID%.xml, and then click OK. 6. In the Send pipeline list, click PassThruTransmit. 7. In the left pane of the Send Port Properties dialog box, click Filters. 8. Click the top box in the Property column, and in the drop-down list, click BTS.ReceivePortName. 9. In the Value box, type SalesOrder. 10. On the second line, click AdvWorks.Messaging.PropertySchema.OrderType in the Property list, and then type CRED in the Value box. 11. In the left pane, click Outbound Maps. 12. In the Map list, click SalesOrder_To_LoanApp, and then click OK. Exercise 7: Testing the Port Configuration Overview Each sales order received through the SalesOrder receive port is sent to two of the three send ports, based on message context. In this exercise, you will start and enable the newly created ports and submit messages to test the message routing configuration. Start the Application Procedure List 1. In the BizTalk Server Administration Console, right-click the BizTalk Application 1 application, and then click Start. 2. In the Start ‘BizTalk Application 1’ Application dialog box, click Options. 3. Verify that all check boxes are selected, and then click Start. Test the Port Configuration Procedure List 1. In Windows Explorer, navigate to C:\AllFiles\LabFiles\Lab5. 2. Double-click CashSalesOrder1.xml, which will open the file in Microsoft Internet Explorer. Notice that the OrderType value is CASH. 3. In Windows Explorer, double-click CashSalesOrder2.xml. Notice that the OrderType value is CASH. 4. In Windows Explorer, double-click CredSalesOrder1.xml. Notice that the OrderType value is CRED. 5. In Windows Explorer, double-click CredSalesOrder2.xml. Notice that the OrderType value is CRED. 6. Close Internet Explorer. 7. Copy the four XML messages to the SalesOrderIN folder. It is important that you copy the messages to the SalesOrderIn, rather than moving them. Once the messages are processed, they can not be recovered for later testing. 8. Open the SalesOrderIN folder. The messages will disappear when BizTalk Server receives them. 9. In Windows Explorer, open the AllSalesOrders folder. It may take up to one minute for the messages to appear, but eventually, all four messages will be routed to this folder. 10. In Windows Explorer, open the CashOrders folder. The two CASH messages have been routed to this folder. 11. In Windows Explorer, open the CreditOrders folder. The two CRED messages have been routed to this folder. 12. Open one of the Credit{GUID}.xml messages. Notice that the message is in the LoanApp format because it was processed through the SalesOrder_To_LoanApp map used by the CreditOrders send port.