BizTalk Throttling
& Thresholds
Who am I?
• BizTalk Administrator for Bouvet in Norway
• Active contributer to the community
– Blog, www.biztalkadmin.com,
www.biztalkadminsblogging.com
– Microsoft TechNet Wiki, forums and galleries
• Speaker
• Trainer
Want to learn more?
BizTalk Admin Deep Dive class
Los Angeles 2nd - 4th of december
400$ discount for all attending BizTalk Boot Camp
Read more here:
www.biztalkadmin.com
So, is throttling good
or bad?
What is thresholds?
• Containers
• Limits
• Bounderies
How does thresholds affect BizTalk
• Designed to…
– manage server resources
– manage rate-based flow
– manage dehydration behavior
Who should change the default settings?
• When you see thresholds being hit due to
– Rate of messages
– Use of server resources
However 70% of all companies world-wide never
change the defualt setting for threshold, but should
they?
Why would you change them?
• Grant more resources to an application
• Limit the application to perform less
However these changes only comes into play when
you have to change them due to some of the above
bullets.
Resource-based throttling
Rate-based throttling
Orchestration throttling
So what is throttling then?
• Throttling is a first-aid kit to the environment
– Prevent melt-downs
– Prevent memory leaks
– Prevent developer to bring problems into production
• But it may also
– Help you to manage the use of resources based on
physical resources or rate of data flow
Are we throttling now?
• You can monitor your throttling states
• Performance Monitor
– Built in with BizTalk
• BizTalk Message Agent
• BizTalk360
– Built in throttling analyzer
Two differences!
Publishing
• Messages being sent into
the messagebox
Delivery
• Messages being sent out
from BizTalk
Publishing happens in
• Receive Ports
and may also appear in
• Orchestrations
• Send Ports with two-way communication
Delivery happens in
• Send ports
and may also appear in
• Orchestrations
• Receive ports with two-way communication
Publishing throttling states
2 – Incoming rate exceeds outgoing rate
Counter
Sub counter
BizTalk:Message Agent
Message publishing incoming rate
BizTalk:Message Agent
message publishing outgoing rate
Publishing throttling states
4 – memory exceeds specified threshold
Counter
Sub counter
BizTalk:Message Agent
High process memory
BizTalk:Message Agent
process memory usage (MB)
BizTalk:Message Agent
process memory usage threshold (MB)
Publishing throttling states
5- System memory exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
Physical memory usage threshold (MB)
BizTalk:Message Agent
Physical memory usage (MB)
Publishing throttling states
6 – Host message size
Counter
Sub counter
BizTalk:Message Agent
database size
BizTalk:MessageBox : General
Counters
Spool size
Tracking data size
BizTalk: MessageBox : Host Counters
Host queue length
Host queue suspend message length
Publishing throttling states
8 – Database session exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
Database session
Publishing throttling states
9 – Process count exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
Thread count
BizTalk:Message Agent
Thread count threshold
Delivery throttling states
1 – Incoming rate exceeds outgoing rate
Counter
Sub counter
BizTalk:Message Agent
Message publishing incoming rate
BizTalk:Message Agent
message publishing outgoing rate
Delivery throttling states
3 – in-process delivery exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
In-process message count
BizTalk:Message Agent
In-process message count threshold
Delivery throttling states
4 – Process memory exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
High process memory
Process memory usage (MB)
Process memory usage threshold (MB)
Active instance count
In-process message count
Delivery throttling states
5 - System memory exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
Physical memory usage
Delivery throttling states
9 -Process thread count exceeds threshold
Counter
Sub counter
BizTalk:Message Agent
Thread count
Thread count threshold
Demo
• Slow running application vs. normal application
I ask again, is throttling
good or bad?
BizTalk Tracking
Tracking, what?
• When is tracking needed
• What can it be used for?
• What are default settings
• Should you disable it in production
Artefact tracking
• Information about data going through the
environment
• Information gathered from artifacts in BizTalk
Global Tracking
• Setting to turn tracking on or off for the entire
environment
• Can be changed by right-clicking the group
Orchestration tracking
Track Events - Orchestration start and end
Select this check box to track the
orchestration instance before and after
processing of the entire business process.
Orchestration tracking enables you to see
the instances in the tracking query result
views.
Track Events - Message send and receive
Select this check box to track message
send and receive events. This check box is
available only if you select the Orchestration
start and end check box.
Track Events - Shape start and end
Select this check box when you need to
debug orchestration instances in the
Orchestration Debugger. When this check
box is selected, the event list in the
Orchestration Debugger is populated. This
check box is available only if you select the
Orchestration start and end check box.
Orchestration tracking
Track Message Bodies - Before orchestration
processing
Select this check box to save and track the
actual message content prior to processing by
the orchestration instance. This check box is
available only if you select the Message send
and receive check box.
Track Message Bodies - After orchestration
processing
Select this check box to save and track the
actual message content after processing by
the orchestration instance. This check box is
available only if you select the Message send
and receive check box.
Track Message Properties - Incoming
messages
Select this check box to track the promoted
properties of an inbound message.
Track Message Properties - Outgoing
messages
Select this check box to track the promoted
properties of an outbound message.
Receive Port tracking
Track Message Bodies - Request
message before port processing
Select this check box to save and track
message content before the message
is received.
Track Message Bodies - Request
message after port processing
Select this check box to save and track
message content after the message is
received.
Track Message Properties - Request
message before port processing
Select this check box to track the
promoted properties of an inbound
message.
Track Message Properties - Request
message after port processing
Select this check box if you want to
track the promoted properties of an
outbound message.
Send Port tracking
Track Message Bodies - Request message before
port processing
Select this check box to enable you to save and
track message content before the message is
received.
Track Message Bodies - Request message after
port processing
Select this check box to enable you to save and
track message content after the message is
received.
Track Message Properties - Request message
before port processing
Select this check box to track the promoted
properties of an inbound message.
Track Message Properties - Request message
after port processing
Select this check box if you want to track the
promoted properties of an outbound message.
BRE tracking
Fast activity
Select this check box to track the
instance data on which the policy
operates.
Condition evaluation
Select this check box to track the
true/false results of conditions in the
selected policy.
Rule firings
Select this check box to track the
actions started as a result of the
policy.
Agenda updates
Select this check box to track updates
to the agenda. The agenda contains a
list of actions that are "true" and need
to fire.
Schema tracking
• Select the schema you want to track
Pipeline tracking
Port start and end
events
Select this check box to track only when an instance starts and ends. Details
include item name, assembly, and other metadata.
Message send and
receive events
Select this check box to track message send and receive events. This check
box is available only if Port start and end events is selected.
Messages before
pipeline processing
Messages after
pipeline processing
Select this check box to save and track the message bodies received by the
pipeline, which holds metadata such as URLs and promoted properties. If this
is a receive pipeline, the message body is the raw message as submitted to
the pipeline by the transport component. Depending on the application, the
message might be encrypted, signed, or encoded. When using a BizTalk
map, if this is a receive pipeline, tracking takes place after the inbound map is
processed.
This check box is available only if Message send and receive events is
selected.
Select this check box to save and track the message bodies sent by the
pipeline, which holds metadata such as URLs and promoted properties. If this
is a receive pipeline, the message body is the processed message to be
submitted to the MessageBox database, which may be XML depending on
your application. When using a BizTalk map, if this is a send pipeline, tracking
takes place before the outbound map is processed.
This check box is available only if Message send and receive events is
selected.
Two ways to move tracking data
• One for moving the content of the messages
• One for the rest of the tracking data
Dedicated tracking host
• Moves all events and execute cleanup within the
tracking database
• One per MessageBox + one for redundancy
SQL Agent job
• TrackedMessages_Copy_BizTalkMsgBoxDb
moves all content to the tracking database
Tables in the MessageBox
• TrackingData_0_x contains data waiting to be
transferred to BAM
• TrackingData_1_x contains data waiting to be
transferred to the DTA database
Gathering statistics
• How long did my orchestration take?
• How long did it take from receiving till it was sent
• How many messages do I transfer
• Specific information regarding tracking based on
artifacts.
• When should you have maintenance hours
• How to keep statistics
How does it evolve
• Information gathered over time
• Simplify gathering by using PowerShell,
databases or other software programs like Excel
and PowerPivot
In and Out Events
Service Instances
Combination
Scripts and queries
•
Powershell script to delete old files
•
•
Find information for orphaned
messages
BizTalk Powershell to clean
messagebox and tracking database
•
Statistics from in and out events in
BizTalk
Count Send Ports by adapters in
BizTalk
•
Statistics from in out events grouped by
portname and date
Count Receive Location based on
adapter usage
•
Load more information from your
BizTalk Tracking database
•
Clean orphaned messages in BizTalk
•
Find recieve locations by address
•
Find send ports by port address
•
•
•
Large files in tracking database
•
Get statistics from Instances in BizTalk
•
Instances in Messagebox being
processed by a host instance
•
Stop all applications
Resources online for tracking
• BizTalk Server 2010:
BizTalkDTADb
• BizTalk Server Resources
on the TechNet Wiki
• BizTalk Server:
Administrator Queries and
Scripts
• White Paper: Proactivity in
BizTalk