Proactivity in BizTalk? Tord Glad Nordahl 1 18. October 2012 Proactivity - All environments are unique Who am I? – – – – – – – BizTalk Administrator Expert Training Health Checking Debugger Speaker Active contributor at TechNet Active blogger www.biztalkadmin.com Profile Profile 2 18. October 2012 Proactivity - All environments are unique Proactivity in BizTalk • • • • 3 What is meant by proactivity How can you know Where do you look And how do you prevent it? 18. October 2012 Proactivity - All environments are unique Make and keep it stable • Monitoring tools – – – – – 4 SCOM BizTalk 360 Messagebox Viewer BizTalk Best Practice Analyzer Microsoft Baseline Analyzer 18. October 2012 Proactivity - All environments are unique Nothing beats your knowledge! BizTalk • Performance Counters • PAL • Statistics • Disk space • Throttling • Thresholds • Event log • Known bugs • Infrastructure 5 18. October 2012 SQL • SQL Agents • Performance counters • Databases • Event log • Disks • SQL Queries Proactivity - All environments are unique How long does it take? • • • • • • • • • • • • • 6 Backup BizTalk Server CleanupBTFExpiredEntriesJob_BizTalkMgmtDb MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb MessageBox_Message_Cleanup_BizTalkMsgBoxDb MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb MessageBox_Parts_Cleanup_BizTalkMsgBoxDb MessageBox_UpdateStats_BizTalkMsgBoxDb Monitor BizTalk Server Operations_OperateOnInstances_OnMaster_BizTalkMsgBoxDb PurgeSubscriptionsJob_BizTalkMsgBoxDb Rules_Database_Cleanup_BizTalkRuleEngineDb TrackedMessages_Copy_BizTalkMsgBoxDb DTA Purge and Archive 18. October 2012 Proactivity - All environments are unique Performance counters • Needs to be monitored over time • Difference between server related and SQL • Provides you with live and old data 7 18. October 2012 Proactivity - All environments are unique Most important databases • • • • 8 BizTalkMsgBoxDb BizTalkDTADb BizTalkMgmtDb SSODB 18. October 2012 Proactivity - All environments are unique BizTalkMsgBoxDb • • • • • 9 Spool TrackingData_x_x Instances Marklog X-RefcountLog 18. October 2012 Proactivity - All environments are unique BizTalkDTADb • • • • 10 DebugTrace MessageInOutEvents ServiceInstances Marklog 18. October 2012 Proactivity - All environments are unique SQL queries and deadlocks • Make sure you avoid deadlocks • Performance monitor • SQL Profiler 11 18. October 2012 Proactivity - All environments are unique Disks • If your using it, make it fast • Check for I\O issues • Never go below 15% free disk space 12 18. October 2012 Proactivity - All environments are unique Tip: To see all counters at least one host must be running! BizTalk Performance counters 13 18. October 2012 Proactivity - All environments are unique Performance Counter • BizTalk: Message Agent – – – – – – – – – – – – – – – Active Instance Count Database sessions Database session threshold Database size High database session High database size High in-process message count High message delivery rate High message publishing rate High Process memory High System memory High thread count In-process message count In-process message count threshold Message delivery delay (ms) Performance Counter • BizTalk: Message Agent – – – – – – – – – – – – – – – Message delivery incoming rate Message delivery outgoing rate Message delivery throttling state Message delivery throttling state duration Message delivery throttling user override Message publishing delay (ms) Message publishing incoming rate Message publishing outgoing rate Message publishing throttling state Message publishing throttling state duration Message publishing throttling user override Physical memory usage (MB) Process memory usage (MB) Service Class ID Thread count Performance Counter • BizTalk: Message Agent – – – – Thread count threshold Total batches omitted Total message delivered Total messages published Performance Counters • BizTalk: Message Box: General counters – – – – – – – – – Instances (total number) MsgBox Dead processes cleanup (purge job) MsgBox Msg Cleanup (purge job) MsgBox Parts Cleanup (purge job) MsgBox Purge Subscriptions job (purge job) Spool size Tracked msg body (purge job) Tracking data size Tracking spool cleanup Performance Counters • BizTalk: Message Box: Host counters – – – – Host queue – Instance state msg refs – length Host queue – length Host queue – number of instances Host queue – suspended messages - lengthr Performance Counters • BizTalk: TDDS – – – – – – – – – – – Batches being processed Batches committed Events being processed Events committed Records being processed Records committed Total batches Total events Total failed batches Total failed events Total records Performance counters • BizTalk:Messaging – – – – – – – – – – – – – – Active Receive Location Active Receive Threads Active Send Messages Active Send Threads Documents processed (per second) Documents received (per second) Documents resubmitted (per second) Documents suspended Documents transmitted/batch ID Process Pending Receive Batches Pending transmitted messages Request/response timeouts Throttled receive batches Performance Counters • BizTalk:Messaging Latency – – – – Inbound Latency (sec) Outbound Adapter Latency (sec) Outbound Latency (sec) Request-Response Latency (sec) Performance counters • BizTalk:<adapterName>_Receive / Send adapter – – – – – – – – Bytes Received Bytes Received per second Deleted retries Lock failures Lock failures per second Message received Messages received per second Time to build batch Throttling • Good or bad? • Thresholds met • Performance stalls! 23 18. October 2012 Proactivity - All environments are unique Thresholds • Add more resources and change limits • Remove resources and limit the application 24 18. October 2012 Proactivity - All environments are unique Event log • Look for errors • Monitor and trigger on errors • Keep it clean 25 18. October 2012 Proactivity - All environments are unique Known bugs • You should be aware of all known bugs – – – – 26 CU updates WCF-SQL, Oracle Custom thread issue on receive 5649 event ID etc 18. October 2012 Proactivity - All environments are unique Infrastructure Have some idea of your infrastructure: • Virtualization how and where • Locations • Connections and people 27 18. October 2012 Proactivity - All environments are unique Questions? 28 18. October 2012 Proactivity - All environments are unique