Transaction Processing In a Dynamic IT environment, the back-end systems that are used to process transactions and store data must provide the availability, performance, and security demanded by the organization. This document will explain how to achieve a Dynamic level of availability, scalability, and security for your processing servers as dictated by business requirements. Capability: Transaction Processing Applies to: Microsoft BizTalk, Microsoft SharePoint, Microsoft SQL Server, SQL Azure, Internet Information Services Attributes: High Availability, Management, Performance, Security Author: Allan Hirt Published: December 2010 Disclaimer This document is provided “as-is.” Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. © 2010 Microsoft. All rights reserved. ii Contents Introduction .................................................................................................................................................. 1 Transaction Processing ............................................................................................................................. 1 Steps to a Dynamic Environment .............................................................................................................. 2 Plan Available, Scalable, and Secure Application Solutions .......................................................................... 2 Deploy Available, Scalable, and Secure Application Solutions ..................................................................... 7 Manage Available, Scalable, and Secure Application Solutions.................................................................. 11 Conclusion ................................................................................................................................................... 17 iii Introduction Microsoft Infrastructure Optimization (IO) is based on three information technology (IT) models: Core IO, Application Platform Optimization, and Business Productivity IO. Each of these models contains four levels of process maturity and capability classifications as logical groupings of requirements for each level of maturity. Application Platform Optimization focuses on aligning applications with core business processes, assessing the workload performance of existing business-critical applications, and prioritizing the application and infrastructure investments that can drive the greatest return for businesses. Maturity levels in the Application Platform optimization model are Basic, Standardized, Advanced, and Dynamic. This guide contains checklists to help move from the Advanced level to the Dynamic level for the Transaction Processing capability in the Application Platform Optimization model. See Application Platform Optimization at http://www.microsoft.com/enterprise/solutions/application-platformoptimization/default.aspx for more information about Application Platform Optimization. Transaction Processing IT systems must meet the uptime, performance, and security needs demanded by the business. The only way to achieve that is to plan, deploy, and manage these systems correctly—not just by putting the right technologies in place, but also the right people and processes. In contrast to implementing a platform or technology on its own with no links to anything else, each application may support only certain technologies or features, so any choices made would be specific to what is ultimately deployed. Application owners and developers must take that into account from the start, as IT groups cannot always compensate for issues that are thrust upon them via application decisions. The whole IT ecosystem—including the applications chosen—must meet your company’s standards. In a Standardized environment, there may be little attention to availability, scalability, and comprehensive security; the goal is often to just get up and running. Rationalized environments acknowledge availability, security, and performance needs related to applications in more detail, but the processes can often be manual or ad hoc; often advanced solutions to support events (such as site failures) are not taken into account. A Dynamic IT environment should automate most of the common tasks to ensure availability, security, and performance for an application, and take into account offpremise solutions where applicable. Steps to a Dynamic Environment Figure 1. Steps for deploying a Dynamic, highly available, scalable, and secure application solution This document explains the steps organizations can take to plan, deploy, and administer highly available and scalable systems for applications. Figure 1 illustrates steps involved in achieving these goals; the remaining sections will provide the specific tasks necessary for each step. Plan Available, Scalable, and Secure Application Solutions Figure 2. Plan available, scalable, and secure application solutions This step will guide you through the pre-deployment planning tasks for systems to support the availability, performance, and security needs of the applications. These tasks, along with most others in this document, are based upon those in the Plan and Operate phases of Microsoft Operations Framework (MOF). If you have not had prior experience with MOF, it is important to familiarize yourself with its concepts. Table 1 lists the inputs (prerequisites or necessary information) for completing this step and the outcomes (or deliverables) of its completion. Table 1. Plan Available, Scalable, and Secure Application Solutions: Inputs and Outcomes Category Description Inputs 2 MOF experience: Understanding iterative processes based on MOF to properly plan, implement, and operate your environments is crucial to any Dynamic IT environment. Business requirements: You cannot implement any solutions unless you understand the availability and performance requirements for each system. Those Category Description start at the business level. Outcomes Service Level Agreements (SLAs) and other related agreements: Based on the business needs and the capabilities of your IT environment, you can then devise the formal availability and performance metrics that the deployments will need to meet. Capacity planning information: For Dynamic IT, you must have the proper information to scale the systems so that they can meet the long term needs of the business. Technical requirements: To perform the implementation, you must have all the availability and other technical requirements in place. Table 2 explains the pre-deployment tasks for systems that will be used for an application solution. After completing this step, you will be ready to deploy the servers that will perform the application work (see Figure 2). Table 2. Plan Available, Scalable, and Secure Application Solutions Task Description Reference Understand the basics of high availability. Understand fundamental concepts such as uptime, service level agreements (SLAs), recovery point objectives (RPOs), recovery time objectives (RTOs), and how to calculate availability. Once you understand them, make sure they are standardized and applied in the same way across your organization. Windows Server 2008 High Availability Program whitepaper Define availability requirements and put formal agreements in place. No technology can be chosen, planned for, deployed, or administered unless the requirements are fully documented and understood by everyone. All requirements start from the business, not IT, and are rationalized by IT. Documents found in MOF Job Aids – Plan.zip: SLAs should be put in place along with other formalized agreements, such as the operations expectations and recovery objectives. Anything that needs to be defined should be done at this stage of pre-deployment. Agreements may need to be put in place 3 Microsoft High Availability Overview White Paper MOF Job Aid – Operating Level Agreement.doc MOF Job Aid – Operations Level Agreement.doc MOF Job Aid – Service Level Agreement Template.doc MOF Job Aid – Operations and Services Description Template.doc in MOF Job Aids – Task Determine risks associated with the deployment. Description Reference between multiple parties (IT and the application owner, IT and the business, IT and the DBAs, and so on) to ensure the availability is met at all levels and proper expectations are set. Operate.zip Risk is a key factor in determining the architecture for an availability solution. Availability is based on the concept that risk is mitigated to increase uptime. “MOF Job Aid – Risk Template Tool.xls” in MOF Job Aids – Manage.zip, available for download at http://www.microsoft.com/do wnloads/en/details.aspx?Famil yId=457ED61D-27B8-49D1BACAB175E8F54C0C&displaylang=en All ZIP files can be downloaded at http://www.microsoft.com/do wnloads/en/details.aspx?Famil yId=457ED61D-27B8-49D1BACAB175E8F54C0C&displaylang=en Determine a How much can be spent (including the licensing budget for the of all software-based components) must be determined early on. A mission-critical, Dynamic solution. IT environment cannot afford to under- or overspend, as that may lead to capacity issues at a later point. The budget may need to be adjusted based on the documented requirements, or the requirements may need to be adjusted if the budget will not support them. Microsoft Operations Framework Understand and determine the right availability technologies based on application support. BizTalk: To deploy the availability architecture, you must understand what availability technologies and methods each application platform supports. Planning for High Availability Performance and Capacity Planning SharePoint: Plan for Availability SQL Server: 4 Task Description Reference High Availability Solutions Overview Understand the security requirements for the application platform. To secure each application platform, you must understand what options are available to you and what is important to secure. BizTalk: Security and Protection Internet Information Services: IIS 7 Security Improvements SharePoint Server: Plan Site and Content Security SQL Server: Security Overview (Database Engine) Understand the performance requirements for the application platform. To ensure that each layer scales and performs to its best abilities, you must understand what options are available to you. Scaling Strategies for ASP.NET Applications Configure a test and/or staging environment. A test environment that is set up to support and mimic the eventual production environment is a key success factor. Microsoft Operations Framework For IT, nothing is more important during the planning phase than properly testing the application under the proposed workload and then using the numbers captured as part of a capacity planning exercise to purchase the right hardware, or to see if current hardware or solutions (such as virtualization) can be leveraged. A Dynamic IT environment will have automated testing for the required workloads, which should represent the peak that is Testing the Application (Visual Studio Test Manager) Perform testing and capacity planning. 5 Capacity Management for Share Point 2010 BizTalk 2009 Benchmark and Assessment Questions Task Description Reference anticipated for the length the systems will be in place. Ensure that change control is implemented in your environment. All mission-critical environments require a formal change control process to ensure that all work is tracked. Purchase support agreements. A crucial component of any mission-critical deployment is the purchase of support agreements from all hardware and software vendors that meet the availability requirements. While these may be costly, in a server- (or site-) down situation, you will not be able to meet your goals with support agreements that exceed the agreed-upon SLAs. Hire the right people and put them in the right roles. Having the right people in the right place at the right time means more than just having someone on call or on a schedule. If a system goes down, the right person should be doing the work to bring it up, whether that person is on site or on call (perhaps in a accessing the system remotely). For example, you should never have your most junior administrator as the only available staff at a crucial time for the business. Always ensure that someone who is more senior is available, and use a tiered system to ensure that those who are working the most will not become overtaxed. Qualify any hosting or cloud vendor if applicable. Many IT organizations are outsourcing their IT environments or moving to the cloud. If you are either considering these moves or are in the process, make sure that you scrutinize the vendor carefully. For example, if a server goes down, what is their response time? Can you get access to their data center 24/7 should you need to go onsite? Even if your organization does not 6 Microsoft Operations Framework Task Description Reference own certain servers, you must take steps to ensure their availability. Deploy Available, Scalable, and Secure Application Solutions Figure 3. Deploy available, scalable, and secure application solutions Once the planning tasks are complete, you can implement the application platform architecture. The tasks in this step account for the various popular application servers Microsoft produces. Table 3 lists the inputs (prerequisites or necessary information) for completing this step and the outcomes (or deliverables) of its completion. Table 3. Deploy Available, Scalable, and Secure Application Solutions: Inputs and Outcomes Category Description Inputs Outcomes Capacity planning information: For Dynamic IT, you must have the proper information to scale the systems so that they can meet the long-term needs of the business. Technical requirements: To perform the implementation, you must have all the availability and other technical requirements to properly put things in place. Required software and settings: To standardize how each server will ultimately look and feel, a Dynamic IT environment must devise a list of what the server build should have. Application Platform Deployed: The components used for the application platform from an infrastructure perspective will be deployed. Table 4 documents the key tasks involved in deploying the application platform infrastructure. After completing this set of tasks, your application infrastructure will be deployed. 7 Table 4. Deploy Available, Scalable, and Secure Application Solutions Task Description Reference Configure the hardware or virtual machines. Ensure that the underlying hardware or virtual machines are configured properly, and that their architecture is adequate for performance. Remember that for an application platform, I/O performance may matter, so implementing your disk subsystem will matter. BizTalk: BizTalk virtualization support policy BizTalk Server 2009 Hyper-V Guide SQL Server: SQL Server virtualization support policy Planning, Implementing, and Supporting SQL Server Virtualization with Windows Server 2008 R2 and Hyper-V Live Migration SharePoint: SharePoint virtualization support policy SharePoint 2010 Virtualization Notes Virtualization for SharePoint 2010 Automate and standardize base Windows deployments. Deploy a failover cluster (optional). 8 Good application deployments start with a solid Windows build. Builds that are standardized will ensure a consistent experience and increase reliability. “Automate and Standardize Base Windows Deployments” in the Mission Critical document High Availability Depending on both your requirements and what the application platform supports, you may need to configure a Windows failover cluster as part of the implementation. “Deploy a Failover Cluster” in the Mission Critical document High Availability Improving Fault Tolerance in BizTalk Server by Using a Windows Server Cluster Task Description Reference Planning, Implementing and Supporting SQL Server Virtualization with Windows Server 2008 R2 Hyper-V and Live Migration Deploy a network load balancing cluster (optional). Deploy application platforms (optional). Depending on both your requirements and what the application platform supports, you may need to configure a Network Load Balancing cluster as part of the implementation. “Deploy a Network Load Balancing Cluster” in the mission critical document High Availability Each major application has its own method of installation. Some are partially referenced above with the clustering and virtualization links. BizTalk: IIS Server (Web Farm) Deployment SharePoint: Deployment Internet Information Services: Setting up a Server Farm with the Web Farm Framework 2.0 Beta for IIS 7 Consolidate SQL Server workloads. As part of the deployment and optimization process, you may want to consolidate your workloads to save money and resources. SQL Server Consolidation Guidance Consolidation Using SQL Server 2008 Consolidation of BizTalk Server Databases Implement specific features for availability or disaster recovery. In addition to building on top of a Windowsbased cluster (which is generally as a part of installation), other methods can be used to make the application platforms (or parts thereof) available. SQL Server: Database Mirroring Log Shipping SQL Server Replication 9 Task Description Reference High Availability: Interoperability and Coexistence BizTalk: Clustering the BizTalk Server Databases Scaling Out the BizTalk Server Databases Disaster Recovery SharePoint: Plan for disaster recovery: hot standby farms Using Database Mirroring with Office SharePoint Server and Windows SharePoint Services Availability configuration Be Prepared: A Guide to SharePoint Disaster Prevention and Recovery Step-by-Step SharePoint DisasterRecovery (WindowsITPro magazine article) Synchronize databases with SQL Azure (optional). If you want to synchronize your SQL Server databases to create business continuity in the cloud using SQL Azure, you can take advantage of the methods linked in the column to the right. Introduction to Sync Framework Database Synchronization SQL Azure Data Sync SQL Server to SQL Azure Synchronization using Sync 10 Task Description Reference Framework How to Sync Large SQL Server Databases to SQL Azure Ensure a secure deployment. As part of overall deployment, the back-end application server platforms must be secured according to the guidelines set by the vendors. SQL Server: Secure Deployment Security Overview for Database Administrators BizTalk: Securing Your Deployment of BizTalk Server 2010 Internet Information Services: Configuring Security SharePoint: Security and Authentication for SharePoint 2010 Manage Available, Scalable, and Secure Application Solutions Figure 4. Manage available, scalable, and secure application solutions You must remain vigilant to maintain the availability, performance, and security of your systems. Some tasks only need to occur occasionally, while others may need constant care and feeding. 11 Table 5 lists the inputs (prerequisites or necessary information) for completing this step and the outcomes (or deliverables) of its completion. Table 5. Manage Available, Scalable, and Secure Application Solutions: Inputs and Outcomes Category Description Inputs Outcomes SLAs and other related requirements: As with deployment, you need the availability, performance, and security-related requirements to be able to put the proper administration in place to maintain systems on a day-to-day basis. Skill set evaluation: Additionally, you must understand the skills of the administrators working in your environment to put the proper administration in place. Putting in technologies and solutions that cannot be maintained by your administrators can do more harm than good. Process evaluation: Know where the lines of delineation are in your IT group (for example, what constitutes a DBA task versus one for a security administrator). Communication and vendor relationships: A core part of any administration is smooth communication—both internally and with any vendor whose products are represented in the application solution stack. Systems properly administered: Systems will have the right administration in place to maintain their performance, availability, and security. Table 6 documents the administration tasks that ensure availability. After completing this step, you will administer servers to maintain their availability (see Figure 6). Table 6. Manage Available, Scalable, and Secure Application Solutions Task Description Administer systems remotely. One way to minimize risk to your servers is to administer them remotely with the proper tools (for example, by using SQL Server Management Studio to administer SQL Server databases). If you do this, there is a much smaller chance of someone doing the wrong thing accidentally, such as shutting down the server instead of logging off from a remote session. If you do need to be on the server to perform tasks, use a remote tool such as Remote Desktop Connection. Remote Desktop Services Document everything and Documentation is an important tool for availability. It is crucial to have not only the Asset Intelligence in Configuration Manager 12 Reference Task Description Reference keep the documentation updated. required task steps documented (in the event anyone would need to execute them) but also contact information, application owner information, and everything in between. In the event of an outage, this information will be invaluable. However, if the information is not kept up to date, the problem could be compounded. Microsoft Assessment and Planning Toolkit Test availability and/or disaster plans. You must devise end-to-end plans for availability and/or disaster recovery must which incorporate all aspects of the systems (applications, Windows, SQL Server, and so on). Documents: Plans are only helpful if they are tested and executed. In the event that you actually have an outage, it is important to know you have a working availability plan and know how long it takes to execute. Implement monitoring for performance and availability. Audit for security events and 13 MOF Job Aid – Test Cases Workbook.doc MOF Job Aid – Test Plan.doc MOF Job Aid – Test Specification.doc in MOF Job Aids – Deliver.zip, found here All servers in an IT environment will require monitoring. With the proper monitoring, problems of all sorts can be detected proactively, whether they are related to availability, performance, or another factor. Items related to uptime, health, and performance are important for system reliability; keep in mind that something as simple as a performance problem can lead to an availability problem. It is also important to configure a monitoring solution so that you can be proactive and address problems before they cause outages. Information gathered during monitoring should also be stored for a period of time and archived. This historical information will help during troubleshooting, and will also assist with capacity planning during an upgrade. “Checklist: Monitoring BizTalk Server with Operations Manager 2007” in the BizTalk Server 2009 Operations Guide Maintaining security is a continual process. You must check for violations not only of your basic Run the Microsoft Baseline Security Monitoring BizTalk Server SharePoint: Health Monitoring Task Description Reference compliance. security rules and regulations, but also configuration changes which would put your applications at risk. Analyzer periodically SharePoint: Item-Level Auditing with SharePoint Server 2007 SQL Server: Auditing (Database Engine) Administering Servers by Using Policy-Based Management Implement a backup and restore strategy. The first administrative task any IT administrator must do to ensure availability is to devise a backup and restore strategy. In the event of a catastrophic problem, the only thing you may have at your disposal is a backup of the missioncritical system or data to be restored. BizTalk: BizTalk backup tool Backing Up and Restoring BizTalk Server SharePoint: Backup and recovery planning Backup and recovery Troubleshoot performance problems. As systems are used over time, they may experience some performance degradation. It might be caused either by the usage of the application, or by other factors, such as hardware or networking. These incidents must be handled as they occur. In a Dynamic IT environment, they should ideally be automatically detected and recurring common issues should be remediated for. SQL Azure: Troubleshooting and Optimizing Queries with SQL Azure whitepaper SQL Server: Troubleshooting Performance Problems in SQL Server 2008 whitepaper BizTalk: 14 Task Description Reference BizTalk Server 2009 Performance Optimization Guide SharePoint: Improving SharePoint Performance SharePoint Performance Tuning References Perform normal operations. Outside the specific tasks listed here, there are some general tasks specific to each platform that IT must perform. SharePoint: Operations for SharePoint Server 2010 Database Maintenance for Microsoft SharePoint Products and Technologies BizTalk: BizTalk Server 2009 Operations Guide Test and apply updates periodically. At one point or another, various components will require a patch or some sort of update. For example, Microsoft releases security updates on the second Tuesday of every month; they are documented in corresponding security bulletins at the Microsoft Security Center. Updates may be deployed as part of a scheduled outage, or if a system is down, a patch may be required to get it back up and running. Effective management of server updates is crucial to server availability; introducing a change—especially one that is not tested—can affect both performance and reliability. Updates such as service packs for Microsoft products do have an expiration date, so you 15 See the Patch and Update Process documentation in the Mission Critical document set. Microsoft Support Lifecycle website Task Description Reference should always plan to test and implement a newer version as soon as is reasonable after its release. Periodically refresh servers and major versions. All product versions, including those from Microsoft, have a lifecycle determined by their manufacturer. If servers are left running older versions, they become unsupported. Missioncritical systems should have full support to ensure that in the event of a problem, the vendor’s support organization can effectively handle a case. Most companies that deploy solutions have a hardware lifecycle for the equipment purchased, so version replacements or upgrades should be considered in the prescribed timeframes to minimize disruption and increase uptime. 16 Microsoft Support Lifecycle website Conclusion While front-end applications are the tools end users work with daily, it is the back end systems in any organization that do the most demanding work. These systems must provide the reliability and performance required by the business. If they do not, the business may not be operating optimally. To ensure high availability, scalability, and security for the application platforms, a Dynamic IT environment must take the steps outlined in this document. For additional information, see the following resources: 17 SharePoint 2010 documentation: http://technet.microsoft.com/en-us/library/ee428287.aspx BizTalk 2010 documentation: http://technet.microsoft.com/en-us/library/ee317850(BTS.10).aspx SQL Server 2008 R2 documentation: http://technet.microsoft.com/enus/library/bb418432(SQL.10).aspx SQL Azure documentation: http://technet.microsoft.com/en-us/library/ee336279.aspx Microsoft BizTalk Server 2009 Performance Optimization Guide: http://msdn.microsoft.com/enus/library/ee377064(v=BTS.10).aspx SQL Server 2005 failover clustering: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=818234dc-a17b-4f09-b282c6830fead499&DisplayLang=en SQL Server 2008 failover clustering: http://download.microsoft.com/download/6/9/D/69D1FEA75B42-437A-B3BA-A4AD13E34EF6/SQLServer2008FailoverCluster.docx High Availability with SQL Server 2008 R2: http://technet.microsoft.com/enus/library/ff658546(SQL.100).aspx How to Implement Kerberos Constrained Delegation with SQL Server 2008: http://technet.microsoft.com/en-us/library/ee191523(SQL.100).aspx Troubleshooting Performance Problems in SQL Server 2008: http://technet.microsoft.com/enus/library/dd672789(SQL.100).aspx Partitioned Tables and Index Strategies Using SQL Server 2008: http://technet.microsoft.com/enus/library/dd578580(SQL.100).aspx Plan Caching in SQL Server 2008: http://technet.microsoft.com/enus/library/ee343986(SQL.100).aspx Disk Partition Alignment Best Practices for SQL Server: http://technet.microsoft.com/enus/library/dd758814(SQL.100).aspx SQL Server 2005 Waits and Queues: http://technet.microsoft.com/en-us/library/cc966413.aspx SQL Server Customer Advisory Team website: http://sqlcat.com/ Scaling out with SQL Azure: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=4931D76F-0E21-44A7-8699852EEC6EE029 Geo-Replication Performance Gains with Microsoft SQL Server 2008 Running on Windows Server 2008: http://technet.microsoft.com/en-us/library/dd263442(SQL.100).aspx