DEP 313 Windows Server 2003 Active Directory Branch Office Deployment Lothar Zeitler Snr. Consultant Microsoft Services Germany AD Branch Office Scenario ROOTDC1- GC 10.0.0.1 DNS corp.contoso.com HUBDC1 - FSMO DNS 10.0.0.10 branches.corp.contoso.com ROOT2DC2 - FSMO 10.0.0.2 DNS corp.contoso.com HQDC1 - FSMO 10.0.0.3 DNS hq.corp.contoso.com HQDC2 - GC 10.0.0.5 DNS hq.corp.contoso.com MOMSVR 10.0.0.26 MOM Server corp.contoso.com TOOLMGRSVR 10.0.0.4 Monitoring Server corp.contoso.com Data-Center-Site HUBDC2 DNS 10.0.0.11 branches.corp.contoso.com BHDC1 - GC BHDC2 - GC DNS 10.0.0.14 DNS 10.0.0.12 branches.corp.contoso.com 10.0.0.13 branches.corp.contoso.com branches.corp.contoso.com STAGINGDC1 DNS GC 10.0.0.25 branches.corp.contoso.com TSDCSERVER 10.0.0.20 ADS Server corp.contoso.com BHDC4 - GC DNS 10.0.0.15 BHDC3 - GC branches.corp.contoso.com DNS BOSite1 BOSite2 BOSite3 BOSite4 BOSiten BODC1 DC BODC2 DC BODC3 DC BODC4 DC BODC n DC Staging-Site What Makes a Branch Office Design Interesting? IP connectivity incl. WAN, link speed, Dial on demand, routers, firewalls, IPSEC Name resolution incl. DNS server, zone and client configuration Active Directory replication to a large number of replication partners FRS replication Group policy implementation Happy users Considerations Proper care of DNS name resolution will guarantee replication success IPSEC preferred firewall solution Active Directory Branch Office Guide Windows 2000 version published http://www.microsoft.com/technet/treeview/default.as p?url=/technet/prodtechnol/ad/windows2000/deploy/ adguide/default.asp Recommends managing connection objects manually Windows 2003 can be deployed using that guide Windows 2003 version in draft All deployment steps are tested in scalability lab Biggest deployment so far: 2.400 sites Goal: 3.500 sites Published September 2003 New Features in Windows 2003 for Branch Office Deployments KCC improvements KCC/ISTG inter-site topology generation Bridgehead Server load-balancing and connection object loadbalancing tool KCC redundant connection object mode for branch offices No more “keep connection objects” mode if replication topology is not 100% closed Better event logging to find disconnected sites Replication improvements Linked-Valued Replication More replication priorities Intra-Site before Inter-Site NC priorities: Schema -> Config -> domain -> GC Notifications clean-up after site move Lingering Object detection New Features in Windows 2003 for Branch Office Deployments No GC full-sync In Windows 2000, schema changes that changed the PAS triggered GC full sync Removed in Windows 2003 Universal Group Caching DNS Improvements Install from media FRS improvements Plus many more…. Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Forest Design Follow recommendations in Windows 2003 Deployment Kit (Chapter 2) http://www.microsoft.com/downloads/details.aspx?familyid=6cde6ee 7-5df1-4394-92ed-2147c3a9ebbe&displaylang=en Reasons for having multiple forests Political / organizational reasons Unlikely in branch office scenarios Too many locations where domain controllers must be deployed Complexity of deployment Too many objects in the directory Should be partitioned on domain level GCs too big? Evaluate not deploying GCs to branch offices Windows 2003: Universal group caching Recommendation: Deploy single forest for Branch Offices Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Centralized vs. Decentralized Domain Controller Deployment The number of sites with domain controllers defines the scope of the deployment Deployment options Centralized deployment Domain controllers are located in datacenters / hub sites only Users in branches logon over WAN link De-centralized deployment All branches have domain controllers Users can logon even if WAN is down Mixed model Some branches have DCs, some don’t Centralized deployment has lower cost of ownership Easier to operate, monitor, troubleshoot Design Considerations for Domain Controller Placement Local DC requires physical security Domain controller management Monitoring, auditing, SP deployment etc. must be guaranteed Required services – business drivers File & Print, email, database, mainframe Most of them require Windows logon Logon requires DC availability Can the business still run even if WAN is down? Is the business in the branch focused on a LOB application that requires WAN access (mainframe)? Logon locally or over the WAN WAN logon requires acceptable speed and line availability WAN only an option if WAN is reliable Cached credentials only work for local workstation logon Terminal Service clients use local logon In many cases, network traffic is important Client logon traffic – directory replication traffic Design Considerations for Global Catalog Placement No factor in single domain deployment Turn on GC flag on all DCs No extra cost associated GC not needed for user logon anymore in multidomain deployments Universal Group Caching GC placement driven by application requirements in multi-domain deployments Exchange 2000 servers Outlook Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Domain Design Recommendation for Branch Office Deployment Use single domain Typically only single administration area Central administration (users and policies) Replication traffic higher, but more flexible model (roaming users, no GC dependencies) Database size no big concern If high number of users work in central location Create different domains for headquarters and branches If number of users very high ( > 50,000) Create geographical partitions High number of domains discouraged Examples: One domain / branch, one domain / state Increases complexity of deployment Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites DNS Design Recommendations DNS server placement Put DNS server on all domain controllers DNS client (resolver) configuration Primary DNS server: Local machine Secondary DNS server: Same site DNS server or hub DNS server Windows 2000: Different configuration for forest root DCs DNS zone configurations Use AD integrated zones (application partitions) Use DNS forwarding No NS records for Branch Office DCs Configure zones DNS Design Managing SRV (locator) records and autositecoverage SRV records are published by netlogon in DNS On site level and domain/forest level Clients search for services in the client site first, and fall back to domain/forest level Branch Office deployments require specific configuration Large number of domain controllers creates scalability problem for domain level registration If more than 850 branch office DCs want to register SRV records on domain level, registration will fail Registration on domain/forest level is in most cases meaningless DC cannot be contacted over WAN / DOD link anyways If local look-up in branch fails, client should always fallback to hub only Disable autositecoverage Use group policy for configuration Using GPOs for DNS Settings Create new Global Group for Hub DCs Add all non-Branch Office DCs as group members Create new GPO (BranchOfficeGPO) Configure DC locators records not registered by branch DCs Configure refresh interval In BranchOfficeGPO properties, deny “Apply Group Policy” to Hub DCs Negative list is easier to manage than positive list No damage if DC is not added to group Smaller number of hub DCs than Branch Office DCs Edit Default Domain Controllers Policy Disable automated site coverage Important that this is configured for ALL DCs, not only Branch Office DCs Netlogon Group Policy Netlogon Group Policy Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Replication Planning Improvements in Windows 2003 Windows 2000 Topology creation had scalability limits Required to manage connection objects manually Windows 2003 has many improvements to fully automate topology management New KCC / ISTG algorithm Bridgehead server loadbalancing KCC redundant connection object mode Specifically developed for Branch Office deployments Replication Planning KCC/ISTG ISTG = Inter-Site Topology Generator Computes least cost spanning tree Inter-Site replication topology Does not require ISM Service Windows 2000: ISTG uses ISM service Runs every 15 minutes by default Replication Planning KCC/ISTG Vastly improved inter-site topology generation (KCC/ISTG) scalability Complexity: approximately O(d*s) d = number of domains s = number of sites Win2000: approximately O(d*s²) Scales to more than 5,000 sites Still single threaded – uses only one CPU on SMP DCs Performance: 4,000 sites: 10 secs (700 Mhz test system) Ongoing tests in scalability lab Can generate different topology than Windows 2000 KCC/ISTG Requires Windows 2003 forest functional level Replication Planning Inter-Site Messaging Service (ISM) Creates cost matrix for Inter-Site replication Sends and receives SMTP messages if SMTP replication is used Runs recalculates matrix When ISM service starts up When changes happen in site configuration (new sites, site-links, site-link-bridges) Every 24 hours Information is used by Netlogon for auto-site coverage Load-Balancing tool Universal Group Caching DFS Has performance impact Replication Planning ISM – Performance Impact Number Sites Runtime / secs Working Set / MBytes Virtual Bytes 500 1,000 1,500 2,000 3,000 4,000 4 7 16 22 56 174* 18 60 130 227 456 540 57 100 170 275 894 893 Reference Hardware: 1.7 Ghz, 512 MB RAM Replication Planning ISM - Recommendations Up to 2,000 sites Use ISM service More than 2,000 sites Evaluate hardware platform (RAM, CPU) Use ISM if DCs do not start swapping Evaluate services not available without ISM service Domain based DFS needed? Replication Planning Bridgehead Server Selection Windows 2000 On a per site basis, for each domain, one DC per NC used as Bridgehead Windows 2003 On a per site basis, for each domain, all DCs per NC used as Bridgehead KCC picks DC randomly amongst bridgehead candidates when connection object is created For both incoming and outgoing connection objects Replication Planning Bridgehead Server Load-Balancing KCC/ISTG randomly chooses Bridgehead Server Both incoming and outgoing replication Once connection object is established, it is not rebalanced when changes happen Adding new servers does not affect existing connection objects Has to be used with care in Branch Office Deployments Necessary to control what servers are used as Bridgehead Servers Recommendation: Use preferred Bridgehead Server List and load balancing tool Replication Planning Preferred Bridgehead Server List Some servers should not be used as Bridgeheads PDC operations master, Exchange facing GCs, Authentication DCs Weak hardware Solution: Preferred Bridgehead Server List Allows administrator to restrict what DCs can be used as Bridgehead Servers If Preferred Bridgehead Server List is defined for a site, KCC/ISTG will only use members of the list as Bridgeheads Warning: If Preferred Bridgehead Server List is defined, make sure that there are at least 2 DCs per NC in the list If there is no DC for a specific NC in the list, replication will not occur out of site for this NC Don’t forget application partitions If branches have GCs, all bridgeheads should be GCs Replication Planning Active Directory Load Balancing Tool (ADLB) ADLB complements the KCC/ISTG Real load balancing of connection objects Stagers schedules using a 15 minute interval Hub-outbound replication only Hub-inbound replication is serialized Does not interfere with the KCC KCC is still needed / prerequisite Tool does not create manual connection objects, but modifies “from-server” attribute on KCC created connection objects Can create a preview Allows using the tool as an advisor Single exe / command line tool Runs on a single server / workstation Uses ISTG in hub site to re-balance connection objects Not needed for fault tolerance, only as optimization Can be run on any schedule Replication Planning KCC Redundant Connection Objects Mode Goal Create stable, simple and predictable replication topology Like mkdsx scripts for Windows 2000 Enabled on a per site level Implementation Creates two redundant connection objects Each branch site replicates from two different Bridge Head Servers Two different Bridge Head Servers replicate from each site Replication schedule is staggered between connection objects Fail-over is disabled If replication from one Bridge Head fails, the branch can still replicate from the other Bridge Head Schedule hashing is enabled Inbound connections start replication at random time inside the replication window Only DCs in same site are used for redundant connection objects Demoting DC causes KCC to create new connection object Replication Planning KCC Redundant Connection Objects Mode Schedule for redundant connection objects Use schedule defined on site-link Like, window open 8pm to 2am, replicate once every 180 minutes (= 2 replications) Divide by “2” and stagger Connection object 1 replicates once between 8pm and 11pm Connection object 2 replicates once between 11pm and 2am Second replication usually causes little network traffic Monitoring becomes even more critical Important to act quickly if hub DC becomes unavailable Replication Planning KCC Redundant Connection Objects Mode BH1 BH2 HUB Site Site Link 1 Duration 8h Replicate every 240 Min. Site Link 2 Duration 8h Replicate every 240 Min. BranchDC01 BranchDC02 Branch01 Branch02 Replication is open from 0:00 and 8:00 a.m. Replication is open from 0:00 and 8:00 a.m. 240 Min 240 Min 0:00 - 0:15 and 4:00 - 4:15 and 2:00 -2:15 6:00 - 6:15 240 Min 240 Min 0:16 - 0:30 and 4:16 - 4:30 and 2:16 - 2:30 6:16 and 6:30 Replication Planning Recommendations: Sites, Site-Links and Topology Create single site for hub site Leverage KCC load-balancing between Bridgehead servers Create site-links between Branch Office sites and hub site No redundant site-links or connection objects are needed Disable transitivity of site-links Not only for performance, but also to avoid branch-branch failover connection objects Disable auto-site coverage Use both ISM and KCC/ISTG services Use KCC redundant connection objects mode Use ADLB to load-balance connection objects Use Universal Group Caching to remove requirement for GC in branch Unless branch application requires GC Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Capacity Planning Replication Planning Branch Office DCs Usually low load only Use minimum hardware Datacenter DCs Depends on usage See Windows 2003 Deployment Kit for DC capacity planning Bridgehead servers Require planning Capacity Planning Formulas to compute number of Bridgeheads Hub outbound replication is multi-threaded Hub inbound replication is single-threaded Hub outbound: OC = (H * O) / (K * T) OC = outbound connections H = sum of hours available for outbound replication O = concurrent connection objects K = Number of replications required / day T = time necessary for outbound replication (usually one hour) Hub inbound: IC = R / N IC = inbound connections R = Length of replication in minutes Capacity Planning Example Requirements: Replication twice a day (= K) WAN 8 hours available (= H), 480 minutes (= R) High performance hardware (= 100 concurrent connections) (= O) Outbound replication will always finish within 1 hour (= T) DOD lines – conservative replication time hub inbound = 2 minutes (= N) Applying the formula: Outbound: OC = (H * O) / (K * T) = (8 * 100) / (2 * 1) = 400 If number is too high / low, change parameters: I.e., WAN available for 12 hours: 600 branches I.e., replicating only once a day: 800 branches Inbound IC = R / N = 480 / 2 = 240 Branches One BS can support 240 branches Capacity Planning Bridgehead Server Overload Cause Unbalanced site-links Unbalanced connection objects Replication schedule too aggressive Panic trouble-shooting Symptoms Bridgehead cannot accomplish replication requests as fast as they come in Replication queues are growing Some DCs NEVER replicate from the bridgehead Once a server has successfully replicated from the bridgehead, its requests are higher prioritized than a request from a server that has never successfully replicated Monitoring Repadmin /showreps shows NEVER on last successful replication Repadmin /queue <DCName> Capacity Planning Bridgehead Server Overload - Solution Turn off ISTG prevents new connections from being generated Delete all inbound connection objects Correct site-link balance and schedule Enable ISTG again Monitor AD and FRS replication for recovery Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Monitoring Design Monitoring is must for any Active Directory Deployment DCs not replicating will be quarantined DCs might have stale data Not finding issues early can lead to more problems later I.e., DC does not replicate because of name resolution problems, then password expires Use MOM for datacenter / hub site Monitor replication, name resolution, performance Windows Server 2003 Deployment Kit ships with Toolsmanager System to push and run scripts to Branch DCs Results copied to central server Web page presents Red/Yellow/Green state per server Evaluate available monitoring tools MOM and third parties Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Deployment - Overview ROOTDC1- GC 10.0.0.1 DNS corp.contoso.com HUBDC1 - FSMO DNS 10.0.0.10 branches.corp.contoso.com ROOT2DC2 - FSMO 10.0.0.2 DNS corp.contoso.com HQDC1 - FSMO 10.0.0.3 DNS hq.corp.contoso.com HQDC2 - GC 10.0.0.5 DNS hq.corp.contoso.com MOMSVR 10.0.0.26 MOM Server corp.contoso.com TOOLMGRSVR 10.0.0.4 Monitoring Server corp.contoso.com Data-Center-Site HUBDC2 DNS 10.0.0.11 branches.corp.contoso.com BHDC1 - GC BHDC2 - GC DNS 10.0.0.14 DNS 10.0.0.12 branches.corp.contoso.com 10.0.0.13 branches.corp.contoso.com branches.corp.contoso.com STAGINGDC1 DNS GC 10.0.0.25 branches.corp.contoso.com TSDCSERVER 10.0.0.20 ADS Server corp.contoso.com BHDC4 - GC DNS 10.0.0.15 BHDC3 - GC branches.corp.contoso.com DNS BOSite1 BOSite2 BOSite3 BOSite4 BOSiten BODC1 DC BODC2 DC BODC3 DC BODC4 DC BODC n DC Staging-Site Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Deploying Non-Branch Domains Not different from normal deployment Documented in Windows 2003 Deployment Kit Build forest root domain Create all sites (incl. branches) Build other non-branches domains as needed Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Deploying Branches Domain in Hub Site Install operations master Install bridgehead servers Install and configure ADLB Modify domain GPO for DNS settings Auto-site coverage Configure DNS zone for NS records Create branches DNS GPO SRV record registration Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Deploying Staging Site Staging Site has special characteristics All replication topology must be created manually KCC turned off Inter- and Intra-Site Scripts will be provided Should not register DNS NS records Create manual connection objects between staging site and production Staging DC needs to be able to replicate 7/24 Install Automated Deployment Services (ADS) Create image for branch DCs pre-promotion Active Directory Deployment For Branch Offices Active Directory Design Forest design Decide on centralized or decentralized deployment Domain design DNS design Site topology and replication design Capacity planning Monitoring design Active Directory deployment Deploying and monitoring non-branch domains Deploying branches domain in hub site Deploying and monitoring a staging site Deploying and monitoring the branch sites Deploying Branch Sites Build branch DCs in staging site from image Run quality assurance scripts (provided) Move branch DC into branch site Ship DC General Considerations for Branch Office Deployments Ensure that hub is a robust data center Monitor the deployment Use MOM for hub sites Do not deploy all branch office domain controllers simultaneously Monitor load on Bridgehead servers as more and more branches come on-line Verify DNS registrations and replication Balance replication load between Bridgehead Servers Keep track of hardware and software inventory and versions Include operations in planning process Monitoring plans and procedures Disaster recovery and troubleshooting strategy Personnel assignment and training Personnel assignment and training Summary Windows 2003 has many improvements for Branch Office deployments New KCC algorithm: no more scalability limit KCC redundant connection object mode: Provides stability Less replication traffic through LVR replication and DNS in app partitions Deployments are much easier to manage No manual connection object management GPO for DNS locator settings No more island problem Bridgehead servers more scalable Branch Office guide will have step by step procedures for deployment and tools Total cost of deployment will be much lower Branch Office Deployment on Windows Server 2003 AD Scale Lab Goal for Deployment Testing Deploy 3500 domain controllers. Provide feedback for developing the Branch Office Guide for Windows Server 2003. Automate deployment to achieve least amount of manual intervention. Use ADS for automated imaging across 3500 machines. Monitor the Datacenter site using MOM. Monitor all branch offices with set of scripts to verify basic functionality and generate report at common location. Monitor FRS. Deploy in 6 weeks. Branch Office Deployment Servers HUB Site Staging Site Root DC’s Staging Site TM Server ADS Server TSDC Server Datacenter TM Server MOM Server Staging Site DC Bridgehead DC’s Staged DC Staged DC Branch Office Site 2 Branch Office Site 1 Keys to a successful deployment Plan well. Double check to avoid simple human errors. Progress with controlled set of changes. Monitoring is essential. Community Resources Community Resources http://www.microsoft.com/communities/default.mspx Most Valuable Professional (MVP) http://www.mvp.support.microsoft.com/ Newsgroups Converse online with Microsoft Newsgroups, including Worldwide http://www.microsoft.com/communities/newsgroups/default.mspx User Groups Meet and learn with your peers http://www.microsoft.com/communities/usergroups/default.mspx evaluations © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.