Ing. Ondřej Ševeček | GOPAS a.s. |
MCM: Directory Services | MVP: Enterprise Security |
ondrej@sevecek.com | www.sevecek.com |
CLIENT INTERACTIONS
Active Directory Troubleshooting
CLIENT INTERACTIONS
Client Applications
 Kerberos and NTLM authentication
 Secure Channel
 password changes, NTLM pass-through, Kerberos
PAC validation
 Group Policy client
 DFS client
 Certificate Autoenrollment client
Client Applications
 NPS (IAS), RRAS, TMG (ISA), RD Gateway (TS
Gateway)
 group membership, Dial-In tab
 RD Host (Terminal Server)
 Remote Control tab etc., Licensing servers
 DHCP Server
 authorization
 IIS
 account and group membership for SSL certificate
authentication
 WDS
 computer MAC addresses or GUIDs
Site Design Scenarios
Branche
Branche
Branche
Branche
Branche
Central
Branche
Branche
Branche
Branche
Site Design Scenarios
Office
Office
Office
Site Design Scenarios
Branche
Branche
Branche
Central
Branche
Branche
Branche
Network Interactions Racap
(DC Location)
SRV: Any DC List
Client
2000+
SRV: My Side DC
DNS
DNS
LDAP
UDP
My Site DC
2000+
Get My Site
Any DC
2000+
Network Interactions Recap
(2008/Vista+ DC Location)
SRV: Any DC List
Client
Vista+
SRV: My Site DC
DNS
SRV: Close Site
DNS
LDAP
UDP
Close Site
DC
2000+
My Site DC
2000+
Get My Site
Next Closest Site
Any DC
2008+
Network Interactions
(Network Logon)
Client
2000+
App Traffic
Server
2000+
In-band
TGS: Server
NTLM
Kerberos
TGT: User
SMB
D/COM
TGS: Server
Occasional PAC
Validation
NTLM
Pass-through
D/COM Dynamic
TCP
DC
2000+
DC
2000+
Connection Properties
 Bandwidth (Mbps)
 forget about this
 Latency (ms)
 round-trip-time (RTT)
 SMB, D/COM, SQL
 Packet Loss (per sec., per Mb)
 packet loss rate (PLR)
 VPN such as PPTP, SSTP, IP-HTTPS
Timeouts
 DNS



primary DNS = 1 sec.
secondary DNSs = 2 sec.
... 1 2 2 4 8 ...
 ARP

... 600 ms 1000 ms
 LDAP UDP Site Location

600 ms
 TCP



SYN = 21 sec. (3x retransmission)
PSH/ACK = 93 sec. (5x retransmission)
... 3 6 12 24 48 ...
 Kerberos (TCP, 3 attempts, KdcSendRetries)

63 sec.
Basic DC location
 Know the DNS name of the domain
 Query general DNS DC SRV records
 _ldap._tcp.dc._msdcs.idtt.local
 Ping DC
 Windows 2003-
 LDAP UDP (ping) DC
 to get the client’s site/close site
Site DC Location
 Site unaware lookup
 NSLOOKUP
 SET Q=SRV
 _ldap._tcp.dc._msdcs.idtt.local
 Site specific lookup
 NSLOOKUP
 SET Q=SRV
 _ldap._tcp.Paris._sites.dc._msdcs.idtt.local
Lab: Finding DCs Manually
 Use NSLOOKUP to query for the generic DC
list
 NSLOOKUP
 SET q=SRV
 _ldap._tcp.dc._msdcs.idtt.local
Site Example – Single Site
DC1
DC2
DC3
DC5
DC4
Client
London 10.10.x.x
Site Example – Multihomed DC
(DNS Bitmask Ordering OK)
DC1
DC2
DC3
DC5
DC4
Client
Paris
10.20.x.x
London 10.10.x.x
Site Example – Multihomed DC
(DNS Bitmask Ordering Error)
DC1
DC2
DC3
DC5
Client
DC4
Paris
10.20.x.x
London 10.10.x.x
Roma
10.30.x.x
DNS Record Priority and Weight
Site Awareness
DC4
DC6
Paris
10.20.x.x
Roma
10.30.x.x
DC1
DC2
DC3
London
10.10.x.x
Anonymous
LDAP
UDP
where I am?
DC5
Client
Berlin
10.50.x.x
General Operation
 Use DNS to find generic DC list
 Ping selected DC
 Windows 2003-
 Anonymous LDAP (UDP) to determine site
 DC defines site from the request source IP address
(NAT?)
 Use DNS to find close DC in site
 Ping or LDAP UDP to determine availability
DC Locator
 NetLogon Service
 nltest /sc_query:idtt
 no network access
 nltest /sc_verify:idtt
 tries to authenticate with the DC
 nltest /sc_reset:idtt
 always performs new DNS lookup
 nltest /dsgetsite
 anonymous query against selected DC
Lab: Check NLTEST Usage
 Try the NLTEST to query, verify and reset
secure channel from Seven2 to its London
DCs
Limit UDP Site Location to a
Central Site?
DC4
DC6
Paris
10.20.x.x
Roma
10.30.x.x
DC1
DC2
DC3
London
10.10.x.x
Anonymous
LDAP
UDP
where I am?
DC5
Client
Berlin
10.50.x.x
Limiting Generic DC List
 Limit creation of generic DC DNS records
 GPO: Computer Configuration –
Administrative Templates – System –
Netlogon – DC Locator DNS Records
 DC Locator DNS Records not Registered
 Dc Kdc
Limiting Generic DC List
(Wise?)
Branche
Branche
Branche
Branche
Branche
Central
Branche
Branche
Branche
Branche
Limiting Generic DC List
(Wise?)
Office
Office
Office
DFS Client (MUP)
 Multiple UNC provider (MUP) driver
 Determines its own DFS server referrals
 obtains the list of DFS root servers from AD using
the default DC from Netlogon
 SYSVOL may be accessed from a different DC
 DFSUTIL /PKTINFO
 Windows Server 2003/Windows XP
 DFSUTIL CACHE REFERRAL
 Windows Server 2008/Windows Vista
DFS Context Menu
Site Example – Empty Site
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
DC4
DC6
DC5
Berlin
10.50.x.x
Roma
10.30.x.x
Client
DC7
Cyprus
10.40.x.x
Site Example – Empty Site
DC4
DC3
DC1
DC2
DC1
DC3
DC2
London
10.10.x.x
DC5
Paris
10.20.x.x
Client
DC4
DC6
DC5
Berlin
10.50.x.x
Roma
10.30.x.x
DC7
Cyprus
10.40.x.x
Site Example – Empty Site
DC3
DC1
DC2
London
10.10.x.x
DC1
DC3
cost 50
DC2
Paris
10.20.x.x
Client
cost 100
DC4
DC6
DC5
Berlin
10.50.x.x
Roma
10.30.x.x
DC7
Cyprus
10.40.x.x
Automatic Site Coverage
 Each DC registers itself for its neighboring
empty sites
 HKLM\System\CurrentControlSet\Services\N
etlogon
 AutoSiteCoverage = DWORD = 1/0
 GPO: Sites Covered by the DC Locator DNS
SRV Records
Active Directory Troubleshooting
MISPLACED OR CONFUSED
CLIENTS
Site Example – Out of Site
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
Client
DC6
10.100.0.7
Roma
10.30.x.x
DC7
Cyprus
10.40.x.x
Berlin
10.50.x.x
Super-netting or Sub-netting
Out-of-site Clients
Out-of-site Clients
Limiting Generic DC List
DC1
DC2
DC3
Paris
10.20.x.x
London
10.10.x.x
Client
10.100.0.7
Roma
10.30.x.x
Cyprus
10.40.x.x
Berlin
10.50.x.x
DC Stickiness
 When one close selected, client sticks to it
 even when moved into a different site
 must reset secure channel
 Force rediscovery interval GPO
 Vista+
 hotfix for Windows XP
 also registry value ForceRediscoveryInterval
Site Example – Until Restart/24
hours
DC1
DC3
DC2
Client
Client
Client
Client
Client
Client
Client
Client
Client
London
10.10.x.x
Site Example – Moving Client
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
DC4
DC6
DC5
Berlin
10.50.x.x
Roma
10.30.x.x
Client
DC7
previously in
Paris
Cyprus
10.40.x.x
Lab: Moving the Client
 On Seven2 verify the current DC in use
 NLTEST /sc_query:idtt
 Move the client into Paris and update group
policy
 GPUPDATE
 Verify the current DC in use again
 the client should use the same DC still although in
remote site (stick)
 Reset the secure channel several times and
determine the result
 NLTEST /sc_reset:idtt
Active Directory Troubleshooting
CLIENT FAILOVER
Site Example – Failed DC
DC4
DC6
Paris
10.20.x.x
Roma
10.30.x.x
DC1
DC2
DC3
London
10.10.x.x
DC7
DC5
Client
Cyprus
10.40.x.x
Berlin
10.50.x.x
Lab: Client Failover
 Move the client into Cyprus
 Reset the secure channel and verify it has
been connected to DC5
 Unplug DC5 from network
 Update group policy
 GPUPDATE
 Verify the resulting DC in use
 NLTEST /sc_query:idtt
Non-close Site DC
 Close site
 client’s site
 next closest site if enabled
 If there is not DC available in the close site,
rediscovery every 15 minutes
 HKLM\System\CurrentControlSet\Services\Netlog
on\Parameters
 CloseSiteTimeout = REG_DWORD = x seconds
Site Example – Next Close Site
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
DC6
Berlin
10.50.x.x
Roma
10.30.x.x
DC7
Client
Cyprus
10.40.x.x
Site Example – Close Site
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
cost 50
DC6
Client
Berlin
10.50.x.x
Roma
10.30.x.x
cost 100
DC7
Cyprus
10.40.x.x
Site Example – Close Site
DC1
DC2
DC4
DC3
DC5
Paris
10.20.x.x
London
10.10.x.x
cost 100
DC6
Client
Berlin
10.50.x.x
Roma
10.30.x.x
cost 50
DC7
Cyprus
10.40.x.x
Try Next Closest Site
 First get any DC name from DNS
 Second query the DC for clients site name
 returns the clients site
 plus the closest site (determined by the DC)
 Then query DNS for DCs in its current site and
then tries to use the DCs
 If none responds, the client queries DNS for
its next closest site and tries to use the found
DCs
Try Next Closest Site
 Does not consider RODC sites by default
 Can be change in registry
 NextClosestSiteFilter
 Windows 2003- cannot return the next closest
site information
 problem if the hit “any DC” is Windows 2003 it is then going to be used regardless of its site
Lab: Next Closest Site
 Enable Try next closest site in a GPO
 Have DC5 unplugged from network
 Update group policy
 Check the resulting DC in use
 NLTEST /sc_query:idtt
Client Rules Recap
 Windows 2003 In current site
 In any site
 Windows Vista+ with Next closest site
 In current site
 In the closest site
 In any site
 If the client is out of any site, find any dc
 consider creating subnets for VPNs etc.
General Best Practice
 Use only AD DNS servers on clients
 Do not use multi-homed DCs
 Define all IP ranges in AD
 may use super-netting if necessary
 Limit the generic DC list
 site UDP location, out-of-site clients, DC failure
 may use static GPO Site assignment
 Force rediscovery
 Try next closest site
Active Directory Troubleshooting
RODC
Read/only DC
 Physically insecure locations
 Only specified password hashes
 Read/only database
 other DCs are not willing to replicate back from
the RODC
 Local Administrator
 Managed By tab in the DC properties
RODC scenario
DC1
DC2
DC3
2003
2003
2008
GC
London
10.10.x.x
SRV
DC5
SRV
CL1
2008
Cyprus
10.40.x.x
Requirements
 Forest functional level 2003
 Domain functional level 2003
 Global catalogue 2003+
 understands confidential attributes
 At least one writable 2008+ DC
RODC and Windows 2003
 Windows 2003 does
not consider RODC
 Do not construct
replication
connections
RODC and Windows 2003
 Disable Auto Site Coverage
 HKLM\SYSTEM\CurrentControlSet\Services\Netlo
gon\Parameters
 AutoSiteCoverage = REG_DWORD = 0
 or install RODC compatibility pack
 Windows 2003, XP (11 issues)
 KB 944043
 Windows 2003, XP
DNS locator records
Password caching
 Passwords are only cached
 once the user logs on using writable DC first time
 can be prepopulated
 If the login fails on RODC, the request is
forwarded to another writable DC
 if offline, password expiration is ignored
Password caching/forwarding


DC1
DC2
DC3
2003
2003
2008
GC
not cached yet
not cached yet after
recent password change
 wrong password
 expired password
 account locked
London
10.10.x.x
SRV
DC5
SRV
CL1
2008
Cyprus
10.40.x.x
Write referrals
DC1
DC2
DC3
2003
2003
2008
GC
 try update on RODC
 referral returned
 try update on the
referred writable DC
directly
London
10.10.x.x
SRV
DC5
SRV
CL1
2008
Cyprus
10.40.x.x
Write Referral Problems
 BitLocker
 SP1 for Windows 2008/Vista
 Managed Service Accounts
 SP1 for Windows 2008 R2/Windows 7
Account lockout
 Accounts locked locally
 not replicated
 But the failure attempt is also reattempted
on a writable DC
 so this then replicates
Expired passwords
 pwdLastSet older than allowed by policy
 Logon attempt fails completely
 Password must be changed out-of-band and
logon then attempted again
Expired password
pwdLastSet
before 3 months
logon
error: expired
pwdLastSet
password
change
actual
logon
DC
ok
CL1
Discarding RODC
RODC DMZ Scenario
 Only RODC has internal domain access
 Cannot join domain normally
 use a join script (+ RODC compatibility pack)
 Cannot change machine passwords
 Cannot determine their site from the "any DC
list"
 HKLM\SYSTEM\CCS\Services\Netlogon\Parameters
SiteName = REG_SZ
 Cannot update AD account
 operating system
 service principal names
Active Directory Troubleshooting
DNS INTEGRATION
DNS Integration
 Clients find DCs by domain/site name
 DCs find replication partners according to
their GUID
 Netlogon de/registers locator records
 DNS stores its data in
 domain partition
 DomainDnsZones application partition
 ForestDnsZones application partition
Netlogon de/registration
 Netlogon de/registers its own records at
startup and deregisters them at shutdown
 requires DNS registration enabled on at least one
network adapter
 does not require DNS/DHCP Client service
 %windir%\System32\Config\netlogon.dns
 It does not touch others’ records
 Autosite coverage
 turned on by default
Netlogon de/registration
 Restarting Netlogon
 NLTEST /DSREGDNS
 force reregistration
 NLTEST /DSQUERYDNS
 query last status
 does not require DNS/DHCP Client service
and does not react on /REGISTERDNS
AD Integrated Zones
 Offer Secure Dynamic Update
 Timestamping
 trimmed to whole hour
 Aging and scavenging
 records deleted by default between 14-21 days of
their age
DNS Application Partitions
 Domain partition
 CN=MicrosoftDNS,CN=System,DC=...
 DomainDnsZones
 replicated to all DNS Server which are also DCs for
the domain
 ForestDnsZones
 replicated to all DNS Server which are also DCs for
the forest
Secure Dynamic Update
 Client side feature
 DHCP Client on Windows 2003 DNS Client on Windows Vista+
 IPCONFIG /REGISTERDNS
 DNS Server must be on DC to authenticate
clients with Kerberos
 All Authenticated Users
 can create new records
 When a record is created, only the
creator/owner can modify/update it
Secure Dynamic Update
 Updates done regularly by clients
 once a day by default by DNS/DHCP Client
 once a day by Netlogon
 once a day by Cluster Service
 Default TTL is 20 minutes
 Disable DHCP dynamic updates
 insecure!
Dynamic Update
Primary DNS
3
Update
Secondary DNS
1
Secondary DNS
Client DNS
SOA
Secondary DNS
2
Adjust A/PTR Record TTL
Dynamic Update and
Replication
DNS
DNS
0 sec.
AD
0-3 min.
15-21 sec.
schedule
AD
Dynamic Update and
Replication
Speed up the refresh
DHCP and dynamic update
 DHCP acts only on behalf of its clients
 client must provide its name (anonymously)
 Domain member computers since Windows
2000 do register themselves
 DHCP registers only
 workgroup computers, mobile phones
 printers, scanners, network devices, crap…
 Insecure, chaotic, unnecessary, corrupting
Disabling DHCP dynamic update
Dynamic DNS Update on RODC
 Each writable DC returns itself as a primary
DNS
 RODC returns either (random) writable DC as
the primary DNS
Dynamic DNS Update on RODC
Client
2
DNS
Upd
R/O
DNS
0 sec.
AD
RODC
1
SOA
Dynamic DNS Update on RODC
DsRemoteReplicationDelay
default 30 sec.
R/O
DNS
DNS
0 sec.
replicateSingleObject
AD
0 sec.
0-3 min.
RODC
Client
DsRemoteReplicationDelay
 Determines how long RODC's DNS server
waits until it requests replication of the single
object
 Default = 30 sec.
 Minimum = 5 sec.
 Do not forget the DsPollingInterval
Time stamping/Aging
 Record Created
 timestamp trimmed to whole hour
 No-refresh period starts
 by default 7 days
 timestamp does not change if the record does not
change
 Refresh period follows
 by default next 7 days
 timestamp gets updated at the first update
Scavenging
 Server wide configuration
 Should be done by only one DNS Server as
best practice
 By default ocurres only once per 7 days
DNS Aging and Scavenging
 per-zone setting
 implemented by all
DNS servers
 timestamp updates
only during the refresh
interval
 limits replication traffic
DNS Aging and Scavenging
 per-server setting
 should be done only by
one of the DNS servers
DNS Aging and Scavenging
DnsTombstoned = TRUE
 Scavenged records remain in AD yet for
another time DsTombStoneInterval before
they are deleted from AD
 default 7 days
 checked and potentially deleted everyday at 2:00
 Aimed to decrease replication traffic and limit
DNT/USN exhaustion
DNS Best Practice
DC1
DC2
AD
AD
DNS
DNS
DNS Waiting for AD
DNS Best-Practice Reasons
 Faster boot time without errors and timeouts
 Deregistration at shutdown is recorded in live
DNS Server
 would have problems replicate if sent into
shutting-down DC
Client DNS balancing
 Clients do not balance DNS servers
 queries/updates
 use the first one always if possible
 DHCP server does not use round robin
 Configuration must be done “manually”
 manual on servers
 more DHCP scopes for clients
Client DNS non-balancing
 Always alternate
DNS server
IP addresses
Client DNS non-balancing
Client1
DNS1
DNS1
DNS2
DNS2
Client2
Client3
DNS1
DNS1
DNS2
DNS2
DNS Client Settings
 HKLM\System\CurrentControlSet\Services\Tc
pip\Parameters
 Timetouts
 DNSQueryTimeouts
 Disjoint namespace on multihomed machines
 DisjointNameSpace
 PrioritizeRecordData
 GPO – DNS Suffix appending on Vista+
DNS Server UDP Pool
 After applying KB 953230, DNS Server
reserves 2500 UDP ports
 HKLM\System\CurrentControlSet\Services\D
NS\Parameters
 SocketPoolSize = DWORD = 2500
 DNSCMD /Config /SocketPoolSize 2500
DNS Cache Pollution


rogue attacker's DNS server: idtt.com, 1.2.3.4
server: idtt.com authoritative DNS server



question: www.idtt.com, type A
answer: no records
authority answer:
 idtt.com SOA
 idtt.com NS a.gtld-servers.net
 a.gtld-servers.net A 1.2.3.4

server: idtt.com authoritative DNS server



question: www.idtt.com, type A
answer: no records
authority answer:
 microsoft.com NS ns.idtt.com
 ns.idtt.com A 1.2.3.4

Enabled by default since 2000 SP3

SecureResponses
DNS Cache Locking
 Further limits cache poisoning as already
improved by the UDP pool
 Records present in the cache cannot be
updated before their TTL expires
 prevents cache poisoning in some scenarios
 frequently visited sites are already in the cache
 Windows 2008 R2
 enabled by default - 100%
 CacheLockingPercent = DWORD = 0-100
Performance Considerations
 MaxCacheTtl
 maximum Ttl limit on cached RRs
 by default 1 day maximum
 MaxNegativeCacheTtl
 by default 15 minutes
General Best Practice
 More than 2 DNS servers are usually
unnecessary for a site
 Enable DNS Aging and Scavenging
 may decrease DsPollingInterval
 may shorten the client update refresh interval
 Alter clients’ DNS settings to rotate the DNS
server addresses
 Disable DHCP dynamic update