University of Washington
1
Why knowing LDAP will help you
manage & troubleshoot your AD
Disclaimer: We won’t talk much about LDAP as a
protocol or API (I’m anticipating huge sighs of relief);
instead we’ll focus on the directory part of LDAP & how
this knowledge might be useful in the context of AD.
2
Who am I?
• Software Engineer at Univ. of Washington
– ~2000 managed workstations **
– Large forest (~40 domains/90 DCs)
• Helped design Stanford’s AD deployment **
• Help manage the windows-hied mailing list **
• Wrote LDAP Directories Explained for
Addison-Wesley—intro to LDAP **
3
Premise
Microsoft has done a better job integrating LDAP
into the user experience than any other directory
product. But nearly-seamless integration doesn’t
mean you should ignore the technology.
Knowledge empowers understanding. If you
understand the technology behind a product,
then you can understand the product and it’s
behavior better. **
4
Overview
• LDAP basics & a little about how it underlies all AD tools & APIs
• LDAP search filters can let you quickly grab info—a quick primer
• The forgotten partition: the config partition is central to many key
operations.
• Some things can't be done w/o LDAP; there aren't MS tools to do
what you want.
• When things go wrong, be prepared for sub-optimal PSS solutions.
• Distributed directories and powerful AD functionalities like persistent
searches.
• Understand what major product upgrades are doing behind the
scenes.
– Windows 2003 “forest prep” and “domain prep”
• VBS Scripting example
• AD LDAP strengths & a wish list
5
Directory basics
•
LDAP adopts the X.500 directory standards. (this adoption is mostly implicit
which has lead to some problems in the schema assumptions).
– This means that a LDAP directory is a tree structure, where any given child entry
only has one parent; no web-like structure.
– An axiom of this structure is that the name of an entry also gives you it’s location.
This is a powerful feature of the design that isn’t often recognized.
•
•
A directory is really just a special purpose “fast, read-oriented” DB. You
wouldn’t use a directory for a modification-intensive purpose—you’d use a
DB. The converse axiom seems to be less recognized; too many people (or
should I say products?) use a database for a low-mod/hi-read purpose
when a directory would work better.
As with ALL protocols, there are special characters that get interpreted in
special ways when using that protocol. So you have to be careful to encode
those characters (escape with a \). So if I wanted to search the location
cn=C&C,dc=endor,dc=sw—I’d better ask for cn=C\&C,dc=endor,dc=sw. The
converse axiom is to avoid special characters in attribute values as much as
possible. You should also be sure that your LDAP vendor knows about
buffer overflow & command injection schemes that make use of these
special characters. **
6
Basics (2)
•The 10 LDAP operations are:
–Bind
–Search
–Compare
–Add
–Delete
–Modify
–ModifyRDN (rename)
–Unbind
–Abandon
–Extended
These operations are standardized, i.e. every LDAP server & client must support them in
the same way. Further, every LDAP API supports them the same way, with a common
data structure
Controls modify these operations to create a new action (e.g. server-side-sort). Controls
aren’t mandatory.
Options modify all the operations in a session (e.g. SSL). Options aren’t mandatory.
There are tons of new IETF proposals for extensions to LDAP all the time. Many of the
most interesting proposals have gotten implemented in AD. ** We’ll see more on AD
specifics of controls later.
7
Basics (3)
8
LDAP underlies most tools
•
•
•
•
ADSI uses LDAP
AD MMC tools use LDAP
Exchange uses LDAP
AD reskit tools use LDAP
…
If you want to access data in AD, you are
using LDAP. **
9
Finding a directory entry—or a set of entries
• A LDAP search involves 3 mandatory parameters:
– A base DN, i.e. where to start
• Ex: dc=mydomain,dc=myschool,dc=edu
– The scope, i.e. how far to look
• Subtree=the base DN & all entries below
• One=all entries at same level as base DN
• Base=Just base DN
– A search filter, i.e. what to look for
• Search filters are a composition of statements, which are surrounded by parentheses
• A statement has 3 elements:
– Attribute type, e.g. objectclass or mail
– Comparison operator e.g. = or >=
– Attribute value – Wildcards are possible to match zero or more characters
• Combination of statements is possible with filter operators which precede the statement to
form a composite statement (with parentheses)
– &=AND
|=OR
!=NOT
• Ex: (|(objectclass=user)(objectclass=computer)) i.e. the user or computer entries **
• Ex: (&(objectclass=user)(cn=a*)) i.e. the user entries with names that begin with a **
•
Note that the common GUI tools that are built on top of LDAP either assume this info or
have you select it (while ignoring the optional parameters). They also make use of
something called Display Specifiers to get a GUI look, but that’s outside this talk. **
10
LDAP Search (part 2)
•
LDAP search optional parameters
– What attributes to return
• Blank = all (operational left out)
• 1.1 = none (1.1 is a special OID) **
• Attrib1,attrib2,etc for a limited list (this is only way to get operational attribs)
– derefAliases (AD doesn’t support aliases!!) **
•
•
•
•
neverDerefAliases=don’t follow aliases
derefInSearching=lookup all except baseobject
derefFindingBaseObj=lookup only baseobject
derefAlways=lookup all
– sizeLimit = number of entries returned
• 0 is default & means give me all results
• May be limited by configuration (which AD sets to 1000)
• If more are found, this is noted in response.
– timeLimit = Max time (in seconds) to search
• 0 is default & means give me all results
• May be limited by configuration (which AD sets to 120)
• If more are found, this is noted in response.
– typesOnly **
• True = lists only attribute types, i.e. sn,cn, etc.
• False (default) = lists both attribute type and value, i.e. sn=Arkills, cn=Brian Arkills, etc.
11
LDAP Clients
• The Search Assistant/Address Book application hides
the complexity of LDAP search filters nicely. This means
users don’t need to know LDAP. **
• BUT …. if you need the search filter to behave
differently, then it isn’t necessarily easy to change. You
will want to check out a few places:
– Q311829 describes how to turn off displayname searching.
– Mapisvc.inf in the [EMABLT] section has the default search filter
used by the Address Book. You can modify it. By default it uses:
(&(mail=*)(|(mail=%s*)(|(cn=%s*)(|(sn=%s*)(givenName=%s*)))))
i.e. Must have mail attrib set and have your search string (with
wildcard afterward) in the cn, sn, mail, or givenName attribs
• MMC ADUC has some configurable parameters—check
out the View menu>Filter Options **
• ADSIEdit (support tools) is nice for single operation use,
but slower than ldp.exe. I use it for 3 things: schema
consultation, for the few operational attribs ldp doesn't
permit, & the security GUI so I don't have to decode the
12
ntsecuritydescriptor. **
The forgotten partition
• The configuration partition is central to many key operations.
Viewing its data directly via ldp or adsiedit instead of one of the
other dumbed-down MS tools is very enlightening, and gives you
much more control. **
• Examples of critical data in the config partition:
– Cn=ExtendedRights holds special rights (which you can create for use
in your app).
– Cn=Partitions holds a list of all the known partitions in the forest. This is
useful for a number of forest-wide search scripts. Beware crossref
entries that get mangled; at UW we’ve seen it 3 times—more on this
later.
– Cn=Services holds a variety of configuration data for network services.
Most notable are the LDAP administrative parameters such as the
maximum page size, and a default forest-wide policy for these settings
(more later). And Exchange lives here--more on that soon
– Cn=Sites holds the AD replication topology. An entry for every domain
controller is here; this is useful for forest-wide DC search scripts.
– Cn=WellKnownSecurityPrincipals holds the built-in user & group
accounts like Authenticated Users. Dunno if you can create your own …
but w2k3 creates new ones, so I’m guessing you can.
13
AD LDAP configuration parameters
•
•
•
A default forest-wide policy is at cn=Default Query Policy,cn=QueryPolicies,cn=Directory Service,cn=Windows
NT,cn=Services,CN=Configuration,DC=Mycompany,DC=com. The
lDAPAdminLimits attribute stores the below config settings in a string
format. Additionally, the lDAPIPDenyList attribute stores any IPs (can use
subnet mask) that should be denied connections. **
Additional policies can be created and linked to specific domain controllers,
so individual domain controllers have settings different from the default.
Types of parameters you can set:
–
–
–
–
–
–
–
–
–
–
•
MaxPageSize: largest size (1000)
MaxActiveQueries: max queries from all clients (20)
InitRecvTimeout: timeout between two LDAP servers (120)
MaxConnections: max client connections (5000)
MaxConnIdleTime: client timeout (900)
MaxNotficationsPerConn: max number of psearch per client (5)
MaxQueryDuration: max time on one operation (120)
MaxResultSetSize: max size of data (262144)
MaxTempTableSize: max temp. table used for sorted search (10000)
MaxPoolThreads: number of directory threads (4)
Ntdsutil also offers a way to modify these. **
14
Exchange lives in the config partition
• Critical data it stores here includes:
–Address book definitions, i.e. which accounts are displayed
–Where RUS lives for each domain partition
–SMTP connections
–Recipient Policies
–Admin Group specific settings, e.g. store/MTA/SA parameters, supported
protocols, Exch SP level, etc.
• An example … how Exchange forms the list of accounts
that are displayed in the Global Address List isn't
documented anywhere publicly, but it is a functionality
you can modify via LDAP. The default value is:
purportedSearch=(&(mailnickname=*)(|(&(objectCategory=person)(objectClass=user)
(!(homeMDB=*))(!(msExchHomeServerName=*)))(&(objectCategory=person)(objectC
lass=user)(|(homeMDB=*)(msExchHomeServerName=*)))(&(objectCategory=person)
(objectClass=contact)(objectCategory=group)(objectCategory=publicFolder)))
Complex looking, huh?
15
Exchange (part 2)
With a little reformatting for better readability:
purportedSearch=(&(mailnickname=*)(|
(&(objectCategory=person)(objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))
(&(objectCategory=person)(objectClass=user) (|(homeMDB=*)(msExchHomeServerName=*)))
(&(objectCategory=person)(objectClass=contact)
(objectCategory=group)(objectCategory=publicFolder) ))
In a slightly simplified statement, this equates to
Gotta be mail-enabled and either
a. person+user (with useless additions)
b. person+contact
c. group
d. public folder
You can modify this definition to further restrict the GAL (for
privacy) or modify other address lists or recipient policies that
have a purportedSearch attribute (and circumvent the e2k GUI
field limit at the same time).
16
Left without a MS tool
• Like the last example, there are some things that can't be done without
direct LDAP manipulation; there aren't high-level tools to do what you
want.
• For example, what do you do when you need to manually delete a DC
entry in the config partition (so the replication topology is clean) when
the domain naming context (i.e. entry under cn=partitions) is gone?
– ntdsutil can't manually remove a dead DC with no NC; it gives an error. You
might be able to do a metadirectory cleanup, but that requires a reboot and
time. And those two tools don’t really have less obscure syntax than LDAP,
so it’s far easier to just delete the entry via LDAP, so why bother with
ntdsutil?
Well … I’ve been told that ntdsutil in some cases does some extra nonLDAP stuff to the DB cross-linked entries behind the curtain. But I’ve never
seen this make any difference in practice.
• What do you do when you need to manually delete a domain, but the
domain is still running?
– You delete the domain trust entry in the forest root domain, the domain
partition entry in the config partition, and any DCs in the config partition. If
you don’t delete the trust, the other entries will come back from the running
domain. 
• I’m sure there are other examples—I just haven’t run into them yet.
17
Troubleshooting & being prepared for the MS engineer
Earlier, I mentioned a crossref mangling issue. We had a problem in our forest where replication was failing to
several domains. Most (but not all) of the partners of those domains would complain on the replication trigger
about an invalid SRV record. The SRV record for the DCs in those domains was registered. Looking more
closely at the event descriptions, I noticed that the SRV record noted had a mangled name with a tell-tale
collision-style prefix (usually CNF). Where was this coming from? I used a LDAP search to find all the collision
records in AD, and found some suspicious values on the partition entry for those domains (which had mangled
values only on some of the DCs in our forest).
Turns out that the problem was caused by a directory collision on the object representing the domain partition.
The collision probably happened when the domain was demoted, then recreated prior to full replication
throughout the forest.
After some extensive troubleshooting (and trying to manually change those mangled values), we got PSS
involved. Initially, they thought a hotfix would have to be applied to EVERY domain controller in our forest, plus
we’d need to run a special version of ntdsutil in DS Restore Mode on each DC. With ~90 DCs geographically
dispersed across several campuses, we were not thrilled with pursuing that solution.
I went into the case with the conviction that we simply needed to modify the attribute value that had been
mangled, and the change would be replicated. This attribute was system-only, meaning that only AD itself can
modify it (not to be confused with the local system account). I knew that every other LDAP product had a way
to modify system-only attribute values. So I remained steadfast through 4 levels of engineers that MS must
have a way to modify system-only data. I ended up with the fellow who wrote repadmin—nice guy. And they
did … flip a registry bit, modify the value, replicate, and flip the registry bit back.
I’ve heard that MS doesn’t want folks to know how to modify system-only data. I can imagine that some basic
architectural assumptions might be subverted if that knowledge was widespread—although only by those
administrators who already have the ACLs and own the forest root domain. Maybe MS is nervous about AD
being modified to work better with products they are in direct competition with. Whatever the case, I still have
never seen this documented by MS.
18
Distributed Directories
• You likely have more than one directory—many
of which are databases, even though they don’t
have a high volume of modification. You’ve
probably got HR, students, ID cards, NetID,
alumni, etc.
• Unifying distributed directories is about how to
make all these disparate source systems come
together. Some folks call this the metadirectory
problem, but that is only one of several possible
solutions.
19
Distributed directories (2)
• As you try to fit two or more “directories” together,
the issues include:
– Protocol: what do they talk? LDAP is nice because it has
universal clients. Often LDIF is used--although I think
DSML will replace LDIF. **
– Schema design: I call the department “department”, but
you call it “dept”. How do we reconcile the two “labels”
(attributes)? Hard problem … more on this later.
– Data consistency: I call our department “C&C”, you call it
“Computing and Communications”, and he calls it “CAC”.
How do we reconcile the multiple values (attribute
values)? Also a hard problem … more on this later.
– Data Structure: The way data is structured can affect the
usability of that data. And different directories will
structure the data in the way that most benefits its own
purposes. So how do you reconcile different structures
where the data should have close relationships (like the
20
NetID to the respective people systems)?
Distributed directories (3)
• Solutions include:
– Metadirectories—hardest to implement because it must tackle all the
issues. Usually a “connector” or “harvester” sucks data into a middle
system, scrubs it to compliance with scripts, then publishes it (or
returns it to the original systems). There are both commercial &
homegrown solutions e.g. Microsoft Metadirectory Services or
Stanford’s Registry Harvester.
– Loosely connected directories—easiest to implement, but requires
LDAP or X.500 directories and it doesn’t solve big discrepancies to
any great satisfaction. You use referrals to paste namespaces
together. A client can connect to any of the systems and get
seamless referrals to the others.
– Client portal or middleware service—fix the discrepancies and bring
together the data in a website where clients have a limited subset of
queries. This has many of the same problems to tackle as a
metadirectory (plus UI design), but when complete it isn’t as useful.
For example, you can’t use it to populate a network OS directory like
AD, whereas a metadirectory would work.
21
Advanced AD functionality
•
Controls allow you to modify the standard LDAP operations with a new behavior.
Common examples of controls are the sorted search (sorted results) and the
paged search (usually to bypass size limits). AD supports the most controls of
any LDAP server product.
– Persistent Search functionality is very useful for metadirectory
synchronization. The basic idea is to allow a search that functions over a
time period, instead of requiring repetitive searches. AD supports
• dirsync, “give me all the changes since a point in time (based on a USN)”,
• psearch, “give me all the changes from now on until I cancel the operation”.
– Tree delete—allows deletion of a container and children. Not widely
supported.
– Lazy commit—allows batch changes to be queued to optimize performance.
– Virtual List View—allows search to specify sort order, the number, and the
position in the results. Useful for “scrolling” behavior when the result set is
large.
http://msdn.microsoft.com/library/en-us/netdir/ad/polling_for_changes_using_usnchanged.asp
•
Referrals allow you to extend your directory to other LDAP servers. There are 2
types of referrals: a default referral or a referral entry. A default referral is
returned on any search that asks for something outside the server’s naming
contexts. A referral entry is an actual directory entry that can be returned to the
client on any search. It redirects the client to another directory space, and by
default the client executes the same search on that namespace too. You end up
w/ the composite of the two results. The bottom line is that you can create a
distributed directory that spans multiple implementations.
22
Windows 2003 forestprep
%systemroot%\System32\Debug\Adprep\Logs\Latest_log folder
• Schema changes:
– 49 objectclasses added. (was 142 or 300 w/e2k)
– 208 attributes added. (was 863 or 1716 w/e2k) (now I can store my
favorite drink)
– 17 Extended rights added. (was 48)
– 9 syntaxes added. (was 18)
– Inetorgperson is finally added; special actions to take if you added it &
e2k already.
• Other changes! (not documented):
– ACL modification to support new features on:
•
•
•
•
•
•
cn=sites,cn=configuration
cn=configuration
cn=schema
Sam-domain objectclass
Computer objectclass
Other less notable entries
– Dunno what any of the changes are as they aren’t logged (I didn’t have
extended time to compare & contrast to another non-forestprep’ed
forest)
23
Windows 2003 domainprep
MS documentation incorrectly states that domainprep just verifies that
forestprep has run. What it does:
• Verifies that forestprep was run
• Creates a bunch of directory entries to support new functionality
(such as the WMIPolicy, Program data, & IP Security entries)
• Makes ACL & attribute changes to the root of your domain & 8 other
locations. These amount to providing new extended rights support.
• Adds the anonymous logon SID to the pre-windows 2000 group
• Makes ACL changes to GP directory entries & their container
• Adds 4 new SID prefixes to the foreignsecurityprincipals container
(I've always wondered about this functionality ... anyone know
more?):
–
–
–
–
S-1-5-11
S-1-5-20
S-1-5-4
S-1-5-9
S-1-1-0
<Authenticated User>
<Network Service>
<Interactive>
<Server Logon>
<Everyone> was already there.
24
Scripting (using LDAP via ADO)
There are lots of other scripting/programming resources you
can use with AD. ADSI is an obvious one. Microsoft also has a
Windows LDAP API. I’ve chosen to demo a VBS using ADO to
access an ADSI provider as it is a very accessible option for
those of us who haven’t scripted much.
We’ll look at an basic-to-intermediate set of scripts to illustrate
several of the facts I’ve noted. I’ll use two different ADO-ADSI
dialects to illustrate the diversity of functionality.
Sidfilterwrapper: will find all domains in forest, find a DC for
each domain, scan that domain for external trusts, then for each
external trust scan for sidfiltering. I’ve omitted the code for the
wrapper to save time.
25
Scripting (part 2)
– List domains in forest
Set rs = CreateObject("ADODB.Recordset")
Set con = CreateObject("ADODB.Connection")
Set com = CreateObject("ADODB.Command")
strContext = "GC://dc=endor,dc=sw"
con.Provider = "ADsDSOObject"
con.Open ("DS Query")
Set com.ActiveConnection = con
com.CommandText = "Select name FROM '" & strContext & "'
WHERE objectclass = 'trusteddomain'"
Set rs = com.Execute
rs.MoveFirst
While Not rs.EOF
wscript.echo rs.Fields("name").Value
rs.MoveNext
Wend
26
Scripting (part 3)
–Find DCs for domain
domain = wScript.Arguments.Item(0)
Set rs = CreateObject("ADODB.Recordset")
Set con = CreateObject("ADODB.Connection")
Set com = CreateObject("ADODB.Command")
strContext = "GC://cn=configuration,dc=endor,dc=sw"
strLDAPfilter = "(&(objectclass=server)(dnsHostName=*." & domain & "))"
strLDAPattr = "dnsHostName"
strLDAPscope = "subtree"
strLDAP = "<" & strContext & ">;" & strLDAPfilter & ";" & strLDAPattr & ";" &
strLDAPscope
con.Provider = "ADsDSOObject"
con.Open ("DS Query")
Set com.ActiveConnection = con
com.CommandText = strLDAP
Set rs = com.Execute
rs.MoveFirst
While Not rs.EOF
if split(rs.Fields(0).Value, "." & domain)(0) = split(rs.Fields(0).Value, ".")(0) then
wScript.Echo rs.Fields(0).Value
end if
rs.MoveNext
Wend
27
Scripting (part 4)
–Scan domain for external trusts: arg0= “LDAP://” & outputOfLastScript –can’t
use GC ‘cuz trust attribs aren’t there
strContext = wScript.Arguments.Item(0)
Set rs = CreateObject("ADODB.Recordset")
Set con = CreateObject("ADODB.Connection")
Set com = CreateObject("ADODB.Command")
con.Provider = "ADsDSOObject"
con.Open ("DS Query")
Set com.ActiveConnection = con
com.CommandText = "Select name,trustdirection,trusttype FROM '" & strContext & "'
WHERE objectclass = 'trusteddomain'"
Set rs = com.Execute
rs.MoveFirst
While Not rs.EOF
If rs.Fields("trusttype").Value = 1 then
If rs.Fields("trustdirection").Value > 1 then
wscript.echo rs.Fields("name").Value
End If
End If
rs.MoveNext
Wend
28
Scripting (part 5)
–Scan for sidfilter
forestdomain = wScript.Arguments.Item(0)
externaldomain = wScript.Arguments.Item(1)
Set wshShell = CreateObject("wScript.Shell")
wshShell.CurrentDirectory = "c:\Program Files\support tools"
sCommandpath = "netdom.exe"
sCommand = sCommandpath & " trust " & forestdomain & " /D:" & externaldomain & "
/filtersids"
set WshExec = wshShell.Exec(sCommand)
sStdOut = WshExec.StdOut.ReadAll
If instr(lcase(sStdOut),"filtering is not enabled") > 0 then
wScript.Echo "None"
Else
wScript.Echo "Enabled"
End If
29
AD as a LDAP Product
• Microsoft has *great* client integration but could
give better customization.
• Microsoft has superior advanced features
(controls, replication, etc.), with only Directory
Server (netscape/sun) in the same league
• With AD/AM they’ve shed the biggest knock
against them, i.e. the requirement to run a
network OS in order to use their LDAP server.
30
Brian’s AD Wish List
•
•
Reduce number of directory collisions or at least raise the visibility of collisions to a
higher level (like logging them with a warning event!). The algorithm AD uses is
extensive & fancy, but it doesn’t do the job well enough. And the tools for fixing
collisions aren’t sophisticated at all.
Fix non-Internet standard schema mistakes. You can’t claim great integration w/o
following standards.
– sn is single-valued; should be multi-valued
– Alias object class support. It’s implied as part of the LDAP standard. Every LDAP server
except AD supports it. Would allow institutions like HiEd to have dummy entries for people
in multiple locations, helping with the “multiple affiliations--which OU does the account go
into?” problem.
•
•
•
•
•
Dynamic access control like Directory Server (either Netscape or Sun—used to be
iPlanet) i.e. ability to set an ACE that changes the access depending on the value of
an attribute on the target entry or the user’s entry.
Extended match filter support. This would allow searching on the DN of an entry.
Implement the LDAP standards wrt what characters are considered special & in need
of being escaped. Document what customers should do with respect to this.
Chaining support. This would mean that AD could be configured to chase referrals
instead of the client. This would pave the way for Global (or not so global) catalog
functionality with cross-forest trusts.
Improve usability of ldp.exe:
– Some operational attributes won’t display. Why?
– Smart/auto fill-in would be helpful for typing long x500 strings.
– Make results window focus by default on results of last operation.
•
AD on a non-Windows platform 
31
University of Washington
32