JMP402: Master Class:
Reaping Rewards and
Avoiding Risks - The
Secrets of Consolidating
Domains
Andy Pedisich, President - Technotics, Inc.
Rob Axelrod, Vice President - Technotics, Inc.
© 2014 IBM Corporation
Your presenters
 They are two hard working IBM® Notes® Administrators/Developers who have worked with
IBM® Notes® and IBM Domino® since version 2.1
– From Technotics, Inc. in Philadelphia, Pennsylvania - USA
 Andy Pedisich
– 29 years in IT
– 19 years with Lotus Notes
 Rob Axelrod
– 25 years in IT
– 19 years with Lotus Notes
2
About Technotics, Inc.
Technotics was founded in 1998 as a consultancy to
focus on collaboration in the enterprise.
Since that time we have provided strategic advice,
project management and technical support to
organizations world wide, focusing on high levels of
customer engagement and long term relationships.
Rob Axelrod
Our services include environmental audits, premium
support, executive briefings on cloud based
collaboration and migrations between messaging and
collaboration systems.
Contact Andy at andyp@technotics.com or Rob at
rob@technotics.com.
3
Andy Pedisich
What’s under our finger tips during this session
 Our host laptop is a Dell Studio™ 1555
– Intel® Core™ 2Duo CPU T9600 2.80 GHz
– 8 GB memory
 The operating system is Microsoft © Windows 7 Ultimate – 64 bit
– Copyright © 2009 Microsoft Corporation. All rights reserved
– Microsoft and Windows are trademarks of Microsoft Corporation in the United States,
other countries, or both.
 IBM® Domino® Server Release 9.0.1 64 bit
– Running on host system, Using IP address of the laptop's network interface card
– Running IBM Traveler
 IBM Notes® Release 9.0.1
4
Examples and demos from this session…
 We’ve packed up the files and code used in this session
– They are available on my blog
– http://www.andypedisich.com
 Look for the link at the top right of the screen that will take you to the files you can download
5
Why we are doing this presentation
 There are a myriad of products, tools, and professionals that can assist you during a Domino
domain consolidation
 We’ve done more than our share of them over the years
– We’ve done a lot of really good ones
• We’ve also found “error conditions” that can trip you up
 We’re here to share what we know and what we’ve learned
– To make it easier for you to do it yourself
• It will be easier to manage if you have others do it for you
– You’ll have a better idea of the tools and the expected outcomes
6
What a Master Class is all about
 Since you are attending this Master Class, we assume that you know that:
– “Master Classes” are designed for the practitioners already in the trenches and are
targeted to those who are developing sophisticated applications that support business
processes and departmental/enterprise administration, who want to kick their skill set up
to the next level.”
7
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
8
Why are you thinking about a domain consolidation?
 Most of our domains are not new
– Many were started in the mid 1990s
– We go back to version 2.1
 What we designed back then was great for back then
– Lots of Domino servers implemented close to the users
– Replication topology that worked great over slow data links
9
Why are you thinking about a domain consolidation?
 If you started your domain back then, you were
– Working with modems
– Thinking about saving a few dollars by having your servers replicate at night
• Because the telephone rates were cheaper
- Modem files don’t ship with the Domino server and Notes client anymore
10
The 1990s were a time of growth and change
 The National Science Foundation said this about the 1990s and the Internet
– “Entire new industries sprang up (and in some cases crashed back down) as humanity
embraces the biggest technological breakthrough since the Industrial Revolution.”
• “The Information Age had arrived and the world would never be the same.”
 During the 1990s, the National Science Foundation says the number of users on the Internet
grew to 248 million!
11
The 1990s were a time of growth and change
 Technologists like us relied less on the phone and more on the Internet in the 90s
– And we designed our domains to exploit it
• Now there are billions of Internet users world wide
• And our appliances are hooked into the Internet too
 Saw a story last week about a refrigerator that was hacked
– It was sending out 100,000 messages 3 times a day
– http://www.dailymail.co.uk/sciencetech/article-2541306/Cyber-criminals-hackREFRIGERATOR-send-malicious-emails-worlds-household-attack.html
12
Designed with slow bandwidth in mind
 Relatively slow Internet bandwidth gave us a habit of placing servers at remote locations
– We would end up with servers that were called home for a small numbers of users
 Just one of the many reasons that today we look at consolidation for our server
environments
– We got bigger
– With bandwidth that’s faster than ever
• No need to have servers at remote locations when they can be reached quickly from
just about anywhere
 We’re not saying that this is the new normal
– There are plenty of places on earth where bandwidth is still scarce
– And we can still design to them
• We just need to optimize
13
Here are some real reasons behind domain consolidation
These are directly from our customers’ mouths
 We want to save money and resources
 We want fewer servers and to exploit new faster networks
 We were just bought by another company
 We buy smaller companies every year and want dedicated processes to migrating to our
messaging systems
14
Here are some real reasons behind domain consolidation
These are directly from our customers’ mouths (cont’d)
 We’ve bought other companies that were running other brands of cloud based messaging
and want to centralize on Notes
 We’ve moved mail to the cloud and want to consolidate what we’re using for applicaitons
 When Notes was deployed we created regional server farms that are no longer a required
component
 We are still running IBM Domino R6.5.4 and wish to do a technology upgrade
– Might as well consolidate the infrastructure while we’re at it
15
More reasons to consolidate from our customers
 Started out with multiple domains, but realized should only have one centralized domain
– Earlier decisions about multiple domains were made without a real understanding of the
Domino infrastructure requirements
– Earlier decisions were made based on technology available, such as modems and
slower networks
 We need a technology refresh
– Just want to start fresh with a smaller number of new servers to replace aging ones
– Establish best practice configuration
16
A common thread of complaints that make us consolidate
 There is just too much of everything
– It is out of control
 There is too many of everything
• Too many mail routes
• Too many servers
• Too many connection documents
• Too many mail-in databases
• Too many applications
- We don’t even know if anyone is using these applications
17
What happened to the creators of this domain?
 Plus, the domain was designed by people who no longer work here
– It’s difficult to find the causes of problems
 It’s time to straighten it up
– Optimize the domain
– Review the security
– Reassess applications
– It’s a fresh start, a new day
18
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
19
Overall do’s and don’ts with your migration
 Do the migration as quickly as possible
– Being in a state where you are in the middle of the migration with your users are in the
state of flux can be very risky
 Run a pilot with a small number of users if possible to ensure any techniques are going to
work as expected
– Make sure you understand the risks of having your users straggled between new domain
and old domains
 Here are some of the issues you might experience during migration
20
A problem with old domain’s meeting invitations
Here’s a good example of what can happen
 Here’s the type of problem you have with meetings during a migration
 You invite your good friends to a meeting in the future
– Maybe two months from now
– You constantly collaborate and arrange meetings like this with people from different
legacy domains in your enterprise
 You invite people from another existing legacy domain, Funlab
– Homer Simpson/Corp/FunLab@Funlab
– Marge Simpson/Corp/FunLab@Funlab
21
Here’s the meeting invitation
 Note that the Domino domain name of each invited person has been appended to their
abbreviated Notes name
22
Where’s the problem?
 The invitation you sent was for a meeting 2 months from now
 Two weeks after you sent it, Homer and Marge are migrated to the Domlab domain
 Their names change from this:
– Homer Simpson/Corp/FunLab@Funlab
– Marge Simpson/Corp/FunLab@Funlab
 To this:
– Homer Simpson/Corp/FunLab@Domlab
– Marge Simpson/Corp/FunLab@Domlab
 This change is not reflected in your calendar
– It has not been updated and will not be updated, ever!
– As far as your calendar goes Homer and Marge are still in the Funlab Domino domain
23
Oh! So maybe updates to the meeting won’t reach them?
 Updates for the meetings will probably still get to them
 You probably made some changes to person documents to make sure their old names with
the FunLab domain are still recognized and mail can still route to them
– We have code to help with that
 One of the problems will happen when you try to suggest new time for the meeting
24
Owner can’t find the info to suggest a new meeting time
 The owner of the meeting can’t see the FunLab users’ free time because they are no longer
from the FunLab domain and the free time lookup can’t find them
– Even if that domain still exists, Homer and Marge no longer live there
• And it appears that their “info is restricted” – which usually generates a help desk call
25
Free time lookup doesn’t show the domain name for users
But it is stored in the calendar document
 This is confusing for users since the names appear to be correct, but the names that are
stored still have the old domain name
26
Invitee wants to suggest a new time but can’t do it
 To add to the frustration, even the invitee can’t see their own free time lookups
– If you rarely collaborate with users in the other domains, this will be less of a problem
27
How do you fix this?
 Since calendar entries live in everyone’s calendar it is very difficult to modify existing
calendar entries without completely cancelling and re-creating them
 Can you imagine how this affects repeating meetings that go way into the future?
28
Another possible issues during domain consolidation
The “recent contacts” problem
 Recent contacts are accumulated automatically by the Notes client
– It’s used to assist you in addressing and in meeting invitations
 The name you use in addresses will be stored in recent contacts
 The names will include the domain name of the contact
 Each time you create a new meeting or mail message you will be presented with recent
contacts that have the domain name appended
 When the name and domain are correct, it works perfectly
– When the domain is not correct, it still gets used
• And thus all of that fun with meetings and free time lookups continue
29
How to fix the recent contacts issue
 You can just turn off recent contacts
– Most of the time I like the way recent contacts handle giving me addresses
• The problem is not exactly recent contacts themselves
• It’s a fact that they don’t seem to refresh as often as they are supposed to from the
legacy address books
• The old domains remain in recent contacts
 It is possible to surgically remove them from the recent contacts view
30
Removing legacy domain entries from recent contacts
 You can prevent legacy domain entries from polluting your recent contacts by inserting a few
parameters into the NOTES.INI file on the local client
 See this technote for the details
– http://www-01.ibm.com/support/docview.wss?uid=swg21415228
 If you needed to prevent the Domino domains FunLab and Simco out of your recent
contacts you would use this parameter:
– DPABRemoveRule=FunLab, Simco
 If you also want to also stop aggregating users from funlab.com and simco.com, use this:
– DPABRemoveRuleSetting=1
 This can be done using a policy with a desktop policy settings document
31
Desktop policy settings to control Recent Contacts
 In the desktop policy settings select custom settings
– Then use edit list to add the parameters as shown below
– You can still use recent contacts, but prevent the names from certain domains from
being added
32
Contacts added to personal address book from Domino
directory
 The issue of the retained legacy domains on person documents will also happen when users
copy names from the Domino directory to their local Contacts, NAMES.NSF
– You can socialize this issue with the right communications
– Make sure the help desk knows how to handle this type of problem
 Sometimes it is simpler and more coherent to solve these issues socially than to come up
with a complex technical solution
– You can send a post open event to the user in a mail file, but really the user just has to
know that they need to delete the users from their contacts and re-add them
33
Special cases
 Any user with initials after their name (VP, CIO, CEO) might need some hand holding or
some automated method of fixing this
34
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
35
Cross certification needed, authority needed
 There will be a period of time when your domains have to work together
– Regardless of whether you’re planning on decommissioning some of them
 Cross certification will be needed if your domains don’t run under the same certifier
 Create an administrator account that has manager access to all files in all domains
– Turn on password checking on the servers
– Turn on password checking for this powerful account
• That will eliminate the risk of someone stealing this ID
- Once you change the password on the ID, anyone who has an ID but doesn’t
have the current password won’t be able to use it
36
Changing the building blocks of the domain
Server changes
 Server changes
– When the servers documents must be configured for a new domain
• There is one place in the server document that holds the Notes domain name
– The NOTES.INI parameter Domain= must be modified for the new domain
 That’s all there is to it!
– Change those two things and bounce the server, the server is in the NewDomain
37
Here’s a simple agent that modifies the mail domain field
 Here’s an agent that uses simple formula actions and modifies the MailDomain field
38
Person documents
 Server documents are easy to change by editing a the server document
 Person documents require an agent because there are more person documents than server
documents and there are more changes per document
– Change the maildomain field to NewDomain
 And we like to add an alias to the fullname field that reflects the old domain of the user
– If another user replies to an old email send by out this user the mail router still finds
him/her
39
Adding the username with the old domain to fullname
 This is a little more complicated that just changing the domain name
– It is done in LotusScript
– Administrators who know Lotusscript can get more done in a shorter time
 Let’s say that the user is in the Notes domain called DOMLAB
– And the user is to move to NewDomain
 The code looks to see if there is an alias in the FullName field in the person document
– It would look for Andy Pedisich/technotics@domlab
– If it’s not there it would append it to the text list
• This code is in the package downloadable from http://www.andypedisich.com
40
Roaming user changes
 If the user is a roaming use, the RoamSrvr field must also be changed to represent the new
server
– This is also part of the agent that modifies the person document and is downloadable
from Andy’s blog
41
Group Documents
 The MailDomain field must be changed to the NewDomain
42
Changing maildomain does not require LotusScript
 Changing the maildomain field in group documents uses the same formula code as
changing the maildomain field in person documents
43
Mail-in databases
 Similar to person documents, there are many of these
 But there is not much to change
 Again, an agent using simple formula actions can easily modify the MailDomain field
44
Changing connection documents
 There are two domain fields in connection documents
– SourceDomain
– DestinationDomain
 Change the appropriate field to the new domain name
– This will be probably be done manually since each connection document needs to be
evaluated to see if changes are needed
45
Change both fields at once or create two agents
 You can change the connection documents using easy code like this
– Or you can split the code into two agents so that you can change only the destination
domain in one agent and the source domain in another
46
Domino directory objects you won’t have to modify
 The following document types do not contain a domain name
– You don’t have to worry about these during a domain consolidation
• Server configuration documents
• Program documents
• Adjacent domain documents
• Foreign SMTP domain documents
• Policy and policy settings documents
• Web site documents
47
Don’t forget that the local NAMES.NSF needs work
 Location documents must be evaluated
– The DOMAIN field must be changed in location docs where old domain name appears
• Critical to correct functionality of calendaring and scheduling
- Which uses the domain information in the location document
 We generally send the user a button with code
– The user clicks the button and the changes are made
– The agent logs the actions success or failures in a mail-in database so that progress can
be tracked
48
Setting up for cross domain adminp requests
 If there are multiple domains being
consolidated it is sometimes necessary
create a special cross domain adminp
request configuration in the admin4.nsf
database
 Start by opening admin4.nsf and
navigating to the Cross Domain,
Configuration section
49
Add a configuration document
 Select inbound or outbound, depending on
what domain you are configuring for
 Then select the actions you would like to be
send across the domains
50
Cross domain requests (cont’d)
 Enter the server names that will be
permitted to submit requests for Create
Replica and Update Delegated User’s Mail
File
– Also indicate the domain of that server
– The domain name must be listed in the
“Domains to Submit Adminp requests
to” field
– These are case sensitive
 Then select the list of approved signers of
those requests
51
Connection documents required for cross domain adminp
 You must create connection documents between the servers you are using for cross domain
adminp requests
 Since these servers two or more different domains, they will not be in the same Named
Notes Networks
– Mail routing will not naturally occur with point to point routing
– Connection documents must be created in each domain that point directly to the server
receiving adminp mail in the other domain
 Connection documents are needed in situations where you don’t use cross domain adminp
requests
52
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
53
Consolidation three different ways
Let’s look at the rationale for two different methods of consolidation
 Fictitious Customer #1 has the following scenario
– Many companies held by a parent company
– Couple thousand users – many organizations cross certified
– Environment – Domino Release 8.5.3
– 7 domains – pockets of Domino servers dispersed – distributed administration
• US and UK take care of their own technology
– Multilingual situation – South America, India, China, USA
– Mail jail is used – If you’re over quota you can’t sent or receive mail
– Not big on using policies
 They want one Notes domain and one Internet domain to help them manage their
messaging/collaboration environment and to help brand their products
54
What’s the approach?
 Since they were basically happy with their infrastructure we see this one as a really simple
domain consolidation to collapse the 7 domains into a single domain
 We would flip existing servers into the new domain during off hours and restart them in the
new domain
 Here’s one way to do that
55
Build a new domain – The Big Bang Theory
 First step is to build a new domain on a server we rent from the cloud
– The cloud is the perfect place to build a new server rather than in our office
– Because we don’t need to make an permanent investment in hardware
• This isn’t a requirement, it’s just something we find convenient
 We create a new domain with the name they want to use and copy the contents of the other
Domino directories into it
– Then we use agents to modify the contents of person document, server documents,
connection documents, mail-in databases and all the others we spoke about in the
previous part of this session
56
Did you hear that? Build a new domain!
 How many times have you had the opportunity to build a new domain?
– This is a “greenfield” operation
– Build it as a “perfect” domain
 Your chance to implement a new domain according to best practices
– ID Vault
– DAOS
– Tight security
57
EVENTS4.NSF Is a Key File in the new domain
 We want to monitor all of our servers and be notified when certain conditions occur
– We specify what we want to watch for
• Low disk space
• Common error messages about security – illegal access to files
• Corruption in databases
• Warning messages about messages not being transferred
• Server is down
 And how we want to be notified
– Pager
– Email
– Phone call
 We will use the EVENTS4 database to configure all of this
58
Same replica on first server and all servers added
 To be effective, the EVENTS4 database should be the same replica on every server in your
domain
– I have found many, many cases where EVENTS4 was not the same replica everywhere
in the domain
• That ruins the monitoring architecture
- The monitoring configuration can’t replicate to some servers
 Servers will be isolated
– Your environment won’t be consistently handling and reporting problems
 It’s not only the EVENTS4.NSF, it’s other system databases you need to treat this way
59
We Know What the Replica ID Should Be for EVENTS4
 The replica ID of system databases like EVENTS4 are derived from replica ID of directory
Database
NAMES.NSF
CATALOG.NSF
EVENTS4.NSF
ADMIN4.NSF
DDM.NSF
Replica ID
852564AC:004EBCCF
852564AC:014EBCCF
852564AC:024EBCCF
852564AC:034EBCCF
852564AC:0A4EBCCF
– Notice that the first two numbers after the colon for the EVENTS4.NSF replica are 02
• Determine your address book’s replica ID, and you’ll know the replica ID of
EVENTS4 and other system databases
60
Verify EVENTS4
 You must verify that every server has the same replica
of EVENTS4
– You can find this info in the catalog, if your catalog
architecture is getting file info from all servers
• Or you need to go to every server and open EVENTS4
- It’s vital that you validate EVENTS4
 If it is not right on a server, then you must down the server, delete EVENTS4, and restart
your server
– The correct EVENTS4 will be re-created automatically
– This works with the other databases as well
61
Errors You Might See If DDM.NSF Is Not Right
 If there is a DDM.NSF on every server but they aren’t all the same replica ID, you’ll see the
following error on the console every couple of minutes:
– Unable to replicate with server Server2: None of the selected databases have a replica
on the server
• You’ll get this even if you have a much longer replication interval scheduled
 To fix problems related to EVENTS4.NSF and DDM.NSF replica IDs, you must delete the
bad databases and restart the server
62
This Configuration Must Be Correct
 Make sure that EVENTS4.NSF and DDM.NSF have the correct replica ID throughout the
domain by opening each and putting it on your desktop
– The icons should stack, showing they are the same replica
• Check the replica ID by looking at the properties of
the databases
63
Want to Add Every EVENTS4.NSF to Your Desktop?
 Add this code to a button on your toolbar
– This is courtesy of Thomas Bahn
• www.assono.de/blog
- It’s a great tool that I use almost every day
_names := @Subset(@MailDbName; 1) : "names.nsf";
_servers := @PickList([Custom]; _names; "Servers"; "Select servers"; "Select servers to add database
from"; 3);
_db := @Prompt([OkCancelEdit]; "Enter database"; "Enter the file name and path of the database to add.";
"log.nsf");
@For( n := 1; n <= @Elements(_servers); n := n + 1;
@Command([AddDatabase]; _servers[n] : _db) )
64
Add Databases to the Desktop
 This code will prompt you to pick the servers that have the database you want on your
desktop
– Then it will prompt for the name of the database
• And open it on all the servers you’ve selected
 Use it to make sure all the EVENTS4.NSF are the same replica in your domain
65
How we flip a server into a new domain
 Create replicas of the system databases from the new domain to the server we want to flip
 And rename as you replicate them to existing servers
66
Normal file name
Migration file name
names.nsf
namesnew.nsf
ddm.nsf
ddmnew.nsf
events4.nsf
events4new.nsf
catalog.nsf
catalognew.nsf
activity.nsf
activitynew.nsf
admin4.nsf
admin4new.nsf
How we flip a server into a new domain (cont‘d)
 Shut down server
 Rename the following system files
67
Server to be migrated
Archive file name
names.nsf
namesold.nsf
ddm.nsf
ddmold.nsf
events4.nsf
events4old.nsf
catalog.nsf
catalogold.nsf
activity.nsf
activityold.nsf
admin4.nsf
admin4old.nsf
Some more renames while server is down
 Rename the following system files
 Lower case letters are extremely important with some OS – Unix, AIX, Linux
68
New files you’ve replicated
New system file name
namesnew.nsf
names.nsf
ddmnew.nsf
ddm.nsf
events4new.nsf
events4.nsf
catalognew.nsf
catalog.nsf
activitynew.nsf
activity.nsf
admin4new.nsf
admin4.nsf
Some additional adjustments
 Change the NOTES.INI file parameter DOMAIN= to the new domain name
 Restart the server in the new domain
– Replicate with the first server you built
– Perhaps make this new server your administration server for the address book
 Take these steps with each server in the legacy domains until all are running in the single
domain
 Restart the server in the new domain
 Take these steps with each server in the legacy domains until all are running in the single
domain
69
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
70
Fictitious customer #2 would like to start over
 Fictitious Customer #2 has the following scenario
– Over 1,000 users
– 3 Domino domains
• Each with their own mail servers
• Some with application and development servers
• Administrators are at each site but have other jobs - don’t want to manage Domino
• There are not a lot of cross domain meetings to worry about
• They use managed replicas
• They are huge on using policies
71
It’s a brand new day!
 Customer #2 wants to:
– Move to a centralized server farm that uses Domino clusters and load balancing
– Move all users and everything in the legacy domain to the new servers
– Other objectives such as id vault implementation, DAOS, and some third party tools to
make domain management easier
 And the kicker – they want to gradually move groups of people to the new domain
– Offices move together
– Regions move together
– Functional groups move together
72
You will grow weary looking at spreadsheets full of users
 They will send you list of people
– Alphabetical by first name or last name
 You will have to select those person documents for those users
– Send them an email to deal with their location documents
– Select them in their legacy domain
• Modify them
– Copy them into the new domain’s directory
– Delete them from their legacy domain
73
An tedious yet effective migration method
 We call this method of migration
– “Death by a thousand cuts”
• Because it is the most arduous method for administrators
- Lots of matching up lists
- Making sure replicas are created
- Counting people
74
So, what’s the approach
 After the centralized empty servers are stood up in a brand new domain
– Start the work of creating replicas in the centralized server environment
 We use the Notes Administrator Client to select all the files we will create replicas of
– Then use the Manage ACL to add the new server names and managing groups
75
Selecting the files that need a replicas in the new domain
 The files you select to create replicas of will depend a lot on your current infrastructure
 Fictitious Company #2 has multi-purpose servers
– They contain
• Mail files
• Mail-in databases
• Archives
• Roaming files
• And lots of applications
76
Selecting the files that need a replicas in the new domain
 It is a good time to determine if you need special purpose servers that might be new for you
 Starting over is a great time to re-evaluate your requirements
– Mail servers
– App servers
– Gateways
– LDAP
– Instant messaging and meeting
– Mobile device
77
Selecting the files that need a replicas in the new domain
 Review these files to be certain they are still being used
– Each mail file should have a person document associated with it
– Mail-in databases should be represented by mail-in database documents
– Archived should be linked to mail files, and they should still be needed
– Work with Human Resources to keep your directory from filling with people who no
longer work there
• We’ll talk about applications later
78
Modify ACLs on all databases to be moved
 Add the new mail servers to the ACL
– Since there are 3 domains consolidating we ensure that the admins of the central
domain have manager access to all files by adding LocalDomainAdmins
• Or whatever you might call your administrator group
79
Ensuring the correct advanced database properties apply
 Using the Notes Administrator client, we
select all the databases on a legacy
server and apply these settings
– Replicate unread marks, all servers
• Users like their unread marks
– Use LZ1 compression to ensure
attachments are all compressed the
same way so that DAOS is efficient
– Use DAOS
– Compress database design
– Compress document data
 By doing this in advance we won’t need to
compact the databases to use DAOS
80
Use the Create Replica tool
 Using the Notes Administrator client, we create replicas from the existing domain
– To each of our new centralized mail servers and their clustered partners
– Each file we create a replica for generates 2 create accelerated replica requests
• We generally select them in groups of 25 to 50 during the day, 100 at night
 Be sure to check the “Exchange Unread marks on replication
– Take the opportunity to create full text indexes if needed
81
Change database administration server configuration
 Use the Notes Administration tool to modify ACLs and most importantly
– Change the administration server for all databases
• Pass control of renames and deletions to the new domain
82
We throttle Create Replica adminp requests
 We generally do less creating of replicas during the day
– The admin4.nsf file gets so filled with requests
– It delays other types of adminp task requests that should have more priority
• Such as password changes and renames
• User deletions
• Mail file delegations
83
We throttle Create Replica adminp requests
 This replication method must be planned enough in advance prior to domain consolidation
– Also make sure that new files that are added between the creation of the first set of
replicas are created on the new clustered servers
 There are other methods of getting the replicas over to the centralized clustered servers
– Restore a backup onto the new servers
– Shut down the servers and using OS level copies to get the files to the new servers
• These methods were not appropriate for fictitious Company #2
84
Policy accord necessary and certs must be in place
 Since we are folding multiple domains into a single domain, the administrators of the 3
legacy domains have to agree on policy settings they should all use
 The 3 domains use separate certifiers and organizations
– The recertification of uses to a single certifier is another project entirely
 Policy settings documents need to be created
– Policy documents for each of the 3 organizations must be created
– Certifiers from the 3 legacy domain directories must be copied to the new domain
directory
– You can’t just copy your existing policies and policy documents
– They won’t link up
85
Staging and backout directory
 We’ve created some views and agents to help us control the migration process
– We’ve added them to the staging directory, the backout directory, and to the production
Domino directories in the legacy domains
– They are used as a holding area for all documents in the legacy domains and in the
staging directory
86
Review groups before the move
 There are some group names that are common to most Domino domains
– LocalDomainAdmins
– LocalDomainServers
– OtherDomainServers
• These groups obviously should not be copied to the new domain
 In most cases we do not find duplicate group names
– In large domain an agent should be used to check for duplicate names in large
– In smaller domains the legacy and new domain directories can be placed in different
windows and visually compared
87
Communications with the user to be migrated
 We also must create an email to
the users that are being migrated
– This will include a button to
change their location
documents
• More on the message in a
second
88
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
89
About this part of the following migration example
 Our example will be using the following servers and domains
– The RED domain uses the server Mail01/Red
– The BLUE domain uses the server NewMail2/Servers/Blue
 Our goal will be to collapse the two domains together into the Blue domain
90
Ready to migrate to the new servers
We create some files and agents to help us
 To assist us in consolidating fictitious Company #2 we created two Directory files
– A “Staging Domino Directory” was created from the server based public name and
address book template pubnames.ntf
– A “Backout Domino Directory” was also created from the same template
– Both are customized with added agents and views
 The Staging directory is used to hold copies of person, group, and other documents that can
then be modified using agents before being copied into the directory of the new domain
 The staging directory has several simple formula agents and several Lotusscript agents
91
Agents in the staging and production directory
 Change Person doc – staging directory only
 Change Mail-in doc – staging directory only
 Change MigStat to Prep
– Staging and production directory
 Change MigStat to In Progress
– Staging and production directory
 Change MigStat to Completed
– Staging and production directory
92
Change person document, mail-in database document
 Change person document is a LotusScript
agent makes critical changes to the person
docs in the staging directory
– It is found only in the staging directory
– It is used to prepare the old legacy
person documents to be pasted into the
new domain
 Change Mail-in database document is a
simple formula agent
– It adjusts mail-in documents to be
pasted into the new domain
93
Changing MigStat – the migration status for person docs
 The migration status agents add two fields
to the person documents
– In the production directory
– In the staging directory
 The two fields are:
– MigStat which is the migration status
– MigDate which is the date the agent
was executed
 As you will see in a few moments, this gives
us a clear picture of where the users are in
the migration process
– Give managers access to the staging
directory if you want clear
communications
94
New view in production and legacy directories
 A view has been added to the staging
directory and to the production directory
– People\by migration status email date
 Fields are going to be added to the person
documents
– This view will help us track those fields
– Let us know where people are in the
migration process
95
Why track the movers?
 It’s important to keep track of the users that are being migrated as they step through the
process
– There will be dozens, perhaps hundreds or thousands of users at different stages
• We want to be able to track them
 Fictitious Customer #2 always communicates with their users to prepare them for migration
and sends us lists of users they would like migrated
 We move them through the following 7 step process
96
The back out directory
 The Backout directory is used to hold copies of those documents that are not modified
– If we need to back anyone out of the migration we can put back these documents if
others were deleted from the legacy domains
 Speaking of safe, always make a safe copy of your Domino directories before changing
anything
97
Overview of the 7 step process
 Step 1: Mark the users person documents in the legacy domain directory
 Step 2: Copy the person documents into the staging directory and the backout directory
 Step 3: Run agent on their person documents in staging directory
 Step 4: Send users a button that changes location documents
 Step 5: Copy their person documents from staging directory to directory in their new domain
 Step 6: Delete their person documents from the legacy domain
 Step 7: Make sure all directory catalogs are updated
 …and here come the details
98
Step 1: Mark user’s person docs in Red legacy domain
 Select those user’s person documents in the legacy Domino Red domain directory
 Run agent on them to mark them as being in the “prep” status (Prepared to migrate)
 There is a new view copied from the Staging directory on the legacy domino where the
status is evident and it shows the status and dates the status has changed
 This makes it easier to follow users through the process of migrating
99
Step 2: Copy person docs into staging/backout directories
 Copy the person documents into the staging
directory and the backout directory
 The backout directory will hold a copy of the
original document which will never be
changed
100
Paste the person docs into the staging directory
 The person docs pasted into the staging directory will be displayed in the same view as the
one in the production directory
101
A you progress you’ll be able to track users
 This method of marking the person
documents has value
– You’ll be able to easily identify the users
that are in prep, in progress or
completed
– It will be easy to determine what
happens next to each user
102
Possible http password issue when restoring a backout
 The user might change their http password after they are migrated
 If the users person document is subsequently copied from the backout and pasted into the
Red legacy domain
– And removed from the new Blue domain
– They won’t have the new password in their person document
– Until the HTTP password was re-entered into the person doc
 This never happened to me in the real world, but it is possible
– Your help desk should be made aware of this scenario
103
Step 3: Run agent on person docs in staging directory
 This agent runs on selected person documents in the staging directory
 Modifies the mailserver field, replacing the contents with the name of the new server
 Modifies the maildomain field, replacing contents with new mail domain name
 If the user is a roaming user, it replaces the RoamSrvr (Roaming server) field with the name
of the new mail server or the name of the new roaming server
– In most situations the roaming server is the same as the mail server
 It adds an alias to the FullName field in the person document
– This is FirstName LastName@OldDomain
• This will cover use cases where a user is replying to an email where the old Notes
domain was being used
• Mail will still route to the person in that scenario
104
Running the agent on docs in the staging directory
 All the changes are made
– No typos
– No fat finger missed keystrokes because it is all automated
105
Example of person doc before the agent runs
 This is a person doc in the staging directory prior to the agent running
 Note that there is one single entry in the fullname field
– Domain name is Red
– Mail server is Mail01/Red
106
Example of person doc after the agent runs
 This is a person doc in the staging directory after to the agent running
 Note that there is a second entry in the fullname field – David Lowery/red
– Domain name is Blue
– Mail server is NewMail2/servers/blue
• The person doc is ready to be copied and pasted into the Blue domain directory
107
Step 4: Send users button that changes location docs
 The button code is signed by a functional account (not a human) who is listed on the
Administrative Execution Control List (ECL) and is allowed to make changes to the
workstation without causing an ECL alert to display
 The code finds all the location documents that contain the legacy domain name and builds a
collection
 The code changes the DOMAIN field to the new domain in all the docs belonging to the new
domain
 It places the address book and the person’s mail file on their desktop
 Sends an email to a tracking database where it posts the results of the run and if there were
any errors
108
Button will be sent in an email
 The button should be sent only to those users
who are being migrated right now
– This should be mailed prior copying and
pasting the documents from the staging
directory to the Blue domain directory
 It’s pretty easy to address the email message
 First, select the users in the view in the staging
directory and copy them as a table
109
Paste into spreadsheet – copy email address
 Paste into a spreadsheet
 Copy out the email addresses
110
Create an email
 Create an email and paste what copied from the spreadsheet into the To field
– Addressing an email to 7 or 70, or 150 users takes about the same amount of time
111
Helpful tips regarding communications with users
 Be sure to ramp up the intensity of the
message that goes out to the users
– Mark it as “High importance”
– Include a courtesy copy (Cc: )
• To the manager of the user
• Or some other person the user
might fear
 Clearly state in the subject that immediate
action is required
– That will encourage the user to act now
112
Helpful tips regarding communications with users (cont’d)
 Mention the advantages of the moving to
the new servers
 Tell the users how long it will take when
they click the button
 Provide instructions for users that have
special situations
 This includes
– Users with multiple workstations with
Notes installs
– Traveler
– iNotes
 Provide web links with help
113
Helpful tips regarding communications with users (cont’d)
 Don’t forget to include the help desk
phone number
– This is a very important touch
– The user might not know how to
create a bookmark
 Even though you’ve tested the button
there can still be situations you haven’t
counted on
114
Step 5: Copy person docs from staging to new domain
 Mark the users as completed using the
agent
 Copy their person documents from staging
directory to directory in their new domain
 Replicate the Domino directory in the new
domain to all servers to ensure that the
new additions are on all servers
– This ensures that mail routing to the
users in the new domain will be correct
115
Step 6: Delete person docs from the legacy domain
 Delete their person documents from the legacy domain
– Sometimes we check the mail in database that is the log for the button action being
clicked before copying to the new domain and deleting from the legacy domain
– If the mail in database has a successful log document from the user clicking, they no
longer need the person document in the legacy domain
• Replicate legacy address books for all domains to ensure user has been removed
 You can use the Delete Person button if you want to also remove their mail file
– Generally customers like to leave the mail file on the legacy server for a week or longer
– When everything is stable, mail files on the old servers can be removed
 Mail files can also be deleted using the Notes Administrator Client
116
Step 7: Make sure all directory catalogs are updated
 Make sure all directory catalogs are updated with the new domain and servers for the
migrated users
 Any extended directory catalogs or compressed directory catalogs should be refreshed or
rebuilt
– Generally we shorten the refresh interval setting for directory catalogs when we migrate
users to the new domain
 At this point you can mark the user as having completed in the staging directory using the
agent “Change Mig Status to Completed”
117
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
118
Mail-ins are easy to migrate compared to people
 Copy the mail-in database documents into the staging directory
 Run the agent “Migrate mail-ins to new server”
 This agent will modify the maildomain and mailserver fields to their new domain and server
 Copy the changed documents to the Domino directory in the new domain
 Remove the original mail-in database documents from the legacy domain’s directory
 Replicate changes to all servers
 Delete the mail-in databases from the legacy domain using the Notes Admin client
119
Be sure to leave a Notes reference file behind
 When using the Notes Admin client to delete the mail-in database use the Delete button
provided in the navigator on the right of the screen
 Using this button will give you the option of leaving a marker that will allow clients to update
their bookmarks and icons to this new location
– And it will remove the icons and bookmarks from the file that was deleted
120
Application migration
 There are lots of issues that come with migrating applications
 If the application is a web app accessed using an Internet browser and you move it to a new
server users will need to update their bookmarks
 If the application contains agents or design elements that refer to a particular server name
that is hard-coded, you might find yourself a designer to help you
 If the application references other databases, make sure the other databases are still
available in the new domain
– Some changes might have to occur in the application to keep it viable
 How do you know what applications are actually used
121
Using activity logging to determine if an app is being used
 Activity Logging servers account for their time, recording user
activity by person, database, and access protocol.
 Use server configuration documents to turn on Activity logging
 Marking your prime shift helps you in interpreting results that look
at that time period
122
Turn on Activity trends
 Also turn on activity trends which will run every day you specify at 3:23 AM by default and
collect information from the LOG.NSF and
 Activity logging stores information in the Domino log
– Expect logs to grow much larger
123
Results of activity logging
 Activity logging produces a generous
amount of raw data and calculates growth
rates for storage
 It also tracks user activity and database
activity
 But what best suits our purposes during a
consolidation is the applications that are not
very active
124
Inactivity of databases helps you to eliminate applications
 This view sorts by date of last activity
 It makes it easy to see apps that haven’t been used in a while
– Keep in mind that an app might only be used a few times a year by the board of directors
• That would mean it’s still important to retain despite it’s low usage
125
Plus you can see trending details about the databases
 Use the Open Database Trends button to see the details about the database
– Check out how quickly it’s growing and get a usage summary
• This will definitely help you in determining apps to migrate
126
Agents and consolidation
 Agents can be quite problematic
 Agents are generally set to run on a particular server
– Analyze all of your agents in applications
– Change the name of the server the agents are supposed to run on
 Improperly written agents might also have hard coded server references in them
– That’s not going to work if the server that’s hard coded is a legacy server that you will be
decommissioning
– You must search out that code and correct it
 There are third party tools to help you do this
• Check out the vendors in the Solutions Showcase at Connect 2014
127
Agent signers must have rights
 The agents were probably signed by an account in the legacy domain
– The agent signers must be an account in your new domain
– Do the research on this in the early stages of the migration project
• Or you’ll be forces to deal with it at the end
• Domino Domain Monitoring can help you track if this is happening
 Moving from a non-clustered environment to a clustered one can be a problem when agents
are set to “run on any server”
– In a cluster, agents configured that way will run on the replicas on both servers in the
cluster
– If two agents process and change all the documents in the database it will create a ton of
replication conflicts
128
Clusters can be a pain
 Moving from a non-clustered environment
to a clustered one can be a problem when
agents are set to “run on any server”
– Agents configured that way will run on
the replicas on both servers in the
cluster
– If two agents process and change all
the documents in the database it will
create a ton of replication conflicts
 Make sure you include an analysis of where
agents are configured to run
129
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
130
Last server standing
 Throughout the migration process we’ve been careful about mail routing in terms of a
particular use case
– Users are going to reply to an email that has the legacy domain address
– Frank Fish/red@Red
 We have provided an alias so that mail can still reach that person
– The email that is sent is routed back to Mail01/red which is in the Red domain
 When it arrives there the router looks at it and sees that the person is actually in the Blue
domain and routes it back to Mail2/servers/red
131
What happens when the legacy domain is removed
 Unfortunately when you remove the Mail1/red server and the Red domain is gone mail
routing breaks
 In some cases we’ve seen very small servers left in the legacy domains with a directory from
the new domain as a secondary directory
– Mail is still routed to the old domain
– Everything still works
• Nobody really likes that solution
132
A better solution
 We need a configuration where mail sent to Frank Fish/red@Red is routed to
Frank Fish/red@Blue
 One rather simple way to do that is to create a foreign domain document
 And a mail.box for the legacy domain
133
Creating a mail.box for the legacy domain’s messages
 Create a new application from the Mail
Router Mailbox template
– Title it “Red Domain’s Mail”
– Name the file redmail.box
 Then create a Foreign Domain Document in
the Domino directory
134
Creating a foreign domain document
 Open the Domino directory
– It’s configures quickly if you open it on
the server that will contain the Blue
domain’s foreign domain mail.box
 Navigate to the Messaging\Domains section
 Click the Add Domain button
135
Creating a foreign domain document (cont’d)
 Create a new Foreign Domain document
– Use the domain type of Foreign Domain
– For the foreign domain name use Red
136
Switch to Mail Information tab
 For the Gateway Server name, use the name of the server where you created redmail.box
 For the Gateway mail file name enter redmail.box
137
Go to the server console
 Go to the console on the server and enter this command:
– Tell router update config
• Or wait 15 minutes and the router will update it’s configuration automatically
 From now on mail sent to Frank Fish/red@Red will be sent to the redmail.box file
 And it will wait there for an agent to run that will remove @Red from each recipient’s
address and replace it with @blue
 Frank Fish/red@blue will then be sent back into the system and the mail will be properly
routed
138
What we’ll cover…
 Determining why you want to consolidate domains
 Why you have to move fast during your domain consolidation
 Understanding the basic elements of Domain consolidation
 Flipping servers into a new consolidated domain
 Preparing to move people, groups, Mail-in DBs, and other objects
 Migrating the users to the new domain
 Migrating mail-in databases, applications and agents
 Keeping mail routing after decommissioning legacy domains
 Finishing up with random thoughts, tips, and risks
139
Something to remember when combining domains
 When combining domains, remember that policy settings are super important
 Imagine collapsing 3 domains into a single domain
– And those three domains had their own policy settings
• How would you set security and desktop settings across those 3 domains
 If the 3 domains were using different organization or OU levels it would be easy to keep the
same settings
– But do you really want to do that with your new centralized domains
 Getting agreement on settings can be difficult
– But in the end everything will be easier to manage
140
It’s a matter of case
 And by that I mean uppercase and lowercase
 If you’re moving to another operating systems from Wintel boxes, remember that some
operating systems are case sensitive in respect to file names
– Be careful when making replicas of files
 As a rule of thumb make all replicas in lower case
– The Notes spell checker likes to make the initial character a capital letter
– Mail file locations become Mail\apedisic.nsf
• The location will have to match what’s in the Domino directory
141
Do your users want full text indexes?
 A user can search their mail file using the
Search This View unless it’s been
specifically turned off by the advanced
properties in the database
– If you’re making new replicas of mail
files make those full text indexes when
you create the replica
142
Create full text indexes when you make the replica
 It’s pretty simple to create the full text index
when you create the replica
– Then it’s done and no need to back
track to complete the task
143
Creating files for new servers
 The Create Replica adminp request can be slow when moving several thousand files
– Consider shutting down both servers
– Do an OS level copy over a weekend
 Or use backup tapes to restore the data directory to new hardware
 Or create the files on a SAN snapshot
144
Combining domains – consider creating directory catalog
 Mobile and desktop users will have faster
typeaheads if you provide them with an
Extended Directory Catalog
 Create a directory catalog using the
pubnames.ntf template
 Control the aggregation of the people in the
directory catalog in the extended directory
catalog’s configuration document
145
Control which server aggregates EDC in the server doc
 Use the Server Tasks/Directory Cataloger
task to control which server does the
aggregation of the address books
– And how often that occurs
146
That’s just about it
 You now know a lot of what we know about domain consolidation
 Take it with you, implement, and enjoy
 …And get it over with already!
147
Resources
 Frequently Asked Questions - The Administration Process (AdminP)
– http://www-01.ibm.com/support/docview.wss?uid=swg21212760
 Domino Server Consolidation: Best Practices to Get the Most Out of Your Domino
Infrastructure
– http://www.redbooks.ibm.com/abstracts/redp4181.html
 Merging Domino domains into one
– http://www.ibm.com/developerworks/lotus/library/lsMerging_Domino_domains/index.html?S_TACT=105AGX13&S_CMP=EDU
 IBM Lotus Notes 8 Recent Contacts and Type-ahead features: FAQs
– http://www-10.lotus.com/ldd/dominowiki.nsf/dx/ibm-lotus-notes-8-recent-contacts-andtype-ahead-features-faqs
148
Wrapping it up… things to take home
 You may not have been around when your domains were created, but you can still make it a
jewel with consolidation
 The most risky time during migration is when you haven't completed it
 Every administrator should have some skills writing LotusScript and formula agents
 Using a staging and backout directory during user migrations reduces risk and gives you lots
of info about the status of your moving users
 Events4.nsf and other system databases should have the same replica ID on every server in
a domain
 When migrating lots of users the more you prepare with automation the more relaxed you
can be when it's actually happening
 Do your homework with agents when migrating or you might find processes stop working or
lots of replica conflicts being created
149
 Access Connect Online to complete your session surveys using any:
– Web or mobile browser
– Connect Online kiosk onsite
 Find us on http://www.technotics.com
 And at:
– http://www.andypedisich.com
– Andyp@technotics.com
– Rob@technotics.com
150
Acknowledgements and Disclaimers
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither
intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information
contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise
related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or
its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and
performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you
will result in any specific sales, revenue growth or other results.
© Copyright IBM Corporation 2014. All rights reserved.
 U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
 IBM, the IBM logo, ibm.com, IBM IBM® Domino® Server, and IBM Notes® are trademarks or registered trademarks of International Business Machines Corporation in the United States,
other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S.
registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A
current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
151