JMP402: Master Class: Reaping Rewards and Avoiding Risks - The Secrets of Consolidating Domains Andy Pedisich, President - Technotics, Inc. Rob Axelrod, Vice President - Technotics, Inc. © 2014 IBM Corporation Your presenters They are two hard working IBM® Notes® Administrators/Developers who have worked with IBM® Notes® and IBM Domino® since version 2.1 – From Technotics, Inc. in Philadelphia, Pennsylvania - USA Andy Pedisich – 29 years in IT – 19 years with Lotus Notes Rob Axelrod – 25 years in IT – 19 years with Lotus Notes 2 About Technotics, Inc. Technotics was founded in 1998 as a consultancy to focus on collaboration in the enterprise. Since that time we have provided strategic advice, project management and technical support to organizations world wide, focusing on high levels of customer engagement and long term relationships. Rob Axelrod Our services include environmental audits, premium support, executive briefings on cloud based collaboration and migrations between messaging and collaboration systems. Contact Andy at andyp@technotics.com or Rob at rob@technotics.com. 3 Andy Pedisich What’s under our finger tips during this session Our host laptop is a Dell Studio™ 1555 – Intel® Core™ 2Duo CPU T9600 2.80 GHz – 8 GB memory The operating system is Microsoft © Windows 7 Ultimate – 64 bit – Copyright © 2009 Microsoft Corporation. All rights reserved – Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. IBM® Domino® Server Release 9.0.1 64 bit – Running on host system, Using IP address of the laptop's network interface card – Running IBM Traveler IBM Notes® Release 9.0.1 4 Examples and demos from this session… We’ve packed up the files and code used in this session – They are available on my blog – http://www.andypedisich.com Look for the link at the top right of the screen that will take you to the files you can download 5 Why we are doing this presentation There are a myriad of products, tools, and professionals that can assist you during a Domino domain consolidation We’ve done more than our share of them over the years – We’ve done a lot of really good ones • We’ve also found “error conditions” that can trip you up We’re here to share what we know and what we’ve learned – To make it easier for you to do it yourself • It will be easier to manage if you have others do it for you – You’ll have a better idea of the tools and the expected outcomes 6 What a Master Class is all about Since you are attending this Master Class, we assume that you know that: – “Master Classes” are designed for the practitioners already in the trenches and are targeted to those who are developing sophisticated applications that support business processes and departmental/enterprise administration, who want to kick their skill set up to the next level.” 7 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 8 Why are you thinking about a domain consolidation? Most of our domains are not new – Many were started in the mid 1990s – We go back to version 2.1 What we designed back then was great for back then – Lots of Domino servers implemented close to the users – Replication topology that worked great over slow data links 9 Why are you thinking about a domain consolidation? If you started your domain back then, you were – Working with modems – Thinking about saving a few dollars by having your servers replicate at night • Because the telephone rates were cheaper - Modem files don’t ship with the Domino server and Notes client anymore 10 The 1990s were a time of growth and change The National Science Foundation said this about the 1990s and the Internet – “Entire new industries sprang up (and in some cases crashed back down) as humanity embraces the biggest technological breakthrough since the Industrial Revolution.” • “The Information Age had arrived and the world would never be the same.” During the 1990s, the National Science Foundation says the number of users on the Internet grew to 248 million! 11 The 1990s were a time of growth and change Technologists like us relied less on the phone and more on the Internet in the 90s – And we designed our domains to exploit it • Now there are billions of Internet users world wide • And our appliances are hooked into the Internet too Saw a story last week about a refrigerator that was hacked – It was sending out 100,000 messages 3 times a day – http://www.dailymail.co.uk/sciencetech/article-2541306/Cyber-criminals-hackREFRIGERATOR-send-malicious-emails-worlds-household-attack.html 12 Designed with slow bandwidth in mind Relatively slow Internet bandwidth gave us a habit of placing servers at remote locations – We would end up with servers that were called home for a small numbers of users Just one of the many reasons that today we look at consolidation for our server environments – We got bigger – With bandwidth that’s faster than ever • No need to have servers at remote locations when they can be reached quickly from just about anywhere We’re not saying that this is the new normal – There are plenty of places on earth where bandwidth is still scarce – And we can still design to them • We just need to optimize 13 Here are some real reasons behind domain consolidation These are directly from our customers’ mouths We want to save money and resources We want fewer servers and to exploit new faster networks We were just bought by another company We buy smaller companies every year and want dedicated processes to migrating to our messaging systems 14 Here are some real reasons behind domain consolidation These are directly from our customers’ mouths (cont’d) We’ve bought other companies that were running other brands of cloud based messaging and want to centralize on Notes We’ve moved mail to the cloud and want to consolidate what we’re using for applicaitons When Notes was deployed we created regional server farms that are no longer a required component We are still running IBM Domino R6.5.4 and wish to do a technology upgrade – Might as well consolidate the infrastructure while we’re at it 15 More reasons to consolidate from our customers Started out with multiple domains, but realized should only have one centralized domain – Earlier decisions about multiple domains were made without a real understanding of the Domino infrastructure requirements – Earlier decisions were made based on technology available, such as modems and slower networks We need a technology refresh – Just want to start fresh with a smaller number of new servers to replace aging ones – Establish best practice configuration 16 A common thread of complaints that make us consolidate There is just too much of everything – It is out of control There is too many of everything • Too many mail routes • Too many servers • Too many connection documents • Too many mail-in databases • Too many applications - We don’t even know if anyone is using these applications 17 What happened to the creators of this domain? Plus, the domain was designed by people who no longer work here – It’s difficult to find the causes of problems It’s time to straighten it up – Optimize the domain – Review the security – Reassess applications – It’s a fresh start, a new day 18 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 19 Overall do’s and don’ts with your migration Do the migration as quickly as possible – Being in a state where you are in the middle of the migration with your users are in the state of flux can be very risky Run a pilot with a small number of users if possible to ensure any techniques are going to work as expected – Make sure you understand the risks of having your users straggled between new domain and old domains Here are some of the issues you might experience during migration 20 A problem with old domain’s meeting invitations Here’s a good example of what can happen Here’s the type of problem you have with meetings during a migration You invite your good friends to a meeting in the future – Maybe two months from now – You constantly collaborate and arrange meetings like this with people from different legacy domains in your enterprise You invite people from another existing legacy domain, Funlab – Homer Simpson/Corp/FunLab@Funlab – Marge Simpson/Corp/FunLab@Funlab 21 Here’s the meeting invitation Note that the Domino domain name of each invited person has been appended to their abbreviated Notes name 22 Where’s the problem? The invitation you sent was for a meeting 2 months from now Two weeks after you sent it, Homer and Marge are migrated to the Domlab domain Their names change from this: – Homer Simpson/Corp/FunLab@Funlab – Marge Simpson/Corp/FunLab@Funlab To this: – Homer Simpson/Corp/FunLab@Domlab – Marge Simpson/Corp/FunLab@Domlab This change is not reflected in your calendar – It has not been updated and will not be updated, ever! – As far as your calendar goes Homer and Marge are still in the Funlab Domino domain 23 Oh! So maybe updates to the meeting won’t reach them? Updates for the meetings will probably still get to them You probably made some changes to person documents to make sure their old names with the FunLab domain are still recognized and mail can still route to them – We have code to help with that One of the problems will happen when you try to suggest new time for the meeting 24 Owner can’t find the info to suggest a new meeting time The owner of the meeting can’t see the FunLab users’ free time because they are no longer from the FunLab domain and the free time lookup can’t find them – Even if that domain still exists, Homer and Marge no longer live there • And it appears that their “info is restricted” – which usually generates a help desk call 25 Free time lookup doesn’t show the domain name for users But it is stored in the calendar document This is confusing for users since the names appear to be correct, but the names that are stored still have the old domain name 26 Invitee wants to suggest a new time but can’t do it To add to the frustration, even the invitee can’t see their own free time lookups – If you rarely collaborate with users in the other domains, this will be less of a problem 27 How do you fix this? Since calendar entries live in everyone’s calendar it is very difficult to modify existing calendar entries without completely cancelling and re-creating them Can you imagine how this affects repeating meetings that go way into the future? 28 Another possible issues during domain consolidation The “recent contacts” problem Recent contacts are accumulated automatically by the Notes client – It’s used to assist you in addressing and in meeting invitations The name you use in addresses will be stored in recent contacts The names will include the domain name of the contact Each time you create a new meeting or mail message you will be presented with recent contacts that have the domain name appended When the name and domain are correct, it works perfectly – When the domain is not correct, it still gets used • And thus all of that fun with meetings and free time lookups continue 29 How to fix the recent contacts issue You can just turn off recent contacts – Most of the time I like the way recent contacts handle giving me addresses • The problem is not exactly recent contacts themselves • It’s a fact that they don’t seem to refresh as often as they are supposed to from the legacy address books • The old domains remain in recent contacts It is possible to surgically remove them from the recent contacts view 30 Removing legacy domain entries from recent contacts You can prevent legacy domain entries from polluting your recent contacts by inserting a few parameters into the NOTES.INI file on the local client See this technote for the details – http://www-01.ibm.com/support/docview.wss?uid=swg21415228 If you needed to prevent the Domino domains FunLab and Simco out of your recent contacts you would use this parameter: – DPABRemoveRule=FunLab, Simco If you also want to also stop aggregating users from funlab.com and simco.com, use this: – DPABRemoveRuleSetting=1 This can be done using a policy with a desktop policy settings document 31 Desktop policy settings to control Recent Contacts In the desktop policy settings select custom settings – Then use edit list to add the parameters as shown below – You can still use recent contacts, but prevent the names from certain domains from being added 32 Contacts added to personal address book from Domino directory The issue of the retained legacy domains on person documents will also happen when users copy names from the Domino directory to their local Contacts, NAMES.NSF – You can socialize this issue with the right communications – Make sure the help desk knows how to handle this type of problem Sometimes it is simpler and more coherent to solve these issues socially than to come up with a complex technical solution – You can send a post open event to the user in a mail file, but really the user just has to know that they need to delete the users from their contacts and re-add them 33 Special cases Any user with initials after their name (VP, CIO, CEO) might need some hand holding or some automated method of fixing this 34 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 35 Cross certification needed, authority needed There will be a period of time when your domains have to work together – Regardless of whether you’re planning on decommissioning some of them Cross certification will be needed if your domains don’t run under the same certifier Create an administrator account that has manager access to all files in all domains – Turn on password checking on the servers – Turn on password checking for this powerful account • That will eliminate the risk of someone stealing this ID - Once you change the password on the ID, anyone who has an ID but doesn’t have the current password won’t be able to use it 36 Changing the building blocks of the domain Server changes Server changes – When the servers documents must be configured for a new domain • There is one place in the server document that holds the Notes domain name – The NOTES.INI parameter Domain= must be modified for the new domain That’s all there is to it! – Change those two things and bounce the server, the server is in the NewDomain 37 Here’s a simple agent that modifies the mail domain field Here’s an agent that uses simple formula actions and modifies the MailDomain field 38 Person documents Server documents are easy to change by editing a the server document Person documents require an agent because there are more person documents than server documents and there are more changes per document – Change the maildomain field to NewDomain And we like to add an alias to the fullname field that reflects the old domain of the user – If another user replies to an old email send by out this user the mail router still finds him/her 39 Adding the username with the old domain to fullname This is a little more complicated that just changing the domain name – It is done in LotusScript – Administrators who know Lotusscript can get more done in a shorter time Let’s say that the user is in the Notes domain called DOMLAB – And the user is to move to NewDomain The code looks to see if there is an alias in the FullName field in the person document – It would look for Andy Pedisich/technotics@domlab – If it’s not there it would append it to the text list • This code is in the package downloadable from http://www.andypedisich.com 40 Roaming user changes If the user is a roaming use, the RoamSrvr field must also be changed to represent the new server – This is also part of the agent that modifies the person document and is downloadable from Andy’s blog 41 Group Documents The MailDomain field must be changed to the NewDomain 42 Changing maildomain does not require LotusScript Changing the maildomain field in group documents uses the same formula code as changing the maildomain field in person documents 43 Mail-in databases Similar to person documents, there are many of these But there is not much to change Again, an agent using simple formula actions can easily modify the MailDomain field 44 Changing connection documents There are two domain fields in connection documents – SourceDomain – DestinationDomain Change the appropriate field to the new domain name – This will be probably be done manually since each connection document needs to be evaluated to see if changes are needed 45 Change both fields at once or create two agents You can change the connection documents using easy code like this – Or you can split the code into two agents so that you can change only the destination domain in one agent and the source domain in another 46 Domino directory objects you won’t have to modify The following document types do not contain a domain name – You don’t have to worry about these during a domain consolidation • Server configuration documents • Program documents • Adjacent domain documents • Foreign SMTP domain documents • Policy and policy settings documents • Web site documents 47 Don’t forget that the local NAMES.NSF needs work Location documents must be evaluated – The DOMAIN field must be changed in location docs where old domain name appears • Critical to correct functionality of calendaring and scheduling - Which uses the domain information in the location document We generally send the user a button with code – The user clicks the button and the changes are made – The agent logs the actions success or failures in a mail-in database so that progress can be tracked 48 Setting up for cross domain adminp requests If there are multiple domains being consolidated it is sometimes necessary create a special cross domain adminp request configuration in the admin4.nsf database Start by opening admin4.nsf and navigating to the Cross Domain, Configuration section 49 Add a configuration document Select inbound or outbound, depending on what domain you are configuring for Then select the actions you would like to be send across the domains 50 Cross domain requests (cont’d) Enter the server names that will be permitted to submit requests for Create Replica and Update Delegated User’s Mail File – Also indicate the domain of that server – The domain name must be listed in the “Domains to Submit Adminp requests to” field – These are case sensitive Then select the list of approved signers of those requests 51 Connection documents required for cross domain adminp You must create connection documents between the servers you are using for cross domain adminp requests Since these servers two or more different domains, they will not be in the same Named Notes Networks – Mail routing will not naturally occur with point to point routing – Connection documents must be created in each domain that point directly to the server receiving adminp mail in the other domain Connection documents are needed in situations where you don’t use cross domain adminp requests 52 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 53 Consolidation three different ways Let’s look at the rationale for two different methods of consolidation Fictitious Customer #1 has the following scenario – Many companies held by a parent company – Couple thousand users – many organizations cross certified – Environment – Domino Release 8.5.3 – 7 domains – pockets of Domino servers dispersed – distributed administration • US and UK take care of their own technology – Multilingual situation – South America, India, China, USA – Mail jail is used – If you’re over quota you can’t sent or receive mail – Not big on using policies They want one Notes domain and one Internet domain to help them manage their messaging/collaboration environment and to help brand their products 54 What’s the approach? Since they were basically happy with their infrastructure we see this one as a really simple domain consolidation to collapse the 7 domains into a single domain We would flip existing servers into the new domain during off hours and restart them in the new domain Here’s one way to do that 55 Build a new domain – The Big Bang Theory First step is to build a new domain on a server we rent from the cloud – The cloud is the perfect place to build a new server rather than in our office – Because we don’t need to make an permanent investment in hardware • This isn’t a requirement, it’s just something we find convenient We create a new domain with the name they want to use and copy the contents of the other Domino directories into it – Then we use agents to modify the contents of person document, server documents, connection documents, mail-in databases and all the others we spoke about in the previous part of this session 56 Did you hear that? Build a new domain! How many times have you had the opportunity to build a new domain? – This is a “greenfield” operation – Build it as a “perfect” domain Your chance to implement a new domain according to best practices – ID Vault – DAOS – Tight security 57 EVENTS4.NSF Is a Key File in the new domain We want to monitor all of our servers and be notified when certain conditions occur – We specify what we want to watch for • Low disk space • Common error messages about security – illegal access to files • Corruption in databases • Warning messages about messages not being transferred • Server is down And how we want to be notified – Pager – Email – Phone call We will use the EVENTS4 database to configure all of this 58 Same replica on first server and all servers added To be effective, the EVENTS4 database should be the same replica on every server in your domain – I have found many, many cases where EVENTS4 was not the same replica everywhere in the domain • That ruins the monitoring architecture - The monitoring configuration can’t replicate to some servers Servers will be isolated – Your environment won’t be consistently handling and reporting problems It’s not only the EVENTS4.NSF, it’s other system databases you need to treat this way 59 We Know What the Replica ID Should Be for EVENTS4 The replica ID of system databases like EVENTS4 are derived from replica ID of directory Database NAMES.NSF CATALOG.NSF EVENTS4.NSF ADMIN4.NSF DDM.NSF Replica ID 852564AC:004EBCCF 852564AC:014EBCCF 852564AC:024EBCCF 852564AC:034EBCCF 852564AC:0A4EBCCF – Notice that the first two numbers after the colon for the EVENTS4.NSF replica are 02 • Determine your address book’s replica ID, and you’ll know the replica ID of EVENTS4 and other system databases 60 Verify EVENTS4 You must verify that every server has the same replica of EVENTS4 – You can find this info in the catalog, if your catalog architecture is getting file info from all servers • Or you need to go to every server and open EVENTS4 - It’s vital that you validate EVENTS4 If it is not right on a server, then you must down the server, delete EVENTS4, and restart your server – The correct EVENTS4 will be re-created automatically – This works with the other databases as well 61 Errors You Might See If DDM.NSF Is Not Right If there is a DDM.NSF on every server but they aren’t all the same replica ID, you’ll see the following error on the console every couple of minutes: – Unable to replicate with server Server2: None of the selected databases have a replica on the server • You’ll get this even if you have a much longer replication interval scheduled To fix problems related to EVENTS4.NSF and DDM.NSF replica IDs, you must delete the bad databases and restart the server 62 This Configuration Must Be Correct Make sure that EVENTS4.NSF and DDM.NSF have the correct replica ID throughout the domain by opening each and putting it on your desktop – The icons should stack, showing they are the same replica • Check the replica ID by looking at the properties of the databases 63 Want to Add Every EVENTS4.NSF to Your Desktop? Add this code to a button on your toolbar – This is courtesy of Thomas Bahn • www.assono.de/blog - It’s a great tool that I use almost every day _names := @Subset(@MailDbName; 1) : "names.nsf"; _servers := @PickList([Custom]; _names; "Servers"; "Select servers"; "Select servers to add database from"; 3); _db := @Prompt([OkCancelEdit]; "Enter database"; "Enter the file name and path of the database to add."; "log.nsf"); @For( n := 1; n <= @Elements(_servers); n := n + 1; @Command([AddDatabase]; _servers[n] : _db) ) 64 Add Databases to the Desktop This code will prompt you to pick the servers that have the database you want on your desktop – Then it will prompt for the name of the database • And open it on all the servers you’ve selected Use it to make sure all the EVENTS4.NSF are the same replica in your domain 65 How we flip a server into a new domain Create replicas of the system databases from the new domain to the server we want to flip And rename as you replicate them to existing servers 66 Normal file name Migration file name names.nsf namesnew.nsf ddm.nsf ddmnew.nsf events4.nsf events4new.nsf catalog.nsf catalognew.nsf activity.nsf activitynew.nsf admin4.nsf admin4new.nsf How we flip a server into a new domain (cont‘d) Shut down server Rename the following system files 67 Server to be migrated Archive file name names.nsf namesold.nsf ddm.nsf ddmold.nsf events4.nsf events4old.nsf catalog.nsf catalogold.nsf activity.nsf activityold.nsf admin4.nsf admin4old.nsf Some more renames while server is down Rename the following system files Lower case letters are extremely important with some OS – Unix, AIX, Linux 68 New files you’ve replicated New system file name namesnew.nsf names.nsf ddmnew.nsf ddm.nsf events4new.nsf events4.nsf catalognew.nsf catalog.nsf activitynew.nsf activity.nsf admin4new.nsf admin4.nsf Some additional adjustments Change the NOTES.INI file parameter DOMAIN= to the new domain name Restart the server in the new domain – Replicate with the first server you built – Perhaps make this new server your administration server for the address book Take these steps with each server in the legacy domains until all are running in the single domain Restart the server in the new domain Take these steps with each server in the legacy domains until all are running in the single domain 69 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 70 Fictitious customer #2 would like to start over Fictitious Customer #2 has the following scenario – Over 1,000 users – 3 Domino domains • Each with their own mail servers • Some with application and development servers • Administrators are at each site but have other jobs - don’t want to manage Domino • There are not a lot of cross domain meetings to worry about • They use managed replicas • They are huge on using policies 71 It’s a brand new day! Customer #2 wants to: – Move to a centralized server farm that uses Domino clusters and load balancing – Move all users and everything in the legacy domain to the new servers – Other objectives such as id vault implementation, DAOS, and some third party tools to make domain management easier And the kicker – they want to gradually move groups of people to the new domain – Offices move together – Regions move together – Functional groups move together 72 You will grow weary looking at spreadsheets full of users They will send you list of people – Alphabetical by first name or last name You will have to select those person documents for those users – Send them an email to deal with their location documents – Select them in their legacy domain • Modify them – Copy them into the new domain’s directory – Delete them from their legacy domain 73 An tedious yet effective migration method We call this method of migration – “Death by a thousand cuts” • Because it is the most arduous method for administrators - Lots of matching up lists - Making sure replicas are created - Counting people 74 So, what’s the approach After the centralized empty servers are stood up in a brand new domain – Start the work of creating replicas in the centralized server environment We use the Notes Administrator Client to select all the files we will create replicas of – Then use the Manage ACL to add the new server names and managing groups 75 Selecting the files that need a replicas in the new domain The files you select to create replicas of will depend a lot on your current infrastructure Fictitious Company #2 has multi-purpose servers – They contain • Mail files • Mail-in databases • Archives • Roaming files • And lots of applications 76 Selecting the files that need a replicas in the new domain It is a good time to determine if you need special purpose servers that might be new for you Starting over is a great time to re-evaluate your requirements – Mail servers – App servers – Gateways – LDAP – Instant messaging and meeting – Mobile device 77 Selecting the files that need a replicas in the new domain Review these files to be certain they are still being used – Each mail file should have a person document associated with it – Mail-in databases should be represented by mail-in database documents – Archived should be linked to mail files, and they should still be needed – Work with Human Resources to keep your directory from filling with people who no longer work there • We’ll talk about applications later 78 Modify ACLs on all databases to be moved Add the new mail servers to the ACL – Since there are 3 domains consolidating we ensure that the admins of the central domain have manager access to all files by adding LocalDomainAdmins • Or whatever you might call your administrator group 79 Ensuring the correct advanced database properties apply Using the Notes Administrator client, we select all the databases on a legacy server and apply these settings – Replicate unread marks, all servers • Users like their unread marks – Use LZ1 compression to ensure attachments are all compressed the same way so that DAOS is efficient – Use DAOS – Compress database design – Compress document data By doing this in advance we won’t need to compact the databases to use DAOS 80 Use the Create Replica tool Using the Notes Administrator client, we create replicas from the existing domain – To each of our new centralized mail servers and their clustered partners – Each file we create a replica for generates 2 create accelerated replica requests • We generally select them in groups of 25 to 50 during the day, 100 at night Be sure to check the “Exchange Unread marks on replication – Take the opportunity to create full text indexes if needed 81 Change database administration server configuration Use the Notes Administration tool to modify ACLs and most importantly – Change the administration server for all databases • Pass control of renames and deletions to the new domain 82 We throttle Create Replica adminp requests We generally do less creating of replicas during the day – The admin4.nsf file gets so filled with requests – It delays other types of adminp task requests that should have more priority • Such as password changes and renames • User deletions • Mail file delegations 83 We throttle Create Replica adminp requests This replication method must be planned enough in advance prior to domain consolidation – Also make sure that new files that are added between the creation of the first set of replicas are created on the new clustered servers There are other methods of getting the replicas over to the centralized clustered servers – Restore a backup onto the new servers – Shut down the servers and using OS level copies to get the files to the new servers • These methods were not appropriate for fictitious Company #2 84 Policy accord necessary and certs must be in place Since we are folding multiple domains into a single domain, the administrators of the 3 legacy domains have to agree on policy settings they should all use The 3 domains use separate certifiers and organizations – The recertification of uses to a single certifier is another project entirely Policy settings documents need to be created – Policy documents for each of the 3 organizations must be created – Certifiers from the 3 legacy domain directories must be copied to the new domain directory – You can’t just copy your existing policies and policy documents – They won’t link up 85 Staging and backout directory We’ve created some views and agents to help us control the migration process – We’ve added them to the staging directory, the backout directory, and to the production Domino directories in the legacy domains – They are used as a holding area for all documents in the legacy domains and in the staging directory 86 Review groups before the move There are some group names that are common to most Domino domains – LocalDomainAdmins – LocalDomainServers – OtherDomainServers • These groups obviously should not be copied to the new domain In most cases we do not find duplicate group names – In large domain an agent should be used to check for duplicate names in large – In smaller domains the legacy and new domain directories can be placed in different windows and visually compared 87 Communications with the user to be migrated We also must create an email to the users that are being migrated – This will include a button to change their location documents • More on the message in a second 88 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 89 About this part of the following migration example Our example will be using the following servers and domains – The RED domain uses the server Mail01/Red – The BLUE domain uses the server NewMail2/Servers/Blue Our goal will be to collapse the two domains together into the Blue domain 90 Ready to migrate to the new servers We create some files and agents to help us To assist us in consolidating fictitious Company #2 we created two Directory files – A “Staging Domino Directory” was created from the server based public name and address book template pubnames.ntf – A “Backout Domino Directory” was also created from the same template – Both are customized with added agents and views The Staging directory is used to hold copies of person, group, and other documents that can then be modified using agents before being copied into the directory of the new domain The staging directory has several simple formula agents and several Lotusscript agents 91 Agents in the staging and production directory Change Person doc – staging directory only Change Mail-in doc – staging directory only Change MigStat to Prep – Staging and production directory Change MigStat to In Progress – Staging and production directory Change MigStat to Completed – Staging and production directory 92 Change person document, mail-in database document Change person document is a LotusScript agent makes critical changes to the person docs in the staging directory – It is found only in the staging directory – It is used to prepare the old legacy person documents to be pasted into the new domain Change Mail-in database document is a simple formula agent – It adjusts mail-in documents to be pasted into the new domain 93 Changing MigStat – the migration status for person docs The migration status agents add two fields to the person documents – In the production directory – In the staging directory The two fields are: – MigStat which is the migration status – MigDate which is the date the agent was executed As you will see in a few moments, this gives us a clear picture of where the users are in the migration process – Give managers access to the staging directory if you want clear communications 94 New view in production and legacy directories A view has been added to the staging directory and to the production directory – People\by migration status email date Fields are going to be added to the person documents – This view will help us track those fields – Let us know where people are in the migration process 95 Why track the movers? It’s important to keep track of the users that are being migrated as they step through the process – There will be dozens, perhaps hundreds or thousands of users at different stages • We want to be able to track them Fictitious Customer #2 always communicates with their users to prepare them for migration and sends us lists of users they would like migrated We move them through the following 7 step process 96 The back out directory The Backout directory is used to hold copies of those documents that are not modified – If we need to back anyone out of the migration we can put back these documents if others were deleted from the legacy domains Speaking of safe, always make a safe copy of your Domino directories before changing anything 97 Overview of the 7 step process Step 1: Mark the users person documents in the legacy domain directory Step 2: Copy the person documents into the staging directory and the backout directory Step 3: Run agent on their person documents in staging directory Step 4: Send users a button that changes location documents Step 5: Copy their person documents from staging directory to directory in their new domain Step 6: Delete their person documents from the legacy domain Step 7: Make sure all directory catalogs are updated …and here come the details 98 Step 1: Mark user’s person docs in Red legacy domain Select those user’s person documents in the legacy Domino Red domain directory Run agent on them to mark them as being in the “prep” status (Prepared to migrate) There is a new view copied from the Staging directory on the legacy domino where the status is evident and it shows the status and dates the status has changed This makes it easier to follow users through the process of migrating 99 Step 2: Copy person docs into staging/backout directories Copy the person documents into the staging directory and the backout directory The backout directory will hold a copy of the original document which will never be changed 100 Paste the person docs into the staging directory The person docs pasted into the staging directory will be displayed in the same view as the one in the production directory 101 A you progress you’ll be able to track users This method of marking the person documents has value – You’ll be able to easily identify the users that are in prep, in progress or completed – It will be easy to determine what happens next to each user 102 Possible http password issue when restoring a backout The user might change their http password after they are migrated If the users person document is subsequently copied from the backout and pasted into the Red legacy domain – And removed from the new Blue domain – They won’t have the new password in their person document – Until the HTTP password was re-entered into the person doc This never happened to me in the real world, but it is possible – Your help desk should be made aware of this scenario 103 Step 3: Run agent on person docs in staging directory This agent runs on selected person documents in the staging directory Modifies the mailserver field, replacing the contents with the name of the new server Modifies the maildomain field, replacing contents with new mail domain name If the user is a roaming user, it replaces the RoamSrvr (Roaming server) field with the name of the new mail server or the name of the new roaming server – In most situations the roaming server is the same as the mail server It adds an alias to the FullName field in the person document – This is FirstName LastName@OldDomain • This will cover use cases where a user is replying to an email where the old Notes domain was being used • Mail will still route to the person in that scenario 104 Running the agent on docs in the staging directory All the changes are made – No typos – No fat finger missed keystrokes because it is all automated 105 Example of person doc before the agent runs This is a person doc in the staging directory prior to the agent running Note that there is one single entry in the fullname field – Domain name is Red – Mail server is Mail01/Red 106 Example of person doc after the agent runs This is a person doc in the staging directory after to the agent running Note that there is a second entry in the fullname field – David Lowery/red – Domain name is Blue – Mail server is NewMail2/servers/blue • The person doc is ready to be copied and pasted into the Blue domain directory 107 Step 4: Send users button that changes location docs The button code is signed by a functional account (not a human) who is listed on the Administrative Execution Control List (ECL) and is allowed to make changes to the workstation without causing an ECL alert to display The code finds all the location documents that contain the legacy domain name and builds a collection The code changes the DOMAIN field to the new domain in all the docs belonging to the new domain It places the address book and the person’s mail file on their desktop Sends an email to a tracking database where it posts the results of the run and if there were any errors 108 Button will be sent in an email The button should be sent only to those users who are being migrated right now – This should be mailed prior copying and pasting the documents from the staging directory to the Blue domain directory It’s pretty easy to address the email message First, select the users in the view in the staging directory and copy them as a table 109 Paste into spreadsheet – copy email address Paste into a spreadsheet Copy out the email addresses 110 Create an email Create an email and paste what copied from the spreadsheet into the To field – Addressing an email to 7 or 70, or 150 users takes about the same amount of time 111 Helpful tips regarding communications with users Be sure to ramp up the intensity of the message that goes out to the users – Mark it as “High importance” – Include a courtesy copy (Cc: ) • To the manager of the user • Or some other person the user might fear Clearly state in the subject that immediate action is required – That will encourage the user to act now 112 Helpful tips regarding communications with users (cont’d) Mention the advantages of the moving to the new servers Tell the users how long it will take when they click the button Provide instructions for users that have special situations This includes – Users with multiple workstations with Notes installs – Traveler – iNotes Provide web links with help 113 Helpful tips regarding communications with users (cont’d) Don’t forget to include the help desk phone number – This is a very important touch – The user might not know how to create a bookmark Even though you’ve tested the button there can still be situations you haven’t counted on 114 Step 5: Copy person docs from staging to new domain Mark the users as completed using the agent Copy their person documents from staging directory to directory in their new domain Replicate the Domino directory in the new domain to all servers to ensure that the new additions are on all servers – This ensures that mail routing to the users in the new domain will be correct 115 Step 6: Delete person docs from the legacy domain Delete their person documents from the legacy domain – Sometimes we check the mail in database that is the log for the button action being clicked before copying to the new domain and deleting from the legacy domain – If the mail in database has a successful log document from the user clicking, they no longer need the person document in the legacy domain • Replicate legacy address books for all domains to ensure user has been removed You can use the Delete Person button if you want to also remove their mail file – Generally customers like to leave the mail file on the legacy server for a week or longer – When everything is stable, mail files on the old servers can be removed Mail files can also be deleted using the Notes Administrator Client 116 Step 7: Make sure all directory catalogs are updated Make sure all directory catalogs are updated with the new domain and servers for the migrated users Any extended directory catalogs or compressed directory catalogs should be refreshed or rebuilt – Generally we shorten the refresh interval setting for directory catalogs when we migrate users to the new domain At this point you can mark the user as having completed in the staging directory using the agent “Change Mig Status to Completed” 117 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 118 Mail-ins are easy to migrate compared to people Copy the mail-in database documents into the staging directory Run the agent “Migrate mail-ins to new server” This agent will modify the maildomain and mailserver fields to their new domain and server Copy the changed documents to the Domino directory in the new domain Remove the original mail-in database documents from the legacy domain’s directory Replicate changes to all servers Delete the mail-in databases from the legacy domain using the Notes Admin client 119 Be sure to leave a Notes reference file behind When using the Notes Admin client to delete the mail-in database use the Delete button provided in the navigator on the right of the screen Using this button will give you the option of leaving a marker that will allow clients to update their bookmarks and icons to this new location – And it will remove the icons and bookmarks from the file that was deleted 120 Application migration There are lots of issues that come with migrating applications If the application is a web app accessed using an Internet browser and you move it to a new server users will need to update their bookmarks If the application contains agents or design elements that refer to a particular server name that is hard-coded, you might find yourself a designer to help you If the application references other databases, make sure the other databases are still available in the new domain – Some changes might have to occur in the application to keep it viable How do you know what applications are actually used 121 Using activity logging to determine if an app is being used Activity Logging servers account for their time, recording user activity by person, database, and access protocol. Use server configuration documents to turn on Activity logging Marking your prime shift helps you in interpreting results that look at that time period 122 Turn on Activity trends Also turn on activity trends which will run every day you specify at 3:23 AM by default and collect information from the LOG.NSF and Activity logging stores information in the Domino log – Expect logs to grow much larger 123 Results of activity logging Activity logging produces a generous amount of raw data and calculates growth rates for storage It also tracks user activity and database activity But what best suits our purposes during a consolidation is the applications that are not very active 124 Inactivity of databases helps you to eliminate applications This view sorts by date of last activity It makes it easy to see apps that haven’t been used in a while – Keep in mind that an app might only be used a few times a year by the board of directors • That would mean it’s still important to retain despite it’s low usage 125 Plus you can see trending details about the databases Use the Open Database Trends button to see the details about the database – Check out how quickly it’s growing and get a usage summary • This will definitely help you in determining apps to migrate 126 Agents and consolidation Agents can be quite problematic Agents are generally set to run on a particular server – Analyze all of your agents in applications – Change the name of the server the agents are supposed to run on Improperly written agents might also have hard coded server references in them – That’s not going to work if the server that’s hard coded is a legacy server that you will be decommissioning – You must search out that code and correct it There are third party tools to help you do this • Check out the vendors in the Solutions Showcase at Connect 2014 127 Agent signers must have rights The agents were probably signed by an account in the legacy domain – The agent signers must be an account in your new domain – Do the research on this in the early stages of the migration project • Or you’ll be forces to deal with it at the end • Domino Domain Monitoring can help you track if this is happening Moving from a non-clustered environment to a clustered one can be a problem when agents are set to “run on any server” – In a cluster, agents configured that way will run on the replicas on both servers in the cluster – If two agents process and change all the documents in the database it will create a ton of replication conflicts 128 Clusters can be a pain Moving from a non-clustered environment to a clustered one can be a problem when agents are set to “run on any server” – Agents configured that way will run on the replicas on both servers in the cluster – If two agents process and change all the documents in the database it will create a ton of replication conflicts Make sure you include an analysis of where agents are configured to run 129 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 130 Last server standing Throughout the migration process we’ve been careful about mail routing in terms of a particular use case – Users are going to reply to an email that has the legacy domain address – Frank Fish/red@Red We have provided an alias so that mail can still reach that person – The email that is sent is routed back to Mail01/red which is in the Red domain When it arrives there the router looks at it and sees that the person is actually in the Blue domain and routes it back to Mail2/servers/red 131 What happens when the legacy domain is removed Unfortunately when you remove the Mail1/red server and the Red domain is gone mail routing breaks In some cases we’ve seen very small servers left in the legacy domains with a directory from the new domain as a secondary directory – Mail is still routed to the old domain – Everything still works • Nobody really likes that solution 132 A better solution We need a configuration where mail sent to Frank Fish/red@Red is routed to Frank Fish/red@Blue One rather simple way to do that is to create a foreign domain document And a mail.box for the legacy domain 133 Creating a mail.box for the legacy domain’s messages Create a new application from the Mail Router Mailbox template – Title it “Red Domain’s Mail” – Name the file redmail.box Then create a Foreign Domain Document in the Domino directory 134 Creating a foreign domain document Open the Domino directory – It’s configures quickly if you open it on the server that will contain the Blue domain’s foreign domain mail.box Navigate to the Messaging\Domains section Click the Add Domain button 135 Creating a foreign domain document (cont’d) Create a new Foreign Domain document – Use the domain type of Foreign Domain – For the foreign domain name use Red 136 Switch to Mail Information tab For the Gateway Server name, use the name of the server where you created redmail.box For the Gateway mail file name enter redmail.box 137 Go to the server console Go to the console on the server and enter this command: – Tell router update config • Or wait 15 minutes and the router will update it’s configuration automatically From now on mail sent to Frank Fish/red@Red will be sent to the redmail.box file And it will wait there for an agent to run that will remove @Red from each recipient’s address and replace it with @blue Frank Fish/red@blue will then be sent back into the system and the mail will be properly routed 138 What we’ll cover… Determining why you want to consolidate domains Why you have to move fast during your domain consolidation Understanding the basic elements of Domain consolidation Flipping servers into a new consolidated domain Preparing to move people, groups, Mail-in DBs, and other objects Migrating the users to the new domain Migrating mail-in databases, applications and agents Keeping mail routing after decommissioning legacy domains Finishing up with random thoughts, tips, and risks 139 Something to remember when combining domains When combining domains, remember that policy settings are super important Imagine collapsing 3 domains into a single domain – And those three domains had their own policy settings • How would you set security and desktop settings across those 3 domains If the 3 domains were using different organization or OU levels it would be easy to keep the same settings – But do you really want to do that with your new centralized domains Getting agreement on settings can be difficult – But in the end everything will be easier to manage 140 It’s a matter of case And by that I mean uppercase and lowercase If you’re moving to another operating systems from Wintel boxes, remember that some operating systems are case sensitive in respect to file names – Be careful when making replicas of files As a rule of thumb make all replicas in lower case – The Notes spell checker likes to make the initial character a capital letter – Mail file locations become Mail\apedisic.nsf • The location will have to match what’s in the Domino directory 141 Do your users want full text indexes? A user can search their mail file using the Search This View unless it’s been specifically turned off by the advanced properties in the database – If you’re making new replicas of mail files make those full text indexes when you create the replica 142 Create full text indexes when you make the replica It’s pretty simple to create the full text index when you create the replica – Then it’s done and no need to back track to complete the task 143 Creating files for new servers The Create Replica adminp request can be slow when moving several thousand files – Consider shutting down both servers – Do an OS level copy over a weekend Or use backup tapes to restore the data directory to new hardware Or create the files on a SAN snapshot 144 Combining domains – consider creating directory catalog Mobile and desktop users will have faster typeaheads if you provide them with an Extended Directory Catalog Create a directory catalog using the pubnames.ntf template Control the aggregation of the people in the directory catalog in the extended directory catalog’s configuration document 145 Control which server aggregates EDC in the server doc Use the Server Tasks/Directory Cataloger task to control which server does the aggregation of the address books – And how often that occurs 146 That’s just about it You now know a lot of what we know about domain consolidation Take it with you, implement, and enjoy …And get it over with already! 147 Resources Frequently Asked Questions - The Administration Process (AdminP) – http://www-01.ibm.com/support/docview.wss?uid=swg21212760 Domino Server Consolidation: Best Practices to Get the Most Out of Your Domino Infrastructure – http://www.redbooks.ibm.com/abstracts/redp4181.html Merging Domino domains into one – http://www.ibm.com/developerworks/lotus/library/lsMerging_Domino_domains/index.html?S_TACT=105AGX13&S_CMP=EDU IBM Lotus Notes 8 Recent Contacts and Type-ahead features: FAQs – http://www-10.lotus.com/ldd/dominowiki.nsf/dx/ibm-lotus-notes-8-recent-contacts-andtype-ahead-features-faqs 148 Wrapping it up… things to take home You may not have been around when your domains were created, but you can still make it a jewel with consolidation The most risky time during migration is when you haven't completed it Every administrator should have some skills writing LotusScript and formula agents Using a staging and backout directory during user migrations reduces risk and gives you lots of info about the status of your moving users Events4.nsf and other system databases should have the same replica ID on every server in a domain When migrating lots of users the more you prepare with automation the more relaxed you can be when it's actually happening Do your homework with agents when migrating or you might find processes stop working or lots of replica conflicts being created 149 Access Connect Online to complete your session surveys using any: – Web or mobile browser – Connect Online kiosk onsite Find us on http://www.technotics.com And at: – http://www.andypedisich.com – Andyp@technotics.com – Rob@technotics.com 150 Acknowledgements and Disclaimers Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. © Copyright IBM Corporation 2014. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM, the IBM logo, ibm.com, IBM IBM® Domino® Server, and IBM Notes® are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Other company, product, or service names may be trademarks or service marks of others. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. 151