NiCE WhitePapers
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 1 von 17
© NiCE GmbH

NiCE WhitePapers
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 2 von 17
© NiCE GmbH

NiCE WhitePapers
IT Challenges
The IT staff faces three significant challenges: managing a more and more complex environment, working efficiently with limited resources, and diagnosing and addressing problems in real-time.
Complexity
In today’s interconnected IT environments, the underlying infrastructure is very complex and involves a wide range of servers, databases, and operating systems.
They all resemble business critical functions.
Limited Resources
IT is expected to keep a close watch over these business-critical applications, infrastructures, and third-party service dependencies — and to be ready to resolve issues before they impact the end user. The reality is a bit different: IT people live in a ―do more with less‖ world where inadequate time, resources, and tools often lead to a continuous state of firefighting. As such, proactive monitoring — and the time, effort, and expense it requires to achieve this — may seem out of reach to many IT departments.
Diagnosing and Addressing Problems in Real-Time
This still seems to be an unreachable goal. According to an IT Management Solution
Study (March 2007, Ziff Davis) 60 percent of the service outage problems require the involvement of up to ten different people within the organization. 80 percent of the time required to fix a problem is spent on addressing the problem itself.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 3 von 17
© NiCE GmbH

NiCE WhitePapers
When using Lotus Domino as your backbone for both internal and external communications as well as being the basis for lots of customized applications, it is most critical to ensure that all of those applications work in perfect harmony to provide a good user experience. Therefore it is most important that the server – being the fundament for all these various application services that run main parts of your business critical communication – is in best shape regarding availability and performance.
Performance and availability may be interpreted in many different ways. To one company it may simply mean that the services are available 24x7. To others it may mean that certain transactions need to be faultlessly completed within milliseconds.
Starting with these two topics: ―what means performance to you‖ and ―which business critical Lotus Domino transactions rely on the health of my Lotus Domino Server‖, leads to a set of questions that need to be matched when Lotus Domino monitoring is considered.
 How do I know which business critical services rely on the Domino server
 How do I categorize them, once found
 Which details do I need to monitor
 How easily can I scale from high-level monitoring to in-depth monitoring
 How easily can I adjust monitoring features, in order to react to business changes
 How can I co-relate Domino events with other business resources so I understand the full impact of the event
 How is the information presented to me
 Can I split monitoring tasks within the IT staff, so that the most advanced staff is not flooded with low level events
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 4 von 17
© NiCE GmbH

NiCE WhitePapers
Monitoring Solutions are an optimal way to address these needs. However, there are still plenty of things to consider in choosing the right monitoring solution.
The best stand-alone monitoring solution may be worthless if there is no way to correlate the data gathered with other business sources in an appropriate manner.
Isolated problem statements will not show how they are caused or how they affect other business services. This leads to a silo state that will not show the complete picture and is therefore powerless when interconnected problems need to be solved.
Even worse, when reacting to isolated problem views, this may cause a whole cascade of other events, which in turn - simply by the meaning from what change they derive – may not be interpreted at once. Such an unintended improvement for the worse can seriously harm your business.
What a monitoring solution should provide:
 It must integrate with the overlaying IT Management Solution, so the data gathered can be correlated with other monitoring sources and business processes.
 The overlaying Management Solution must provide ways to aggregate and visualize Lotus and Domino (Lotus Notes) service driven business processes.
 It must provide a good set of predefined monitoring conditions so you can start using the solution right away, adjusting it to individual needs when they arise.
 It needs to be easily adaptable to different and changing business needs.
 The choice, which granularity you need to have insight to at what point in time, is of great importance. So, the monitoring solution must be able to scan high-level issues - such as overall availability as well as response times and key performance indicators like dead mail message - which must not violate Service Level
Agreements (SLA).
 The given advice created from event correlation must be in the same
―IT language‖ as all other advices.
 Event messages need to be sent out to different administrator groups and roles depending on their expertise.
 Reporting for various timeframes and business levels is important to prove that
Service Levels have been matched as well as to provide reliable data for future planning.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 5 von 17
© NiCE GmbH

NiCE WhitePapers
Considering all these challenges when looking for the right monitoring solution, you will need to check the following criteria:
 A mature, high-end monitoring solution
 Designed to fit into the Management Solution in place
 Comes along with a comprehensive pre-set of monitoring metrics
 Can be easily adjusted to your individual needs
 Easy to understand by your staff
 Gives you the flexibility to handle different monitoring levels easily
 Sends out messages to different user groups
 Correlates events a) within the Domino environment itself and b) within business services based on Domino services
 Creates various reports on availability and performance
 Provides manufacturer commitment to support new releases of Lotus Domino and monitoring trends
The Domino Management Pack is such a mature, high-end monitoring solution that neatly integrates into one of the top IT Business Management Solutions – the Microsoft
Operations Manager.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 6 von 17
© NiCE GmbH

NiCE WhitePapers
Lotus Domino is an IBM server product that provides enterprise-wide e-mail, collaboration capabilities, and custom application platform. It can be used as an application server for Lotus Notes applications and/or as a web server.
As outlined before, good performance is essential to each business. It’s even a key topic when it comes to communications.
If the basis of the communication infrastructure is ailing, this will be infectious to all layers of your business. Symptoms may be delayed or even missing information, which in turn will slow down or even kill your projects and the critical ones will be the first to suffer. This in turn will lead to unpredictable timely efforts, not just to reactivate the project flow, but also to explain to the project staff, why things are delayed or went wrong. This may be a particularly sore point when you need to bring this up to the attention of your customers, on whom your business depends on for ordering your services or solutions.
Avoiding such scenarios upfront by laying out clever 101 fundamentals, helps you to free giant powers to drive your business with excellence, having the strength to overcome all challenges waiting along the way.
The goal of performance optimization is a stable, well tuned system that continuously meets and supports your business’ functional and service level agreements. It is a continuous, iterative process, diminishing event returns over time, empowering you to focus purely on your interaction with your customers.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 7 von 17
© NiCE GmbH

NiCE WhitePapers
Details of Lotus Domino Performance Tuning
Topics such as Topology Design, Capacity Management, System Management, and
Application Design Management influence the overall Domino architecture performance.
However, this needs to be seen within a greater picture, and can not be achieved by any
Monitoring Solution. This paper focuses in detail on the Lotus Domino Server tuning.
Performance tuning initiatives will take effect within the network when Domino
Servers are communicating and collaborating with other Domino servers.
As Domino is not designed as a stand-alone application, individual server performance tuning is most beneficial to the network as a whole.
Part A – Different kinds of Lotus Domino Servers and how to treat them
Performance Tuning means modifying a system to improve its efficiency at meeting specified functional service levels.
The systems in question can be
 a single application
 one Domino server
 a collection of servers participating in a common activity, such as mail routing or replication
 an entire Domino domain
Kinds of Lotus Domino Servers
There are a number of different roles that a Domino Server may perform.
Separating servers by their usage enables targeted and therefore most effective monitoring and performance tuning.
Mail servers
Mail servers are used in the Domino network to store and route mail messages among users, servers, and applications.
Database (or application) servers
Database servers are used to store the data for Domino applications.
Hub servers
Hub servers are designed to be used as dedicated servers to route mail and replicate databases to other servers within the network to minimize the performance impact on production servers.
Each hub server is required to support the storage of all common databases and be capable of replicating these databases to each additional server in a timely fashion, capable of routing mail and ensuring that messages are delivered within an acceptable time frame.
Web servers
Web servers provide data requested by Web browsers. This information can be normal
Web pages consisting of graphics files and text files stored on the server in HTML format, or in a Domino environment, pages built from Notes databases.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 8 von 17
© NiCE GmbH

NiCE WhitePapers
Web servers usually carry a lot of network traffic and can have a significant CPU load if complex Web applications are implemented.
So, your monitoring solution must provide pre-sets of event conditions for each Lotus
Domino server type. This enables you to
 automatically establish and understand the baselines (see page 10) for each dedicated server role
 automatically monitor each server type individually by its usage, to see, understand and fine-tune even the minor details. Even overseeing small issues may build up to cause a major disaster if not taken care of proactively.
Part B - Server Capacity and System Management
Managing server capacity according to your infrastructure also improves the overall infrastructure performance. Some examples are listed below:
 On a Domino mail server cluster, the average load of each server within the cluster is reduced by balancing the number of concurrent users between the cluster members.
 For Web servers in a cluster, load balancing the http port on a network layer improves access times and the application performance itself on each cluster partner.
 Exercising proper storage capacity management, including utilizing monitoring facilities on Domino servers, prevents server performance reduction or crashes caused by missing storage resources.
System Management includes Domino related system management (monitoring Domino server tasks and managing transaction logging) as well as management of third-party software components running per server instance, such as virus scanning and backup management.
Since each of these tasks demand system resources, they will impact the performance of the overall system. Therefore the configuration and schedule of these tasks must be adapted to the servers’ requirements (out of peak times), reduced to a minimum, centrally controlled to avoid redundancies and, where possible, monitored to detect scheduling conflicts.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 9 von 17
© NiCE GmbH

NiCE WhitePapers
Part C – Back to the whole picture of Monitoring the Lotus Domino Environment
Monitoring the Lotus Domino environments is a multi-faceted exercise and reaching optimal performance is not an easy goal, but a process of ongoing improvement. You need to know how factors such as memory, physical design and maintenance can affect performance.
To tune and maintain, you must start to continuously monitor the entire environment.
Solely focusing on tuning standard tasks and operations is a good start, but by far not enough.
Your Lotus Domino environment is like an ecosystem that is constantly changing by means of physical design, transactions, and memory – and it still needs to be balanced at any time to work with the best possible performance.
The next section will give you an insight into the key areas for Lotus Domino monitoring.
Part D – Key areas of Lotus Domino monitoring
Before you begin the iterative optimization process it is important to verify that you have a consistent and accurate data baseline to measure against, prior making changes to the system. Tools to achieve that are proper monitoring and management solutions.
Establishing the baseline
A baseline is established by monitoring and capturing/tracking performance characteristics of a system under certain conditions. Also, performance tuning and measurement require multiple baselines for comparison, for example, little or no load, normal peak load, and during a maintenance window.
This sounds like a
Sisyphean labor .
A monitoring solution with a comprehensive set of outof-the-box monitoring capabilities (metrics and thresholds set) is optimal to establish, verify and adjust such multiple baselines.
Domino specific performance tuning
As outlined above each Domino server should be tuned for its specific role in the infrastructure. Here's a short overview of the main areas, which you may dispatch with a proper monitoring solution, and therefore free valuable time.
Server tasks
Minimizing the number of server tasks on the server, their frequency and specific execution times will enable you to optimize performance of the Domino server.
You will need a monitoring solution to identify these tasks, separate them by their value, and help you to set thresholds for each of the tasks, which may not be violated to ensure constant high performance.
Replication
The database replicator (replica) on a server is responsible for handling scheduled replication requests, as configured in server connection documents. Replication
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 10 von 17
© NiCE GmbH

NiCE WhitePapers plays such a significant part in any multiple Domino server environment, that it is critical to optimize replication scheduling with the help of a monitoring solution.
Failure to do so may directly, and indirectly, affect performance and availability in many other areas within Domino.
HTTP
Domino’s HTTP task reserves significant resources as soon as it is started.
Correlate system performance events with HTTP performance events to avoid false alerts.
Update
One of the most common, preventable causes of poor response time and low performance of Domino servers is excessive and unnecessary activity of the
Domino Update task. You may track the Updates and then find ways to minimize them.
Directory Cataloger / Catalogs (Dircat)
A Directory Catalog is an optional directory database that aggregates information from multiple Domino Directories. Because Dircat is a CPU-intensive task, it is often beneficial to dedicate one server solely to Dircat processing. However, it’s highly important to monitor that specific server to make sure Dircat is always performing faultlessly.
Router
Domino mail servers use a MAIL.BOX database to hold messages that are in transit. Monitoring and also reporting on message delivery (Maximum delivery threads, Setting transfer limits, Maximum concurrent transfer threads, and
Maximum transfer threads) and health of the MAIL.BOX database is also highly recommended.
AdminP
The Administration Process (AdminP) automates routine administrative tasks, such as ID management, database moves, and so on. Some of these administration requests can require significant server resources.
Implementing a designated administration server for AdminP is useful and you may want to monitor this task.
Domino Memory Manager
On a typical Domino server, the vast majority of memory managed by Domino is shared. Furthermore, the majority of that shared memory is usually part of the
Unified Buffer Manager (UBM) , also known as the NSF Buffer Pool. The UBM may use 70-80% of the total memory allocated to Domino. For this reason, it is critical to effectively monitor and manage the UBM.
Database performance
Often, the most pervasive performance problems are due to poor database design. However, monitoring the database itself is most important to optimize performance on a given database.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 11 von 17
© NiCE GmbH

NiCE WhitePapers
In a well-tuned environment, indexing should be relatively transparent; views and full-text indexes should be updated quickly. In an environment experiencing performance problems, you may see the following symptoms:
 Long delays when opening a database, opening a view, switching views, scrolling through a view, or saving a document
 Delays when opening a document that uses lookups (In fact, you may see the form pause at certain points as it waits for these lookups to compute.)
 Performance problems throughout the working day, but excellent performance during the off-hours
 Out-of-date full-text indexes
To troubleshoot view indexing problems, start monitoring the log.nsf
Additions
Hard disk performance
CPU and memory are also important for stability, and often receive more attention when talking about hardware. Typically, servers install software and load and save data frequently. All of these operations require frequent reading / writing to the disk. Therefore, hard disk performance and reliability is a critical issue.
So it's important to also monitor CPU and memory usage.
Minimizing logging activity
“Do not log things that you never review!!!” is one of the most common statements.
However, you need to clearly understand what log file entries may be worth gold when it is interpreted by a smart monitoring solution reading from log files.
Domino clustering
A Domino cluster links multiple Domino servers so that they appear as one resource. If any given server is unavailable or too busy when the request arrives, the cluster transparently passes the request to a server that is capable of handling the work.
So, the monitoring solutions must support Domino Clustering.
The impact of third-party applications
Domino may not be the only application installed on a system, so it is important to understand the impact that other applications may have on a Domino server’s performance. Each process actively running on a machine is using some amount of memory, CPU, and so on. This is true whether the application is running in the foreground or in the background.
Antivirus and backup software tend to be two of the most common third-party applications sharing resources with Domino.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 12 von 17
© NiCE GmbH

NiCE WhitePapers
Conclusion
Good performance is truly a measure of customer satisfaction.
When you plan a performance tuning strategy, be sure to understand the business needs of the customer. Identify the most critical business applications and focus your tuning efforts on those first. And remember that tuning is not a ―set it and forget‖ job. Instead it is a process of ongoing improvement.
Constantly monitor critical applications for any type of environmental changes and adjust the applications accordingly.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 13 von 17
© NiCE GmbH

NiCE WhitePapers
Microsoft® System Center solutions help IT pros manage the physical and virtual information technology (IT) environments across data centers, client computers, and devices. Using these integrated and automated management solutions, IT organizations can be more productive service providers to their businesses.
System Center solutions capture and aggregate knowledge about the infrastructure, policies, processes, and best practices so that IT pros can optimize IT structures to reduce costs, improve application availability, and enhance service delivery.
(OpsMgr)
The Operations manager solves the end-to-end service management challenge.
System Center Operations Manager 2007 provides best of breed end-to-end service management for Microsoft Windows®, helping you increase efficiency and achieve greater control over your IT environment. Operations Manager enables you to focus your time and resources on initiatives that add value to the business.
Benefits
 Improves service levels and visibility with end-to-end service management of
Microsoft and custom line of business applications
 Enables Operations and IT Management teams to more easily identify and resolve issues affecting the health of distributed IT services
 Accelerates problem resolution through expert server, client and application knowledge for Windows
 Lowers client support costs by integrating and extending problem reporting and operations monitoring to client systems
 Reliably and securely scales management responsibility across teams and infrastructure with role-based, more scalable architecture
 Reduces IT management complexity by automating routine administration
The strength of a Management Pack is the expert knowledge it contains about the technology it is designed to manage. To extend Operations Manager to technologies in which Microsoft is not an expert, Microsoft relies on its partners. Microsoft partners have created Management Packs for various hardware platforms, non-Microsoft applications, and even non-Microsoft operating systems, such as Linux and UNIX.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 14 von 17
© NiCE GmbH

NiCE WhitePapers
The Domino Management Pack keeps an eye on important details
It monitors the critical functions in your Lotus Domino environment easily and precisely.
It keeps you informed about the status by configuring automatic checks on threshold settings and filtering critical messages. The Domino Management Pack triggers you with prompt and precise preventative measures.
The Domino Management Pack uses out-of-the-box intelligence
A core set of predefined rules and unit monitors provide you with the details you need.
Further additional monitors can be added with just a few mouse-clicks.
Fine tuning made easy with the Domino Management Pack
With the Domino Management Packs you set individual rules and monitors to monitor specific details and selected components. This information helps you to configure resources to suit your ever changing business needs and requirements day by day.
The Domino Management Pack looks behind the scenes
Coordinated and meaningful monitors add further value to the management of your
Lotus Domino based services. At any state you will be able to get information about even the smallest event in your Lotus Domino environment, events, which under certain circumstances could have far-reaching consequences.
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 15 von 17
© NiCE GmbH

NiCE WhitePapers
The development of application management solutions is a key business at NiCE for more than ten years. Bringing our in-depth knowledge of application management to the entire
Microsoft Enterprise Management market was just a logical step after a long-term experience in the UNIX world. Starting with the DB2 MP for the System Center
Operations Manager (SCOM) is a big achievement to roll out a long-standing monitoring solution into the SCOM world.
NiCE is a leading manufacturer that develops HP Certified Smart Plug-Ins for application management based on HP Business Technology Optimization (BTO) software, and also maintains some of the most advanced monitoring solutions for IBM DB2 databases,
PeopleSoft and Remedy, as well as Lotus Domino and BlackBerry Enterprise Servers.
NiCE software solutions are used worldwide by many major companies. The software solutions are available directly from NiCE, its US affiliate inConIT Corporation, and software partners like Hewlett Packard, Microsoft and their valued resellers.
Contact: NiCE GmbH, Liebigstrasse 9, 71229 Leonberg, Germany, Ms. Braun, Marketing
& Sales, TollFree Phone: +1-877-778-3730, www.nice.de
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 16 von 17
© NiCE GmbH

NiCE WhitePapers
Microsoft Homepage http://www.microsoft.com/en/us/default.aspx
Microsoft System Center Homepage http://www.microsoft.com/systemcenter/en/us/default.aspx
Microsoft System Center Operations Manager Homepage http://www.microsoft.com/systemcenter/en/us/operations-manager.aspx
Microsoft Operations Manager Management Packs Homepage http://www.microsoft.com/systemcenter/en/us/operations-manager/om-managementpacks.aspx
Microsoft PinPoint – Solution Portal http://pinpoint.microsoft.com/en-US/Default.aspx?rdRetry=1
IBM Lotus Domino http://www-01.ibm.com/software/lotus/products/domino/
NiCE Domino Management Pack http://www.nice.de/en/nice-products/domino-management-pack
Other NiCE Management Packs http://www.nice.de/en/nice-products/nice-db2-management-pack-for-microsoft-scom
Other NiCE Solutions http://www.nice.de/en/nice-products
NiCE Homepage http://www.nice.de/
Monitoring_DominoServers_WhitePaper_2010Q4 Seite 17 von 17
© NiCE GmbH