Front cover QuickPlace 3 for Sun Solaris 8 Features and benefits of running QuickPlace 3 on Solaris 8 Installation and setup hints and tips Performance tuning and testing William Tworek Michael Kapfer Jackson Williams ibm.com/redbooks Redpaper International Technical Support Organization Quickplace 3 for Sun Solaris 8 March 2003 Note: Before using this information and the product it supports, read the information in “Notices” on page v. First Edition (March 2003) This edition applies to IBM Lotus QuickPlace 3.0, and Sun Solaris 8. © Copyright International Business Machines Corporation 2003. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The team that wrote this Redpaper . . . . . . . . . . . . . . . . . . Become a published author . . . . . . . . . . . . . . . . . . . . . . . . Comments welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... ....... ....... ....... ...... ...... ...... ...... . . vii . . viii . . viii . . . ix Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 . . . . . . . . . . . . . 1 1.1 Introduction to QuickPlace 3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Key features and benefits of QuickPlace 3 . . . . . . . . . . . . . . . . . . . . . 2 1.1.2 QuickPlace 3 supported platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.3 The QuickPlace environment/architecture . . . . . . . . . . . . . . . . . . . . . 4 1.1.4 Key QuickPlace components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1.5 For more information on QuickPlace 3.0. . . . . . . . . . . . . . . . . . . . . . 10 1.2 Introduction to Sun Solaris 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.1 Highlights of Sun Solaris 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.2 System requirements for Sun Solaris 8. . . . . . . . . . . . . . . . . . . . . . . 11 1.2.3 For more information on Sun Solaris 8 . . . . . . . . . . . . . . . . . . . . . . . 11 1.3 What’s new in QuickPlace 3.0 for Solaris 8. . . . . . . . . . . . . . . . . . . . . . . . 12 Chapter 2. Installation hints and tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1 Preparing for an installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 Installing Domino . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3 Setting up the Domino server for QuickPlace usage. . . . . . . . . . . . . . . . . 17 2.4 Installing QuickPlace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris . . . . . . . . 21 3.1 Sun Solaris performance monitoring options . . . . . . . . . . . . . . . . . . . . . . 21 3.1.1 vmstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.1.2 mpstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1.3 iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.1.4 netstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Our test environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2.1 Test categories/classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2.2 Basic server environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.3 LDAP directory setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.4 Performance load strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2.5 Simulated/sample QuickPlaces used . . . . . . . . . . . . . . . . . . . . . . . . 29 © Copyright IBM Corp. 2003. All rights reserved. iii 3.3 Performance testing and tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3.1 Out-of-the-box performance numbers . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3.2 Tuning QuickPlace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3.3 Performance results after tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.4 Performance results compared . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 iv QuickPlace 3 for Sun Solaris 8 Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces. © Copyright IBM Corp. 2003. All rights reserved. v Trademarks The following terms are trademarks of the International Business Machines Corporation and/or Lotus Development Corporation in the United States, other countries, or both: ™ Redbooks(logo) AIX® Domino™ IBM eServer™ IBM® ™ iNotes™ iSeries™ LearningSpace® Lotus Notes® Lotus® Notes® OS/390® QuickPlace™ Redbooks™ Sametime® z/OS™ zSeries™ The following terms are trademarks of other companies: ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. Solaris and SunONE are registered trademarks of Sun Computer Corporation. Other company, product, and service names may be trademarks or service marks of others. vi QuickPlace 3 for Sun Solaris 8 Preface IBM® Lotus® QuickPlace™ 3 is the Web-based solution for creating team workspaces for collaboration. With QuickPlace, companies give users a way to securely work with colleagues, suppliers, partners, and customers. QuickPlace provides teams with workspaces where they can reach consensus through discussions, collaborate on documents, and coordinate plans, tasks and resources. The Solaris 8 Operating Environment is the foundation for Sun systems, from smaller departmental servers to massive, clustered servers with more than 100 CPUs. Designed for multiprocessing and 64-bit computing, Solaris software delivers a consistent computing environment that scales to handle heavy traffic, huge data sets, and CPU-intensive problems. By minimizing planned and unplanned downtime, reducing administration errors, and simplifying troubleshooting, the Solaris Operating Environment keeps mission-critical applications available, and ensures high-speed, reliable access to data. The Solaris 8 Operating Environment(OE) is scalable, available, manageable, and secure. This IBM Redpaper describes how to leverage the Solaris 8 environment as a stable and scalable platform for QuickPlace 3.0. Topics covered in this Redpaper are: New features included with QuickPlace 3.0 for Solaris 8 Basic installation and setup guidelines for QuickPlace 3.0 on Solaris 8 General Solaris performance monitoring techniques appropriate for QuickPlace 3.0 Performance tuning suggestions for optimizing a QuickPlace 3.0 install on Solaris 8 Results of basic performance benchmarks demonstrating the scalability of QuickPlace 3.0 on Solaris 8 Since QuickPlace 3.0 executes on top of a basic Domino™ 5.0.10 server, readers unfamiliar with the Domino platform may want to first review the IBM Redbook Lotus Domino R5 for Sun Solaris 8, SG24-5969. © Copyright IBM Corp. 2003. All rights reserved. vii The team that wrote this Redpaper This Redpaper was produced by a team of specialists from around the world working at the International Technical Support Organization, Cambridge Center. William Tworek is a Project Leader with the International Technical Support Organization, working out of Westford, Massachusetts. He manages projects which produce Redbooks™ and other material on various topics involving IBM and Lotus Software technologies. Prior to joining the ITSO, Bill was an IT Architect in the consulting industry, working first for Andersen Consulting/ Accenture, then for IBM Software Services for Lotus (ISSL). His areas of expertise include system integration and systems infrastructure design. Michael Kapfer is a Consulting IT Architect with ISSL. He spent a year in the ISSL eMeetings practice, specializing in Sametime® and QuickPlace technologies. Previously, he designed and implemented UNIX applications for various clients, specializing in security solutions. Prior to joining IBM in 1996, Michael wrote network and security applications for the Federal Government, working for a Lotus Business Partner. Jackson Williams is a Performance Specialist within the IBM Product Introduction Engineering (PIE) group, and specializes in UNIX analysis. He works with major software releases and assists the performance team in identifying areas where code can be improved. Prior to joining IPIE in 2000, Jack was a Senior Consultant within ISSL. The authors would also like to thank the following people for their contributions to this project: Catherine Stone, IBM Lotus Software Mike Dempsey, IBM Lotus Software Bob Sommers, Sun Microsystems Julie Pettibon, Sun Microsystems Become a published author Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You'll team with IBM technical professionals, Business Partners and/or customers. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability. viii QuickPlace 3 for Sun Solaris 8 Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html Comments welcome Your comments are important to us! We want our papers to be as helpful as possible. Send us your comments about this Redpaper or other Redbooks in one of the following ways: Use the online Contact us form found at: ibm.com/redbooks Send your comments in an Internet note to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept HYJ Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 Preface ix x QuickPlace 3 for Sun Solaris 8 1 Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 This chapter introduces the IBM Lotus QuickPlace 3.0 and Sun Solaris 8 technologies, including a high-level description of the QuickPlace architecture. It also includes an overview of those features new to QuickPlace 3.0 on the Solaris platform. Those already familiar with both QuickPlace and the Solaris platform may want to skip directly to 1.3, “What’s new in QuickPlace 3.0 for Solaris 8” on page 12. 1.1 Introduction to QuickPlace 3.0 IBM Lotus QuickPlace is a Web-based collaborative team workspace. QuickPlace integrates an enterprise's existing applications—such as Lotus Notes® 6 and Notes R5, Lotus Sametime, and Microsoft Office XP—to help users schedule meetings, manage tasks, communicate in real-time, present ideas, and create and edit documents using familiar tools. QuickPlace is a self-service application, so once administrators install the software on the server, users can take responsibility for creating a new team workspace and managing users for the workspace. © Copyright IBM Corp. 2003. All rights reserved. 1 QuickPlace has the customization capabilities that allow companies to maximize their investment in Web applications. Web designers and developers can customize the look and features of QuickPlace to fit either general needs or an industry segment's specific requirements. Changes to the look and functions of multiple QuickPlaces can be propagated simultaneously through centralized administration and template designs. A QuickPlace can also be integrated in a Web portal using the QuickPlace Java/XML API. QuickPlace's design allows team members to find information quickly by giving them the ability to search across all the “Places” they have membership in as well as to search within a single “Place.” Unlike most Web applications, team members can also bring information with them “off-line” when they disconnect from the network using Domino Off-Line Services. Administrators can make use of existing directories and manage multiple servers and clusters from a single console. 1.1.1 Key features and benefits of QuickPlace 3 My Places: Provides a central, personalized listing of all team workspaces to which the “Member” belongs, simplifying viewing, finding, and moving from one Place to another. Advanced search: Search across all the team workspaces within an enterprise, or scope the search to a single team workspace and “Rooms” and “Folders” within it. Lotus Sametime integration: Increase productivity by adding presence awareness, IM (instant messaging) and Web conferencing to any Place. This lets you see Member availability, initiate IM, or schedule or launch a Web conference. Team calendar: Keep track of shared calendar events created in the team workspace using IBM Lotus Notes or Microsoft Outlook. Room map: Find information faster by quickly navigating throughout a Place using this hierarchical list of all the rooms to which a user has access. Personalize and customize: Easily adapt a team workspace or fully customize to address a vertical or horizontal business process. Collaborate in context: Embed team collaboration in any Web-based application, Web site, or portal. 2 QuickPlace 3 for Sun Solaris 8 QuickPlace empowers people QuickPlace gives users the ability to: Create rich content: Author and edit documents right from a Web browser and share in popular formats including HTML, GIF and JPEG, animated GIF, e-mail with attachments, and more. Find information fast: Easily retrieve information with the help of full-text indexing. Search all Places or scope a search to a single Place. Personalize: “Point-and-click” to change the basic look and feel based on personal preference or corporate guidelines. Take it off-line: Establish a local copy on a desktop and seamlessly read, author, and edit documents while disconnected. Synchronize changes made to the local copy when reconnected to the network. QuickPlace inspires teams QuickPlace enables the team to: Collaborate in real-time: See if team members are available, engage in instant messaging dialog, and schedule and launch a Web conference. Manage projects: Assign tasks and track milestones specific to a project or initiative. Communicate using e-mail: Send e-mails directly to a Place, or receive notifications about postings, updates, team calendar events, and outstanding action items. QuickPlace extends enterprises QuickPlace can help your enterprise: Integrate with directories: Consistently support user names and passwords with other directory-based applications. Authenticate against any LDAP directory, including IBM Lotus Domino. Update multiple Places from a single template: Change a common template and update Place Types, simplifying administration. Apply existing IT skills: Tap existing IT skills and technology investments. QuickPlace supports standards and runs on leading server platforms. 1.1.2 QuickPlace 3 supported platforms Supported server operating systems: Microsoft Windows NT 4.0 operating system Microsoft Windows 2000 operating system (Server and Advanced Server editions) IBM AIX® operating system, Version 4.3.3 Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 3 iSeries™/OS 400 -versions: V5R1, V5R2 zSeries™ -OS/390® 2.10, and zSeries OS, z/OS™ 1.1 through 1.4. Sun SPARC, Sun Solaris 8 Supported Web servers: IBM Lotus Domino 5.0.10 Supported client operating systems Windows NT Windows 98 Windows 2000 Windows XP Professional Macintosh System 9 Supported browsers: Microsoft Internet Explorer 5.5 for Windows Microsoft Internet Explorer 6 for Windows Microsoft Internet Explorer 5.1 for Macintosh Netscape 4.79 for Windows 1.1.3 The QuickPlace environment/architecture QuickPlace is built upon Lotus Domino technology, and requires an underlying Domino server to provide much of its functionality. Minimal knowledge of Domino is required to set up, use, administer, or manage places within a QuickPlace. The only knowledge required is when setting up the server environment. QuickPlace uses the Domino mail infrastructure to invite members to join places and to notify them of changes. Search Places, the QuickPlace tool that enables users to search for information across multiple places and servers, relies on Domino Domain Search to index and locate place content. Using Domino single sign-on (SSO), QuickPlace users who sign on to a given place and server can access all the places on any server in the domain. Using Domino Off-Line Services (DOLS), users can take places offline and work while disconnected from the network. In addition, QuickPlace integrates with other technologies to augment the collaborative experience. For instance, QuickPlace draws on the capabilities of Sametime to provide online awareness and real-time communication, and implements the Lightweight Directory Access Protocol (LDAP) to enable connection to an external LDAP-compliant directory for authenticating users and obtaining user and group information. 4 QuickPlace 3 for Sun Solaris 8 1.1.4 Key QuickPlace components QuickPlace relies on some services obtained from other products, such as Domino and Sametime. This section includes some of the most important components, and provides a brief description of each. Together, these components make up the high-level architecture of QuickPlace. QuickPlace service A QuickPlace service is a multiple server deployment of QuickPlace servers and QuickPlace clusters that reside on the same network (that is, they are not separated by a firewall) share the same user directory and authentication system, connect to the same Place Catalog, and are managed by the same administration team. A QuickPlace server can belong to only one QuickPlace service. Servers within a QuickPlace service share information and have a consistent configuration. Members can use single sign-on authentication to access all servers in the service. Typically, servers in a QuickPlace service reside within a single Domino domain, but a QuickPlace service can span Domino domains as long as servers are cross-certified with the Place Catalog server and have at least Editor access in the ACL of the Place Catalog database. QuickPlace cluster A QuickPlace cluster is a group of QuickPlace servers that replicate data among themselves in real time. The servers that make up the cluster must all be running the same release of QuickPlace. QuickPlace does not support mixed-release clusters. QuickPlace servers can be clustered for failover using Domino clustering. The QuickPlace service treats a QuickPlace cluster as a single virtual server. In clusters that provide failover only, one member server, or node, is configured as the master server, and other nodes are configured as slaves. Under normal circumstances, the master server handles all incoming client requests. If the master server is unavailable for some reason, service fails over to the slave servers. In a complex cluster configuration, load-balancing services are provided in addition to failover. Load balancing requires the installation of a special computer, known as a network traffic dispatcher. The network dispatcher is not a QuickPlace server, but is responsible for receiving all incoming client requests and distributing them among the cluster nodes based on the availability of each node. In this type of cluster, individual nodes remain hidden. From the user’s point of view, all interaction is with the virtual server. For example, consider a cluster that includes the servers apple.acme.com, orange.acme.com, and kumquat.acme.com, and the designated virtual server for the cluster is the Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 5 network dispatcher fruit.acme.com. Users address all QuickPlace requests to fruit.acme.com, rather than to the individual servers in the cluster. Fruit.acme.com then directs the requests to one of the cluster nodes based on availability. QuickPlace server A QuickPlace server is a Domino server on which you install QuickPlace software. Each QuickPlace server is part of a QuickPlace service and hosts places created by QuickPlace users. You can set up specialized QuickPlace servers that do not have places on them to host the Place Catalog for the QuickPlace service or a Domino domain catalog for searches across multiple QuickPlace servers. Although a pilot installation of QuickPlace can reside on the same computer as Sametime and LearningSpace®® Virtual Classroom, this configuration is supported for evaluation purposes only. In the enterprise, the computer on which QuickPlace is installed must be a dedicated QuickPlace server. Although QuickPlace takes advantage of Domino features such as security, replication, mail, and so forth, do not use a QuickPlace server to provide Domino mail, directory, or application services outside of QuickPlace. Places A Place is a virtual collaborative workspace, created on a QuickPlace server, composed of a collection of Domino databases. The name assigned to a place must be unique among all places on the server as well as within the entire QuickPlace service. When a user creates a place, QuickPlace first verifies that the QuickPlace data directory on the server does not already include a subdirectory with this place name. To ensure that a place name is unique across the entire service, QuickPlace automatically registers each new place in the Place Catalog for the service. A place manager can create rooms or folders within a place, create online meetings, enable or disable mail for the place, specify the room where mail is delivered, and otherwise customize the place. The place manager controls membership for the place and designates each member as a Reader, Author, or Manager of the place. Rooms A place manager can create rooms to expand a place and create private areas for smaller teams to work in. Rooms can contain pages, folders, and other rooms. Access to a room can be restricted to a subset of group members who need to work together privately. 6 QuickPlace 3 for Sun Solaris 8 Folders To help organize the content of a place, members can create folders for storing pages related to a particular topic. Folders appear in the table of contents of a place or room and have their own security. The member who creates the folder can limit author access to managers only. By default, anyone who has Author access to the place or room can create pages in a folder. All members of a place or room can always see and open all the folders it contains. By hiding the “New” option, members with Manager access to a folder can prevent other members from creating pages. For more information on excluding the option to create pages in a folder, see the online QuickPlace user help. Page Pages are the basic unit for adding content to a place and can include formatted text, images, and file attachments. Authors can create pages in place using the QuickPlace rich text editor, or import content from an existing Microsoft Office, HTML, JPEG, or GIF file. Place members can create pages in places, rooms, and folders to which they also have Author or Manager access. A member who creates a page or who has been given edit rights to it can edit the page after it is published. Pages can be published as private drafts, or to a restricted set of readers and authors. Pages can be checked in and out to control access during editing. Place Catalog The Place Catalog is a Notes® database that collects information about all places that are on the QuickPlace servers and clusters in a QuickPlace service. The Place Catalog contains information about the servers, places, and members of a service. Using this information, users and administrators can work with all places in the entire service, rather than being confined to working with places on a single server only. Administrators use the information in the Place Catalog to establish management policies and ensure the integrity of places across multiple servers. The QuickPlace administration tool, qptool, queries the Place Catalog and generates reports that help track QuickPlace use. When building the My Places view for a user, the QuickPlace server uses the Place Catalog to track the user’s membership in places and to determine which servers to search when a user performs a cross-place search. By default, the QuickPlace setup program creates a Place Catalog in the data directory on the server. To configure multiple servers to use a single Place Catalog, add information to each server’s QuickPlace configuration Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 7 file (qpconfig.xml). The Place Catalog server must be accessible over the Notes port (1352) and HTTP to all servers in the QuickPlace service. An organization may have one or more Place Catalogs, depending on the number of services. A multiple-server enterprise requires at least one dedicated Place Catalog server. For best results, large-scale enterprise deployment should cluster Place Catalog servers so that at least one failover server is available. Search Places Search Places provides knowledge management capabilities for locating information across all the places in a QuickPlace service. Because QuickPlace content is stored within Notes databases, all the information in a QuickPlace can be full-text indexed. You can then use Domino Domain Search to search QuickPlace documents across multiple servers. With Search Places, place members added from an external LDAP directory can search across multiple places in a domain. Search Places requires configuration of a Domain Catalog server to index the databases. Search Places manages security using the access controls built into Domain Search. As a result, the search results returned to a place member display only those pages to which a user has access. My Places Each QuickPlace service provides users with a My Places portal that displays a personalized listing of all the places in a QuickPlace service that a user is a member of. My Places works only for members added to the service from an external LDAP directory. QuickPlace uses the directory credentials to find out which groups a user belongs to and then queries the Place Catalog to find all the places that the user is a member of, either explicitly or as the result of a group membership. It then provides links to those places, along with information about their approximate size and the date when they were last modified. User directory This is an external LDAP directory that provides a central repository for user information. QuickPlace refers to the user directory to fulfill requests from members and applications and to locate contact information and other attributes for users, groups, and other entities in an organization. For example, when a user invites members into a newly created place, QuickPlace uses the directory to find the invitees and their e-mail addresses. When an invited member signs in to the place, QuickPlace again refers to the user directory to authenticate the user’s password. 8 QuickPlace 3 for Sun Solaris 8 The QuickPlace server administrator selects the directory for each server. In QuickPlace services that have multiple servers, you must configure all servers to use the same directory in order to use multi-server session authentication. Online awareness, chat, and meetings With online awareness, provided through Sametime, QuickPlace members can log in and see other members who are logged in, use instant text messaging to communicate with other users in real-time, and share applications with other members. To enable Sametime features, install a Sametime server on a dedicated Domino server in the QuickPlace domain. The server must be running a Sametime release compatible with the current QuickPlace release. E-mail QuickPlace uses the Domino SMTP service to send mail to members. Members can generate messages from links with a page and the QuickPlace service can be configured to automatically send members information about updated QuickPlace content. QuickPlace can rely on the native Domino SMTP routing, or you can configure the underlying Domino server to send outbound mail through a dedicated relay host. For inbound mail, each QuickPlace has its own e-mail address and uses a mail-in database to receive mail. Contributions may be mailed in to QuickPlace, allowing critical e-mails from external sources to be gathered for future reference. Off-line access QuickPlace provides the ability to work with places off line through the use of Domino Off-Line Services (DOLS). QPTool QPTool is a command-line tool that you run manually, as a server task, or from a batch file, to perform QuickPlace administrative tasks. QPTool can accept XML files as input and can log the results of a command to a specified XML output file. QPTool commands can be run either locally or remotely, with the exception of the QPTool report command, which can be run only against places on the local server. To run QPTool commands remotely, the QuickPlace server administrator requires remote file system access, or Domino administrator access to the server. XML configuration file (qpconfig.xml) This is an XML file that provides most of a QuickPlace server’s configuration information. A QuickPlace server comes with a sample XML configuration file called qpconfig_sample.xml, which is installed in the data directory. Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 9 Server settings room The server settings room is a special room on each QuickPlace server for configuring options that govern security, directory use, and other server-specific information. Only designated QuickPlace server administrators can enter the server’s QuickPlace server settings room. 1.1.5 For more information on QuickPlace 3.0 To learn more about QuickPlace 3.0, visit http://www.lotus.com/quickplace, or the Lotus Developer Domain at http://www.lotus.com/ldd. 1.2 Introduction to Sun Solaris 8 As the foundation for Sun systems, the Solaris 8 Operating Environment enables your IT organization to deliver on the promise of massive scale, continuous real-time computing, and secure systems—while increasing service levels, reducing risk, and decreasing costs. With Solaris 8 software, you can efficiently manage resources and provide a higher level of service. It also helps reduce complexity by providing easy-to-use interfaces and comprehensive administration tools, all designed to help lower your IT risk. And by optimizing your resources and enabling them to scale to meet demand, the Solaris 8 Operating Environment helps you to minimize costs and lower your total cost of ownership. The Solaris platform supports: One million simultaneous processes on a single system Up to 128 CPUs on a single system More than four billion network connections 32- and 64-bit applications Two-, four-, and eight-node clusters IPv4 and IPv6 network addresses Up to 512 CPUs in a clustered environment 1.2.1 Highlights of Sun Solaris 8 The leading UNIX operating environment, it combines power, stability, and predictability with complete backwards compatibility. Offers reliability, availability, and serviceability comparable to that of a data center, but at a fraction of the cost of a mainframe. 10 QuickPlace 3 for Sun Solaris 8 Assures superior availability through a smaller, more stable kernel design and increased load balancing across multiple processors. Scales to handle heavy traffic, huge data sets, and CPU-intensive problems. Tightens an already secure environment with increased support for major security protocols as well as new technologies. Offers a complete global solution with extensive support for 37 languages and 123 locales. Provides the premier deployment platform for Java technology. Enhances ease of use through simplified software installation and setup, plus comprehensive integration capabilities. Supports the latest networking protocols and adheres to all major industry standards. Standards supported by Sun Solaris 8 Interface: X/Open UNIX 98 Graphics: X11, Adobe PostScript, Display PostScript, OpenGL Desktop: CDE, Motif Object: Java IDL Connectivity: ONC, ONC+, NFS, WebNFS, SMB, and optional NetWare IPX/SPX, SNA, AppleTalk, DECnet, and others Internet: HTTP, FTP, Telnet, DNS, NTP, IMAP4, DHCP, SNMP, SMTP, IPv6, IPSec, Kerberos, SASL, OCF Protocols: LDAP v3 IETF RFCs 1323, 1510, 1652, 1869, 1870, 1891-1894, 1985, 1996, 2018, 2136, 2045, and 2078 1.2.2 System requirements for Sun Solaris 8 SPARC 32- and 64-bit platforms Disk space: 600 MB for desktops; one Gbyte for servers Memory: 64 MB minimum 1.2.3 For more information on Sun Solaris 8 To learn more about the Solaris Operating Environment, visit: http://www.sun.com/solaris For additional technical and development information, visit the BigAdmin portal at: http://www.sun.com/bigadmin Chapter 1. Introduction to QuickPlace 3 on Sun Solaris 8 11 1.3 What’s new in QuickPlace 3.0 for Solaris 8 Version 3.0 of QuickPlace enhances previous versions by adding significant functionality for users and administrators. Some of the key additions include: My Places provides a central listing of all team workspaces to which the person is a member. This is personalized to the individual and simplifies viewing, finding, and moving from one wokspace place to another. Advanced Search allows searches across all team workspaces within an enterprise, or to a single team workspace and all rooms and folders within. Integration with external Lotus Sametime adds presence awareness, instant messaging, and Web conferencing to any Place. This allows members to view the availability of other members, initiate instant messaging, and schedule or launch a Web conference. Team calendar allows members to track a shared calendar of events created within the workspace place, using IBM Lotus Notes or Microsoft Outlook. Improved LDAP Support allows administrators to map their customized LDAP directory attributes to required QuickPlace directory attributes. QPTool allows administrators to manage changes to specific QuickPlace Places if access through the Web interface is not possible (for instance, if place manager is removed from the directory prior to transitioning ownership of the place). Additionally, in previous versions of QuickPlace the version released for the Solaris platform was not fully functional, so several key features were not available on the Solaris platform. With QuickPlace 3.0, this issue has been fixed, and the full functionality of QuickPlace 3.0 is now available on the scalable Solaris 8 platform. For more information on the new features of the QuickPlace 3.0 product, see the following recent Lotus Developer Domain articles: New Features in QuickPlace 3.0 http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/869d d0fa3a87875e85256c290015d867?OpenDocument Comparing QuickPlace 3.0 with QuickPlace 2.0.8 http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/2a73 8f04a34a537b00256c270062a8c1?OpenDocument 12 QuickPlace 3 for Sun Solaris 8 2 Chapter 2. Installation hints and tips This chapter provides some overall hints, tips, and suggestions for successfully installing QuickPlace 3.0 on the Sun Solaris platform. It is intended to be used in conjunction with the more formal installation documentation that can be found on the Lotus Developer Domain (ldd) at: http://www-10.lotus.com/ldd/products.nsf/products/quickplace Additionally, since QuickPlace 3.0 executes on top of a basic Domino 5.0.10 server, readers unfamiliar with the Domino platform may want to first review the IBM Redbook Lotus Domino R5 for Sun Solaris 8, SG24-5969. 2.1 Preparing for an installation Prior to starting the installation process for any QuickPlace server, some basic planning is recommended. Specifically, consider the following: Deployment Options - Various deployment options and choices are available with QuickPlace. Before you start, considering the following: – Decide whether to install Quickplace in an existing Domino domain or a dedicated Domino domain. For reasons we present later in this chapter, we generally recommend a dedicated domain, but you should choose the option most suitable for your environment. © Copyright IBM Corp. 2003. All rights reserved. 13 – Decide whether or not to cluster or distribute QuickPlace services across multiple servers. – Determine your directory strategy. QuickPlace 3.0 supports the use of local place member lists (CONTACTS1.NSF) to define place members. However, for most organizations, using an external LDAP directory is recommended. Additionally, several QuickPlace features require the use of an external LDAP directory, including: • Use of Sametime features such as online awareness • Use of Search Place to perform searches across places and servers • Use of My Places • Ability to assign a super user • Multiple-server single-session authentication • User names in double-byte character sets Security considerations - As with any application, proper security planning is crucial. QuickPlace actually provides multiple types and layers of security; the installation- and setup-related aspects are: – Physical and platform security: The hardware and OS itself must be properly secured. – Protocol security: You can protect QuickPlace communications using secure socket layer encryption (SSL) and Notes/Domino port encryption. – Authentication: To allow users to have single sign-on (SSO) capabilities across multiple quickplaces, the Lightweight Third-Party Authentication (LTPA) token-based Domino multi-server single sign-on (MSSO) must be enabled. For more information on the deployment options and security considerations, refer to QuickPlace 3.0 Deployment Guide available in the Product Documentation section of the Lotus Developer Domain Web site at: http://www-10.lotus.com/ldd/products.nsf/products/quickplace Naming conventions - Establish naming conventions for the names of your QuickPlace servers, the OS User IDs, and the OS Group names you plan to use. In our installations for this paper, we used a three character datacenter designator, plus a two to three character application designator, plus a two character instance number as our recommended naming convention. (For example, the first QuickPlace server in our Cambridge datacenter is camqp01, and tstqp01 is our development/test QuickPlace server as used for this Redpaper.) We also suggest giving the OS user ID running the software the same name as the server. Having a clear server and ID naming strategy can greatly simplify system management. 14 QuickPlace 3 for Sun Solaris 8 2.2 Installing Domino This section describes the steps necessary to install the underlying Domino server code that is required for QuickPlace: 1. Log into the Solaris root account. 2. Create a Solaris group for group ownership purposes. We used a group name of notesgrp. 3. Create a Solaris user ID from which the QuickPlace server will be executed. (That is, the Solaris user ID that will own the QuickPlace server process.) Tip: We recommend using at least two Operating System IDs. The first ID is a general ID for ownership of the binary files and any scripts used to manage or monitor the QuickPlace server. The second ID is the owner of running the server process. This two ID approach makes it easier to set up and manage multiple server instances if you choose to partition servers. In our setup, we used the ID domino for storing the code, and the ID tstqp01 for running the server. 4. Mount the cd-rom drive with the QuickPlace 3.0 for Solaris CD at /cdrom. 5. Identify or create a file system on the Solaris system to be used for the install. Tip: We recommend using file systems spread across multiple physical devices, or I/O channels, to optimize I/O performance/throughput. 6. Change directories to /cdrom/sol. 7. Run the command ./install 8. Answer the prompts. Select the following choices from the appropriate prompts: – Domino Enterprise Server – iNotes™ Web Access for this Domino Server 9. Select directories owned by the OS User IDs you created in step 3. We used /export/home/domino/lotus for the program directory setting and /export/home/tstqp01/data for the data directory. A symbolic link will automatically be created for /opt/lotus. 10.After you have completed all the prompts, you will be given a confirmation screen to verify your selections. This is shown in Figure 2-1 on page 16. Chapter 2. Installation hints and tips 15 Figure 2-1 Installation confirmation 11.Press Tab to install the Domino server code. After installation, the code will prompt you to start the server from the owning operating system ID. Switch to the Solaris ID that you set up to own the QuickPlace server process using the su command, for example: su - tstqp01 12.Change directories to make the Domino data directory you specified during installation active, for example: cd data 13.Start the server by issuing the command: /opt/lotus/bin/server. This will start the Domino server in httpsetup mode so you can configure the server from a Web browser. Note: At this point the HTTP server will be running on port 81 instead of port 80. To configure your Domino server, you need to connect to it using port 81. The URL would be: http://server_name:81, where server_name is the name of the server (for example, http://tstqp01.lotus.com). 16 QuickPlace 3 for Sun Solaris 8 Figure 2-2 Server ready to configure 2.3 Setting up the Domino server for QuickPlace usage At this point, the Domino server can configured for QuickPlace usage. 1. If you are already familiar with Domino Domains and have existing ones, you need to decide now if you want the QuickPlace server to be part of your existing Domino Domain. If you do, select Additional Server from the Web browser. If QuickPlace is going to be part of its own Domain, select First Server from the Web browser. Tip: We recommend putting QuickPlace in its own domain, separate from any existing Domino infrastructure that you may have. This simplifies server configuration records, and administration. If you set up multiple QuickPlace servers, have them all participate in the same domain. 2. When configuring your Domino server, accept the default services along with HTTP and SMTP. Do not select LDAP, iMAP, or other server services which will not be used by QuickPlace. We don’t recommend running other server services (LDAP, IIOP, and so forth) from a QuickPlace server since this will adversely affect the performance of your QuickPlace server. It is important to note that while QuickPlace uses an LDAP client to access an LDAP directory for authentication, this LDAP client is different from the LDAP server listed in this configuration section. Chapter 2. Installation hints and tips 17 Figure 2-3 Server configuration information 3. Once you have completed the configuration information, your Web browser will indicate that the server has been stopped, and the console/telnet session of your Operating System ID will show that the http server has been shut down. We recommend that you restart the server process to verify that everything has been set up correctly and to confirm there are no errors. To do this, issue the command: server The most common error during this server startup is a conflict with the SMTP service. Prior to running the server command, you may want to check whether the sendmail SMTP server is already running on the Solaris system. Use the ps -ef command to determine if the process is running. If it is, you can kill the sendmail task via ps, and then remove it from the init tables before the next reboot of the Solaris OS. 18 QuickPlace 3 for Sun Solaris 8 Figure 2-4 Checking the sendmail process 4. Once you have confirmed that the Domino server is running without any errors, issue a quit command from the console/telnet session and prepare to install QuickPlace. 2.4 Installing QuickPlace After Domino is installed and set up for QuickPlace usage, you are ready to take the final (and easiest) step of installing QuickPlace itself on top of Domino. 1. Ensure you are no longer accessing the server under the Operating System ID which is used by the Domino server. Verify that you are signed in as root by issuing the ID command from the command line. 2. Go to the QuickPlace CD, which should still be mounted at the /cdrom mount point. Change your working directory to server (for example, /cdrom/server) and run the .install program. 3. You will be asked to again specify the group and user ID names that you specified when installing Domino. The program file and data directories for QuickPlace should also be identical to those selected for Domino. 4. Specify a Quickplace super user when installing the QuickPlace server. This user will be used to initially configure and manage the QuickPlace server. Tip: We recommend that you use an administrator name that does not exist in any of your current directories. After the server is configured, you can specify users from your production directory to also manage the QuickPlace server. This approach will allow you to manage the QuickPlace server if your directory service becomes unavailable for some reason. We used qpadmin as the administrator name. Chapter 2. Installation hints and tips 19 5. Once the QuickPlace server has been installed, restart your Domino server using the /opt/lotus/bin/server command. Append an ampersand (&) at the end of the command to allow the server to be started in the background. Tip: We recommend that when starting the server, you redirect the standard input and output with files. This way you can monitor your QuickPlace server and issue console commands to it remotely if you don’t have access to your Notes Administration Client. To do this, create a directory called log in a separate file system and set an environment variable called $LDIR to point to the log directory. We used console.input as the name of the input file to be redirected, and console.log the name of the output file. The startup command is: /opt/lotus/bin/server <$LDIR/console.input >$LDIR/console.log & Once the server is started you can issue a tail -f $LDIR/console.log to watch everything happening on the server as it happens, or echo "command" >> $LDIR/console.input to redirect "commands" to the QuickPlace server. Of course, implementing this approach will require access to the redirected files to be restricted to secure user IDs; otherwise, overall Domino security will be greatly weakened. 20 QuickPlace 3 for Sun Solaris 8 3 Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris In this chapter we describe some key performance tuning measures you can take to fine tune your new QuickPlace environment on the Solaris Platform. We first describe the various performance tools that can be used in any Sun Solaris environment, and then we describe the test environment that was built for this paper. Finally, we provide our test results, showing the actual performance improvements provided by each tuning-related configuration change. 3.1 Sun Solaris performance monitoring options For this Redpaper, we wanted to highlight performance tools that are readily available on any Solaris system. This way, no special software is required for the reader to duplicate these steps in their own environment. Following this plan, we primarily used a combination of Solaris commands which are included with the base operating system and the GNU debugger to calculate the relative impact of various performance tuning parameters. However, we also used the performance monitors included with Mercury Interactive’s LoadRunner © Copyright IBM Corp. 2003. All rights reserved. 21 test suite to generate the more user-friendly throughput and performance graphs shown later in this chapter. The specific Sun Solaris commands that we used for performance monitoring are described in Table 3-1. Table 3-1 Key Solaris performance monitoring commands Command Description vmstat 30 180 Provides statistic information from the Solaris virtual machine. The 30 sets the collection/reporting interval to 30 seconds and the 180 specifies the number of iterations to be repeated. This will cause vmstat to report statistics every 30 seconds for 1.5 hours. mpstat 30 180 Provides statistics on each of multiple processors. This helps us determine if some CPUs are working harder than others. The parameters are the same as vmstat. iostat -x 30 180 Provides statistics on the input/output generated on the system. This helps us determine if some disks are more heavily utilized than others. The -x option provides extended statistics. netstat -s Provides statistics about the network. The -s option provides output for all interfaces and allows us to check for network errors. 3.1.1 vmstat The vmstat command generates a listing containing a number of columns that display information about the state of the system. It includes information about the processes, memory, page, disk, faults, and CPU. For the purposes of tuning QuickPlace, we are interested in information about the processes, paging, disk, and CPU. Figure 3-1 shows sample vmstat ouput from a mostly idle system. There are 0 processes waiting to be executed in the run queue (column r), there is minimal paging (pi, po columns), no disk activity, and minimal CPU in use (all idle). QuickPlace is tracked as user CPU (that is, the us column). 22 QuickPlace 3 for Sun Solaris 8 Figure 3-1 Sample vmstat output for an idle system 3.1.2 mpstat The mpstat command generates a listing showing what each CPU is spending its time doing: for example, the distribution of system, user, wait, and idle time; system calls made, lock contention, interrupts, faults, and cross calls. There is some Solaris tuning that can be performed to balance the CPU load if it is unbalanced, but these tunings are not commonly needed. Instead, we use mpstat to see what the actual CPU load is on the system, since vmstat will only report on the most heavily used CPU. mpstat is most useful when we are using a large system which has been carved into multiple Solaris “domains” (meaning system partitions). This helps us identify if we are holding unused CPUs in our domain (in which case we can reallocate them to other domains), or if we need additional CPUs assigned to our domain. The output shown in Figure 3-2 shows the key columns of interest for our purposes: The interrupts are spread unevenly across the CPUs (intr column) . An uneven distribution may point to a problem with the system’s “SBus” configuration (that is, the most common Solaris system bus), although in this particular case it is not significant. The number of failed mutex enters (smtx column) shows internal lock contention. Low numbers are common, but when it gets into the mid thousands or higher—or if there appears to be a correlation between high system time and high smtx levels—then you have some contention that should be addressed. System call overhead (syscl column) can be high on a QuickPlace server. When it gets over 5000/sec, you will want to address this by adding additional capacity, or investigate what else is running on the system that can be moved to another platform. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 23 The time spent in I/O wait (wt column) will determine if the system is I/O-bound. If this is an issue, you may need to spread out your IO array by adding channels or increasing buffering. Your wait time should be less than the time spent in processing QuickPlace server requests (usr column). Any issues or changes in this area is really a balancing act between limiting the OS ID under which the Quickplace server is running (that is, within etc/limits), and satisfactory user response time. Relatively significant amounts of system time on the CPU (sys column) generally indicates a problem if it is sustained. Either the load on the system is too high and additional capacity must be added, or other tuning is required to allow the system to operate more optimally. Figure 3-2 Sample mpstat output - for an idle system 3.1.3 iostat The iostat command provides information about disk utilization. The main item to look at is the svc_t column, which describes the average service time (in milliseconds) required to complete an I/O request. You can also use iostat to determine if the disks are not load balanced. We segmented our test system with multiple I/O channels in some cases, but for the majority of the testing we used only a single channel. Thus, our output will reflect one file system (containing the QuickPlace data) having a significant load and the other file systems having a minimal load. Because most people use RAID 5 arrays for their disk, this information is useful for comparison of load across the volume groups. Utilization numbers may often reflect close to 100% utilization; however, this does not necessarily indicate a problem. See the SUN article, “What does 100% busy mean,” for more details. You can find this article at: http://www.sun.com/sun-on-net/itworld/UIR990801perf.html 24 QuickPlace 3 for Sun Solaris 8 Figure 3-3 Sample iostat output for an idle system 3.1.4 netstat The netstat command provides information about the network system. We are looking for any network buffer overload errors caused by not allocating enough space to the TCP/IP buffers of the network. There are no errors reported when you have allocated too many buffers; but we want to ensure we allocate the minimal amount of memory to network interfaces to support the sustained load, as well as most of the peak load. Minimal increments of network buffer errors reflect a system with the right balance of memory allocated to the network. We are specifically interested in any dropped frames from the TCP/IP stack (tcpTimRetransDrop and tcpListenDrop), as highlighted in Figure 3-4. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 25 Figure 3-4 Sample netstat ouput 3.2 Our test environment This section describes the test environment that we established to execute performance loads/tests for this paper. 3.2.1 Test categories/classes We established three different test environments for our performance tuning experiments. The environments we established are classified as shown in Table 3-2. 26 QuickPlace 3 for Sun Solaris 8 Table 3-2 Test environment server classifications Category Server Type # CPUS Memory Developer Class Sun Ultra 5 1 256 MB Small Company or Division Sun Blade 2000 2 4 GB Enterprise Sun F6800 4 16 GB 3.2.2 Basic server environment We installed Sun Solaris 8 (SunOS 5.8) on each of the test environments, and applied no Sun maintenance/fixes/patches except for the Generic Patch (October 2001) included in the general installation. On each server, we installed Domino 5.0.10 with out-of-the box functionality. We then applied a stability patch to Domino 5.0.10 which was made available to the QuickPlace 3.0 beta program. This patch brought the version of Domino 5.0.10 to release 5.0.10CF1. Note: At the time of this writing, this patch was still required. The reader should refer to the QuickPlace portion of the Lotus Developer Domain Web site for the latest information on the requirement and availability of this patch. QuickPlace 3.0 was installed as described in 2.4, “Installing QuickPlace” on page 19. 3.2.3 LDAP directory setup To duplicate what many of our clients have expressed as “typical” for their environments, we set up an LDAP directory to be used with the QuickPlace server. The first LDAP directory we utilized was hosted by a Domino 5.0.10 server. The second LDAP directory was hosted by a Sun ONE Directory Server. Tip: An LDAP directory is recommended as the directory of choice for QuickPlace. If you already have a Domino environment, simply designate one of your servers to publish the directory using LDAP, and point your QuickPlace installation to that server. Domino LDAP Configuration The Domino LDAP directory was configured to allow anonymous access, and contained approximately 300,000 users. The LDAP server was located in a separate network segment from the QuickPlace server and the load driver Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 27 machines. This was to duplicate what we expected to be the “worst case” scenario for directory lookups. We expect that for performance reasons, organizations will typically locate their Directory servers close to their application servers (to reduce network latency and minimize network load). Because we used a Domino LDAP server, no mapping was required between the attributes provided by the directory and the attributes required by QuickPlace. SunONE Directory Server LDAP configuration As this paper is focused on Sun Solaris as the platform for QuickPlace, we chose to make the SunONE Directory Server a directory of choice for some of our testing, in addition to the Domino LDAP directory. The Sun directory was also configured remotely from the test server. It was based on one of our larger Domino customers, whose directory implementation requires mapping of attributes, and authenticated access. To perform the mappings between the required LDAP attributes and the attributes contained in the Sun ONE directory, we utilized the qpconfig.xml file provided with QuickPlace. The documentation for the use of this file is contained within the file itself. 3.2.4 Performance load strategy We used LoadRunner 7.02 from Mercury International to generate the simulated user loads required for the QuickPlace server in our testing. The load was configured to utilize all places on the server, and some places contained more concurrent users than others. We loaded 8 users every minute from 2 different run groups. This allowed us to mix the processing of “logins” with workload activity. We developed a scenario to try to simulate fairly accurately the user load a typical environment would experience. The tasks shown in Table 3-3 were generated as part of the scenario. Each task is a complete multi-click event, and represents several QuickPlace transactions. Each virtual user will execute any one of the listed tasks during a single test run according to the listed priorities. 28 QuickPlace 3 for Sun Solaris 8 Table 3-3 User tasks for simulated QuickPlace load Task Probability (%) Login 80 Read attachment 50 Read a simple page in the table of contents 90 Read an imported page in the table of contents 90 Read a simple page in the start of a folder 60 Read an imported page in the start of a folder 60 Read a simple page in the end of a folder 5 Read an imported page in the end of a folder 5 Read first page in a folder 80 Read last page in a folder 5 Open the calendar 60 Read a calendar event 20 Open the task list 5 Read a task 5 Open a page in a room 6 Open a page in an inner room 6 Create new page 20 Unauthorized access 20 When the virtual users were created, they had some randomization built in. The randomization chose which QuickPlace would be used during the test session, as well as which tasks would be performed (according to the probability listed in the table). 3.2.5 Simulated/sample QuickPlaces used Over 260 places were created on the test servers, for use by our test load drivers. These places took up approximately 26 GB of space at the start of the testing. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 29 Table 3-4 represents the average topology of the places. Table 3-4 Average topology of QuickPlaces used Place object Average on server Rooms per place 4 Folders per room 5 Documents per folder 1000 Forms per room 2 Fields per form 5 Local users per place 2 Local groups per place 2 External groups per place 5 External users per place 100 Themes per place 1 PlaceBots per room 0 3.3 Performance testing and tuning This section provides the results of our actual performance tests, and details the tuning settings/parameters that we applied to improve performance. Our overall tuning strategy was as follows: Baseline the system using “out-of-the-box” default settings and parameters. Apply performance tunings based on the baseline results. Re-test the “tuned” system, to verify performance improvements. 3.3.1 Out-of-the-box performance numbers We set up QuickPlace 3.0 on each of our system classes (developer, division/small company, and enterprise) as described previously. We then ran some basic user loads across the systems: Developer System (Sun Ultra 5) - 5 concurrent users Division/Small Company System (Sun Blade 2000) - 50 concurrent users Enterprise System (partitioned Sun 6800) - 500 concurrent users 30 QuickPlace 3 for Sun Solaris 8 Developer system: Out-of-the-box results Prior to tuning, we found that the developer system performed adequately well. The system continued to respond adequately (that is, under 2 seconds average response time), although the systems resources were fully consumed. Figure 3-5 shows the relative response time on the system versus the number of concurrent users. Figure 3-5 Developer system: Out-of-the-box performance results The vmstat information showed that the system resources were pretty heavily utilized, and that the Sun Ultra 5 with 256 MB was memory-constrained, causing jobs to back up in the run queue. Thus, no tuning can really be performed on this system, until its memory constraints are addressed. Figure 3-6 Developer system vmstat results: Out-of-box performance results Division and enterprise systems: Out-of-the-box results The characteristics of the Sun Blade and the Sun 6800 systems were similar in our testing. We found that neither system was memory- or resource-constrained, and that both responded similarly to a test load of 250 users prior to tuning. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 31 Essentially, neither could support 250 concurrent users with out-of-the box configurations. Each ran into problems around 100 concurrent users! Figure 3-7 Enterprise system: Out-of-the-box test results Figure 3-7 shows the results of the load test on the Enterprise (Sun 6800) system. The red line (the line that starts at a steeper angle) shows the load on the system, peaking at 100 users. Once we reached a load of 100 users, the CPU averaged a 67% load from QuickPlace. Our vmstat results showed that the disk was similarly affected and response time averaged about 7 seconds. Thus, running more than 100 concurrent users on either system resulted in response times spiking to approximately 18 seconds, and once this spike began to occur, response times never returned to below 9 seconds. 3.3.2 Tuning QuickPlace It became obvious in our performance tests that the out-of-the-box configurations would not perform adequately, regardless of the system upon which the QuickPlace environment was installed. The performance modifications that can then be made fall into several categories: Domino-, QuickPlace application-, and Solaris-related tuning. In this section we describe the key tuning techniques in each category. 32 QuickPlace 3 for Sun Solaris 8 Domino-related tuning The first thing to do is to tune the Domino Server underlying QuickPlace. We tuned the Domino server for servicing Web clients, since a Web server has very different needs than a server that performs another role, such as mail routing. Concurrent Web agents The first thing we did was allow the Domino server to run multiple Web agents at the same time. This is done by updating the server document in the address book, under Internet Protocols → HTTP tab. The setting shown in Figure 3-8 allows multiple users to run QuickPlace bots simultaneously. Figure 3-8 Enabling concurrent Web agents HTTP threads While we are in the server document, we should also increase the number of available active HTTP threads. Each thread consumes approximately 30 KB of memory, so we don’t want to increase this too high. On the division/small company server, we increased this number to 96; on the enterprise server, we increased it to 120. Sun does not recommend increasing this value to above 100, but if you have available memory, increasing this setting should not have a negative effect. However, adjust this number carefully in your environment, looking for memory constraints. Figure 3-9 Increasing the number of http threads Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 33 HTTP memory cache Another Domino tuning that can be made is to increase the HTTP server’s memory cache settings themselves. While we did not tune these values at all in our test environment, you may want to consider adjusting these settings in your environment. For example, increase the maximum number of cached users from 64 to 512, and increase the cached user expiration from 120 to 300 seconds. Memory cache options appear on the Internet Protocols → Domino Web Engine tab of the Server document. QuickPlace application tunings There are a few tunings that are specific to QuickPlace on top of Domino, which may also provide for an overall performanc improvement. Caching QuickPlace pages Web page caching can greatly improve the response time of a QuickPlace server. Without a cached copy of a Web page, the HTTP server must access the database upon every HTTP request, which results in a slower response time for the QuickPlace server. If a Web page is cached on the QuickPlace server, the server only has to pick up the page from the database one time, and then create a user-specific, cached copy of the page at that time. Upon subsequent requests for the page, the server retrieves and provides the cached copy, as long as it is still valid. If the server is brought down, the existing cache is maintained after the server is brought back up. Despite the advantages of Web page caching, we did not enable this capability in our environment because we were able to get more than satisfactory response time with the other performance tunings mentioned in this paper. However, this is a key parameter that you may want to consider adjusting in your environment. To enable the cache Set QuickPlaceWebCacheEnabled=1 in the NOTES.INI file. To disable server caching, set QuickPlaceWebCacheEnabled=0 The server cache is enabled by default. To set the cache directory Set QuickPlaceWebCacheDir= <full path> in the NOTES.INI file. If this variable is not set in the NOTES.INI file, then it is automatically set to the default directory: <NOTESPROGRAM>\data\domino\quickplace\cache. If a specified directory path is invalid, the server cache is disabled. 34 QuickPlace 3 for Sun Solaris 8 To set the cache size limit Set QuickPlaceWebCacheLimitInMB = <size in MB> in the NOTES.INI file. If this variable is not set in the NOTES.INI or if the size given is not a positive number, then the variable is automatically set to the default size of 50 MB. To set the time interval for cache cleaning Add QuickPlaceWebCacheGCIntervalInMIN= <time interval in minutes> to the NOTES.INI file. If this variable is not set in the NOTES.INI file, or if the value given is not a positive number, then it is automatically set to the default value of 60 minutes. To set the cache for anonymous users only Add QuickPlaceWebCacheUsers= Anonymous to the NOTES.INI file. The default value for this setting allows server caching for all users when the cache is enabled. Changing this setting disables caching for all other users. To enable logging for the server cache Set QuickPlaceWebCacheLogging = <log level> in the NOTES.INI file. The logging setting has three levels: 1, 2, or 3, where 1 is the least detailed and 3 is the most detailed. Logging is written to LOG.NSF. Caching QuickPlace users After a QuickPlace server successfully authenticates a user, it adds the user’s name, password, and the groups of which the user is a member to its user cache. The next time the user attempts to authenticate, the server can quickly access the information in the cache to speed up authentication. Specifying the number of user entries allowed in the cache To specify the maximum number of user entries allowed in the cache, use the following NOTES.INI setting: QuickPlaceMaxCachedUsers=number where number is a number of user entries. When the cache reaches the specified number, older entries are removed to make room for new ones that are needed. By default, 64 user entries are allowed in the cache. Specifying the length of time user entries remain in the cache To specify the length of time user entries remain in the cache before the server removes them, use the following NOTES.INI setting: QuickPlaceExpireCachedUsers=interval where interval is the length of time in seconds. By default, the interval is 120 seconds. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 35 One of the issues with our testing was that, by default, QuickPlace will flush its user cache every 2 minutes. This causes any connected Web browsers to resubmit any pending HTTP requests, along with session credentials. Unfortunately, our load tool was unable to handle this automatic refresh of the session information in the manner that a standard Web browser would. So, we were required to set very high max cached users (1500) and expiration (3600) values to circumvent this issue. Solaris-related tuning In addition to tuning the default Domino and QuickPlace parameters, there are several operating system parameters that we modified to improve our overall system usage. Changing file descriptors To change the number of file descriptors, you have to edit the system file in the /etc directory. Use the root userid to make this change. We changed this value to 64K on both our servers. This allows the QuickPlace server ID to manage up to 64K of file descriptors simultaneously. The line added to the system file was: set rlim_fd_max=65536 Increasing memory available to TCP/IP To change the amount of memory available to IP networking, we had to make a change to the Domino HTTP configuration daemon file (httpd.cnf) and increase the limits within the Solaris TCP/IP networking code. To increase the value Domino uses within its HTTP backlog queue, add the following line to the httpd.cnf file: listenbacklog SIZE_IN_BYTES We used different values for the Sun Blade (Division) server and the Sun 6800 (Enterprise) server. Thus, the SIZE_IN_BYTES value of the change for the Blade 2000 server was 2K, while for the 6800 server it was 4K. Once you make this change to allow Domino to address more IP memory, you also need to change the limits in two of the Solaris TCP listen queues to make more available to Domino. The Solaris queues in question are: tcp_conn_req_max_q tcp_conn_req_max_q0 These changes must be made by issuing the following commands from the root account: /usr/sbin/ndd -set /dev/tcp tcp_conn_req_max_q SIZE_IN_BYTES 36 QuickPlace 3 for Sun Solaris 8 /usr/sbin/ndd -set /dev/tcp tcp_conn_req_max_q0 SIZE _IN_BYTES SIZE_IN _BYTES is again the same value set in the httpd.cnf file for the listenbacklog parameter. We knew we needed to increase the value of these parameters because our tcpListenDrop parameter was showing dropped network packets in the netstat -s parameter. So, we increased these values by 1K increments until we stopped dropping packets, resulting in the final 2K and 4kK values mentioned earlier. In general, these changes should be done incrementally and one at a time. This will help you identify the right settings for your environment. On the Division server, we were able to drive loads of up to 500 concurrent users with the 2K block. On the Enterprise server, we drove loads of up to 750 concurrent users with the 4K block. When we reached the level we were trying to achieve, we stopped adjusting this parameter. System file (kernel) tuning We did not make any more tuning adjustments to the Division/small company server. However, we did make some additional changes to the enterprise system. Specifically, we changed the system file (/etc/system) to tune some of the base operating system, or kernel, characteristics. Changes were made on this enterprise system to: System sync queue (sq_max_size), which is used to transfer network packets from the hardware driver to the TCP/IP protocol driver Characteristics of the file system flush daemon (tune_t_fsflushr) Size (bufhwm) and life (autoup) of the buffer cache Hard limit on file descriptors (rlim_fd_max, as discussed earlier) These changes are shown in Figure 3-10. Figure 3-10 System file tuning changes on the Enterprise class system Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 37 Note: Additional information on Solaris-specific performance and kernel tunings can be found in the “Solaris Developer Connection” articles that are listed in “Referenced Web sites” on page 41. Parameters that should not be changed There are some common Domino tuning parameters that have been applied to R5 Domino servers running on Solaris, and described in other redbooks and documentation. However, we found all of the following parameters to have either a minimal positive, or even an adverse affect, on the performance characteristics of our QuickPlace servers. Thus, we would recommend that these parameters be left at their defaults: NSF_BUFFER_POOL_SIZE or NSF_BUFFER_POOL_SIZE_MB PercentAvailSysResources set maxpgio=16384 However, if you are using multiple swap devices, or very fast RPM disks, setting this value may not adversely affect you as it did our environment. Priority_Paging=1 Update_No_FullText=1 3.3.3 Performance results after tuning After performing the tuning described, we re-executed the same test scripts against the enterprise server to determine the improvements that were achieved. You will recall, that with the out-of-the-box settings we were only able to load this system with 100 concurrent users before response time became unacceptable. With the described performance tuning modifications, we were able to easily load this system with 600 concurrent users, all with a ~1 second response time! In fact, we are confident that even more concurrent users would easily be supported by this system, as our 4 CPU domain on this Sun 6800 system was running with very minimal CPU load with the 600 concurrent users. Unfortunately, we were not able to demonstrate this higher load based on the user cache time-out issue described previously, as well as due to limitations in our load tool licensing. The “tuned” performance results are shown in Figure 3-11. 38 QuickPlace 3 for Sun Solaris 8 Figure 3-11 depicts the successful 600 concurrent user load test. Figure 3-11 Enterprise system results after performance tuning 3.3.4 Performance results compared To provide some context around these performance results, it is useful to compare them to results of similar testing recently performed on the Windows NT platform, as part of the Lotus Developer Domain article “QuickPlace 3 performance,” which can be obtained at: http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/5076 cae0219bd20885256c78006bab36?OpenDocument This LDD article shows test results for the same user load scenarios used in this paper, as executed against a Quad-processor Pentium III Intel server running Microsoft Windows NT Server. This Intel/Windows NT platform was only able to support 275 concurrent users at an approximately 1 second response time - this is compared to the 600+ concurrent users that were easily supported on our tuned Sun Solaris system. The significant benefits of a high-end UNIX platform, such as the Sun 6800 system used for this paper, are clearly shown from these performance results. Chapter 3. Performance tuning QuickPlace 3.0 on Sun Solaris 39 40 QuickPlace 3 for Sun Solaris 8 Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of some of the topics covered in this Redpaper. IBM Redbooks The following IBM Redbooks provide good background reading behind some of the technologies discussed in this paper: Lotus Domino R5 for Sun Solaris 8, SG24-5969 Deploying QuickPlace, SG24-6535 You can order hardcopy Redbooks, as well as view, download, or search for Redbooks at the following Web site: ibm.com/redbooks Referenced Web sites These Web sites were referenced within this paper as relevant further information sources: Lotus Developer Domain General QuickPlace information can be found at the QuickPlace “place” within the Lotus Developer Domain: http://www-10.lotus.com/ldd/products.nsf/products/quickplace These specific LDD articles and documents were discussed within this paper: QuickPlace 3.0 Deployment Guide http://doc.notes.net/uafiles.nsf/docs/QP30/$File/NA5D3G2J.pdf Quickplace 3.0 Installation and Upgrade Guide http://doc.notes.net/uafiles.nsf/docs/qp30/$File/NA5D3FZJ.pdf “New Features in QuickPlace 3.0” http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/869d d0fa3a87875e85256c290015d867?OpenDocument © Copyright IBM Corp. 2003. All rights reserved. 41 “QuickPlace 3 performance” http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/5076 cae0219bd20885256c78006bab36?OpenDocument “Comparing QuickPlace 3.0 with QuickPlace 2.0.8” http://www-10.lotus.com/ldd/today.nsf/71b8f499e6b9399885256b7d006b9611/2a73 8f04a34a537b00256c270062a8c1?OpenDocument Solaris Developer Connection The Solaris Developer Connection is full of technical articles and whitepapers that will help ensure successful deployment on the Solaris platform. Point your browser to: http://soldc.sun.com/ These specific LDD articles and documents were discussed within this paper: “Performance and Tuning on the Solaris™ 2.6, 7, and 8 Operating Environments” http://soldc.sun.com/articles/tuning_solaris.html “Performance Tools and Tuning Tips for Java™ Server Applications on the Solaris™ Operating Environment” http://soldc.sun.com/articles/performance_tools.html “Understanding and preventing system slowdowns” http://soldc.sun.com/articles/systemslowdowns.html “What does 100% busy mean” http://www.sun.com/sun-on-net/itworld/UIR990801perf.html 42 QuickPlace 3 for Sun Solaris 8 Back cover QuickPlace 3 for Sun Solaris 8 Features and benefits of running QuickPlace 3 on Solaris 8 This IBM Redpaper describes how to leverage the Sun Solaris 8 Operating Environment as a stable and scalable platform for the IBM Lotus QuickPlace 3.0 (Team Workplace) Web-based collaboration platform. Installation and setup hints and tips Topics covered in this Redpaper are: Performance tuning and testing - New features included with QuickPlace 3.0 for Solaris 8 - Basic installation and setup guidelines for QuickPlace 3.0 on Solaris 8 - General Solaris performance monitoring techniques appropriate for QuickPlace 3.0 - Performance tuning suggestions for optimizing a QuickPlace 3.0 install on Solaris 8 - Results of basic performance benchmarks demonstrating the scalability of QuickPlace 3.0 on Solaris 8 Since QuickPlace 3.0 executes on top of a basic Domino 5.0.10 server, readers unfamiliar with the Domino platform may want to first review the IBM Redbook Lotus Domino R5 for Sun Solaris 8, SG24-5969. ® Redpaper INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment. For more information: ibm.com/redbooks