3/28/11 12:03 AM Rocks as Cluster Management DongInn Kim Graduate Student Indiana University Bloomington dikim@osl.iu.edu Abstract 1. Introduction Modern high performance Setting up a cluster requires a lot of system administration computing (HPC) is mainly works including the installation and configuration of a lot of fulfilled on the cluster software on many computers connected with specially environment. Several cluster designed high bandwidth network. Rocks is a software management software came out package that is designed to simplify cluster installation. The to make it easier to build primary goal for Rocks is to make cluster installation as clusters. Rocks is one of them. easy as possible. Rocks include a collection of open source In this paper I go through the cluster software for a dedicated, high-performance cluster. main features of Rocks and In my survey paper, I examine the installation of Rocks in analyze its pros and cons by Section 2, introduce its main features in Section 3, and comparing with other clustering analyze its pros and cons by comparing other clustering toolkits such as OSCAR (Open toolkits in Section 4. In Section 5 I discuss my analysis of Source Cluster Application Rocks features and conclude my survey paper. Resources) and Perceus. Keywords Cluster, Rocks, Clustering toolkit, OSCAR, Perceus starts, the previous existing system is wiped out and a new 2. Installation operating system (RedHat Enterprise Linux) is installed. The I went through the Rocks installation a few years ago and it new operating system is installed and all the selected was pretty straightforward. I do not install Rocks this time packages in Rocks are installed to setup the preferred roll’s because the installation steps have not changed since then. features with their default configuration. The frontend is Instead, I examine the installation instructions posted at the ready to serve as a head node in a cluster. Rocks’ website. Rocks installation consists of two steps: 1) Install Compute Nodes Install and configure frontend 2) Install compute nodes. The frontend is ready to find all the connected compute Rocks requires the following minimum hardware to setup a nodes in its subnet by running the “insert-ethers” command cluster. on the frontend. When a compute node is booted up with • • Frontend Node PXEboot (Preboot Execution Environment), its MAC address o Disk Capacity: 30 GB shows up on the frontend and a new IP is assigned to the o Memory Capacity: 1 GB compute node. Once the compute node is fully connected to o Ethernet: 2 physical ports (e.g., "eth0" and "eth1") the frontend, it requests a kickstart file from the frontend o BIOS Boot Order: CD, Hard Disk and it starts to install all the software in kickstart file. The Computed Node rest of compute nodes will be installed in the same manner. o Disk Capacity: 30 GB o Memory Capacity: 1 GB 3. Main Features o Ethernet: 1 physical port (e.g., "eth0") Default installation of Rocks is very simple and goes o BIOS Boot Order: CD, PXE (Network Boot), Hard Disk smoothly. Rocks assume that users deal with all the software problems that may happen on a node by Install and Configure frontend reinstalling the system on that node rather than trying to Users need to prepare all the required and preferred roll figure out the issues and to fix them. Rocks supports CDs for the installation of frontend. Once the installation various HPC solutions such as Area 51, Bio, Ganglia, PVFS2, Sun Grid Engline (SGE), Grid, and so on by providing its own independent roll with CD. The Area 51 roll includes tripwire and chkrootkit to have security enhancement in Rocks. The Grid roll has a full complement of grid software, such as Globus toolkit, Condor, and MPICH-G2. Especially Rocks is well designed to support the grid computation. Setting up a cluster is associated with the following features that Rocks tries to satisfy all. • Various Linux distributions and architectures • Customizing the head node o User account management (synchronize access through the whole system) • Figure 1. Rocks fundamental architecture. o Monitoring systems o Firewall to disallow the direct access to the client One of main features in Rocks is the web-based nodes management tools which can provide a flexible access to Customizing the client nodes the management interface. Rocks tries to provide recently o Updating packages many other features: new rolls to support more HPC o Changing disk partitions solutions, Virtual LANs (VLANs), and Logical VLAN Bridges. Rocks is tightly bound to RedHat Enterprise Linux (RHEL) The command lines to manage a cluster are very useful. For distribution and it supports most features that RHEL example, the “sync” command with some proper options provides. The RPM package management tool and Kickstart can sync the configuration, DNS, firewall, networks, and provisioning solution are the features that Rocks inherited users automatically through the whole nodes. from RHEL automatically. As Figure 1 shows the main architecture of Rocks, all the compute nodes are sitting behind the Ethernet Switch so that the internal communication among the nodes is secure As Table 1. shows, Rocks, OSCAR, and Perceus have their and can be very fast depending on the network own provisioning solutions and I can not tell which one is infrastructure. the best for the cluster management but I try to analyze their features depending on their solution. Rocks’ flexibility 4. Pros and Cons on the re-configuration of the system is low in comparison I introduce two other clustering toolkits to compare Rocks. to the others because Rocks does not allow the users to OSCAR (Open Source Cluster Application Resources) and tweak the systems to fix the problems but to re-install the Perceus have been developed later than Rocks that started problematic nodes to remove the issues. It is simple but not its development in 2000 but they have played an enormous many options are available for the users. Rocks is designed role to improve the HPC world like Rocks. They are well to the specific roll of features and it is completely stable for equipped with their own features to compete Rocks. the target features. The usability of Rocks is high enough The OSCAR group started the OSCAR development in 2000 because it does not require much learning curve to setup a and they released their first official software in 2002. cluster like OSCAR. All three toolkits are not fully developed OSCAR provides very flexible cluster deployment solution to support the multiple clusters. It is possible for users to but it is not designed to fully support for Grid and PVFS. setup the multiple clusters by re-configuring the existing Perceus is designed to provide the clustering provision clusters that are built by the three toolkits. The easiness of solution for the large quantities of machine in a scalable maintenance of Rocks cluster is not perfect because it does fashion. Perceus uses Virtual Node File System (VNFS) to not provide a functional solution to update the running enable the client nodes to run on either the stateless client nodes without shutting down them or re-installing the system or stateful system. The stateless system is the new system on them. The existing systems would be system that the operating system is installed on RAM or via removed when Rocks is installed but OSCAR can keep the the network file system. The normal system with the existing head node only. On contrast to them, Perceus can operating system installed on the hard disk is the stateful keep all the nodes as they are because its client nodes can system. be deployed with VNFS. The ability to setup the heterogeneous system on Rocks is not good because Rocks because of its simplicity and stability in the cluster works with only RHEL. Even with some drawbacks Rocks installation and management. has the highest market share in the HPC world. I think it is Provisioning solution Rocks OSCAR Perceus Kickstart SIS VNFS (System Installation Suite) Flexibility Low Medium High Stability High Medium Medium Usability High High Medium Multiple clusters Low Low Low Medium High High Low Medium High Heterogeneous system Low High High Market share High Medium Low Maintainability Re-usability of the existing system Table 1. Comparison among Rocs, OSCAR, and Perceus in terms of their features 5. Conclusions clustering solutions. The production cluster does not really In this paper, I identified analysis of features in Rocks by care about the flexibility and heterogeneous system support comparing with OSCAR and Perceus. Rocks is not the but the stability and availability of the system. Rocks hits perfect clustering toolkit but it has a high market share in the edge and I believe that two other clustering toolkits the HPC world because it is well designed for the specific keep up the good works. References Joseph D. Sloan (2004): High Performance LINUX CLUSTERS with OSCAR, Rocks, OpenMosix, and MPI Gregory F. Pfister: In Search Of CLUSTERS (2nd Edition) Rocks user manual in http://www.rocksclusters.org/ OSCAR user manual in http://svn.oscar.openclustergroup.org/ Perceus user manual in http://www.perceus.org/ DongInn Kim, Jeffrey Squyres, Andrew Lumsdaine (2005): Revamping the OSCAR Database: A Flexible Approach to Cluster Configuration Data Management DongInn Kim, Jeffrey Squyres, Andrew Lumsdaine (2006): The Introduction of the OSCAR Database API (ODA)