3/28/11 12:03 AM
Rocks as Cluster Management
DongInn Kim
Graduate Student
Indiana University Bloomington
dikim@osl.iu.edu
Abstract
1. Introduction
Modern high performance
Setting up a cluster requires a lot of system administration
computing (HPC) is mainly
works including the installation and configuration of a lot of
fulfilled on the cluster
software on many computers connected with specially
environment. Several cluster
designed high bandwidth network. Rocks is a software
management software came out
package that is designed to simplify cluster installation. The
to make it easier to build
primary goal for Rocks is to make cluster installation as
clusters. Rocks is one of them.
easy as possible. Rocks include a collection of open source
In this paper I go through the
cluster software for a dedicated, high-performance cluster.
main features of Rocks and
In my survey paper, I examine the installation of Rocks in
analyze its pros and cons by
Section 2, introduce its main features in Section 3, and
comparing with other clustering
analyze its pros and cons by comparing other clustering
toolkits such as OSCAR (Open
toolkits in Section 4. In Section 5 I discuss my analysis of
Source Cluster Application
Rocks features and conclude my survey paper.
Resources) and Perceus.
Keywords
Cluster, Rocks, Clustering toolkit, OSCAR, Perceus
starts, the previous existing system is wiped out and a new
2. Installation
operating system (RedHat Enterprise Linux) is installed. The
I went through the Rocks installation a few years ago and it
new operating system is installed and all the selected
was pretty straightforward. I do not install Rocks this time
packages in Rocks are installed to setup the preferred roll’s
because the installation steps have not changed since then.
features with their default configuration. The frontend is
Instead, I examine the installation instructions posted at the
ready to serve as a head node in a cluster.
Rocks’ website. Rocks installation consists of two steps: 1)
Install Compute Nodes
Install and configure frontend 2) Install compute nodes.
The frontend is ready to find all the connected compute
Rocks requires the following minimum hardware to setup a
nodes in its subnet by running the “insert-ethers” command
cluster.
on the frontend. When a compute node is booted up with
•
•
Frontend Node
PXEboot (Preboot Execution Environment), its MAC address
o
Disk Capacity: 30 GB
shows up on the frontend and a new IP is assigned to the
o
Memory Capacity: 1 GB
compute node. Once the compute node is fully connected to
o
Ethernet: 2 physical ports (e.g., "eth0" and "eth1")
the frontend, it requests a kickstart file from the frontend
o
BIOS Boot Order: CD, Hard Disk
and it starts to install all the software in kickstart file. The
Computed Node
rest of compute nodes will be installed in the same manner.
o
Disk Capacity: 30 GB
o
Memory Capacity: 1 GB
3. Main Features
o
Ethernet: 1 physical port (e.g., "eth0")
Default installation of Rocks is very simple and goes
o
BIOS Boot Order: CD, PXE (Network Boot), Hard Disk
smoothly. Rocks assume that users deal with all the
software problems that may happen on a node by
Install and Configure frontend
reinstalling the system on that node rather than trying to
Users need to prepare all the required and preferred roll
figure out the issues and to fix them. Rocks supports
CDs for the installation of frontend. Once the installation
various HPC solutions such as Area 51, Bio, Ganglia, PVFS2,
Sun Grid Engline (SGE), Grid, and so on by providing its
own independent roll with CD. The Area 51 roll includes
tripwire and chkrootkit to have security enhancement in
Rocks. The Grid roll has a full complement of grid software,
such as Globus toolkit, Condor, and MPICH-G2. Especially
Rocks is well designed to support the grid computation.
Setting up a cluster is associated with the following features
that Rocks tries to satisfy all.
•
Various Linux distributions and architectures
•
Customizing the head node
o
User account management (synchronize access
through the whole system)
•
Figure 1. Rocks fundamental architecture.
o
Monitoring systems
o
Firewall to disallow the direct access to the client
One of main features in Rocks is the web-based
nodes
management tools which can provide a flexible access to
Customizing the client nodes
the management interface. Rocks tries to provide recently
o
Updating packages
many other features: new rolls to support more HPC
o
Changing disk partitions
solutions, Virtual LANs (VLANs), and Logical VLAN Bridges.
Rocks is tightly bound to RedHat Enterprise Linux (RHEL)
The command lines to manage a cluster are very useful. For
distribution and it supports most features that RHEL
example, the “sync” command with some proper options
provides. The RPM package management tool and Kickstart
can sync the configuration, DNS, firewall, networks, and
provisioning solution are the features that Rocks inherited
users automatically through the whole nodes.
from RHEL automatically.
As Figure 1 shows the main architecture of Rocks, all the
compute nodes are sitting behind the Ethernet Switch so
that the internal communication among the nodes is secure
As Table 1. shows, Rocks, OSCAR, and Perceus have their
and can be very fast depending on the network
own provisioning solutions and I can not tell which one is
infrastructure.
the best for the cluster management but I try to analyze
their features depending on their solution. Rocks’ flexibility
4. Pros and Cons
on the re-configuration of the system is low in comparison
I introduce two other clustering toolkits to compare Rocks.
to the others because Rocks does not allow the users to
OSCAR (Open Source Cluster Application Resources) and
tweak the systems to fix the problems but to re-install the
Perceus have been developed later than Rocks that started
problematic nodes to remove the issues. It is simple but not
its development in 2000 but they have played an enormous
many options are available for the users. Rocks is designed
role to improve the HPC world like Rocks. They are well
to the specific roll of features and it is completely stable for
equipped with their own features to compete Rocks.
the target features. The usability of Rocks is high enough
The OSCAR group started the OSCAR development in 2000
because it does not require much learning curve to setup a
and they released their first official software in 2002.
cluster like OSCAR. All three toolkits are not fully developed
OSCAR provides very flexible cluster deployment solution
to support the multiple clusters. It is possible for users to
but it is not designed to fully support for Grid and PVFS.
setup the multiple clusters by re-configuring the existing
Perceus is designed to provide the clustering provision
clusters that are built by the three toolkits. The easiness of
solution for the large quantities of machine in a scalable
maintenance of Rocks cluster is not perfect because it does
fashion. Perceus uses Virtual Node File System (VNFS) to
not provide a functional solution to update the running
enable the client nodes to run on either the stateless
client nodes without shutting down them or re-installing the
system or stateful system. The stateless system is the
new system on them. The existing systems would be
system that the operating system is installed on RAM or via
removed when Rocks is installed but OSCAR can keep the
the network file system. The normal system with the
existing head node only. On contrast to them, Perceus can
operating system installed on the hard disk is the stateful
keep all the nodes as they are because its client nodes can
system.
be deployed with VNFS. The ability to setup the
heterogeneous system on Rocks is not good because Rocks
because of its simplicity and stability in the cluster
works with only RHEL. Even with some drawbacks Rocks
installation and management.
has the highest market share in the HPC world. I think it is
Provisioning solution
Rocks
OSCAR
Perceus
Kickstart
SIS
VNFS
(System Installation Suite)
Flexibility
Low
Medium
High
Stability
High
Medium
Medium
Usability
High
High
Medium
Multiple clusters
Low
Low
Low
Medium
High
High
Low
Medium
High
Heterogeneous system
Low
High
High
Market share
High
Medium
Low
Maintainability
Re-usability of the
existing system
Table 1. Comparison among Rocs, OSCAR, and Perceus in terms of their features
5. Conclusions
clustering solutions. The production cluster does not really
In this paper, I identified analysis of features in Rocks by
care about the flexibility and heterogeneous system support
comparing with OSCAR and Perceus. Rocks is not the
but the stability and availability of the system. Rocks hits
perfect clustering toolkit but it has a high market share in
the edge and I believe that two other clustering toolkits
the HPC world because it is well designed for the specific
keep up the good works.
References
Joseph D. Sloan (2004): High Performance LINUX
CLUSTERS with OSCAR, Rocks, OpenMosix, and MPI
Gregory F. Pfister: In Search Of CLUSTERS (2nd Edition)
Rocks user manual in http://www.rocksclusters.org/
OSCAR user manual in
http://svn.oscar.openclustergroup.org/
Perceus user manual in http://www.perceus.org/
DongInn Kim, Jeffrey Squyres, Andrew Lumsdaine (2005):
Revamping the OSCAR Database: A Flexible Approach to
Cluster Configuration Data Management
DongInn Kim, Jeffrey Squyres, Andrew Lumsdaine (2006):
The Introduction of the OSCAR Database API (ODA)