PLUG-IN FOR SINGLETON SERVICE IN CLUSTERED ENVIRONMENT
AND
IMPROVING FAILURE DETECTION METHODOLOGY
Srinivasa Chakravarthy Kodali
B.E., Nagarjuna University, India, 2007
PROJECT
Submitted in partial satisfaction of
the requirements for the degree of
MASTER OF SCIENCE
in
COMPUTER SCIENCE
at
CALIFORNIA STATE UNIVERSITY, SACRAMENTO
SPRING
2011
PLUG-IN FOR SINGLETON SERVICE IN CLUSTERED ENVIRONMENT
AND
IMPROVING FAILURE DETECTION METHODOLOGY
A Project
by
Srinivasa Chakravarthy Kodali
Approved by:
, Committee Chair
Chung-E-Wang, Ph.D.
, Second Reader
Ahmed Salem, Ph.D.
Date
ii
Student: Srinivasa Chakravarthy Kodali
I certify that this student has met the requirements for format contained in the University
format manual, and that this project is suitable for shelving in the Library and credit is to
be awarded for the project.
, Graduate Coordinator
Nikrouz Faroughi, Ph.D.
Date
Department of Computer Science
iii
Abstract
of
PLUG-IN FOR SINGLETON SERVICE IN CLUSTERED ENVIRONMENT
AND
IMPROVING FAILURE DETECTION METHODOLOGY
by
Srinivasa Chakravarthy Kodali
In the time of social networking, number of users accessing web applications has
reached millions. As a result the user load on web applications has increased, resulting in
the evolution of different technologies to manage this increase. One such technology is
clustering web application i.e. a large number of servers aka cluster nodes, each hosting
the same web application. Thus, creating a distributed web application where the user
load will be distributed among each of the cluster nodes. However while running an
application in a clustered environment different challenges arise, one of which is
handling singleton objects.
Singleton object is an instance of a class when only one such object is created per
application. In a clustered environment each node runs the same web application creating
its own singleton object as it requires. Thus each of the cluster nodes aka servers, ends up
with one singleton object each. This surplus of singleton objects will cause unnecessary
system overload or repetitive actions of the web application. One way of addressing this
problem is to create a singleton service for the entire cluster which can provide access to
singleton objects for each member of the cluster as and when needed.
iv
The goal of this project is to design a plug-in which can automatically create a
singleton service in any clustered web application. The plug-in should be capable of
starting and stopping singleton service in any one node of the cluster. Once a node
imbibes the singleton service it can cater to all singleton object requirements of the entire
cluster. Since only one node acts as a singleton service it is more susceptible to be a
single point of failure. The way of fixing this problem is by designing the plug-in in such
a way that it detects the failure and immediately chooses another node to become the
singleton service. Therefore providing a singleton service i.e. highly available for
clustered web applications.
, Committee Chair
Chung-E-Wang, Ph.D.
Date
v
DEDICATION
Affectionately Dedicated to
Respected Parents & Teachers
vi
ACKNOWLEDGEMENTS
I take this as a wonderful opportunity to thank Dr. Chung-E Wang and Dr. Ahmed
Salem who gave me the opportunity to work under their guidance. Especially, I would
like to thank professor Wang for his valuable insights and thoughts regarding the project.
I also thank my family and friends for their support and cooperation.
vii
TABLE OF CONTENTS
Page
Dedication ..........................................................................................................................vi
Acknowledgements ........................................................................................................... vii
List of Figures .................................................................................................................... x
Chapter
1. INTRODUCTION ......................................................................................................... 1
2. BACKGROUND .......................................................................................................... 4
2.1 Clustering Implementation............................................................................... 4
2.2 Singleton Service ............................................................................................. 5
3. DESIGN SPECIFICATION .......................................................................................... 7
3.1 System Overview ............................................................................................. 7
4. IMPLEMENTATION .................................................................................................. 11
4.1 Cluster Configuration...................................................................................... 11
4.2 Apache HTTP Load Balancer ......................................................................... 12
4.3 Database Connection Pooling ......................................................................... 14
4.4 Building Singleton Service ............................................................................. 15
4.4.1 RMI Service Configuration........................................................... 15
4.4.2 Plug-in Thread .............................................................................. 16
5. FAILURE DETECTION METHODOLOGY.............................................................. 19
6. CONCLUSION AND PROSPECTS FOR IMPROVEMENT .................................... 22
viii
6.1 Conclusion ...................................................................................................... 22
6.2 Prospects for Improvement ............................................................................. 22
Appendix Source Code ..................................................................................................... 24
Bibliography ..................................................................................................................... 57
ix
LIST OF FIGURES
Page
1.
Figure 2.1 Basic Clustering Architecture ................................................................... 4
2.
Figure 3.1 Plug-in Development Environment .......................................................... 8
3.
Figure 5.1 Application Server Cluster ....................................................................... 19
x
1
Chapter 1
INTRODUCTION
Of the 6.8 billion people in the world today there are around 2 billion internet
users. The internet provides a platform for users to pursue diverse functions ranging from
day to day chores (bills, e-commerce etc.,), business, education, research, networking to
entertainment. Most of these functions are bought to the users by the use of web
applications, which are applications that are accessed over the internet or intranet. Each
web application has a limit on the number of users it can handle at a given time. In
today’s world with the increased number of internet users most web applications are over
loaded in no time, therefore reducing their efficiency.
The two major reasons for reduction in efficiency are either due to lack of
resource availability in the server or bad coding practices. Problems related to coding are
either due to bad design practices or badly written code. Both of which can be resolved
by following the correct software design practices or by fixing the code where ever
necessary. When it comes to problems related to resource availability in the server they
are caused either by one or both of the following; the number of concurrent threads that
the server can handle, the number of socket connections that it can establish etc. The
solution to this is by clustering web application into n number of servers aka, cluster
nodes which are similar in functionality.
Clustering is one of the technologies used to make web applications scale to
millions of users. When clustering of web application is done, as the number of usage
2
instance increases, the load is distributed to each node. This system is called a distributed
system. Designing such a system comes with its own set of challenges in making the
entire system work in a manner similar to a single system. Some of which are handling
user sessions, load balancing, handling singleton objects, etc.
Handling singleton objects in a clustered environment is one of the important
challenges in a distributed system. As per definition of singleton object, there should be
only one instance (aka singleton object) of the class in a complete system. But as the
application is clustered, each cluster node holds its own singleton object. One of the
solutions to this problem is having one node provide access to all singleton objects in a
cluster. This service which provides all singleton objects to all the nodes in the cluster is
called singleton service.
Any application that needs to be clustered and contains singleton objects should
implement singleton service. Therefore every application that is similar to above needs to
develop a singleton service. It is a waste of development cycles to design a distinctive
singleton service for each web application. The goal of this project is to design a code
(plug-in) that is universal, but specific in its function of creating a singleton service in
any clustered environment. With the current web application usage trend this
functionality of the plug-in, in automatically creating a singleton service in any clustered
environment is highly desirable.
A major problem in using this plug-in is that the singleton service runs in only
one node of a cluster, therefore making it a single point of failure. If this node fails for
3
any reason, singleton objects will not be available to any node in the cluster. Thus, the
plug-in should also be capable of starting a singleton service in another node of the
cluster if the node running singleton service fails. By resolving this issue we get a
comprehensive and highly effective singleton service plug-in for clustered web
application.
4
Chapter 2
BACKGROUND
2.1 Clustering Implementation
A cluster is a group of computer systems connected together to provide a solution
or to solve a problem. Computers in a cluster are called nodes.
Application servers
Node-1
L
O
A
D
Presentation
server
B
A
L
A
N
C
E
R
Node-2
Node-3
Figure 2.1 Basic Clustering Architecture
Figure 2.1 shows the basic clustering architecture with a cluster of application
servers. The presentation server is a system that executes user interface related tasks and
the application server is the location where business logic is executed. The presentation
server makes calls to the load balancer, and the load balancer holds some information
5
about the number of application nodes available and their location. This information is
stored in configuration files. Based on the configuration, calls are redirected to
application servers and these servers process the calls and return with a response. Thus,
the load balancer is responsible for distributing load to the different nodes in the cluster.
There are two ways of clustering

Vertical clustering: When multiple instances of the server are run in a single
physical machine, it is called vertical clustering. A drawback of this
implementation is that resources of a single system are shared by multiple
nodes in a cluster.

Horizontal clustering: Running each node of the cluster in its own physical
machine is called horizontal clustering. This is a very expensive solution
because as the nodes increase, the number of physical machines to maintain
also increases.
An effective clustering mechanism is to use both ways. However in this project
only vertical clustering has been used to distribute the load.
2.2 Singleton Service
Singleton objects are single instances of class maintained in a complete
application. These objects are made globally accessible. Singleton objects are static
objects which are instantiated in the static function of the class. Below is the snippet of
code that shows how singleton objects are created:
6
private static instance = null;
public void static getInstance() {
if(instance == null) {
instance = new SingletonObject();
}
return instance;
}
Singleton service is the node in the cluster that provides access of singleton objects to all
the nodes in cluster.
7
Chapter 3
DESIGN SPECIFICATION
3.1 System Overview
A 3-tiered sample web application is needed to create and test a plug-in for
singleton service. As a result the web application is divided into a presentation server, an
application server and a database. The application server is the one that holds the
business logic and the presentation server holds the user interface code and makes http
calls to the application server. As the number of http calls made to the application server
increases, the performance of the application slows down. The only solution is to cluster
the application server so that all the http calls are distributed among the application server
cluster. Apache Tomcat 6.0 is the Servlet container, which is where the application server
and presentation server codes are executed.
Figure 3.1 is the design diagram of the environment used to build the plug-in. The
presentation code is loaded into the browser as I am using javascript and html. From
javascript ajax calls are made to the Apache 2.2 http web server, which acts as a load
balancer with some configuration changes and with the new module called mod_jk. This
can redirect the calls from the load balancer to each tomcat instance that contains the
application server code and, in turn, writes the data from the nodes to the database.
8
Tomcat Instances
L
O
A
D
Browser
B
A
L
A
N
C
E
R
Node-1
MySQL database
Database
Node-2
Node-3
Figure 3.1 Plug-in Development Environment
There are two kinds of load balancers:

Software Load Balancer: A software program that listens to the http calls at a
particular port and redirects calls to the application servers. There are different
strategies in choosing an application server to which http calls should be
redirected. Some of the strategies in selecting an application server are, using
round robin technique or the number of connections already given per node.

Hardware Load Balancer: A physical device that splits all the network calls
across multiple application server nodes. These use strategies similar to those
mentioned above to select the application server.
9
The performance of the hardware load balancer is better when compared to the
software load balancer. In this project, a software load balancer is being used
as it is cost effective and easy to configure.
Singleton service should expose a set of methods that provide singleton objects to
all the nodes in the cluster. One of the best implementations is to make singleton service
as RMI (remote method invocation) service, as it is a core java functionality and works in
all systems that use java. By using this service, all nodes in the cluster can make a remote
call to get access to the singleton objects. But we need to let all the nodes know where
RMI service is currently running by specifying the path of RMI service. In a cluster,
global information can be stored in a shared file system, database, or some cache. So, if
any node wants to access the singleton objects, it needs to look for the RMI service
location and call a particular method in the service.
Initially, no node will be running the RMI service. Server initialization needs to
look up the RMI location and ping that server to determine whether RMI service is
located in a specified location and if the specified location is not available, then the server
should start RMI service in any of the live nodes. Initialization of the RMI service and
verification of whether RMI service is running needs to be synchronized as two or more
servers might be doing the same thing and start RMI service in two different nodes.
If a node running RMI service fails, the application needs to start RMI service in
another node in the cluster. To do this, a ping servlet is used which pings the machine
running the RMI service. This is done by all the nodes in the cluster. If the ping to the
10
node running RMI service fails, it means RMI service is down. We need to reinstate the
initialization of RMI service, which starts a new RMI service in another node. This
makes sure that singleton service is highly available.
11
Chapter 4
IMPLEMENTATION
4.1 Cluster Configuration
As the application is running in a tomcat container, we need to cluster the tomcat
server, which holds the application server code. Every J2EE container has its own
clustering implementation. The tomcat cluster configuration is one of the simplest
configurations of all J2EE containers.
The configuration file that holds the entire configuration variables like ports and
protocols used is in the tomcat container server.xml. As we are implementing vertical
clustering, we create multiple tomcat instances in the same machine where each server
holds the same application server code. All the port conflicts need to be fixed as all
tomcat instances will be running in a single machine. A few more configuration steps are
needed to enable clustering in the tomcat J2EE container:

Step 1: Add the following line in server.xml to help enable communication
between the load balancer and application servers:
<Connector port="8109" protocol="AJP/1.3" redirectPort="8443" />
Apache Jserv Protocol (AJP) is used to communicate between the load balancer
and the application server over TCP connections. This reduces the socket creation time
and keeps the TCP connection live until the request handling cycle is terminated. This is
similar to connection pooling used in a database.
12

Step 2: The line below is responsible for setting up a cluster and provides
callers with a valid multicast receiver/sender:
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>

Step 3: In all the three application server’s web.xml file, add the following
line:
<distributable />
This makes sure the code is suitable to run in a clustered environment. Hence, all new
user sessions and changes to existing user sessions will be replicated in the other
members of the cluster.

Step 4: Add jvmRoute property to the Engine tag in server.xml:
<Engine name=”Catalina” defaultHost=”localhost” jvmRoute=”tomcatA”>
JvmRoute acts as an identifier to the tomcat instances. It will be used by the load balancer
to identify the tomcat worker.
By incorporating the above changes, the tomcat cluster will be available to accept
calls from the load balancer given that a few configuration changes in the load balancer
are made.
4.2 Apache HTTP Load Balancer
Apache Http server loaded with mod_jk is used as a software load balancer for the
tomcat instances. Mod_jk is the connector that establishes communication between the
13
apache http server and tomcat servers. All the configurations related to apache http server
need to be added to the httpd.conf file. The configuration steps are:

Step 1: Add latest Mod_jk binary file into the modules folder of http server

Step 2: Add the following lines to httpd.conf file:
1) LoadModule jk_module modules/mod_jk-apache-2.2.4.so
This action helps in loading the mod_jk file on the http server when it is started.
2) JkWorkersFile=” C:\Program Files (x86)\Apache Software
Foundation\Apache2.2\conf\workers.properties”
Helps in specifying the location of workers.properties file.
3) JkMount /chatAppServer/* loadbalancer
All the calls from the presentation server to the application server are redirected to
the loadbalancer property, which is configured in workers.properties file.

Step 3: Add the following lines to the workers.properties file
worker.tomcatA.port=8109
worker.tomcatA.host=localhost
worker.tomcatA.type=ajp13
This identifies the port, the host of the tomcat instance and the protocol, which
specifies communication between the http server and the tomcat instance. A
similar set of configuration values need to be added for each tomcat instance in
the cluster.
14
4.3 Database Connection Pooling
MySQL database has been used to store the configuration details and all state
information of the singleton service. A database connection is established using the
mysql jdbc connector. There are two ways to establish a database connection and to
query the database. One of them is to open a connection every time we need information.
This can be expensive due to opening of a socket connection and checking credentials
every time a connection is made.
The other way of opening a database connection is to use database connection
pooling (DBCP). In DBCP, a list of connections can be maintained to connect to a
database, these connections can be reused every time a connection is needed. One of the
popular types of database connection pooling is tomcat database connection pooling. For
tomcat to establish a connection pooling datasource, a resource needs to be specified in
conext.xml. The code below makes sure that connection pooling is established for the
application:
<Resource name="jdbc/chatApp" auth="Container" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000" username="root"
password="passenger" driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/chatapplication"/>
JNDI lookup is used to get a connection to the database from the pool of
connections.
15
4.4 Building Singleton Service
Now that the application server is clustered and configured with a load balancer,
the system is ready to implement singleton service and test it. A sample chat application
is built, which creates singleton objects and the singleton service is built for these objects.
There are two major parts for creating singleton service. One is configuring RMI service
in a web application and the other one is implementing a thread that initializes the
singleton service.
4.4.1 RMI Service Configuration
Remote method invocation (RMI) is the core java function used for invoking
methods of remote java objects. Thus RMI service is used to access singleton object
methods running in different nodes in the cluster.
All singleton object creation or availability is provided by one class which
extends a Remote java class. This Remote java class makes sure that these methods are
available to the external JVMs. For example in chat application, the http request that
creates a new thread, based on roomId will be done in this class, which extends Remote.
Also, all the chat entries related to that particular room go through this Remote class. So
before calling these methods from any location, we should know where remote service is
running (host address) and the port number of the RMI service which is available in the
database. With this remote service host location and port number we can get access to the
RMI registry.
16
Initialization of the RMI service is done when servers are starting up via the
singleton service plug-in thread. This initialization creates RMI registry, which can be
accessed by all nodes. The RMI registry looks up the remote class object by acquiring the
key that the object is bound to. Once acquired the object can be used to call its methods.
4.4.2 Plug-in Thread
One of the Singleton service tasks is to initialize RMI service in a single node in
the cluster. It is also capable of starting and stopping the RMI service. On the startup of
each server, a servlet thread is initialized in all the nodes in the application responsible
for singleton service tasks. This thread looks into the database for the location of RMI
service and pings the service and checks to see if it is live. If the service is already live,
servlet will not initialize RMI service in any node. If the service is not running in any
node, a thread will initialize RMI service randomly in any node of the cluster.
Initialization of RMI service is done by a plug-in thread calling a servlet that
binds the class, which extends Remote to some key. This is the same key used to retrieve
the object from the RMI registry. A plug-in thread call will pass the port address which is
available in configuration so that RMI is initialized at that port address. Below is a
snippet of code used to bind an object to RMI service:
Public InitRemoteService( int port) {
try {
service = new RemoteServiceImpl();
17
IRemoteService stub = (IRemoteService)
UnicastRemoteObject.exportObject(service, 0);
Registry registry = LocateRegistry.createRegistry(port);
Registry.bind(“singletonSerive”, stub);
}
Catch (Exception e) {
e.printStackTrace();
}
}
The above piece of code will bind the remoteServiceImpl object to a key used for
look-up in the registry.
One of the problems is if all the servers are started at the same time, all the nodes
try to initialize remote service in some location creating multiple RMI services in the
cluster. To avoid this problem, we need to add a distributed lock on a piece of code that
initializes the start of singleton service. There are lots of distributed locking mechanisms
currently in the industry. One of the simple locking mechanisms is Hazelcast distributed
locking. Hazelcast is an open source jar, which provides solutions to distribution
problems. Once the Hazelcast jar is included and all the nodes are configured into the
configuration.xml, we get access to all the distributed solutions provided by Hazelcast.
Below is a snippet of code that shows how distributed locking is implemented:
Lock lock = Hazelcast.getLock(obj);
18
lock.lock();
try {
… ….
}finally {
lock.unlock();
}
This piece of code locks the try block if the same class in some other node holds a
lock on the object. Once the lock is released from the node holding it, the next node that
requested for the lock gets a hold of the lock and goes on. But once the RMI service is
initialized by the first node, the database is updated with the location at which RMI
service is running. This makes sure a new RMI service cannot be initialized in other
nodes, as every time before RMI service is initialized we look to see whether service is
running fine. Therefore at any give point of time only one RMI service is running in the
cluster.
19
Chapter 5
FAILURE DETECTION METHODOLOGY
If the node running the singleton service fails, no node has access to singleton
objects in the cluster; thus, it is a single point of failure for the application. To overcome
a failure of a node running singleton service, a plug-in should automatically detect the
failure and start singleton service in another node. This ensures that singleton service is
highly available.
Node - 2
Node-1
Node3
Node-4
Figure 5.1 Application Server Cluster
Let us say we have four application server nodes running in a cluster and node-2
running singleton service dies because of network failure or for any other reason. The
20
nodes in the cluster should be able to pick up another node as the singleton service. In a
clustered environment, the simplest way to find whether a node is running or not is by
pinging the nodes in small intervals of time. The same is applied to find out whether
singleton service is running or not. Each node running in the cluster will ping the node
running the singleton service. This ensures that even if two or more nodes fail, singleton
service is always available. As we are already saving the information regarding where the
singleton service is running, each node gets access to this information and starts pinging
the singleton service URL.
For implementation of the ping servlet, the same servlet thread that starts the RMI
service is used. Thus, the thread in this servlet will see whether singleton service is
running every 2 seconds. Below is the snippet of code that pings the given url:
U = new URL(url);
Uc =u.openconnection();
is = new InputStreamReader(uc.getInputStream());
bin = new BufferedReader(is);
bin.readLine();
The code above opens a URL connection for the provided singleton service url and
returns some data back from the provided url. This proves that the provided server is live
and available.
If one node figures out that the singleton service node is not responding to the
ping, then that thread will get the Hazelcast distributed lock on the code and start
21
singleton service in some random available node. Once the RMI service is started, the
database is updated with host location, and the port where the singleton service is
running. While starting the singleton service in a new node due to service failure, some
state information can be passed on to the new singleton service if required. By
implementing this ping servlet we ensure that one node in the cluster will always be
running the singleton service. Therefore making the singleton service highly available.
22
Chapter 6
CONCLUSION AND PROSPECTS FOR IMPROVEMENT
6.1 Conclusion
From the conception of this project, the aim was to develop a singleton service
plug-in that is highly available and not restricted to a single J2EE container. During the
development process many issues (which have been elaborated in previous chapters)
were resolved so as to achieve the above goal. This was done using open source
software’s like tomcat container, apache http load balancer and Hazelcast. However, the
plug-in is independent of any of these software’s, as it is built using RMI, servlets and
core java functionalities except for Hazelcast, which is used for the distributed locking
system. The above singleton service plug-in works well for all large-scale as well as
small-scale J2EE applications. Whereas this is not the case with other singleton service
providers like JBOSS and Weblogic, their singleton services are tightly tied up with their
containers and cannot be used in other J2EE containers and are not open source.
6.2 Prospects for Improvement

A distributed JNDI look-up can be implemented by the plug-in to look up
service information such as, which cluster node is running, ports, and
configurations.
23

Failure detection methodology can be improved because in the current
implementation, each node in the cluster pings the singleton service node,
which causes unnecessary traffic between nodes.

When a singleton node dies, a plug-in should be capable of passing state
information of the failed node to a new singleton service node.
24
APPENDIX
Source Code
/**
*The plug-in class responsible for starting singleton service. Ping servlet is also part of
*this class which checks whether the singleton node is running in the cluster
*environment.
*/
package plugin;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Serializable;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.rmi.AccessException;
import java.rmi.NotBoundException;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
25
import java.rmi.registry.Registry;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.locks.Lock;
import com.hazelcast.core.Hazelcast;
import rmi.IRemoteService;
import chatApp.DBconnection;
public class PingJVMThread extends Thread {
private String remoteServiceUrl = null;
private SampleLock obj = new SampleLock();
public static void startThread() {
new PingJVMThread().start();
26
}
public void init() {
this.setName("pingjvmthread");
}
/**
* Infinite loop which keeps on pinging the server which is running the singleton remote
*service in cluster check if the "/ping" returning "alive" else get distributed lock to start a
* new singleton remote service. if some one already got the lock update
*"remoteServiceUrl" with new location where remote service is running
*/
public void run() {
init();
while(true) {
//TODO: remoteServiceUrl - this is server which is running remote
service -populate this from configuration file
Lock lock = Hazelcast.getLock(obj);
lock.lock();
try{
remoteServiceUrl = readRemoteServiceUrl();
27
if(remoteServiceUrl == null || remoteServiceUrl.length() == 0 ||
!makeUrlConnection(remoteServiceUrl+"/ping").equals("alive") || getRemoteService()
== null) {
try {
remoteServiceUrl = startRemoteServiceInCluster();
} catch (Exception e) {
System.err.println("remote service is already running in cluster");
e.printStackTrace();
}
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} finally {
lock.unlock();
}
}
}
28
/**
* start a new singleton remote service in cluster
* See if any jvm is alive and check whether remove service on that machine is
already running
* if not call "startremoteservice" servlet on that JVM
* @throws Exception
*/
private String startRemoteServiceInCluster() throws Exception{
String runningJVM = null;
//read from configuration location list of jvms running
Map<String, Integer> JVMnRMIMap = new HashMap<String, Integer>();
JVMnRMIMap.put("http://localhost:8081/chatAppServer", 9345);
JVMnRMIMap.put("http://localhost:8082/chatAppServer", 9346);
JVMnRMIMap.put("http://localhost:8083/chatAppServer", 9347);
//check whether jvm is alive
Set<String> jvms = JVMnRMIMap.keySet();
for(String jvmuri : jvms) {
if(makeUrlConnection(jvmuri+"/ping").equals("alive")){
runningJVM = jvmuri;
break;
29
}
}
//if jvm is live check whether remoteservice is running in that location
IRemoteService service = (IRemoteService)rmiLookUp(null,
JVMnRMIMap.get(runningJVM), IRemoteService.serviceName);
if(service == null) {
makeUrlConnection(runningJVM+"/startremoteservice?port="+JVMnRMIMap.g
et(runningJVM));
} else {
throw new Exception();
}
//TODO:update configuration where rmi service is running
//write to db
updateRemoteServiceUrl(runningJVM,
JVMnRMIMap.get(runningJVM));
getRemoteService().startReadThread(0);
return runningJVM;
}
private String makeUrlConnection(String url) {
30
String inputLine = "dead";
URLConnection uc;
BufferedReader bin = null;
InputStreamReader is = null;
InputStream inputstream = null;
URL u = null;
try {
u = new URL(url);
uc = u.openConnection();
inputstream = uc.getInputStream();
is =new InputStreamReader(inputstream);
bin = new BufferedReader(is);
inputLine = bin.readLine();
bin.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}
return inputLine;
}
private Object rmiLookUp(String url, int port, String serviceName) {
31
Registry registry;
try {
registry = LocateRegistry.getRegistry(url, port);
return registry.lookup(serviceName);
} catch (AccessException e) {
e.printStackTrace();
} catch (RemoteException e) {
System.out.println("Unable to find RMI service at port:" + port);
} catch (NotBoundException e) {
e.printStackTrace();
}
return null;
}
private String readRemoteServiceUrl() {
Connection con = DBconnection.getConnection();
String url = null;
try {
ResultSet rs = con.createStatement().executeQuery("SELECT
rmiurl FROM CONFIGURATION");
while(rs.next()) {
32
url = rs.getString(1);
}
} catch (SQLException e) {
e.printStackTrace();
}
DBconnection.releaseConnection(con);
return url;
}
private void updateRemoteServiceUrl(String url, int port) {
Connection con = DBconnection.getConnection();
try {
PreparedStatement preparedStmt = con.prepareStatement("update
configuration set rmiurl=?, port=? where id=1");
preparedStmt.setString(1, url);
preparedStmt.setInt(2, port);
preparedStmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
DBconnection.releaseConnection(con);
33
}
private IRemoteService getRemoteService() {
Connection con = DBconnection.getConnection();
//TODO:Read from configuration where rmi is running. port is hard coded
for now
IRemoteService service = null;
try{
int port = 9345;
ResultSet rs = con.createStatement().executeQuery("SELECT port
FROM CONFIGURATION");
while(rs.next()) {
port = rs.getInt(1);
}
Registry registry = LocateRegistry.getRegistry(null,port);
service = (IRemoteService)
registry.lookup(IRemoteService.serviceName);
} catch (Exception e) {
System.err.println("Remoteservice exception: RMI is not
running");
34
}
DBconnection.releaseConnection(con);
return service;
}
}
/* Class responsible for initializing RMI service in a particular cluster node.*/
package rmi;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.rmi.server.UnicastRemoteObject;
public class InitRemoteService {
public static IRemoteService service;
public InitRemoteService(int port){
try {
service = new RemoteServiceImpl();
IRemoteService stub =
(IRemoteService) UnicastRemoteObject.exportObject(service, 0);
Registry registry = LocateRegistry.createRegistry(port);
35
registry.rebind(IRemoteService.serviceName, stub);
System.out.println("Remote service bound");
} catch (Exception e) {
System.err.println("Remote service exception:");
e.printStackTrace();
}
}
}
/*Servlet responsible for calling to initialize the remote service object*/
package servlets;
import javax.servlet.ServletConfig;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import rmi.InitRemoteService;
public class StartRemoteService extends HttpServlet{
@Override
36
public void init(ServletConfig config) {
}
@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp){
new InitRemoteService(Integer.parseInt(req.getParameter("port")));
}
@Override
public void doPost(HttpServletRequest req, HttpServletResponse resp) {
doGet(req, resp);
}
}
/*Class returning a response that the cluster node is live */
package servlets;
import java.io.IOException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
37
import javax.servlet.http.HttpServletResponse;
import plugin.PingJVMThread;
/**
* (1). This servlet is responsible for pinging all the JVM in a cluster and starting a thread
on the process of servlet to do this.
* (2). Provide a servlet method (doGet) that responds to all the ping calls coming from
other instances
* @author skodali
*
*/
public class PingServlet extends HttpServlet{
@Override
public void init(){
PingJVMThread.startThread();
}
@Override
38
public void doGet(HttpServletRequest req, HttpServletResponse res) {
try {
res.getWriter().print("alive");
} catch (IOException e) {
e.printStackTrace();
}
}
public void doPost(HttpServletRequest req, HttpServletResponse res) {
doGet(req, res);
}
}
/* RMI service interface. These methods are called for any in the cluster. Thus, these
methods are *available only through remote service
*/
package rmi;
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface IRemoteService extends Remote{
39
public final String serviceName = "RMIService";
public void startReadThread(int roomId) throws RemoteException;
public void stopReadThread() throws RemoteException;
public void enterChat(String username, String chat, int roomId) throws
RemoteException;
}
/*
*RMI service interface implementation where the singleton objects are initialized.
*/
package rmi;
import java.rmi.RemoteException;
import java.util.Random;
import chatApp.ReadChatsThread;
import chatApp.WriteChats;
40
public class RemoteServiceImpl implements IRemoteService {
@Override
public void startReadThread(int roomId) throws RemoteException{
ReadChatsThread.getInstance().startReadThread(roomId);
}
@Override
public void stopReadThread() throws RemoteException{
//TODO: see how we can stop thread
}
@Override
public void enterChat(String username, String chat, int roomId) throws
RemoteException {
WriteChats.getInstance().write(username, chat, roomId);
}
}
/* Chat example restful webservices class. All the methods is this class are available as
*webservice methods to the external world
*/
41
package restfulWS;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.sql.Connection;
import java.sql.ResultSet;
import java.util.Random;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import chatApp.DBconnection;
import rmi.IRemoteService;
42
@Path("/chatapp")
public class ChatWS{
/**
*All members who hit join will call this method, user will be subscribed to the
channel where
*chats will be pushed to. This method creates a new room or a new read Thread
* @throws RemoteException
*/
@GET
@Path("/createRoom")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public String createRoom() throws RemoteException {
getRemoteService().startReadThread(0);
//TODO: subscribe to the channel for pushing messages
return "true";
}
/**
43
* members entering room number will be automatically pushed into a currently
running thread
* he or she will receive chats related to that room.
* @throws RemoteException
*/
@GET
@Path("/joinroom")
@Consumes(MediaType.APPLICATION_JSON)
public void joinRoom(@QueryParam(value="roomid") int roomId) throws
RemoteException{
//TODO: subscribe user to particular roomid and he or she will start
receiving chats related to that room
}
/**
* write entered chats to DB
* @throws RemoteException
*/
@GET
@Path("/enterchat")
@Consumes(MediaType.APPLICATION_JSON)
44
@Produces(MediaType.APPLICATION_JSON)
public String enterChat(@QueryParam(value = "username") String username,
@QueryParam(value = "chat") String chat, @QueryParam(value="roomid") int roomId)
throws RemoteException {
getRemoteService().enterChat(username, chat ,roomId);
return "true";
}
/**
* looks up RMI service and returns its object
* @return IRemoteService Object
*/
public IRemoteService getRemoteService() {
Connection con = DBconnection.getConnection();
//TODO:Read from configuration where rmi is running. port is hard coded
for now
IRemoteService service = null;
try{
int port = 9345;
ResultSet rs = con.createStatement().executeQuery("SELECT port
FROM CONFIGURATION");
45
while(rs.next()) {
port = rs.getInt(1);
}
Registry registry = LocateRegistry.getRegistry(null,port);
service = (IRemoteService)
registry.lookup(IRemoteService.serviceName);
} catch (Exception e) {
System.err.println("Remoteservice exception:");
e.printStackTrace();
}
DBconnection.releaseConnection(con);
return service;
}
}
/*Chat example class which Write chats to database
*/
package chatApp;
import java.sql.Connection;
46
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.text.SimpleDateFormat;
import java.util.Date;
public class WriteChats {
private static final String writeQuery = "INSERT INTO chatapplication.chatdata
VALUES (?, ?, ?, ?)";
private static WriteChats instance = null;
public static WriteChats getInstance() {
if(instance == null) {
instance = new WriteChats();
}
return instance;
}
public void write(String username, String chat, int roomId) {
try {
Connection connection = DBconnection.getConnection();
47
PreparedStatement pstmt1 =
connection.prepareStatement(writeQuery);
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd
hh:mm:ss");
String messageUserName =
username+System.currentTimeMillis();
String message = chat+System.currentTimeMillis();
pstmt1.setString(1, sdf.format(new Date()));
pstmt1.setString(2, messageUserName);
pstmt1.setString(3, message);
pstmt1.setInt(4, roomId);
pstmt1.executeUpdate();
DBconnection.releaseConnection(connection);
} catch(SQLException e) {
e.printStackTrace();
}
}
}
/*Chat example thread responsible for reading the chats from the database and pushing
them to UI
*/
48
package chatApp;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.Date;
public class ReadChatsThread {
private static ReadChatsThread instance = null;
public static ReadChatsThread getInstance() {
if(instance == null) {
instance = new ReadChatsThread();
}
return instance;
}
public void startReadThread(int roomId) {
49
new readThread(roomId).start();
}
/**
*
* thread responsible for reading chat entries for every 1000 ms.
* create a json string for each call and PUSH to web UI
*
* @author skodali
*/
public class readThread extends Thread{
private static final String readQuery = "SELECT * FROM
chatapplication.chatdata WHERE ID >= ? AND ROOMID = ?";
private static final String readLastPopulate = "SELECT lastpopulated
FROM chatapplication.lastpopulated";
private static final String writeLastPopulate = "update
chatapplication.lastpopulated set lastpopulated = ? where id=1";
private static final long READ_WAIT_TIME = 1000;
Connection connection = null;
PreparedStatement pstmt1 = null;
50
PreparedStatement pstmt2 = null;
PreparedStatement pstmt3 = null;
ResultSet readResultSet1 = null;
ResultSet readResultSet2 = null;
ResultSet lastPopulatedResultSet = null;
int roomId = 0;
boolean readTheadInfiniteLoop = true;
public readThread(int roomId) {
this.roomId = roomId;
this.setName("readThread=" + roomId);
}
private void init() {
try {
connection = DBconnection.getConnection();
pstmt1 = connection.prepareStatement(readQuery);
pstmt2 = connection.prepareStatement(readLastPopulate);
pstmt3 = connection.prepareStatement(writeLastPopulate);
} catch (SQLException e) {
51
System.out.println("***** unable to get handle to database
conncetion*******");
e.printStackTrace();
}
}
@Override
public void run() {
this.init();
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd
hh:mm:ss");
Timestamp lastPopulatedId = null;
try {
while(readTheadInfiniteLoop) {
readResultSet1 = pstmt2.executeQuery();
while(readResultSet1.next()) {
lastPopulatedId =
readResultSet1.getTimestamp(1);
}
pstmt1.setTimestamp(1, lastPopulatedId);
52
pstmt1.setInt(2, roomId);
readResultSet2 = pstmt1.executeQuery();
pstmt3.setString(1, sdf.format(new Date()));
pstmt3.executeUpdate();
while(readResultSet2.next()) {
String userName =
readResultSet2.getString(2);
String data = readResultSet2.getString(3);
//TODO:create a json string and push it to
web UI using comet connection
System.out.println(userName);
System.out.println(data);
}
Thread.sleep(READ_WAIT_TIME);
}
}
catch (SQLException e) {
System.out.println("***** sql exception*******");
e.printStackTrace();
} catch (InterruptedException e) {
53
System.out.println("***** sleep interrrupted*******");
e.printStackTrace();
}
}
}
}
/*Class responsible for getting access to the database connection pool and get the
connection *object from the pool.
*/
package chatApp;
import java.sql.Connection;
import java.sql.SQLException;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.sql.DataSource;
/**
* Class responsible to initialize database connection object
54
* @author skodali
*
* DBCP is the by Tomcat apache commons DBCP
* how to setup resource DBCP is http://tomcat.apache.org/tomcat-6.0-doc/jndiresources-howto.html
*/
public class DBconnection {
static Connection con = null;
static DataSource ds = null;
static {
InitialContext initialContext;
try {
initialContext = new InitialContext();
Context envctx =
(Context)initialContext.lookup("java:comp/env");
ds = (DataSource) envctx.lookup("jdbc/chatApp");
con = ds.getConnection();
if(con != null) {
System.out.print("*********connectionestablished**********8");
55
}
} catch (NamingException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
}
public static Connection getConnection() {
//
try {
//
Class.forName ("com.mysql.jdbc.Driver").newInstance ();
//
con = DriverManager.getConnection
("jdbc:mysql://localhost:3306/chatapplication", "root", "passenger");
//
} catch (Exception e) {
//
e.printStackTrace();
//
//
}
return con;
try {
return ds.getConnection();
} catch (SQLException e) {
e.printStackTrace();
56
}
return null;
}
public static void releaseConnection(Connection connection){
try {
if(connection != null)
connection.close();
} catch(SQLException e) {
e.printStackTrace();
}
}
}
57
BIBLIOGRAPHY
[1] The Apache Software Foundation. (n.d.a). Apache tomcat. Retrieved from
http://tomcat.apache.org/download-60.cgi
[2] The Apache Software Foundation. (n.d.b). Apache tomcat 6.0: Clustering/session
replication how-to. Retrieved from http://tomcat.apache.org/tomcat-6.0doc/cluster-howto.html
[3] The Apache Software Foundation. (n.d.c). Apache tomcat 6.0: JNDI resources howto. Retrieved from http://tomcat.apache.org/tomcat-6.0-doc/jndi-resourceshowto.html#JDBC_Data_Sources
[4] The Apache Software Foundation. (2011). Apache http server project. Retrieved from
http://httpd.apache.org/
[5] Datadisk.com.uk. (n.d.). Tomcat and Apache setup. Retrieved from
http://www.datadisk.co.uk/html_docs/java_app/tomcat6/tomcat6_apache_server.h
tm
[6] Easy Way Server. (n.d.). Implementation of tomcat clustering. Retrieved from
http://www.easywayserver.com/implementation-tomcat-clustering.htm
[7] Jersey.java.net. (n.d.). Jersey 1.5 user guide. Retrieved from
http://jersey.java.net/nonav/documentation/latest/user-guide.html#d4e8
[8] Oracle. (n.d.). Java servlet technology. Retrieved from
http://www.oracle.com/technetwork/java/javaee/servlet/index.html