DIAMONDS
DIstributed Agents for MObile & Dynamic Services
Iosif C. Legrand
California Institute of Technology
CHEP 2003
Iosif Legrand
This project was developed in collaboration with
a team from NUST, Pakistan :
Saad Kiani
Maria Riaz
Aamir Shafi
Anjum Shehzad
Umer Farooq
coordinated by :
Arshad Ali
CHEP 2003
Iosif Legrand
Distributed Service Architecture
Design Considerations
The prototype for Distributed Dynamic Services we are developing gather,
disseminate and coordinate time-dependent information and services for
specific HEP applications.
It aims to extend and improve the (centralized) client – server architectures
on large scale distributed systems.

Hierarchical structure of loosely coupled services which are
independent & autonomous entities able to cooperate using a dynamic
set of proxies or self describing protocols.

These services should facilitate an efficient use of the distributed
resources and provide adaptability.

Reliable on a large scale network distributed environment


Avoid single points of failure

Automatic re-activation of components and services
Scalable & Flexible for adding dynamically new services and
automatically replicate existing ones to cope with time dependent load
CHEP 2003
Iosif Legrand
JAVA RMI & JINI
Distributed Object Systems
“Traditional” Distributed Object Models
(CORBA, DCOM)
Server
Lookup
Skeleton
Service
The Stub is linked to the Client.
The Client must know about the
service from the beginning and
needs the right stub for it
Lookup
Stub
Service
CLIENT
“IDL” Compiler
Lookup
Stub
Service
(Proxy)
Service
CLIENT
Lookup
Stub
Service
(Proxy)
Dynamic Code Loading
Less Protocols !!
CHEP 2003
Any service can be used dynamically
Iosif Legrand
Jini & Code Mobility
JINI extends the Java Platform providing support for Distributed Applications
The purpose of the Jini architecture is to federate groups of devices and soft-ware
components into a single, dynamic distributed system.
Jini is a set of specifications that enables services to find each other on a network
and allows these services to participate within certain types of operations within the
framework. It allows these services to interact in a dynamic and robust way: no user
intervention when devices are added or removed, adaptability when services come
and go, no prior knowledge of the services implementation are needed simplifying
administrative duties.
It provides support for a service oriented architecture with code mobility
CHEP 2003
Iosif Legrand
JINI Overview
The purpose of the Jini architecture is to federate groups of devices and soft-ware
components into a single, dynamic distributed system.
Lookup Service
Services are found and resolved by a lookup service. The lookup service is the
central bootstrapping mechanism for the system and provides the major point of
contact between the system and users of the system.
Leasing
Access to many of the services in the Jini system environment is lease based. A lease
is a grant of guaranteed access over a time period. Each lease is negotiated between
the user of the service and the provider of the service as part of the service
protocol.
Remote Events
The Jini architecture supports distributed events. An object may allow other
objects to register interest in events in the object and receive a notification of the
occurrence of such an event. This enables distributed event-based programs to be
written with a variety of reliability and scalability guarantees.
CHEP 2003
Iosif Legrand
JINI Overview(2)
Transactions Manager
A series of operations, either within a single service or spanning multiple ser-vices,
can be wrapped in a transaction. The Jini Transaction interfaces supply a service
protocol needed to coordinate a two-phase commit.
The JavaSpaces Service
which can be used for simple communication and for storage of related groups of
objects written in the Java programming language
The Mailbox Service
Can be used to provide asynchronous communications (based on any type of messages )
between distributed services
Security
The design of the security model for Jini technology is built on the twin notions of a
principal and an access control list. Jini services are accessed on behalf of some entity
which generally traces back to a particular user of the system. Services themselves
may request access to other services based on the identity of the object that
implements the service.
CHEP 2003
Iosif Legrand
Lookup Discovery Service
A Service Registers with at least one
Lookup Service using the same ID.
jar
Web
Server
Publish the
“Interface” jar
It provides information about its
functionality and the URL addressed
from where interested clients may
get the dynamic code to use it.
CLIENT
Lookup
Lookup
Service
The Service must ask each Lookup
Service for a lease and periodically
renew it.
Register
Service ID
Register with ID
Servic
e
Ask for a lease
Get a lease for DT
Lookup
Service
If a Service fails to renew the lease, it
is removed form the Lookup Service
Directory. When problems are
solved, it can re-register.
The lease mechanism allows the Lookup Service to keep an up to date
directory of services and correctly handle network problems.
Discovery mechanism implementation can be enhanced with UDDI , JXTA protocols.
CHEP 2003
Iosif Legrand
Remote Events
CLIENT
Lookup
Service
Service
Event Provider
Register for Event Notification
Servic
e
Ask for a lease
Get a lease for DT
Any Interested Service or
Client in certain types of
events must register. It
must ask for an event
notification lease and
periodically renew it.
If the Event Provider or the
Event Listener fails the
system will correctly
identify such a state as well
as the case when such a
unit becomes again
available.
The lease mechanism provides for any Event Producer a real list of active
Listeners. Any Event Consumer is informed if the Producer is alive .
CHEP 2003
Iosif Legrand
JavaSpaces
JavaSpace ->Tuplespace computation model
(LINDA) & Distributed Object Model
Process
Take

Loosely coupled/anonymous
communication. Participants are no
aware of each others identity and the
message objects are selected and
retrieved using associative lookup
based on pattern matching

Persistence of the Tuple Space does not
require that participants exist
concurrently (timely uncoupled)

Allows Code and Data Migration
(tuples=code & data)
Write
Process
Process
Tuplespace
Well suited for many parallel algorithms in dynamic configurations.
CHEP 2003
Iosif Legrand
Transaction Services
Transactions enable operational grouping into all successful or all
failed. It allows a two phase commit operations in a distributed
environment.

A Client hands a factory method to a Transactions Manager
reference and a lease duration and receives a Transaction Object
and a Lease.

The Client pass the Transaction Object to the Services it wants to
use asking to do the task “under the transaction”.

If the Client or any participant aborts the transaction the
Transaction Manager instruct all the participates to “rollback”.

If the Client asks to commit the transaction, the Transaction
Manager queries all the participants and if all are “prepared” it asks
all of them for “ roll forward” and commit the transaction.
CHEP 2003
Iosif Legrand
The System Functionally
This prototype system, based on Jini, aims to provide:
Customized dynamic services
“Smart Proxies”
“Mobile Agents”
These conceptual entities should be able to work together and
interact in a distributed environment by:
Remote Event Subscription/ Notification
Synchronous and asynchronous communication
Code Mobility
Persistent Objects
CHEP 2003
Iosif Legrand
Dynamic Services
jar
Web
Server
Publish the
“Interface” jar
Service Discovery
Lookup
Service
Client
The Station Servers register the
dynamic services with the lookup
services, test them and continuously
keep them available by renewing the
leases.
Tuplespace
Registration
Dynamic
Service
Station
Server
The Station Servers allows to
dynamical load into the system local
services. This procedure may be
initiated by a user or by an other
service.
Dynamic
Service
The additional “mobile code”, partners
may need to use a service is either
loaded on a http server or on a
JavaSapace.
CHEP 2003
Iosif Legrand
Dynamic Proxies
“Smart” Proxies extend the functionality of the network services by
trying to provide an optimized performance based on the locally
available resources, services or data.
Station
Server
Service
Lookup
Service
Lookup
Proxy
Service
CLIENT
Smart proxies may :
•
•
•
Dynamically load part of the service code to the client to improve
performance.
Cache locally frequently used data from the remote service and keep them
synchronized
Discover and select the most convenient places to access common services
from the “client” perspective.
CHEP 2003
Iosif Legrand
Mobile Agents
Mobile Agents are automatons entities able to migrate between Station
Servers.
Each Station Server offers the runtime environment, priority and controls the
access permission for the hosted agents.
Mobile Agents interact with the Station Servers network to get access to the
necessary information to migrate and to perform a certain task.
The agent migration is done as a weak migration ( e.g. the agent prepare itself
before it is moved and save its state into structured which a serialized)
Mobile Agents should remain small and simple entities. To achieve this, and
provide complex functionality they must use and access “static” services hosted
by the Station Server network.
CHEP 2003
Iosif Legrand
Mobile Agent Framework Development
DIAMONDS
 Develop a framework that allows for hosting dynamic services in the






form of Mobile Agents
Allows for registration/publishing and discovery of services
Support for remote event notification
Trust Establishment between services
Enabling access to remote resources through Agents
Reliability over a large distributed network
 Avoid loss/duplication of agents – transactions
Dynamic Interaction with other services to
 Use other service’s generated data
 Help another service in decision making
CHEP 2003
Iosif Legrand
Mobile Agents and Code Mobility
Object moves with its code definition
Agents are packed in Jar files which serve as the ‘codebase’ for
the agents
 Marshalled Object for the agent is created which stores the
codebase information.
 On un-marshalling, the agent’s definition is requested from its
codebase.
Weak Mobility
 Code and Variable state is transferred.
Dynamic Services register their proxies with the discovery services.
 Client interested in using these services needs to download the proxy
 Client need not be aware of the actual location of the service.
 Service’s proxy communicates back to its service for the client.
Redundant codebases for downloading proxy definition
“Proxy” is the entire service
CHEP 2003
Iosif Legrand
Remote User Interfaces for Dynamic Services





Dynamic Services in this framework may have a GUI
component (act as complex proxy) that is created at runtime,
on request by a client.
These user interfaces are transferred to the client and
opened at the Client side
Clients can perform operations on the services, remotely,
through their User Interfaces.
Each client has a unique state for the GUI depending on the
operations it is performing.
When a service un-registers, its remote GUI is closed and on
re-registration, it is re-opened automatically at the clients that
were previously viewing the GUI.
CHEP 2003
Iosif Legrand
Transaction Service controls Mobile Agents
Transactions are used in the framework wherever atomic operations
are required
Agent Movement from one Station to another needs to be atomic
 (Un) registration
 Code and state transfer / Migration
 Re registration
Either the move operation is carried out completely and successfully,
otherwise, it stays where it was.
 Avoid agents loss and duplication
Implemented using JINI Transaction Service
CHEP 2003
Iosif Legrand
Security
Asymmetric keys and Certificates
 Each service signed with owner/administrator’s
Agent Station
private key
 Hosting Services (Stations) maintain a trust store
Private
filled with trusted certificates
Trust
Keystore
Store
Trust between AgentStation and Agent’s Owner
 Key Stores
 Signing Jars, Generating Certificates
 Trust Stores
 Trusted Certificates
 Import, Send, Delete Certificates
 Certificate verification on loading and move
Trust between Agent and Remote Client
 Open Agent – Remotely accessible To All Clients
 Close Agent – Remotely accessible To Owner Only
 Agent Verifies if the requesting client is its true owner
CHEP 2003
Iosif Legrand
Agent Station
An Agent Station is a network service that can host Dynamic Services.
Station Servers are dynamically interconnected (peer-to-peer) providing
a distributed frame for Services.
Handling Remote events
Keeps a dynamic list of
available services
Lookup
Service
Lookup
Service
Performs security
authentification / encrypted
communication
Support for Inter-Service
communications (Roaming
for mobile Agents)
Station
Server
Station
Server
Station
Server
CHEP 2003
Iosif Legrand
Agent Stations
Agent Stations are deployed at various
locations across the network (LAN/WAN)
Agents can be loaded by a user.
Station Proxy
The agent can move to other stations.
Where ever the agent goes, it always registers
itself as a service, making remote
administration/monitoring possible for the
user
Agent may be open or closed
A
 Open – Anyone can view what the
agent is doing i.e.it’s a free service
Agent Station A
 Closed – The activities of this agent
can only be monitored and results
retrieved by its owner.
The agent mobility and its monitoring are all
subject to its security credentials
Registration Service
Agent Proxy
B
Agent
CHEP 2003
Iosif Legrand
Agent Station B
Mobile Agents Services
Agents are Dynamic & Mobile Services
 Generic Agent with mobility mechanism to carry the soft
state
 Service Agents with registration mechanism & remote UI,
extended from Generic Agents
Basic Application Agents
 Test Service Agent
 Database Access Agent
 File System Access Agent
 Search agent
The Basic Application Agents should be extended for specific
applications
CHEP 2003
Iosif Legrand
Database Access Agent
Provides access to databases nearby the host AgentStation.
 MySQL
 MS-Access
 RmiJDBC
Host AgentStation keeps record of the dbs on the LAN.
Load and send the Agent to an AgentStation and then access the
databases there remotely through the Agent’s UI.
Remote DB Browser
CHEP 2003
Iosif Legrand
Remote File Access Agent
Provides access to the host Agent Station's File System
depending on security parameters
 View, Edit files
 Remote File Browser
CHEP 2003
Iosif Legrand
Read Performance of Agent
10
9
8
Speed (MB/sec)
7
6
5
Local Agent
LAN(100Mbps)
LAN(10Mbps)
4
3
2
1
0
1.95
4.88
7.81
11.72
14.65
File Size (MB)
CHEP 2003
Iosif Legrand
17.58
Write Performance of Agent
8
7
Transfer Rate(MB/sec)
6
5
4
Local Agent
LAN(100Mbps)
LAN(10Mbps)
3
2
1
0
1.95
4.88
7.81
11.72
14.65
File Size (MB)
CHEP 2003
Iosif Legrand
17.58
Summary
 A prototype system, based on Java & Jini is currently under
development to provide a distributed mobile agents service system to
help the physics data analysis in a multi-tier Regional Centre
architecture.
 Code mobility paradigm provides the mechanism for a consistent,
correct invocation of components in large, distributed systems.
 These services need to use Monitoring information services
(MonaLisa) in decision making and the strategy they use to adapted to
available resources.
 This prototype aims prove the advantages of such an approach
compared with a client-server model for handling dynamic information
in a large scale distributed system
CHEP 2003
Iosif Legrand