SmartFrog
Overview
HP Labs
Version: 0.6 (Draft)
Localized for US English
© 2003 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without
What is SmartFrog?
SmartFrog: “Smart Framework for Object Groups”
•
•
Describes services as collections of components
Activates services by realizing service descriptions
•
Framework elements:
–
language
– engine
– components
•
•
A framework to build from, not a packaged solution
Mature; already embedded in products
July 2003
page 2
SmartFrog framework
Language
language
transforms
parser
security
model
config.
models
live-ness
process
model
templates
transport
protocol
discovery
scripting
config
servers
Components
July 2003
Engine
transactional
lifecycle
naming
workflows
component
model
binding
deployment
reliability
services
page 3
What problem does it solve?
•
How to configure & automatically
activate complex, distributed apps
–
–
–
•
flexible
repeatable
sequencing, failure-recovery,
evolution, adaptation, …
Ideal for the utility computing
model
–
–
July 2003
automatic app activation
on utility resources
enables rapid resource repurposing
for different apps
page 4
How does SmartFrog address the problem?
1. The common problems
•
No standard method to capture service config., multiple:
–
–
–
•
No method of validation for the configuration as a whole
–
–
–
•
variable lifecycles
no up-front consideration of complete system startup/shutdown
lack of facility to compose systems from sub-systems
No separation of concerns
–
–
•
definitions of each value
mechanisms for access
notations (XML, ini files, SQL, ...)
functionality and configuration mixed together
rigid location and binding integrated into code
Lack of auto-discovery, self-monitoring, and automation
July 2003
page 5
How does SmartFrog address the problem?
2. Philosophy
•
Services are collections of components
–
•
software, data: disk files, hardware
Components collaborate to achieve some goal for
which they must be appropriately organised:
–
required components must be defined
– correctly initialised, with appropriate attributes
– be able to locate each other as required
– exchange information regarding their state
July 2003
page 6
How does SmartFrog address the problem?
3. Approach
•
•
•
Language allows rich system descriptions
Engine automates application deployment, activation,
management and shutdown
Component model allows the engine to be extended
July 2003
page 7
Examples
•
Network monitoring application
–
–
–
–
•
many monitoring components spread throughout the
network
each component must be correctly configured
service as a whole must be correctly configured /
customized
service components must come up in the correct sequence
Three-tier web application
–
–
July 2003
database, application logic engine and web-server
must each be installed, configured and started correctly
must be started on the correct host(s), in the correct
sequence
page 8
SmartFrog elements
•
Language
– templates / descriptions
• Engine
– interprets descriptions to
activate running services
• Components
– make up the running
service
– deployed, configured &
activated by the engine
Language
which service components?
running where?
how is each component initialised?
how are components related?
how are the component lifecycles
sequenced?
Engine
activates
running
components
July 2003
•
•
•
•
•
managed,
monitored
through lifecycle
page 9
Language properties
•
•
Attribute:value pairs in a structured containment
hierarchy
Properties
–
declarative (a data description language, not a
programming language)
– instance-inheritance (prototype-based);
allow templates to be progressively redefined
– flexible linking between attributes and values
– value resolution can be delayed to deployment time
– composition: build larger descriptions out of smaller parts
•
Note: not XML-based, but can use XML as an input or
output format if needed
July 2003
page 10
Language example
wstemplate.s
fwebServerTemplate
extends {
sfProcessHost “localhost”;
port 80;
useDB;
}
dbtemplate.s
f
dbTemplate extends {
userTable extends {
columns 4;
rows 3;
}
dataTable extends {
columns 2;
rows 5;
}
July 2003
webservice.s
f
#include “wstemplate.sf”
#include “dbtemplate.sf”
sfConfig extends {
commonPort 8080;
ws1 extends webServerTemplate {
sfProcessHost “webserver1.hpl.hp.com”;
port ATTRIB commonPort;
}
ws2 extends webServerTemplate {
sfProcessHost “webserver2.hpl.hp.com”;
port ATTRIB commonPort;
useDB LAZY ATTRIB db;
}
db extends dbTemplate {
userTable:rows 6;
}
}
page 11
Language: composing descriptions
software
component
descriptions
E.g. Apache
Web-server
resource
descriptions
composed:
solution
templates
application
patterns
e.g. servers, storage
networking
e.g. 3-tier web service on
Linux cluster front-end,
Superdome back-end.
e.g. reliability & recovery
thresholding
customer
preferences
e.g. SLAs
July 2003
parameterized
fully-specified
solution
description
‘farm’
patterns
e.g. n-tiers including
firewall rules
page 12
Engine
•
Fully distributed engine comprised of SmartFrog
daemons that run on each node
–
•
Daemons interpret SmartFrog descriptions
–
–
•
•
•
deploy service descriptions on any node and the engine
will create components in the right place
load components in the right sequence, in the right place,
with correct configuration parameters
orchestrate the whole system by stepping components
through their complete lifecycles
Provides liveness and component-tree navigation
mechanisms
Many services can be deployed simultaneously using
the same engine
Service un-deployment allows clean service removal
July 2003
page 13
Security
•
•
•
•
•
Engine provides secure deployment based on PKI
All nodes have certificates installed
All component code and descriptions must be signed
If node has valid certificate and code/descriptions are
correctly signed, then deployment is allowed
Single-level security model (all deployed services
operate within the same trust domain for a given
certificate set)
July 2003
page 14
Component model
•
•
SmartFrog components all implement a simple lifecycle
that allows the engine to control them
Allows systems of components to be started
consistently
import com.hp.SmartFrog.Prim.*;
import java.rmi.*;
public class MyPrim extends PrimImpl implements Prim, … {
/* any component specific declarations */
public MyPrim() throws RemoteException {}
public void sfDeploy() throws Exception {
super.sfDeploy();
/* any component specific initialization code */
}
public void sfStart() throws Exception {
super.sfStart();
/* any component specific start-up code */
}
public void sfTerminateWith(TerminationRecored tr) {
/* any component specific termination code */
super.sfTerminateWith(tr);
}
/* any component specific methods */
}
July 2003
page 15
Components
•
•
SmartFrog has an extensible component set
Some basic components form the core of the system,
e.g.:
“Prim”: the primitive component – everything inherits from
it
– “Compound”: implements a “shared-fate” lifecycle for
groups of components
– Sequence: implements a sequential lifecycle for
components
–
•
Other components provide system services: discovery
mechanisms, reliability building blocks, host scripting,
JMX support, …
•
Components are written as needed to support new
July 2003
page 16
Encapsulating non-SmartFrog components
•
•
We don’t need to write our whole application/service in
SmartFrog Java components -- it’s easy to encapsulate
non-SmartFrog things, e.g.
Apache web-server: wrapper components to
–
install and configure software (and remove)
– stop and start the service
•
•
•
We describe Apache and its required configuration in a
SmartFrog description (using an Apache SF template)
When deployed, the SF Apache component can install,
configure, start, stop and remove Apache
Apache can be used as a component of larger services
July 2003
page 17
Implementation details
•
Implemented completely in Java (using JDK1.4)
–
•
•
works across all standard Java platforms
Uses Java RMI as transport
Core system size
–
–
–
July 2003
~10K non-commented lines of Java
~750 lines of description files
binaries: 800-1200 KBytes depending on services
included
page 18
Research questions
•
Rich research space spanning language, engine, and
components
–
reliability (basic mechanisms, generic recovery patterns)
– security
– combining reliability and security
– scalability
– language specification
– service debugging tools
– standardized interfaces to resource management
subsystems
– standardized interfaces to node image deployment
subsystems
– generic system/service visualization tools
July 2003
page 19
Business opportunity
•
App developers
–
–
–
•
Solution providers
–
–
–
•
better enable apps for Grid and utility computer models
optimize application for specific hardware configurations
improve development (rapid deployment)
improve app configuration, deployment, and management
more secure and reliable solutions
quicker time to deployment and adaptation
Happier users
–
–
–
July 2003
more stable systems
customization to specific needs
more control over solutions
page 20
Framework status
•
•
Developing an open source (LGPL-licensed) release of
the core framework (availability calendar ‘03)
Earlier preview / alpha releases planned
Deployed within HP/Agilent products
•
Experience with:
•
–
–
–
–
July 2003
Apache
BEA Weblogic
Maya CGI rendering engine
Oracle
page 21
Some related work
•
•
•
•
•
•
Grid standards for Grid/utility computing
Imaging / provisioning systems: Altiris, Rembo,
Novadigm (+ many others)
MS Dynamic Systems Initiative, MS Provisioning
System
Adaptive systems: ThinkDynamics, Corosoft
IBM eLiza, autonomic computing
Proprietary (product-specific) solutions in place today
for installation / configuration / activation
July 2003
page 22
Standardization goals
•
Improve Grid applications
–
•
•
Standardize SF language, app templates, and SF
engine
Create multiple reference implementations of
–
–
•
•
specification, deployment, configuration & life-cycle mgmt
engine
app templates (Application Server, JBOSS, WebLogic)
Define SF integration with Grid
Form a GGF WG
July 2003
page 23
SmartFrog benefits
•
•
•
•
•
•
Increased operational reliability
Improved quality
Assured correctness and consistency
Increased security
Reduced cost
Improved customer experience
July 2003
page 24
Summary
•
SmartFrog framework (language, templates, engine)
supports:
–
•
•
•
•
specification, deployment, configuration, and life-cycle mgmt
Well suited for a utility computing model
Addresses needs of complex distributed applications
Business benefits for application developers, ISPs, &
users
We are seeking partners & lighthouse users
–
–
–
–
July 2003
jointly standardize key elements
develop with reference implementations
create templates for specific applications
establish a wide technical community
page 25
HP logo