WS-VLAM Tutorial:
Part III (part3): Learn how to
WS-VLAM JAVA API
Adam Belloum
Outline
• Part I: Hands on the User Graphical Interface
• Part III: Port a Legacy application to WS-VLAM
• Part III: Learn how to use WS-VLAM API
• Notation
(Title background):
– Generic step
– Tutorial specific step
Outline (Part III): Learn how to use
WS-VLAM API
Step 1: Define an Interface for your component
Step 2: Create a description for your component
Step 3: Define your component dependencies
Step 4: JAVA Tutorial (create a WS-VLAM JAVA components)
Step 6: C/C++ Tutorial (create a WS-VLAM C/C++ components)
Step 5: PYTHON Tutorial (create a WS-VLAM PYTHON components)
Step 6: Deploy the component/dependencies lib
Step 7: Compose a workflow using the component
Step 8: execute the workflow / monitor
Step 1: Define the component interface
• The Interface defines how WS-VLAM will interact
with your components
– Parameter Interface: is use to interact at runtime
with your module
– Port interface: is used to stage in/out the data
consumed/produced by your module
• Port in allows to stage in the consumed data
• Port out allows to stage out the produced data
Step 1: Example File Reader
• The file reader “fileReader” opens a file specified as a
URI using the parameter Interface “filename” and
sends the content through an output port “port_out”.
– fileReader support many protocols: GSIFTP, ftp, http, file
Step 2.1: Create a description
• To create a description of your component: use
CLAM workflow component configuration tool
Action:
• Click on CLAM
Button in the WSVLAM menubar
Step 2.2: Create a description
• CLAM workflow component configuration tool
allows you to define: the name of the component,
default host, ports, parameters, classification, and
category
Action:
• Enter the name of the
component
•
Entre the name of the
default host
“fs2.das3.science.uva.nl”
Step 2.3: Create a description
• CLAM workflow component configuration tool allows
you to define: the name of the component, default
host, ports, parameters, classification, and category
Action:
• Click on ports tab
• Click + to add new
input or output port
• Type the name of the
ports exactly as you
will define them the
source code.
Step 2.4: Create a description
• To add a port a pop up window appears “Add input
Port” or “Add output port”
Action:
• Type the name of the
ports exactly as you did
it in source code
• Specify the type of the
port either a simple
data type, a file type,
or Complex data type
(see documentation for
more details)
Step 2.5: Create a description
• Create a description of your component: use
CLAM workflow component configuration tool
Action:
• Click on save to
created the description
file “component-name.xml”
Step 2.6: Create a description
• Create a description of your component: use
CLAM workflow component configuration tool
Action:
• You will see in the
filed file path the full
path to the create
Component-name
Step 2.7: Example File Reader
• Create a description of fileReader: use CLAM
workflow component configuration tool
• The FileReader has
• One output: port-name = “port_out” and Type=“file”
• One parameter parameter-name = “filename” and
Type=“String”
Action:
• Perform the tasks described in step 2.1 to step 2.6 to create
the fileReader.xml
Step 3.1: Define the component
dependencies
• Dependencies are libraries & auxiliary software
that are required by your component but are not
available on standard Operating systems
• You have to list the name of these libraries &
auxiliary software in a file called “dependencies”
• Every libraries & auxiliary software listed in the
dependencies files have to be packaged as a
“pseudo-module”
Step 3.2: Create a pseudo-module
• A pseudo module is just a
simple software package
for which you need to
create a profile.sh in the
root directory
• Using your favorite text
editor created a file call
profile.sh
# ############ NOTE ####################
# In the profile.sh export all the environment
# variables needed to execute your package
#
# bash environment
#
# export VARIABLE_NAME=VARIABLE_VALUE
#
# Add your VARIABLES bellow the line
######################################
Step 4.1: YOUR JAVA Module
• From a developer’s point of view your
components are applications that use a special
library provided by the WS-VLAM framework:
the “vlport” library
Step 4.2: JAVA modules (pre-requisite)
• Before you build the jwrapper make sure that the
SWIG toolkit and Python are installed in your system.
• The SWIG version should be higher then 1.3.19.
• To compile the java wrapper
–
–
–
–
the environment variable VLAM_INSTALL must be set.
Just type ‘‘make dist‘‘.
You will find the library in dist/lib subdirectory.
The wrapper consists of an ”adapter” shared library with the
name ” jwrapper.jar” and ”jvlport.jar”
Step 4.3: Extend your JAVA code
1. Initialize
– import nl.wtcw.vlamg.jwrapper. ∗;
2. Create a reference to the input port
–
portIRef = VLIPort.getInstance("portIn");
3. Create a reference to the output port
– VLOPort portORef =VLOPort.getInstance("portOut");
– Note: should provide the same names for the ports as
specified at the initialization stag
Step 4.4: API for reading from input port
• void readInt()
– reads an integer number from an input port;
• void readFloat()
– reads a float number from an input port;
• void readDouble()
– reads a double precision number from an input
port;
• void readString()
– reads a string from an input por
Step 4.4: API for reading from input port
• void hardSync ()
– prepares an output buffer and waits until it is sent
to network.
• void readBytes (byte[ data)]
– reads a byte array (raw data) from an input port.
Array must be prepared before;
• void readShort ()
– reads a short number from an input port;
»
• void readLong ()
– reads a long integer number from an input port;
Step 4.5: API for writing to output port
• void writeBytes (byte[ data)]
– writes a byte array to an output port
• void writeShort (short number)
– writes a short to an output port
»
• void writeLong (long number)
– writes a long integer to an output port
»
• void writeInt (int number)
– writes an integer to an output port
Step 4.5: API for writing to output port
• void writeDouble (double number)
– writes a double precision number to an output
port;
• void writeString (String str)
– writes a string to an output port;
• void flush ()
– prepares an output buffer to be sent over network
(flush internal library buffer)
Step 4.6: API for reading/writing
to/from parameter
• getParameter (String param−id)
– Reads the value of “param−id”
• setParameter (String param−id, String para−value)
– Writes the value of “para−value” to “param−id”
Step 4.7: JAVA component
• XXXXXXXX:
// YOU MUST IMPORT THE JWRAPPER PACKAGE
import nl.wtcw.vlamg.jwrapper. ∗;
VLOPort outputPortRef = null;
VLIPort inputPortRef = null;
…
// IT IS ASSUMED THAT REFERENCES TO THE PORTS ARE ALREADY TAKEN
try {
double rndn = 12345.12345;
// WRITING TO THE WS −VLAM
Output Port outputPortRef.writeDouble(rndn); ...
// READ FROM THE WS −VLAM INPUT PORT
rndn = inputPortRef.readDouble();
...
}
catch(VLAPPException e){
// HANDLE EXCEPTION ...
}
...
Step 4.8: JAVA code of the File Reader
import
• nl.wtcw.vlamg.jvlport.VLAppFactory;
The Example:
import nl.wtcw.vlamg.jvlport.Vlport;
import nl.wtcw.vlamg.jvlport.OutputStream;
VLAppFactory.activate(null, outputPorts)
PrintWriter pw =
new PrintWriter(
import java.io.*;
new OutputStreamBufferedReader
reader = new BufferedReader(
/**
new FileReader("/etc/passwd"));
* Created by IntelliJ IDEA.
while((line = reader.readLine()) != null){
* User: dvasunin
System.out.println(line);
* Date: Jul 29, 2005
pw.println(line);
* Time: 3:49:34 PM
}
* To change this template use File | Settings | File Templates.
pw.flush();
*/
} catch (IOException e) {
public class JFileReader {
e.printStackTrace();
static public void main(String args[]){
} finally{
String line = null;
VLAppFactory.deactivate();
String[] outputPorts;
System.exit(0);
try {
}
outputPorts = new String[] {"port_out"}; }
Vlport vlport =
}
(vlport.getOutputPort("port_out")));
Step 4.9: build your JAVA component
• build.xml for FileReader (½)
<project name="WSVLAM-java-modules" default="dist" basedir=".">
<description>
Project : VLE-wsvlam
Component : JFileReader source
Info
:
Ant file to call 'make'.
Builds wsvlam java modules.
This component can be used as standalone.
In that case the Makefile should be called directly.
</description>
<path id="compile.classpath"/>
<property name="src.home"
value="${basedir}/src"/>
<target name="init" description="making build directory">
<mkdir dir="build/java"/>
</target>
Step 4.9: build your JAVA component
• build.xml for FileReader (2/2)
<target name="build" depends="init" description="compile the source ">
<!-- Compile the java code from the srclist } into ${build} -->
<javac srcdir="src" destdir="build/java">
<classpath refid="compile.classpath"/>
</javac>
</target>
<!-- use custom 'invoke' target to pass build parameters & environment -->
<target name="dist" depends="build" description="generate the distribution">
<copy todir="dist/JFileReader/bin">
<fileset dir="${basedir}/build/java" excludes="**/*.java"/>
</copy>
<copy todir="dist/JFileReader">
<fileset dir="${basedir}" includes="JFileReader.xml,main.sh"/>
</copy>
</target>
</project>
Step 4.10: package your FileReader
• Create a directory with the
fileReader name as the
component
• Copy the files you have
created:
– build.xml (step 4.9)
– fileReader.xml (step 2.7)
• Copy any other files or
directory related to your
component:
– src/fileReader.java (step 4.8)
(1) Compile the fileReader
$ cd fileReader
$ ant
# this will created java classes in
# fitleReader/bin
(2) Create a tar file of your component:
$ tar cf fileReader.tar fileReader
(3) Compress the tar file you have created
$ gzip fileReader.tar
Step 5: Deploy the fileReader
• Test mode: you can
test your component
– Perform the steps
described in the blue
box
• Shared mode: you
can share your module
with other users:
– by moving the tar file
you have created in
step 4 to a component
repository
# On a machine where both GT4 & WS-VLAM
# runtime system are installed
# Create in your HOME directory called “.wsvlam”
(1) save fileReader.tar.gz to the default location
$ mkdir .wsvlam
$ cd .wsvlam/
$ mkdir modules
$ cd modules
#untar the component tar file you have created
# in step 4
(2) untar fileReader.tar.gz
$ tar –zxvf fileReader.gz
Step 6: Complete examples in JAVA
components
• Complete examples in JAVA can be found here:
– http://www.science.uva.nl/~gvlam/wsvlam/Documentation/wsvl
am-module-examples.tar.gz
• Download wsvlam-module-examples.tar.gz untar the file
and save the examples in a directory:
$HOME/.wsvlam/modules (see step 5)
Step ?: Template
for
component
in JAVA?>
<?xml
version="1.0"
encoding="ISO-8859-1"
• Is this needed
• ????????
<config>
<ports>
<!-- define here the list of input and output ports -->
</ports>
<java PropertyName="PropertyValue">
<!-- Here we can define System properties for jvm -->
<!-- Here an entry point to your program is defined -->
• Dmitry do we
still need
this??????
<main>
your-java-main-class
</main>
<options>
<!-- Here we define additional options to jvm -->
<option> -server </option>
</options>
<!-- optional classpath -->
<classpath> /path/to/something.jar </classpath>
</java>
</config>
Step 6.1: Compose a workflow fileReader
• In the component list window you will see the
created FileREader components as well as other
components
Action:
• Click on the
fileReader in the list
and drag and drop in
the composition
panel
Step 6.2: Compose a workflow using LA
• An instance of fileReader will be instantiated
Action:
• Click on the fileReader
in the list and drag and
drop in the composition
panel
Step 6.3: Compose a workflow using LA
• Instantiate the fileWriter component:
• This component will move the file produced by fileReader
to any remote location the user specifies
Action:
•
Click on the WriterReader in
the list and drag and drop in the
composition panel
Step 6.4: Compose a workflow using LA
• Create a communication channel:
• This component will move the file produced by FileReader
to any remote location the user specifies
Action:
•
Create a communication
channel by dragging between
an output port (read square)
and input port (blue square)
Step 7.1: customize the composed workflow
• In the step you should check if all the default values
of the parameters of the workflow components
composing your workflow are set correctly
Action:
•
Click on the fileReader in
composition panel
•
In the property window
select the parameter tab
•
Type in the URI to the file you
want stage in FileReader
support many protocols:
GSIFTP, http, file, ftp
Step 7.1: customize the composed workflow
• In the step you should check if all the default values
of the parameters of the workflow components
composing your workflow are set correctly
Action:
•
Click on the fileWriter in
composition panel
•
In the property window select
the parameter tab
•
Type in the URI to the file you
want stage in FileReader
support many protocols:
GSIFTP, http, file, ftp
Step 7.1: customize the composed workflow
• Input file: fileReader parameter
• fs2.das2.science.uva.nl: Type in the following
URI for parameter in each fileReader
• gsiftp://fs2.das.3science.uva.nl/home2/<useraccount>/<filename-you-want-to-cp>
• output file: fileWriter parameter
• fs0.das3.cs.vu.nl: Type in the following URI for
parameter in each fileReader
• gsiftp://fs0.das3.cs.vu.nl/home0/<useraccount>/<filename-you-want-to-created>
Step 8: execute the composed workflow
• Now you are ready to execute the omnimatch
workflow, which automatically stage in the data file
needed for the omnimatch application, and stage out
the result files
Action:
•
Click on the run button
Some References
1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section
on workflow systems and applications in e-Future Generation
Computer Systems 25 (5), 525-527
2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WSVLAM: a GT4 based workflow management system , The 2nd
International Workshop on Scientific Workflows, In
conjunction with ICCS 2007, Beijing, China, 2007
3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M.
Bubak Collaborative e-Science experiments: from scientific
workflow to knowledge sharing JULY/AUGUST, IEEE Internet
Computing, 2011
4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn
Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher,
Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative
Studies Through Interoperable Workflow Provenance,
IPAW2010, Troy, NY, USA
Some References
5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and
applications , Future Generation Comp. Syst. 25 (5): 525-527
(2009)
6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of
aggregated multi domain workflows using an agent framework
The 1st IEEE International Workshop on Scientific Workflows,
Salt Lake City, U.SA, 2007
7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans,
Bob Hertzberger Using Jade agent framework to prototype an
e-Science workflow bus Authors Cluster Computing and the
Grid, 2007. CCGRID 2007