Developing with OMII
Part 2: The Application Integration
Tutorial
1
What we will cover…
The application deployment tutorial documentation
The example application we wish to install
The 5 steps to deploying an application



1.
2.
3.
4.
5.

Create a subdirectory structure on the server
Write the application wrapper scripts
Configure the Job Service
Add new application to the Resource Allocation Service
Restart Tomcat
Testing the deployment
2
The application deployment
tutorial documentation

Application deployment documentation
teaches how to deploy the mbio1 example
Java application

Can easily be generalised to installing any
application
3
The application deployment
tutorial documentation

Developer pre-requisites…




Basic shell script (Bash) expertise
Knowledge of overall OMII architecture
Familiarity with OMII Client command line
In context…
Application
Integration
Tutorial
Client
Command
Line Tutorial
Java Client
Tutorial
Services
Development
Tutorial
In development…
4
Tutorial pre-requisites

Installation of an OMII Client (Windows/Linux)



Development platform
Used to submit jobs
Access to an OMII Server


With OMII Base/Extensions and Services
An open, credit-checked account on the server
5
The example application we
wish to install

The mbio1 application

Can be obtained from /tmp/OMII

As root, unzip mbio1.zip in home directory:






Server subdirectory – materials for installing the application
Client subdirectory – materials for testing the install on client
Written in Java – mbio_1.java
Accepts single bio_1_data.txt file as input (fixed)
Outputs single mbio_1_results.csv as output
(fixed)
Ensure Java environment set…


Compile it: javac –classpath . mbio_1.java
Run it: java –cp . mbio_1
6
Interface between the server
and executable
Job Service architecture
Used to
start mbio1
mbio1
Used to
check status
of mbio1
Not strictly
required
7
Step 1: create a subdirectory
structure


We refer to location of OMII server as OMII_HOME
Applications have own directory stored in
OMII_HOME/demo



As root… (sudo –s)
For mbio1, create a OMII_HOME/demo/mbio1 directory
(e.g. mkdir OMII_HOME/demo/mbio1)
Then, create two subdirectories:
 mbio1/application – copy mbio1 class in here


e.g. cp mbio_1.class OMII_HOME/demo/mbio1/application
mbio1/wrapper – location of wrapper scripts
8
Step 2: Write application
wrapper scripts



Interface between server and application
Helps to understand the job run process
When a job run is requested from a client…

Service provider responsible for creating a workspace
directory for the running job
 Workspace in OMII_HOME/datastore/<date-time>_storeid
 Contains logs/status files for job
 Creates 2 subdirectories:



/stagedzips: input/output files moved to/from the staging areas
/work: temporary storage (scratch) area for running job
Service provider responsible for invoking start script
through the platform start script
9
Step 2: Write application
wrapper scripts

Application start script responsible for 3 things:
1.
2.
3.

Moving all input files from /stagedzips directory to
/work directory (unzipping, for example, if necessary)
Executing the application command
Preparing the output files for the client (e.g. zip the
files), and moving them to /stagedzips directory
Input/output file locations passed to start
wrapper script in following manner:

-i ../stagedzips/input0.zip -i ../stagedzips/input1.zip …
-i ../stagedzips/inputN.zip -o ../stagedzips/output0.zip
-o ../stagedzips/output1.zip … -o
../stagedzips/outputN.zip
10
Step 2: Write application
wrapper scripts

When job status is requested from a client…



Service provider responsible for invoking the application’s
status script through the platform status script
Application status script responsible for returning textual
status information to the client
 Perhaps to be parsed, or just output to the screen
Status file argument passed to status script:
 e.g.: -f .app_sp_status
 Any output from application status script is placed in this file
 File contents sent to client
11
Step 2: Write application
wrapper scripts – start script
#!/bin/sh
exec > log
echo mbio1 wrapper started
…
INPUT ="$2"
OUTPUT ="$4"
Pipe output to log file –
used for status script
Input/output file locations (in
stagedzips directory) passed from
platform script
echo Copying input to work directory ...
cp "$INPUT" bio_1_data.txt || exit 1
echo Run mbio1 ...
/usr/java/j2sdk1.4.2_04/bin/java –cp
/usr/local/OMII/demo/mbio1/application
 mbio_1 || exit 1
echo Copying result to output staging ...
cp mbio_1_results.csv "$OUTPUT" || exit 1
echo mbio1 completed successfully
1. Copy our staged input
to working directory
2. Location of
mbio_1 class –
may need to edit
this!
3. Copy our output to
stagedzips directory
12
Step 2: Write application
wrapper scripts – start script

Set execute permissions on the script:



Type: chmod ugo+x mbio1-wrapper.sh
Copy mbio1-wrapper.sh to
OMII_HOME/demo/mbio1/wrapper
Can now test the script:





cd to OMII_HOME/demo/mbio1/wrapper
Create subdirectory: mkdir stagedzips
Copy bio_1_data.txt to stagedzips
Run script:
 ./mbio1-wrapper.sh –i ./stagedzips/bio_1_data.txt –o
./stagedzips/mbio_1_results.csv
mbio1_results.csv should be in /stagedzips directory
13
Step 2: Write application
wrapper scripts – start script

Look at log file: cat log

Should have contents like the following:
mbio1 wrapper started
Copying input to work directory
...
Run mbio1 ...
10
10
2.0
Copying result to output staging
...
mbio1 completed successfully
14
Step 2: Write application
wrapper scripts – status script
#!/bin/sh
cat log > $2
Output status file name path to use,
passed from platform status script
15
Step 2: Write application
wrapper scripts – status script

Set execute permissions on the script:



Type: chmod ugo+x mbio1-status-wrapper.sh
Copy mbio1-status-wrapper.sh to
OMII_HOME/demo/mbio1/wrapper
Can now test the script:


cd to OMII_HOME/demo/mbio1/wrapper
Run script:


./mbio1-status-wrapper.sh –f status_file.txt
Contents of log file should now reside in
status_file.txt
16
Step 3: configure the Job
Service

Individual applications form part of an
application suite and require unique names


There is a recommended convention…
Need to decide on unique application suite name


We’ll use http://omii.org/mbio1_suite
Our application suite only contains one
application – also requires a unique name

We’ll use http://omii.org/mbio1_suite/mbio1
17
Step 3: configure the Job
Service


Need to inform service provider of locations
of wrapper scripts so they can be executed
Locations of platform and application scripts
held in jobservice.properties:



Located in OMII_HOME/jakarta-tomcat5.0.25/webapps/axis/WEB-INF/classes/
First, backup the file (e.g. cp jobservice.properties
jobservice.properties.bak)
Pre-written template provided in zip file – copy
jobservice.properties to the above location
18
Step 3: configure the Job
Service

Edit the copied jobservice.properties file:



Change the jobservice.url.prefix to reflect your host and
port e.g.:
http://lab-19.nesc.ed.ac.uk:18080/axis/services/JobService
You will notice these two lines at the end:




jobservice.appscript.run.http\://omii.org/mbio1_suite/mbio1
=OMII_HOME /demo/mbio1/wrapper/mbio1-wrapper.sh
jobservice.appscript.getStatus.http\://omii.org/mbio1_suite/
mbio1=OMII_HOME /demo/mbio1/wrapper/mbio1-statuswrapper.sh
Lines should have no whitespace
Replace OMII_HOME with your server location
19
Step 4: Add application to the
Resource Allocation Service

Need to inform Resource Allocation Service
of…



The existence of a new application suite
The machines that have it installed
We use the ra_admin web tool to do this


Start a browser, and point it to:
http://<yourhost>:<yourport>/ra_admin
Enter ra as the user, and admin as the password
20
Step 4: Add application to the
Resource Allocation Service

First, need to add a new application suite:



Scroll down to “Application Suites” -> “Add a New
Application Suite”
Enter application suite URI
(http://omii.org/mbio1_suite) in the “Application
Suite URI:” field
Select “Add Application Suite”
21
Step 4: Add application to the
Resource Allocation Service

Next, need to add mbio1 application to application
suite




Scroll down to “Applications” -> “Add a New Application to
a Suite”
Select application suite URI (http://omii.org/mbio1_suite)
from the “Select an App Suite URI” dropdown list
Enter application name (http://omii.org/mbio1_suite/mbio1)
in the “Application Name” field
Select “Add Application to Suite”
22
Step 4: Add application to the
Resource Allocation Service

Lastly, need to specify machine name that has the
application suite installed




Scroll down to "Application Suites Installed on Machines" > "Add an Application Suite to a Machine“
Select the machine name, (name of the machine on which
the application is deployed), from the "Select a Machine"
dropdown list
Select your application suite URI
(http://omii.org/mbio1_suite) from the "Select an App Suite
URI" dropdown list.
Select "Add Application Suite to a Machine"
23
Step 5: restart Tomcat

Have to restart Tomcat so server will re-read
the modified jobservice.properties file




cd to OMII_HOME/jakarta-tomcat-5.0.25/bin
Execute: ./shutdown_base.sh
Followed by: ./start_base.sh
The mbio1 application has been deployed!
24
Testing the deployment


As the omii user…
Copy the mbio1/client directory contents to
the OMII_CLIENT_HOME directory (e.g.
~/OMIICLIENT)


e.g. cp –r client/* ~/OMIICLIENT
Copy the Accounts.xml file generated earlier
to the mbio1 subdirectory as Account.xml:

e.g. cp test-services/test/Account-test.xml
mbio1/Account.xml
25
Testing the deployment

OMII_CLIENT_HOME should now contain:


The mbio1 subdirectory contains files used by script:



test-mbio1.sh: script to test deployment
bio_1_data.txt: input file to mbio1
Work.xml: example of individual job requirements to run a
single mbio1 job
Ensure java environment is set


export JAVA_HOME=/usr/java/j2sdk1.4.2_04
export PATH=$JAVA_HOME/bin:$PATH
26
Testing the deployment –
client test script test-mbio1.sh
DIR=./mbio1
java CreateRequirementsFile http://omii.org/mbio1_suite
$DIR/Requirements.xml
./ogre_client.sh tender $DIR/Account.xml
$DIR/Requirements.xml MyTask
./ogre_client.sh upload MyTask $DIR/bio_1_data.txt
./ogre_client.sh run MyTask
http://omii.org/mbio1_suite/mbio1 $DIR/Work.xml --input
bio_1_data.txt --output mbio_1_results.csv
./ogre_client.sh download mbio_1_results.csv
./ogre_client.sh finish
27
Testing the deployment

Run the test in OMII_CLIENT_HOME directory as
user omii:
…
 ./test-mbio1.sh
Contacting
http://omiitest12.omii.ac.uk:18080/axis/services/JobService#58
http://omii.org/mbio1_suite/mbio1:
URL
:
http://omiitest12.omii.ac.uk:18080/axis/services/JobService#58
Status: Status is now output-staging-complete (FINISHED)
> JOB_STATUS
FINISHED
> mbio1 wrapper started
> Copying input to work directory ...
> Run mbio1 ...
> 10
> 10
> 2.0
> Copying result to output staging ...
> mbio1 completed successfully
Success.
OGRE Client
Downloading:
…
28