Scripting Astrogrid
Noel Winstanley
Jodrell Bank,
The University of Manchester
A PPARC funded project
Scripting Languages
●
concise
–
minimal boilerplate
multiple ways of doing the same thing.
● no compilation phase
● do the 80% well
● comfortable syntax
● forgiving type system
●
–
●
loose typing, automatic conversion
rich operating environment
pre-configured system objects
– most work done with system objects.
–
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p2
Printed: 08/11/04
Scripting in Astrogrid
●
glue between CEA applications in workflows
extract parameters for a step from the results of previous steps
– date / time calculations
– reformatting data – myspace file management
–
●
interactive script console into astrogrid
development of workflow glue-code
– quick queries / experiments
–
●
commandline utilities
–
integrate jes / myspace / registry with standard unix tools. eg –
grep, cron, mail
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p3
Printed: 08/11/04
Astrogrid System Objects
resuse the existing java client libraries.
● tie these together with some script-friendly system objects
●
astrogrid
– jes
– user
– homeIvorn
www.astrogrid.org/maven/docs/HEAD/jes/api-reference.html
–
●
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p4
Printed: 08/11/04
What Language?
●
Astrogrid System Objects are accessible from any scripting
language that runs in a JVM
Jython (Python) - www.jython.org
– Javascript - www.mozilla.org/rhino/
– Jacl (TCL) - www.tcl.tk/software/java/
– beanshell / judoscript
– Groovy - groovy.codehaus.org/
–
●
Non JVM scripting languages (perl, python, idl, etc)
possibly
– depends on language's support for calling into java
– or could call soap interfaces directly.
–
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p5
Printed: 08/11/04
You're probably more groovy than
you realize...
Groovy is the language that should have been called
Javascript.
● looks like Java / C / Javascript
● strict superset of java
● extensions to make it better for scripting
● dynamically interpreted
● Why groovy?
●
Tight integration with Java
– actively maintained
– going into the JSR process
–
●
Default language used for workflow scripts
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p6
Printed: 08/11/04
Examples: Date Manipulation
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p7
Printed: 08/11/04
Examples: Extract data from Votable
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p8
Printed: 08/11/04
Interactive Astrogrid Console
●
Groovy console into Astrogrid
–
webstartable from www.astrogrid.org/jnlp
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p9
Printed: 08/11/04
Commandline Astrogrid Scripts
●
Groovy commandline interpreter bundled
astrogrid jars
– configuration file for endpoints and user info
–
same system objects as Astrogrid console
● executable scripts using #! notation
● eg – resolve service to endpoint by registry
●
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p10
Printed: 08/11/04
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p11
Printed: 08/11/04
Future of Scripting
Tidy up object model
● Taglibs – hide scripts in reusable tags
● Workflows reference external scripts by URI / IVORN
●
–
●
develop libraries of shared code.
Publish libraries in registry
reg entry contains script source, doc and examples.
– enables routines to be shared, standardized, harvested
–
●
Script servers – distributed script execution.
move script execution out of JES, closer to data
– needs a formal declaration of data flows, so location decisions
can be made. - will look like another CEA tool
–
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p12
Printed: 08/11/04
Future of Scripting
●
Provide script servers for other languages
keep the perl hackers happy
– language-specific issues with sandboxing / trust
– reg entry can describe properties of the server
–
●
●
●
time and space bounds
language versions
Bundle script servers with cea servers
extend cea interface to specify pre- and post-processing scripts.
– a more structured use of scripts.
– registry entry for cea application can relate to registry libraries of
relevant pre- and post-scripts
–
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p13
Printed: 08/11/04
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p14
Printed: 08/11/04
Workflow Document
●
●
●
●
Activities
Authentication & Identity
Results and Logs
Extension Buffers
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p15
Printed: 08/11/04
XLST compiles activities to rules
●
●
●
traversal
name generation
declarative compilation
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p16
Printed: 08/11/04
Rule
●
●
●
trigger – condition on set of states
body – action to change states
all expressed in groovy
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p17
Printed: 08/11/04
Groovy exp has access to set of states
●
●
rules, vars and states serializable to xml
in this way can single-step through execution of activities and persist
execution state – in extension block of workflow.
AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004
p18
Printed: 08/11/04