Scripting Astrogrid Noel Winstanley Jodrell Bank, The University of Manchester A PPARC funded project Scripting Languages ● concise – minimal boilerplate multiple ways of doing the same thing. ● no compilation phase ● do the 80% well ● comfortable syntax ● forgiving type system ● – ● loose typing, automatic conversion rich operating environment pre-configured system objects – most work done with system objects. – AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p2 Printed: 08/11/04 Scripting in Astrogrid ● glue between CEA applications in workflows extract parameters for a step from the results of previous steps – date / time calculations – reformatting data – myspace file management – ● interactive script console into astrogrid development of workflow glue-code – quick queries / experiments – ● commandline utilities – integrate jes / myspace / registry with standard unix tools. eg – grep, cron, mail AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p3 Printed: 08/11/04 Astrogrid System Objects resuse the existing java client libraries. ● tie these together with some script-friendly system objects ● astrogrid – jes – user – homeIvorn www.astrogrid.org/maven/docs/HEAD/jes/api-reference.html – ● AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p4 Printed: 08/11/04 What Language? ● Astrogrid System Objects are accessible from any scripting language that runs in a JVM Jython (Python) - www.jython.org – Javascript - www.mozilla.org/rhino/ – Jacl (TCL) - www.tcl.tk/software/java/ – beanshell / judoscript – Groovy - groovy.codehaus.org/ – ● Non JVM scripting languages (perl, python, idl, etc) possibly – depends on language's support for calling into java – or could call soap interfaces directly. – AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p5 Printed: 08/11/04 You're probably more groovy than you realize... Groovy is the language that should have been called Javascript. ● looks like Java / C / Javascript ● strict superset of java ● extensions to make it better for scripting ● dynamically interpreted ● Why groovy? ● Tight integration with Java – actively maintained – going into the JSR process – ● Default language used for workflow scripts AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p6 Printed: 08/11/04 Examples: Date Manipulation AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p7 Printed: 08/11/04 Examples: Extract data from Votable AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p8 Printed: 08/11/04 Interactive Astrogrid Console ● Groovy console into Astrogrid – webstartable from www.astrogrid.org/jnlp AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p9 Printed: 08/11/04 Commandline Astrogrid Scripts ● Groovy commandline interpreter bundled astrogrid jars – configuration file for endpoints and user info – same system objects as Astrogrid console ● executable scripts using #! notation ● eg – resolve service to endpoint by registry ● AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p10 Printed: 08/11/04 AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p11 Printed: 08/11/04 Future of Scripting Tidy up object model ● Taglibs – hide scripts in reusable tags ● Workflows reference external scripts by URI / IVORN ● – ● develop libraries of shared code. Publish libraries in registry reg entry contains script source, doc and examples. – enables routines to be shared, standardized, harvested – ● Script servers – distributed script execution. move script execution out of JES, closer to data – needs a formal declaration of data flows, so location decisions can be made. - will look like another CEA tool – AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p12 Printed: 08/11/04 Future of Scripting ● Provide script servers for other languages keep the perl hackers happy – language-specific issues with sandboxing / trust – reg entry can describe properties of the server – ● ● ● time and space bounds language versions Bundle script servers with cea servers extend cea interface to specify pre- and post-processing scripts. – a more structured use of scripts. – registry entry for cea application can relate to registry libraries of relevant pre- and post-scripts – AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p13 Printed: 08/11/04 AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p14 Printed: 08/11/04 Workflow Document ● ● ● ● Activities Authentication & Identity Results and Logs Extension Buffers AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p15 Printed: 08/11/04 XLST compiles activities to rules ● ● ● traversal name generation declarative compilation AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p16 Printed: 08/11/04 Rule ● ● ● trigger – condition on set of states body – action to change states all expressed in groovy AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p17 Printed: 08/11/04 Groovy exp has access to set of states ● ● rules, vars and states serializable to xml in this way can single-step through execution of activities and persist execution state – in extension block of workflow. AstroGrid Science Demos: :AG Consortium Meeting #7: NeSC: : Dec 14, 2004 p18 Printed: 08/11/04