Experiences in Teaching a Geographically Distributed Undergraduate Grid Computing Course Barry Wilkinson Department of Computer Science University of North Carolina at Charlotte Mark Holliday Department of Mathematics and Computer Science Western Carolina University Clayton Ferner Department of Computer Science University of North Carolina at Wilmington Grid Computing Course (Fall 2004) Originated from WCU on NCREN network. Broadcast to: – UNC-Wilmington – NC State University – UNC-Asheville – UNC-Greensboro – Appalachian State University – NC Central University – Cape Fear Community College – Elon University 43 students, several faculty Instructors: Barry Wilkinson and Clayton Ferner Participating Sites Level and Prerequisites Listed as an undergraduate course but can be taken for graduate credit. Graduate students expected to do more work. Preferably have programming skills in Java on a Linux system. Topics Review of Internet technologies Introduction to grid computing Web services Grid services Security, Public Key Infrastructure Open Grid Services Architecture (OGSA) Globus 3.2 Condor-G MPI and grid-enabled MPI UNC-W workflow editor and other GUI tools Grid computing applications Assessment Programming assignments, submitted using WebCT – – – – – – Creating a web service Creating a grid service Submitting a Globus job Submitting a CondorG job MPICH-G2 program (not done in Fall 2004) Using UNC-W GUI workflow editor Several WebCT quizzes/tests Weeks 1 - 3 Grid computing Internet Tech. Web Services Assignment 1 Virtual organizations, grid projects around the world, grid challenges IP addresses, HTTP, URL, HTTP, XML, FTP, SSL Service-Oriented Architecture, XML documents, XML schema, namespaces, SOAP, examples, Axis, WSDL "Simple" Web service Java programming assignment. Tomcat, axis, JWS facility Assignment 1 Objective Learn how to write, deploy, and use a web service, using Apache Tomcat Java Servlet container and tools. Steps 1. Create a simple math web service (Java Web Service file). 2. Generate required files from jws file and compile. 3. Create client source file and compile. 4. Execute client to access service. 5. Extend service functionality and test. Weeks 3 - 4 Grid Service (GT3) Concepts, Open Grid Services Architecture (OGSA), OGSI, grid service factory, introduction to GT3. Assignment 2 "Simple" grid service Java programming assignment. Globus 3.2 environment.Tools: ant Assignment 2 Objective Learn how to write, deploy, and use a grid service, using Globus 3.2 Toolkit. Steps 1. 2. 3. 4. 5. 6. 7. 8. 9. Run name change script. Create a math grid service with several methods. Define grid service interface using GWSDL. Write deployment descriptor. Build and deploy grid service. Create client source file and compile. Start Globus container on a free port. Execute client to access service. Extend service functionality and test. Weeks 4 - 6 Security Globus Assignment 3 Secure connection requirements, symmetric and asymmetric key cyptography, digital signatures, certificates, CAs, X509 certificate Basic structure (GT 3.2), grid service container, service browser, GRAM, job submission, GSI, Globus certificates, simpleCA, proxies, resource management, MMJFS, RSL and RSL-2, examples. Submitting GT 3 job specified in RSL-2 Assignment 3 Objective Learn how to submit jobs in Globus 3.2. Steps 1. Start Globus container on a free port. 2. Start a proxy process (obtain proxy certificate). 3. Submit a pre-existing job using the Globus managed-job-globusrun command. This command uses an RSL-2 job description file. 4. Create your own job, compile and submit. 5. Write and deploy your own grid service accessed by multiple identical clients started using the managed-job-globusrun command. Weeks 6 - 7 Inf. Services GT3 information directory services, LDAP, resource discovery Schedulers and resource brokers Condor, submit description file, DAGMan, Checkpointing, ClassAd, Condor-G, other systems Assignment 4 Submitting a Condor-G Job Assignment 4 Objective Learn the Condor-G scheduler and Globus Resource Allocation Manager. Steps 1. Start a proxy process (obtain proxy certificate). 2. Check status of Condor pool. 3. Create a Condor job description file for a simple job and submit a job. 4. Check status of job and manage job using Condor commands. 5. Write and test a computationally intensive Java job according to specification. 6. Submit job to a Condor Java universe and test. 7. Submit job to a Condor Globus universe, test. Weeks 7 - 8 HPC Grand challenge problems, parallel computing, potential speed-up, types of parallel computers Message passing Basic message passing techniques, system software, programming models), synchronous message passing, asynchronous message passing, message tags, collective routines Weeks 8 - 9 MPI Grid-enabled MPI Assignment 5 Process creation, communicators, unsafe message passing, point-topoint, blocking, non-blocking, communication modes, collective communication, running an MPI program on a cluster MPI-G2 internals, mpirun command, RSL script, programming techniques suitable for a Grid Running a simple MPI-G2 program Assignment 5 Objective Learn how to write and submit MPI programs to a grid using MPICH-G2. Steps 1. 2. 3. 4. Start a proxy process (obtain proxy certificate). Create and compile a simple MPI program. Execute MPI program on two machines. Add communication to MPI program and reexecute program. 5. Write, compile and execute an MPI ping program. 6. Write, compile and execute MPI program for numerical integration using broadcast/reduce. Weeks 10 to 11 Workflow GridNexus I GridNexus II Grid portals Assignment 6 Concept, examples. GridNexus UNC-W GUI for Workflow management. Implementation of GridNexus, JXPL. Presented by Prof. Jeff Brown. Grid portals, purpose, application-based portals, historical examples, portal implementation, portlets, JSR 168, OGCE portal. GridNexus assignment. Assignment 6 Objective Learn how to create a GridNexus workflow that uses a web service and a grid service. Steps 1. Download GridNexus and install on your computer. 2. Check that your service from assignment 1 and 2 are deployed. If not, deploy them, or use the deployed services provided. 3. Create a web service workflow. 4. Create a grid service workflow. 5. Create workflow that uses both web & grid services. 6. Extend service functionality and test. GridNexus Workflow Editor Developed by UNC-Wilmington www.gridnexus.org Simple Workflow Example Computes -2 * (23 + 6) More Complex Example using Grid Services Weeks 14 to 15 Guest Speakers Professor Daniel A. Reed, Chancellor's Eminent Professor, Vice Chancellor for IT and CIO, UNC-Chapel Hill, Director of Institute for Renaissance Computing, University of North Carolina at Chapel Hill, Duke University, and NC State University. Title of presentation: “Grid computing: 21st Century Challenges.” Wolfgang Gentzsch, Managing Director, MCNC Grid Computing and Networking Services. Title of presentation “Grid Computing in the Industry” Chuck Kesler, Director, Grid Deployment and Data Center Services, MCNC. Title of presentation: “Security Policy, Legal, and Regulatory Challenges in Grid Computing Environments.” Professor Ian Foster: Taped presentation “The Grid: Beyond the Hype,” by Ian Foster, Argonne National Laboratory and University of Chicago, (originally given at Duke University, Sept. 14th, 2004). Course Home Page No assigned course textbook. Materials and links provided on the home page. http://www.cs.wcu.edu/~abw/CS493F04 Used for posting announcements, slides, assignments, reading materials, tests dates, etc. WebCT also used for quizzes. Fall 2005 Grid Computing Course To originate from UNC-Charlotte Again in collaboration with UNC-Wilmington Can be for undergraduate or graduate credit. Acknowledgements Partial support for this work was provided by the National Science Foundation’s Course and University of North Carolina, Office of the President. 1. National Science Foundation, “Introducing Grid Computing into the Undergraduate Curricula,” ref. DUE 0410667, PI: A. B. Wilkinson, $100,000, 2004-2006. 2. University of North Carolina Office of President, “A Consortium to Promote Computational Science and High Performance Computing,” PI: B. Kurtz (Appalachian State University) co-PI B. Wilkinson and others at various universities, total $650,000, 2004-2006. 3. University of North Carolina Office of President, “Fostering Undergraduate Research Partnerships through a Graphical User Environment for the North Carolina Computing Grid,” PI: R. Vetter (UNC-Wilmington), co-PI B. Wilkinson and others at various universities, total $557,634, 2004-2006. Questions?