Contents
UNICORE: Seamless Access to
Distributed Resources
The GRID Vision
Goals of UNICORE
Dietmar Erwin
Zentralinstitut für Angewandte Mathematik
Forschungszentrum Jülich
D.Erwin@fz-juelich.de
Edinburgh, April 25, 2002
UNICORE Status
Experience and Challenges
Summary
April 2002
• Distributed Computing
– synchronous processing
• High-Throughput Computing
– asynchronous processing
• On-Demand Computing
– dynamic resources
• Data-Intensive Computing
– databases
• Collaborative Computing
– scientists
Software
catalogs
Flexible, highperformance access
to all significant
resources
Computers
Sensor
nets
Colleagues
Data archives
On-demand creation of powerful
virtual computing systems and organizations
Ian Foster and Carl Kesselman, editors, “The Grid: Blueprint for a New Computing
Infrastructure,” Morgan Kaufmann, 1999, http://www.mkp.com/grids
Edinburgh 2002
2
The Vision of the Grid
The Grid: Five Computing Models
April 2002
Edinburgh 2002
3
April 2002
Edinburgh 2002
4
UNICORE Goals
UNICORE
UNICORE:
UNICORE is funded in part by BMBF, the German
Ministry of Education and Research under
project grant:
UNiform Interface to COmputing Resources
• conceived prior to the ‘invention’ of the GRID
• UNICORE has both a modest and an ambitious
goal:
UNICORE Plus: 01-IR-001
• Create
January 1, 2000 - December 31, 2002
– Seamless
– Secure
– Intuitive
http://www.unicore.de
access to distributed German HPC resources
• Create a system that can be used in production
at the German HPC centers
April 2002
Edinburgh 2002
5
April 2002
UNICORE Goals
Edinburgh 2002
6
UNICORE Goals
• Creation of system-independent jobs
• Transfer of data to and from the workstation
• Execution at different sites
• Automatic data transfer between systems and
sites
• Creation of complex interdependent multi-system
and multi-site jobs
• Access to data in Unix file systems and archives
• Monitoring and control of jobs
• Application support
• Dependencies
• No changes to third party applications
• Workflow
• Support for legacy job scripts
• Performance analysis using VAMPIR
April 2002
Edinburgh 2002
7
April 2002
Edinburgh 2002
8
UNICORE Goals
UNICORE Goals
• Single sign-on
• No changes to local user management and
naming conventions
• Continued development and support after the
end of the project
• Secure communication over the Internet
• Basis for additional national and international
projects
• Support for all platforms, operating systems, and
batch systems at partner sites
• Production quality prototype
• Retaining of administrative autonomy and
security policies of participating sites
April 2002
Edinburgh 2002
9
April 2002
Software
catalogs
Flexible, highperformance access
to all significant
resources
Edinburgh 2002
10
The GRID Vision
Computers
Goals of UNICORE
Sensor
nets
UNICORE Status
Experience and Challenges
Colleagues
Summary
Data archives
On-demand creation of powerful
virtual computing systems and organizations
April 2002
Edinburgh 2002
11
April 2002
Edinburgh 2002
12
Project Partners
•
•
•
•
•
•
•
•
•
•
PC²
Forschungszentrum Jülich (FZJ, Project coordination)
Deutscher Wetterdienst Offenbach (DWD)
Pallas GmbH Brühl (Pallas)
Rechenzentrum Universität Stuttgart (RUS)
Konrad-Zuse-Zentrum Berlin (ZIB)
Leibniz-Rechenzentrum München (LRZ)
Paderborn Center for Parallel Computing (PC²)
Rechenzentrum Universität Karlsruhe (RUKA)
Technische Universität Dresden (TUD)
fecit*
* Subcontractor to Pallas
April 2002
Edinburgh 2002
LRZ: SR 8000, VPP
RUS: Hitachi, SX-5
FZJ: 2xT3E, T90, SV1
FZJ
ZIB: T3E, SGI
DWD: T3E, IBM SP
RUKA: IBM SP
PC²: hpcLine
Pallas
TUD: SGI
RUKa
Pallas: Sun
fecit: VPP, Sun
13
April 2002
DWD
ZIB
TUD
RUS
LRZ
Edinburgh 2002
14
UNICORE Status
• UNICORE achieves seamlessness through
abstraction
Research groups using
– Abstract Job Objects (AJO) model most user actions
Jülich’s
– Represented as Java classes and XML
• Abstract jobs are translated to system and site
specific jobs and actions at the target system
John von Neumann
Institute for
• Changes at the destination don’t affect the user
Computing (NIC)
April 2002
Edinburgh 2002
15
April 2002
Edinburgh 2002
16
UNICORE Status
UNICORE Architecture
• UNICORE implements a very strong security
based on X.509 certificates
User Workstation
– Single sign-on
UNICORE Client
– Mutual authentication of clients and servers
SSL
– Signing of code by developers
UNICORE Server
UNICORE Server
– Signing of jobs
Gateway
– Encryption of communication
Gateway
SSL
Network Job Supervisor
Network Job Supervisor
TCP/IP
TCP/IP
Batchsystem
BSS + Batchsystem
System
Interface
Batchsystem
BSS + Batchsystem
System
Interface
Batch SubSystem
Batch SubSystem
UNICORE Site 1
April 2002
Edinburgh 2002
UNICORE Site n
17
April 2002
Edinburgh 2002
18
19
April 2002
Edinburgh 2002
20
UNICORE Status
• The functions to meet the project objectives have
been implemented (final version end 2002)
• Pallas GmbH in Brühl will continue to support
UNICORE commercially
• UNICORE sources are available to research free
of charge under community licence
(download from www.unicore.org in May)
• UNICORE is used in other projects:
– EUROGRID, GRIP, OpenMol GRID, …
• UNICORE is available in IBM's new
GRID Innovation Centre
April 2002
Edinburgh 2002
Edinburgh 2002
Edinburgh 2002
April 2002
April 2002
23
21
April 2002
April 2002
Edinburgh 2002
Edinburgh 2002
24
22
UNICORE Status
• The functions to meet the project objectives have
been implemented (final version end 2002)
• Pallas GmbH in Brühl will continue to support
UNICORE commercially
• UNICORE sources are available to research free
of charge under community licence
(download from unicore.org in May)
• UNICORE is used in other projects:
– EUROGRID, GRIP,OpenMol GRID, ….
• UNICORE is available through IBM's new Grid
Innovation centre in Montpellier
April 2002
Edinburgh 2002
25
Geography
April 2002
Edinburgh 2002
26
GRIP
GRIP: GRID Interoperability Project
Funded in Part by EC: IST-2000-32257
Duration: January 2002- December 2003
Partners:
April 2002
Edinburgh 2002
27
•
Forschungszentrum Jülich, D (CO)
•
Pallas GmbH, D
•
University of Manchester, UK
•
University of Warsaw, PL
•
Deutscher Wetterdienst, D
•
University Southampton, UK (AC)
•
Argonne National Laboratory, US (SC)
•
fecit, UK (SC)
April 2002
Edinburgh 2002
28
GRIP Objectives
• Develop software to facilitate interoperation
between UNICORE and Globus combining the
unique strength of each system
The GRID Vision
• Build and demonstrate biomolecular and
meteorological inter-grid applications
Goals of UNICORE
UNICORE Status
• Contribute to and influence international Grid
standards through the Global Grid Forum
Experience and Challenges
Summary
+
UNICORE's Object model is well suited to be
translated into Web Services
April 2002
Edinburgh 2002
29
Experience
April 2002
Edinburgh 2002
30
Challenges
• Support
– Long term commitment by developers
• UNICORE software works extremely well
– Support by the computing centers
• Users demand application support
• Users need reliable solutions
This is requires funding beyond projects terms
• Retaining site autonomy enables integration
• Strong security is essential
April 2002
Edinburgh 2002
31
April 2002
Edinburgh 2002
32
Challenges
Challenges
• Security
• Administrative Autonomy
– Sharing of resources between centres is a major
obstacle
– Certificate management is a major hurdle for users
– Mutual acceptance of certificates of different CA s
must be solved (equivalence of policies)
– different funding agencies
different local governments
different nations
– Firewalls and firewall policies can hinder
communication
prevent effective Grid computing
This is both a technical and an administrative
challenge
This is only an administrative challenge
April 2002
Edinburgh 2002
33
April 2002
Edinburgh 2002
34
Summary
Web addresses:
• http://www.unicore.de
Information about the project
Addressing both the technical and non-technical
issues is mandatory for success
• http://www.fz-juelich.de/unicore-test
Free UNICORE Client software and test system
• http://www.unicore.org
Congratulations to the UK Science Councils to
include both in the e-Science program
Download of software + source (as of May 2002)
• http://www.eurogrid.org
Information about EUROGRID
• http://www.grid-interoperability.org
Information about GRIP
April 2002
Edinburgh 2002
35
April 2002
Edinburgh 2002
36