Introduction to ILDG
Dirk Pleiter
DESY Zeuthen
HackLatt 2008, 3 April 2008, Edinburgh
Outline
■
Introduction
■
Security
■
GUI Installation
■
ILDG Services
■
Monitoring
■
Infrastructure overview
■
Summary
2
INTRODUCTION
3
What is Grid?
■
According to I. Forster's (2002) check-list a Grid
is a system:
■
that coordinates resources that are not subject to
centralized control
■
■
■
Resources that live within different control domains
using standard, open, general-purpose protocols
and interfaces
to deliver non-trivial qualities of service
■
Coordinated use
4
Grid Middleware
■
■
Layer model:
Application
Service examples:
■
resource discovery
■
uniform data access
■
authentication
■
identity management
Middleware
Network
5
What is ILDG?
■
■
Goal of ILDG:
■
Establish an international grid infrastructure
■
Long-term storage and global sharing of data
Organised as a federation of Grids
■
Different middleware stacks
■
■
■
gLite
http://www.glite.org
Globus Toolkit http://www.globus.org
Different grid infrastructures
■
■
EGEE, OSG, WLCG
Use available infrastructure
(tiny fraction of LHC storage)
6
Interoperability Challenge
■
■
Adopt common Grid standards
■
Provide service which implements given standard
■
Example: Storage Resource Manager (SRM)
Define and implement
interface services
■
■
Standardized interface
service
Interface service executes
request on behalf of user
■
Interface
Service A
Interface
Service B
Service A
Service B
Problem: credential delegation
7
Use cases and services
■
Search for scientific data
■
■
Identify copy of file
■
■
Query file catalogue (FC)
Download file
■
■
Query metadata catalogue (MDC)
Access storage element (SE)
Register and manage users
■
Virtual organisation management
8
Credits
■
Members of the ILDG Middleware Working
Group:
■
Australia: Paul Coddington, Shunde Zhang
■
Japan:
■
Germany: David Melkumyan, Dirk Pleiter
■
UK:
George Beckett, Radoslaw Ostrowski
■
US:
Balint Joó, James Simone, Chip Watson
Noriyoshi Ishii, Mitsuhisa Sato
9
SECURITY
10
Basics
■
Authentication
■
■
Authorization
■
■
Process of determining whether the person we are
communicating with is who it is declared to be
→ Who are you?
Since we know with whom we are communicating
we can restrict the access to certain services
→ What are you allowed to do
Accounting (not considered here)
→ How much have you used?
■
Auditing (not considered here)
→ What did you do? Were you allowed to do that?
11
Public Key Cryptography
■
■
■
Based on two keys:
■
Key A: “secret” or “private” key
■
Key B: “public” key
By knowing the public key, it is computationally
infeasible to reveal the secret key
The decryption key is not the same as the
encryption key
12
Public Key Encryption (2)
Figure: uic.edu
13
Authentication: Step 1
■
■
■
Modified procedure:
■
Bob sends random message to Alice
■
Alice encrypts message using private key
■
Encrypted message is returned to Bob
■
Bob decrypts message using Alice's public key
If decryption is successful, Bob knows that
Alice is owner of private key
But:
■
Does Bob trust relation of
Alice and her key?
Anyone can generate a private and public key
14
Digital Signatures
Figure: uic.edu
15
Authentication: Step 2
■
■
Establish Certification Authority (CA) that can
assure that a public key (or certificate) can be
trusted
Procedure:
■
■
CA has private/public key pair
CA signs user's certificate after having confirmed
identity of user
■
■
Typically done via Registration Agency (RA)
Everybody can check the authenticity of a user's
certificate by checking the authenticity of the CA's
signature
16
Certificates
■
■
Certificate includes:
■
Public key
■
Distinguished name (DN)
■
Identity of CA that signed certificate
■
Digital signature of CA
X.509 format defined by IETF
17
Certificate Authorities
■
CAs organised in Policy Management
Authorities (PMAs)
TAGPMA
■
■
■
APGridPMA
Coordinate trust fabric
Provide repository for
CA certificates and
Certificate Revocation
Lists (CRL)
Roof organisation:
International Grid Trust Federation (IGTF)
http://www.gridpma.org/
18
Single Sign-on
■
■
Private key should be password protected
→ Each use requires use of password
Solution: Proxy certificate
■
New certificate with new private/public key
■
Certificate signed by owner → trust chain:
■
Allows authentication with typing password
→ Limited lifetime reduces security risk
19
Tutorial 1a: Install GUI
■
■
Goto http://www-zeuthen.desy.de/latfor/ldg
→ UserDoc → Software and follow instructions
Install
■
■
■
■
■
■
■
gui-3.0-1.*.noarch.rpm,
gui-default-3.0-1.*.noarch.rpm
env.rpm
cert.rpm requires regular update
cert.rpm
jre.rpm, jre-default.rpm
srmcp.rpm, srmcp-default.rpm
ildg-client.rpm ildg-client-default.rpm
Alternatively:
export LROOT=$HOME/ildg/lroot; . $LROOT/etc/env.sh
20
Tutorial 1b: Certificates
■
■
■
■
Locations:
■ Certificate:
■ Your private key:
$HOME/.globus/usercert.pem
$HOME/.globus/userkey.pem
View certificate:
■
grid-cert-info [-file $HOME/.globus/usercert.pem]
■
Location should be defined in $X509_USER_CERT
Generate proxy certificate
■
grid-proxy-init [-verify] [-valid <h:m>]
■
Location defined in $X509_USER_PROXY
View proxy certificate
■
grid-proxy-info
Remove $HOME/.globus from lab
machines after lecture!
21
Virtual Organisation (VO)
■
■
Collection of people from different institutions
working to solve a common goal
VO is consumer of resources (e.g. storage)
provided by resource providers
■
■
■
VO members share common resources
VO establishes resource-usage agreements with
grid resource providers
VO manages membership
22
VOM Registration Service
(VOMRS)
■
GUI: https://grid-voms.desy.de:8443/vo/ildg/vomrs
■
User action:
■
Load certificate into browser and connect to VOMRS
■
Fill registration form and nominate representative
■
■
After email address verification acknowledge VO
usage rules
Representative action
■
Approve/deny application
■
≥ 2 representatives per regional Grid
23
AuthZ Architecture
24
AuthZ in Practice
■
Simplest and popular mechanism: grid-mapfile
■
List of authorized users
...
"/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=christopher kelly" .ildg
"/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=christopher maynard" .ildg
"/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=eilidh grant" .ildg
"/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=george beckett" .ildg
...
■
■
Local account name: map Grid to Unix credentials
VO is not persistent
■
Services regularly pull updated membership list
■
Automatic update of grid-mapfile (from time to time)
→ Dynamic propagation of membership information
25
VO Groups
■
VOMRS allows definition of groups
■
■
E.g. /ildg/ukqcd
Mapping Grid to Unix credentials may depend
on group membership
■
Group members mapped to different accounts
26
Use cases and services
■
Search for scientific data
■
■
Identify copy of file
■
■
Query file catalogue (FC)
Download file
■
■
Query metadata catalogue (MDC)
Access storage element (SE)
Register and manage users
■
Virtual organisation management
27
Linking Metadata and Data
Objects
Links
Ensemble XML document
Configuration XML document
Binary data file
markovChainURI
dataLFN
dataLFN
28
Information/Data Storage
Ensemble XML
Config XML
markovChainURI
dataLFN
Metadata Catalogue
dataLFN
SURL
File Catalogue
Binary Confs
SURL
Storage Element
29
URI Conventions
■
markovChainURI: mc://__GRID__/...
■
__GRID__
■
Example:
= regional grid name
mc://ldg/qcdsf/clover_nf2/b5p40kp13660-32x64
■
dataLFN:
■
lfn://__GRID__/...
Example:
lfn://ldg/qcdsf/clover_nf2/b5p40kp13660-32x64
/qcdsf.597.00554.lime
30
METADATA CATALOGUE
31
Metadata Catalogue
■
■
■
Catalogue = database with following
functionalities:
■
Query and download XML documents
■
Insert/update/delete XML documents
Different implementations:
■
Generic XML database (e.g. eXist)
■
XML relational mapper + RDBMS (e.g. MySQL)
ILDG interface services provide access only to
some functions
32
MDC Interface Services
■
ILDG-complient MDCs have to provide the
following services:
■
getMDCinfo
■
■
doEnsembleURIQuery
■
■
■
Return information on MDC, e.g.: name of regional grid,
supported query types, version number
Query the MDC for matching ensemble XML IDs
For each matching XML ID its markovChainURI is
returned
doConfigurationLFNQuery
■
Equivalent service for configurations XML IDs
33
MDC Interface Services (2)
■
getEnsembleMetadata
■
■
getConfigurationMetadata
■
■
Return ensemble XML ID with a given markovChainURI
Return configuration XML ID with a given dataLFN
Particular MDC implementation may provide
more services
■
E.g. for uploading and managing XML IDs
34
Web Service Technology
■
■
ILDG interface services are implemented as
web services
Definition by W3: [Web Services Glossary, 2004]
■
■
■
A software system designed to support
interoperable machine to machine interaction over
a network
It has an interface described in a machineprocessable format, i.e. WSDL
Other systems interact with the Web service in a
manner prescribed by its description using SOAPmessages
35
SOAP
■
■
■
Originally SOAP = Simple Object Access
Protocol
A protocol for exchanging XML-based
messages over computer networks, normally
using HTTP/HTTPS
Typically RPC type messaging pattern:
■
■
■
Client sends request to server
Server sends immediate response message back
to client
Lengthy XML syntax potential disadvantage
36
Web Services Description
Language (WSDL)
■
XML-based language for describing web-services
■
Example:
■
getMDCinfo service
■
Define binding:
<wsdl:binding name="mdcSoapBinding"
type="impl:MDCInterface">
<wsdl:operation name="getMDCinfo">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="getMDCinfoRequest"/>
<wsdl:output name="getMDCinfoResponse"/>
</wsdl:operation>
</wsdl:binding>
37
WSDL (2)
■
Define interface implementation:
<wsdl:portType name="MDCInterface">
<wsdl:operation name="getMDCinfo">
<wsdl:input message="impl:getMDCinfoRequest"
name="getMDCinfoRequest"/>
<wsdl:output message="impl:getMDCinfoResponse"
name="getMDCinfoResponse"/>
</wsdl:operation>
</wsdl:portType>
■
Define message implementations:
<wsdl:message name="getMDCinfoRequest"/>
<wsdl:message name="getMDCinfoResponse">
<wsdl:part name="getMDCinfoReturn" type="impl:MDCinfo"/>
</wsdl:message>
38
WSDL (3)
■
Define complex types:
<complexType name="MDCinfo">
<sequence>
<element name="groupName" type="xsd:string"/>
<element name="groupURL" type="xsd:string"/>
<element name="queryTypes" type="impl:ArrayOf_string"/>
<element name="version" type="xsd:string"/>
</sequence>
</complexType>
39
Tutorial 2: Simple Perl Client
■
Perl script using SOAP::Lite module
■
Client only parses SOAP message
■
no reference to WSDL
40
C/C++ Client
■
Build C/C++ web-service clients using gSoap
http://gsoap2.sourceforge.net
■
Procedure:
■
Run WSDL parser tool wsdl2h that imports WSDL
and generates header files with
■
■
■
web-service operations
C/C++ data types used by the services
Run stub compiler soapcpp2 to generate
■
■
XML serialisers for the data types
Client side stubs
41
gSoap getMDCinfo client
42
Xpath: XML Path Language
■
Defines syntax for portions of XML documents
■
Example XML ID:
<?xml version="1.0"?>
<list>
<item><value>1</value></item>
<item><value>2</value></item>
</list>
■
Example XPath expression:
■
/list/item
All item elements
■
/list/item[value>1]
Only the last item element
43
XPath Examples
■
Query ensemble XML IDs:
■
All ensemble XML IDs:
■
All ensembles using Wilson plaquette action:
/markovChain
/markovChain/physics/action/gluon/plaquetteGluonAction
■
Same but restricted to β>5.3:
/markovChain//plaquetteGluonAction[beta>5.3]
■
Query configuration XML IDs:
■
All configurations that belong to a ensemble:
/gaugeConfiguration/markovStep[markovChainURI=”<uri>"]
■
It is better not to query for all configuration XML IDs
44
Tutorial 3: MDC Portals
■
■
Web interfaces:
■
http://globe-meta.ifh.de:8080/lenya/hpc/live/LDG/mdc.html
■
http://cssm.sasr.edu.au/ildg/
ILDG Browser Client
■
■
http://forge.nesc.ac.uk/projects/qcdgrid/
Download metadata using ildg-client:
ildg-get --mdc-only {__URI__|__LFN__}
% ildg-get --mdc-only \
uri://ldg/qcdsf/clover_nf2/b5p20kp13420-16x32
% ildg-get --mdc-only \
lfn://ldg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
45
FILE CATALOGUE
46
Grid File Catalogues
■
■
File catalogue task
■
Map logical file name to physical file names
■
Possibly: file metadata (e.g. checksums)
Popular implementations:
■
GT Replica Location Service (RLS)
■
■
Flexible set of attributes
LCG File Catalogue (LFC)
■
Optimised for scalability
47
Example: LFC
■
■
Objects
Replica 1
LFN 1
LFN 2
GUID
Replica 2
…
…
LFN N
Replica M
■
Logical file name (LFN)
■
Global Unique Identifier (GUID)
■
Site URL (SURL, aka physical file name)
Hierarchical namespace organised as virtual
file system
% lfc-ls -l /grid/ildg/ldg/qcdsf/clover_nf2p1
drwx---r-x 798 44054
3413 0 May 29 2007 b7p20kp12450kp12450-16x32
drwx---r-x 440 44054
3413 0 Nov 22 14:44 b7p20kp12500kp12500-16x32
drwx---r-x 599 44054
3413 0 Sep 25 2007 b7p20kp13350kp13350-16x32
■
Virtual user and group IDs + support for ACL
■
Authenticated access (like RLS)
48
Interface Service
■
Services
■
getFCinfo
■
getURL
■
■
■
Input:
Output:
List of LFNs
List of SURLs associated with each LFN
Interface service queries file catalogue on
behalf of user
→ Credential delegation required
49
Tutorial: Query File Catalogue
■
Fetch list of replicas for a particular LFN using
ildg-client:
% ildg-get --fc-only \
lfn://ldg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
srm://ccsrm02.in2p3.fr/pnfs/in2p3.fr/data/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
srm://dcache.fz-juelich.de/pnfs/fz-juelich.de/data/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
srm://dcache.zib.de/pnfs/zib.de/data/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
srm://globe-door.ifh.de/pnfs/ifh.de/acs/qcdsf/pleiter1/ukqcdsf/b5p20/kp13420-16x32/confs-ildg/ape.001.000305.dat
srm://grid-se3.desy.de/pnfs/desy.de/data/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
■
First call takes long ← credential delegation
■
■
Interface service stores delegated credential
Watch details: ildg-get –debug=3 ...
50
STORAGE ELEMENT
51
What Is a Storage Element?
■
Very simple storage element:
Storage
■
Grid
Interface
Will not work for large storage (multi-PBytes)
→ Need scaling architecture
52
GridFTP
■
■
GridFTP protocol developed by GGF as
extension of FTP protocol
Features:
■
■
■
Host A
Grid Security Infrastructure
(GSI) support
Parallel data transfer
(multiple TCP streams)
Third-party
control of data
transfer
GridFTP
Server
Host B
(source)
data
GridFTP
Server
Host C
(destination)
53
SRM
■
Storage Resource Management protocol
■
■
Reservation and scheduling of storage resources
Standardised access to heterogeneous storage
■
■
■
■
Simple disk
Robotic tape storage systems
Distributed disk cache, hierarchical storage system
Features
■
File transfer protocol negotiation
■
Dynamic transfer URL (TURL) allocation
54
SRM get
■
Client sends get to SRM server
■
■
SRM server returns
TURL
■
■
■
SURL, protocol list
GridFTP server
selected, e.g.,
depending on load
Client
Data
TURL
SURL, protocol list
Possibly wait until file
has been staged
GridFTP
Server 1
GridFTP
Server 2
TURL
SRM
Server
Client initiates file transfer from GridFTP
server
GridFTP
Server N
55
SRM Function Overview
■
Data transfer functions
■
■
Directory functions
■
■
srmCopy, srmPrepareToGet, srmRemoveFiles
srmMkdir, srmLs
Permission functions
■
srmSetPermission, srmCheckPermission
■
Space management functions
■
Status functions
56
ILDG client
■
ILDG client has to support
■
SRM
■
Transport: HTTP, GridFTP
57
ILDG File Format
■
■
All files uploaded to ILDG have to conform to
file format specification
Format based on structured files which are
packaged using LIME file format
■
LIME defined by USQCD
■
Structure:
■
■
■
Records containing ASCII or binary data
One or more records combined in one message
One or more messages in one file
58
LIME record
■
Header (144 bytes)
■
Control word (8 bytes)
■
Data length in bytes (8 bytes)
■
LIME type (128 bytes)
■
Data (1...263 bytes)
■
Null padding (0...7 bytes)
59
ILDG records
■
ildg-format
<?xml version="1.0" encoding="UTF-8"?>
<ildgFormat>
<version> 1.0 </version>
<field> su3gauge </field>
<precision> 32 </precision>
<lx>20</lx> <ly>20</ly> <lz>20</lz> <lt>64</lt>
</ildgFormat>
■
ildg-binary-data
■
■
Gauge configuration data
ildg-data-lfn
■
Logical file name, reference to metadata
60
ILDG File Format (2)
■
Users free to add own messages and records
(but must not use record type name 'ildg-*')
Message Record LIME record type
#1
...
#n
...
#m
...
...
...
...
#i
...
#j
...
...
#1
...
...
...
...
ildg-format
...
ildg-binary-data
...
...
ildg-data-lfn
...
61
How to read LIME files?
■
Minimal
example:
(coding style not
recommended)
■
Link to LIME
library from
USQCD
62
Tutorial 4: Download File
■
Demo: download file
% ildg-get \
lfn://ldg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat
■
■
For more details increase debug level, e.g.:
--debug=3
Verify checksum:
ildg_cksum <file>
■
Check contents:
lime_contents <file>
63
REGIONAL GRID OVERVIEW
64
Regional Grids: Australia
■
Catalogue services
■
dCache based storage element
■
Host of global ILDG services:
■
■
Monitoring service
■
Global web-site http://www.lqcd.org/ildg
See http://cssm.sasr.edu.au/ildg/
65
Regional Grids: Japan
■
■
■
■
Catalogue services
Relevant computing
centres access global
file system Gfarm
Grid-FTP server
provides external
access
See http://www.jldg.org
66
Regional Grids:
France/Germany/Italy
■
LatFor Datagrid (LDG)
■
Catalogue services
■
dCache based storage elements in
■
France: Lyon
■
Germany: Berlin, Hamburg, Jülich, Zeuthen
■
Italy: Parma (planned)
■
Operates user registration service
■
See http://www-zeuthen.desy.de/latfor/ldg
67
Regional Grids: UK
■
Catalogue services
■
6 storage nodes
■
See talk by George Beckett
and http://www.gridpp.ac.uk/qcdgrid/
68
Regional Grids: US
■
Catalogue services
■
dCache storage element at FNAL
■
See http://www.usqcd.org/ildg/
69
MONITORING
70
INCA Based Monitoring
■
■
■
Report manager
executes
regularly test
scripts accessing
Grid resources
Information
collected in
repository
Failures
generate email
notifications
71
ILDG Monitoring Service
■
http://www.sapac.edu.au/inca
■
Monitored services:
■
MDC
■
FC
■
SE
72
History
■
Review service stability,
e.g.:
→ UKQCD MDC is actually quite stable!
(ignore Christmas break)
73
SUMMARY
74
Summary
■
ILDG concept: Federation of regional Grids
■
■
■
■
5 regional grids in operation
Interoperability challenge:
■
Adopt common Grid standards
■
Define and implement interface services
Short introduction into Grid security
■
Certificate based authentication
■
Concept of virtual organisation
Web-service technologies (SOAP, WSDL)
75
Summary
download
operation:
76
Conclusions
■
Growing number of large physics collaborations
use ILDG infrastructure
■
Gaining grid access requires efforts
■
■
■
■
Install Grid client software
Request certificate from CA
Join the VO
Efforts pay-off
■
Distributed generation and analysis of data
77
More sites to come ...
78
QUESTIONS?
79