http://www.grid-support.ac.uk
http://www.ngs.ac.uk
The National Grid Service
Mike Mineter
mjm@nesc.ac.uk
http://www.nesc.ac.uk/
http://www.pparc.ac.uk/
http://www.eu-egee.org/
Acknowledgements
• Most NGS and GOSC slides are taken from a
talk by Stephen Pickles, Technical Director of
GOSC
• Also slides from Malcolm Atkinson on e-Science
programme
• A few I even did myself (If you find an error,
you’ve one of mine! Tell me please.)
3
Overview
• The UK e-science programme:
building and using e-Infrastructure
• GOSC and the NGS
• NGS middleware and services
4
What is e-Infrastructure –
Political view
• A shared resource
– That enables science,
research, engineering,
medicine, industry, …
– It will improve UK /
European / … productivity
• Lisbon Accord 2000
• E-Science Vision SR2000
– John Taylor
– Commitment by UK
government
• Sections 2.23-2.25
– Always there
• c.f. telephones, transport,
power, internet
5
UK e-Science Budget
(2001-2006)
Total: £213M + £100M via JISC
EPSRC Breakdown
MRC (£21.1M)
10%
EPSRC (£77.7M)
37%
Staff costs only Applied (£35M)
45% Grid Resources
HPC
(£11.5M)
BBSRC (£18M)
15%
8%
NERC (£15M)
7%
Computers & NetworkCore (£31.2M)
(£57.6M)
40%
funded separately PPARC27%
CLRC (£10M)
5%
ESRC (£13.6M)
6%
+ Industrial Contributions £25M
Source: Science Budget 2003/4 – 2005/6, DTI(OST)
6
The e-Science
Centres e-Science
Institute
Globus Alliance
National
Centre for
e-Social
Science
Grid
Operations
Support
Centre
Digital
Curation
Centre
Open
Middleware
Infrastructure
Institute
CeSC (Cambridge)
http://www.nesc.ac.uk/centres/
EGEE
National
Institute
for
Environmental
e-Science
7
GOSC and the NGS
8
The National Grid Service
The NGS is the core UK grid, intended for the
production use of computational and data grid
resources. NGS is the core service resulting
from the UK's e-Science programme. NGS is
supported by JISC, and is run by the Grid
Operations Support Centre (GOSC).
9
GOSC
The Grid Operations Support Centre is a
distributed “virtual centre” providing deployment
and operations support for the UK e-Science
programme.
10
GOSC
More on this tomorrow!!
11
NGS Organisation
•
Operations Team
–
–
–
–
–
•
Technical Board
–
–
–
–
–
–
•
led by Andrew Richards (RAL)
representatives from all NGS core nodes
meets bi-weekly by Access Grid
day-to-day operational and deployment issues
reports to Technical Board
led by Stephen Pickles
representatives from all sites and GOSC
meets bi-weekly by Access Grid
deals with policy issues and high-level technical strategy
sets medium term goals and priorities
reports to Management Board
GOSC Board meets quarterly
–
–
representatives from funding bodies, partner sites and major stakeholders
sets long term priorities
12
Managing middleware
evolution
• Important to coordinate and integrate this with deployment and
operations work in EGEE, LCG and similar projects.
• Focus on deployment and operations, NOT development.
EGEE…
Other software
sources
Prototypes &
specifications
NGS
ETF
Software with proven
capability & realistic
deployment
experience
OMII
‘Gold’ services
UK
Campus
Operations and
other
Grids
Feedback & future requirements
Engineering Task Force
Deployment/testing/advice
13
http://www.ngs.ac.uk
NGS core nodes:
Need UK e-Science
certificate (1-2 days)
Apply through NGS
web site (1-2 weeks)
14
Gaining Access
NGS core nodes
National HPC services
•
•
data nodes at RAL and Manchester
compute nodes at Oxford and Leeds
•
HPCx
•
•
•
•
free at point of use
apply through NGS web site
accept terms and conditions of use
light-weight peer review
•
CSAR
–
•
•
1-2 weeks
to do: project or VO-based application
and registration
Must apply separately to research councils
all access is through digital X.509
certificates
Digital certificate and
Conventional (username/ password)
access supported
–
–
from UK e-Science CA
or recognized peer
15
UofD
U
of
A
H
P
C
x
Commercial
Provider
PSRE
Man. Leeds
GOSC
RAL
Oxford
C
S
A
R
U
of
B
U
of
C
NGS Core Nodes: Host core services, coordinate integration, deployment and support
+free to access resources for all VOs. Monitored interfaces + services
NGS Partner Sites: Integrated with NGS, some services/resources available for all VOs
Monitored interfaces + services
NGS Affiliated Sites: Integrated with NGS, support for some VO’s
Monitored interfaces (+security etc.)
16
http://www.grid-support.ac.uk
http://www.ngs.ac.uk
Middleware & Services
http://www.nesc.ac.uk/
http://www.pparc.ac.uk/
http://www.eu-egee.org/
Globus Toolkit
• Globus Toolkit version 2
– GT 2.4.3 from VDT 1.2
– (VDT comprises several middleware packages used
in EU, US and UK grids)
•
•
•
•
Job submission (GRAM)
File transfer (GridFTP)
Shell (GSI-SSH)
Information Services (MDS/GIIS/GRIS)
– Information providers from GLUE schema
18
NGS software
• Computation services based on GT2
– Use compute nodes: for sequential or parallel jobs,
primarily from batch queues
• Data services:
– Storage Resource Broker:
• Primarily for file storage and access
– “OGSA-DAI”: Data Access and Integration
• Primarily for grid-enabling databases (relational, XML)
• Note especially: the NGS Oracle service
– Used by NGS services
• OGSA-DAI and Oracle are available to users – but apply via
GOSC
19
SRB
• Storage Resource Broker from SDSC
– Location transparent access to storage
• Access a file in SRB across a grid, interface is similar to POSIX I/O
• Mechanism used to access same files from any NGS compute node
(unless you synch your accounts)
– Metadata catalog
– Replica management
• Clients on compute nodes
• Servers on data nodes
• Java and Windows clients for desktop/UI machines
20
Oracle
• Oracle 9i database
• Only on data nodes
• Populated by users/data providers
• Infrastructure maintained by NGS database
administrators
• Used directly or via OGSA-DAI
21
OGSA-DAI
• Open Grid Services Architecture (OGSA)
• Database Access and Integration (DAI)
• Developed by UK e-Science projects OGSA-DAI and
DAIT
• OGSA-DQP (Distributed Query Processor)
• Experimental service based on OGSI/GT3 on
Manchester data node only
– considering WS-I and WSRF flavours
– Uses Oracle underneath
• Early users from e-Social Science (ConvertGrid)
22
The NGS portal
• More on this later…
• NGS has services for “data grid” and “compute grid”;
portal gives easier access to some of these and also
support for the “people grid”
• Portlets for
–
–
–
–
Collaborations to use – e.g. chat
Job submission
Access to information services
GridFTP
• http://portal.ngs.ac.uk/
23
Key facts
• Production: deploying middleware after selection
and testing – major developments via
Engineering Task Force.
• Evolving:
– Middleware
– Number of sites
– Organisation:
• VO management
• Policy negotiation: sites, VOs
• International commitment
24
Web Sites
• NGS
– http://www.ngs.ac.uk
– To see what’s happening: http://ganglia.ngs.rl.ac.uk/
• GOSC
– http://www.grid-support.ac.uk
• CSAR
– http://www.csar.cfs.ac.uk
• HPCx
– http://www.hpcx.ac.uk
25
Summary
• NGS is a production service
– Therefore cannot include latest research protoypes!
– ETF recommends what should be deployed
• Core sites provide computation and also data services
• NGS is evolving
– OMII, EGEE, Globus Alliance all have m/w under assessment by
the ETF for the NGS
• Selected, deployed middleware currently provides “low-level” tools
– New deployments will follow soon
– New sites and resources being added !
26