Grid Computing and the Role of NeSC
Prof Richard Sinnott
Technical Director National e-Science Centre
r.sinnott@nesc.gla.ac.uk
26th October 2006
Purpose of Today
• To provide Stirling folk with
– Broad overview of Grids
• Flavour of different kinds of e-Research possible
– Explain the role of NeSC
• What we have done, what we are doing, and plans for the future
– What resources we have at our disposal
• Why?
– NeSC-III continuation grant
– Grids are successful when widely adopted
• User communities essential need to be more altruistic
• Avoid the “me-science” culture
– I’m not that nice a guy, I see this as the possibility to tap into whole new
areas of collaboration with Stirling University
• Aquaculture, environmental sciences, …
e-Science and the Grid
‘e-Science is about global collaboration in key
areas of science, and the next generation of
infrastructure that will enable it.’
‘e-Science will change the dynamic of the
way science is undertaken.’
John Taylor
Director General of Research Councils
Office of Science and Technology
Grid is infrastructure used for e-Science
•Metaphor of Power Grid: compute and data resources
on demand
Foundation for e-Science
• e-Science methodologies transforming science,
engineering, medicine and business
– driven by exponential growth in data, compute demands
• enabling a whole-system approach
computers
software
Grid
sensor nets
instruments
colleagues
Shared data
archives
Data Grids for High Energy Physics
~PBytes/sec
1 TIPS is approximately 25,000
~100 MBytes/sec
Online System
SpecInt95 equivalents
Offline Processor Farm
~20 TIPS
There is a “bunch crossing” every 25 nsecs.
There are 100 “triggers” per second
~100 MBytes/sec
Each triggered event is ~1 MByte in size
~622 Mbits/sec
Tier 1
France Regional
Centre
Tier 0
Germany Regional
Centre
Italy Regional
Centre
CERN Computer Centre
FermiLab ~4 TIPS
~622 Mbits/sec
Tier 2
~622 Mbits/sec
InstituteInstitute Institute
~0.25TIPS
Physics data cache
Tier2 Centre
Tier2 Centre
Tier2 Centre
Tier2 Centre
~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS
Physicists work on analysis “channels”.
Institute
~1 MBytes/sec
Tier 4
Physicist workstations
Caltech
~1 TIPS
Each institute will have ~10 physicists working on one or more
channels; data for these channels should be cached by the
institute server
Populations
Organisms
Physiology
Tissues
Protein-protein interaction (pathways)
Protein Structures
Gene expressions
Nucleotide structures
The e-Health Future…
Next Generation Transistor Design
3D
+
Statistical
UK e-Science Budget
(2001-2006)
Total: £213M + £100M via JISC
EPSRC Breakdown
MRC (£21.1M)
10%
EPSRC (£77.7M)
37%
Applied (£35M)
Staff costs 45%Grid Resources
HPC (£11.5M)
BBSRC (£18M)
15%
8%
NERC (£15M)
7%
Computers & Network Core (£31.2M)
40%
(£57.6M)
funded separately PPARC27%
CLRC (£10M)
5%
ESRC (£13.6M)
6%
+ Industrial Contributions £25M
Source: Science Budget 2003/4 – 2005/6, DTI(OST)
Slide from Steve Newhouse
e-Science in the UK
NeSC
e-Science
Institute
Scottish
Grid Service
•
National
Centre
forUK e-Science Grid based
Previous
work on
on GT2 e-Social
– Demonstrated
Sciencebroad set of applications across it
Core NGS
Nodes
+ HPCx
+ CSAR
• Monte Carlo simulations of ionic diffusion
through radiation damaged crystal structures
• Integrated Earth system modelling
• BLAST on the Grid
• Grid Integration Test
Script Suite
Digital
• …
NERC
e-Science
Centre
OMII-UK
OMII-UK
Grid
Operations
Support
Centre
Curation
Centre
CeSC (Cambridge)
National
Institute
for
Environmental
e-Science
NeSC Mission Statement
• To stimulate and sustain the development of e-Science in the UK, to
contribute significantly to its international development and to ensure that its
techniques are rapidly propagated to commerce and industry.
• To identify and support e-Science projects within and between institutions in
Scotland, and to provide the appropriate technical infrastructure and support
in order to ensure rapid uptake of e-Science techniques by Scottish
scientists.
• To encourage the interaction and bi-directional flow of ideas between
computing science research and e-Science applications
• To develop advances in scientific data curation and analysis and to be a
primary source of top quality systems and repositories that enable
management, sharing and best use of research data.
NeSC Continuation Grant
• One of the few to be offered continued funding
– Runs from August 2006 – end July 2008
– PI is Prof. Peter Clarke
• However - limited resources
•
•
•
•
•
•
•
•
•
0.25 FTE Prof. R. Sinnott - Technical Director
0.25 FTE Dr D. Berry - Project Manager
0.5 FTE Susan Andrews - Web & Database Developer
0.33 FTE Iain Coleman - technical writer/ content editor
0.5 FTE Chris Bayliss - Software Engineer
1 FTE event team staff
0.5 FTE computing staff
0.25 FTE administrative support staff
Total 3.5FTE (over 2 sites)
NeSC-3 Plans
• 1. National leadership and coordination, and
International Representation
– National and International Activities including OGF and JISC
e-Infrastructure
– Knowledge base for UK e-Science web site
– Plan for 6 key community events per annum
– Outreach through other events
– AHM Support
– e-Science centre directors’ meetings
– Host international delegations.
– home for GridNet
NeSC-3 Plans …ctd
• 2. Promote research excellence based upon
existing and newly developing areas of strength
–
–
–
–
–
–
–
–
Scientific Data Exploitation
High-performance networking
Bioinformatics
Clinical Sciences
Electronic Engineering
Security
Emergency Response Computation Dynamics
BioMedical Image Analysis
NeSC-3 Plans …ctd
• 3. Inform/contribute to development of
national e-Infrastructure and international
standards
– Technical Development
• ETF, STF, ATF and GOSC
• Middleware OGSA-DAI/DAIT, eDIKT2
– Security expertise
– Standards such as GGF/OGF
– Infrastructure
• Scottish e-Infrastructure (more soon)
– AccessGrid
– Working closely with EUCS and GUCS
NeSC-3 Plans …ctd
• 4. E-Science outreach and uptake in the
Regional community
– plan for 3 events per year to foster uptake of
e-Science in the Scottish Region
• You are the first!!!!
• Next one in Aberdeen 13th December
– organise workshops and seminars at various
Scottish institutions on a variety of application
domains of regional interest
– fostering inter-institute projects
• SBRN, GS SFHS, GEODE, nanoCMOS, …
NeSC-3 Plans …ctd
• 5. Stimulate e-Science Education
– Grid Computing module in advanced MSc at
Department Computing Science in Glasgow
– E-Science MSc in Edinburgh
– EGEE Training Team in Edinburgh
– Involved in National Grid Service training courses
– Various summer schools
Glasgow e-Science Hub
• E-Science Hub
– Externally
• Glasgow end of NeSC
– Involved in UK wide activities
» Involved in numerous projects (more later!)
– Public visibility of NeSC
» responsible for NeSC web site (www.nesc.ac.uk)
David Martin
(ScotGrid
sys-admin)
– Internally
A. Ajayi
• Focal point for e-Science research/activities at
Glasgow
• Work closely with foundation departments
» Department of Computing Science
» Department of Physics & Astronomy
• Also working with other groups including
»
»
»
»
Bioinformatics Research Centre, Biostatistics
Electronics and Electrical Engineering
Clinicians, Hospitals, across Scotland,
Arts & Humanities, University Services …
– NeSC GU now part of University Services!
Campbell
Millar
Nano
CMOS
PC
Gordon
Stewart
Mohd
Noor
(PhD)
J. Jiang
Chris
Bayliss
In the beginning at Glasgow…
• Consolidation of resources
– Story started with building around ScotGrid
• Providing shared Grid resource for wide
variety of scientists inside/outside Glasgow
– HEP, CS, BRC, EEE, …
» Target shares established
» Non-contributing groups encouraged
Hardware
•
59 IBM X Series 330 dual 1 GHz Pentium III with 2GB memory
•
2 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory
•
3 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory
and 100 + 1000 Mbit/s ethernet
•
1TB disk
•
LTO/Ultrium Tape Library
•
Cisco ethernet switches
And then came…
•
IBM X Series 370 PIII Xeon with 32 x 512 MB RAM
•
5TB FastT500 disk 70 x 73.4 GB IBM FC Hot-Swap HDD
•
eDIKT 28 IBM blades dual 2.4 GHz Xeon with 1.5GB memory
•
eDIKT 6 IBM X Series 335 dual 2.4 GHz Xeon with 1.5GB
memory
•
CDF 10 Dell PowerEdge 2650 2.4 GHz Xeon with 1.5GB memory
•
CDF 7.5TB Raid disk
•
•
ScotGrid [ Disk ~15TB
CPU ~ 255 1GHz ]
~2.81 million CPU hours
completed
–
Compare Sun $1 CPU/hr
•
~313,500 jobs completed
•
Stats on usage available to
user groups
Glasgow e-Science Infrastructure …ctd
• Now includes…
– Computer Services second HPC facility (128 Opteron based)
– University SAN (50TB – 25TB mirrored across campus)
– ~£850k investment
– SMP donations to NeSC Glasgow
– Access to campus wide resources
• Campus wide Condor provisionally ok’ed…
• EEE compute clusters and larger SMP machines
• others…
– Scottish Bioinformatics Research Network equipment funds (~£80k)
– BBSRC REI equipment grant success (~£120k)
– Connection to UKLight network (up to 10Gbit/s)
• (ask Pete for more details on UK Light)
– Use of National Grid Service
ScotGrid Infrastructure Now
• £800k of SRIF-3 funding
– Glasgow investment builds on SRIF-1/SRIF-2 funding,
ScotGrid and eDIKT projects
• Clustervision cluster now procured
– installation well advanced
• consists of 140 dual core, dual CPU Opteron worker
nodes
– provides more than 1 million SI2K
• 100TB raw disk
– In process of becoming affiliate of the NGS
• 10% made available for NGS
Similar Story in Edinburgh
• Created the Advanced
Computing Facility
– secure site outside Edinburgh
– contains 155TB SAN and
HPC servers
– initial investment £3.8M
– Further £2M SRIF-3
investment on-going for new
compute cluster
– These investments and
Glasgow’s laying foundations
for Scottish Grid Service
Scottish Grid Service
• Case is currently being formulated
– Initial proposal outline agreed by SE as “strategically important for
Scotland”
– Aberdeen, Dundee, Edinburgh, Glasgow (lead), Heriot-Watt,
Strathclyde, Stirling (Ken), St Andrews, …
• Abertay, Highlands & Islands, …
– Be inclusive!!!!!!!!!!!!!!!
– Offer more than Grid computing resources
• Move towards service based infrastructure
• Key application areas
– High energy physics, life sciences and bioinformatics, electronics,
computing science, …
• Arts & humanities, social sciences, other things of relevance/importance
for Scotland …
– Can’t support everyone, so will likely need to cherry-pick
Finally
• Lots of opportunities in this space
– In the last 2 years I have personally been involved in 48 funding
proposals … (more in a bit!!!)
• (18 funded, 6 being reviewed, 8 in progress, don’t mention the rest!)
• We are happy to provide training
– Offer lectures/seminars on Grids/e-Science here at Stirling?
– Follow up workshop here?
• Organise events through e-Science Institute
– Themes, workshops, …
– What are Stirling interested in?
• Best place to learn more is NeSC web site
– Primary source of information on UK e-Science
– What projects, what is happening in what areas, …