The Hartree Centre
&
The Daresbury Research Collaboratory in association with IBM
Jonathan Follows
HPC activities at
STFC Daresbury Laboratory
e-Infrastructure and HPC Capital Investment
• £145M funded by BIS covering
– High capacity networks
– Massive data storage
– Advanced software development for massively parallel
computers
– Space infrastructure
– HPC
• National Tier 0/1 facilities
• Distributed Tier 2/3 facilities
From BIS Presentation To Cabinet Office
“Capital Projects in Support of
Software Development Activities”
From BIS Presentation To Cabinet Office
• ICE-CSE Phases 1 and 2 (Hartree, STFC)
• Rationale for investment
– Focus on leveraging new collaborations and investment into
Daresbury to promote UK economic growth and rebalance
the economy;
– Bring together communities from academia, Government
and industry to develop a programme of multi-disciplinary
projects that will focus on developing multi-scale, efficient
and effective modelling and simulation software;
From BIS Presentation To Cabinet Office
• Provide a step-change in analytical and predictive
capabilities across many strategic themes, including
energy, life sciences, environment, materials,
security;
• In addition, the centre will develop and demonstrate
services to a diverse range of end users to
demonstrate the value of software development to
them and the positive effects on their products;
From BIS Presentation To Cabinet Office
• The investment in ICE-CSE will provide:
– Capability to host truly world class, next generation
HPC systems;
– Equip it with world leading computer systems as
software development and demonstration platforms;
• Applications areas that will benefit directly from the
investment include
– Earth Observation data-mining, Square Kilometre
Array data receiving/mining, genome sequencing,
financial transactions and predictions, seismic
exploration data interpretation
From BIS Presentation To Cabinet Office
• Lack of investment will require Daresbury to continue to
work with better equipped partners and procure run-time
elsewhere in order to develop its new algorithms and
applications software.
• This will result in a smaller, slower applications
development programme with reduced competitiveness of
UK industry
IDC said ….
• Software leadership will become the New
Battleground
– Only 1% of HPC codes can exploit 10,000 or more cores
– There May Be More Emphasis on Software … Finally
• Big Data Methods Will Start to Transform the HPC
Market, Including Storage
– HPC and commercial Big Data are starting to collide
• Petascale Performance on Big Systems Will Create
New Business Opportunities
• The HPC Staffing Shortage Will Grow More Acute
Return on investment
•
•
•
•
UK tax revenues exceed 35% of GDP
Forecast tax revenue for 2016 is >£700,000,000,000
Forecast GDP for 2016 is >£2,000,000,000,000
A 0.1% increase in GDP leads to additional
annual£700,000,000 in tax revenues (net taxes and
national insurance contributions) (and IDC and others
forecast a much greater increase in GDP than this)
• So a £145,000,000 investment shows a pretty decent
return on investment; even a 0.01% increase in GDP
pays for itself after 2 years
• And, by the way, we paid £6,300,000 in VAT for our
HPC systems!
Aims of the Centre (1)
• Drive the adoption of “HPC” in UK industry
– to improve their competitiveness and generate wealth Nextgeneration applications
• Delivering step-change improvements in scientific
capabilities
• Develop world-class “HPC” technology & skills in
Daresbury Laboratory
Aims of the Centre (2)
• Develop & enhance research collaboration
– nationally & internationally
– on key focus areas for the UK
• Foster innovation in academia & commerce
• Educational focus on computational science & engineering
• Establish a better career structure for staff
– who support computational science and engineering in academia
An “ecosystem” for HPC
• The Hartree Centre needs to be, and is designed to
be complementary to other HPC facilities in the UK
– HECToR, ARCHER, new Tier 2 systems (such as HPC
Midlands), University HPC systems
• One model for the Hartree Centre is one in which the
entire system can be reserved for one purpose for a
period of time – akin to a telescope facility
• Software generation and a wide range of
collaborators are the focus areas for the Hartree
Centre
Douglas Rayner Hartree
• Hartree – Fock
• Appleton – Hartree Equation
• Differential Analyser
• Numerical Analysis
• Father of Computational Science
Douglas Hartree with Phyllis Nicolson at the
Hartree Digital Analyser at Manchester University
Hartree Digital Analyser
• He said in 1946
It may well be that the high-speed digital computer will
have as great an influence on civilization as the
advent of nuclear power.
• He wrote Numerical Analysis in 1952 which became
a classic of the subject.
Anyone intending to undertake a serious piece of calculation should realise
that adequate checking against mistakes is an essential part of any
satisfactory numerical process.
No one, and no machine, is infallible, and it may fairly be said that the
ideal to aim at is not to avoid mistakes entirely, but to find all mistakes
that are made, and so free the work from any unidentified mistakes.
The Project
• To establish a world leading centre with a world
leading service ethos
– Phase A (Initiation):
• Establish the staff, operating procedures, and systems
• Establish the IBM Collaboratory
– Phase B (On-going):
• On-going operation of the centre
• Detailed governance structure
How (1)
• Purchase of internationally competitive DONE
–
–
–
–
–
–
massively parallel systems
a high-throughput system
a data-intensive computation system
advanced visualization systems
145 days from launch of tender to contract signature
Majority of capital spend in 2011-12 financial year
• Establish a collaboration with IBM DONE
– for R&D and business development
– as part of a single contract for £31½m
• STFC funding for recurrent spend DONE
How (2)
• Establishing the centre with a service ethos
• Tackle with industry major computational problems.
• Promote uptake of “HPC” to commerce
• Foster collaboration & inward investment to DL
• Outreach programme
– raise awareness with the public and policy-makers
How(3)
It is important to place a higher value upon the position
of “scientific programmer” and also “data curator” in
the academic environment and to offer more career
opportunities to these staff.
Scientific programmers combine the knowledge of
the underlying scientific discipline with
implementation, optimisation and parallelisation for
high-end systems: they are important in obtaining
highly-efficient application implementations.
– Dominic Tildsley, e-Infrastructure report
Our hardware
• Data centre upgrade – water cooling
• 7 racks of BG/Q 1.46Pflop/s
• IBM iDataPlex with 8192 cores 170Tflop/s
– Including 8x4TB ScaleMP large memory systems
• ~6PB of DDN disk storage with GPFS
• Tape library
• Visualisation systems in Daresbury and Rutherford
Appleton Laboratory
• Computers undergoing final installation and
commissioning
• Visualisation waiting on some building work to
complete
Other Collaborators
Scientific Computing
(CSED &
e-Science)
Hartree
Centre
Business
Innovation
Department
Daresbury Research Collaboratory in association with IBM
Initial Governance
STFC
A d viso ry G ro u p
V is u a liz a tio n A d v is o ry G ro u p
IB M
H a rtre e C e n tre
M anagem ent Team
E xte rn a l In p u t
U se r G ro u p
O p e ra tio n a l T e a m s
Main Elements of Plan
Area
One Word Description
Systems Support
Hardware
Applications Support
Software
User Support
People
Science & Engineering
R&D
Business Development
Pipeline
Governance
Monitoring
Marketing & Communications Information
The Service
• Includes all aspects of our work
– Users, R&D projects, collaborators (commerce & academia),
public, policymakers
– The service is the centre, not just the delivery of computer
systems
• World Class
– Process for everything
– Metrics
• But only those which matter in the context of our centre
– Those processes under continuous improvement
Structure
User
Domain Specific Support
Service R&D
Computational Science & Engineering
Support
Commercial Applications
Support
Service Manager
In House Application
Support
User Management & User Support
HPC System Administration
Core IT Infrastructure
Networks, Security, Server Admin, User Desktop, …
The HPC
Service
Progress
Interim Management Team
In Place
Project Managers
In Place
Project Plan
Nearly Ready
Systems
Nearly Accepted
Centre Structure & Governance
Initial version agreed
Staffing levels
Initial version agreed
Early Access Users
Panel approved 15 projects
Thank you for listening
Any Questions?