The UK e-Science Programme & The National e-Science Centre Malcolm Atkinson Director of NeSC Universities of Edinburgh and Glasgow Scottish Regional Forum 17th January 2002 Contents What is e-Science? What do we expect from the Grid? Some examples of e-Science projects The UK e-Science Programme NeSC’s Role & Structure e-Science Institute The Road ahead What is e-Science? An acceleration of a trend? A sea change in scientific method? A new opportunity for science? Accelerating Trend More and More data Instrument resolution doubling /12 months Instrument and telemetry speeds increasing Storage capacity doubling / 12 months Number of data sources doubling / ?? months More and More Computation Computations available doubling / 18 months Analyses and simulations increasing Faster networks Raw bandwidth doubling / 9 months These Integrate and Enable More interplay between computation and data More collaboration among scientists, medics, engin…. More international collaboration Sea Change In Silico discovery Exploration of data and models predicts results Verified by directed experiments Combinatorial chemistry Gene function Protein Structure, … Shared Resources Researcher’s Workbench Laboratory team Multi-national network of labs + modellers Public instruments, repositories and simulations Floods of (public) data + diversity More than can be used by human inspection Gene sequence doubling / 9 months 1. 2. 3. Searches required doubles / 4.5 months Prior test against data and models Experimental Procedures Sanity check on results against data and models But … Skilled scientists and computer scientists Roughly static in number Diminishing in available attention for any task Distributed systems remain hard E.g. component failures and latency are always with us Important data still in documents More subjects experiencing the Data deluge Analysis avalanche Simulation bonanza Collaboration growth Must therefore find general solutions And make technology easier to use The New Behaviour Shared Infrastructure Intrinsically distributed Intrinsically multi-organisational Multiple uses interwoven Shared Software A new attempt at making distributed computing economic, dependable and accessible Scientists from all disciplines share in its design and use Shared & Automated System Administration Replicated farms of replicated systems Autonomic management Immediate benefit Faster transfer of ideas and techniques between disciplines Amortisation of development, operation and education Not Just Scientists Engineers They already travel the same path Finance, economy, politics, … We can expect best use of data and models to guide the decisions that affect our lives e.g. home climate simulation may moderate greenhouse gas emissions Medicine See above Industry & Commerce See above The UK Office of Science & Technology Has these extensions firmly in mind So have twelve computing & S/W companies Signed agreements with GGF Several Assumptions The Technology is Ready Not true — its emerging Joining in the task of building middleware Of Advancing Standards Of Developing Dependability The Scientists / Engineers, … want this Not universally true Addressed by Pilot projects and Demonstrators Addressed by The e-Science Institute One Size Fits All Not true Addressed by a minimum set of composable virtual services But starting with Globus It’s only for “big” science No — “small” science collaborates too! We know how we will use grids No — Disruptive technology Online Access to Scientific Instruments Advanced Photon Source wide-area dissemination real-time collection archival storage desktop & VR clients with shared controls tomographic reconstruction DOE X-ray grand challenge: ANL, USC/ISI, NIST, U.Chicago From Steve Tuecke 12 Oct. 01 Supernova Cosmology Requires Complex, Widely Distributed Workflow Management Mathematicians Solve NUG30 Looking for the solution to the NUG30 quadratic assignment problem An informal collaboration of mathematicians and computer scientists Condor-G delivered 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites) 14,5,28,24,1,3,16,15, 10,9,21,2,4,29,25,22, 13,26,17,30,6,20,19, 8,18,7,27,12,11,23 MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin From Miron Livny 7 Aug. 01 Network for Earthquake Engineering Simulation NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other On-demand access to experiments, data streams, computing, archives, collaboration NEESgrid: Argonne, Michigan, NCSA, UIUC, USC From Steve Tuecke 12 Oct. 01 Home Computers Evaluate AIDS Drugs Community = 1000s of home computer users Philanthropic computing vendor (Entropia) Research group (Scripps) Common goal= advance AIDS research From Steve Tuecke 12 Oct. 01 UK e-Science e-Science and the Grid ‘e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ ‘e-Science will change the dynamic of the way science is undertaken.’ John Taylor Director General of Research Councils Office of Science and Technology From presentation by Tony Hey e-Science Programme DG Research Councils E-Science Steering Committee Director’s Awareness and Co-ordination Role Grid TAG Director Director’s Management Role Generic Challenges Academic Application Support EPSRC (£15m), DTI (£15m) Programme Research Councils (£74m), DTI (£5m) PPARC (£26m) BBSRC (£8m) MRC (£8m) NERC (£7m) £80m Collaborative projects ESRC (£3m) EPSRC (£17m) CLRC (£5m) Industrial Collaboration (£40m) From Tony Hey 27 July 01 UK e-Science Initiative (1) £120M 3 Year Programme to create the next generation IT infrastructure to support e-Science and Business SR2000 – Funded UK e-Science Grid and Grid Support Centre, e-Science Application research projects and industrial collaboration SR2002 – Bidding for additional funding to extend scope of e-Science programme Essential that UK plays a leading role in Global Grid development with the USA, EU and Asia From presentation by Tony Hey UK e-Science Initiative (2) £120M Programme over 3 years £75M is for Grid Applications in all areas of science and engineering £10M for Supercomputer upgrade £35M for development of ‘industrial strength’ Grid middleware Require £20M ‘matching’ funds from industry Prof. Tony Hey Director of the Core Programme From presentation by Tony Hey UK Grid Network Edinburgh Glasgow Newcastle Belfast Manchester DL Cambridge Hinxton Oxford Cardiff RAL London Southampton From Tony Hey 27 July 01 SuperJanet4, June 2002 Scotland via Glasgow 20Gbps 10Gbps 2.5Gbps 622Mbps 155Mbps Scotland via Edinburgh WorldCom Glasgow WorldCom Edinburgh NNW NorMAN YHMAN Northern Ireland MidMAN WorldCom Manchester WorldCom Leeds EMMAN WorldCom Reading WorldCom London EastNet TVN South Wales MAN WorldCom Bristol External Links WorldCom Portsmouth LMN SWAN& BWEMAN LeNSE Kentish MAN From presentation by Tony Hey Access Grid Access Grid Nodes Technology Developed by Rick Stevens’ group at Argonne National Laboratory Access Grid will enable informal and formal group to group collaboration Distributed lectures and seminars Virtual meetings Complex distributed grid demos Uses MBONE and MultiCast Internet Technologies From presentation by Tony Hey Grid Middleware R&D £16M funding available for industrial collaborative projects £11M allocated to Centres projects plus £5M for ‘Open Call’ projects - approved £0.5M ‘Centre’ project with Imperial College Sun Centre of Excellence Set up two Task Forces - Database Task Force (Chaired by Norman Paton from Manchester Centre) - Architecture Task Force (Chaired by Malcolm Atkinson, Director of NeSC) From presentation by Tony Hey IRC ‘Grand Challenge’ Project Equator: Technological innovation in physical and digital life AKT: Advanced Knowledge Technologies DIRC: Dependability of Computer-Based Systems MIAS: From Medical Images and Signals to Clinical Information From presentation by Tony Hey e-Healthcare Grand Challenge Funding £0.5M projects to give Grid dimension to these IRCs Funding £2M Joint IRC projects with MIAS on e-Healthcare application Example: Breast cancer surgery – normalization of mammography and ultrasound scans - FE modelling of breast tissue Deliver useful clinical information to surgeon ensuring privacy and security From presentation by Tony Hey UK e-Science Projects £75M for e-Science application ‘pilots’ - spans all sciences and engineering Particle Physics and Astronomy (PPARC) - £20M GridPP and £6M AstroGrid Engineering and Physical Sciences (EPSRC) - funding 6 projects at around £3M each Biology, Medical and Environmental Science - projects with total value of £20M will be announced soon From presentation by Tony Hey Particle Physics and Astronomy e-Science Projects GridPP links to EU DataGrid, CERN LHC Computing Project, US GriPhyN and PPDataGrid Projects, and iVDGL Global Grid Project AstroGrid links to EU AVO and US NVO projects From presentation by Tony Hey EPSRC e-Science Projects (1) Comb-e-Chem:Structure-Property Mapping Southampton, Bristol, Roche, Pfizer, IBM DAME: Distributed Aircraft Maintenance Environment York, Oxford, Sheffield, Leeds, Rolls Royce Reality Grid: A Tool for Investigating Condensed Matter and Materials QMW, Manchester, Edinburgh, IC, Loughborough, Oxford, Schlumberger, … From presentation by Tony Hey EPSRC e-Science Projects (2) My Grid: Personalised Extensible Environments for Data Intensive in silico Experiments in Biology Manchester, EBI, Southampton, Nottingham, Newcastle, Sheffield, GSK, Astra-Zeneca, IBM, Sun GEODISE: Grid Enabled Optimisation and Design Search for Engineering Southampton, Oxford, Manchester, BAE, Rolls Royce Discovery Net: High Throughput Sensing Applications Imperial College, Infosense, … From presentation by Tony Hey Comb-e-Chem Structure-Property Mapping Goal is to integrate structure and property data sources within knowledge environment to find new chemical compounds with desirable properties - Accumulate, integrate and model extensive range of primary data from combinatorial methods - Support for provenance and automation including multimedia and metadata Southampton, Bristol, Cambridge Crystallographic Data Centre, Roche Discovery, Pfizer, IBM From presentation by Tony Hey MyGrid e-Science Workbench Goal is to develop ‘workbench’ to support: Experimental process of data accumulation Use of community information Scientific collaboration Provide facilities for resource selection, data management and process enactment Bioinformatics applications Functional genomics, pattern database annotation Manchester, EBI, Newcastle,Nottingham, Sheffield, Southampton GSK, AstraZeneca, Merck, IBM, Sun, ... From presentation by Tony Hey e-Science Demonstrators Dynamic Brain Atlas Biodiversity Chemical Structures Mouse Genes Robotic Astronomy Collaborative Visualisation Climateprediction.com Medical Imaging/VR From presentation by Tony Hey Contents What is e-Science? What do we expect from the Grid? The UK e-Science Programme NeSC’s Role & Structure The Road ahead NeSC’s context Coordination e-Science Centres Application Pilots IRCs … e-Scientists, Grid users, Grid services & Grid Developers GNT DBTF ATF TAG NeSC GSC UK Core Directorate eSI CS Research Global Grid Forum … NeSC’s Roles Stimulation of Grid & e-Science Activity Users, developers, researchers Education, Training, Support Think Tank & Research Coordination of Grid & e-Science Activity Regional Centres, Task Forces, Pilots & IRCs Technical and Managerial Fora Support for training, travel, participation Developing a High-Profile e-Science Institute Meetings Visiting Researchers International Collaboration Regional Support Portfolio of Industrial Research Projects NeSC — The Team Director Malcolm Atkinson (Universities of Glasgow & Edinburgh) Deputy Director Arthur Trew (Director EPCC) Commercial Director Mark Parsons (EPCC) Regional Director Stuart Anderson (Edinburgh Informatics) Chairman Richard Kenway (Edinburgh Physics & Astronomy) Initial Board Members Muffy Calder (Glasgow Computing Science) Tony Doyle (Glasgow Physics & Astronomy) Centre Manager Anna Kenway Conference Manager Andrea Grainger e-Science Institute The Story so Far August & September 3 workshops week 1: DF1, GUM1 & DBAG1 HEC2 and the Grid preGGF3 & DF2 October Steve Tuecke Globus tutorial (oversubscribed) 4-day workshop Getting Going with Globus (G3) – Reports on DataGrid & GridPP experience Biologist Grid Users’ Meeting 1 (BiGUM1) November GridPP Configuration management December Architecture & Strategy with Ian Foster et al. AstroGrid DIRC meeting 625 participants, 107 organisations, so far eSI Highlights cont. 2002 January Regional meeting Steve Tuecke et al. 4 day Globus Developers’ Workshop Pilot project workshop Grid Portals & Problem Solving Environments Workshop February — closed for renovation March Protein folding Workshop 14th to 17th IBM sponsor April XML, XML Schema, Web Services Tutorials Getting OGSA Going Workshop Managing Grid Software Projects Advanced Tutorials Digital Libraries, Librarians, Museums and the Grid May Mind and Brain Workshop eSI continued 21st to 26th July 2002 GGF5 & HPDC 11 EICC August Research Festival ? 14th to 16th April 2003 Dependability Suggestions Please e-Science Institute Welcomes suggestions and organisers Any topic related to e-Science How your subject may use e-Science How your technology may benefit e-Science Any format Tutorial, advanced tutorial, workshop, scientific meeting We can give travel, organisation, accommodation support This building renovated! Mail director@nesc.ac.uk Contents What is e-Science? What do we expect from the Grid? The UK e-Science Programme NeSC’s Role & Structure The Road ahead Where to Concentrate International & Industrial Collaboration Ideas, experiments, software, standards Integrating Data across the Grid Data growth demands new methods Data ownership expects respect & security Data is hard to scan — indexing & query Data is hard to move — query & move code Human attention is scarce but essential Machine-assisted annotation, provenance, archiving Machine-assisted data mining Machine-assisted ontology construction & integration Human-factors must drive designs Dynamic, Dependable and Virtual Fabric Improved Programming Models For more Information Ask me www.nesc.ac.uk director@nesc.ac.uk Thank you for your attention or for arriving early for the next talk