Commercialisation at NeSC
An operational perspective
Dr Rob Baxter
Software Development Group Manager
NeSC Review, 30/09/2003
Overview
NeSC project strategy
The GCPs and edikt
Some highlights of the year
How do we do it?
Where are we going?
NeSC project strategy
The Grid must be built on standards
cf. the Internet, the World Wide Web
Core middleware is fundamental
cf. TCP sockets libraries, http servers
Our strategy has been
identify e-Science application needs
develop standards and core middleware
build applications, leveraging middleware
This has proved very successful for e-Science
and provides a solid foundation for e-Business
Software projects at NeSC
NeSC has two flavours of project
development - the GCP projects
research and development - the edikt programme
GCP projects
funded by £3.6m EPSRC/DTI (+ equal in-kind)
allow focused development on key e-Science problems
demand direct engagement with business
edikt
funded by £2.3m SHEFC RDG
allows exploration of new ideas, some good, some bad
NeSC project portfolio
GCP:
SunDCG - m/w
OGSA-DAI - m/w
GridWeaver - fabric
MS.NETGrid - m/w
FirstDIG - app
PGPGrid - app
BRIDGES - app
ODD-Genes - demo
edikt:
• BinX - m/w
• Eldas - m/w
• AstroBinX - app
• Osage - app
[chronological starts]
Collaborators roll-call
In the last two years we’ve worked with
IBM, Sun Microsystems, HP
Oracle, Microsoft
First plc, Pepper’s Ghost Productions ltd, Raytheon
Company inc
and
AstroGrid, myGrid
ESNW, NEReSC, LeSC, GSC
Scottish Centre for Genomic Technology & Informatics,
MRC Human Genetics Unit
Some highlights
OGSA-DAI
£1.8m, NeSC, EPCC, ESNW, NEReSC, IBM, Oracle
core data access middleware based on GGF standards
X
OGSA-DAI team helping write the specifications
in use in e-Science projects
X
AstroGrid, myGrid, FirstDIG, BioSimGrid, BioGrid (Jp)
1000 downloads reached last week
follow-on project planned (DAIT, £1.5m)
gained EPCC/NeSC entry to the Globus Alliance
X
only UK site alongside Argonne, ISI and PDC
Some highlights cont.
MS.NETGrid
demonstration and training of OGSI on .NET
full house at AHM 2003!
X
great interest at the NeSC booth too
delivered course & demos 9-10/09/03 @ eSI
invited by MS to lecture on Grid/.NET, Budapest, Oct
FirstDIG
EPCC, FirstBus South Yorkshire
application of OGSA-DAI and related data mining
X
to draw together real, disparate, dirty data sources to add real value
“the results of this exercise will revolutionise the way we do things
in the bus industry”
X
Darren Unwin, Divisional Computing Manager, FirstBus South Yorkshire
Some highlights cont.
ODD-Genes
EPCC, GTI, MRC HGU
a demonstration of OGSA-DAI and SunDCG in action
the highlight of the NeSC booth at AHM 2003!
“This project has demonstrated how Grid technologies
can be used to enable true e-Science - discoveries
that would not otherwise have been achieved without
this infrastructure in place”
X
Professor Peter Ghazal, Director, GTI
How do we do it?
The NeSC/EPCC SDG thinks like a company
we do fixed-priced contracts to deliver software X against
customer requirements
must plan and execute properly or we get cost overrun
but we also operate on the technology bleeding edge…
X
…so we multiply all our task estimates by ×2.3 ☺
Software Development Group: 30 staff
SDG Manager
3 Project Managers, 2 Architects
5 Principal Consultants/Team Leaders
19 software developers
Well-defined project lifecycle
selection, planning, execution, delivery and review
Project selection
Two flavours, two methods
GCP Projects
Commercial Group prospect based around CommStrat
proposals developed with clients and SDG project staff
“project licence” agrees handover from CG to SDG
edikt
edikt management define proposals based on work with
our e-Science stakeholders
project proposals reviewed by Advisory Board
Project planning
Older projects used standard development plan
Newer projects use more comprehensive project
management plan
a quality plan template based on NASA SEL and
other sources (CMU SEI, Microsoft, Oracle)
covers all aspects of the project
X
X
X
X
X
development plan
risk and issue monitoring
project tracking metrics
documentation and coding standards
QC and QA
Project planning cont.
Leading edge development needs flexibility
We use phased development models
staged delivery
X
X
good where scope is ill-defined and we expect requirements to
change/evolve, e.g. OGSA-DAI, SunDCG
define scope in quarterly blocks (say)
design to schedule
X
X
good where scope is well-understood and initial requirements are
well defined, e.g. MS.NETGrid, FirstDIG
work against prioritised requirements until effort runs out
evolutionary prototyping
X
good for demos or exploring requirements, e.g. ODD-Genes
Project execution
All project teams have
project leader
developers
technical reviewers
Bigger projects will have an architect…
e.g. OGSA-DAI, edikt projects
…and/or a Review Board
e.g. OGSA-DAI, SunDCG, edikt
Weekly team meetings are expected
action lists, task breakdowns, risk & issue lists
Project execution cont.
Based on standard document templates
requirements, component design, test…
Use standard tools
e.g. Rational Rose, TogetherSoft, JUnit, ant
Test early, test often
aim for high unit test coverage
have developed automated test setups for distributed
middleware over course of GCP projects
Monitor overruns, apply triage
review plans regularly, revise if required
Delivery and review
All projects postmortemed
what was good? what was bad? what can we learn?
For customer-led projects, delivery is easy
site visit, installation, training
limited period of free support
longer term support contracts always offered
Middleware is a bit different
monitor web downloads
set up web-based bug reporting
disseminate, disseminate, disseminate!
Where is this taking us?
Solutions for e-Science today…
…are solutions for e-Business tomorrow
universal data access and integration
universal data format description
Web and Grid services
X
WSDL, SOAP, HTTP, Java, .NET
distributed resource management
distributed, heterogeneous systems integration
Our projects reflect our vision for the
commercial Grid
In summary
NeSC’s software development programme has been - and
continues to be - a great success
We leverage off existing strengths at EPCC
the institute that thinks it’s a company
Professional planning and execution is key
backed by an unrivalled CS research base
International recognition
from a standing start to the Globus Alliance in 2 years
SunDCG software already distributed by Sun
Real business impact
“this will revolutionise the way we do things”
Darren Unwin, FirstBus South Yorkshire