• Spin out of Warwick (& Oxford) University
• Specialising in distributed (technical) computing
– Cluster and GRID computing technology
• 14 employees & growing; focussed expertise in:
– Scientific Computing
– Computer systems and support
– Presently 5 PhDs in HPC and Parallel Computation
– Expect growth to 20+ people in 2003

• Establish an HPC systems integration company..
• ....but re-invest profits into software
– Exploiting IP and significant expertise
– First software product released
– Two more products in prototype stage
• Two complementary ‘businesses’
– Both high growth

• Installations include:
– Largest Sun HPC cluster in Europe (176 proc)
– Largest Sun / Myrinet cluster in UK (128 proc)
– AMD, Intel and Sun clusters at 21 UK Universities
– Commercial clients include Akzo Noble, Fujitsu,
Maclaren F1, Rolls Royce, Schlumberger, Texaco….
• Delivered a 264 proc Intel/Myrinet cluster:
– 1.3 Tflop/s Peak !!
– Forms part of the White Rose Computational Grid

• Pre-configured ‘grid’-enabled systems:
– Clusters and farms
– The SCore parallel environment
– Virtual ‘desktop’ clusters
• Grid-enabled software products:
– The Distributed Debugging Tool
– Large-scale distributed graphics
– Scaleable, intelligent & fault tolerant parallel computing

• Choice of DRMs and schedulers:
– (Sun) GridEngine
– PBS / PBS-Pro
– LSF / ClusterTools
–
–
Condor
Maui Scheduler
• Globus 2.x gatekeeper (Globus 3 ???)
• Customised access portal

• Developed by the Real World Computing
Partnership in Japan (www.pccluster.org).
• Unique features, that are unavailable in most parallel environments:
– Low latency, high bandwidth MPI drivers
– Network transparency: Ethernet, Gigabit and
Myrinet
– Multi-user time-sharing (gang scheduling)
– O/S level checkpointing and failover
– Integration with PBS and SGE
– MPICH-G port
– Cluster management functionality

• Linux Workstation Strategy
– Integrated software stack for HPTC (compilers, tools & libraries) – cf. UNIX workstations
• Aim to provide a GRID at point of sale:
– Single point of administration for several machines
– Files served from front-end
– Resource management
– Globus enabled
– Portal
• A cluster with monitors !!

• A debugger for distributed parallel application
– Launched at Supercomputing 2002
• Aim is to be the de-facto HPC debugging tool
– Linux ports for GNU, Absoft, Intel and PGI
– IA64 and Solaris ports; AIX and HP-UX soon…
– Commodity pricing structure !
• Existing architecture lends itself to the GRID:
– Thin client GUI + XML middleware + back-end
– Expect GRID-enabled version in 2003

• Aims
– To enable very large models to be viewed and manipulated using commodity clusters
– Visualisation on (local or remote) graphics client
• Technology
– Sophisticated data-partitioning and parallel I/O tools
– Compression using distributed model simplification
– Parallel (real-time) rendering
• To be GRID-enabled within e-Science ‘Gviz’ project

• Aim to invest in new computing paradigms
• Developing parallel applications is far from trivial
– OpenMP does not marry with cluster architecture
– MPI is too low-level
– Few skills in the marketplace !
– Yet growth of MPPs is exponential…
• Most existing applications are not GRID-friendly
– # of processors fixed
– No Fault Tolerance
– Little interaction with DRM

• Throughput of parallel jobs is limited by:
– Static submission model: ‘mpirun –np …..’
– Static execution model: # processors fixed
– Scaleability; many jobs use too many processors !
– Job Starvation
• Available tools can only solve some issues
– Advanced reservation and back-fill (eg Maui)
– Multi-user time-sharing (gang scheduling)
• The application itself must take responsibility !!

• Job scheduler should decide the available processor resource !
• The application then requires:
– In built partitioning / data management
– Appropriate parallel I/O model
– Hooks into the DRM
• DRM requires:
– Typical memory and processor requirements
– LOS information
– Hooks into the application

• Additional resources may become available or be required by other applications during execution…
• Ideal situation:
– DRM informs application
– Application dynamically re-partitions itself
• Other issues:
– DRM requires knowledge of the application (benefit of data redistribution must outweigh cost !)
– Frequency of dynamic scheduling
– Message passing must have dynamic capabilities

• Optimal scheduling requires more information:
– How well the application scales
– Peak and average memory requirements
– Application performance vs. architecture
• The application ‘cookie’ concept:
– Application (and/or DRM) should gather information about its own capabilities
– DRM can then limit # of available processors
– Ideally requires hooks into the programming paradigm…

• On large MPPs, processors/components will fail !
• Applications need fault tolerance:
– Checkpointing + RAID-like redundancy (cf SCore)
– Dynamic repartitioning capabilities
– Interaction with the DRM
– Transparency from the user’s perspective
• Fault-tolerance relies on many of the capabilities described above…

• Commitment to near-term GRID objectives
– Turn-key clusters, farms and storage installations
– On going development of ‘GRID-enabled’ tools
– Driven by existing commercial opportunities….
• ‘Blue’-sky project for next generation applications
– Exploits existing IP and advanced prototype
– Expect moderate income from focussed exploitation
– Strategic positioning: existing paradigms will ultimately be a barrier to the success of (V-)MPP computers / clusters !