MUST - Multicast Streaming Technology for the Grid
Dr John Brooke, Michael Daw (University of Manchester)
Dr Maziar Nekovee, Dr Sverrir Olafsson, Ivan Skenderoski (BT Exact)
Abstract
The aim of the project is to increase the awareness and use of multicast technologies within the
Grid community, so making more efficient use of bandwidth and increasing data throughput.
This aim will be achieved through three main goals. The first is to implement selected reliable
multicast technologies on RealityGrid’s testbed, evaluate their performance through a sequence
of traffic monitoring experiments and document the result in an overview report. Based on this
evaluation, our second aim is to implement a reliable multicast toolset in RealityGrid’s
middleware (possibly within the framework of Grid FTP) and make it available to the Grid
community. Our third goal is to perform a sequence of traffic monitoring and user experience
experiments with Access Grid, which uses RTP/UDP stack and IP multicast, and document our
findings in an evaluation report.
IP multicast contrasts with more
conventional unicast networking because
data packets are only copied when
necessary. Consider the scenario of a source
computer broadcasting data to many
receiver computers. Via unicast, multiple
copies of the same data must be made and
transmitted. This is not an efficient use of
the network. Via multicast, copies of the
data are made only when required. Only
one copy of the data is sent along parts of
the route that are the same for any subset of
the receivers.
Initial work on the MUST project will
implement a number of open source reliable
multicast protocols in the RealityGrid
testbed and evaluate their performance
through a sequence of experiments with
high data rate one-to-many transport
scenarios. This evaluation will be in terms
of the improvement in throughput and
bandwidth saving that they provide in
comparison to unicast GridFTP, their
possible performance bottlenecks, their
impact on network traffic and their relative
merits. The result will be documented in an
overview document.
Our focus here will be on protocols which
build reliability on top of UDP/IP multicast
transport through a combination of
acknowledgement mechanisms and forward
error correction, and for which pilot
implementations for different platforms
already exist or are in the final development
phase. The important feature of these
protocols, which make them suitable for
Grid application, is that they achieve
reliability and scalability without any
additional assistance from the network
infrastructure. Thus their deployment does
not require any change to network routers.
The specific protocols we have in mind are:
•
1
Multicast Dissemination Protocol
(MDP)1 and its successor NORM2
(currently under development by
Reliable Multicast Working Group
of IETF) which make use of
negative
acknowledgements
(NACKs) combined with timerbased feedback control3 and
forward error correction to achieve
scalable and reliable multicast.
See MDP web site
http://manimac.itd.nrl.navy.mil/MDP/ for
further information and relevant technical
publications.
2
See NORM web site
http://norm.pf.itd.nrl.navy.mil/
3
M. Nekovee and S. Olafsson, "Timer-based
Feedback in Multicast Communication"
(submitted for publication).
•
•
Local Group based Multicast
Protocol (LGMP)4 developed by
LGC working group at University
of Karlsruhe, Germany.
InterGroup Protocol5, currently
under development at Lawrence
Berkeley National Laboratory, US.
This is attempting to implement
secure and reliable multicast.
Once this initial evaluation is complete,
work will begin on optimising the
performance of these protocols (in terms of
system performance and network impact).
We will use these optimised protocols to
deliver a reliable multicast toolset for use
by Grid applications.
MUST also aims to prove the claimed
benefits of using Digital Fountain (DF) as
an alternative or replacement for GridFTP
transfers. This will involve transport level
experimental runs over the RealityGrid
network testbed using multicast and unicast
streams on UDP level according to DF
solution implementations6.
There will be comparative network analysis
between the transport utilising TCP and
GridFTP on one side and DF's FEC
(Forward
Error
Correcting)
coding
technology running reliably over UDP
level.
The final project aim is to investigate the
use of multicast on Access Grid7 sessions.
This will enable assessment of its potential
for providing highly robust networking to
increase Access Grid performance and
improve users' experience of this
4
See LGMP web site
http://lgmp.planethofmann.com/ for further
information and relevant technical publications.
5
See http://wwwitg.lbl.gov/InterGroup/index.html for further
information and relevant technical publications.
6
See http://www.digitalfountain.com/ for
further information and some relevant technical
publications.
7
See http://www.accessgrid.org/ for further
information.
technology. The results of this assessment
will be documented in an overview report.
We envisage that multicast will become the
enabling technology for one-to-many and
many-to-many transport and distribution of
high-volume data on the Grid. The project
will impact e-Science by directly testing the
maturity of current multicast technology in
real-life Grid transport scenarios and by
testing the capabilities of e-Science’s
network infrastructure for supporting highvolume multicast traffic.
The outputs of the project should help push
future development of multicast technology
in Grid computing as well as the wider
context of group communication for
distributed systems.
Notes
MUST is due to begin in September 2003.
The project involves collaboration between
Complexity Research Group (CRG) of BT
Exact (www.BTexact.com) and the
RealityGrid consortium of EPSRC’s eScience initiative (www.RealityGrid.org).
For further information, please contact
Michael Daw (michael.daw@man.ac.uk) or
Maziar Nekovee (maziar.nekovee@bt.com).