A critique on the research paper:
LogP: Towards a Realistic Model of Parallel Computation
[Authors : David Culler, Richard Karp y , David Patterson, Abhijit Sahay, Klaus Erik
Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken]
by
Rajvikram Singh
1
Paper : LogP: Towards a Realistic Model of Parallel Computation
[Authors : David Culler, Richard Karp y , David Patterson, Abhijit Sahay, Klaus Erik
Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken]
Main Contribution(s):
The authors of the paper have stressed more than once that the present popular
models used for depicting parallel architectures do not accurately describe the machines.
The present models, which are derived from the ideal, PRAM models are described by
the authors as too “simple” One of the main drives for presenting a new model was given
as the trend that the computing industry has been taking since the past few years.
Statistical and empirical data has been given in the paper to show that the latest trend has
been towards more “multi computer” type of machines. These are generally massively
parallel machines constituted of many nodes, each containing a powerful processor and
substantial memory and cache, so as to be regarded as independent computer in itself.
In such ‘distributed architecture’ machines, the following features can be
considered as the salient ones:  Performance of the interconnect network – The bandwidth, latency, jitter,
congestion aspects of the interconnect networks typically become important in
deciding the overall performance of the machine.
 Unsynchronized nodes – Since each of the nodes is more or less an independent
computer in itself, the various nodes in a MPP machine can no longer considered
to be synchronous.
 Shared memory absent – Since each node now carries its own memory modules,
the architecture is more suited for a message-passing system than a shared
memory one.
All the three points mentioned above are starkly opposite to the features of the
popular PRAM model, which describes a synchronized, unbounded shared memory,
MIMD model. Thus acknowledging the latest trend in the HPC industry, the authors have
suggested the LogP model, which takes into consideration, the parameters, namely:
latency, communication overheads, communication bandwidth and the number of
processors to describe a more accurate model. The model also takes care of some other
important aspects such as affect of load on network performance and affect of routing for
describing a more pragmatic model.
Evaluation of the contributions to advancing science and the state-of-the-art of the topic:
The LogP model suggested by the authors does seem to alleviate quite a few of
loopholes of the overly simplified PRAM model and in the present scenario where indeed
focus on MPP machines has shifted towards a more distributed architecture, the LogP
model definitely provides a much more accurate depiction of practical MPP machines.
Thus though the model was suggested in’93, it can still be considered as state-of-the-art
2
and of relevance even now and with HPC systems like clusters becoming popular day by
day, the model might well be of use even in the future.
Deficiencies:
Though the authors have presented a model, which they feel is accurate for
depicting present day MPP architectures, they have used parameters, which concentrate
primarily on the performance of the interconnection network. They seem to neglect the
other factors such as network topology and message-passing v/s shared memory
topology, which also affect performance.
The paper also seems to sort of re-instate the obvious concerns for designing a
parallel computing machine. And though the authors have published results of their
experiments showing the predicted performance and the actual performance for some
algorithms but they have not compared the results to those predicted by the existing
models.
Directions for future improvements:
The model seems to give too much importance to the underlying interconnect
network. Maybe some of the parameters in the model should be dropped and replaced
with others pertaining to the nodes. Especially with the recent development of
inexpensive and high speed optical interconnects, which provide excellent latency, near
zero contention environments (WDM) and bandwidths higher than the processors, it
might be a good idea to drop some of the parameters like L and o or g in lieu of future
trends. Also the authors agree that in most of the cases all the parameters are not required.
Questions –Regarding the paper:
 Can one of the parameters out of ‘o’ or ‘g’ be dropped? Or can one be
expressed as a function of the other? They seem to depict the same thing.
 The authors claim that the model takes care of network saturation due to
loading of the network. How do L, o , g or P reflect the loading of the network,
when there is no parameter to define the network topology ?
 Why haven’t the results of performance predicted by other known models
compared to the ones predicted by the LogP model? (eg. For the FFT
algorithm)
3
SUPPLEMENTARY PAPERS
Paper : Algorithms in CGM, BSP and BSP Model – A Survey
[ Author: Silvia Gotz]
Summary:
The paper has been written with the intention of evaluating the pluses and
minuses of some of the most popular parallel models that have received considerable
attention in the past few years of its publication. The paper talks about the following
models : The BSP (Bulk synchronous Parallel Model)
 The BSP* - Which is an extension of the basic BSP model but takes into account
an additional parameter B (minimum size of transmitted message that fully
utilizes n/w bandwidth) - which defines the interconnect more accurately and
leads to better bandwidth utilization.
 The CGM (Coarse Grained Multicomputer ) model
After surveying these popular models and evaluating them for different
algorithms, the author concludes that the BSP model is the one that comes close to being
the one model that can be standardized. But at the same time there exist whole classes of
problems which can not use the BSP model.
Note : Since the paper was just a survey of existing models, it by itself does not contribute
anything, nor does it contain any deficiencies which can be attributed to the survey.
4
Paper: BSP - The Bulk Synchronous Parallel model
[University of Oxford Parallel Applications Center]
Main contribution(s):
This paper is not a research paper and provides a brief but insightful view into the
BSP model developed primarily at the University of Oxford. The paper defines the
characteristics of a BSP computer, the parameters that define a BSP model and the
computation and programming model for a typical BSP computer.
5
Paper : Models of Parallel Computation: A survey and Synthesis
[ Authors: B.M Maggs, L.R Matheson and R.E Tarjan ]
Main Contribution(s):
The paper tries to form present the past trends in the field of parallel computing
and tries to explain why there has been no standard model which can define all the
existing and possible future parallel machines.
The paper evaluates the research undertaken by the computer scientists in coming
up with a unified model and the levels at which models have been introduced (eg.
Memory, network etc)
The authors stress on the point that the models should not only be ‘descriptive’ i.e they should not only reflect the existing parallel architectures, but they should also be
prescriptive – i.e they should be able to provide guidelines for the machines of the future.
And while the authors agree that the models should not contain a lot of parameters, they
have described a few parameters, which they feel symbolize a parallel architecture. They
recommend that a subset of these parameters be used for modeling.
6
Paper : Models for Parallel Computation
[ Author: Susanne E. Hambrusch]
Summary:
The paper gives a brief discussion of the most popular models, which have
received the most attention over the past years and describes the major advantages /
disadvantages of one over the other. The paper acknowledges that in order for a
successful standard model to evolve, the model should:  Accurately reflect the constraints of the existing classes of machines
 Should also broadly apply to existing as well as future machines
 Allow accurate measure and prediction of performance.
The paper was not intended as a research paper and thus does not make any major
contributions or has any deficiencies.
7