CAREER: Research in Elastically Redundant Computing as a Catalyst for
Seamless Hardware/Software Education
Project Description
a. RESULTS FROM PRIOR NSF SUPPORT
NSF/ONR-0224743
PI: Dr. Agustín Irizarry-Rivera. Amount: $399,978 Effective Dates: Oct. 1, 2002-Sept. 30, 2004
Title: Intelligent Power Routers for Distributed Coordination in Electric Energy Processing Networks
The PI is involved in this NSF/ONR project as a CoPI. The project goal is to develop a model for the
next generation of power distribution networks using a distributed concept based on scalable
coordination by an Intelligent Power Router (IPR). The project hopes to demonstrate that by distributing
network intelligence and control functions using the IPR improved survivability, security, reliability, and
reconfigurability can be achieved. The IPR project gathers a multidisciplinary team from power
engineering, systems, control, distributed computing, and computer networks. In the IPR system control
can be detached from central control sites, and delegated to intelligent power routers that are strategically
distributed over the entire Electric Energy Processing Network. The first model of a programmable IPR
is currently being developed by the PI in collaboration with a multidisciplinary team professors and
gradate students using the University of South Carolina Virtual Test Bed Environment (VTB). The
prototype is expected to be available for experimentation by October of 2003.
b. CAREER DEVELOPMENT PLAN
b.1. OVERVIEW
The PI proposes a career plan comprised of a strong research component aimed at the design and
development of elastically redundant computing systems coupled with an aggressive educational
component conducing to a qualitative improvement of the Computer Science and Engineering curricula
at the University of Puerto Rico Mayaguez.
Elastically Redundant Computing Systems (ERCS) incorporate algorithms that sustain a desired level of
data availability even in the presence of online changes to the topology of a distributed storage system.
Central to this work is the thesis that a dynamic balance among data replication and data migration must
be carefully maintained in order to sustain a desired level of availability while maximizing storage
utilization.1 The research pretends to empirically demonstrate the inherent trade off between utilization
and availability (Figure 1) in distributed storage systems as well as to introduce a simple modeling
framework of a distributed storage cluster (DSC). A DSC is comprised of one or more storage nodes
collaborating to provide an image of a single storage system. The need for ERIS arises because DSC’s
often suffer changes in the number of storage nodes to accommodate dynamic changes in the demands
for storage bandwidth and/or capacity. Adding storage nodes to a DSC may increase the expectation of
the number of failures thus potentially decreasing data availability. We are particularly interested in
DSC’s to (from) which nodes can be added (removed) at any time during the operation of a system
without interruption. Preliminary results from simulations of different elastic redundancy algorithms
1
Data replication is one of the most common mechanisms for introducing redundancy to a system. In the remainder
of this proposal the terms redundancy and replication will be used interchangeably.
using this DSC model suggest that ERCS are desirable even in small scale DSC’s (order of ten storage
nodes). This early exploratory research evidences a practical need for ERCS.
Availability
Desirable
F(#nodes) = Near Constant
Minimum
Utilization (U)
Maximum
Utilization (U)
No Distribution
Uniform
distribution
Maximum
Availability (A)
Minimum
Availability (A)
Threshold
A(t)
(Minimal tolerable availability)
A
U
systems with
static redundancy
?
# Nodes
8
#Nodes
15
Figure 1. Relationship between availability (A(t), utilization U and DSC size
ERCS arises in several different contexts and at different levels of abstraction. Examples of relevant
distributed storage systems include: distributed files systems, RAID systems and distributed database
systems, among others. Some of these systems typically belong to the realm of Computer Architecture
while others are the subject of study in courses in Operating Systems. As a result, the proposed research
component provides a strong foundation for the adaptation of existing, and the development of new,
learning tools and laboratory experiences to help students bridge the often obscured abstraction barriers
that lie between hardware and software. Although much progress has been achieved, the PI believes
than, from his experiences teaching hardware and software courses at different institutions, further work
in this direction is still much need.
Since his affiliation with the UPRM in 2000, the PI has been actively leading important processes at the
UPRM conducting to significant outcomes expected to materialize within the next two to three years
including: (1) the creation of a brand new undergraduate program in Computer Science and Engineering
offered by the UPRM School of Engineering, a program unique in its class in Puerto Rico, (2) the
reorganization of the Department of Electrical and Computer Engineering (ECE) into two sister
departments; one in ECE and a new Computer and Information Sciences and Engineering Department
(CISE), (3) a major reorganization of the recently created doctoral program in CISE. The PI seeks NSF
support from this CAREER award to provide a critical final boost to these ongoing curricular
improvement processes.
b.2. OBJECTIVES AND SIGNIFICANCE OF PROPOSED ACTIVITIES
Objectives
The main research objectives of the proposed career plan here are:

Develop a solid theoretical framework for studying ERCS. Develop a modeling framework to
empirically demonstrate that ERCS are of real practical importance. Uncover the functional
relationships between availability and DSC size for different ERCS approaches. Build the first
ERCS prototype system.

Catapult the PI’s research agenda into the area of systems research with a particular emphasis on
gaining a deep understanding as well as designing and developing elastically redundant
computing systems.

Stimulate the involvement of talented Puerto Rican engineering students in competitive state-ofthe art research projects leading to the development of new distributed system technologies of
both scientific and commercial value.

Contribute to the continued growth of thriving computing research groups at the University of
Puerto Rico Mayaguez Campus (UPRM).

Create new and strengthen existing collaborative research relationships between the UPRM and
world-leading research institutions in CISE.

Demonstrate that the UPRM can become a world class scientific research institution in CISE
areas.
The main educational objectives are:

Provide critical support to the final stages of development and eventual ABET accreditation of
the first Computer Science and Engineering undergraduate program ever implemented in a
Puerto Rican School of Engineering.

Provide critical support for the ongoing administrative and academic reorganization of both
undergraduate and graduate CISE academic programs at the UPRM.

Enhance the undergraduate Computing curricula at UPRM to improve the integration between
Computer Architecture (Hardware) and Operating Systems (Software) courses by adapting and
developing learning tools and laboratory experiences to help students deeply understand how the
OS implements important abstractions on top of the Architecture.
Intellectual Merits
The PI proposes an ambitious yet viable career plan with strong research and education components.
On the research side, the intellectual merit mainly resides on the importance and relevance of the
proposed research problem in today’s world. The proliferation of Internet access and consequently of
Internet services has renovated the interest of the computing research community on the security and
reliability of computing systems. Fault tolerance computing has evolved from a feature affordable only
by large institutions to a commodity item. However, multi-scalar high performance computer
architectures have not suffered the same fate and still remain an item affordable by the few. In order to
provide fast response times and high transaction throughput to a potentially large customer base,
companies have resorted to loosely coupled architectures or clusters build form off-the-shelf
components. In order to provide continuous 24/7 service clusters must tolerate dynamic additions and
removals of computing nodes in response to component failures or resource reallocations. The PI
proposes re-visiting the idea of replication, vastly explored since the 70’s, but this time with a different
set of design constraints.
The education component leverages on the research component in order to improve the way in which
students learn about the Hardware/Software interface. ERCS provides a fertile ground for the
development of laboratory experiences along these lines. The PI proposes adapting and extending
currently available computer architecture learning tools to allow the development of laboratory modules
illustrating the implementation of OS abstractions at the bare instruction set architecture (ISA) level. For
instance, the popular SPIM simulator (MIPS) supports a comprehensive set of the MIPS instructions set
and basic support for exceptions. However, SPIM does not quite implement MIPS support for system
calls, virtual memory or I/O devices. Such features are essential to provide students with a
comprehensive view of the inner workings of an OS.
Broader Impacts
Both research and education components of the proposed career plan have an enormous potential for
achieving broader impacts. Research in ERCS has the potential to yield new system designs and
algorithms for providing sustained availability at an affordable cost thus reaching a broad customer base.
With this goal in mind the PI’s effort will be focused on exploring ERCS for the types of cluster systems
commonly used to support Internet services (e-services). NSF/CAREER funds are being requested to
equip the PI’s research group with a small cluster of this kind as an experimental test bed.
The education component has the potential to significantly deepen student understanding of OS
abstraction implementation not only at the UPRM, but world-wide through the distribution, free-ofcharge, of all educational material developed under this grant on existing web accessible portals
dedicated to the dissemination of learning materials and tools in Computer Science. Moreover, the PI’s
leadership in the reorganization and curricular implementations of Computing Programs at the UPRM
will undoubtedly benefit talented Puerto Rican engineering students wishing to pursue CISE careers, but
currently deterred from doing so due to a limitation of resources required to attend institutions in
mainland US or in other countries. The UPRM undergraduate student population is virtually 100%
Hispanic and fifty percent of all Computer Engineering are women. So this grant will have a decisive
positive impact on underrepresented minorities and women, a current national priority, not only during
the award period, but for generations to come.
b.3. RELATION OF CAREER PLAN TO CURRENT STATE OF KNOWLEDGE
In general, data replication has been a subject of active research for over 30 years. The literature on the
subject is extensive, but due to space limitations only an extract of the most relevant previous work is
presented here. Approaches to data replication can be classified into four different groups: Consensus
Based, Data Trading, Caching, and RAID.
Consensus-based systems, also known as voting replication system, disseminate replicated data to
various nodes in a network with a certain degree of correctness. In order to identify the correctness of
the data that arrive to a given node, the method requires comparing the data that arrive to that of the other
nodes. Since the correctness of the data at each receiving node in the network is uncertain, in order to
decide whether the data is correct, the method uses consensus between the nodes. Each node casts a vote
and the majority determines which data is correct. If a node does not belongs to the majority, is either
disabled or the data is resent to it. The availability of the system depends on the correctness of the data,
which depends in turn, on the count of nodes involved in the voting process. The count of nodes has to
be sufficiently large to overcome the “Byzantine failure assumption”, this is, a majority of incorrect
votes. Consensus-based replication is more concerned with the problem of coordinating object updates
so that the latest update is always available, ERCS is more concerned with determining how many copies
should be kept in order to sustain a specific availability level In fact, ERCS and consensus based systems
complement each other. Examples of consensus-based systems are [20] and [5].
The next replication method is Data Trading. This method is characterized, as the name suggests, by
the exchange of dating among nodes. For instance, if a node A has 1GB of data and 2GB of spare
allocation space, and a node B, with a similar description, interchange part or all of their data, the result
is that data from node A will be replicated in node B, and vice versa. The actual interchange is usually
done thru a communication network. This interchange is driven by the help of a set of particular policies.
These policies determine the conditions under which the nodes form a relationship. The relationship can
be a free exchange society or one based economic factors. The rules involved in the relationship have to
be determined, as well. For example, the amount of resources used in an interchange can be determined
by a load balancing scheme, or be restricted by time constraints, or by other type of policy. Although the
level of redundancy in the system may vary dynamically, the system does not have enough control of
how much it redundancy varies across the system. Data Trading was never designed with the same
constraints as ERCS. For instance, the system may reach a state in which some objects have no
replication and objects nodes are excessively replicated. Examples of data trading systems include [13]
and [33].
In Caching, storage is partitioned so that the primary copies of objects remain separate from the
replicas. Typically, the replicas are located closer to the data consumer and in smaller memories for
faster access. This type of storage organization is primarily designed for performance instead of
reliability. When the cached data is modified the primary copies are modified according to a policy
specifying how and when the change propagation is going to be performed. The Caching method is
adequate for data that does not change too often. Otherwise, it requires a mechanism for getting
consistent updates and conflict resolutions among the participating nodes. This is usually done by a
Consensus method. Two examples of caching systems are [1] and [26].
Redundant arrays of inexpensive disks (RAID) are similar to ERCS in their use of replication to
overcome the loss in reliability due to the use of multiple storage nodes with independent failure modes.
Different RAID levels were originally proposed based on four parameters: the percentage of replication,
the type of redundancy used (parity or full), the size of the data unit (striped or not striped), and the
location or arrangement (distributed or not) of the units in the group of nodes (discs). What distinguishes
RAID form ERCS is that these parameters are normally fixed at the time the system is configured.
Examples of RAID systems are: [28] and [12].
A distributed storage system is said to be elastically redundant if it incorporates algorithms that
automate the choice between migration and replication in order to maintain a desired level of availability
in the presence to dynamic changes to the number of nodes. The PI is not aware of any system that
incorporates such algorithms. There have been distributed file systems that provide a level of replication
that is fixed at system configuration time [22]. Others provide facilities to create objects with different
levels of replication [19]. Deceit [31] provides operations to dynamically alter the level of replication of
a file. However, none of these systems automatically adjusts the level of replication in response to a
change in the number of nodes.
b.4. DETAILED PLAN OF WORK
Research Task #1: ERCS Theoretical Framework
Ideally, when a DSC node fails, the remaining nodes should transparently take over the responsibilities
of the failed node. The system as a whole should not fail, but rather should recover and continue
operating at the maximum capacity possible. This requires that the data stored by the failed node be
made accessible to the node or nodes taking over its duties. The process of making data from a failed
node available to other nodes is called data failover. When the failed node recovers it should get its
(possibly updated) data from the nodes that replaced it. This process is called data failback. A process
similar to failback is also applied when nodes are added to a DSC to increase system capacity. In this
case data can be reorganized and the new node is assigned some portion of the data. In order to take
advantage of the increase in storage capacity the cluster software has two basic alternatives. It can
replicate, that is, store additional copies of objects in the new node (Figure 2a) or it can migrate (Figure
2b) some copies of objects to the new node.
2 nodes
X
y
2 nodes
X
y
X
y
2
nodes
added
X
y
X
y
X
y
2
nodes
added
X
y
X
y
X
x
4 nodes
y
y
4 nodes
(a)
(b)
Figure 2. Replication vs. Migration: (a) Replication increases the number of copies, (b) Migration
redistributes existing copies.
If objects are very seldom updated and most often read, replication is typically an acceptable
alternative. This is due to the fact that many replication algorithms, including voting and primary copy
are capable of implementing much faster writes than reads [19]. When the frequency of writes is large
enough that the increase in the cost on write operations is not acceptable, migration should be considered.
But why not just use migration? This section presents informal arguments as to why relying exclusively
on migration will decrease availability. Formalizing these arguments is one important goal of the
proposed research plan.
Elastic replication requires solving two problems: (1) deciding when to replicate and when to migrate,
and (2) carrying out the operations consistently in the presence of failures. The proposed research will
focus on (1). The existence of systems that allow dynamic changes to the level of replication of objects
demonstrates that (2) is achievable with known algorithms. The PI proposes to structure this research
around the following hypotheses:
H1: If a system relies exclusively on migration as a way to use the storage capacity contributed by
a new node, its availability will decrease.
H2:
A system must alternate migration with replication if it wishes to maintaining a desired level
of availability at minimal cost
H3:
Replication should be applied much less often than migration
H2 is not quite independent form the others. It follows from H1 and the fact that replication increases
operation costs. A simple informal argument that H1 and H3 are true can be formulated under the
following simplifying assumptions about a DSC with n nodes:
A1: DSC keeps exactly k of copies of each object
A2: Initially n = k
A3: n increases k nodes at a time
A4: No network partitions occur
Under these assumptions a system of n nodes keeping k copies of each object can always be
represented in matrix form as shown in Figure 3. Each box represents a storage node and contains one
or more objects represented by the letters inside the boxes. It will be assumed that objects can be
partitioned by a object-node equivalence relation that assigns objects whose copies are stored in the same
set of nodes to the same equivalence class. Two objects are related if their copies are stored in exactly
the same set of nodes. The number q represents the number of equivalence classes in such partition. The
rows in Figure 3 represent the set of nodes holding objects of one equivalence class. In the figure, X
and W belong to a different classes, but X and Y belong to the same class.
Example: k = 3, n = 6, q = 2
X
Y
X
Y
X
Y
W
Z
W
Z
W
Z
q
k
32
Figure 3. A DSC n=6 nodes and k=3 copies of each of the objects
The argument is twofold. First, if only migration is used, the maximal availability attained by a system
of q classes after adding k nodes is obtained by re-partitioning the objects into q+1 groups so that the set
of nodes where each group has copies is disjoint with that of any other class. The reason why this is true
is that in such a system each single node failure can only touch one group. The probability that k nodes
of the same group will fail, with the consequent total failure, is smaller that when the number of groups
touched by each failure increases. But this must be the case when the sets of nodes of different groups
overlap.
For the second part of the argument notice that a total failure occurs when all the nodes in one class
fail. The probability that this happens is q * p ^ k where p is the probability that each node fails.
Therefore the availability A(k,q) of the system is one minus this probability that is,
A(k,q) = 1 - q * pk
This result informally demonstrates that a system relying exclusively on migration will suffer from
steady decreases in availability. A formal proof will be elaborated with support form this CAREER
award. It remains to be shown whether this decrease in availability is significant for real systems. Also,
the impact that relaxing the simplifying assumptions will have on this result must determined. These
questions are central to the theoretical foundations aspects of the proposed research plan.
Research Task #2: ERCS Modeling Framework
A DSC can be modeled as a Boolean O x D storage matrix SM[o,d]. The storage matrix relates a set of
objects O to the set pf storage nodes D where they reside at any given point in time. The value of
SM[o,d] is 1 when at least one copy of object o resides in node d. SM[o,d] equals zero otherwise.
Figure 4 shows three storage matrices. The matrix on the left represents a DSC storing a total of 4
objects, all of them stored in storage node 0. Storage nodes 1 to 3 are empty. The middle matrix shows
another DSC storing 4 objects evenly distributed amount its 4 storage nodes. This middle DSC may have
resulted from the migration of objects stored in the left DSC. The matrix on the right illustrates a DSC
storing 2 copies of each of 4 objects. The copies have been fairly evenly distributed among storage nodes
1 to 3, but storage node 0 holds one copy of each object.
0
1
2
3
0
1
0
0
0
1
1
0
0
0
2
1
0
0
0
3
1
0
0
0

Migration
0
1
2
3
0
1
0
0
0
1
0
1
0
0
2
0
0
1
0
3
0
0
0
1

Replication
0
1
2
3
0
1
1
0
0
1
1
0
1
0
2
1
0
0
1
3
1
0
1
0
Figure 4. DSC matrix, [rows, columns] = [objects, nodes]. Migration and replication example.
In this simplified model it is assumed that all objects stored in the DSC are equally necessary and
therefore that the system is available
is available. The system fails if
0 1 if
2 at
3 least
4 5 one
6 7copy
8 9of each
60% object
Redundancy
all copies of any single object0 become
1 0 1 unavailable.
0 0 0 0 0 In
0 general,
0
40%the
Up availability A(t ) of a system at some
1 0 1 0 1 0 0 0 0 0 0
point in time t is the probability
that the system is available at time t whether or not it has suffered any
2 1 0 1 0 0 0 0 0 0 0
failures in the past. In our case,
the DSC is available until we encounter a system failure.
3
0
1
0
1
0
0
0
0
0
0
4 1 0 the
1 likelihood
0 0 0 0 of
0 system
0 0
Replication can be used to reduce
failures. Note that if one node of the DSC
5 0in 1that
0 node
1 0 all
0 0become
0 0 0unreachable. However, had those objects been
becomes unreachable the objects
6 1 0 0 0 0 0 0 0 0 0
10% Down
replicated to other nodes of the
DSC they would still be reachable,
meaning that the system as a whole
7 1 0 0 0 0 0 0 0 0 0
would remain available. The way in which the objects are arranged in the DSC and the total amount of
8 1 0 0 0 0 0 0 0 0 0
replicas of each object are crucial
factors in determining the availability of the system.
9 1 0 0 0 0 0 0 0 0 0
0
1
2
3
4
5
6
7
8
9
60%
Redundancy
0
1
0
1
0
0
0
0
0
0
0
40%
Up
1
0
1
0
1
0
0
0
0
0
0
2
1
0
1
0
0
0
0
0
0
0
3
0
1
0
1
0
0
0
0
0
0
4
1
0
1
0
0
0
0
0
0
0
5
0
1
0
1
0
0
0
0
0
0
6
1
0
0
0
0
0
0
0
0
0
20%
Down
7
0
1
0
0
0
0
0
0
0
0
8
1
0
0
0
0
0
0
0
0
0
9
0
1
0
0
0
0
0
0
0
0
Figure 5. DSC Combining Replication with Migration
Availability is easy to achieve via replication if one is not concerned with storage utilization and
performance. From the theoretical discussion above it follows that some combination of migration and
replication must be used if one desires to achieve high utilization and performance and simultaneously
maintain availability above a threshold. Figure 5 illustrates a storage matrix for a DSC storing 10
objects across 10 storage nodes. Objects 0-5 are stored in nodes 0-3, and objects 6-9 are stored in nodes
0-1. Storage nodes 4-9 remain empty. Lets call that section of the matrix containing rows 0-5 the upper
section and the section containing rows 6-9 the lower section. The upper section is the section that holds
replicated objects, while the lower section holds migrated objects. This DSC uses 4 out of 10 storage
nodes (40% utilization) in the upper section and 2 out of 10 (20% utilization) in the lower section. Note
that the upper section has a highest redundancy factor than the lower section. The number of rows in the
upper region is the remainder of dividing the number of copies by the number of original objects. In
Figure 5, the upper region contains 6 rows, the remainder of dividing 6 (copies) by 10 (original objects).
Clearly, there are many ways in which objects can be arranged in a DSC. Let’s first consider two
extreme cases. In Figure 6 all objects are stored in a single node, an approach that can be named zerodistribution. In Figure 7, objects are evenly distributed in as many nodes as possible, an approached that
can be named full-distribution. Each of these two arrangements embodies a distinct, although trivial,
family of elastic redundancy algorithms. The zero-distribution algorithms simply ignore additions of
nodes to the DSC. All objects are stored in a single node at all times. Full distribution algorithms
migrate some of the objects to maintain a uniform distribution of objects across nodes.
0
1
2
3
4
5
6
7
8
9
0
1
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
2
1
0
0
0
0
0
0
0
0
0
3
1
0
0
0
0
0
0
0
0
0
4
1
0
0
0
0
0
0
0
0
0
5
1
0
0
0
0
0
0
0
0
0
6
1
0
0
0
0
0
0
0
0
0
7
1
0
0
0
0
0
0
0
0
0
8
1
0
0
0
0
0
0
0
0
0
9
1
0
0
0
0
0
0
0
0
0
Figure 6. No distribution extreme case.
These two extreme algorithms achieve widely opposing objectives. The zero-distribution algorithm
maximizes availability at the expense of reducing storage utilization. Since all objects are stored in a
single node, the system only fails if the specific node holding the objects fails. The full-distribution
algorithm maximizes utilization at the expense of reducing availability. In this case, any node failure will
cause a system failure. The proposed research plan will investigate alternatives between these extremes.
0
1
0
1
2
0
1
2
3
4
5
6
7
8
9
0
1
0
1
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
1
1
1
0
1
1
0
1
0
1
0
1
0
0
0
0
0
0
0
0
2
1
2
1
0
2
0
0
1
2
0
0
1
0
0
0
0
0
0
0
3
1
3
0
1
3
1
0
0
3
0
0
0
1
0
0
0
0
0
0
4
1
4
1
0
4
0
1
0
4
0
0
0
0
1
0
0
0
0
0
5
1
5
0
1
5
0
0
1
5
0
0
0
0
0
1
0
0
0
0
6
1
6
1
0
6
1
0
0
6
0
0
0
0
0
0
1
0
0
0
7
1
7
0
1
7
0
1
0
7
0
0
0
0
0
0
0
1
0
0
8
1
8
1
0
8
0
0
1
8
0
0
0
0
0
0
0
0
1
0
9
1
9
0
1
9
1
0
0
9
0
0
0
0
0
0
0
0
0
1
0
(A)

(B)

(C)

(D)
Figure 7. Full distribution ALGORITHM. (A) DSC initial state. (B) DSC after adding one node. (C) DSC
after adding next node. (D) Keep adding nodes until #nodes = #objects.
This simple modeling framework serves as a starting point in the exploration of alternative elastic
redundancy algorithms. DSC growth (shrinking) can be modeled by adding columns to the matrix and
reorganizing the objects. The Availability of each configuration will be approximated using a discrete
event simulation that calculates, for each node, whether the node has failed since the last time step. A
total system failure will occur whenever at least one of the objects becomes inaccessible. With the
support form a small seed sub-grant from the PRECISE project (NSF-EIA Grant 99-77071 PI Domingo
Rodriguez. See attached support letter) the PI and a masters level student developed a rudimentary
simulator and are beginning to obtain interesting results. For instance Figure 8 plots the MTTF vs DSC
size for three ERCS algorithms: full-distribution, zero-distribution (centric) and a hybrid distribution
algorithm currently under experimentation. The Figure confirms the rapid decrease in Availability when
objects are uniformly distributed across al nodes and the sustained level of availability possible in no
distribution of objects is done. The PI is convinced that in order to make significant progress provision
of additional funds for a sustained period of time will be required in order to involve UPRM PhD
students. Due to the expiration of the awarding period, PRECISE support to this project will end in May
2004.
MTTF (in week)
Comparison of Uniform Full Distribution,
Centric Distribution and Hybrid Distribution
1800
1600
1400
1200
1000
800
600
400
200
0
Hybrid
Centric
Uniform
Algorithm
0
20
40
60
80
100
#Nodes
Figure 8. MTTF vs. DSC size in nodes: alg1 is full distribution, alg2 is no distribution.
Research Task #3: ERCS Prototype Implementation
Beginning the third year of the award period (2006) the PI will lead a team of PhD students in the
design the first real elastically redundancy system. The ideal abstraction level at which the system will
be build is still a matter of research. Some possibilities include the development of a distributed file
system in software or the development a new RAID-type distributed storage system in a
Hardware/Software design.
Research Component Time Frame
2004
2005
2006
2007
2008
RT1: Development of Theoretical
Framework
RT2: Development of Modeling
Framework
RT3: Implementation of ERCS Prototype
Education Task #1: Development of undergraduate CISE Curricula at UPRM
To this day the only bachelor level academic degrees in Computing offered at the UPRM is a BS in
Computer Engineering (CpE) offered by the Department of Electrical and Computer Engineering and a
BS in Mathematics with a Computer Science option offered by the Department of Mathematics. The
Computer Engineering program is a five year program heavily loaded with courses in many areas of
Electrical Engineering. The Computer Science component consists of 16 (out of a total of 168 credits)
credits in programming, data structures, programming languages, compilers, operating systems and
software engineering. At the UPRM a course is typically 3 credits worth. The program is monolithic in
nature permitting a mere 9 credits in technical elective courses. In response to the most recent ABET
accreditation cycle (2002) the CpE program is currently undergoing a major curricular revision including
the creation of specialty tracks in computing systems, embedded systems and digital signal processing.
Three years ago, under the leadership of the PI, a new program in Computer Science and Engineering
was designed and a formal proposal was presented to the UPRM authorities. As of summer of 2003, the
proposal has already been approved by the academic senate and is on its way to the upper layers of the
UPRM administration. Much work remains to be accomplished before the proposal gets finally approved
and the program can start operating. Each approval requires an enormous amount of effort in order to
explain and justify the benefits of approving the program to the corresponding decision makers. A
CAREER award will further strengthen the credibility of the PI and will provide the necessary time
release to carry out this extenuating, but crucial process. The PI strongly believes that the new program
in CSE offered by the UPRM will open unique doors of opportunity to Puerto Rican students for
generations to come. Hi is firmly convinced that the UPRM School of Engineering is the only place
within the University of Puerto Rico system amenable to the development of a world class Computer
Science research and education initiative.
Education Task #2: Reorganization of Computing Programs at UPRM
The PI will continue his proactive involvement and leadership in the reorganization of the Department
of Electrical and Computer Engineering into two sister departments; one in ECE and one in CISE. A
formal proposal was submitted to the UPRM authorities and as of summer of 2003 it has already been
approved by the Department of Electrical and Computer Engineering unanimously. The proposal is
expected to obtain approval by the School of Engineering by August of 2003. Due to strong support
from the Central Administration, fast track treatment of the proposal is expected and a new CISE
Department is forecasted to be operational by August 2004.
The PI will also continue his involvement in the restructuring of the multidisciplinary doctoral program
in CISE both as member of the CISE PhD steering committee as well as member of the doctoral faculty.
The goal of this restructuring is to provide the program with and identity in terms the thriving research
thrusts that have demonstrated the greatest potential during the first three years since the establishment of
the program. With strong support from the Direction (See attached support letter) the PI will work
towards a major reformulation of the curriculum in terms of a Computer Science core and a set of
specialty tracks organized around the selected research thrusts. The current research thrusts as organized
in various research groups namely; Advanced Data Management, High Performance Computing,
Computational Signal Processing and Scientific Computing. Again, support from an NSF CAREER
award will provide the PI with release time required to carry out this important effort.
Education Task #3: Integrative Architecture/OS Learning Tools
Work on this task will be aimed at providing students with suitable tools to heighten their
understanding of hardware/software interfaces. The PI experience both taking courses and teaching
courses in computer organization in different competitive programs and at different levels has led him to
believe that undergraduate students often complete their course work without a deep understanding of the
mechanisms used to implement some of the important abstractions offered by modern operating systems.
Two examples of such abstractions are a kernel and a process. Students know that processes exist at the
OS level, but they seldom appreciate the “magic” involved in the implementation of this abstraction on
top of architectures providing different levels of support. The PI believes that laboratory experiences
designed to illustrate the type of support required/desirable from the architecture to implement these
abstractions are scarce.
The PI proposes the development and adaptation of existing learning tools geared toward illustrating,
via simulation, alternative mechanisms for implementing atomic context switches on top of the bare
instruction set architecture. More specifically, the deliverables of this effort will include:

A VHDL specification of a simple processor and its testing framework circuitry specially crafted
to be used as a learning tool by undergraduate students in combination with a VHDL simulation
environment.

An extension to the SPIM simulator (http://www.cs.wisc.edu/~larus/spim.html) to implement the
MIPS processor support for system calls, preemptive scheduling and virtual memory.

SPIM modules to implement a simple pre-emptive multithreaded operating system kernel in
MIPS assembly language, including a simple yet extensible scheduler and a timer interrupt
device.
Education Component Time Frame
2004
2005
2006
2007
2008
ET1: Development of BS CISE Program
ET2: Reorganization of CISE Programs
ET3: Development/Adaptation of
Integrative Architecture/OS learning tools
b.5. RELATION OF CAREER PLAN TO PI’S CAREER GOALS AND JOB RESPONSIBILITIES
The research component of this career plan is intended to catapult the PI’s research effort into the area
of reliable distributed systems.
Since his affiliation with the UPRM in 2000, the PI has been actively leading important processes
leading to the improvement of computing programs at the UPRM. Soon after his affiliation the PI was
elected by his peers as the coordinator of the software specialty area committee. The coordinator
foresees all the curricular issues pertaining to the courses traditionally taught in Computer Science
programs including Computer Programming, Data Structures, Programming Languages, Compilers,
Architecture and Operating Systems, among others. The committee is often consulted on issues
pertaining to Computing programs as was the main force behind the development a brand new program
in Computer Science and Engineering. To this day the PI remains as the coordinator, after being reelected once.
The PI also serves a a member of the interdepartmental committee coordinating the recently approved
multidisciplinary doctoral program in CISE (www.ece.uprm.edu/phd). In addition, the PI is a member of
the ECE Department’s Planning Committee responsible for the design and implementation of the
Department’s master developmental plans. This committee proposes any internal academic and
administrative reorganization of the personnel associated with the ECE Department. Finally , the is the
ECE Department representative at the School of Engineering academic affairs committee, the committee
that approves all new program proposals and curricular revisions in the School of Engineering, before
they go to the UPRM Academic Senate.
The PI’s educational and teaching background serves as a strong foundation for developing Computer
Science and Engineering curricula. During his academic preparation he attended four (4) institutions of
higher learning: the University of Puerto Rico at Río Piedras (transferred 1984), Cornell University (BA
Computer Science 1986), the University of California at Berkeley (MS Computer Science 1988) and the
Massachusetts Institute of Technology (PhD Computer Science 1999). Each of these institutions offers
a different perspective of what a top rated education in Computer Science should be. The PI has served
as teaching assistant in several CS courses at both the undergraduate and graduate levels including
undergraduate compilers, graduate programming language design and undergraduate programming and
data structures.
By December of 2003, the PI would have offered three core undergraduate courses for CSE Students:
Advanced Programming (CS1), Computer Architecture, Programming Languages, and Compiler Design.
As a graduate student the PI served as a teaching assistant in undergraduate introductory Computer
Science courses (e.g. 6.001 at MIT taught by Prof. Gerald J. Sussman), undergraduate compiler design
courses (CS 164 at UC Berkeley taught by Prof. Lawrence Rowe) and a graduate level programming
languages course (6.821 at MIT taught by Prof. David K. Gifford). This broad spectrum of previous
educational experiences in Computer Science has provided the PI with a deep understanding of both the
current state of Computer Science education as well as of the strengths and weaknesses of the UPR
Computing curriculum across multiple CS subdisciplines and intellectual levels.
NSF support in the form of a CAREER award coupled with strong support from people in key
leadership positions in the UPRM (See Support Letters) administration will place the PI in an ideal and
unique position to propel the thorough execution of the integrated research and education career plans
outlined in this proposal. The consecution of these outcomes will undoubtedly result in a qualitative
improvement of the academic and research offerings locally available to talented Puerto Rican students.
Many of these students do not have the means necessary to leave the Island to pursue a degree at an
institution in mainland United States.
b.6. SUMMARY OF PREVIOUS RESEARCH AND EDUCATION ACCOMPLISHMENTS
Research Accomplishments
The PI’s major previous research accomplishments can be summarized as follows:

HyPursuit Project (1997): Developed a novel version of MIT’s content routing system that
exploited document clustering techniques based on both the content and the link structure of web
pages in order to produce content labels or summaries if the information contained inside a
search engine. This project was one of the first attempts to analyze the hypertext structure of
web pages and put it to work in the improvement of information retrieval effectiveness.
Graduate students collaborating in this project included Ron Weiss and Mark Sheldon.
Most Relevant Publication: [39]; Website: www.psrg.lcs.mit.edu

Inforadar Project (1999): As part of his doctoral work at MIT’s Laboratory of Computer
Science, under the supervision of Dr. David K. Gifford, the PI developed a novel search engine
capable of automatically and dynamically generating categories based on the documents
retrieved in response to a query. Inforadar generates and populates the categories with absolutely
no human intervention. Inforadar introduced the concept of query lookahead, that is, the eager
evaluation of queries derived from a user query in order to improve the quality of categories
selected for organizing the results sets.
Most Relevant Publication: [35] ; Website: admg.ece.uprm.edu/~inforadar
More recently the PI has been and continues to be involved in the following research projects:

InforadarML Project (2000): This project consists of extending the Inforadar system with the
capability to process and serve multilingual document collections. The UPRM has proven to be
an ideal environment for this type of experimentation due to the inherently multilingual life style
in Puerto Rico. The project is currently focused on exploring alternative document ranking
algorithms as well as on developing performance metrics suitable to assess the effectiveness of
multi-lingual IR systems. The design of InforadarML is uniquely focused on users with equal
proficiency in multiple languages. Most previous research efforts in multilingual information
retrieval systems centered around the problem of query translation.
Most Relevant Publication: [36]; Website: admg.ece.uprm.edu/~inforadar

TU-PIS Project (2001): The TU-PIS pretends to develop a ubiquitous and interactive passenger
information system for the brand new 16 Km subway system currently being deployed in the San
Juan metropolitan area. The PIS integrates various state-of-the art information technologies
including: world wide web, virtual reality, real time computing and interactive movie authoring
environments. Completely developed in Macromedia Flash from scratch TU-PIS offers an aerial
vivid view of the real-time locations of the mass transit vehicles transiting in the 500 meters
periphery of each train station. Flash allows the PI and his students to assess the suitability of
interactive movie authoring environments for developing new types of applications.
Most Relevant Publication: [24]; Website: admg.ece.uprm.edu/~tu-pis

Terrascope Project (2002): The emergence of multimedia technologies and the possibility of
sharing and distributing geospatial data through broadband computer networks have exacerbated
the need for geographical information technologies. In the typical scenario, geospatial data is
periodically collected at geographically distributed ground stations which often span several
technical, administrative and even political domains. Solutions attempting to offer integrated
access to distributed data by relying on centralization of data repositories are often not feasible.
The inherently distributed nature of geospatial data should be of no concern to scientists,
students and other data consumers. The information system should enable and facilitate
collaboration among a set of distributed data providers who wish to provide an image of a single
data repository with minimal loss of individual autonomy over their data. Based on this principle
we are designing and developing the TerraScope Earth Science distributed peer-to-peer
database middleware system.
Most Relevant Publication: [11]; Website: admg.ece.uprm.edu/~terrascope
It was mentioned early in this proposal that one of the PI’s primary career plan objectives in obtaining
NSF support is to catapult the PI’s research agenda into the area of systems research with a particular
emphasis on the design and development of novel elastically redundant computing systems. The PI
firmly believes that elastically ERCS are fertile ground for research breakthroughs in what IBM has
recently begun to call Autonomic Computing. The following is an extract from IBM’s autonomic
computing website (http://www.research.ibm.com/autonomic):
“As access to information becomes omnipresent through PC's, hand-held and wireless devices, the
stability of current infrastructure, systems, and data is at an increasingly greater risk to suffer outages
and general disrepair. IBM believes that we are quickly reaching a threshold moment in the evolution
of the industry's views toward computing in general and the associated infrastructure, middleware,
and services that maintain them. The increasing system complexity is reaching a level beyond human
ability to manage and secure.”
Although the PI first conceived ERCS in 1998, well before the autonomic computing project was
established, the design goals for ERCS have a stinking resemblance to the design goals of autonomic
computing systems which in IBM terms are:

Flexibility. The system will be able to sift data via a platform- and device-agnostic approach.

Availability. The nature of the autonomic system is that it is always on.

Transparency. The system will perform its tasks and adapt to a user's needs without dragging the
user into the intricacies of its workings.
Elastic redundancy simply pretends to seek ways to automate the necessary adjustments to the level of
redundancy maintained by a system necessary to guarantee a minimal level of Availability above. The
PI, together with other UPRM computer scientists recently submitted a $100K equipment grant proposal
to IBM after conversations with IBM research staff form the IBM Austin Research demonstrated a string
interest and support from IBM in collaborating hand in hand with the UPRM in the ERCS project. The
equipment obtained from this grant will be used as a test bed for ERCS ideas, experiments and
prototypes.
Education Accomplishments
The major educational accomplishments achieved by the PI can be summarized as follows:

Has been coordinating the ECE Department specialty area committee in computing systems for
over three years during which critical events occurred that will shape the future of computing
programs at the UPRM.

Represented the computing faculty in dialogues with the EE faculty which resulted in the
reorganization of the ECE Department into two sister Departments: ECE and CISE. The
reorganization is unanimously supported by the entire ECE faculty, the Dean, the Chancellor and
the President of UPR. A formal proposal has been approved by the ECE department an wil be
under the consideration of the College of Engineering in August of 2003. Due to strong support
at all levels it is expected that a new department will be in place in August of 2004.

Lead in the formulation of the first revision of the UPRM Computer Engineering curriculum in
twenty years. The original curriculum only left room for 9 out of 168 credit units in computes
science electives and was heavily oriented toward electrical engineering. The new curriculum is
structured in various tiers allowing the students to specialize on one of three areas: computing
systems, hardware and embedded systems and digital signal processing.

Proposed and designed the first undergraduate engineering program in Computer Science in
Puerto Rico. The five year program offers a degree in CSE as well as a degree in Software
Engineering. This required a major transportation and much lobbing by the PI and other UPRM
professors.

Modernized the computer organization course for undergraduate Computer Engineering students.
For over twenty years the course had been taught with a heavy emphasis on two aspects: learning
assembly language programming and learning the specifics of the Intel family of
microprocessors. The PI, aware that this approach will not prepare students adequately for upper
level course in, for instance, compilers and operating systems, decided to teach the course
beginning in the spring of 2002. The PI implemented a holistic approach that focused on
fundamental pervasive concepts in Computing like: interpretation, abstraction, computing
models, computability, among others. The need to support higher level computing and data
abstractions at the OS and upper layers is a central theme throughout the course.