-^^^
PROPERTIES OF STORAGE HIERARCHY
SYSTEMS WITH MULTIPLE PAGE SIZES
AND REDUNDANT DATA
Chat-Yu Lam
Stuart E. Madnick
April 10, 1979
Revised
CISR //A2
Sloan WP No. 1047-79
This material is based upon v;ork
supported, in part, by the
National Science Foundation
under Grant No. MCS77-20829.
Center for Information Systems Research
Massachusetts
Institute of Technology
Sloan School of Management
77 Massachusetts Avenue
Cambridge, Massachusetts, 02139
PROPERTIES OF STORAGE HIERARCHY
SYSTEMS WITH MULTIPLE PAGE SIZES
AND REDUNDANT DATA
Chat-Yu Lam
Stuart E. Madnick
April 10, 1979
Revised
CISR //42
Sloan WP No. 1047-79
This material is based upon work
supported, In part, by Mie
National Science Foundation
under Grant No. MCS77-20829.
Center for Information Systems Research
Massachusetts Institute of Technology
Alfred P. Sloan School of Manaciement
50 Memorial Drive
Cambridge, Massachusetts, 02139
617 2!)3-10nO
PREFACE
The Center for Information Systems Research (CISR) is a research
center of the M.I.T. Sloan School of Management; it consists of
a
group of Management Information Systems specialists, including faculty
members, full -time research staf^, and student research assistants.
The Center's general research thrust is to devise better means of
designing, generating and maintaining application software, information
systems and decision support systems.
Within the context of the research effort sponsored by the National
Science Foundation under Grant No. MCS77-20829, CISR proposes to
investigate the architecture oT the INFOPLEX Data Base Computer which
is particularly designed for large-scale information management.
applies the theory of hierarchical decomposition in its design
INFOPLEX
and makes use of multiple microprocessors in its implementation to
obtain high performance, high reliability, and large storage capacity.
Research issues to be addressed include optimal decomposition of
information management functions into
implemented by
a
decomposition of
a
functional hierarchy to be
hierarchy of microprocessors, and optimal physical
a
data storage hierarchy to support the memory
requirements of the information management functions.
In Technical
Report No. 1, we discussed the INFOPLEX concept and its
research directions.
This I'eport focuses on the study of
a
generalized
data storage system for very large databases which can be used to support
the memory requirements of INFOPLEX.
multiple page sizes in
a
This data storage system makes use of
hierarchy of storage levels and maintains multiple
copies of the same information across the storage levels.
of such
a
data storage system are derived here.
q^nnn
\
Important properties
ABSTRACT
The need for high performance, highly reliable storage for very large
on-line databases, coupled with rapid advances in storage device
technology, has made the study of generalized storage hierarchies an
important area of research.
This paper analyzes properties of a data storage hierarchy system
To
specifically designed for handling very large on-line databases.
attain high performance and high reliability, the data storage hierarchy
makes use of multiple page sizes in different storage levels and
maintains multiple copies of the same information across the storage levels.
Such a storage hierarchy system is currently being designed as part of
Previous studies of storage
the INFOPLEX database computer project.
hierarchies have primarily focused on virtual memories for program
storage and hierarchies with a single page size across all storage levels
and/or a single copy of information in the hierarchy.
INFOPLEX design, extensions to the Least Recently Used (LRU)
The Read-Through
algorithm are used to manage the storage levels.
technique is used to initially load a referenced page, of the appropriate
size, into all storage levels above the one in which the page is found.
Since each storage level is viewed as an extension of the immediate
higher level, an overflow page from level 'i' is always placed in level 'i+1
It is shown that,
Important properties of these algorithms are derived.
depending upon the types of algorithms used and the relative sizes of the
storage levels, it is not always possible to guarantee that the contents
of a given storage level 'i' is always a superset of the contents of
The necessary and sufficient
its immediate higher storage level 'i-l'.
and proved.
identified
hold
are
property
to
for
this
conditions
of intermediate
size
the
increasing
possible
that
Furthemore, it is
to lower
of
references
number
storage levels may actually increase the
necessary
Conditions
storage levels, resulting in reduced performance.
proved.
and
identified
to avoid such an anomaly are also
In the
Key Words and Phrases
CR Categories
:
:
database computer, very large databases,
data storage hierarchy, storage management algorithms,
inclusion properties, modelling, performance and
reliability analysis.
4.3 Supervisory Systems,
4.33 Data Base
5.2 Metii theory
6.22 Special -Pui'pose Computers
6.34 Storage Units
2
32
TABLE OF CONTENTS
Introduction
I.
II.
11.1
Storage Management Algorithms
4
11.
Basic Model of Data Storage Hierarchy
9
1.
111.1
1
VI.
Formal Definitions of Storage Management Algorithms 9
13
Properties of Data Storage Hierarchy
III.
V.
4
Model of a Data Storage Hierarchy
1
IV.
1
11.
Summary of Properties
Derivation of Properties
Conclusions
.
.
.
,
16
19
37
Acknowledgment
38
References and Bibliography
39
1
Introduction
.
Two and three-level memory hierarchies have been used in practical
computer systems
5,
[
9,
13 ].
However, there is relatively little experience with general hierarchical
Rapid advances in storage technology coupled with the
storage systems.
need for high performance, highly reliable on-line databases makes the
idea of using
a
generalized storage hierarchy as the repository for
very large shared data bases very attractive.
One major area of theoretic study of storage hierarchy systems in
placement of information in
the past has been the optimal
hierarchy system.
(1)
a
storage
Three approaches to this problem have been used:
Static placement
[
1.
4,
22
]
-
this
approach determines the optimal placement strategy statically, at the
initiation of the system; (2) Dynamic placement
[
7,
16
-
]
this approach attempts to optimally place
information in the hierarchy, taking into account the dynamically
changing nature of access to information; (3) Information structuring
r
11
14
"1
-
approach manipulates
this
structure of information so that information items that are
the internal
frequently used together are placed adjacent to each other.
Another major area of theoretic study of storage hierarchy systems
has been the study of storage management algorithms
[
2,
3,
8,
10,
17, 21
].
Here
the study of storage hierarchy and the study
of virtual memory systems for program storage have overlapped
considerably.
This is largely due to the fact that most of the studies
-2-
of storage hierarchies in the past have been aimed at providing a
virtual memory for program storage.
These studies usually do not
consider the effects of mjltiple page sizes across storage levels,
nor the problem of providing redundant data across storage levels.
These considerations are of great importance for
a
storage hierarchy
designed specifically for very large data bases.
Madnick
of
a
[
15,
18,
19
proposed the design
]
generalized storage hierarchy for large data bases that makes use
of multiple data redundancy against failure and multiple page sizes
in different storage levels for high performance.
Such
a
storage
hierarchy system is to be used in the INFOPLEX database computer
[
12, 20 ].
Conceptually, the INFOPLEX database computer consists of
functional hierarchy and
a
physical
(storage) hierarchy (See figure
functional
hierarchy
virtual storage
Interface
storage
hierarchy
FIGURE
1
INFOPLfl Ddta
Ortenlzatlon
Basi"
a
Computer Conceptual
1)
-3-
The functional hierarchy implements all the information management
functions of
a
database manager, such as query language interpretation,
security verification, and data path accessing, etc.
In
INFOPLEX,
the functional hierarchy is implemented using multiple microprocessors.
Both pipeline and parallel processing are exploited to realize high
performance and high reliability.
To support the storage requirements
of the functional hierarchy, INFOPLEX makes use of a generalized
data storage hierarchy system.
In this paper, we extend this work by developing a model
of the
data storage hierarchy, proposing extensions to the Least Recently Used
(LRU) algorithm for managing the storage hierarchy, and deriving
important properties of the data storage hierarchy.
-4-
2
Model Of A Data Storage Hierarchy
.
A Data Storage Hierarchy consists of
m\
.
M
.,
.
.
.
The page size of M^
pages each of size Q.
m.
.
M^,
.
is
is always an
The unit of information transfer
h.
.,
Q^-
.
page, of size Q.
Figure
.
2
levels of storage devices,
h
Q^.
and the size of M^ is
integral multiple of Q^_-|. for
and M
between M
i
=
is a
illustrates this model of the Data
Storage Hierarchy.
All
references are directed to M
The storage management
.
algorithms automatically transfer information among storage levels.
a
a
As
result, the Data Storage Hierarchy appears to the reference source as
M^
storage device with the size of M
.
As a result of the storage management algorithms
(to be discussed next),
multiple copies of the same information may exist in different storage
levels.
Storage Management Algorithms
2.1.
,
focus our attentions on the basic algorithms to support
We shall
the read- through
[
18
]
operation.
Algorithms to support other
operations can be derived from these basic algorithms.
In a
read- through
,
the highest storage level
that contains the
addressed information broadcaststhe information to all upper storage
levels, each of which simultaneously extracts the page (of the appropriate
size) that contains the information from the broadcast.
information is
reduces to
Figure
3
a
If the addressed
found in the Fiighest storage level, the read - through
simple reference to the addressed information in that level.
illustrates the read- t hrough operation.
2,3
-5-
REFERENJES
I
J...
I
Common
<****
..l^"*'
'1"
between
y//////71
/sue O,//
data
path
'^
L.
transfer
.
,
m'
and M^
A
--I
JUnlt of data transfer
j
between
,
-,
H^ and
Y//P Agc
of size Q
Reference to page P
READ-
1
P,
Page containing P
THROUGH
f^^/XTcverflow from
h'
ya
V\^Y^
t
|
Kodel of « data storage hierarchy system
Figure 2
Page
.,
M-"
>>.
•„
-..,
•
X
,
Page containing
'
,
iy7y77!/,"!overflow from M
''y////A
p'
[,1
v^r.^>y///AL
i
'/^(Overflow from H
'^
RFAO-TIIROUGri
OVER'LOW-HAflDLING (OH)
(RT)
^t
1''^
't+1
't-
t
REPfRINCt CVr.Lt
figure
3
~~~
IlluUralloii of the
operation
Rfftt)-
THROUGH
X
.
-6-
Note that in order to load
a
new page into
a
storage level an existing
page may have to be displaced from that storage levelphenoinonon as overflow
.
We refer to this
Hence, the basic reference cycle consists of two
sub-cycles, the read-through cycle (RT), and the overflow handling cycle
(OH), with
RTpreceeding OH.
For example. Figure
handle
illustrates the basic reference cycle to
3
reference to the page
a
Y
the highest storage level
taining
to all
P
(M
)
P,,^
During the Read-Through (RT) subcycle,
.
ya
that contains
1
P
broadcasts the page con-
upper storage levels, each of which extracts the page of
appropriate size that contains
P
As result of the
from the broadcast.
Read-Through, there may be overflow from the storage levels.
These are
handled in the Overflow-Handling (OH) subcycle.
It is
necessary to consider overflow handling because it is desirable
to have information overflowed from a storage level
to be in the immediate
lower storage level, which can then be viewed as an extension to the higher
storage level
One strategy of handling overflow to meet this objective is to treat
overflows from M
as references to M
.
We refer to algorithms that in-
corporate this strategy as having dynamic - over flow- placement (DOP).
Another possible overflow handling strategy is to treat an overflow
from M
in M
i
+1
as a reference to M
.If
only when the overflow information is not already
the overflow information is already in M
handling is necessary.
i
+1
,
no overflow
We refer to algorith-ns that incorporate this
strategy as having static -overflow - placement (SOP).
-7Lct us consider the algorithms at each storage level for seli-cting
the page to be overflowed.
Since the Least Recently Used (LRU) .ilgorithm [^> ^^3
consider natural
serves as the basis for most current .algorithms, we shall
extensions to LRU for managing the storage levels in the Data Storage
Hierarchy system.
Consider the following two strategies for handling the Read-Through
Cycle.
First,
let every storage level above and including the level
containing the addressed information be updated according to the LRU
storage levels lower than the addressed information
Thu., all
strategy.
do not know ab)ut the reference.
This class of algorithms is called
This is illustrated in Figure
LOCAL-LRU algori thm.
4.
The other class of algorithms that we shall consider is called
GLOBAL -LRU algorithm.
In
this case, all
storage levels are updated
according to the LRU strategy whether or not that level actually participates
This is illustrated in Figure
in the read-through.
Although
tlie
5.
read-through operation leaves supersets of the page P^^ in all
depends upon the replacement
levels, the future handling of each of these pages
We would like to
'algorithms used and the effects of the overflow handling.
guarantee that the contents of each storage level,
immediately higher level, \^''\
m\
is always a
This property is called
Conditions to guarantee MLI will be derived in
a
superset of its
MullH^ve^ InclusionjML,:!.
later section.
where handling overflows generates
It is not difficult to demonstrate situations
more references.
references which produce overflows, which generate yet
another important question to resolve
is
to determine the conditions under which an
overflow from M^ is always found to already exist in
storage levels lower than m'^^
property
will
is
is
generated as
a
a
M^"^
,
i.e., no reference to
result of the overflow.
called MvmiJ^eJ_Ove^rllow_IiKjiJsioi^^
be derived in
Hence
This
Conditions to guarantee MLOI
later section.
light of four basic algorithm
We shall consider these important properties in
Formal
overflow.
alternatives based on local or global LRU and static or dynamic
the basic model of the
definitions for these algorithms will be provided af'.er
"
Data Storage Hiei.irthy system is introduced.
RIFERFNCE to
P
-c:
i
L
Read-
!
through
H
Is the highest
level where P Is found
-{'
A)l these
,xtl
::j
levels ers
not affected
€Z
Figure
4
LOCAL-LRU
REFERENCE to
ReadThrough
l^
These level s\
are also
updated as
If rcfererxe
to P were
made to then
P
.
-9-
2.2
S torage Hi
Basic Model of Data
erarchy
*
For the purposes of this paper, the basic model
6
Figure
illustrated in
sufficient to model the Data Storage Hierarchy.
is
As far as
the Read-Through and Overflow-Handling operations are concerned, this basic
M*"
can be viewed as
a
pages each of size Q^.
M^
pages each of size nQ. where
It has m.
the next level.
reservoir which contains all the information.
It has m^
is the top level.
M^
storage hierarchy system.
is general izable to a h-level
model
n
(j=^i+l)
is
is an integer
•J
greater than
1
References
__:.,.$
M^
m.
„j
|m.
pages of size Q.
Common
Data
pages of size nQ.
Path
M'
Figure
2
.
Denote
is
Basic model of
6
Definitions of
Formal
3
S tora ge
reference string by
a
reservoir
J
r
a
data storage hierarchy
Management A l gori thms
-
"r,
,
,"
rn^^...v
where
the page being referenced at the t-th reference cycle.
stack for M
to LRU.
r.
Let
(If^t^n)
S.
be the
at the beginning of the t-th reference cycle, ordered according
That is,
SJ.
=
(sj.(l),
sj.(2),
..., sJ.(K)), where sj.(l) is the
most recently referenced page and S.(K) is the least recently referenced
page.
pages).
Note that
Kim.
(m.
1
1
=
capacity of M
The number of pages in
By convention,
S,
-
,
ls,|-
S
is
in
terms of the number of
denoted as
S
,
hence
10Define M
an ordered set.
IS
S^
Similarly, we can define
ordering.
Let us denote the pages in
consists of an equivalent of
and \V
S;:
by
M'^
the contents of
as
P:j
(M^)^
P2
,
•
,
•
Each page,
•
-
(P:J)^
pi
''
breaking down
where
the pages in mJ
t
=
x
pi
y2
'
•
•
'
'
)^
(P
called
is
Formally,
.
X
.
(M:J)^
t
.
U
=
t
1^^^
the f amily from the
•
(sj(k))^
parent page
P
J
Any pair of pages,
denoted by
P^
f
-
ya
and p\ from
yo
P^
ya
p\
(P'^)^
y
Furthermore,
.
yu
a
are said to be family equivalent
parent page
(|
S^l
,
a
page
P
for
=
k
...
2, 3,
1,
,
yz
p""
yz
g
P;^
.
Y
denoted by
w, where
Intuitively, two stacks are in corresponding
|S::| ).
order if, for each element of the shorter stack, there
is
a
corresponding
page in the other stack at the same stack distance (The stack distance
for page
and M^ are said to be correspondingly equivalent
M
p
.
.
='
mJ
mJ.
is defined to be k.).
(k)
S.
.
k
=
1,
2,
.
(x) ^
sj.
(y)
.
I
such that
sj.
(k)
-
denoted by
.
Mj.|=|Mj|and for any
il"
,
(x) and sj
S:J
.,|mJ.
there exists x,
for all y f k.
Intuitively,
the two memories are correspondingly equivalent when each page in one
memory corresponds to exactly one page in the other memory.
The red uced stack
for
k
and
s],
-
1
,
(k)
.
j<
.
sJ.
.,|s||
(j)
A
,
S^,
where
of
j^^
for j<j|^.
is
S
defined to be
the minimum
is
Intuitively,
S'
(k) = S'
jj^
where J|^>j\_i
sJ.
is
(j.)
(jg = 0)
obtained from
by collecting one page from each family existing in sJ
,
sj^
such that the
page being collected from each family is the page that has the smallest
stack distance within the family.
,
liZ^n)
(for
^
denoted by
,
and S^ are said to be in corresponding order
min
=
and
P
y
S[ = sj, if Sj,(k) = sj(k)
w
.
-J
from its family are said to be corresponding pages
S.
^
yn
'
the set of pages, each of size Q^, obtained by
is
"
Q./n.
=
Q.
in M^,
,
P:J
pi
yl
.
"
.
smaller pages, each of size
n
Denote this set of pages by (P^)\ i.e.,
In general,
for M
without any
S.
,
I
11-
In the following, we define the storage
assume that the page referenced at time t is
In each case,
LRU (S;
,
P^
Case
)
ya
L
1
management algorithms
=
P^
:
is
sl.,
t+
ya
s!
e
,
'
t
ya
defined as follows:
f
=
P^
sl(k)'
ya
t'
= pL
si.id)
't.r--^;a'
si^^d)
=
If \s]{
= m.
t'
1
P^g
:
'
(x-1
S
(
)
<
x
<'
k
,
1
^
k<x<(s;|
Si,,(x)
.
I
P
't.l(^)=J3i(,)
sj^^(x)
,
then
= sj.(x-l)
< x^min(m., |sj|+l)
sl(m.) is the overflow, else
=
P^
1
,
oa
t
1
there is no overflow.
LOCAL-LRU-SOP
Case
1
:
(sj.
,
sj
P^
e
s!
ya
2
P^
ya
=
LRU
u (sl
^
-
,
P^^
(sj.^^
sj^^
,
defined as follows:
is
)
t
^
^Ul
Case
)
,
t
sl
i
t
=
sj..
,
'
P^
e
pM
ya
sl
sj
,
't+1
t
:
t_
y_
.
_
_
LRU (S^
P^g)
,
S^,
,
LRU (sj
=
,
P^
If there is no overflow from S.
=
then si^^
If overflow from S.
P^
ya
i
sj.
then (S^^^
,
sj^^)
=
SOP {s[,
4-
'
^^
^
i'
4+1
3
-
the page
P„,
^ ^
oa
is
t
Case
S^^
and
S\,
S^^
t
=
and P^ ^^
y
,
^^^"
sj.
P^^) defined as
,
4+1
=
4-
if pj i sj. then sj^^
=
LRU (sJ.
S:j
''o
'
,
:
t
(handled as in Case 2)
S^
I
'U
-
LOCAL-LRU- POP
Case
1
:
p'
ya
2
:
P^
ya
S[.
defined as
s:
6
t
iM
=
4.1
Case
is
)
^ S^
and
(4
.
P-^
6
t
=
IPM {S\
y
ya
s{
'
4
:
t
,
P^g)
If no overflow from S
,
sJ.
then
=
LRU (sJ
S^^.^
=
S^..
y
and
S:J^i
=
S'^.
pj)
,
-12-
If overflow from S.
is
P
t
^^t+1
=
^li
Case
3
:
P^
—
4'
^"d
sl and P^
^
ya
"
^t+1^
'
'
^^t"
•
^t'
''oa^
^^^^^
""^
defined as:
pJ)
LRU (sj.
-
i.^
sl
i
y
t
then
oa
,
:
t
(handled as in Case 2 above)
•
GLOBAL-LRU-SOP
S^,
=
•
•
•
P^^
(S^^
,
sj
LRU (S[
,
P^g) and sj.
,
)
•
(s|^^
=
LRU (sj
=
If no overflow from S.
then S.^,
If overflow from s! is
p'!^
L
GLOBAL-LRU-D OP (s}
,
S]..
=
LRU (S[
S^
,
t
t
,
p\)
ya
P]^)
=
S
then (sj
Oa
sj^^
,
sj^J
L+l
,
(sl^,
and sj.
,
t+i
=
If no overflow from S.
then s|^,
If overflow from sj is
P^^ then
S^ ,
t+i
LRU (sj
=
s|,
(sj.^^
,
follows:
,
and S^
L+l
=
is defined as
pj)
,
,
,
)
,
=
=
SOP (sI,
S"^
,
L
S^
p""
,
,
sj,
,
X,
)
oa
defined as:
)
is
,
P^)
and S^
,
=
sj^^
=
OOP (sj.
)
5;^,
,
P^^)
-133
Properties of Data Storage Hierarchy
,
One of the properties of
leaves
Read-Through operation
a
is
that it
"shadow" of the referenced page (i.e., the corresponding
a
storage levels.
pages) in all
Does this multiple redundancy exist at all times?
for the page.
That is, if
a
page exists in storage level M
this as the Multi-Level
for the
7
,
its corresponding
will
storage levels lower than K
pages always be in all
in Figure
This provides multiple redundancy
Inclusion (MLI) property.
?
We refer to
As illustrated
LOCAL-LRU algorithms and in Figure
8
GLOBAL-LRU algorithms, it is not always possible to
for the
that the MLI property holds.
^(a)
in Figure
is not found in
the page
M"^.
In this
guarantee
For example, after the reference to P_,
P-,-,
exists in M
but its corresponding page
P-i
paper we shall derive the necessary and
sufficient conditions for the MLI property to hold at all times.
Another desirable property of the
Date.
Storage Hierarchy is to
That is, under what
avoid generating references due to overflows.
conditions will overflow pages from M
already existing in the storage level
find their corresponding pages
M^"*"
?
We refer to this as the
Multi-Level Overflow Inclusion (MLOI) property.
We shall
investigate the
conditions that make this property true at all times.
Refering to the basic model of
for high performance
to M
r
a
data storage hierarchy in Figure
6,
it is desirable to minimize the number of references
i
(the reservoir).
If we increased the number of pages in M
,
"^
or in H
i
,
or in both, we might expect the number of references to M
to decrease.
As illustrated in Figure
9
for the LOCAL-LRU-SOP
algorithm, this is not always so, i.e., for the same reference string,
the number of references to the reservoir actually increased from 4 to 5
—
-14-
reference to M
reference to M
contents of H
overflow from H
r
reference to H
contents of M
reference to M
reference to M
(b)
(a) LCCAL-LRU-SOP
Figure
LOCAL-LRU-DOP
Examples of MLI violations for
7
Local-LRU algorithms
reference to M
pi
1
•^11
I
pi
I
21
p1
31
I
'
I
'
pi
i
^11
I
pi
21
I
Ul
pi
I
rsi
overflow from M
M"*
p),
=
^
I
I
pJ
2
I
!
pJ
3
pJ
I
^}
\
^21
I
pi
"^31
I
pJ
?J
A
2
I
I
pj
5
!
p1, '/P
2)
from M
overflow; Trom
pj
p1
I
contents of M
(m^
reference to
^11
i
1/^41 \
\
reference to M
I
V\
],0\
'/31
pi
I
1*^51
J
1
contents of H
pi
1
J
reference to
M'
cor^tents of
M'
contents of H
(.j=2)
I
p,
I
2
1
I
reference to
p-'
V
P''
f
P'^
\J.'\
1
\
p-'
\Jy]
*
M*"
(a)
i
I
p-'
2
*
(m.
"j
'
2)
reference to M
GLOBAL-LRU-SOP
(b)
Figure
8
Examples of MLI violations for
Global -LRU algorithms
GLOBAL-LRU-DOP
15-
reference to M
M1
21
'")
mi]
;
11
I
I
^21
i''ii!''ii
-pi
Tpi
pi
'31
pi
'
pi
I
^31
^11
jMlj'^Al
I
contents of M
I
(m^
=
2)
>
'
'
IP 1
1
P
*^11
J_
'
21
P
1
11
i
!
"31
1
''ll
T'
overflow from H
'1.!
31
-4
(a) .n.=2
reference to H^
p\
I
pI
pj
I
pj
pj
I
2
p|
Ip^
'\\'Z
,•3
'3
I
I
2)
pJ1
pJ1
,
i
.
r>Ji
.1
^—
1
r
pj
/i
I
(.J
reference to M
I
<*
*
,
H•
=
*^3
I
2
1
contents of
pJ
'
i
^
PJ
I
;
i
number^ of references to M
reference to M
P^ [p^
'^21
•^lll
p^
•^11
;7Tp''
*^11
*^31
'
,
-
1
contents of H
(m,
-
'11
\
pi
'
*^31
I
pi
11
3)
(b) m.-3
overflow from H
_l.
reference to
H"'
M
I
pJ
I
2
oJ
.
OJ
pJ
'.'
contents of M,J
(mj » 2)
•^1
reference to M
I
*.l
1
t
*
.^\
number of references to M
I
L,_.
p
=
Figure
after M
is
ncreased by
as a Multi-Level
1
5
MLPA for LOCAL-LRU-SOP
page in size.
feging Anomaly (MLPA)
.
We refer to this phenomona
One can easily find
situations where MLPA occurs for the other three algorithms.
Since
occurrenceof MLPA reduces performance in spite of the costs of
increasing memory sizes, we would like to investigate the conditions
to guarantee that MLPA does not exist.
-16-
3
.
Summary of Properties
1
The MLI, MLOI, and MLPA properties of the Data Storage Hierarchy have been
derived in the form of eight theorems.
These theorems are briefly explained
and summarized below and formally proven in the following section.
Multi-Level
of pages in M
is
Inclusion
(
MLI
shown in Theorem
It is
:
)
greater than the number of pages in M
larger than those of M
)
,
that if the number
(note M
pages are
then it is not possible to guarantee MLI for all
It turns out that using LOCAL-LRU-SOP, or
reference strings at all times.
LOCAL-LRU-DOP, no matter how many pages are in
a
1
M-"
or
m\
one can always find
reference string that violates the MLI property (Theorem 2).
Using the
GLOBAL-LRU algorithms, however, conditions to guarantee MLI exist.
For
the GLOBAL-LRU-SOP algorithm, a necessary and sufficient condition to
guarantee that MLI holds at all times for any reference string is that the
number of pages in M
be greater than the number of pages in M
For the 6L0BAL-LRU-D0P algorithm,
a
necessary and sufficient condition to
guarantee MLI is that the number of pages in M
twice the number of pages in M
be greater than or equal
to
(Theorem 4).
Multi-Level Overflow Inclusion
(
MLOI
)
:
be guaranteed then MLOI cannot be guaranteed.
cannot guarantee MLOI.
(Theorem 3).
It
is
obvious that if MLI cannot
Thus, the LOCAL-LRU algorithms
For the GLOBAL-LRU-SOP algorithm, a necessary and
sufficient condition to guarantee MLOI is the same condition as that to
guarantee MLI
(Theorem 5).
For the GLOBAL-LRU-DOP algorithm, a necessary
and sufficient condition to guarantee MLOI is that the number of pages in
is
strictly greater than twice the number of pages in M^
(Theorem 6).
Thus, for the GLOBAL-LRU-DOP algorithm, guaranteeing that MLOI holds
will
also guarantee that MLI will
hold, but not vice versa.
M'^
17-
Multi-Level Paging Anomaly (MLPA)
:
We have identified and proved
sufficiency conditions to avoid MLPA for the GLOBAL-LRU algorithms.
For the GLOBAL-LRU-SOP algorithm, this condition is that the number of pages
in M
must be greater than the number of pages in M
increase in the sizes of the levels (Theorem 7).
before and after any
For the GL08AL-LRU-D0P
algorithm, this condition is that the number of pages in
twice the number of pages in M
M"^
must be greater than
before and after any increase in the sizes
of the levels (Theorem 8).
In summary, we have shown that for the LOCAL-LRU algorithms,
of sizes for the storage levels can guarantee that
a
no choice
lower storage level always
contains all the information in the higher storage levels.
For the
GLOBAL-LRU algorithms, by choosing appropriate sizes for the storage levels,
we can (1) ensure that the above inclusion property holds at all times for all
reference strings,
(2)
levels are generated as
guarantee that no extra page references to lower storage
a
result of handling overflows, and (3) guarantee that
increasing the sizes of the storage levels does not increase the number of
references to lower storage levels.
the following eight Theorems.
in the following section.
These results are formally stated as
Formal proofs of these Theorems are presented
18-
THEOREM
1
Under LOCAL-LRU-SOP, or LOCAL-LRU-OOP, or GLOBAL-LRU-SOP,
or GLOBAL-LRU-DOP, for any
THEOREM
m^.
>
£
2, m
implies^ r,t, (M:^)^^
m^
2
Under LOCAL-LRU-SOP, or LOCAL-LRU-DOP, for any m^
3
THEOREM
r,t.
(MJ)''
^
,
>
2,'Vr,t,
(mJ)^
3
mJ.
iff m
>
2,\f-r,t,
(mJ)""
2
M^
iff m
>
2,
.
>
m.
>
2m
5
finds its corresponding page in
V'r,t, an overflow from
M^
m.>m.
iff
M"^
6
V
Under GLOBAL-LRU-DOP, for any m. > 2,
finds its corresponding page in M
THEOREM
.
4
Under GLOBAL-LRU-SOP, for any m^
THEOREM
2, and any m
3
Under GLOBAL-LRU-DOP, for any m.
THEOREM
>
M^
Under GLOBAL-LRU-SOP, for any m.
THEOREM
MJ.
r,t, an overflow from K
iff m.
>
2m.
7
Let M^
Let M'^
Let m.'
(with m. pages),
(with m.'
>
m.
pages),
and m.'
>
m.
no MLPA can exist if m.
(with m. pages) and M
M"^
>
M'"^
pages) and
m'
be System B.
Under GLOBAL-LRU-SOP, for any m. > 2,
.
m.
J
(with m.'
be System A.
>
and m.'
J
1
m.'
1
THEOREM 8
Let System A and System B be defined as in THEOREM 7.
Let m.'
>
m.
and m.'
>
no MLPA can exist if m.
m.
""
.
2m.
Under GLOBAL-LRU-DOP, for any m. > 2,
and
.
ni
'
>
2m.'
19-
3.2
Derivation of Properties
THEOREM
1
Under LOCAL-LRU-SOP, or LOCAL-LRU-DOP, or GLOBAL-LRU-SOP,
>
or GLOBAL-LRU-DOP, for any m.
2, m. <.m.
B
implies
r,t, (mJ)^
PROOF
Case
1
<
m.
:
m.
*
Consider the reference string r="
'
P,
i
P^
,
,
.
.
.
,
P/
^-j
\
".
vJ
Using any one of the algorithms, the following stacks are
obtained at t=m.+2
:
J
\
^t
"
^
^m.+l)a
"
^
^m
+1)
Thus, Pj^ e
Case
2
:
.
'
'
""m-a
""m
,
'
•
•
•
.
.
.
,
but p]^ i
mJ.
m. = m.
=
^2a
'
P^
P^
.
^
''la
)
i.e., (mJ)^^ M^
,
(mJ)""
'
.
w
Consider the reference string
=
r
"
P^
P2g
,
"
,
.
.
Using any one of the above algorithms, the following
stacks are obtained at t=w+2
\
^t
^
^
^(w+l)a
^
n
Thus, P^g
'
6
'
^/a
'^(w+1)
m],
'
'
^w
'
•
'
:
•
•
^3a
'
•
but P^g i (MJ)^
•
'
,
'
M
^
''2a
'
^3
'
i.e., (mJ)^^
Q.E.D.
mJ.
.
.
,
P(y^+-i )3
^
mJ,
-20-
THEOREM
2
Under LOCAL-LRU-SOP, or LOCAL-LRU-DOP, for any m. > 2, and any m.
PROOF (For LOCAL-LRU-SOP)
For
m.;^m. the result follows directly from THEOREM
For
m.>
r =
^
"
m.
using the reference string
,
P^
P^
^za
P^
P^
^la
'
1.
'
^za
•
'
•
•
za
'
"
P^
P^
^2a
'
be produced at t=2m.+l
the following stacks will
'
m.a
'
:
J
^t
"
^m.a
^
^a
'
^(m.-l)a
'
rj _
/
^t
^
Thus
'
J
pJ
2
pJ
pJ
m.
.
J
P^
6
P{m -m +2)a
'
•
•
•
J
J
m.-l
'
•
'
•
'
•
pJ
^
1
^
'
^
'
J
.
.
but P^, i (M^)^
mI
(MJ)^i
i.e.,
,
M^
Q.E.D.
.
PROOF (For LOCAL-LRU-DOP)
<
m.
m.>
m.
For m
For
„ _
^ ~
.
II
the result follows directly from THEOREM 1.
,
pi
^za
'
using the following reference string
pi
pi
^la
^za
pi
'
p""
*^2a
'
•
•
•
'
P^
P^
Where for
1
since
overflow from M
is the only
-^
P
z
i
< m.
.
a.
ma
'
be produced at t=2m.+l
The following stacks will
<
p""
^za
6
|
Thus, P^^ 6 M^ but P^g ^ (MJ)'
,
,
i.e.,
,
•
.
.
,
.
(mJ)^^ mJ
Q.E.D,
"
'
:
P3
,
P^
,
P^
C
,
-21-
THEOREM
3
V
Under GLOBAL-LRU-SOP, for any ni.> 2,
r,t,
3
(M^)^
m!
iff m
.
>
m.
m!.
=^
.
PROOF
This proof has two parts.
or equivalently, m
£
.
J
:
m
3
V
m. =v
m.>m.^.
Part (b) to prove
PROOF of Part (a)
Part (a) to prove
.
£
.
3
=>
r.t,
r,t,
(M;^)^
^
hI
r,t,
(M;?)^
^
m!
C
I
m
v
r,t,
^
(mJ)""
3
(M;^)
m.>m
L
mJ.
This follows directly from THEOREM 1.
Q.E.D.
To prove Part (b), we need the following results.
LEMMA 3.1
t
such that
"v/"
>".
(a)
(MJ)'2Mi
.
-
M:^
and (b)
m.
if m
,
=
.
+
m.
1
then
,
? sj
sj
PR OOF of LEMMA 3.1
For t=2 (i.e., after the first reference),
(a) and
(b) are true.
Suppose (a) and (b) are true for t, such that JM^^j^m.
Consider the next reference
Cas e
1
:
:
It is a reference to M
There is no overflow from M
Since Global-LRU is used,
Case 2
:
It is a
There is no overflow from
arguement as Case
1
or M
so
,
is still
(b)
reference to
:
M"^
(a)
is still
true.
:
If no overflow from M
M'^.
this overflow can be treated as a "no-op".
:
It is a reference to
There is no overflow from
M'^
,
the same
If there is overflow from M
applies.
overflow page finds its corresponding page in M
Case 3
true.
m"^
,
the
Since SOP is used,
.
Thus (a) and (b) are preserved.
:
since
|
M'?^iK m^
•
Thus the same reasoning
as in Case 2 applies.
Q.E.D.
1
-22-
LEMMA 3.2
V
r,
(a)
such that
t,
(MJ)S
MJ
(b)
m.
=
M;:
$1
if m.=m.+l
,
"=
SJ
Let us denote the conditions
(a)
PROOF of LEMMA
,
,
then
and (c)
(b) and
(sj(nij))^ f]
= ^
t<t*
for all
by the t*-th reference.
and sj(m.) f
for all
From LEMMA 3.1 we know that (a) and (b) are true for all
t* + 1.
=
t^
t* + 2,
induction on t, starting at
that Z(t,
Case
1
«*
3 .2
S:?'(m.)
Let t
^
jointly as Z(t).
(c)
Suppose the first time S^(m.) is filled is
That is,
4
is true as
)
:
,
t,
etc.
Vie
t*.
^t*.
show, by
shall
that Z(t) is true
,
t
t>
First we show
follows:
m].^
mJ^ §
S^^ Q SJ^ and MJ^
m[^ =>sj^(nK-l)
e
sj^(m.)
g
As a result of the reference at t* (to m'), S'J^^i(m.) = S:^^(m.-1)
and S,^(m.) overflows from M
corresponding page in
Vr
This overflow page finds its
.
becau:ie there is no overflow from M
can be treated as
Since SOP is used, the overflow from M
Furthermore, since Global-LRU
reference,
Case 2
(b)
(mJ^)^
:
(mJJ'3
3 M^^
m. ,
J-
>
(i.e., the last page of
S^(m^) overflows from M
page from M
true.
1,
(b)
(b)
is true after the t*- th
x
^
S:J^
.
^
Thus, Z(t^
)
Thus Z(t^) is true.
sj^(k) such that (sj^(k)
=
^k>rsj.J|
k.
Thus (S:J^(m.
f
is not
is
true.
mJ*
=
J-
,
))^ fl
'
1
'
s!+ =
t*
in s|,J
Th^^re is no
preserved,
)''n
and (sj^(x))'"n mJ^ =
overflow from
finds its corresponding page in M
is still
"no-op".
a
and mJ^ e mJ^
mJ^ and mJ^ i MJ^=>
for all X, where
used,
[sj^^J >|s[.^^J =»(a) and (c).
and
SJ* Q Sj^ and (sJ^(k))VlMJ^
Case
is
and (a).
(b)
and
t*+l
.
M'^.
Thus the overflov.
For the same reasons as in
S^*+J->(a) and
(c)
are
Assume that
true; to show that Z(t
is
Z(t|^)
the next reference, at time
t,
,,
i
S"!
h
t.
,
'
does not exist, i.e.,
1
(m.) =
S'i
k
i
i
or M;
,
h
'
•^
^k
J
then (a) and (b)
hold because Global -LRU is us 3d and because overflow from M^
still
its corresponding page in
If
:
to a page in M
is
true, we consider
is
k+1
Imagine that the last page of
If the reference at
)
tfie
(See tne proof of LEMMA 3.1).
M"^
reference at t,-, is to
the argument
as
finds
a
pa le not in M^
^k
then we can apply
,
that used in considering the reference at time t, above
to show that Z(t.
is still
-,)
true.
Q.E.D.
LEMMA 3.3
r,t, if m.-m.+l
"V
then (a) (nj)^
3K
^"^ (^) (S^Cm ))^ fl sl =
PROOF of LEMMA 3.3
For
such that lM:^|<m.
t
(a) follows directly from LEMMA 3.1
and
(b) is true because S^(m.) =
For t such that
[mJU
m.
(a) and
follows directly from LEMMA 3.2
(b)
Q.E.D.
LEMMA 3.4
•Vr,t,
if m,
>m.
then (a)
(M'M^D
m1 and
(b)
(sj{m.))^
sl =
PROOF of LEMMA 3.4
Let m.
J
For k=l
=
m.+k
We shall
.
provi? this
(a)
and (b) are true from LEMMA 3.3.
Suppose that (a) and (b) Are true for
Consider m.^m.+(k+l).
1
lemma by induction on k.
1
page in size
:
k.
That is consider the effects of increasing
M'^
by
-24-
Since M
is
(with m.+k+l
unchanged, M
pages) sees the same
Applying the stack
(with m.+k pages).
reference string as M
inclusion property (Mattson.et al., 70), we have
M"^(with m.+k+l
pages
Suppose (S^(m.+k+l))
)
D
M"^
f] S.
corresponds to this page.
(with m.+k pages).
f
^
then there is
But
Thus
a
This shows that (b) is
still
S^(m.+k+l) is not in M
true.
Q.E.D.
:
m.
J
>
m. .->
V
r,t,
(mJ)^
3
mJ.
:
This follows directly from LEMMA 3.4,
Q.E.D.
is
page in M
This contradicts the property that (mJ)^
m.+k pages).
PROOF of Part(b)
(a)
still
that
(with
3
mJ.
.
true.
25-
THEOREM 4
Under GLOBAL-LRU-DOP, for any m
>
Vr,t, (M^)^3 mJ
2,
iff m. >2in.
PROOF
This proof has two parts:
Part (a)
:
m^
< 2m.=>3r,t, (mJ)^*
Part (b)
:
m.
>
PROOF of Part (a)
For m.
<
:
2ni.^
V
m.<
2m.-=^
r,t,
M^
(mJ)^3.mJ^
3
r,t,
(mJ)^^
m].
the result follows from THEOREM
in.
Consider the
for 2m.
cas.e
The reference string r
>
"
=
m.
>
p]
m.
:
pj,
,
la
1,
p],
,
2a
,
.
.
.
3a
"
p]„
(2m.} a
v
,
will produce the following stacks:
^t "
^
^2m.)a
si =
(
a,
a„
,
''(2m.-l)a
'
a.
,
.
.
,
'
.
•
•
a
,
alternatively, starting from
L,
•^
•
where a.'s are picked from
)
=
L,
.
1
^'
P(m.-fl)a
'
(
1
P^
m.
,
P-^
i \ ,
(m,.-l
•
.
.
and L^
L,
.
P,
)
1
)
1
^''"^
4
^
^2m.
^
^^2m.-l)'
'
If m. is even, then
elements of
(a,
L,
and (a„
1
2
elements in L„.
,
a_
a.
4
,
'
"
.
i
i
S::
{P.
t
.
.
h'
^m,+l^
'
...
.
Mm.
If m. is odd, then
J
a
.
a
)
_
m.'
i
T
^t.
(m. + l }a
is
We see that P,
2
is not in
.
,
,
'
corresponds to the first m./2
corresponds
to the first m./2
^
J
_
i
(a,
is
not in Si since m./2 <m.).
t
,
L,
a.^
and
,
.
(a^,
first (m.-l)/2 elements in L^.
c
J
.
,
.
r
J
$
a^,
)
corresponds to the first
....
a
,
)
In both ca?es,
is at most the
corresponds to the
i
i
We see that the page
P/
>, is in S.
"
L
im.+ ;a
,
•,
I
but its corresponding page is not in S^ because max{
_ix
z,
:^
t
"i
,.\
+ l)
(m.+l)/2 elements in
thus, a-
corresponding
page
r
r
in S^ but its
(m.-l)/2
(m.-l)-th element of Lp.Pop,
(M"J)'i^fi]^
Q.E.D.
_(g,
)
=
m.-l,
."D+i^'P,^ +2
•
-26-
To prove Part (b), we need the following preliminary results,
LEMMA 4.1
Under GLOBAL-LRU-DOP, for
distance
>
2, m. >.2ni.
,
a
page found at stack
implies its corresponding page can be found within
in M
k
ni.
stack distance 2k in
M":
.
PROOF of LEMMA 4.1
We prove by induction on
t.
At t=2 (i.e., after
At t=l, the statement is trivially true.
the first reference) s|^(l) and its corresponding page are both at
the beginning of the stack, hence the induction statement is still
true.
Suppose the induction statement is true at time t, i.e.,
=
P^
=^
-^
sl(k)
za
t
can be found within stack distance 2k within
P"^
z
Suppose the next reference is to
'^
P,^,
Wa
"^
Case
1
P^^^
:
wa
6
= si(x)
(P^
t
wa
mJL
From the induction statement,
There are three cases
•
S'J
.
t
:
)
is found within stack distance 2k
P''
w
in S^ as illustrated in Figure 10.
-stack distance
—
"
=^
w^
M'
'<''W^I
-
"'
"
2x
--^
"
p/^
M
I
I
A'
Figure 10.
Consider the page movem,ents in the two stacks as
the reference to
(1)
p\ and P
Wa
V/
P
a
result of handling
:
wa
are both moved to the top of their stack,
'
the
induction
.
(2)
Each page in A increases its stack distance by 1, but its
corresponding page is in A', each page of which can at most
increase its stack distance by
Thus the induction statement
1.
holds for all pages in A.
(3)
None of the pages in
moved.
B
are moved.
(See previous diagram)
None of the pages in
Suppose a
(k>x), has its corresponding page,
P?
Then P^ can at most increase its stack distance by
1.
page in B,
But
are
If a page in B has its corresponding
page in B', the induction statement is not violated.
in A'.
B'
S.(k)
=
P,
w<2x because
P^ e A'
=
^t^^^
Since 2k>2x, the induction statement
.
is not violated.
Case 2
P^
:
wa
M^
4.
^
Each page in M
page in
^
w ^
P"^
,
t
M-j
t
increases its stack distance by
slik), k<m.
=
,
=
then P^
within stack distance 2(k+l) in
Case
3
:
^ M^
t
P^
wa
,
P"^
w
overflow from M
increased by
Each page in
H'^
a
.
Hence if
sl,^(k^l), and P^ can be found
M"^
at time t+1
4 mJ
t
As a result of the read-through from M
is
Each corresponding
can at most increase its stack distance by 2, one d ue
M'^
to the reference and one due to an
P\
1.
stack distance of
r
each page in M
That is, for
1.
i
•
,
k <, m^
,
can at most increase its stack distance by 2,
one due to loading the referenced page and one due to an cverfl ow
from M
.
Hence, the page
2k+2 in M^.
P
Since max(2k+2)
is
=
found within stack distance of
2m. «c m.
1—
J
,
P'^
is still
z
Q.E.D.
in
M"^
-28-
COROLLARY to LEMMA 4.1
m.
J
L
I
J
X,
PROOF of COROLLARY
For any P^^ in S
,
its corresponding page can be found within
stack distance 2m. in
S.
,
and since pages in
the information in the last page of
i.e..
(sJ(m^.))'*
PROOF of Part (b)
:
n
si
=
j
A
is not found in S'
(M'j)^O M^
This follows directly from LEMMA 4.1.
Q.E.D.
are unique,
S'
.
m.>2m.^Vr.t.
S'J
,
-29-
THFOREM
5
Under GLOBAL-LRU-SOP, for any m. > 2,
findso its
v,wii(;j|_ii_iiiuiii^ page
loj corresponding
(jayc in
III
V r,t,
m.>
'"
iff
t^
M'^
an overflow from
m""
m.
COROLLARY
Under GLOBAL-LRU-SOP, for any m. > 2,
finds its corresponding page in
V
r,t, an overflow from M^
iff \' r,t, (M:j[)^p M^
M"^
PROOF
This Proof has two parts as shown below.
PROOF of Par t (a)
:
m.>m.
v r,t, an overflow from M
=^
finds its corresponding
page in M
From LEMMA 3.4 m. > m
V
=^
.
Suppose the overflow from
Then just before
P
is
oa
After the overflow,
P
03
Suppose the overflow,
(m|)Sm|. and (sj(m.))^n
r,t,
m\
P^^ is caused by
overflowed, P^ exists in
p'!^
Oa
is
,
caused by
m\
a
means that the last page of
finds its corresponding page
m.j<m.^ 3
not
is
1-1
still
'"'t,
in
P
M"-^
,
P^
za
G
M^
t
1,
and
m.
P~^
z
M-^
,
t
M"^
and
(S:^(m.)) f]S.
This
.
after an overflow from
,
M'^
occurs.
does not
M"^
then there exists
Wo can find a reference string Such that
za
string of references to
M"^
existing in
such that an overflow from M
at the time of the overflow of P^
A
f>v\
thus, the overflow page P
< m.=^3 r,t, (M^)''^mJ^
^^
<i
m"^.
is not in M
find its corresponding page in
From THEOREM
still
reference to
M'^
=
M'^
P^ exists in
i.e., the information in the last page of
:
reference to
finds its corresponding page
Then just before the overflow from
PROOF of Part (b)
a
sj.
M
from M^
,
P'^
is still
not in
M"^.
z
will produce this condition.
Then at the time of overflow of P^
za
,
it will
page it M
Q.E.D.
not find its corresponding
=
^
-30-
THEOREM
6
Under GLOBAL-LRU-DOP. for m.
>
V
2,
finds its corresponding page in M
r,t, an overflow from M^
iff m. >
2in.
CORO LLARY
V
Under GLOBAL-LRU-DOP, for m. > 2,
finds its corresponding page in
r,t, an overflow from
M^'
implies that V" r,t, (M'J)^^M^
M-"
PROOF
This Proof has two parts as shown below.
PROOF
of Part (a)
m.>
:
^\/r,t, an overflow from
2m.
M^
finds its
corresponding page it M
THEOREM
4
ensures that m.>2m.^Vr,t, (M'j)^o
ensures that (S'J(m.))^
of THEOREM
in Part (a)
PROOF of Part (b)
^—^
m.<
:
J—
sl =
m].
and LEMMA 4.1
we then use the same argument as
,
5.
2m.
^3
-"
r,t, such that an overflow from M^
1
_
.
does not find its corresponding page in M
Case
1
m.^
2m. =^
:
m.
<
3
2m.
r,t,
(m|)^^
argument
We then use the same
Case
2
:
in.
=
(from the proof of
M].
part(a) of THEOREM 4)
as in Part (b) of THEOREM 5.
2m.
J
1
The reference string r -
"
p\^
"
Pj^
,
.
,
.
.
,
p'^^^_^^
,
P(2m. + l)a
will produce the following stacks (at t=2m.+l):
^t
"
^^2m.)a
/pJ
^
m.
c3 _
^t
'
pJ
2m.
^(2m.-l)a
pJ
'
ni.-l
•
i
P
P/
^-.s
,1
m.+l
P(m. + l)a^
'
pJ
'
^2m.-1
In handling the next reference,
and
•
•
'
•
•
•
•
pJ
^]
to page P/p
pJ
'
^,\^
\
^^.+i^
»
the pages ?,
overflow at the same time, hence the overflow page
^ ^
from M
does not find its corresponding page in M^
"i
Q.E.D.
^-j
\
-31-
THEOREM
.
7
i
i
(with m. pages),
Let M
(with m.'
Let M'
pages),
Let m.' > m. and m.'
>
no MLPA can exist if m.
Y*
(with m. pages) and M
M'^
m.
>
M'"^
.
m.
(with m
.
be System A.
pages) and
'
M*"
be System B.
Under GLOBAL-LRU-SOP, for any m.
>
and m.'
>
2,
m.'
PROOF
We
Shan show that V"r,t,
This will
Since
(mJ
U
(M'j)
"
)
^
U(M'j)M
(M'
J
ensure that no MLPA can exist.
m.'>m.
and LRU is used in M
stack inclusion property to obtain M
From THEOREM
5,
and M'
c
M'
we can apply the LRU
,
.
we know that overflows from M
find their corresponding pages in M^ and M
i
or from M'
i
respectively.
always
Since
SOP is used, these overflows can be treated as "no-ops".
Thus, M
and
M''^
see the same reference string and we can apply
the LRU stack inclusion property to obtain
\A.
'£
\A
.
(since
and LRU is used).
m].C
Mj.^
and
M^CM'I =^
(M^
U
(M^j)^') CL (f^'j LJ(M'-J)^')
Q.E.D.
m.'>
m.
-32-
TIOREM
8
Let System A and System B be defined as in THEOREM 7.
Let
m/ >
m.
m
and m.'
no KiLPA can exist if m.
Under GLOBAL-LRU-DOP, for any m.
.
.
>
and m.'
2m.
J
>
2m.
J
1
'
>
2,
.
1
PROOF
We need the following preliminary results for this proof.
LEMMA 8.1
Let
be partitioned into two disjoint stacks, W
S'J
follows:
minimum
V|.(k)
W (k) = S:^(j.
=
j^ > j^_^
k=l,...,|V J
^'^^
^t^Ji,;)
for k=l ,..., |wj where JrfO, and
)
such that 3 P^^
J|^>J|^_-|
such that
V
and V
P^g
Sj
6
6
j
is the
and P^^ g
^t^^k^'
sj.
where
,
defined as
and
jr^=0,
P^^ p ^^(j^).
is the
j,
minimum
(Intuitively, W^
is the stack obtained from S^ by collecting those pages that have
their corresponding pages in
in S.
is
preserved.
Then, i/ r,t,
(a) W
g
such that the order of these pages
what is left of
is
V
M.
C
and (b) V
sj^
pages corresponding to all
after
S.
where
W.
is formed.)
is the set of
the pages that ever overflowed from M
,
up to time t.
PROOF of LEMMA 8.1
From THEOREM 4, m.
in M
,
Vr,t, (M"J)\o
2m. :^
its corresponding page is in
M^ is exactly H
and W
for V
/
,
and W
e
s],
.
.
Thus, for each page
This set of pages in
by definition.
Since the conditions
are mutually exclusive and collectively exhaustive,
the other pnges in
M't
Since
does not have
a
M;?
M^
page in
V
are by definition in V
that are not in W
a
corresponding page in M
corresponding page must have once been in
and later overflowed from M
.
Thus
a
M^
,
.
its
because of Read-Through,
page in
V,
is a
Q.E.D.
page in 0.
.
.
,
-33-
LEMMA
8.??
Any overflow page from
is a page in V
M;^
PROOF of LEMMA 8 .2
^V
^
From THEOREM 4. m.
>
2m.
From THEOREM 6, m.
>
2m.t^'V' r,t, an overflow from M
corresponding page in
An overflow from
(m|)^
mJ
is
caused by a reference to M
M^ also implies that there is an overflow from M
Suppose the overflow page from
p-^^
V.
11;^
is P^
is
M.
c
If P^
oa =
is
P"^
P-^
page in
M"^
c
ya f
P
oa
overflowed from m!
,
THEOREM
overflow at the same time so
P'^
is
overflowed from m!
,
p"!^
oa
W.
,
i.e.,
(y/o).
,^
ya
6
is
violated since
THEOREM
will not find its corresponding
4
....
is violated since
t
after the overflow handling, there exists
but P
.
t
and
If P^
An overflow from
.
Also suppose P^ e
.
either P^, or
L
oa
r
We shall show that this leads to a contradiction.
.
The overflow page from
P^
always finds its
M"^
i
Mt.
r.t,
is no longer
in M
^
a
page
P^j^
g Pj in M^ (since pj 6 W^)
.
Q.E.D.
LEMMA 8.3
If there is no overflow from either M
or M
then X/"r,t,
V.
and V'
have the same reverse ordering.
Two stacks S
and S
are in the same reverse ordering, S
if rS^(k) = rS-^Ck) for
1
^
ki
min( (s^)
,\s^
\
),
rg
S'^
if S^=
or
S"^
=
,
where rS denotes
the stack obtained from S by re/ersing its ordering.
S^
CQ S
By convention,
.
34PROOr of LEMMA 8.3
To facilitate the proof, we introduce the following definitions.
(1) The o rdere d paren t stack
of the stack
(S^)"^,
,
the stack of
is
S^
parent pages corresponding to, and in the same ordering as, the
pages in the reduced stack,
and
(2)
1
S
(3)
S^
.
Formally,
(S
g S
)
2
to produce a new stack,
,
S^, where S{k) -
II
(
(||)» between two stacks,
S,as follows;
s\k)
for k-1 ,2,
S^k)
for k-
.
.
.
.
,
ls^l+1,
]
S^|
+ Is^lS
...,}|s^|
Define
a
new binary operator, ordere d di fference (o), between
and
a
set T, to produce
S
S =
T,
S^
Intuitively,
S
S
is
obtained from
which are also in
Figure
Is^OtI),
S
the
0-
by taking away those elements of
S
T.
ever referenced up to time
,
=
(j. )n"l'
illustrates the LRU ordering of all Level
11
or M'
|s^| -
minimum J|.>Jk_i such that
is the
j,
stack
a
new stack, S,as follows:
a
where S(k)=S^(j|^) for k=l,2,...,(
such that Jq=0,
M"^
S
new binary operator, concatenation
a
and S
S =
of
,
S^
(s"")-^
Define
S
i
pages
Since there is no overflow from either
t.
length of this LRU stack is less than or equal to min(m.
,
_t_..
Least Recently
[Lea
_r^Ref erenced
Recently
Referenced^!
^
By the definition of
But (S'j)J
=
hence V;
(Y^)^'
^
(SJ)J
o
Figure
t'
II
(
VJ.
(
,
=
VJ.
(Y^)"^'
(X^)J
(s[)^^
(
||
hence V^
=
=
((x^)\
(S'|)^^
),
(X^.)^
o
,
V
(S^)^
=
{Z
)
)
)^
c
=
(Y^^
o
((S^)JU(Xt)^')
(sM^
(XjJ'),
((Y^)J'
II
((X^j^o (sj)^)
o
{s\)^
Similarly, by the definition of V
But (Z^)J = (X.)J
11
||
({(Y^J^ o (X^)^) o (si)^)
(Y^)J
(sj.)J)
1
(
{(s[)JU(x^)^'))={(x^.)Jo(s^)^^')
II
1
Thus, the
two stacks
ai'e
in
the same reverse ordering.
Q.E.D.
v;
..)
.
LEMMA 8.4
y'r.t,
(a) ^'^.z> M^
,
and V' are either in the same reverse
(b) V.
ordering or the last element of
V'
is not an element of V.
PROOF of LEMMA 8.4
and
(a)
-
either
Level
(t))
or M'"^.
M'^
j
are true for any time before there is any overflow from
(a)
is
true because any page ever referenced is in
is also found
so a page found in M
,
in M''^.
is true
(b)
because of the result from LEMMA 8.3.
Assume that (a) and (b) is true for
Consider the next reference at t+1
t.
Suppose this reference does not produce any overflow from either M
or M'
then (a) still holds because M'J^f^t ^^^ M't"?
(See THEOREM 7).
^l
(b)
holds because overflows from M
still
the end of stacks V.
are taken from
and M
and V! respectively, and since there is no
overflow from Level j, (b)'s validity is not disturbed.
Suppose this reference does produce overflow(s) from Level j.
Case
1
:
overflow from M
no overflow from M
,
This cannot happen since overflow from M
which in turn implies overflow from M
.
Case
2
:
overflow from M
,
starting from the end of VI
,
implies reference to M
,
is not an element of V,
Then
This
X
Figure 12.
Figure 12.
x4
xsl
x7
x6
1*"^^^^J.
x7
x^J x5_x4
....
i
.
if we eliminate those elements not in
follows from LEMMA 8.3 and is illustrated in
j_
:
be in the same reverse ordering.
the two stacks will
I'*
also.
no overflow from M
t Suppose the last element in V'
V
1
:
x9
VJ.
x5
x3
_x3
x2
x2
xl
|
xi;
,
-36-
Thus we see that overflow from
of V
,
is
(b)
Suppose
will
still
V!
preserved since the last page in
3
:
V!
is
in V'
still
are in the same reverse ordering.
.
not in V
not in V
Then overflowing
.
overflow from \r and overflow from M
a Suppose the last element in V!
is not
in V
:
.
Refering to the diagram
in Case 2, we see the result of overflowing the last element of V!
and the last element of
V.
does not violate (a) and still preserves
the condition that the last element of V| is not in V
6
Suppose
V!
and V.
are in the same reverse ordering.
overflowing the last elements of
in the same reverse ordering,
page is overflowed from M
V!
leaves V| and V
is not violated
(a)
and M
and V
Then
.
PROOF of THEOREM 8
2
M
for the same reasons as those used in THEOREM
From LEMMA 8.4
Hence,
M'^
(mJuImJ)')
^
^
M"^
(^'j
U(M'^)'')
Q.E.D.
still
since the same
Q.E.D.
M'
.
does not violate (a) and results in the last
the last page of V
Case
i.e., overflowing the last page
,
not violate (a) since this page is still
and V
page of VI
M'^
7.
-37-
4
.
C onclusions
We have developed
a
model of
i\
designed for ^ery large databases.
data storage hierarchy system specifically
This data storage hierarchy makes
use of different page sizes across storage levels and maintains
multiple copies of the same informition in the hierarchy.
Four algorithms obtained from natural extensions to the LRU
algorithm are studied in detail
anc
key properties of these algorithms
that affect performance and reliability of the data storage hierarchy
are derived,
.
It is found that for the LOCAl -LRU algorithms,
no choice of sizes
for the storage levels can guarantee that a lower storage level
always contains all the information in the higher storage levels.
For the GLOBAL-LRU algorithms, by choosing appropriate sizes for the
storage levels, we can (1) ensure the above inclusion property to
hold at all times,
(2) guarantee tliat no extra page references to
lower storage levels are generated as
a
result of handling overflows,
and (3) guarantee that no multi-le\el paging anomaly can exist.
Several areas of further study emerge from this investigation.
These include the study of store-behind algorithms
[
18
]
and
the study of extensions to other known storage management algorithms.
We hope that this study motivates further work in the area of
generalized
data storage hierarchy systems for very large databases.
-30-
Acknowledgment
The authors would like to thank Mike Abraham, Sid Huff, and Ken Yip
for reviewing an earlier version of this paper; and the referees for
their editorial comments.
-39-
References
1.
Arora, S.R., and Gallo, A. Optimal sizing loading and reloading in
multi-level memory hierarchy system.
Proc. AFIPS 1971
SJCC
a
38, 337-344.
2.
A study of replacement algorithms for
Belady, L.A.
IBM Systems Journal 5, 2 (1966), 78-101.
computer.
3.
An anomaly in space-time
Belady, L.A., Nelson, R.A., and Shedler, G.S.
characteristics of certain programs running in a paging machine.
Comm. ACM 12, 6 (June 1969), 349-353.
4.
Optimal file allocation in multi-level
Chen, P.P.
Proc. AFIPS 1973 NCC, 277-282.
5.
Conti
6.
Denning, P.J.
153-190.
7.
Frenaszek, P. A., and Bennett, B.T.
Adaptive variation of the transfer
unit in a storage hierarchy.
IBM Journal of Research and Development 22, 4
(March 1978), 405-412.
Concepts for buffer storage.
C.J.
March 1969, 6-13.
,
Virtual memory.
a
virtual -storacie
storage systems.
IEEE Computer Group News,
ACM Computing Surveys 2,
3
(November 1974),
8.
Franklin, M.A., Graham, G.S., and Gupta, R.K.
Anomalies with variable
partition paging algorithms.
Comm. ACM 21, 3 (March 1978), 232-235.
9.
Greenberg, B.S., and Webber, S.H.
IEEE INTERCON 75.
10.
Hatfield, D.J.
Experiments on page size, program access patterns, and
virtual memory.
IBM Journal of Research and Development 16, 1 (January 1972),
58-66.
11.
Hatfield, D.J., and Gerald, J.
Program restructuring for virtual memory.
IBM Systems Journal 10, 3 (1971), 168-192.
12.
Hsiao, D.K., and Madnick, S.E.
Data base machine architecture in the
context of information technology evolution.
Proc. "^evy Large Data Base Conf.,
Tokyo, Japan, Oct. 1977. 63-84.
13.
Johnson,
C.
IBM 3850
-
MULTICS multilevel paging hierarchy.
mass storage system. IEEE INTERCON, 1975.
-AO-
14.
Proqram restructuring for virtual moniory systems.
Johnson, J.
MIT Project MAC TR-14a (March 1975).
15.
INFCPLEX data base computer architecture
Lam, C.Y., and Maclnick, S.E.
MIT Sloan School of Management Working Paper
concepts and directions.
No. 1046-79 (1979).
16.
Lum, V.Y., Senko, M.E., Wong, C.P., and Ling, H.
A cost oriented
algorithm for data set allocation in storage hierarchies.
Comm ACM 18, 6
(June 1975), 318-322.
17.
Madnick, S.E.
Storage hierarchy systems.
-
MIT Project MAC TR-105
(1973).
18.
INFOPLEX
Madnick, S.E.
management system using
hierarchical decomposition of a large information
inicroprocessor complex.
Proc. NCC 44
-
a
(May 1975), 581-587.
19.
Madnick, S.E.
Design of
a
general
hierarchical storage system.
IEEE INTERCON
75.
20.
The INFOPLEX database computer
Madnick, S.E.
concepts and
Proc. IEEE Computer Conference, February 26, 1979, 168-176.
21.
Mattson, R.L., Gecsei J., Slutz, D.R., and Traiger, I.L.
Evaluation techniques for storage hierarchies.
IBM Systems Journal 9, 2
(1970), 78-117.
22.
Ramamoorthy, C.V., and Chandy, K.M.
Optimization of memory hierarchies in
multi programmed systems.
Journal of the ACM 17, 3 (July 1970), 426-445.
:
directions.
,