Disk Storage
The collection of information that makes up files must be physically stored on some storage medium. The
information can then be retrieved, updated and processes as needed. There are two main categories of
computer storage media. These are:
Primary Storage – storage media that can be directly accessed by the central processing unit (CPU) such as
main memory (RAM) and cache memory. Primary storage provides fast access but is limited, due to the
expense.
Secondary Storage – there are various storage devices included in this category, such as magnetic disks,
optical disks, and tapes. These devices have a larger storage capacity than primary storage, but data
residing on secondary storage devices cannot be processed directly by the CPU. Before processing data in
secondary storage, it must first be transferred to primary storage.
The memory hierarchy refers to the hierarchy of storage media where data resides. The highest speed
memory is the most expensive, and is available with the least capacity, the lowest speed memory is the least
expensive and has an almost limitless capacity.
The primary storage level includes cache memory, which is used by the CPU to speed up execution of
programs, Dynamic RAM (DRAM), also referred to as main memory, which keeps programs and data being
used by the CPU. The secondary storage level includes magnetic disks, CDs, and tapes. The storage
capacity is measured in kilobytes, megabytes, gigabytes and terabytes.
Programs reside and execute in DRAM. Large databases reside in secondary storage and required portions
of the databases are read into and written from buffers in main memory as needed. There are cases where
entire databases are kept in main memory, with a backup copy on disk, these are called main memory
databases.
See the course textbook, Chapter 13 for more examples of secondary memory devices.
Storing Databases
Databases store large amounts of information that must be available for long periods of time. During this
period, the data is continually accesses and processed. Databases are generally stored permanently on
magnetic disk secondary storage devices. This is because databases are too large to fit entirely in main
memory, secondary storage is non-volatile, meaning that permanent loss of data is less common than volatile
storage such as main memory, and because the cost of storage is less for magnetic disks than for primary
storage.
Most database applications will only need a small portion of the database at once for processing. When a
portion of the database is needed, it is first located on the disk, copied to main memory for processing, then
rewritten to the disk if changes are made.
There are a number of different primary file organizations. These file organizations determine how the
records of a file are physically written to the disk, and how they can be accessed. A heap file (unordered file),
places records on disk by appending new records to the end of the file, and a sorted file (sequential file) keeps
the records ordered by the value of a particular field, called a sort key. We will go into more detail later in
Chapter 13.
Magnetic Disk Devices
The most basic unit of data stored on a magnetic disk is a bit. By magnetizing an area on a disk in a certain
way, it can be made to represent a bit value of either 0 or 1. These bits are grouped into bytes. Bytes are
typically 4 to 8 bits, depending on the device. Commonly it is assumed that one byte represents a single
character of information. The capacity of a disk is measured by the number of bytes it can store. Disk
capacities can be measured into the terabytes, and continue to grow as technology improves.
Disk Structures
Disks are made of magnetic material shaped as a thin, circular disk. Disks can be single-sided, meaning they
only store data on one surface, or double-sided, meaning data can be stored on both surfaces. Disks are
assembled into disk packs, which includes many disks and many surfaces.
Information is stored on disks in concentric circles, having small width. Each circle is called a track. In disk
packs, the tracks with the same diameter on the multiple surfaces are called a cylinder, because of the shape
it would form if connected in space.
track
spindle
read/write head
disk rotation
cylinder
of tracks
actuator




track – concentric circles on a disk surface which store information.
cylinder – tracks with the same diameter on various surfaces of a disk pack.
read/write head – reads or writes a block of data. For a disk pack, there is one read/write head for each
disk surface.
actuator – moves the read/write heads in unison and positions them over the cylinder of tracks specified
in a block address.
The number of tracks on a disk ranges from a few hundred to a few thousand. Each track has a capacity
ranging from 10 Kbytes to 150 Kbytes.
Tracks are divided into smaller blocks or sectors.
sector

sector – arc of a track, which divides a track into smaller blocks.
The division of a track into disk blocks is set by the operating system during formatting. The block size is
fixed, and cannot be changed dynamically. Blocks are separated by interblock gaps, which include specially
coded control information written during formatting. The information is used to determine which block follows
each interblock gap.
The transfer of data between main memory and the disk takes place in units of disk blocks. The hardware
address of a block (combination of a cylinder number, track number, and the block number) is supplied to the
disk I/O hardware.
The address of a buffer is also provided. A buffer is a reserved area in main storage that holds one block. For
a read command, the block from the disk is copied into the buffer, for a write command, the contents of the
buffer are copied into a disk block.
The disk drives for the hard disks rotate continuously at a constant speed. Once all the read/write heads
have been moved to a particular cylinder, and is positioned on the right track and the block specified in the
block address moves under the read write head, data transfer can begin.
Measuring Transfer Time
There are 3 main parameters used to measure the time it takes to transfer data from a disk to
memory, or from memory to the disk. These are:



seek time: the time it takes to move the arm to the correct cylinder.
rotational latency: the time it takes for the disk to revolve so the correct place on the track is
just under the head.
block transfer time: the time it takes to transfer the bytes of data (block) from the disk to the
memory, or from the memory to the disk.
The bits on the innermost track are stored closer together than the bits on the outermost track, so
the number of bits on each track is the same, and the block transfer time for a given block size is the
same no matter what track the block is on. Because the diameter of the innermost track are smaller
than the outer tracks, there must be a variation in the number of bits per centimetre on the different
tracks.
The time it takes to locate and transfer an arbitrary block, given its address is the seek time, plus the
rotational delay, plus the block transfer time.
Time to Locate and Transfer Block = seek_time(s) + rotational_delay(r) + block_transfer_time(btt)
The seek time and rotational delay are usually larger than the block transfer time, because
mechanical rather than electrical movement is involved.
Disks are random access secondary storage devices, because disk blocks can be accessed at
random once we know the address. Magnetic tapes are sequential access , because to access
the nth block on the tape we must first scan over the first n-1 blocks. For this reason tape access is
slow, and is not usually used to store online data, but is used mostly for backing up databases.
Sequential Reading of Blocks
If the blocks being read are adjacent, the time needed to read n blocks is the time for one average
seek, one average rotational latency and b block transfer times. This is because once the first block
is accessed, because the blocks are adjacent, the seek time and rotational latency to access the
subsequent blocks is 0.
Therefore: Sequential time = s + r + (b x btt)
Random Reading of Blocks
If the blocks being read are placed randomly on the disks, the time to read the blocks is the sum of
the average seek time, the rotational latency, and the block transfer time, multiplied by b (the
number of blocks to read).
Therefore: Random Time = b x (s + r + btt)
Example:
So let’s see an example. Assume that we want to read 100 blocks, with the parameter values
given.
s = 16 ms (milliseconds)
r = 8.3 ms
btt = 0.8 ms
b = 100
First assume that the blocks are placed sequentially on the disk drive, therefore the transfer time is:
Sequential Transfer Time = s + r + (b x btt)
= 16 + 8.3 + (100 x 0.8)
= 104.3 ms
Now let’s assume the blocks are placed randomly on the disks in the disk drive, meaning that we
must make a seek and allow for rotational latency for each block.
Random Transfer Time = b x (s + r + btt)
= 100 x (16 + 8.3 + 0.8)
= 2510 ms
From this example we can see that the random time for reading 10 blocks is slower than the
sequential time. This shows that it is much faster to read blocks which are stored together than to
read blocks that are randomly placed on the disk.
Buffering Blocks
When blocks are transferred to memory, they are held in buffers. If several blocks need to be
transferred from the disk to main memory, several buffers can be reserved in main memory to
speed up the transfer. While one buffer is being read/written, the CPU can process data in the
other buffer. This is possible because the block transfer is controlled by an independent disk I/O
processor, which can transfer data blocks between main memory and the disk, independent of, and
in parallel to the CPU processing data already stored in memory.
There are two ways this processing can proceed concurrently (at the same time), in and
interleaved fashion or in a parallel fashion. The diagram below illustrates the difference.
A
A
B
B
C
D
t1
t2
Interleaved concurrency
t3
Parallel
t4
If there is only a single CPU controls multiple processes, parallel execution is not possible.
However processes can still run concurrently, in an interleaved fashion. Buffering is most useful
when processes can run concurrently in a parallel fashion, where multiple CPUs exist, or there is a
separate I/O disk processor available.
Double buffering can occur when the time required to process a disk block in memory is less than
the time required to read the next block and fill the buffer. It can also be used to write a continuous
stream of blocks from memory to the disk. Double buffering eliminates the seek time and rotational
latency for all but the first block transfer. The CPU is only idle for the time it takes to transfer the
first block into memory.
Disk block: I/O
Fill A
Fill B
process A
Disk Block:
Processing
Fill A
Fill B
process B
process A
time
Placing Records on Disk
As discussed previously, a record is a collection of related data values or items called fields. The
fields along with their data types makes up a record type or record format definition. A data type
specifies the values a field can take. A file is a sequence of records. Of every record in the file has
the same size in bytes, the file is made up of fixed length records. If the records in the file have
different sizes, it is made up of variable length records.
Variable length records may occur if: (a) one or more fields in the record are variable length, (b)
one or more fields may have multiple values for individual records (repeating field), (c) one or more
fields in the record are optional, or (d) the file contains records of different record types (mixed file).
It is important for a system to be able to identify the starting position of a field relative to the starting
position of a record. For example, where within a record does the field “Salary” start. In fixed length
record files this is not difficult, however if the records are variable length, there are different formats
for storing the records.
It is possible to represent variable length records as fixed length records. For example if there is a
field that is optional within a record, each record can contain that field, but may have a special null
character to indicate there is no value for that field in that particular record. Although this may seem
logical, it results in a waste of space when records do not have values for all physical spaces
provided.
Fixed Length Records
For the following example, let’s assume that we have a file with 4 fields, (1) Name, (2) Kimlik No,
(3)Salary, (4) Department. With fixed length, this file may look as follows:
1
Name
21
Kimlik No.
34
Salary
40
Department
In this example, the beginning of each field is located in exactly the same location relative to the
beginning of the record. Name is 20 characters, Kimlik No. is 13 characters, Salary is 6 characters
and Department is 10 characters. Therefore the fixed record length is 49 bytes.
Variable Length Records
Now let’s assume that the fields in the file vary as follows:
(1) Name (variable length), (2) Kimlik No, (3)Salary(optional), (4) Department (optional).
Strategies for storing variable length records:
a) If each record has a value for every field, but some fields have variable lengths, a separator
character (eg. %, ?, $), which will not appear in any field value, can be used to terminate the
variable length field.
Zafer, Can
123456789
55,000
Payroll
Separator
b) If some fields are optional, the strategy used depends on whether the total number of fields for
the record type is large, but the fields that actually appear in a typical record is small. If this is
true, then each record can include a sequence of field names and field value pairs rather than
just the field values. In this case, three types of separators can be used, one to separate the
field name from the field value, one to indicate the end of a field, and one to indicate the end of
the record.
Name= Zafer, Can
Kimlik=123456789
Record Separator
Salary=55,000
Department=Payroll
Field Separator = Field Name/Value Separator
Optional fields can also be stored using a field type code, and using a combination of field type
code/field value pairs.
c) If there are repeating fields in the record, one separator is required to separate the repeating
values of the field, and another to indicate the end of a field. For a file that includes records of
different types, each record is preceded by a record type indicator.
Record Blocking
As we have seen previously, when operations need to be performed on records, the block
containing the record is the unit of data transferred between the main memory and the disk.
Because the block is the unit of data transferred, records must be allocated to disk blocks. When
the block size is larger than the record size, the block will contain multiple records. We will use the
following parameters in this discussion:
B: Block size
R: Fixed length record size in bytes.
If B (size of the block) is greater that R (size of the fixed length record) then we know that we can fit
B/R records per block. This is called the blocking factor (bfr).
Blocking factor: bfr = B/R
In the equation, you will notice that the expression B/R is surrounded by the floor function symbols,
 . The floor function rounds down the result, x, to an integer. For example if we had the
expression, 10/3, without the floor function, the result is 3.33333, but using the floor function, the
result would be 3. It is important to note that the floor function always rounds down. So even
though in the expression, 11/3, 11/3 is equal to 3.666667, using the floor function the result is 3.
If the record size does not divide the block size exactly, the remainder is the unused space in the
block. To calculate unused space in the block, the formula is as follows:
Unused Block Space = B – (bfr * R) bytes
Which is the total block size B, minus the record size multiplied by the number of records in a block.
For example, if the block size is 512 bytes, and the record size is 50 bytes, what is the unused block
space?
B = 512 bytes
R = 50 bytes
First we need to calculate the blocking factor: bfr = B/R = 512/50 = 10
bfr = 10
Then we can calculate the unused space:
Unused space = 512 – (10 * 50) = 12 bytes
This unused space can be used by storing part of the record in one block, and the remaining portion
that does not fit in the next block. This is called record spanning. Records can be spanned or
unspanned. In record spanning, a pointer at the end of the first block points to the block containing
the remainder of the record. In cases where a record is larger than a single block, record spanning
is necessary.
Unspanned organisation is used in cases where B > R, and the records are fixed length, because
the record start and end locations are known, and it simplifies the processing. For variable length
records, either spanned or unspanned organisation can be used.
In order to calculate the number of blocks required, b, for a file containing r records we can use the
following formula:
b = r / bfr  blocks
This formula uses the ceiling function, which is the opposite of the floor function. The ceiling
function rounds up to the nearest integer, x. For example the function 10/3 , without using the
ceiling is 3.33333, using the ceiling function the result is 4. The ceiling function is used because if
we are trying to determine the number of blocks required, and the number of records, r, is 10 and
the number of records per block (bfr) is 3, 4 blocks are required to store the records, 3 blocks would
not be enough to store the records, and partial blocks cannot be used.
Example Questions:
13.23
13.24
13.25
13.26
13.38
Allocating File Blocks on Disk
There are a number of standard techniques for allocating blocks of a file on disk, these are:
a) Contiguous allocation – file blocks are allocated to consecutive dick blocks. Makes reading the
file fast using double buffering, but expanding the file difficult.
b) Linked allocation – each file block contains a pointer to the next file block. Makes expansion of
the file easy, but slow to read the whole file.
c) Clusters – a combination of the previous two. Allocates clusters of consecutive disk blocks and
then links the clusters. Clusters are sometimes called extents or file segments.
d) Indexed allocation – One or more index blocks contain pointers to the actual file blocks.
File Headers
File Header – contains information about a file needed by system programs to access file records.
Includes information on:
- Disk addresses of file blocks
- Record format descriptions (may include field lengths, field order, field type codes, separator
characters and record type codes for variable length records)
To search for a record on a disk, one or more blocks are copied into the memory buffers. The
application programs then search for the desired record using information contained within the file
header. If the address of a block is not known, the application program must do a linear search
through the file blocks. Each block is loaded into memory until the record is found, or all blocks
have been searched.
File Operations
Two categories of operations:
Retrieval Operations: Do not change any data in the file but locate certain records so their field
values can be examined and processed.
Update Operations: Change the file by insertion or deletion of records, or modification of fields
within a record.
For either operation, you must specify a select condition, to identify the record to be processed.
The select condition specifies the criteria the desired record(s) must satisfy. When several records
satisfy a search condition, the first record (with respect to the physical sequence of file records) is
located and designated the current record. The actual operations for locating and accessing file
records vary from system to system.
A file organisation refers to the organisation of the data of a file into records, blocks and acces
structures. This includes the way records and blocks are placed on the storage medium and linked.
Access methods are groups of operations that can be applied to a file. In general, several access
methods can be applied to a file organisation.
File Organisations
An important factor in deciding the primary file organisation is how the information will be accessed.
Primary file organisations are important to ensure the most efficient access of files and records.
Files of Unordered Records (Heap Files)
The most basic type of organisation is when records are placed in the file in the order in which they
are inserted, so new records are placed at the end of the file. This is called a heap file, or a pile
file.
Inserting Records
With this organisation, inserting a new record into the file is very efficient. The last disk block of the
file is copied into a buffer, the new record is added, and the block is rewritten to disk. The address
of the last file block is kept in the file header.
Searching Records
However, searching for a record using any search condition requires a linear search through the file
block by block, which is an procedure, meaning it requires a lot of time and system resources.
With a file of b blocks, if only one record meets the search criteria, on average, a program will need
to read into memory and search half the file blocks before finding the correct record. If no records
exist that meet the search criteria, b blocks will have to be searched.
Therefore,
Number of searches required to locate a record in a file with b blocks = b/2.
Deleting Records
To delete a record, the program must 1) find the block containing the record, 2) copy the block to
the buffer, 3) delete the record from the buffer, and 4) rewrite the modified buffer to disk. As a result
of this, unused space is left in the disk block. Deleting a large number of records results in wasted
storage space.
Another method for deleting records is to have an extra byte stored with the record called a deletion
marker. To delete a record, the deletion marker is set to a specific value. Searching algorithms will
only consider records that are valid (deletion marker not set to deleted value). With both of these
strategies, reorganisation is needed periodically to clean up unused space.
An unordered file can have either spanned, or unspanned organization, and may either have fixed
length or variable length records. If the records are variable length, modifying a record may require
deleting the old record and adding a new record, because the modified record may not fit in the
space occupied by the old record.
For a file with the following characteristics: 1) unordered, 2) fixed length records, 3) unspanned, and
4) contiguous allocation, accessing records is straightforward. The file can be accessed by its
position in the file.
For example, assume the file records are numbered 0, 1, 2, 3….r-1, and the records in each block
are numbered 0, 1, 2, 3…..bfr – 1, where bfr is the blocking factor, then the ith record in the file is
located in the block i/bfr and it is the (i mod bfr)th record in the block. See the image below.
position in file
block 0
block 1
block 2
.
.
.
position in block
0
0
1
1
2
2
3
3
4
0
5
1
6
2
7
3
8
0
9
1
10
2
From the figure above we can see bfr = 4. If we want to locate the 10th record of the file, it is located
in block 10/4 = block 2. The location of the file within block 2 is at the (I mod bfr)th position, which
is the (10 mod 4) = 2nd position.
These files are often called relative or direct files because records can be easily accessed directly
by their positions.
Files of Ordered Records (Sorted Files)
A sorted file is one that is physically ordered based on the values of one of their fields. The field
used to order the file is called the ordering field. If the ordering field is also a key field, then the
field is called the ordering key for the file.
Advantages of ordered records over unordered files:
- Reading records in order of ordering key is efficient because no sorting is required.
- Finding the next record from current record, in order of the key requires no additional block
accesses because it will be found in the same block as the current record (unless the current
record is the last in the block)
- Using a search condition based on the value of an ordering key field results in faster access
because the binary search technique can be used. A binary search usually accesses log2(b)
blocks, whether the record is found or not, whereas a linear search on average accesses
(b/2) blocks if the record is found, and b blocks if the record is not found. As you can see the
binary search is much faster.
Searching Records
Search criteria involving the conditions (>, <, >=, <=) on the ordering field are efficient because all
records satisfying the condition are contiguous in the file. However ordering is not an advantage
for random access, or ordered access of records based on values other than the ordering field. For
example, if a file is ordered by Last_Name, and I wish to find all records whose data of birth is after
August 1, 1980, the ordering would not provide any advantage over non-ordered files.
Inserting and Deleting Records
With an ordered file, inserting and deleting are expensive operations because the file must remain
physically ordered. To insert a record, we must find the correct position in the file, based on the
ordering key, then make space to insert the record into that position. For large files this can be very
time consuming, because on average half the records must be moved to make space for the new
record. This means that half the file blocks must be read and rewritten after records are moved
among them.
To make inserting records more efficient, one method that is used is to keep some unused space in
each block for new records, however once full, it is back to the original problem. Another option is
to create a temporary unordered file called a transaction or overflow file. The ordered file is called
the main or master file. New records are inserted at the end of the overflow file rather than the main
file. Periodically the overflow file is sorted and merged with the main file. Although insertion
becomes very efficient, searching becomes more complex. The master file must be searched using
binary search, and if the records are not found, the overflow file must be searched using a linear
search.
For deleting records, if deletion markers are used, the problem is less severe, and periodic
reorganisation is used.
Hashing Techniques
Another type of file organisation is based on hashing, which provides fast access to records using
certain search conditions. This organisation is called a hash file. In most cases the hash field is a
key field of the file, called the hash key.
Hashing uses a hash function, h, that is applied to the hash field value of a record and gives the
address of the disk block in which the record is stored. (ie. to determine where to store a record,
apply the hash function to the key). A search for a record within the block can be done in the main
memory buffer. To access most records, only a single block transfer is needed.
Internal Hashing
For internal files, hashing is implemented as a hash table, using an array of records. Imagine that
the array index ranges from 0 to M-1. There are M slots whose addresses correspond to the array
indexes.
A hash function is chosen to transform the hash field value into an integer between 0 and M-1. A
common hash function is h(K) = K mod M, which returns the remainder of an integer hash field
value K after dividing by M. The resulting value is then used for the record address.
If the hash field is non integer, it can be transformed to integers before the mod function is applied
using the ASCII value of the character.
Other Hashing Functions
- Folding involves applying an arithmetic function such as addition or logical function such as
exclusive or (Xor) to different portions of the hash field value to calculate the hash address.
- Another technique involves picking some digits of the hash field value (ie. Third, fifth, eighth
digits) to form the hash address.
The difficulty with most hashing functions is that they don’t guarantee that distinct values will hash to
distinct addresses. (ie. Many values can hash to the same address). This is because the hash field
space (number of possible values that a hash field can take) is much larger than the address space
(the number of available addresses for records). The hashing function maps the hash field space to
the address space.
When the hash field value of a record being inserted hashes to an address space already
containing a different record, a collision occurs. There must be some strategy in place for
resolving this collision.
There are a number of collision resolution mechanisms, including the following:
Open Addressing/Linear Probing – if the hash address is occupied, the program checks the next
positions in order until an empty address is found.
Chaining – Overflow locations are used, by extending the array with overflow positions. Each
record location has a pointer field, and if the hash address is occupied, the collision is resolved by
placing the new record in an unused overflow location, and setting the pointer of the occupied hash
address location to the address of the overflow location. A linked list of the overflow records for
each hash address is maintained.
Multiple hashing – If the first hash function results in a collision, a second hash function is applied.
If there is a collision after the second hash function, either a third hash function is applied, or open
addressing is used.
Hashing Example with Open Addressing
Hash Function = K mod M, where K is the field value, and M is the size of the address space. This
will result in the range of values of the hash function to match the address spaces.
M=9
h(k) = K mod M
K
30
45
24
25
36
54
h(K)
3
0
6
7
0
0
Resulting Array:
M
0
1
2
3
4
5
6
7
8
K
45
36
54
30
24
25
Primary Clustering
In this example, as we can see, any K value which is a multiple of 9 hashes to the value 0, and
produces a collision. Using open addressing, when an address is occupied, the program checks
the next position, until it finds an unoccupied position. Because of a collision with any multiple of 9,
the values tend to cluster around the collision value. This is referred to as a primary cluster.
Average Probes
K
h(K)
#
Probes
M
K
45
36
54
30
63
24
25
0
0
0
3
0
6
7
1
2
3
1
5
1
1
0
1
2
3
4
5
6
7
8
45
36
54
30
63
24
25
To determine the average number of probes, count each time a position is probed. If a value
hashes to an empty position the number of probes is 1, if the position is full, the number of probes is
1 plus each additional probe used to find a vacant space.
Once you know the number of probes, sum the total probes, and divide by the number of K values,
which will give you the average probes.
Average Probes = Total Probes / Total Inserts
Therefore, in the example above, the average probes is: 14/7 = 2 probes. This value means that
there are two searches on average to find the next available position.
Hashing with Chaining
K
45
47
36
5
14
23
M
0
1
2
3
4
5
6
7
8
h(K)
0
2
0
5
5
5
K
45
47
36
-
-
5
14
23
-
-
Hashing with Overflow Area
0
1
2
3
4
45
Overflow
Pointer
9
47
Original Table
5
6
7
8
9
10
11
12
13
5
10
36
14
23
11
Overflow Area
Each collision resolution mechanism requires its own algorithms for insertion, retrieval and deletion
of records. The algorithms for chaining are the simplest. Deletion algorithms for open addressing
are more complex.
The ultimate goal of a good hashing function is to distribute records uniformly over the address
space so as to minimise collisions, while not leaving many unused locations.
Load Factor
The load factor, represented by , measures how full the table is. The formula for calculating the
load factor is:
 = # of entries/Total Entries (M)
To minimise the number of collisions, the goal value for the load factor is 70%.
Example Questions:
13.27
External Hashing for Disk Files
-
Hashing for disk files is called external hashing. The target address space is made up of
buckets (blocks). Each bucket holds multiple records. A bucket can be either one disk block, or
multiple contiguous blocks.
-
The hashing function maps a key into a relative bucket number, rather than assign an absolute
block address to the bucket. A table maintained in the file header converts the bucket number
into the corresponding block address. See Figure below for example.
bucket #
Block
address
on disk
0
1
2
= bucket
.
.
.
M-2
M-1
…
Because many records will fit into a bucket, the collision problem is less severe, because many
records can hash to the same bucket. In the case of a bucket becoming full and new records
hashing to a full bucket, collision resolving mechanisms again must be used.
A variation of the chaining mechanism can be used, where every bucket maintains a pointer to a
linked list of overflow records for that bucket. The pointers are record pointers which include the
block address and the relative position within the block.
This hashing scheme is called static hashing. This is because a fixed number of buckets, M, is
allocated for the address space. If m is the maximum number of records that can fit into a bucket,
the total capacity allocated for all buckets is (M * m). If there are significantly fewer than (M * m)
records, then there is a lot of unused record space. Conversely, if there are more than (M*m)
records numerous collisions will result, and performance will be affected because of the long lists of
overflow records.
-
Searching for a record when using external hashing is expensive if the search field is not the
hash field.
-
Record deletion can be implemented by removing the record from its bucket. If the bucket has
an overflow chain, the overflow records can be moved into the bucket to replace the deleted
record.
Dynamic File Expansion
As seen before, a major drawback of static hashing is that address space is fixed. Newer dynamic
file organisations based on hashing allow the number of buckets to vary dynamically with only
localised reorganisation.
Extendible Hashing
One example of dynamic file expansion is called extendible hashing. These hashing schemes are
based on binary representation of a number. By applying a hashing function to a non negative
integer, you can represent the result as a binary number. This binary number is a string of bits
which is called the hash value of a record. Records are distributed among buckets based on the
values of the leading bits in their hash values.
In extendible hashing, a directory of 2d bucket addresses in maintained, where d is the global
depth of the directory. The integer value corresponding to the first d bits of a hash value is used as
an index to the array to determine a directory entry, and the address in the directory entry
determines the bucket in which the corresponding records are stored.
A local depth, d1 stored with each bucket specifies the number of bits on which the bucket contents
are based. The value d can be increased or decreased by one at a time, doubling or halving the
number of entries in the directory.
Doubling is needed if the bucket whose local depth matches the global depth overflows, halving
may occur after some deletions (d > d1).
Example:
0
0
0
0
0
0
0
0
27
26
25
24
23
22
21
20
128
64
32
16
8
4
2
1
The hashing function is: h(k) = K mod 32, therefore we need a directory with at most 32 elements,
(5 bits), 25 = 32.
K
2657
3760
4692
4871
5659
1821
1074
7115
1620
2428
3943
4750
6975
4981
9208
h(K)
1
16
20
7
27
29
18
11
20
28
7
14
31
21
24
Binary h(k)
00001
10000
10100
00111
11011
11101
10010
01011
10100
11100
00111
01110
11111
10101
10111
Go to PowerPoint Presentation for the Extendible Hashing Example, using these values.
Linear Hashing
Linear hashing allows a hash file to expand and shrink its number of buckets dynamically without
needing a directory.
The file starts with M buckets, numbered 0 to M-1, and uses the initial hash function h(K)= K mod M.
Overflow because of collisions is needed, and can be handled using an overflow chain for each
bucket.
When a collision leads to an overflow record in ANY file bucket, the first bucket is split into two
buckets, the original bucket 0, and a new bucket M at the end of the file. The records in bucket 0
are distributed between the two buckets based on a different hashing function, hi+1(K) = K mod M.
An important property of the two hash functions is that any records that hashed to bucket 0 based
on hi will hash to either bucket 0 or bucket M based on hi+1.
As further collisions lead to overflows, additional buckets are split in linear order (first split bucket 0,
then bucket 1, then bucket 2, etc). If enough overflow occurs, all the original buckets will have been
split, so the file will have 2M buckets instead of M buckets, and all buckets use the hash function,
hi+1.
The records in the overflow are eventually redistributed to regular buckets using the hash function
hi+1 via a delayed split of their buckets. There is no directory, only a value n which is initially set to 0
and is incremented by 1 whenever a split occurs.
Class Example:
hi(K) = K mod 5
hi+1(K) = K mod 10
Using Linear Hashing, assign the following records to buckets, using the hash functions above.
K h(K)
16
1
17
2
30
0
49
4
25
0
21
1
48
3
32
2
43
3
54
4
77
2
79
4
26
1
103
3
Example Questions:
13.28
13.29