ANALYSIS OF ALGORITHM
Chapter One: Introduction and elementary data structures
INTRODUCTION
What is an Algorithm?
An Algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition,
all algorithms should satisfy the following criteria.
1. INPUT Zero or more quantities are externally supplied.
2. OUTPUT At least one quantity is produced.
3. DEFINITENESS Each instruction is clear and unambiguous.
4. FINITENESS If we trace out the instructions of an algorithm, then for all cases, the algorithm
terminates after a finite number of steps.
5. EFFECTIVENESS Every instruction must very basic so that it can be carried out, in principle, by
a person using only pencil & paper.
Issues or study of Algorithm:
How to device or design an algorithm creating an algorithm.
How to express an algorithm definiteness.
How to analysis an algorithm time and space complexity.
How to validate an algorithm fitness.
Testing the algorithm checking for error.
Algorithm Specification:
Algorithm can be described in two ways.
1. Flowchart:
 Graphic representation of an algorithm is called as flowchart
 Easy to understand the logic of complicated and lengthy problems
Algorithm to find the average of three numbers
Start
READ
XYZ
COMPUTE
S=X+Y+Z
AVERAGEE
A=S/3
WRITE
XYZ
Stop
Chapter 1 – Introduction and elementary data structures
Page 1
2. Pseudo-code Method:
Pseudo code is a type of structured English that is used to specify an algorithm. This cannot be
compiled and not executed.
In this method, we should typically describe algorithms as program, which resembles language
like Pascal & algol.
Pseudo-code to find the average of three numbers
START
READ values of X, Y, Z
COMPUTE S=X+Y+Z
COMPUTE A=S/3
WRITE the value of A
STOP
ANALYSIS OF ALGORITHM
Performance Analysis
1. Space Complexity
2. Time Complexity
Space Complexity
The space complexity of an algorithm is the amount of memory it needs to run to compilation.
The Space needed by each of these algorithms is seen to be the sum of the following
component.
1. A fixed part that is independent of the characteristics (eg: number, size) of the inputs and
outputs.
The part typically includes the instruction space (ie. Space for the code), space for simple
variable and fixed-size component variables (also called aggregate) space for constants, and so
on.
2. A variable part that consists of the space needed by component variables whose size is
dependent on the particular problem instance being solved, the space needed by referenced
variables (to the extent that is depends on instance characteristics), and the recursion stack
space.
Example
Algorithm sum(a,n)
{
S:=0.0;
for i :=1 to n do
s:= s+a[i];
return s;
}
The problem instances for this algorithm are characterized by n,the number of elements to be summed.
The space needed d by ‘n’ is one word.
Chapter 1 – Introduction and elementary data structures
Page 2
The space needed by ‘a’ is the space needed by variables of type array of floating point
numbers. This is at least ‘n’ words.
So, we obtain Ssum(n)>=(n+3) [ n for a[],one each for n, i & s]
Time Complexity
The time complexity of an algorithm is the amount of computer time it needs to run to
compilation.
The time T(p) taken by a program P is the sum of the compile time and the run time(execution
time) The compile time does not depend on the instance characteristics. Also we may assume
that a compiled program will be run several times without recompilation .This rum time is
denoted by tp(instance characteristics).
The number of steps any problem statement is assigned depends on the kind of statement.
For example, comments 0 steps.
Assignment statements  1 steps.
Interactive statement such as for, while & repeat-untilControl part of the statement.
We can determine the number of steps needed by a program to solve a particular problem
instances in two ways.
First, we introduce a variable, count into the program statement to increment count with initial
value 0.Statement to increment count by the appropriate amount are introduced into the
program. This is done so that each time a statement in the original program is executed count is
incremented by the step count of that statement.
Algorithm sum(a,n)
{
s= 0.0;
count = count+1; // for assignment statement
for i=1 to n do
{
count =count+1;
// For f
s=s+a[i];
count=count+1;
// for assignment statement
}
count=count+1;
// for last time of for
count=count+1;
// for return
return s;
}
If the count is zero to start with, then it will be 2n+3 on termination. So each invocation of sum
executes a total of 2n+3 steps.
The second method to determine the step count of an algorithm is to build a table in which we
list the total number of steps contributes by each statement.
Chapter 1 – Introduction and elementary data structures
Page 3
First determine the number of steps per execution (s/e) of the statement and the total number
of times (ie., frequency) each statement is executed.
By combining these two quantities, the total contribution of all statements, the step count for
the entire algorithm is obtained.
Statement
1. Algorithm Sum(a,n)
2.{
3. S=0.0;
4. for I=1 to n do
5. s=s+a[I];
6. return s;
7. }
Total
S/e
0
0
1
1
1
1
0
Frequency
1
n+1
n
1
-
Total
0
0
1
n+1
n
1
0
2n+3
Best, Worst, Average Cases
 Best-Case Step count
o Minimum number of steps executed by the algorithm for the given parameters.
 Worst-Case Step count
o Maximum number of steps executed by the algorithm for the given parameters.
 Average-Case Step count
o Average number of steps executed by an algorithm.
Inexactness of step count
 Both the instructions x=y and x=y+z+(x/y) count as one step.
 Because of the inexactness of what a step count stands for, the exact step count is not very
useful for comparison of algorithms.
Asymptotic efficiency
 Asymptotic efficiency means study of algorithms efficiency for large inputs
 To compare two algorithms with running times f(n)and g(n),we need a rough measure that
characterizes how fast each function grows as n grows.
 Hint:use rate of growth
 Compare functions asymptotically!
(i.e., for large values of n)
Rate of Growth
Ex:-F(n)=n2+100n+log10n+1000
The low order terms and constants in a function are relatively insignificant for large n
n2+ 100n+ log10n+ 1000 ~ n2
i.e., we say that n2+ 100n+ log10n+ 1000 and n2 have the same rate of growth
Chapter 1 – Introduction and elementary data structures
Page 4
f(n)
n2
100n
log10n
value
Value
value
value
1
1,101
1
10
2,101
100
100
21,002
10,000
1,000
1,101,003
1,000,000
10,000
101,001,004
100,000,000
1,00000
10,010,001005
10,000,000,000
n
%
0.1
4.76
47.6
90.8
99.0
99.9
100
1,000
10,000
100,000
1,000,000
10,000,000
%
9.1
47.6
47.6
9.1
0.99
0.099
0
1
2
3
4
5
1000
%
0.0
0.05
0.001
0.0003
0.0
0.0
value
1,000
1,000
1,000
1,000
1,000
1,000
%
90.83
47.60
4.76
0.09
0.001
0.00
Some more examples
n4+ 100n2+ 10n+ 50 is ~ n4
10n3+ 2n2is ~ n3
n3-n2is ~ n3
constants
–10 is ~1
–1273 is ~1
ASYMPTOTIC/ORDER NOTATIONS
 Asymptotic/order notation describes the behavior of functions for the large inputs.
 Big Oh(O)notation:
–The big oh notation describes an upper bound on the asymptotic growth rate of the
function f.
Definition: [Big “oh’’]
–f(n) = O(g(n)) (read as “f of n is big oh of g of n”) iff there exist positive constants c and
n0such that f(n) <cg(n) for all n, n >n0.
 The definition states that the function f(n) is at most c times the function g(n) except when
n is smaller than n0.
 In other words, f(n) grows slower than or same rate as” g(n).
Chapter 1 – Introduction and elementary data structures
Page 5
Examples
–f(n) = 3n+2
3n + 2 <= 4n, for all n >= 2, 3n + 2 = O(n)
–f(n) = 10n2+4n+2
10n2+4n+2 <= 11n2, for all n >= 5, 10n2+4n+2 = O(n2)
Omega(Ω)notation:
–The omega notation describes a lower bound on the asymptotic growth rate of the function f.
Definition:[Omega]
–f(n) = Ω(g(n)) (read as “f of n is omega of g of n”) iff there exist positive constants c and n0
such that f(n)>cg(n) for all n, n>n0.
The definition states that the function f(n) is at least c times the function g(n) except when n is
smaller than n0.
In other words, f(n)grows faster than or same rate as g(n).
Examples
–f(n) = 3n+2
3n + 2 >= 3n, for all n >= 1, 3n + 2 = Ω (n)
–f(n) = 10n2+4n+2
10n2+4n+2 >= n2, for all n >= 1, 10n2+4n+2 = Ω (n2)
Theta(Θ)notation:
–The Theta notation describes a tight bound on the asymptotic growth rate of the function f.
Definition:[Theta]
–f(n) = Θ(g(n)) (read as “f of n is theta of g of n”) iff there exist positive constants c1, c2,and
n0 such that c1g(n) <f(n) <c2g(n)for all n, n>n0.
Chapter 1 – Introduction and elementary data structures
Page 6
The definition states that the function f(n) lies between c1 times the function g(n) and c2 times
the function g(n) except when n is smaller than n0.
In other words, f(n)grows same rate as g(n).
Examples:–f(n) = 3n+2
•3n <= 3n + 2 <= 4n, for all n >= 2, 3n + 2 = Θ(n)
–f(n) = 10n2+4n+2
•n2<= 10n2+4n+2 <= 11n2, for all n >= 5, 10n2+4n+2 = Θ(n2)
1 < logn < n < nlogn < n2 < n3 < 2n < n!
To get a feel for how the various functions grow with n, you are advised to study the
following figs:
Chapter 1 – Introduction and elementary data structures
Page 7
HASHING
A symbol table is a data structure which allows one to easily determine the presence or absence
of an arbitrary element. It also permits easy insertion and deletion of elements.
Hashing differs from this approach in that the address or location of an identifier, X, is
obtained by computing some arithmetic function, f, of X. f(.X) gives the address where X
should be placed in the table. This address will be referred to as the hash address of X.
The memory available to maintain the symbol table is assumed to be sequential. This memory
is referred to as the hash table, abbreviated HT. The hash table is partitioned into b buckets,
HT(O), ... , HT(b - 1). Each bucket is capable of holding s identifiers. Thus, a bucket is said to
consist of s slots, each slot being large enough to hold 1 identifier.
F(X) maps the set of possible identifiers onto the integers 0 through b - 1. Let T be the size of
the space from which the identifiers are drawn. This space is called the identifier space.
For a table containing n identifiers, the ration/Tis the identifier density, while a = nl(sb) is the
loading density or loading factor.
The hash function f will certainly map several different identifiers into the same bucket. Two
identifiers I 1, I 2 are said to be synonyms with respect to f if f(l 1) = f(l 2).
An overflow is said to occur when a new identifier I is mapped or hashed by f into a full
bucket. A collision occurs when two non identical identifiers are hashed into the same bucket.
When the bucket size s is 1, collisions and overflows occur simultaneously.
As an example, let us consider the hash table HT with b = 26 buckets, each bucket having
exactly two slots, i.e., s = 2. Assume that there are n = 10 distinct identifiers in the program
and that each identifier begins with a· letter. The loading factor, a, for this table is 10/52 = 0.19
The identifiers GA, D, A, G, L, A2, Al, A3, A4 and E will be hashed into buckets 7, 4, l, 7, 12,
l, l, l, 1 and 5 respectively by this function. The identifiers A, Al, A2, A3 and A4 are
synonyms. So also are G and GA.
Figure 2.25 shows the identifiers GA, D, A, G, and A2 entered into the hash table. Note that
GA and G are in the same bucket and each bucket has two slots. Similarly, the synonyms A
and A2 are in the same bucket gets hashed into HT(l2). The next identifier, Al, hashes into the
bucket HT(l). This bucket is full and a search of the bucket indicates that Al is not in the
bucket. An overflow has now occurred.
Chapter 1 – Introduction and elementary data structures
Page 8
Hash Function
A hashing functions, f, transforms an identifier X into a bucket address in the hash table.
One simple and effective choice for a hash function is obtained by using the modulo (mod)
operator. The identifier X is interpreted as an integer and it is divided by some number M and
the remainder is used as the hash address for X.
f(X) = X mod M
This gives bucket addresses in the range 0 to M - 1 and so the hash table is at least of size b =
M.
Overflow Handling
All slots may be initialized to zero.
When a new identifier gets hashed into a bucket already occupied, it is necessary to find
another bucket for this identifier. The simplest solution would probably be to find the closest
unfilled bucket.
Assume the identifiers are GA, D, A, G, L, A2, Al, A3, A4, Z, ZA, E. For simplicity we
choose the hash function fiX) = first character of X.
Initially, all the entries in the table are zero. fiGA) = 7, this bucket is empty, so GA (and any
other information about this identifier) are entered into HT(7). D and A get entered into the
buckets HT(4) and HT(l) respectively. The next identifier G hasfiG) = 7. This slot is already
used by GA. The next vacant slot is HT(8) and so G is entered there. Lenters at HT(l2). A2
collides with A at HT(l), the bucket overflows and A2 is entered at the next vacant slot HT(2).
Al, A3 and A4 are entered at HT(3), HT(S) and HT(6) respectively. Z is entered at HT(26), ZA
at HT(9), (the hash table is used circularly), and E collides with A3 at HT(S) and is eventually
entered at HT(lO).
This method of resolving overflows is known as linear probing or linear open addressing.
procedure LINSRCH(X, HT, b, j)
//search the hash table HT(O:b - 1) (each bucket has exactly 1//
//slot) using linear probing. If HT(j) = 0 then the j-th bucket//
//is empty and X can be entered into the table. Otherwise//
//HT(j) = X and X is already in the table. f is the hash function//
i f(X); j i
while HT(j) ≠ X and HT(j) ≠ 0 do
j (j + 1) mod b //treat the table as circular//
if j = i then call TABLE-FULL endif //no empty slots//
repeat
end LINSRCH
To locate the identifier, ZA, in the table of Figure 2.27, it was necessary to examine HT(26),
HT(l), ... , HT(9), a total of ten comparisons. This is far worse than the worst case behavior for
tree tables. If each of the identifiers in the table of Figure 2.27 was retrieved exactly once, then
the number of buckets examined would be 1 for A, 2 for A2, 3 for Al, 1 for D, 5 for A3, 6 for
A4, 1 for GA, 2 for G, 10 for ZA, 6 for E, 1 for L and 1 for Z for a total of 39 buckets
examined. The average number examined is 3.25 buckets per identifier.
Chapter 1 – Introduction and elementary data structures
Page 9
Hashing with chaining
In the hash table of Figure 2.25, for instance, searching for the identifier ZA involved
comparisons with the buckets HT(l) to HT(8), even though none of the identifiers in these
buckets had a collision with HT(26) and so could not possibly be ZA.
Many of these comparisons could be avoided if we maintained lists of identifiers, one list per
bucket, where each list contains only the synonyms for that bucket.
Since the sizes of these lists is not known in advance, the best way to maintain them is as
linked chains. Each chain will have a head node which will usually be much smaller than the
other nodes since it has to retain only a link.
Using chaining to resolve collisions and the hash function used to obtain Figure 2.27, the hash
chains of Figure 2.28 are obtained. When a new identifier, X, is being inserted into a chain,
the insertion can be made at either end. This is so because the address of the last node in the
chain is known as a result of the search that determined X was not in the list for f(X).
Chapter 1 – Introduction and elementary data structures
Page 10
In the example of Figure 2.28 new identifiers were inserted at the front of the chains. The
number of probes needed to search for any of the identifiers is now 1 for each of A4, D, E, G,
L, and ZA; 2 for each of A3, GA and Z; 3 for Al; 4 for A2 and 5 for A for a total of 24. The
average is now 2.0 which is considerably less than for linear probing. Additional storage,
however, is needed for links.
procedure CHNSRCH(X,HT, b,j)
//search the hash table HT(O:b - 1) for X. Either HT(i) = 0//
//or it is a pointer to the list of identifiers X such that f(X) = i//
//List nodes have fields IDENT and LINK. Either j points//
//to the node already containing X or j = 0//
j HT(f(X)) //compute head node address//
//search the chain starting at j//
While j≠0 and IDENT(j) ≠ X do
j
LINK(j)
repeat
end CHNSRCH
HEAP
Chapter 1 – Introduction and elementary data structures
Page 11
Deletion
Chapter 1 – Introduction and elementary data structures
Page 12
Chapter 1 – Introduction and elementary data structures
Page 13
Chapter 1 – Introduction and elementary data structures
Page 14
Chapter 1 – Introduction and elementary data structures
Page 15
Chapter 1 – Introduction and elementary data structures
Page 16
Chapter 1 – Introduction and elementary data structures
Page 17