Lec. 5
Best , Worst and Average Cases
Ex. 10:
Insertion an element x to sorted array A[0..n-1], where all the
elements of the array are sorted in increasing order before and after
insertion process.
Algorithm Insertelement (A, n, x):
Input: Sorted array A[0..n-1] with size n and the required element x to be
inserted.
Output: Sorted array A[0..n].
1. for i ← n-1 down to 0 while i >= 0 and x < A[i]
2. A[i+1] ←A[i]
3. end for
4. A[i+1] ← x
5. n← n+1
The instance characteristics of this problem is n.
Time complexities (step counts):
1)) Best case:
When x is greater than all elements of A.
1.
2.
3.
4.
T
for
…………………… 1
statement inside for .…………. 0
insert x
…………………… 1
increment the size n ………….. 1
B
Insertelement
(n) = 3
2)) Worst case:
When x is smaller than all elements of A.
1.
2.
3.
4.
T
for
…………………… [(n-1)-0+1] +1 = n+1
statement inside for .…………. n
insert x
…………………… 1
increment the size n ………….. 1
W
Insertelement
(n) = 2n+3
1
3)) Average case:
It is the average of complexities of insertion cases for all positions.
n
T
Av
Insertelement
(n) =
∑ 2(n − i) + 3
i =0
n +1
The best and worst cases are subset from the average case.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Asymptotic Notations
The process of determining step counts of an algorithm was proved
that it is high difficult process, thus Asymptotic Notations are used
in computing space and time complexities of an algorithm, they are
estimation of the complexities in real but they are easer than step counts.
Asymptotic Notations give formula type without interesting with its
details.
1)) Upper Bound Notation (Big-Oh):
f(n) is O(g(n)) if there exist positive constants c and n0 where (c>0),
(n0 ≥0) such that
f(n) ≤ c ⋅ g(n) for all n ≥ n0.
f(n): is space function S(n) or time function T(n).
This definition shows that function f(n) grows no faster than c.g(n).
Theorem:
If f(n) = a0 + a1n + ..... + am −1n m −1 + am n m
then
f(n) = O(nm)
is a polynomial of degree m,
where g(n)= nm.
2
Ex:
f(n)=3n+2
3n+2 = O(n)
where g(n)=n.
To prove this
f(n) ≤ c ⋅ g(n)
3n+2 ≤ 4n
Ex:
where 2 is n0 , c=4.
f(n)=100
100=О(1)
Because 100 ≤ 100*1
Ex:
for all n ≥ 2
where g(n)=n0 =1
for all n ≥ 0.
f(n)=10n2+4n+2
10n2+4n+2 = О(n2)
Because 10n2+4n+2 ≤ 11n2
Ex:
for all n ≥ 5.
f(n)=6*2n + n2
6*2n + n2 = О(2n)
Because 6*2n + n2 ≤ 7*2n
for all n ≥ 4.
The maximum bound of the complexities of space or time function
is c.g(n).
The following example explains this.
Ex:
3n+2 ≠ О(1).
10n2+4n+2 ≠ О(n).
2)) Lower Bound Notation (Big-Omega):
f(n) is Ω(g(n)) if there exist positive constants c and n0 where (c>0),
(n0 ≥0) such that
f(n) ≥ c ⋅ g(n) for all n ≥ n0.
This definition shows that function f(n) grows at least as fast as c.g(n),
and f(n) = Ω(g(n)) if and only if g(n) is O(f(n)) .
Theorem:
If f(n) = a0 + a1n + ..... + am −1n m −1 + am n m
then
f(n) = Ω (nm)
is a polynomial of degree m,
where g(n)= nm.
3
Ex:
f(n)=3n+2
3n+2 = Ω (n)
where g(n)=n.
To prove this
f(n) ≥ c ⋅ g(n)
3n+2 ≥ 3n
Ex:
where 0 is n0 , c=3.
f(n)=100
100= Ω (1)
Because 100 ≥ 100*1
Ex:
for all n ≥ 0
where g(n)=n0 =1
for all n ≥ 0.
f(n)=10n2+4n+2
10n2+4n+2 = Ω (n2)
Because 10n2+4n+2 ≥ 10n2
Ex:
for all n ≥ 0.
f(n)=6*2n + n2
6*2n + n2 = Ω (2n)
Because 6*2n + n2 ≥ 6*2n
for all n ≥ 0.
The minimum bound of the complexities of space or time function is
c.g(n).
The following example explains this.
Ex:
3n+2 ≠ Ω (n2) .
10n2+4n+2 ≠ Ω (n3).
4
3)) Upper-Lower Bound Notation (Big-Theta):
f(n) is Θ(g(n)) if there exist positive constants c1, c2, and n0 such that
c1 ⋅ g(n) ≤ f(n) ≤ c2 ⋅ g(n) for all n ≥ n0.
Theorem:
If f(n) = a0 + a1n + ..... + am −1n m −1 + am n m
then
f(n) = Θ (nm)
is a polynomial of degree m,
where g(n)= nm.
The Θ is more accurate and is achieved if g(n)is the lower and upper
bound of function f(n). This means:
f(n) = Θ(g(n)) if and only if f(n) = O(g(n)) AND f(n) = Ω(g(n))
where O(g(n)) and Ω(g(n)) are equivalents.
Ex:
f(n)=3n+2
3n+2 = Θ (n)
where g(n)=n.
To prove this
c1 ⋅ g(n) ≤ f(n) ≤ c2 ⋅ g(n)
3n ≤ 3n+2 ≤ 4n
Ex:
for all n ≥ 2
f(n)=100
where g(n)=n0 =1
100= Θ (1)
Because 100*1 ≤ 100 ≤ 100*1
Ex:
where 2 is n0 , c1=3, c2=4.
for all n ≥ 0.
f(n)=10n2+4n+2
10n2+4n+2 = Θ (n2)
Because 10n2 ≤ 10n2+4n+2 ≤ 11n2
for all n ≥ 5.
5
3n+2 ≠ Θ (n2) .
Ex:
3n+2 ≠ Θ (1) .
H.W.
Prove that the relation 5n2 – 6n = Θ(n2) is true (correct).
6
Lec. 6
Widespread Notations of The Complexities
Here some widespread Notations that are used in the complexities:
О(1) < О(log n) < О(n) < О(n log n) < О(n2) < О(n3) < О(2n)
1
log n
N
n log n
n2
n3
2n
n!
Constant
logarithmic
linear
n-log-n
Quadratic
cubic
Exponential
factorial
The algorithms which have complexities greater than О(n log n) are
considered non practical, and the algorithms which have exponential
complexities О(2n) are frightful and non practical except when n is very
small (leas than 40).
Explication:
Consider that there is an existing computer which runs
109 instruction / Sec, when f(n)=2n, then:
…….
Time
becomes long
Asymptotic Similarities
Asymptotic Similarities are shown in the following table, whereas the
symbol ⊕ means О, Ω, or Θ:
k
f(n)
C
n
⊕ (1)
n
n
n
∑i
∑ i2
∑ i k , k>0
∑ r i , r>1
n!
⊕ (n )
⊕ (n )
⊕ (n )
⊕ (n )
⊕ (r )
⊕ ((n/e) )
i =0
Asymptotic
n
∑ ci ni
k
i =1
2
i =1
i =1
3
k+1
7
i =0
n
1
∑i
i =1
n
⊕ (log n)
Produced Laws of Asymptotic Notations
Examples:
Analysis some of problems with Asymptotic
notations.
Sum Algorithm (Ex. 1):
1. ……… Θ(1).
2. ……… Θ(n).
3. ……… Θ(n).
5. ……… Θ(1).
Add Algorithm (Ex. 2):
1. ……… Θ(m).
2. ……… Θ(n).
3. ……… Θ(mn).
5. ……… Θ(mn).
Ssum(n) = Θ(1)
, Tsum(n) = Θ(n).
Sadd(m,n) = Θ(mn) = Θ(n2) , if n=m.
Tadd(m,n) = Θ(mn) = Θ(n2) , if n=m.
Fibonacci Algorithm (Ex. 3):
1. ……… Θ(1).
2. ……… Θ(1).
4. ……… Θ(1).
5. ……… Θ(1).
Sfibonacci(n) = Θ(1).
6. ……… Θ(n).
Tfibonacci(n) = Θ(n).
7. ……… Θ(n).
8. ……… Θ(n).
9. ……… Θ(n).
12. ……… Θ(1).
8
Ex. 11: Fibonacci sequence with recursion.
Algorithm Rfibonacci (n):
Input: a positive integer n.
Output: the n'th term of the fibonacci sequence.
1.
2.
3.
4.
if n < 2 then
return n
else
return (Rfibonacci (n-1) + Rfibonacci (n-2))
Time Complexities:
To know the time complexities of Rfibonacci, the tree of calling will
be build, for example, to compute Rfibonacci (5), (where n=5):
…………………………….…….. Level 1
……………………..…….. L 2
……………………. L 3
……………….. L 4
……………... L 5
This is binary tree, its depth is the maximum (last) level.
In a binary tree, the maximum number of nodes is (2k)-1, where k is the
tree's depth whereas k=n , thus (2k)-1 = (2n)-1.
Every node represents a call, then the maximum number of calls is (2n)-1,
thus:
TRfibonacci(n) = О(2n)
it is exponential.
9
Ex. 12:
Sequential search
It is one of search types, it searches about x element in array A.
The instance characteristics is n.
In this idea the search process begins from the end of A.
Space Complexities:
SSeqSearch (n) = Θ(1).
Time Complexities:
1)) Successful Searches:
- Best case: It occurs when the searching element x is in end of A, the
required time for searching in this case is constant and independent on
n value, thus:
TBSeqSearch (n) = Θ(1).
- Worst case: It occurs when the searching element x is in the first
position of A.
TWSeqSearch (n) = Θ(n).
n
- Average case:
TASeqSearch (n) =
∑i
i =1
n
=
n +1
= Θ(n).
2
2)) Failed Searches: It occurs if the searching element x doesn't exist
in the array.
TSeqSearch (n) = Θ(n).
10
Another Idea for Sequential Search
The search process begins from the beginning of A.
Its space and time complexities like them of the previous algorithm.
11