TTIT33 Algorithms and Optimization – Lecture 1
Analysis of Algorithms
• Introductory Example
• Principles of Analysis
• Big-Oh notation
• Time complexity of iterative algorithms
• Time complexity of recursive algorithms
References:
[G&T] 4.2 , or
[L&D] Chapter 1, 2.1-2.2, or
[CL] Chapter 1-3 , or …
Jan Maluszynski - HT 2006 1.1
TTIT33 Algorithms and Optimization – Lecture 1
Example problem:
Checking results of an exam:
• Input:
– A set of students (name, personal no) who passed the exam
– Name of a student
• Output: ”yes” or ”no”
To implement this on a computer we:
• Characterize the data and the needed operations on data
Æ ADT (e.g
ADT Set )
• Choose a representation of the data in the computer? Æ datastructure (e.g. table/array )
• Implement the needed operations: algorithm
(e.g. Table search )
• Analyse the efficiency of the implementation
Jan Maluszynski - HT 2006 1.2
TTIT33 Algorithms and Optimization – Lecture 1
How to measure time/space
• Describe algorithms in pseudocode
(or in a high-level programming language)
• Analyse number of basic operations as function of the size of input data
• Use simple data memory model
• Analyse number of needed memory cells
Jan Maluszynski - HT 2006 1.3
TTIT33 Algorithms and Optimization – Lecture 1
Example of pseudocode: function TableSearch ( table<integer> T [1..
n ] , key k ): boolean for i from 1 to n do if T [ i ] = k then return true if T [ i ] > k then return false return false
...”scope” defined by indentation!
Jan Maluszynski - HT 2006 1.4
TTIT33 Algorithms and Optimization – Lecture 1
Principles of Algorithm Analysis
An algorithm should work for (input) data of any size.
(Example TableSearch: input size is the size of the table.)
Show the resource (time/memory) used as an increasing function of input size .
Focus on the worst case performance.
Ignore constant factors
• analysis should be machine-independent;
• more powerful computers introduce speed-up by constant factors.
Study scalability / asymptotic behaviour for large problem sizes:
• ignore lower-order terms, focus on dominating terms.
Jan Maluszynski - HT 2006 1.5
TTIT33 Algorithms and Optimization – Lecture 1
”Order” notation
• Compares growth rate of functions
• f
O( g )
” f grows at most as
as g” apart of constant factors
• Refer to simple well known functions f(n) grows
as n 2
Jan Maluszynski - HT 2006 1.6
1

TTIT33 Algorithms and Optimization – Lecture 1
Types of growth comparison:
• f , g growing functions from natural numbers to positive real numbers
• f is (in) O ( g ) iff there exist c > 0 , n
0
≥
1 such that f ( n ) ≤ c g ( n ) for all n
≥ n
0
Intuition: Apart from constant factors, f grows at most as quickly as g
• f is (in) Ω ( g ) iff there exist c > 0 , n
0
≥ 1 such that f ( n ) ≥ c g ( n ) for all n ≥ n
0
Intuition: Apart from constant factors, f grows at least as quickly as g
Ω() is the converse of O , i.e. f is in Ω ( g ) iff g is in O ( f )
≥≤
• f is (in) Θ ( g ) iff f ( n ) ∈ O(g( n )) and g(n) ∈ O(f(n))
Intuition: Apart from constant factors, f grows exactly as quickly as g
Jan Maluszynski - HT 2006 1.7
TTIT33 Algorithms and Optimization – Lecture 1
…comparison with simple math function: n
2
16
64 log
2 n
1
4
6 n
2
16
64 n log
2 n
2
64
384 n 2
4
256
4096
2 n
4
6.5
* 10 4
1.84
* 10 19
1.84* 10 19 µ sec = 2.14
* 10 8 days = 5845 centuries
Jan Maluszynski - HT 2006 1.9
TTIT33 Algorithms and Optimization – Lecture 1
Estimating execution time for iterative programs
TTIT33 Algorithms and Optimization – Lecture 1
Types of growth comparison:
Ω ( g ) , Θ ( g ), O ( g ) ..??
Jan Maluszynski - HT 2006 1.8
TTIT33 Algorithms and Optimization – Lecture 1
To check growth rate?
Jan Maluszynski - HT 2006 1.10
TTIT33 Algorithms and Optimization – Lecture 1
Example of ”algebraic” analysis
Jan Maluszynski - HT 2006 1.11
Jan Maluszynski - HT 2006 1.12
2

TTIT33 Algorithms and Optimization – Lecture 1
Example: Dependent Nested Loops
TTIT33 Algorithms and Optimization – Lecture 1
Analysis of Recursive Program (1)
Jan Maluszynski - HT 2006 1.13
TTIT33 Algorithms and Optimization – Lecture 1
Analysis of Recursive Programs...
Jan Maluszynski - HT 2006 1.14
TTIT33 Algorithms and Optimization – Lecture 1
Towers of Hanoi....
Jan Maluszynski - HT 2006 1.15
TTIT33 Algorithms and Optimization – Lecture 1
Average case analysis
Reconsider TableSearch(): sequential search through a table
• Input argument: one of the table elements,
• assume it is chosen with equal probability for all elements.
Time to find the element when it was in the n:th place
Expected search time: t c
+
2 t c
+
3 t c
+
...
+ nt c n
=
( 1
+
2
+
3
+
...
+ n ) t c n
= n ( n
+
1 ) t c
2 n
= n
+
1
∈
O ( n )
2
Jan Maluszynski - HT 2006 1.17
As stated earlier:
• Formulate an equation T(1)=... T(2)=...
• Unroll a few times, get hypothesis for T(n)=...
• Prove the hypothesis!
Jan Maluszynski - HT 2006 1.16
3