CSI - 3105 Design & Analysis of Algorithms
Jean-Lou De Carufel
Fall 2022
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
1 / 20
Course Presentation
Course Outline
Schedule:
Monday
Thursday
11:30 to 13:00
13:00 to 14:30
(CRX - C240)
(CRX - C240)
Website: BrightSpace
My office: STE - 5108
Office hours: Thursday 14:30 to 16:00
The following textbook is amazing.
Sanjoy Dasgupta, Christos H. Papadimitriou and Umesh Vazirani.
Algorithms.
McGraw-Hill Education, 2006.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
2 / 20
Course Presentation
Teaching Assistants
Saman Bazargani
Office:
Email address:
Office Hours:
STE-5116
sbaza033 @ uottawa . ca
Tuesday 15:15 to 16:45
Saeed Odak
Office:
Email address:
Office Hours:
STE-5000-J
sodak073 @ uottawa . ca
Wednesday 9:00 to 10:30
Frédéric Simard
Office:
Email address:
Office Hours:
TBA
fsima063 @ uottawa . ca
Friday 11:00 to 12:30
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
3 / 20
Course Presentation
Assignments
To be submitted directly in BrightSpace.
Weight
Deadline
Assignment 1
0%
Sept. 22, 2022
Assignment 2
0%
Oct. 6, 2022
Assignment 3
0%
Nov. 10, 2022
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
before 13:00
before 13:00
before 13:00
Fall 2022
4 / 20
Course Presentation
Online Mini-Tests
To be done directly in BrightSpace. You will have 30 minutes for each
mini-test. You can decide when you want to write your mini-test within
the time window allocated for it.
Weight
Mini-Test 1
5%
Time Window
between Sept. 29, 2022 at 14:30
and Oct. 1, 2022 at 14:30
Mini-Test 2
5%
between Oct. 13, 2022 at 14:30
and Oct. 15, 2022 at 14:30
Mini-Test 3
5%
between Nov. 17, 2022 at 14:30
and Nov. 19, 2022 at 14:30
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
5 / 20
Course Presentation
In-person Exams
To be done in person only.
Weight
Date & Time
Exam 1
17.5%
Oct. 17, 2022 from 11:30 to 12:50
Exam 2
17.5%
Nov. 21, 2022 from 11:30 to 12:50
Final Exam
50%
TBA
No documentation is authorized for exams.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
6 / 20
Course Presentation
Exercises
Assignments
Mini-Tests
Exams
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
7 / 20
Introduction
Chapter 1: Introduction
What does “algorithm” mean?
Al-Khwarizmi (783 - 850)
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
8 / 20
Introduction
What does “design & analysis of algorithms” mean?
Correctness of algorithms.
Does it terminate?
Efficient (fast):
Estimate the running time.
Count the number of steps.
Is it optimal? Can we do better?
Limits of efficiency (some problems cannot be solved efficiently).
Pseudocode, no programming.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
9 / 20
Introduction
Insertion Sort
Input: An array A[1..n] of n numbers.
Output: An array containing the numbers of A in increasing order.
1: for j = 2 to n do
2:
key = A[j]
3:
i =j 1
4:
while i > 0 and A[i] > key do
5:
A[i + 1] = A[i]
6:
i =i 1
7:
end while
8:
A[i + 1] = key
9: end for
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
10 / 20
insertion
for j
2 to n
key AG
É
is
While i 0 and
Ati
Alita
F i
i
AE
Eadwhile
endfor
Atia Key
keydo
I
II
Introduction
What is “the best” input for Insertion Sort?
What it “the worst” input for Insertion Sort?
How much time does it take to sort n numbers with Insertion Sort ?
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
11 / 20
Introduction
Line
1
2
3
4
5
6
7
Instruction
for j = 2 to n do
key = A[j]
i = j 1W
JIE
Time (in ms.)
c1
c2
c3
while i > 0 and A[i] > key do
Iggy
A[i + 1] = A[i]
i =i
c5
1
end while
Fact
c4
c6
i
0
# of times
n
n 1
n 1
n
P
tj
n
P
j=2
8
j=2
n
P
j=2
n
P
(tj
1)
(tj
1)
(tj
1)
j=2
8
9
A[i + 1] = key
end for
J.-L. De Carufel (U. of O.)
c7
0
Design & Analysis of Algorithms
n
n
1
1
Fall 2022
12 / 20
line
that
tj
valve
of the time of execution of
will depend on
the value of
in a for loop the
determine execution
this
we use
Value of j will
time of one law
t instead of n
a
Introduction
Fit
Carda
te
In the best case, it takes an + b units of time to sort n numbers with
Insertion Sort (for some constants a and b).
In the worst case, it takes cn2 + dn + e units of time to sort n
numbers with Insertion Sort (for some constants c, d and e).
É
een
J.-L. De Carufel (U. of O.)
EY i
n
Design & Analysis of Algorithms
Fall 2022
13 / 20
Introduction
Suppose that a computer can execute 109 = 1000 000 000 operations per
second.
Number of operations
n2
1 2
2n
1
2n
n
191 E
50000
49950
log(n)
2n
50
2100
n = 100
n = 1000 000
0, 000 010 000 sec.
1000 sec.
0, 000 005 000 sec.
500 sec.
0, 000 000 100 sec.
0, 001 sec.
0, 000 000 007 sec.
0, 000 000 020 sec.
4 ⇥ 1013 years
3 ⇥ 10301 013 years
4 ⇥ 1013 years is larger than the age of the universe.
What do you think of the constants a, b, c, ... ?
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
14 / 20
Introduction to Order
you
Definition (O-Notation)
Let
f :N
g :N
! R+ ,
! R+
be two functions. We say that f is O of (or is big O of) g if there exist a
constant c 2 R+ and a number k 2 N such that f (n)  c g (n) for all
n k.
We write
f (n) = O(g (n))
We can say that since for is bounded
gen
or
at k onwards
f = O(g ) .
by
Insertion Sort takes O(n2 ) time in the worst case.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
15 / 20
Introduction to Order
Definition (⌦-Notation)
Let
f :N
g :N
! R+ ,
! R+
be two functions. We say that f is ⌦ of (or is big ⌦ of) g if there exist a
constant c 2 R+ and a number k 2 N such that f (n) c g (n) for all
n k.
We write
f (n) = ⌦(g (n))
or
f = ⌦(g ) .
Insertion Sort takes ⌦(n2 ) time in the worst case.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
16 / 20
Introduction to Order
Definition (⇥-Notation)
Let
f :N
g :N
! R+ ,
! R+
be two functions. We say that f is ⇥ of (or is big ⇥ of) g if there exist
two constants c1 2 R+ and c2 2 R+ and a number k 2 N such that
c1 g (n)  f (n)  c2 g (n) for all n k.
We write
f (n) = ⇥(g (n))
or
f = ⇥(g ) .
Insertion Sort takes ⇥(n2 ) time in the worst case.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
17 / 20
Introduction to Order
An algorithm A takes O(T (n)) time, for a function T , if there exist
a strictly positive constant c
and an implementation of A which takes at most c T (n) units of time
to execute for any input of size n.
This is possible thanks to the Principle of Invariance.
Two di↵erent implementations of the same algorithm will not di↵er
in efficiency by more than some multiplicative constant.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
18 / 20
Introduction to Order
Barometer Instruction
A barometer instruction is one that is executed at least as often as any
other instruction in the algorithm.
There is no harm if some instructions are executed up to a constant
number of times more often than the barometer since their contribution is
absorbed in the asymptotic notation (O, ⌦ and/or ⇥).
The time of computation of the algorithm is then in the order of the
number of executions of the barometer instruction.
Can you identify a barometer instruction in Insertion Sort?
J.-L. De Carufel (U. of O.)
C inner loop
Design & Analysis of Algorithms
Fall 2022
19 / 20
Introduction to Order
To establish the relation between two functions, we can use the following
theorem.
Theorem (Limit Criterion)
Let
f :N
g :N
be two functions. Let
! R+ ,
! R+
f (n)
L = lim
.
n!1 g (n)
If L = 0, then f = O(g ) (and f 6= ⇥(g )).
If L = 1, then f = ⌦(g ) (and f 6= ⇥(g )).
If L 2 R+ , then f = ⇥(g ) (and g = ⇥(f )).
If the limit does not exist, then we cannot conclude.
J.-L. De Carufel (U. of O.)
Design & Analysis of Algorithms
Fall 2022
20 / 20