“Two ideas lie gleaming on the jeweler’s velvet. The first is the
calculus, the second, the algorithm. The calculus and the rich
body of mathematical analysis to which it gave rise made
modern science possible; but it has been the algorithm that
has made possible the modern world”
David Berlinski, 2000
Lecture notes based on the course textbook: Levitin, A.V. (2012). Introduction to the Design and Analysis of Algorithms (3rd ed.).
Alg
Houssain Kettani, Ph. D.
2
Content
What is an algorithm?
Fundamentals of algorithmic problem solving
Important problem types
Fundamental data structures
Alg
Houssain Kettani, Ph. D.
3
Algorithm
An algorithm is a sequence of unambiguous
instructions for solving a problem, i.e., for obtaining
a required output for any legitimate input in a finite
amount of time.
Alg
Houssain Kettani, Ph. D.
4
Algorithm vs. Al-Khawarizm
Derives from the Latin form of al-Khawarizmi's name
Muhammad ibn Musa al-Khawarizmi (780 – 847), an
Arabic Mathematician, was born and lived mostly in
Baghdad, Iraq
It is the title of his text, Hisab al-jabr w'al-muqabala,
that gives us the word "algebra" and, it is the first book
to be written on algebra.
Al-jabr in Arabic means to bring things together
Alg
Houssain Kettani, Ph. D.
5
Historical Perspective
One of the oldest known algorithms is Euclid’s
algorithm for finding the greatest common divisor
Euclid (325 BC – 265 BC) is a Greek mathematician,
was born and lived mostly in Alexandria, Egypt.
Alg
Houssain Kettani, Ph. D.
6
Divisibility
An integer d divides an integer n if d ≠ 0 and there is
an integer k such that n = dk.
We write d|n, to denote the fact that d divides n.
Example:
3 divides 18 because we can write 18 = (3)(6).
But 5 does not divide 18 because there is no integer k such
that 18 = 5k.
All the divisors of 18 are
–18, –9, –6, –3, –2, –1, 1, 2, 3, 6, 9, 18.
Alg
Houssain Kettani, Ph. D.
7
Greatest Common Divisor
The greatest common divisor of two integers, not
both zero, is the largest integer that divides them
both.
i.e., the largest d such that d|a and d|b.
We denote the greatest common divisor of a and b by
gcd(a, b).
Example
The common divisors of 12 and 18 are ±1, ±2, ±3, ±6.
So the greatest common divisor of 12 and 18 is 6,
Thus, we write gcd(12, 18) = 6.
Alg
Houssain Kettani, Ph. D.
8
Euclid’s Algorithm
Based on repeatedly applying the equality:
gcd(m,n) = gcd(n, m mod n)
Until (m mod n) is equal to 0.
Since gcd(m,0), the last value of m is the gcd of the
initial m and n.
Example:
gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.
Alg
Houssain Kettani, Ph. D.
9
Euclid’s Algorithm
Algorithm: Euclid(m, n)
// Computes gcd(m, n) by Euclid’s algorithm
// Input: Two non-negative, not-both-zero integers m and n
// Output: Greatest common divisor of m and n
while n ≠ 0 do
r ← m mod n
m←n
n←r
return m
Alg
Houssain Kettani, Ph. D.
10
Middle-School Algorithm
Follows the following four steps:
1. Find the prime factors of m.
2. Find the prime factors of n.
3. Identify all the common factors in the two prime
expansions found in Step 1 and Step 2.
4. Compute the product of all common factors and return
the greatest common divisor of the numbers given
Example:
60 = 2 . 2 . 3 . 5
24 = 2 . 2 . 2 . 3
gcd(60, 24) = 2 . 2 . 3 = 12.
Alg
Houssain Kettani, Ph. D.
11
Notion of Algorithm
problem
algorithm
input
“computer”
output
Algorithmic solution
Alg
Houssain Kettani, Ph. D.
12
Example of Computational
Problem: Sorting
Statement of problem:
Input: A sequence of n numbers <a1, a2, …, an>
Output: A reordering of the input sequence <a´1, a´2, …, a´n> so
that a´i ≤ a´j whenever i < j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:
Selection sort
Insertion sort
Merge sort
(many others)
Alg
Houssain Kettani, Ph. D.
13
Selection Sort
Input: array a[1],…,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i = 1 to n-1
swap a[i] with smallest of a[i+1],…,a[n]
• see also pseudocode, section 3.1
Alg
Houssain Kettani, Ph. D.
14
Some Well-Known
Computational Problems
Alg
Sorting
Searching
Shortest paths in a graph
Minimum spanning tree
Primality testing
Traveling salesman problem
Knapsack problem
Chess
Towers of Hanoi
Program termination
Houssain Kettani, Ph. D.
15
Basic Issues Related to Algorithms
How to design algorithms
How to express algorithms
Proving correctness
Efficiency
Theoretical analysis
Empirical analysis
Optimality
Alg
Houssain Kettani, Ph. D.
16
Algorithm Design Strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Greedy approach
Dynamic programming
Backtracking and Branch and bound
Space and time tradeoffs
Alg
Houssain Kettani, Ph. D.
17
Analysis of Algorithms
How good is the algorithm?
Correctness
Time efficiency
Space efficiency
Does there exist a better algorithm?
Lower bounds
Optimality
Alg
Houssain Kettani, Ph. D.
18
What Is an Algorithm?
Recipe, process, method, technique, procedure,
routine,… with the following requirements:
1. Finiteness: Terminates after a finite number of steps
2. Definiteness: Rigorously and unambiguously specified
3. Input: Valid inputs are clearly specified
4. Output: Can be proved to produce the correct output given
a valid input
5. Effectiveness: Steps are sufficiently simple and basic.
Alg
Houssain Kettani, Ph. D.
19
Why Study Algorithms?
Theoretical importance
The core of computer science
Practical importance
A practitioner’s toolkit of known algorithms
Framework for designing and analyzing algorithms
for new problems
Alg
Houssain Kettani, Ph. D.
20
Thank You!
Alg
Houssain Kettani, Ph. D.
21