“Two ideas lie gleaming on the jeweler’s velvet. The first is the
calculus, the second, the algorithm. The calculus and the rich
body of mathematical analysis to which it gave rise made
modern science possible; but it has been the algorithm that
has made possible the modern world”
David Berlinski, 2000
Lecture notes based on the course textbook: Levitin, A.V. (2012). Introduction to the Design and Analysis of Algorithms (3rd ed.).
Alg
Houssain Kettani, Ph. D.
2
Content
 What is an algorithm?
 Fundamentals of algorithmic problem solving
 Important problem types
 Fundamental data structures
Alg
Houssain Kettani, Ph. D.
3
Algorithm
 An algorithm is a sequence of unambiguous
instructions for solving a problem, i.e., for obtaining
a required output for any legitimate input in a finite
amount of time.
Alg
Houssain Kettani, Ph. D.
4
Algorithm vs. Al-Khawarizm
 Derives from the Latin form of al-Khawarizmi's name
 Muhammad ibn Musa al-Khawarizmi (780 – 847), an
Arabic Mathematician, was born and lived mostly in
Baghdad, Iraq
 It is the title of his text, Hisab al-jabr w'al-muqabala,
that gives us the word "algebra" and, it is the first book
to be written on algebra.
 Al-jabr in Arabic means to bring things together
Alg
Houssain Kettani, Ph. D.
5
Historical Perspective
 One of the oldest known algorithms is Euclid’s
algorithm for finding the greatest common divisor
 Euclid (325 BC – 265 BC) is a Greek mathematician,
was born and lived mostly in Alexandria, Egypt.
Alg
Houssain Kettani, Ph. D.
6
Divisibility
 An integer d divides an integer n if d ≠ 0 and there is
an integer k such that n = dk.
 We write d|n, to denote the fact that d divides n.
 Example:
 3 divides 18 because we can write 18 = (3)(6).
 But 5 does not divide 18 because there is no integer k such
that 18 = 5k.
 All the divisors of 18 are
–18, –9, –6, –3, –2, –1, 1, 2, 3, 6, 9, 18.
Alg
Houssain Kettani, Ph. D.
7
Greatest Common Divisor
 The greatest common divisor of two integers, not
both zero, is the largest integer that divides them
both.
 i.e., the largest d such that d|a and d|b.
 We denote the greatest common divisor of a and b by
gcd(a, b).
 Example
 The common divisors of 12 and 18 are ±1, ±2, ±3, ±6.
 So the greatest common divisor of 12 and 18 is 6,
 Thus, we write gcd(12, 18) = 6.
Alg
Houssain Kettani, Ph. D.
8
Euclid’s Algorithm
 Based on repeatedly applying the equality:
gcd(m,n) = gcd(n, m mod n)
 Until (m mod n) is equal to 0.
 Since gcd(m,0), the last value of m is the gcd of the
initial m and n.
 Example:
 gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.
Alg
Houssain Kettani, Ph. D.
9
Euclid’s Algorithm
 Algorithm: Euclid(m, n)
// Computes gcd(m, n) by Euclid’s algorithm
// Input: Two non-negative, not-both-zero integers m and n
// Output: Greatest common divisor of m and n
while n ≠ 0 do
r ← m mod n
m←n
n←r
return m
Alg
Houssain Kettani, Ph. D.
10
Middle-School Algorithm
 Follows the following four steps:
1. Find the prime factors of m.
2. Find the prime factors of n.
3. Identify all the common factors in the two prime
expansions found in Step 1 and Step 2.
4. Compute the product of all common factors and return
the greatest common divisor of the numbers given
 Example:
 60 = 2 . 2 . 3 . 5
 24 = 2 . 2 . 2 . 3
 gcd(60, 24) = 2 . 2 . 3 = 12.
Alg
Houssain Kettani, Ph. D.
11
Notion of Algorithm
problem
algorithm
input
“computer”
output
Algorithmic solution
Alg
Houssain Kettani, Ph. D.
12
Example of Computational
Problem: Sorting
 Statement of problem:
 Input: A sequence of n numbers <a1, a2, …, an>
 Output: A reordering of the input sequence <a´1, a´2, …, a´n> so
that a´i ≤ a´j whenever i < j
 Instance: The sequence <5, 3, 2, 8, 3>
 Algorithms:
 Selection sort
 Insertion sort
 Merge sort
 (many others)
Alg
Houssain Kettani, Ph. D.
13
Selection Sort
 Input: array a[1],…,a[n]
 Output: array a sorted in non-decreasing order
 Algorithm:
for i = 1 to n-1
swap a[i] with smallest of a[i+1],…,a[n]
• see also pseudocode, section 3.1
Alg
Houssain Kettani, Ph. D.
14
Some Well-Known
Computational Problems










Alg
Sorting
Searching
Shortest paths in a graph
Minimum spanning tree
Primality testing
Traveling salesman problem
Knapsack problem
Chess
Towers of Hanoi
Program termination
Houssain Kettani, Ph. D.
15
Basic Issues Related to Algorithms
 How to design algorithms
 How to express algorithms
 Proving correctness
 Efficiency
 Theoretical analysis
 Empirical analysis
 Optimality
Alg
Houssain Kettani, Ph. D.
16
Algorithm Design Strategies
 Brute force
 Divide and conquer
 Decrease and conquer
 Transform and conquer
 Greedy approach
 Dynamic programming
 Backtracking and Branch and bound
 Space and time tradeoffs
Alg
Houssain Kettani, Ph. D.
17
Analysis of Algorithms
 How good is the algorithm?
 Correctness
 Time efficiency
 Space efficiency
 Does there exist a better algorithm?
 Lower bounds
 Optimality
Alg
Houssain Kettani, Ph. D.
18
What Is an Algorithm?
 Recipe, process, method, technique, procedure,
routine,… with the following requirements:
1. Finiteness: Terminates after a finite number of steps
2. Definiteness: Rigorously and unambiguously specified
3. Input: Valid inputs are clearly specified
4. Output: Can be proved to produce the correct output given
a valid input
5. Effectiveness: Steps are sufficiently simple and basic.
Alg
Houssain Kettani, Ph. D.
19
Why Study Algorithms?
 Theoretical importance
 The core of computer science
 Practical importance
 A practitioner’s toolkit of known algorithms
 Framework for designing and analyzing algorithms
for new problems
Alg
Houssain Kettani, Ph. D.
20
Thank You!
Alg
Houssain Kettani, Ph. D.
21