Chapter 1 Introduction
Definition of Algorithm An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem.
Example 1 Describe an algorithm for finding the maximum (largest) value in a finite sequence of integers.
Solution
1.
Set the temporary maximum equal to the first integer in the sequence.
2.
Compare the next integer in the sequence to the temporary maximum, and set the larger one to be temporary maximum.
3.
Repeat the previous step if there are more integers in the sequence.
4.
Stop when there are no integers left in the sequence. The temporary maximum at this point is the maximum in the sequence.
Algorithm 1.
Finding the Maximum Element in a Finite Sequence
//Input : n integers a
1
,a
2
,...,a n
// Output : max (the maximum max for i:
:
 a
1
;

2 to n if max
 a i
then max :
 a i of a
1
,a
2
,...,a n
) return max
Instead of using a particular computer language, we use a form of pseudocode.

Algorithmic Problem Solving
Understand the problem
Design an algorithm and proper data structures
Analyze the algorithm
Code the algorithm
• Ascertaining the Capabilities of a Computational Device
• Choosing between Exact and Approximate Problem Solving
• Deciding on Appropriate Data Structures
• Algorithm Design
• Algorithm Analysis
• Coding

Important Problem Types
• Sorting
• Searching
• String processing (e.g. string matching)
• Graph problems (e.g. graph coloring problem)
• Combinatorial problems (e.g. maximizes a cost)
• Geometric problems (e.g. convex hull problem)
• Numerical problems (e.g. solving equations )

Fundamental Data Structures
1. Array a[1] a[2] a[3]
2. Link List
Head of list a
1
3. Binary Tree a
2 a
3
1
4
4
6
7
10
17
9
13
9
20 a[n] a n

4. Heap
Definition 1 A max-heap (or min-heap) A is an array object that can be reviewed as a complete binary tree (each node has two children, except the right part of the last level) , where at each node, there is a key which is larger (or smaller) or equal to the keys of its children.
A[0] A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8] A[9]
20 19 17 13 4 16 2 10 8 3
A[1] 19
A[0]
20
A[2] 17
Node i’s parent and children
Parent:  i
2
1


Left child: 2 i +1
A[3]
13
A[4]
4
A[7] A[8] A[9]
10 8 3
A[5]
16
A[6]
2
Right child: 2 i +2
The height of a max-heap with n elements is O(log n )
Operations in a max-Heap (or min-heap):
• Extract the item with the maximum (or minimum ) key.
• Insert a new item into the heap

5. Binary Search Tree
Definition 1 Binary Search Tree is a Binary Tree satisfying the following condition:
(1) Each vertex contains an item called as key which belongs to a total ordering set and two links to its left child and right child, respectively.
(2) In each node, its key is larger than the keys of all vertices in its left subtree and smaller than the keys of all the vertices in its right subtree.
1
3
4
6
7
9
15
13
17
18
20
Operations a binary search tree support : search, insert, and delete an element with a given key.
How to select a data structure for a given problem?

Fundamental Abstract Data Structures
1. Stack a n
• Operations a stack supports : last-in-first-out (pop, push)
• Implementation: using an array or a list (need to check underflows and overflows) a
2 a
1
2. Queue
• Operations a queue supports: first-in-first-out (enqueue, dequeue)
• Implementation: using an array or a list (need to check underflows and overflows) a m a m

1 a m

2 a n front rear
3. Priority Queue (each elements of a priority queue has a key)
• Operations a priority queue supports: inserting an element, returning an element with maximum/minimum key.
• Implementation: heap

4. Dynamic Dictionary
A Dynamic Dictionary is a data structure of item with keys that support the following basic operations:
(1) Insert a new item
(2) Remove an item with a given key
(3) Search an item with a given key

Some Advanced Data Structure
Graph
• Operations a graph support: finding neighbors
• Implementation: using list or matrix
G

( V , E )
V

( a , b , c , d , e , f )
E

{( a , c ), ( b , c ), ( d , a ), ( c , e ), ( e , c ), ( b , f ), ( d , e ), ( e , f )} a d c e b f c d a b e f f c c e a
Adjacent lists f e c a b c d e f








 f a 0 0 1 0 0 0 b 0 0 1 0 0 1 c 0 0 0 0 1 0 d 1 0 0 0 1 0 e 0 0 1 0 0 1
0 0 0 0 0 0









Adjacent matrix

Assignment 1
(1) Read the sections of the text book about array, linked list, heap, binary search tree, stack, queue, priority queue, and dynamic dictionary.
(2) (i) Implement stack S and queue Q using both of array and linked list.
(ii) Write a main function to do the following operations for S: pop, push 10 times, pop, pop, repeat push and pop 7 times. Do the following operations for Q: dequeue, enqueue 10 times, dequeue, depueue, repeat enqueue and dequeue 7 times. When you use array, declare the size of the array to be 10. Printout all the elements in S and in Q.
Select one of (3) and (4)
(3) Implement a min-heap of integer. The program must contain the following operations:
•
Heapify the min-heap
•
Build the min-heap from a given array.
•
Extract an item with the minimum key.
•
Insert a new item into the min-heap.
•
Print the heap.
Also, write a Main method to test your program as follows:
•
Create an array A of integers.
• Create an empty min-heap.
•
Assign 10 integers one by one into the array A in the following order: 12, 34, 76, 21, 4, 3, 33,
88, 90, 15.
•
Build the min-heap from array A, and print the heap.
•
Extract the smallest integer, and print the heap.
•
Insert a new integer into the heap, and print the heap.

(4) Write the program for BST of student records (student’s id is used as key). The program must contain the following 5 operations:
•
Search a node (given a student id), and print the record when it is found.
• Insert a node (given a new student record)
• Delete a node (given a student id)
• Print out of all student records in the BST in in-order.
Also, write a Main method to test your program as follows:
•
Create an empty BST.
•
Insert 10 student records one by one into the BST in the following order: record1: 1884, name1, score1 record2: -174, name2, score2 record3: 1464, name3, score3 record4: 527, name4, score4 record5: -1158, name5, score5 record6: 1860, name6, score6 record7: -1783, name7, score7 record8: -2014, name8, score8 record9: 1979, name9, score9 record10: 1892, name10, score10.
and print the BST in in-order to confirm 10 records are inserted correctly.
• Search a student record (giving a student id), and print the student record if the id is found.
• Delete a student record (giving a student id), and print the BST in in-order to confirm that the record is deleted.

Chapter 2 Fundamentals of the Analysis of Algorithm
Implementation and Empirical Analysis
Challenge in empirical analysis:
• Develop a correct and complete implementation.
• Determine the nature of the input data and other factors influencing on the experiment. Typically, There are three choices: actual data, random data, or perverse data.
• Compare implementations independent to programmers, machines, compilers, or other related systems.
• Consider performance characteristics of algorithms, especially for those whose running time is big.
Can we analyze algorithms that haven’t run yet?

Example 1 Describe an algorithm for finding an element x in a list of distinct elements a
1
, a
2
,..., a n
.
Algothm 1 The linear Search Algorithm.
Procedure linear sea rch ( x : integer, i:

1 ; while ( i
 n and x
 a i
)
i:
 i

1 ; if i
 n then location:
 i else locatiton:

0 ; a
1
,a
2
,..., a n
: distinct integers)
{ location is the subscript of term that equals x , or is 0 if x is not found}

Algorithm 2 The Binary Search Algorithm.
Procedure binary sea rch ( x : integer, i:

1 ; j:
 n ; while ( i
 j ) a
1
,a
2
,..., a n
: increasing integers) begin m: if x



(i
 a m j)/ 2
then

; i: else j:
 m ;
 m

1 end; if else x
 a i
then location: location:

0 ;
 i
{ location is the subscript of term that equals x , or is 0 if x is not found}

Linear Search and Binary Search written in C++
1.
Linear search int lsearch(int a[], int v, int l, int r)
{ for (int i= l ; i<=n; i++) if (v==a[i]) return i;
} return -1;
2.
Binary search int bsearch(int a[], int v, int l , int n)
{ while (r>= l )
{int m=( l +r)/2; if (v==a[m]) return m;
} if (v<a[m]) r=m-1; else l =m+1; return -1;
}

Complexity of Algorithms
Assume that both algorithms A and B solve the problem P . Which one is better?
•
Time complexity : the time required to solve a problem of a specified size.
•
Space complexity : the computer memory required to solve a problem of a specified size.
The time complexity is expressed in terms of the number of operations used by the algorithm.
•
Worst case analysis : the largest number of operations needed to solve the given problem using this algorithm.
•
Average case analysis : the average number of operations used to solve the problem over all inputs.

Example 2 Analyze the time complexities of linear search algorithm and binary search algorithm
Linear Search Algorithm.
Procedure linear sea rch ( x : integer, i:

1 ; while ( i
 n and x
 a i
)
i:
 i

1 ; if i
 n then location:
 i else locatiton:

0 ; a
1
,a
2
,..., a n
: distinct integers)
{ location is the subscript of term that equals x , or is 0 if x is not found}
1
2(n+1) n
2
3n+5

Binary Search Algorithm.
Procedure binary sea rch ( x : integer, i:

1 ; j:
 n ; while ( i
 j )
1
1
1 a
1
,a
2
,..., a n
: increasing integers) begin m: if x



(i
 a m j)/ 2
then

; i: else j:
 m ;
1
 m

1
2
Let n

2 k
.
It repeats at most k (k
 log
2 n) times.
end; if else x
 a i
then location: location:

0 ;
 i
2
{ location is the subscript of term that equals x , or is 0 if x is not found}
Number of operations = 4 log n+4
2

Orders of Growth
Running time for a problem with size n

10
6 necessary operations Running
Time
Operation
Per second
10
6
10
12 lg instant instant n n n
2
2 n
1 second 11.5 days Never end
2
59350 days
Instant 1 second Never end
2
59340 days
Using silicon computer, no matter how fast CPU will be you can never solve the problem whose running time is exponential !!!

Asymptotic Notations: O-notation
Definition 2.1 A function t(n) is said to be O(g(n)) if there exist some constant 0 and n
0

0 such that t ( n )
 c
0 g ( n ) for all n
 n
0
.
c
0 g ( n ) t ( n ) n
0 n
If lim n
  t g
( n )
( n )
 c (c

0 is a constant ) , then t(n)

O(g(n)) .

Example 3 Prove 2n+1=O(n)
Example 4 Prove 10 n
2 
12 n

5

O ( n
2
)
Example 5
List the following function in O -notation in increasing order: lg n , n , n
2
, n lg n , n
3
, n !
, 2 n
.
Example 6 What is the big oh of the following functions?
5n
5 
100 n
2 
1000 , n lg n
2 
100 n lg n

1000 n ,
0 .
0001

2 n  n

Asymptotic Analysis of algorithms (using O-notation)
Example 7 Analyze the time complexities of linear search algorithm and binary search algorithm asymptotically.
Linear Search Algorithm.
Procedure linear sea rch ( x : integer, i:

1 ; while ( i
 n and x
 a i
)
i:
 i

1 ; if i
 n then location:
 i else locatiton:

0 ; a
1
,a
2
,..., a n
: distinct integers)
It repeats n times.
{ location is the subscript of term that equals x , or is 0 if x is not found} Totally(addition): O(n)

Binary Search Algorithm.
Procedure binary sea rch ( x : integer, i:

1 ; j:
 n ; while ( i
 j ) a
1
,a
2
,..., a n
: increasing integers) begin m: if x



(i
 a m j)/ 2
then

; i: else j:
 m ;
 m

1 end; if else x
 a i
then location: location:

0 ;
 i
Let n

2 k
.
It repeats at most k (k
 log
2 n) times.
{ location is the subscript of term that equals x , or is 0 if x is not found}
Totally(comparison): O( log n)

Example 8 Analyze the time complexities of following algorithm asymptotically.
Matrix addition algorithm
Procedure MatricAddition( A[0..n-1,0..n-1],B[0..n-1,0..n-1] )
for i=0 to n-1 do
for j=0 to n-1 do
Repeat n times
C[i,j] = A[i,j] + B[i,j];
return C;
Repeat n times
O n
2
Totally(addition): ( )

Recursive Algorithms
Example 9 Computing the factorial function F(n)=n!.
F(n) can be defined recursively as follows:
F ( 0 )

1
F ( n )

F ( n

1 )
 n
Factorial Algorithm
Procedure factorial(n) if n = 0 return 1 else return factorial(n-1) * n;
Algorithm factorial calls itself in its body!
Time complexity(multiplication): T(0) = 0
T(n) = T(n-1) +1 when n>0 recurrence

Basic Recurrences
Example 10
Example 11
Solving the following recurrence
T(0) = 1
T(n) = T(n-1) + 1 n>0
Solve the following
T n
T
1

2 T n / 2
 c'
 n recurrence n

1
T ( n )

2 T ( n / 2 )
 cn
T(n) = T(n-1) + 1 
2 ( 2 T ( n / 2
2
)
 c ( n / 2 ))
 cn

2
2
T ( n / 2
2
)
 cn
 cn
= T(n-2) + 1 + 1 = T(n-2) + 2

2
2
( 2 T ( n / 2
3
)
 c ( n / 2
2
))
 cn
 cn

2
3
T ( n / 2
3
)

3 cn
= T(n-3) + 1 + 2 = T(n-3) + 3
......
…

2 i
T ( n / 2 i
)
 icn
......
= T(n-i) + i
…
= T(n-n) + n

2 k T ( n / 2 k )
 kcn ( k
 log n
 nT
1

O (n
 cn log log
n) n
 c '
 cn log n
)
= n

Example 12 Solve the recurrence Example
T ( n )
T
1

1

T ( n

1 )
 n n

1
13 Solve the recurrence
T ( n )
T ( 1 )


1
T ( n / 2 )

1 n

1
T ( n )

T ( n

1 )
 n

T ( n

2 )

( n

1 )
 n

T ( n

3 )

( n

2 )

( n

1 )
 n
......

T
1

2

1

2

3


3
...


...

( n n

2 )

( n

1 )
 n
 n ( n

1 ) / 2
(Assume that n is a power of 2 .
That
T ( n ) is, n

2

T ( n / k
,
2 ) where k

1
 log n ).

T ( n / 2
2
)

1

1

T ( n / 2
3
)

1

1

1
......

T ( n / 2 i
)
 i
......

T ( n / 2 k
)
 k ( k

1
 log n
 log n )