Design and Analysis of
Computer Algorithms
Spring ‘08
Chuan-Ming Liu
Computer Science & Information Engineering
NTUT, Taiwan
1
Mobile Computing & Software Engineering Lab
Instructor
Chuan-Ming Liu (劉傳銘)
Office: Room 502-3
Computer Science and Information Engineering
National Taipei University of Technology
TAIWAN
Phone: (02) 2771-2171 ext. 4251
Email: cmliu@csie.ntut.edu.tw
Office Hours: Thu 10:10-12:00 and Fri 10:10-12:00
2
Mobile Computing & Software Engineering Lab
Text Books
Introduction to Algorithms, 2nd edition. T. Cormen, C.
Leiserson, R. Rivest, and C. Stein. McGraw-Hill, 2001.
Supplementary Texts
1. Computer Algorithms /C++, 2nd Edition. E. Horowitz, S. Sahni, and S.
Rajasekaran, W.H. Freeman and Company, 1998.
2. Computer Algorithms, 3rd edition. S. Baase and A. V. Gelder. AddisonWesley, 2000.
3. Algorithms in C++, 3rd. R. Sedgewick. Addison-Wesley, 1998
4. Algorithms, R. Johnsonbaugh and M. Schaefer, Pearson Prentice Hall, 2004.
5. Algorithm Design, Jon Kleinberg and Éva Tardos, Pearson/Addison-Wesley,
2006.
6. Algorithms, Sanjoy Dasgupta, Christos Papadimitriou, Umesh Vazirani, McGrawHill, 2008.
3
Mobile Computing & Software Engineering Lab
1
Course Outline
•
•
•
•
•
•
•
•
•
•
•
•
Introduction
Asymptotic analysis
Recurrence Relations
Data Structures
Average and worst analysis
Lower Bounds
Dynamic Programming
Graph algorithms
Search and Traversal Techniques
NP-Completeness
Approximation Algorithms
New Trends in Algorithms
4
Mobile Computing & Software Engineering Lab
Course Work
•
•
•
•
Assignments (30%): 6-8 homework sets
Report (15%, by reading a research paper)
Midterm (25%): 2hr exam
Final exam (30%): 2-3hr exam
5
Mobile Computing & Software Engineering Lab
Course Policy (1)
• No late homework is acceptable.
• For a regrade please contact me for the question
within 10 days from the date when the quiz or exam
was officially returned. No regrading after this period.
• Cheating directly affects the reputation of the
Department and the University and lowers the morale
of other students. Cheating in homework and exam
will not be tolerated. An automatic grade of 0 will be
assigned to any student caught cheating. Presenting
another person's work as your own constitutes
cheating. Everything you turn in must be your own
doing.
6
Mobile Computing & Software Engineering Lab
2
Course Policy (2)
• The following activities are specifically forbidden on
all graded course work:
– Theft or possession of another student's solution or partial
solution in any form (electronic, handwritten, or printed).
– Giving a solution or partial solution to another student,
even with the explicit understanding that it will not be
copied.
– Working together to develop a single solution and then
turning in copies of that solution (or modifications) under
multiple names.
7
Mobile Computing & Software Engineering Lab
First Thing to Do
• Please visit the course web site
http://www.cc.ntut.edu.tw/~cmliu/Alg/NTUT_
Alg_S08g/index.htm
• Send an email to me using the email
address:cmliu@csie.ntut.edu.tw. I will make a
mailing list for this course. All the
announcements will be broadcast via this
mailing list.
8
Mobile Computing & Software Engineering Lab
Introduction
Chuan-Ming Liu
Computer Science & Information Engineering
National Taipei University of Technology
TAIWAN
9
Mobile Computing & Software Engineering Lab
3
Outline
•
•
•
•
Algorithms – definition and applications
Algorithm Specification
Performance Analysis
Randomized Algorithms
10
Mobile Computing & Software Engineering Lab
What is an Algorithm?
• Definition
An algorithm is a finite set of instructions that, if
followed, accomplishes a particular task. All the
algorithms must satisfy the following criteria:
– Input
– Output
– Definiteness
– Effectiveness
– Finiteness
11
Mobile Computing & Software Engineering Lab
What is an Algorithm ?
• Definiteness: each instruction is clear and
unambiguous
• Effectiveness: each instruction is
executable; in other words, feasibility
• Finiteness: the algorithm terminates after
a finite number of steps.
12
Mobile Computing & Software Engineering Lab
4
What is an Algorithm ?
Input
Definiteness
Output
Effectiveness
Finiteness
Computational Procedures
13
Mobile Computing & Software Engineering Lab
Procedures vs. Algorithms
• Termination or not
• One example for procedure is OS
• Program, a way to express an
algorithm
14
Mobile Computing & Software Engineering Lab
Applications of Algorithms
Practical applications of algorithms are
ubiquitous, for example,
– Human Genome Project
– Internet
– E-commerce (security issues)
– Resource Management in Manufacturing
and other Commercial Setting
–…
15
Mobile Computing & Software Engineering Lab
5
Short History (1)
• The word algorithm derives from Al Kwarizmi
(about 790 - about 840), an Islamic
mathematician.
• He laid out the basic methods for adding,
multiplying, and dividing numbers, as well as
extracting square roots and calculating digits
of π.
16
Mobile Computing & Software Engineering Lab
Al Kwarizmi
JOC/EFR © July 1999
http://www-history.mcs.st-andrews.ac.uk/Mathematicians/Al-Khwarizmi.html
17
Mobile Computing & Software Engineering Lab
Short History (2)
• The 13th century Italian mathematician
Leonardo Fibonacci developed Al Kwarizmi’s
work further and propagandizing it.
• However, Fibonacci is most widely known for
his famous sequence of numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34 …
18
Mobile Computing & Software Engineering Lab
6
Leonardo Fibonacci
 Corbis
19
Mobile Computing & Software Engineering Lab
Study on Algorithms
•
•
•
•
Devise: useful design techniques will learn
Correctness: learning how to prove
Analysis: why the devised algorithm is good
Testing: alternative way to show correctness
20
Mobile Computing & Software Engineering Lab
Describing Algorithms
• Graphical view (flow charts)
• Programming languages (C/C++, java, …)
• Pseudo-code representation
21
Mobile Computing & Software Engineering Lab
7
Pseudo-Code Conventions
• Indentation as block structure
• Loop and conditional constructs similar to
those in PASCAL, such as while, for, repeat,
if-then-else
• Symbol as the comment in a line
• Using ← instead of = and allowing i ← j ← e
• Variables local to the given procedure
22
Mobile Computing & Software Engineering Lab
Pseudo-Code Conventions
• Array element accessed by A[i] and A[1..j] as
the subarray of A
• NIL representing the pointer referring to no
object
• Call by value procedures
Please refer to page 19 on the textbook.
23
Mobile Computing & Software Engineering Lab
Sorting Problem
Input: Given a sequence of n elements,
a1, a2, …, an
Output: a permutation of these n elements in
non-decreasing order
24
Mobile Computing & Software Engineering Lab
8
Example
10 40 25 36
8
7
10 25 36 40
8
7
8
10 25 36 40
7
25
Mobile Computing & Software Engineering Lab
Insertion Sort – Pseudo Code 1
26
Mobile Computing & Software Engineering Lab
Insertion Sort – Pseudo Code 2
InsertionSort (a, n)
for j=2 to n
do
item = a[j];
i = j-1;
while ((i >= 1) && (item < a[i]))
do
a[i+1] = a[i]; i--;
a[i+1] = item;
27
Mobile Computing & Software Engineering Lab
9
Algorithm Analysis (1)
• Machine-independent
• Storage requirement
Space complexity: maximum amount of memory to
complete an algorithm
• Computing time
Time complexity: amount of time to complete an
algorithm
28
Mobile Computing & Software Engineering Lab
Algorithm Analysis (2)
• What to measure? How to measure?
• Input size: depends on the problem
– Number of items for sorting
– Total number of bits for multiplying two integers
– Sometimes, more than one input
• Running time in terms of the number of
primitive operations or “steps” executed.
29
Mobile Computing & Software Engineering Lab
Analysis of Insertion Sort
30
Mobile Computing & Software Engineering Lab
10
Analysis of Insertion Sort
• The best-case time complexity is Θ(n)
– How come?
• Time complexity for the worst-case is Θ(n2)
• Even though the time complexity is O(n2), the
insertion sort runs fast when n is small, usually
n<=16.
31
Mobile Computing & Software Engineering Lab
Designing Algorithms
• Many techniques for designing algorithms and
we will discuss some of them in the class
• Insertion sort uses an incremental approach
• We now discuss “divide-and-conquer”
approach using merge sort as an example.
32
Mobile Computing & Software Engineering Lab
Divide-and-Conquer (1)
• General Description
– A problem having n inputs
– Divide the problem into k sub-problems
– Solve each sub-problem
– Combine (merge) all the results of the subproblems to derive the solution of the problem
• Usually, the sub-problems keeps the same type
as the original problem
Recursive procedure
takes place
33
Mobile Computing & Software Engineering Lab
11
Divide-and-Conquer (2)
• Can be implemented by a recursive algorithm
• The computing time in general is
g ( n)
small n

T ( n) = 
T
n
T
n
L
T
n
f
n
(
)
+
(
)
+
+
(
)
+
(
)
otherwise
2
k
 1
where g(n) is the time when n is small and f(n) is
the time for dividing and combination
34
Mobile Computing & Software Engineering Lab
Example – Merge Sort
• Sorting Problem:
– Input: Given a sequence of n elements, a1, a2, …, an
– Output: a permutation of these n elements in nondecreasing order
• Using divide-and-conquer strategy
– Divide the input sequence into subsequences with
smaller size
– Merge the results of the subsequences toward the
final result
35
Mobile Computing & Software Engineering Lab
Example
1
2
3
4
5
6
7
8
9 10
310 285 179 652 351 423 861 254 450 520
(1,5)
(1,3)
(1,2)
1
(4,5)
3
2
(6,10)
4
(6,8)
5
(6,7)
6
(9,10)
8
9
10
7
36
Mobile Computing & Software Engineering Lab
12
Example for Merge
179 285 310 351 652
254 423 450 520 861
179 254 285 310 351 423 450 520 652 861
Then, copy to a[]
37
Mobile Computing & Software Engineering Lab
Merge Sort
38
Mobile Computing & Software Engineering Lab
Merge
39
Mobile Computing & Software Engineering Lab
13
40
Mobile Computing & Software Engineering Lab
Time Analysis
• The merge process needs O(n) time since it
scans each element in worst case
a
n =1

T (n) = 
2T (n / 2) + cn n > 1
• T(n) = O(n log n)
41
Mobile Computing & Software Engineering Lab
14