Design and Analysis of Computer Algorithms Spring ‘08 Chuan-Ming Liu Computer Science & Information Engineering NTUT, Taiwan 1 Mobile Computing & Software Engineering Lab Instructor Chuan-Ming Liu (劉傳銘) Office: Room 502-3 Computer Science and Information Engineering National Taipei University of Technology TAIWAN Phone: (02) 2771-2171 ext. 4251 Email: cmliu@csie.ntut.edu.tw Office Hours: Thu 10:10-12:00 and Fri 10:10-12:00 2 Mobile Computing & Software Engineering Lab Text Books Introduction to Algorithms, 2nd edition. T. Cormen, C. Leiserson, R. Rivest, and C. Stein. McGraw-Hill, 2001. Supplementary Texts 1. Computer Algorithms /C++, 2nd Edition. E. Horowitz, S. Sahni, and S. Rajasekaran, W.H. Freeman and Company, 1998. 2. Computer Algorithms, 3rd edition. S. Baase and A. V. Gelder. AddisonWesley, 2000. 3. Algorithms in C++, 3rd. R. Sedgewick. Addison-Wesley, 1998 4. Algorithms, R. Johnsonbaugh and M. Schaefer, Pearson Prentice Hall, 2004. 5. Algorithm Design, Jon Kleinberg and Éva Tardos, Pearson/Addison-Wesley, 2006. 6. Algorithms, Sanjoy Dasgupta, Christos Papadimitriou, Umesh Vazirani, McGrawHill, 2008. 3 Mobile Computing & Software Engineering Lab 1 Course Outline • • • • • • • • • • • • Introduction Asymptotic analysis Recurrence Relations Data Structures Average and worst analysis Lower Bounds Dynamic Programming Graph algorithms Search and Traversal Techniques NP-Completeness Approximation Algorithms New Trends in Algorithms 4 Mobile Computing & Software Engineering Lab Course Work • • • • Assignments (30%): 6-8 homework sets Report (15%, by reading a research paper) Midterm (25%): 2hr exam Final exam (30%): 2-3hr exam 5 Mobile Computing & Software Engineering Lab Course Policy (1) • No late homework is acceptable. • For a regrade please contact me for the question within 10 days from the date when the quiz or exam was officially returned. No regrading after this period. • Cheating directly affects the reputation of the Department and the University and lowers the morale of other students. Cheating in homework and exam will not be tolerated. An automatic grade of 0 will be assigned to any student caught cheating. Presenting another person's work as your own constitutes cheating. Everything you turn in must be your own doing. 6 Mobile Computing & Software Engineering Lab 2 Course Policy (2) • The following activities are specifically forbidden on all graded course work: – Theft or possession of another student's solution or partial solution in any form (electronic, handwritten, or printed). – Giving a solution or partial solution to another student, even with the explicit understanding that it will not be copied. – Working together to develop a single solution and then turning in copies of that solution (or modifications) under multiple names. 7 Mobile Computing & Software Engineering Lab First Thing to Do • Please visit the course web site http://www.cc.ntut.edu.tw/~cmliu/Alg/NTUT_ Alg_S08g/index.htm • Send an email to me using the email address:cmliu@csie.ntut.edu.tw. I will make a mailing list for this course. All the announcements will be broadcast via this mailing list. 8 Mobile Computing & Software Engineering Lab Introduction Chuan-Ming Liu Computer Science & Information Engineering National Taipei University of Technology TAIWAN 9 Mobile Computing & Software Engineering Lab 3 Outline • • • • Algorithms – definition and applications Algorithm Specification Performance Analysis Randomized Algorithms 10 Mobile Computing & Software Engineering Lab What is an Algorithm? • Definition An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. All the algorithms must satisfy the following criteria: – Input – Output – Definiteness – Effectiveness – Finiteness 11 Mobile Computing & Software Engineering Lab What is an Algorithm ? • Definiteness: each instruction is clear and unambiguous • Effectiveness: each instruction is executable; in other words, feasibility • Finiteness: the algorithm terminates after a finite number of steps. 12 Mobile Computing & Software Engineering Lab 4 What is an Algorithm ? Input Definiteness Output Effectiveness Finiteness Computational Procedures 13 Mobile Computing & Software Engineering Lab Procedures vs. Algorithms • Termination or not • One example for procedure is OS • Program, a way to express an algorithm 14 Mobile Computing & Software Engineering Lab Applications of Algorithms Practical applications of algorithms are ubiquitous, for example, – Human Genome Project – Internet – E-commerce (security issues) – Resource Management in Manufacturing and other Commercial Setting –… 15 Mobile Computing & Software Engineering Lab 5 Short History (1) • The word algorithm derives from Al Kwarizmi (about 790 - about 840), an Islamic mathematician. • He laid out the basic methods for adding, multiplying, and dividing numbers, as well as extracting square roots and calculating digits of π. 16 Mobile Computing & Software Engineering Lab Al Kwarizmi JOC/EFR © July 1999 http://www-history.mcs.st-andrews.ac.uk/Mathematicians/Al-Khwarizmi.html 17 Mobile Computing & Software Engineering Lab Short History (2) • The 13th century Italian mathematician Leonardo Fibonacci developed Al Kwarizmi’s work further and propagandizing it. • However, Fibonacci is most widely known for his famous sequence of numbers 0, 1, 1, 2, 3, 5, 8, 13, 21, 34 … 18 Mobile Computing & Software Engineering Lab 6 Leonardo Fibonacci Corbis 19 Mobile Computing & Software Engineering Lab Study on Algorithms • • • • Devise: useful design techniques will learn Correctness: learning how to prove Analysis: why the devised algorithm is good Testing: alternative way to show correctness 20 Mobile Computing & Software Engineering Lab Describing Algorithms • Graphical view (flow charts) • Programming languages (C/C++, java, …) • Pseudo-code representation 21 Mobile Computing & Software Engineering Lab 7 Pseudo-Code Conventions • Indentation as block structure • Loop and conditional constructs similar to those in PASCAL, such as while, for, repeat, if-then-else • Symbol as the comment in a line • Using ← instead of = and allowing i ← j ← e • Variables local to the given procedure 22 Mobile Computing & Software Engineering Lab Pseudo-Code Conventions • Array element accessed by A[i] and A[1..j] as the subarray of A • NIL representing the pointer referring to no object • Call by value procedures Please refer to page 19 on the textbook. 23 Mobile Computing & Software Engineering Lab Sorting Problem Input: Given a sequence of n elements, a1, a2, …, an Output: a permutation of these n elements in non-decreasing order 24 Mobile Computing & Software Engineering Lab 8 Example 10 40 25 36 8 7 10 25 36 40 8 7 8 10 25 36 40 7 25 Mobile Computing & Software Engineering Lab Insertion Sort – Pseudo Code 1 26 Mobile Computing & Software Engineering Lab Insertion Sort – Pseudo Code 2 InsertionSort (a, n) for j=2 to n do item = a[j]; i = j-1; while ((i >= 1) && (item < a[i])) do a[i+1] = a[i]; i--; a[i+1] = item; 27 Mobile Computing & Software Engineering Lab 9 Algorithm Analysis (1) • Machine-independent • Storage requirement Space complexity: maximum amount of memory to complete an algorithm • Computing time Time complexity: amount of time to complete an algorithm 28 Mobile Computing & Software Engineering Lab Algorithm Analysis (2) • What to measure? How to measure? • Input size: depends on the problem – Number of items for sorting – Total number of bits for multiplying two integers – Sometimes, more than one input • Running time in terms of the number of primitive operations or “steps” executed. 29 Mobile Computing & Software Engineering Lab Analysis of Insertion Sort 30 Mobile Computing & Software Engineering Lab 10 Analysis of Insertion Sort • The best-case time complexity is Θ(n) – How come? • Time complexity for the worst-case is Θ(n2) • Even though the time complexity is O(n2), the insertion sort runs fast when n is small, usually n<=16. 31 Mobile Computing & Software Engineering Lab Designing Algorithms • Many techniques for designing algorithms and we will discuss some of them in the class • Insertion sort uses an incremental approach • We now discuss “divide-and-conquer” approach using merge sort as an example. 32 Mobile Computing & Software Engineering Lab Divide-and-Conquer (1) • General Description – A problem having n inputs – Divide the problem into k sub-problems – Solve each sub-problem – Combine (merge) all the results of the subproblems to derive the solution of the problem • Usually, the sub-problems keeps the same type as the original problem Recursive procedure takes place 33 Mobile Computing & Software Engineering Lab 11 Divide-and-Conquer (2) • Can be implemented by a recursive algorithm • The computing time in general is g ( n) small n T ( n) = T n T n L T n f n ( ) + ( ) + + ( ) + ( ) otherwise 2 k 1 where g(n) is the time when n is small and f(n) is the time for dividing and combination 34 Mobile Computing & Software Engineering Lab Example – Merge Sort • Sorting Problem: – Input: Given a sequence of n elements, a1, a2, …, an – Output: a permutation of these n elements in nondecreasing order • Using divide-and-conquer strategy – Divide the input sequence into subsequences with smaller size – Merge the results of the subsequences toward the final result 35 Mobile Computing & Software Engineering Lab Example 1 2 3 4 5 6 7 8 9 10 310 285 179 652 351 423 861 254 450 520 (1,5) (1,3) (1,2) 1 (4,5) 3 2 (6,10) 4 (6,8) 5 (6,7) 6 (9,10) 8 9 10 7 36 Mobile Computing & Software Engineering Lab 12 Example for Merge 179 285 310 351 652 254 423 450 520 861 179 254 285 310 351 423 450 520 652 861 Then, copy to a[] 37 Mobile Computing & Software Engineering Lab Merge Sort 38 Mobile Computing & Software Engineering Lab Merge 39 Mobile Computing & Software Engineering Lab 13 40 Mobile Computing & Software Engineering Lab Time Analysis • The merge process needs O(n) time since it scans each element in worst case a n =1 T (n) = 2T (n / 2) + cn n > 1 • T(n) = O(n log n) 41 Mobile Computing & Software Engineering Lab 14