CISC 621 Algorithm Design and Analysis Dr. Rui Zhang Course Information • Instructor: Rui (Ray) Zhang • Office: 316A FinTech • Email: ruizhang@udel.edu • Meeting time • Section 010: Tuesday/Thursday 2:20PM--3:40PM • Section 011: Tuesday/Thursday 3:55PM--5:15PM • Meeting location • Section 010: Gore Hall Room 315 • Section 011: Smith Hall Room 130 Course Information • Office hour • 11:00AM--3:00PM Wednesday • Appointment via email • TA • Xihan Qin (xihan@udel.edu) • Jiahao Xie (jiahaox@udel.edu) Course Objectives • Upon completion of this course, you will be able to do the following • Demonstrate a familiarity with popular algorithmic problems, e.g., sorting, shortest path, and minimum spanning tree • Understand important algorithmic design paradigms such as dynamic programming, greedy algorithm, and incremental methods • Analyze the (asymptotic) performance of algorithms • Synthesize efficient algorithms in common engineering design situations • Understand the concept of problem hardness, and be able to identify NP-hard problems Course Outline • Asymptotic analysis • Randomized algorithm and probabilistic analysis • Divide-and-conquer paradigm • Dynamic-programming paradigm • Greedy paradigm • Incremental method • Major graph algorithms • Advanced data structures • NP-completeness • Approximation algorithms Required Textbook • Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, McGraw-Hill & MIT Press, 4th edition, ISBN-10: 026204630X, ISBN-13: 978-0262046305 Reference Textbook • Jon Kleinberg and Eva Tardos, Algorithm Design, Pearson; 1st edition, 2017, ISBN-10: 9780321295354, ISBN-13: 9780321295354 Homework • There will be 5 homework assignments • You can discuss homework assignments with others but are required to write up each problem solution yourself and identify your collaborator, if any. • You are required to type homework assignments using a word processor (e.g., MS Word or Latex). Handwriting will not be accepted • You are required to turn in via Canvas by 11:59PM on the specified due date. Policies on Late Submissions • Homework and assignment deadlines will be hard • Late submission will NOT be graded Exam • There will be two midterm exams • Midterm I is scheduled for October 9th • Midterm II is scheduled for November 13th • Both midterms will consist of several problems similar to homework problems • Both are close-books/notes • There will be one final exam • The final exam will follow the university schedule • Close-books/notes Generative AI Tools • Students are not allowed to use advanced automated tools (artificial intelligence or machine learning tools such as ChatGPT or Dall-E 2) on assignments in this course. Each student is expected to complete each assignment without substantive assistance from others, including automated tools. Grading • Final course grades will be based on homework assignments, midterm exams, and the final exam • Homework assignments . . . . . . . . . . . . . . . . . 25% • Midterm I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25% • Midterm II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20% • Final exam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28% • Course evaluation .. . . . . . . . . . . . . . . . . . . . . . . . 2% Grading • A:>=93 • A-: >=90 • B+:>=87 • B: >=84 • B-: >=80 • C+: >=77 • C: >=74 • C-: >=71 • D+:>=68 • D:>=64 • D-: >=60 • F:<60 Lecture 1 Analysis of Algorithm, Insertion Sort, Merge Sort Sorting Problem • Input: • Output: a permutation of input sequence such that Insertion Sort • Good for sorting a small number of elements • Intuition: mimic the way to sort a hand of playing cards How to Express Algorithms? • Express algorithms in whatever way is the clearest and most concise • When issues of control need to be made perfectly clear, we often use pseudocode • Pseudocode is similar to C, C++, Pascal, and Java • The goal is to express algorithms to humans • We sometimes embed English statements into pseudocode • Ignore error handling Pseudocode for Insertion Sort Example Proof of Correctness using Loop Invariant • Loop invariant • A property of a program loop that is true before each iteration • Initialization • It is true prior to the first iteration of the loop • Maintenance • If it is true before an iteration of the loop, it remains true before the next iteration • Termination • When the loop terminates, the invariant—usually along with the reason that the loop terminated—gives us a useful property that helps show that the algorithm is correct Loop Invariant of Insertion Sort • At the start of each iteration of the “outer” for loop consists of the elements originally in but in sorted order Correctness of Insertion Sort • Initialization • Just before the first iteration, . The subarray the single element , which is trivially sorted is • Maintenance • Insert s into the right position of is sorted. such that • Termination • The outer loop ends, when invariant, we get is sorted . Plug into loop How to Analyze Algorithms? • Random-access machine (RAM) model • Instructions executed one after another • Instructions commonly found in real computers • Arithmetic: add, subtract, multiply, divide, remainder, floor, ceiling, etc. • Data movement: load, store, copy • Control: conditional/unconditional branch, subroutine call and return • Each of these instructions takes constant time • Limit on the word size. • It takes bits to store a number How to Analyze an Algorithm’s Running Time • Running time depends on the input • Input size matters • Ex: sorting 10 numbers vs. sorting 10000 numbers • Different inputs of the same size may have different running time • Ex: sorting 1000 random numbers vs. sorting 1000 almost sorted numbers • Input size • Usually, the number of items in the input • Could be something else • Multiplying two integers: the total number of bits of two integers • Graph algorithm: the number of vertexes and the number of edges • etc. Running Time • The number of primitive operations (steps) executed • Define steps to be machine-independent • Assume that each line of pseudocode requires a constant amount of time • One line may take a different amount of time than another • Line takes constant time operations if it consists only of primitive Analysis of Insertion Sort : the number of times that the while loop test is executed for that value of Analysis of Insertion Sort • Let be the running time of insertion sort • Running time depends on Best Case • The array is already sorted • Always find that test (when ) • for all • Running time is given by • can be expressed as upon the first time the while loop , i.e., is a linear function of Worst Case • The array is in reverse sorted order • Always find that • Have to compare • for all • It follows that in while loop test with all elements to the left ( elements) Worst Case • Running time is given by • is a quadratic function of Which One Shall We Care About? • Worst-case running time • Running time guarantee for any input of size • Average-case running time • Expected running time for any input of size • Require assumptions about probability distribution of inputs • What if the input distribution is not known? • Best-case running time Other Types of Analysis • Probabilistic analysis • Expected running time of a randomized algorithm for any input of size • Amortized analysis • Worst-case running time for any sequence of operations • Competitive analysis • The performance of an online algorithm is compared to the performance of an optimal offline algorithm • etc. Order of Growth • Important abstraction to ease analysis and focus on the important features • Drop lower-order terms • Ignore the constant coefficient in the leading term • Example • The worst-case running time of insertion sort is • We say that the running time is to capture the notion that that the order of growth is • More on asymptotic notation later Divide and Conquer Paradigm • Divide the problem into a number of subproblems that are smaller instances of the same problem • Conquer the subproblems by solving them recursively • Base case: If the subproblems are small enough, just solve them by brute force • Combine the subproblem solutions to give a solution to the original problem Merge Sort • A sorting algorithm based on divide and conquer • Its worst-case running time has a lower order of growth than insertion sort • Subproblem • Sorting an array , initially • Divide by splitting into two subarrays • Conquer by recursively sorting the two subarrays • Combine by merging the two sorted subarrays Pseudocode of Merge Sort Example Example Merging: • Input: Array and indices • • Subarrays sorted and such that are both non-empty and • Output: A single sorted subarray in • Design goal • Implement it so that it takes • What is here? • The size of the subproblem ( time ) Idea behind Linear Time Merging Two sorted pile of cards At each step, choose the smaller of the two top cards How many steps in total? Pseudocode Analyzing Divide-and-Conquer Algorithms • Use a recurrence equation (more commonly, a recurrence) to describe the running time of a divide-and-conquer algorithm • Let be the running time on a problem of size (base case) subproblems each of size time to combine solutions time to divide a size- problem Analyzing Merge Sort • For simplicity, assume that is a power of 2 • Each divide step yields two subproblems, both of size exactly • Base case: when • Divide: just compute , so • Conquer: recursively solve 2 subproblems, each of size exactly , so • Combine: MERGE on an -element subarray takes • So we have Solving the Merge-Sort Recurrence • By the Master Theorem (Ch. 4), we can show that this recurrence has the solution • Compared to insertion sort ( running time), merge sort is faster worst-case • Actually a big deal! • Note that insertion sort may be faster on small input Understand the Solution Without Master Theorem • Rewrite the recurrence as (simplified) • We can draw a recursion tree • For the original problem Understand the Solution Without Master Theorem • For each of the size- subproblems Understand the Solution Without Master Theorem • Continue expanding until the problem sizes get down to 1 Summary of Recursion Tree • Each level has cost • Top level has cost • The next down has 2 subproblems, each contributing cost • The next level has 4 subproblems, each contributing cost • … • Total • Total cost is levels , so Reading Assignment • Chapters 2
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )