advertisement

Algorithm Analysis (Big O) 1 Complexity In examining algorithm efficiency we must understand the idea of complexity Space complexity Time Complexity 2 Space Complexity When memory was expensive we focused on making programs as space efficient as possible and developed schemes to make memory appear larger than it really was (virtual memory and memory paging schemes) Space complexity is still important in the field of embedded computing (hand held computer based equipment like cell phones, palm devices, etc) 3 Time Complexity Is the algorithm “fast enough” for my needs How much longer will the algorithm take if I increase the amount of data it must process Given a set of algorithms that accomplish the same thing, which is the right one to choose 4 Algorithm Efficiency a measure of the amount of resources consumed in solving a problem of size n Benchmarking: implement algorithm, time space run with some specific input and measure time taken better for comparing performance of processors than for comparing performance of algorithms Big Oh (asymptotic analysis) associates n, the problem size, with t, the processing time required to solve the problem 5 Cases to examine Best case if the algorithm is executed, the fewest number of instructions are executed Average case executing the algorithm produces path lengths that will on average be the same Worst case executing the algorithm produces path lengths that are always a maximum 6 Worst case analysis Of the three cases, only useful case (from the standpoint of program design) is that of the worst case. Worst case helps answer the software lifecycle question of: If its good enough today, will it be good enough tomorrow? 7 Frequency Count examine a piece of code and predict the number of instructions to be executed for each instruction predict how many times e.g. each will be encountered as the code runs Inst # 1 2 3 Code F.C. for (int i=0; i< n ; i++) n+1 { cout << i; n p = p + i; n } ____ 3n+1 totaling the counts produces the F.C. (frequency count) 8 Order of magnitude In the previous example: best_case = avg_case = worst_case Example is based on fixed iteration n By itself, Freq. Count is relatively meaningless Order of magnitude -> estimate of performance vs. amount of data To convert F.C. to order of magnitude: discard constant terms disregard coefficients pick the most significant term Worst case path through algorithm -> order of magnitude will be Big O (i.e. O(n)) 9 Another example Inst # Code 1 2 for (int i=0; i< n ; i++) for int j=0 ; j < n; j++) F.C. F.C. n+1 n+1 n(n+1) n2+n 3 { cout << i; n*n n2 4 p = p + i; n*n n2 ____ } 3n2+2n+1 discarding constant terms produces : 3n2+2n clearing coefficients : Big O = O(n2) n2+n picking the most significant term: n2 10 What is Big O Big rate at which algorithm performance degrades as a function of the amount of data it is asked to handle For O example: O(n) -> performance degrades at a linear rate O(n2) -> quadratic degradation 11 Common growth rates 12 Big Oh - Formal Definition Definition of "big oh": f(n)=O(g(n)), iff there exist constants c and n0 such that: f(n) <= c g(n) for all n>=n0 Thus, g(n) is an upper bound on f(n) Note: f(n) = O(g(n)) is NOT the same as O(g(n)) = f(n) The '=' is not the usual mathematical operator "=" (it is not reflexive) 13 Big-O Notation Comparing Algorithms and ADT Data Structures 14 Algorithm Efficiency a measure of the amount of resources consumed in solving a problem of size n benchmarking – code the algorithm, run it with some specific input and measure time taken time space better for measuring and comparing the performance of processors than for measuring and comparing the performance of algorithms Big Oh (asymptotic analysis) provides a formula that associates n, the problem size, with t, the processing time required to solve the problem 15 big Oh measures an algorithm’s growth rate how fast does the time required for an algorithm to execute increase as the size of the problem increases? is an intrinsic property of the algorithm independent of particular machine or code based on number of instructions executed for some algorithms is data-dependent meaningful for “large” problem sizes 16 Computing x * x * x .. * x (n times) recursive definition for n >= 0 iterative definition n x x0 = 1 xn = x * xn-1 (for n > 0) another recursive definition x0 = 1 xn = (xn/2)2 xn = x * (xn/2)2 (for n > 0 and n is even) (for n > 0 and n is odd) 17 Iterative Power function double IterPow (double X, int N) { double Result = 1; while (N > 0) { Result *= X; N--; { return Result; } Total instruction count: 1 n+1 n n critical region 1 3n+3 algorithm's computing time (t) as a function of n is: 3n + 3 t is on the order of f(n) - O[f(n)] O[3n + 3] is n 18 Recursive Power function double RecPow (double X, int N) { if (N == 0) return 1; else return X * RecPow(X, N - 1); } Base case 1 1 Recursive case 1 1 + T(n-1) total: 2 2 + T(n-1) Number of times base case is executed: 1 Number of times recursive case is executed: n Algorithm's computing time (t) as a function of n is: 2n + 2 O[2n + 2] is n 19 Another Power Function Base case double Pow3 (double X, int N) { if (N == 0) 1 return 1; 1 else { double halfPower = Pow3(X, N/2); if (N % 2 == 0) return halfPower * halfPower; else return X * halfPower * halfPower; } } total: 2 Recursive case 1 T(n/2) 1 1(even) 1(odd) 3 + T(n/2) Number of times base case is executed: 1 Number of times recursive case is executed: log2 n Algorithm's computing time (t) as a function of n is: 3 log2 n + 2 O[3 log2 n + 2] is log2 n 20 Computational Complexity Computing time, T(n), of an algorithm is a function of the problem size (based on instruction count) T(n) for IterPow is: 3n + 3 T(n) for RecPow is: 2n + 2 T(n) for Pow3 is: 3 log2 n + 2 Computational complexity of an algorithm is the rate at which T(n) grows as the problem size grows is expressed using "big Oh" notation growth rate (big Oh) of 3n+3 and of 2n+2 is: n big Oh of 3 log2 n + 2 is: log2 n 21 Common big Ohs constant logarithmic linear n log n quadratic cubic exponential O(1) O(log2 N) O(N) O(N log2 N) O(N2) O(N3) O(2N) 22 Comparing Growth Rates 2n n2 n log2 n n T(n) log2 n Problem Size 23 An Experiment Execution time (in seconds) 2^25 2^50 2^100 IterPow .71 1.15 2.03 RecPow 3.63 7.42 15.05 .99 1.15 1.38 Pow3 (1,000,000 repetitions) 24 Uses of big Oh compare algorithms which perform the same function search algorithms sorting algorithms comparing data structures for an ADT each operation is an algorithm and has a big Oh data structure chosen affects big Oh of the ADT's operations 25 Comparing algorithms Sequential search growth rate is O(n) average number of comparisons done is n/2 n 100 500 1000 5000 n/2 50 250 500 2500 Binary search growth rate is O(log2 n) average number of comparisons done is 2((log2 n) -1) 2((log2 n)-1) 12 16 18 24 26