Lab of CS103 Data Structure and C++ by Han Young Ryoo, Joseph Gomes Big-O Notation Definition: T(n) is O(f(n)) iff (if and only if) there exist positive constants c and n0 such that T(n) <= c f(n), for all n >= n0 Examples T(n) = 3n + 2 is O(n) since 3n + 2 <= 4n for all n >= 2 where c=4 and n0 = 2. T(n) = 10n2 + 4n + 2 is O(n2) since 10n2 + 4n + 2 <= 11n2 for all n >= 5 where c=11 and n0 = 5. Why do we need Big O To decide whether an algorithm is adequate, For a better implementation * The algorithm will always be too slow on a big enough input. Quicksort vs. Bubble sort (O(n log n) vs. O(n2) Quicksort running on a small desktop computer can beat bubble sort running on a super-computer if there are a lot of numbers to sort. To sort 1,000,000 numbers, the quicksort takes 20,000,000 steps on average, while the bubble sort takes 1,000,000,000,000 steps! How to determine Sequence of statements: statement 1; statement 2; ... statement k; The total time is found by adding the times for all statements: total time = time(statement 1) + time(statement 2) + ... + time(statement k) Rules for using big-O Ignoring constant factors O(c f(N)) = O(f(N)), where c is a constant Example) O(20 N3) = O(N3) Rules for using big-O Ignoring smaller terms If a<b then O(a+b) = O(b) Example) O(N2+N) = O(N2) Rules for using big-O Upper bound only If a<b then an O(a) algorithm is also an O(b) algorithm. Example) an O(N) algorithm is also an O(N2) algorithm (but not vice versa). Rules for using big-O N and log N are "bigger" than any constant, from an asymptotic view (that means for large enough N). So if k is a constant, an O(N + k) algorithm is also O(N), by ignoring smaller terms. Similarly, an O(log N + k) algorithm is also O(log N). Rules for using big-O An O(N log N + N) algorithm, which is O(N(log N + 1)), can be simplified to O(N log N). Simple Statements If each statement is "simple" (only involves basic operations) then the time for each statement is constant and the total time is also constant: O(1). if-then-else statements if (condition) { sequence of statements 1 } else { sequence of statements 2 } Either sequence 1 will execute, or sequence 2 will execute. The worst-case time: the slowest of the two possibilities: max(time(sequence 1), time(sequence 2)). If sequence 1 is O(N) and sequence 2 is O(1), what is the worst-case time for the whole if-then-else statement? loops for (i = 0; i < N; i++) { sequence of statements } The loop executes N times, so the sequence of statements also executes N times. If the statements are O(1), what is the total time for the for loop? Nested loops for (i = 0; i < N; i++) { for (j = 0; j < M; j++) { sequence of statements } } The complexity is O(N * M). Need Generalization: In a common special case where the stopping condition of the inner loop is j < N instead of j < M (i.e., the inner loop also executes N times), the total complexity for the two loops is O(N2). Function/Method calls When a statement involves a method call, the complexity of the statement includes the complexity of the method call. Assume that you know that method f takes constant time, and that method g takes time proportional to (linear in) the value of its parameter k. Then the statements below have the time complexities indicated. f(k); // O(1) g(k); // O(k) Example: for (j = 0; j < N; j++) { g(N);} What is the complexity? A program that calculates execution time in milliseconds #include <sys/timeb.h> #include "stdio.h" #include "math.h" int main(int argc, char* argv[]) { struct _timeb time_start,time_end; _ftime( &time_start ); for (int i=1;i<=10000;i++) { int m = 1; for(int j=1;j<100000;j++) { m *= 2; } } _ftime( &time_end ); printf("elapsed time = %d milliseconds\n",time_end.millitm-time_start.millitm); return 0; } Exercises Two loops in a row: for (i = 0; i < N; i++) { sequence of statements } for (j = 0; j < M; j++) { sequence of statements } How would the complexity change if the second loop went to N instead of M? Exercises A nested loop followed by a non-nested loop: for (i = 0; i < N; i++) { for (j = 0; j < N; j++) { sequence of statements } } for (k = 0; k < N; k++) { sequence of statements } Exercises A nested loop with dependent loop index: for (i = 0; i < N; i++) { for (j = i; j < N; j++) { sequence of statements } } A nested loop in which the number of times the inner loop executes depends on the value of the outer loop index Quiz for (i = 0; i < N; i++) { for (j = 0; j < N; j++) { C[i][j] = 0; for (k = 0; k < N; k++) { C[i][j] = C[i][j] + A[i][k] * B[k][j]; } } } Quiz Sorting for (i = N-1; i > 0; i--) for (j = 0; j < i; i++) if (a[j] > a[j+1]) swap a[j] and a[j+1];

Download
# CS103-31 Lab of Data Structure and C++