Algorithmic Complexity What to measure? Space utilization: Time efficiency: amount of memory required amount of time required to process the data. — Depends on many factors: — size of input, — speed of machine, — quality of source code, — quality of compiler. These factors vary from one machine/compiler (platform) to another count the number of times instructions are executed So, measure computing time as: T(n) = computing time of an algorithm for input of size n = number of times the instructions are executed. Example: Calculating the Mean /* Algorithm to find the mean of n real numbers. Receive: integer n 1 and an array x[0], . . . , x[n–1] of real numbers Return: The mean of x[0], . . . , x[n–1] ----------------------------------------------------------------------------------------* / 1. Initialize sum to 0. 2. Initialize index variable i to 0. 3. While i < n do the following: 4. a. Add x[i] to sum. 5. b. Increment i by 1. 6. Calculate and return mean = sum / n . T(n) = 3n + 4 Big Oh Notation The computing time of an algorithm on input of size n, T(n) is said to have order of magnitude f(n), written T(n) is O(f(n)) if there is some constant C such that T(n) <= C.f(n) for all sufficiently large values of n. Equivalently, the complexity of the algorithm is O(f(n)). Example: For the Mean-Calculation Algorithm: T(n) is O(n) (Note that constants and multiplicative factors are ignored). Worst-case Analysis The arrangement of the input items may affect the computing time How then to measure performance? best case – not very informative average - too difficult to calculate worst case - usual measure /* Linear search of the list a[0], . . . , a[n – 1]. Receive: An integer n an array of n elements and item Return: found = true and loc = position of item if the search is successful; otherwise, found is false. */ 1. found = false. 2. loc = 0. 3. While (loc < n && !found ) 4. If item = a[loc] then found = true // item found 5. Else Increment loc by 1 // keep searching Worst case: Item not in the list: TL(n) is O(n) Average case (assume equal distribution of values) also O(n) Makes sense: each item (or proportion thereof) must be examined! Binary Search /* Binary search of the list a[0], . . . , a[n – 1] in which the items are in ascending order. Receive: integer n and an array of n elements and item. Return: found = true and loc = position of item if the search successful otherwise, found is false. */ 1. found = false. 2. first = 0. 3. last = n – 1. 4. While (first < last && !found ) 5. Calculate loc = (first + last) / 2. 6. If item < a[loc] then 7. last = loc – 1. // search first half 8. Else if item > a[loc] then 9. first = loc + 1. // search last half 10. Else found = true. // item found Worst case: Item not in the list: TB(n) = O(log2n) Makes sense: each pass cuts search space in half! Recursion Recursion a function references (or calls) itself. A very old idea, with its roots in mathematical induction. Induction always involves two things: A base or “trivial” case An inductive case Example: Factorial numbers Base case: 0! = 1 Inductive case: n! = n*(n-1)! Computing times of recursive functions Have to solve a recurrence relation. // Towers of Hanoi void Move(int n, char source, char destination, char spare) { if (n <= 1) // anchor (base) case cout << "Move the top disk from " << source << " to " << destination << endl; else { // inductive case Move(n-1, source, spare, destination); Move(1, source, destination, spare); Move(n-1, spare, destination, source); } } T(n) = O(2n) Recursion inefficient Each recursive call results in Creation and storage of activation record = = expensive overhead Most recursive functions can be rewritten iteratively Much cheaper!! // example counting the number of digits in a positive integer int F(unsigned n, int count) { // recursive, expensive! if (n < 10) return 1 + count; else return F(n/10, ++count); } int F(unsigned n) { // iterative, cheaper int count = 0; while ( n >= 10) { count++ n /= 10; } return count; } Comments on Recursion There are several common applications of recursion where a corresponding iterative solution may not be obvious or easy to develop. Classic examples of such include Towers of Hanoi, path generation, multidimensional searching and backtracking. Note that common textbook examples of recursion usually are tail-recursive, i.e. the last statement in the recursive function is a recursive invocation. Tail-recursive functions can be (much) more efficiently written using a conditional (while) loop. More complicated recursive functions are frequently replaced by iterative functions that use a stack to store the "recursive" call. Recursive Examples The next example explores the examination of a 2D structure (a grid). In this simplistic representation, each element of a grid is blank or marked by a special character. We want to search the grid to find the number of "blobs" -- sets of contiguous asterisks ("*"). This classic example rests on the notion of using the recursive structure to control searching in multiple directions. Linear evaluation is not sufficient because most grid points have neighbors to the north, south, east and west. Care must be taken here to control recursive invocation in order to prevent infinite recursing. For example, one does not want to call one's south neighbor and then be called by one's south neighbor, etc. Typically, one alters the grid to indicate that a point has been visited. Recursive Blobs #include <iostream> using namespace std; const int maxRow = some_value; const int maxCol = some_value; typdef char Grid[maxRow][maxCol]; void initGrid(Grid& g); // each element of the grid is initialized // one can use two values: * and " " // or four (introduce two additional values to indicate that the element has been visited // if a copy of the grid is used then elements can be changed // as they are visited. Some method of marking is necessary // to prevent infinite recursion Recursive Blobs void countBlob(Grid& g, int x, int y) { if (g[x][y] == "*") { g[x][y] = "X"; // mark as visited // thus counted if (x < maxRow-1) countBlob(g, x+1, y); else if (x > 0) countBlob(g, x-1, y); else if (y < maxCol-1) countBlob(g, x, y+1); else if (y > 0) countBlob(g, x, y-1); } return; } Recursive Blobs int main() { Grid grid; initGrid(grid); int count = 0; for (int row = 0; row < maxRow; row++) for (int col = 0; col < maxCol; col++) if (grid[row][col] == "*") { countBlob(grid, row, col); count++; } cout << "The number of blobs in your grids is " << count << endl; return 0; } Recursive Permutations Permutation generation is another classic example of the power of recursion when used for backtracking. Again, the recursive structure is used to complete path generation (in this case permutations) from partial paths at all levels. Here we permute a sequence of digits (characters could have just as easily been used). The basic idea is to fill an array with the digits and use the recursive structure to systematically swap the digits to acquire all permutations. Note the replacement of the digits following the recursive call. Recursive Permutations #include <iostream> using namespace std; const int maxValue = some_value; void PrintPerm(int[] p) { int index = 1; while (p[index] != 0) { cout << p[index]; index = p[index]; // next number in permutation } cout << endl; return; } Recursive Permutations void Permute(int[] p, int k, int n) { int index = 0; do { p[k] = p[index]; p[index] = k; if (k == n) PrintPerm(p); else Permute(p, k+1, n); p[index] = p[k]; // swap back index = p[index]; } while (p == 0); return; } Recursive Permutations int main() { int Perm[maxValue]; int number = 0; while (number <=0 || number > maxValue) { cout << "Enter number of elements to permute, less than " << maxValue-1 << endl; cin >> number; } Perm[0] = 0; Permute(Perm, 1, number); return 0; }