Programming Interest Group
http://www.comp.hkbu.edu.hk/~chxw/pig/index.htm
Tutorial Six
Divide and Conquer and
Backtracking
1
Outline



Recursion
Divide and Conquer
Backtracking

Examples



Constructing all subsets
Constructing all permutations
Pruning

Eight-Queens Problem
2
Recursion

A recursive function is one that calls itself.


There must be a termination condition.
Example:


Factorial function N! = Nx(N-1)x(N-2)x…x2x1 .
0! = 1
int factorial (unsigned int N) {
if (N == 0)
return 1;
return N*factorial(N-1);
}
3
Example: Euclid’s Algorithm

Find the greatest common divisors of two
integers
gcd(314159, 271828)
int gcd (int m, int n) {
if (n == 0)
return m;
return gcd(n, m%n);
}
gcd(271828, 42331)
gcd(42331, 17842)
gcd(17842, 6647)
gcd(6647, 4458)
gcd(4458, 2099)
gcd(2099, 350)
gcd(350, 349)
gcd(349, 1)
gcd(1, 0)
return 1;
4
Recursive Functions

Advantage:



Allow us to express complex algorithms in a compact form
Recursive functions are the cornerstone of several
advanced techniques: backtracking, divide and conquer,
dynamic programming
Disadvantage:



We are nesting function calls
Overhead of function calls
Sometimes the program will fail because of the depth of
recursion is too large!
5
Fibonacci Number



Fn = Fn-1 + Fn-2; F0 = 0; F1 = 1
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …
How about writing a recursive function?
int fib( unsigned int n ) {
if (n == 0 || n == 1)
return n;
Can you figure out the drawback
of the left algorithm?
return fib(n-1) + fib(n-2);
}
6
Divide and Conquer

Divide-and-conquer is a common and important
strategy in problem-solving.



Divide: to split the problem into two roughly equal subproblems, which are then solved recursively.
Conquer: to patch together the two solutions of the subproblems, perform a small amount of additional work, and
arrive at a solution for the whole problem
For divide-and-conquer approach, the sub-problems
should be independent.

Fibonacci number problem cannot be solved by divide-andconquer: Fn = Fn-1 + Fn-2
7
Divide and Conquer

Example: find the maximum among N items
stored in an array a[0], …, a[N-1]
// a common and simple solution
for (t = a[0], i=1; i < N; i++)
if (a[i] > t)
t = a[i];
It’s used to show the concept.
Its performance is not as good
as the left one.
// divide and conquer
int max( int a[], int l, int r ) {
int m, u, v;
if (l == r) return a[l];
m = (l+r)/2;
u = max(a, l, m);
v = max(a, m+1, r);
if (u > v)
return u;
else
return v;
}
8
Divide and Conquer

Example: Fast Exponentiation

Calculate XN where N is an unsigned integer
int Exp( int x, unsigned int n ) {
int temp;
if (n == 0) return 1;
if (n == 1) return x;
if (n % 2 == 0) {
temp = Exp(x, n/2);
return temp * temp;
} else {
temp = Exp(x, n/2);
return temp * temp * x;
}
}
9
Divide and Conquer


Example: binary search
Task: search an item in a sorted array
BinarySearch(int A[], int value, int low, int high) {
int mid;
if (high < low) return -1; // not found
mid = (low + high) / 2;
if (A[mid] > value)
return BinarySearch(A, value, low, mid-1);
else if (A[mid] < value)
return BinarySearch(A, value, mid+1, high);
else
return mid; // found
}
10
Divide and Conquer

Example: Mergesort


A stable sorting algorithm with worst-case running
time of O(NlogN)
Algorithm:



Divide the list into two sub-lists
Sort each sub-list (recursion)
Merge the two sorted sub-lists
11
Mergesort
void msort(int A[], int temp[], int left, int right) {
int mid;
if (left < right) {
mid = (left + right) / 2;
msort(A, temp, left, mid);
msort(A, temp, mid+1, right);
merge(A, temp, left, mid+1, right);
}
}
void mergesort(int A[], int N) {
int *temp = malloc(N * sizeof(int));
if (temp != NULL) {
msort(A, temp, 0, N-1);
free(temp);
}
}
O(N)
12
Backtracking

Backtracking is a systematic method to
iterate through all the possible configurations
of a search space.
Traversal using
right-hand rule
13
Ref: http://en.wikipedia.org/wiki/Maze
Backtracking



Given a problem, it may have a large number
of different possible solutions, called “search
space”, and your task is to find one correct
solution or all correct solutions.
Brute force strategy: go through all possible
solutions, and check them one by one
Today’s CPU (e.g., 3x109Hz) should be able
to handle the problems with search space
size of millions to billions.
14
Backtracking




Size of search space is usually related to the
number of variables of the problem and the possible
values of each variable
E.g., given 20 variables, each one can be either 0 or
1, then we have 220 different candidate solutions,
which is around 1 million (106)
E.g., given 12 variables, each one can be 0, 1, 2, …,
9, then we have 1012 different candidate solutions! --- brute force may not be a good idea for ACM
contest
E.g., given 12 variables, which are the permutation
of 1, 2, 3, …, 12, then we have 12! = 479,001,600
different candidate solutions. Brute force may work!
15
Backtracking

How to iterate through all the possible configurations of a
search space?



Recursion is the answer
Assume the problem has n variables, stored as a vector a = (a1,
a2, …, an), and each variable ai is selected from a finite
ordered set Si.
Backtracking:
1.
2.
3.
Process a partial solution (a1, a2, …, ak). If k == n, it is a complete
solution and we should handle it. Otherwise, goto Step 2.
Add one more variable ak+1, find the candidates for ak+1, i.e., Sk+1.
For each possible value of ak+1, process (a1, a2, …, ak+1) by recursion
16
Search Space

E.g., we have 4 variables, each could be 0 or
1 or 2
start
a1
0
a2
0
a3
a4
0
0
1
1
1
1
2
2
2
2
17
Backtracking

A general programming structure
bool finished = FALSE; /* to control when to stop */
backtrack(int a[], int k, data input ) {
int candidate[MAX]; /* candidates for next variable */
int ncandidates;
/* number of candidates for next variable */
int i;
if ( is_a_solution(a, k, input) )
process_solution(a, k, input);
else {
k++;
construct_candidates(a, k, input, candidate, &ncandidates);
for (i = 0; i < ncandidates; i++) {
a[k] = candidate[i];
backtrack(a, k, input);
if (finished) return; /* we can terminate early by setting flag finished */
}
}
}
18
Some Comments

is_a_solution(a, k, input)



construct_candidates(a, k, input, c, ncandidates)



Test whether the first k elements of vector a are a complete
solution for the given problem
Argument input allows to pass general information into the
routine, e.g., the size of a target solution. It could be ignored in
some cases.
Fill an array c with the complete set of possible candidates for the
kth position of a, given the contents of the first k-1 positions.
The number of candidates returned is denoted by ncandidates
process_solution(a, k)

Process a complete solution once it is constructed
19
Example 1:
Constructing All Subsets


Given a set with n items, we have 2n subsets. How
to output all the subsets?
E.g., the subsets of {1, 2, 3} include








{123}
{12}
{13}
{1}
{23}
{2}
{3}
{}
Introduce three variables, a1, a2, a3. Each of them
can be either true or false.
ai is true means that i is in the subset.
20
Constructing All Subsets
is_a_solution(int a[], int k, int n) {
return (k == n);
}
process_solution(int a[], int k, int n) {
int i;
printf(“{ “);
for(i = 1; i <= k; i++) {
if (a[i] == TRUE)
printf(“ %d”, i);
printf(“ }\n”);
}
Reminder: In this example, the first variable is a[1]. a[0] is not used!
21
Constructing All Subsets
construct_candidates(int a[], int k, int input, int c[], int *n) {
c[0] = TRUE;
c[1] = FALSE;
*n = 2;
}
/* output all the subsets of {1, 2, 3} */
int main()
{
int n = 3;
int a[4];
backtrack(a, 0, n);
return 0;
}
22
Trace
backtrack( ( $, $, $), 0, 3 )
backtrack( ( T, $, $), 1, 3 )
backtrack( ( T, T, $), 2, 3 )
backtrack( ( T, T, T), 3, 3 )
{1 2 3}
$: empty
T: TRUE
F: FALSE
backtrack( ( F, $, $), 1, 3 )
backtrack( ( T, F, $), 2, 3 )
backtrack( ( T, T, F), 3, 3 )
{1 2}
backtrack( ( T, F, T), 3, 3 )
{1 3}
backtrack( ( T, F, F), 3, 3 )
{1}
23
Stone Pile
Time Limit: 2.0 second Memory Limit: 16 MB
You have a number of stones with known weights W1…Wn. Write a program that
will rearrange the stones into two piles such that weight difference between the
piles is minimal.
Input contains the number of stones N (1 ≤ N ≤ 20) and weights of the stones
W1…Wn (1 ≤ Wi ≤ 100000) delimited by white spaces.
Your program should output a number representing the minimal possible weight
difference between stone piles.
Sample input
5
5 8 13 27 14
output
3
24
Example 2:
Constructing All Permutations

Permutation of {1, 2, 3} include






123
132
213
231
312
321
Difference from the previous example:
When constructing the candidates for the next move,
we must ensure that the ith element is distinct from
all the elements before it.
25
Constructing All Permutations
is_a_solution(int a[], int k, int n) {
return (k == n);
}
process_solution(int a[], int k, int n) {
int i;
for(i = 1; i <= k; i++)
printf(“ %d”, a[i]);
printf(“\n”);
}
26
Constructing All Permutations
construct_candidates(int a[], int k, int input, int c[], int *n) {
int i;
int in_perm[NMAX];
for( i = 1; i < NMAX; i++) in_perm[i] = FALSE;
for( i = 1; i < k; i++) in_perm[ a[i] ] = TRUE;
*n = 0;
for (i = 1; i <= input; i++)
if (in_perm[i] == FALSE) { /* candidates must be */
c[*n] = i;
/* different from existing */
*n = *n + 1;
/* elements */
}
}
int main()
{
int n = 3;
int a[4];
backtrack(a, 0, n);
return 0;
}
27
Example 3:
Eight-Queens Problem




The eight queens puzzle is the problem of putting
eight chess queens on an 8×8 chessboard such that
none of them is able to capture any other using the
standard chess queen's moves.
A solution requires that no two queens share the
same row, column, or diagonal.
How many solutions?
http://en.wikipedia.org/wiki/Eight_queens_puzzle
28
Eight-Queens Problem



What is the search space?
If a queen can be placed at any square, the search
space will be 648. Too large!
There are several constrains, which help to reduce
the size of search space significantly.


pruning
Consider the queen on the first column, it has 8
choices. Once it has been fixed, consider the queen
on the second column, which has 7 choices. It’s
easy to see that we have a total of 8! = 40320 cases
only.

We haven’t used the diagonal constrain yet.
29
Eight-Queens Problem
is_a_solution(int a[], int k, int n) {
return (k == n);
}
process_solution(int a[], int k, int n) {
solution_count++;
// global count variable
}
int solution_count;
int main()
{
int n = 8; int a[9];
solution_count = 0;
backtrack(a, 0, n);
printf(“%d\n”, solution_count);
return 0;
}
30
Eight-Queens Problem
construct_candidates(int a[], int k, int input, int c[], int *n) {
int i, j;
int legal_move;
*n = 0;
for (i = 1; i <= input; i++) {
legal_move = TRUE:
for (j = 1; j < k; j++) {
if ( abs(k-j) == abs(i-a[j]) ) legal_move == FALSE; /* diagonal threat */
if ( i == a[j] ) legal_move == FALSE;
/* column threat */
}
if (legal_move == TRUE) {
c[*n] = i;
*n = *n +1;
}
}
31
}