Brute Force Approaches
• Techniques for finding optimal solutions to
hard problems “relatively” quickly
• Backtracking
• Branch and Bound
• Combining efficient solutions with brute
force approaches
• Other issues
The Knapsack Problem
• Input
– Capacity K
– n items with weights wi and values vi
• Goal
– Output a set of items S such that
• the sum of weights of items in S is at most K
• and the sum of values of items in S is maximized
Greedy Does Not Work
• What are possible greedy strategies?
– Highest Density First
– Highest Value First
– Lowest Weight First
• Prove these are not optimal for the 0-1
knapsack problem.
Multi-constraint Knapsack
• Input
– Capacity K1 and K2
– n items with integer weights wi, size si, and values vi
• Goal
– Output a set of items S such that
• the sum of weights of items in S is at most K1
• the sum of sizes of items in S is at most K2
• and the sum of values of items in S is maximized
• Can we use dynamic programming?
Situation where dynamic
programming does not work
What if our problem can’t be described with integers?
wA = 2
vA = $40
wB = 
vB = $50
wC = 1.98
vC = $100
wD = 5
vD = $95
wE = 3
vE = $30
We have to resort to brute force….
Brute Force
• Generate all possible solutions
– With n items, there are 2n solutions to be
generated
– Check each to see if they satisfy the constraint
– Save maximum solution that satisfies constraint
• Can be represented as a tree
Brute Force: Branching
A
In
Out
B
B
C
C
D
E
D
E
E
C
D
E
Weight = 15.12
Value = $315
E
D
E
E
C
D
E
Weight = 8.98
Value = $235
E
D
E
E
D
E
E
D
E
E
Weight = 9.98
Value = $225
E
Backtracking
• In the tree representation, we can think of the
previous algorithm doing a DFS in the tree
• If we reach a point where a solution no longer is
feasible, there is no need to continue exploring
• We can “backtrack” from this point and potentially
cut off much of the tree and many of the solutions
• In the given example, backtracking would be
much more effective if we had even more items or
a smaller knapsack capacity
2, $40
, $50
Backtracking
1.98, $100
5, $95
A
In
3, $30
Out
B
B
C
C
D
E
D
E
E
C
D
E
E
D
E
E
C
D
E
Weight = 8.98
Value = $235
E
D
E
E
D
E
E
D
E
E
Weight = 9.98
Value = $225
E
Branch and Bound
• We can backtrack if we know the best possible
solution in current subtree is worse than current
best solution obtained so far.
• Estimates for improvement given current ordering
–
–
–
–
–
A down can give $315
B down -> $275
C down -> $225
D down -> $125
E down -> $30
2, $40
, $50
Branch and Bound
1.98, $100
5, $95
A
In
Out
B
B
C
C
D
E
D
E
E
D
E
E
Weight = 7.12
Value = $190
3, $30
C
D
E
Weight = 8.98
Value = $235
D
E
C
D
E
Generating a good bound
• The key to branch and bound is
having a good initial bound.
• How might we generate a good
initial bound?
2, $40
, $50
Order of items
1.98, $100
5, $95
3, $30
• Since there is no fixed ordering of
items, is there a better way to order the
input items?
– Highest weight first
• Generate infeasible solutions quickly
– Highest density first
• Generate good solution quickly
– Which is better depends on input
D: 5, $95
B: , $50
Heaviest on Top
E: 3, $30
A: 2, $40
D
In
B
C: 1.98, $100
Out
B
(Max remaining = $220)
E
E
A
A
C
Weight = 8.14
Value = $145
A
C
C
A
C
Weight = 10.0
Value = $165
C
C
Weight = 9.98
Value = $225
Weight = 8.98
Value = $235
C: 1.98, $100
A: 2, $40
Best Density on Top
C
In
A
B
E
D
B
E
B
E
Weight = 8.98
Value = $235
B
E
B: , $50
E: 3, $30
Out
A
D
D: 5, $95
Greedy and Brute Force
• Suppose half the inputs to our knapsack
problem all have the same weight.
– If all inputs had the same weight, we could
implement a greedy solution
– However, since half do not, we cannot use
greedy alone to find optimal solution
• Combine brute-force approach with greedy
to find optimal solution more quickly.
The Breakdown...
F1
In
Out
F2
F2
F3
F4
F3
F4
F5 F5 F5 F5
F4
F3
F4
F4
F5 F5
F5 F5
F3
F4
Greedy
on half!
F4
F5 F5
F4
Algorithm
• Backtrack on half the inputs
• At leafs, apply greedy strategy on the other
half of the inputs
• Comparison
– Pure brute force: O(2n)
– Combination: O(n2n/2)
How much better?
• Assumptions
– Suppose n=50
– We can test 1,000,000 solutions/second.
• 250 would take over 35 years
• 225 can be generated in half an hour
– Plus marginal time to generate greedy solution
for each of the 225 solutions
Another combination
• Suppose half the inputs to our knapsack
problem have small integral weights/values
while the other half have real
weights/values.
• How can we combine approaches to solve
this efficiently?
The n-Queens Problem
• Input
– Positive integer n
• Task
– Place n queens on an n by n chessboard so that
no two queens attack each other (on same row,
column, diagonal), or report that this is
impossible
• Solve the n-queens problem for n = 1, 2, 3
n=4
• Pure brute force search
– 16 squares on a 4 by 4 chessboard
– Leads to 16 * 15 * 14 * 13 = 43,680 possible
solutions
• Improvements
– At most one queen per row: 44 = 256 possible
solutions
– Backtracking: If two queens already attack
before final queen placed, backtrack
Larger values of n
• n=8
– At most one queen per row: 88 = 16,777,216
– Early backtracking: 2057 nodes total
– Time to find first solution: 114 nodes
• n=12
– At most one queen per row: 1212
– Early backtracking: 856,189 nodes total
– Time to find first solution: 262 nodes
The Problem
In the U.S. navy, the SEALS are each specially trained in
a wide variety of skills so that small teams can handle a
multitude of missions.
If there are k different skills needed for a mission, and n
SEAL members that can be assigned to the team, find the
smallest team that will cover all of the required skills.
Andersen knows hand-to-hand, first aid, and camouflage
Butler knows hand-to-hand and snares
Cunningham knows hand-to-hand
Douglas knows hand-to-hand, sniping, diplomacy, and snares
Eckers knows first-aid, sniping, and diplomacy
Greedy Algorithm
• What is the obvious greedy algorithm for
this problem?
• Find a counter-example to the optimality of
greedy for this problem.
Brute Force Approach
• What is the brute-force approach?
• How can we simplify the problem as far as
possible in polynomial time?
• How can we set bounds on the solutions?
• When will we need to do backtracking?
• What order should we test the potential members
in when branching?