Integrating Different Ideas Together
 Reading
Materials:
 Ch 3.6 of [SG]
 Contents:




Incrementing a Binary Counter
How many subsets, Representation
Printing All Subsets Problem
Exponential Time Algorithms
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 1
Four seemingly unrelated problems
P1: You are given a bit-array
A = (A[n−1], A[n−2], ... A[1] A[0]),
where each A[k] is 0 or 1.
Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n
objects, and a subset X of S.
How can we use the bit-vector A to represent the subset X?
P3: Using P1 and P2 above, or otherwise, give a simple
algorithm to generate and print all subsets of S.
P4: What is the running time of your algorithm?
[Note: Recall that there are 2n subsets altogether. (A proof
is provided by P1 and P2 above.)]
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 2
n-bit Binary Counter
x
A[4] A[3] A[2] A[1] A[0]
Cost
0
0
0
0
0
0
0
1
0
0
0
0
1
1
2
0
0
0
1
0
3
3
0
0
0
1
1
4
4
0
0
1
0
0
7
5
0
0
1
0
1
8
6
0
0
1
1
0
10
7
0
0
1
1
1
11
8
0
1
0
0
0
15
9
0
1
0
0
1
16
10
0
1
0
1
0
18
11
0
1
0
1
1
19
Hon Wai Leong, NUS
(CS5234, 28 Aug 2007) Page L3.3
Copyright © 2005-7 by Leong Hon Wai)
Incrementing a Binary Counter
 n-bit Binary Counter: A[0..n1]
x = A[n−1]⋅2(n−1) + . . . + A[1]⋅21 + A[0]⋅20
x  i0 A[i] 2 i
n 1
INCREMENT(A)

1. i  0
2. while i < length[A] and A[i] = 1
3.
do A[i]  0
⊳ reset a bit
4.
ii+1
5. if i < length[A]
6.
then A[i]  1
⊳ set a bit
Hon Wai Leong, NUS
(CS5234, 28 Aug 2007) Page L3.4
Copyright © 2005-7 by Leong Hon Wai)
Four seemingly unrelated problems
P1: You are given a bit-array
A = (A[n−1], A[n−2], ... A[1] A[0]),
where each A[k] is 0 or 1.
Given an algorithm for incrementing A by 1.
Note:
For UIT2201, only need to understand the first 2 slides on
the problem P1 (incrementing a binary counter).
(The other slides are included for info only.)
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 5
Worst-case analysis
Consider a sequence of n insertions. The
worst-case time to execute one insertion is
O(k). Therefore, the worst-case time for
all the n insertions is n · O(k) = O(n k).
In fact, WRONG! In fact, the worst-case
cost for n insertions is only Q(n) ≪ Q(n
k).
Let’s see why.
Hon Wai Leong, NUS
(CS5234, 28 Aug 2007) Page L3.6
Copyright © 2005-7 by Leong Hon Wai)
Tighter analysis
x
A[4] A[3] A[2] A[1] A[0]
Cost
Total cost of n operations
0
0
0
0
0
0
0
A[0] flipped every op
1
0
0
0
0
1
1
A[1] flipped every 2 ops n/2
2
0
0
0
1
0
3
3
0
0
0
1
1
4
4
0
0
1
0
0
7
5
0
0
1
0
1
8
6
0
0
1
1
0
10
7
0
0
1
1
1
11
8
0
1
0
0
0
15
9
0
1
0
0
1
16
10
0
1
0
1
0
18
11
0
1
0
1
1
19
Hon Wai Leong, NUS
n
A[2] flipped every 4 ops n/22
A[3] flipped every 8 ops n/23
…
…
…
…
A[i] flipped every 2i ops
(CS5234, 28 Aug 2007) Page L3.7
Copyright © 2005-7 by Leong Hon Wai)
…
n/2i
Tighter analysis (continued)
lg n 
n

Cost of n increments  
i


2
i 1
 1
 n  i  2n
i 1 2
 Q( n )
Thus, the average cost of each increment
.
operation is Q(n)/n = Q(1).
Hon Wai Leong, NUS
(CS5234, 28 Aug 2007) Page L3.8
Copyright © 2005-7 by Leong Hon Wai)
P2: Using n-bit-array
to represent subsets?
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 9
Copyright © by Leong Hon Wai
Four seemingly unrelated problems
P1: You are given a bit-array
A = (A[n−1], A[n−2], ... A[1] A[0]),
where each A[k] is 0 or 1.
Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n
objects, and a subset X of S.
How can we use the bit-vector A to represent the subset X?
Note:
For UIT2201, only need to understand bit-representation
method for problem P2.
(The other slides are included for info only.)
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 10
PS: (Power-Set Example) (1)
Problem: Given a set S with n elements.
How many subsets of S are there?
Stage 1: Understanding the Problem
PQ: What is the unknown?
[# subsets of S]
PQ: What is the data?
[A set S with n elements]
PQ: What is the condition? [Subsets of S. Need to count all of them.]
PQ: Is it sufficient?
[Yes? Can count one-by-one, but tedious]
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 11
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (2)
Stage 2: Devising a Plan
PQ: Have you seen the problem before?
PQ: Can you try to work out some small instances?
n
S
P(S)
0
ϕ
ϕ
1
1
{x1}
ϕ, {x1}
2
2
{x1, x2}
ϕ, {x1}, {x2}, {x1, x2}
4
3
{x1, x2, x3}
ϕ, {x1},
{x2},
{x1, x2}
{x3}, {x1, x3},{x2, x3}, {x1, x2, x3}
8
#
PQ: Can you see any pattern?
Stage 3: Carrying out the Plan. PQ: Can you prove the result?
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 12
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (3)
Another Approach?
PQ: Can you solve it differently?
PQ: Introduce suitable notations?
Let An be the # of subsets of a S with n element.
Now, consider the set S’ = S U {x} with n+1 elements.
Divide subsets of S’ into
(P1) those containing element x, and
(P2) those that do not contain element x.
Those in P2 are exactly all the subsets of S (and we have An of them).
For every subset T of S in P2,
there is a corresponding subset that contains x, namely T U {x} in P1.
Thus, there is a 1-1 correspondence betw subsets in P1 and those in P2.
Therefore, An+1 = 2  An for all n ≥ 0.
In addition, we know A0 = 1.
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 13
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (4)
Yet Another Approach?
PQ: Draw a figure?
a
b
c
Hon Wai Leong, SoC, NUS
a'
b'
c
'
b
b'
cc
c
c
c
'
'
Where are the subset?
c
'
(Creative Problem Solving) Page 14
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (5)
Yet Another Method?
PQ: Look at the unknown. [Subsets! They come in different sizes]
IDEA: Let’s count those of the same size!
How many of size 0?
How many of size 1?
How many of size 2?
...
How many of size k?
...
How many of size n–1?
How many of size n?
1
n
n(n-1)/2 = nC2
nC
k
n
1
n n
k  2
k 0
n
Total # of subsets is
Hon Wai Leong, SoC, NUS
n
(x  y)   x k y (n k )
k
k 0  
n
n
(Creative Problem Solving) Page 15
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (6)
Yet another method?
PQ: How can the subsets be represented?
Let the vector (b1, b2, …, bn) represent the subset T
where bk=1, if the element xk is in the set T, and 0 otherwise.
Subset
Bit-Representation
ϕ
(0,0,…,0,0)
{x1}
(1,0,…,0,0)
{x2}
(0,1,…,0,0)
{x1, x2}
(1,1,…,0,0)
{x1, x2, ..., xn}
(1,1,…,1,1)
PQ: How many such bit-strings are there?
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 16
Copyright © by Leong Hon Wai
Four seemingly unrelated problems
P1: You are given a bit-array
A = (A[n−1], A[n−2], ... A[1] A[0]),
where each A[k] is 0 or 1.
Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n
objects, and a subset X of S.
How can we use the bit-vector A to represent the subset X?
P3: Using P1 and P2 above, or otherwise, give a simple
algorithm to generate and print all subsets of S.
P4: What is the running time of your algorithm?
[Note: Recall that there are 2n subsets altogether. (A proof
is provided byP1 and P2 above.)]
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 17
Solution to P3 and P4
Outline of algorithm for P3:
Represent subsets using bit vector A
1. Start with A = (0, 0, . . . , 0, 0)
2. Repeat these steps
3. Increment the bit vector A (use algorithm from P1)
4. Print out the subset corresponding to A
5. until A = (1, 1, . . . , 1, 1)
Analysis (P4):
Steps 3,4 done 2n time; (there are 2n subsets!)
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 18
Copyright © by Leong Hon Wai
Solution to P3 and P4
Analysis (P4):
Steps 3,4 done 2n time; (there are 2n subsets!)
Step 3 takes time at most O(n)
Step 4 takes time at most O(n)
Total time: 2n * O(n) = O(n 2n)
This is an exponential time algorithm!
Lower Bound:
Since there 2n subsets;
Just printing them will take at least 2n time!
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 19
Copyright © by Leong Hon Wai
One Final Question
?
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 20
Copyright © by Leong Hon Wai
Exponential Time Problems…
 Intel
designs CPU chips for most of todays’
computers (IBM-compatible, Macs, Linux, etc)
 If
Intel wants to thoroughly verify their
multiplication circuit is correct (i.e. produces the
correct product for all input combinations), what
does it need to do?
 Check
that for every a, b compute and verify the
product (a * b)
 Do
you think that Intel has actually done that for
their 32-bit and 64-bit processors?
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 21
Thank you!
Hon Wai Leong, SoC, NUS
(Creative Problem Solving) Page 22
Copyright © by Leong Hon Wai
© Leong Hon Wai, 2003-2008
LeongHW, SoC, NUS
(UIT2201: AI) Page 23