Sampling Combinatorial Space
Using Biased Random Walks
Jordan Erenrich, Wei Wei and Bart
Selman
Dept. of Computer Science
Cornell University
• Many forms of probabilistic reasoning can be
effectively reduced to sampling satisfying
assignments from a Boolean formula (an
instance of SAT).
• Question: Can state-of-the-art local search
procedures for SAT sample effectively from the
solution space? (as an alternative to standard
Monte Carlo Markov Chain methods)
Characteristics of Solution space:
Solution Clustering
• Visualization with multi-dimensional scaling (MDS)
– Solutions to specific 75 variable, 325 clause 3-SAT instances
– 75 dimensional solution projected to two dimensions
– Distance between points approximates hamming distance
Solution Probability
• Consider a simple 2-SAT problem
– x OR y
• Consider a simple SAT heuristic
– Starts with a random bit assignment
– Randomly flip a bit until a solution is found
• Consider the probability of finding each solution
–
x
y
Solution?
Solution Probability
0
0
No
N/A
0
1
Yes
3/8
1
0
Yes
3/8
1
1
Yes
1/4
Solution Probability Using WalkSat
Algorithm
• Empirically determined each solution’s probability (uf75-01
- 75 variable, 325 clause 3-SAT instance)
• WalkSat finds every solution, but with very large range of
probabilities (1:104)
• Probability Clusters
SA on SAT-75 - Multi-dimensional Scaling Using Hamming Distance Between Solutions
20
25 Most Often Found Solutions
25 Next Most Often Found Solutions
Middle Solutions
25 Next Least Often Found
25 Least Often Found Solutions
15
10
5
0
-5
-10
-15
-20
-25
-10
-5
0
5
10
15
20
25
Probability Ranges in Different Domains
Instance
Runs
Random
50*106
Hits
Rarest
53
Logistics
1* 106
84
Hits
Common-to
Common -Rare Ratio
9*105
1.7*104
4*103
50
Improving the Uniformity of Sampling
Mixed sampling strategy
• To reduce the range of probabilities, we
propose a hybrid local search algorithm:
– With probability p, the algorithm makes a
biased random walk move
– With probability 1-p, the algorithm makes a SA
(simulated annealing) move
• In our experiment, we used
– 50% WalkSat + 50% SA at a fixed temperature
Results of the Hybrid Approach
Our key figure.
Solution Clusters
Results on a random 3-SAT instance (70 vars, 301 clauses, 2531 solutions).
Summary
Proposal: Use SAT solvers to sample solutions from a
combinatorial space.
Findings:
1) WalkSAT does sample all solutions.
2) But, sampling can be highly biased.
3) Using a new hybrid strategy, we can obtain
effective near-uniform sampling.
Lesson: Hybrid of SA and biased walk, is a
promising alternative to MCMC methods
for sampling.