Artificial Intelligence, HW 1 – Search, External Documentation
Prinz Amit 039668603
Rubin Oren 034107581
Please return to box number 39
1. General
We’ve implemented the search in Java (version 1.5). The most “interesting” class
is State, which represent a certain state in our search. For every type of search
(Informed and Uninformed) there’s a class respectively that implements it in its
main() method. The test suite is generated by the class Test, and the beam is
implemented by class SortedList. For a detailed explanation on the structure and
implementation of these classes, see the inner documentation. (This external
documentation does not delve into implementation details). Please note that all
code shown in this HW was written by yours truly.
2. Terminology
a. Bead – A ball. A bead color is either one of azure, green, red or violet.
b. Ring – Strictly speaking, a vector of 20 beads.
c. State – composed mostly by two rings – a left ring and a right ring.
d. Shared Bead – A bead that is shared by both the left and the right ring in a
state. A state has two shared beads – the upper shared bead and the lower
shared bead.
3. The Ring class
a. General – The ring class is an inner class of the State class. It represents a
ring.
b. A Ring instance is not aware whether it represents the left or right ring of
a state. A ring does hold which beads are its shared beads.
4. The State class
a. General - The State class represent a certain state in the search – that is, a
certain combination of beads in the two rings.
b. A state may be an initial state, final state, or some intermediate state which
is neither initial nor final. An intermediate state is achieved by applying a
certain series of rotations on an initial or final state).
c. Ring.lastRingMoved_ – this field can contain 3 values {LEFT, RIGHT,
DON’T_CARE, NIETHER_LEFT_NOR_RIGHT}
Since, by problem definition, there’s no point in rotating the same ring
twice in succession, it is important that a state will remember which ring
was rotated last. This helps us avoid visiting states we’ve already visited.
The last ring to be rotated is kept in the lastRingMoved_.
We should note that the successors are created by rotating the ring which
is not specified in lastRingMoved_ (this accurse in successor () method).
However, this creates some complication when testing for equality
between states. See the inner documentation of the DONT_CARE value.
d. State.h() – The State class hold our heuristic function.
for a State s, h(s) == h1(s) – h2(s), where
h1(s) == the number of maximal-length color sequences in both of the
rings. A color sequence is a succession of beads in a ring of the same
color. A maximal-length color sequence is a color sequence that is
contained only in itself.
A simple way to explain maximal-length color is this: when iterating
sequentially over a ring (the 20 beads), count the times the color changed
(add one when current color is different than the previous color).
For example, if s is a final state, h1(s) = 6, since in every ring there’s 3
maximal-length color sequences.
h2(s) == the discrepancy of both rings. The discrepancy of a ring is
max {color of bead of color c1 + color of beads of color c2} .
c1,c 2,c1c 2
In other words:
On each ring we find which 2 colors have the maximal number of beads,
we sum the two counts, that answer is denoted "the discrepancy"
The minimal value of h1 is 6 and of h2 is 38. Note that only final state can
get both of these values and get heuristic value of -32.
The complexity of the h() is O(RING_SIZE). Strictly speaking, since
RING_SIZE is constant, the complexity of h() is O(1).
Our heuristic function has one more option. After calculating the heuristic
value of a state s, s can get 10 "bad merit" points if :
1. On one of his rings there are exactly two beads of a certain color.
2. The distance between them is odd
* our initial tests proved that this makes the beam search much more
effective.
5. Uninformed Search
a. The BFS open list allows duplicates. We believe the overhead for not
allowing duplicates in the open list is not worth the time saved for the
BFS. Instead of not allowing duplicates in open, we can simply increase
the MAX_MEMORY_USED constant. At any rate, we end up with an
amount of nodes which is very close to MAX_MEMORY_USED.
b. A tricky thing to notice in the uninformed search is that for the state the
IDFS returns as a solution x, we have a path from x to the initial state, and
from x to the final state, which are in a reversed direction. The
State.makeSolutionPath() method deals with that.
6. Informed Search
a. The beam search is implemented with a close list, in which no duplicated
are allowed. We believe that this is critical, since we are likely to return to
nodes with a low heuristic value, and this might make the beam search
enter a loop.
Test Results:
Here follows the results of our test suite (test seed = 0). For every type of search there are
3 tables, one for every difficulty level. The difficulty levels are 3, 5 and 7. In every table,
for every metric computed there are 3 columns – one for every choice of parameter, from
the lowest to the highest.
Uninformed search:
The time column for the uninformed search does not include the BFS time, since the BFS
was computed once for every parameter of the test ( MAX_MEMORY_USED).
The time, in milliseconds, it took to compute the BFS for every parameter choice:
MAX_MEMORY_USED
TIME
1000
62
10000
250
100000
5641
Difficulty level 1: 3 rotations
Nodes developed
Time (sec) Solution
Memory used
Test #
length
1
1343 10343 100039 0
0 0 3
3 3 1069 10056 100000
2
1438 10438 100039 0
0 0 3
3 3 1069 10056 100000
3
1533 10039 100039 0
0 0 3
3 3 1069 10039 100000
4
1400 10400 100039 0
0 0 3
3 3 1069 10056 100000
5
3395 10172 100039 0
0 0 3
3 3 1086 10056 100000
6
4972 10248 100039 0
0 0 3
3 3 1086 10056 100000
7
5580 10267 100039 0
0 0 3
3 3 1086 10056 100000
8
1514 10514 100039 0
0 0 3
3 3 1069 10056 100000
9
1324 10324 100039 0
0 0 3
3 3 1069 10056 100000
10
1115 10115 100039 0
0 0 3
3 3 1069 10056 100000
average 2361 10286 100039 0
0 0 3
3 3 1074 10054 100000
Conclusions for this test suite: For low difficulty level problems, the max memory used is
insignificant – it doesn't help us to get to a solution faster (and the BFS takes longer). The
uninformed search solves these easy problems in negligible time.
Difficulty level 2: 5 rotations
Nodes developed
Time
Solution
Memory used
Test #
(msec)
length
1
135027 17088 107088 0 0 0
5 5 5 1103 10073 99878
2
238539 246019 112522 0 0 0
5 6 5 1103 10090 99878
3
239508 22560 112560 0 0 0
5 5 5 1103 10073 99878
4
58305 13060 103060 0 0 0
5 5 5 1103 10073 99878
5
216765 225765 111382 0 0 0
5 5 5 1103 10090 99878
6
5299151 288864 114688 8 0 0
5 5 5 1120 10090 99878
7
30451 39451 101597 0 0 0
5 5 5 1103 10090 99878
8
129669 16803 100381 0 0 0
5 5 5 1103 10073 99861
9
34897 11825 100134 0 0 0
5 5 5 1103 10073 99861
10
253131 23282 113282 0 0 0
5 5 5 1103 10073 99878
average 663544 90471 107669 0 0 0
5 5 5 1104 10079 99874
Conclusions for this test suite: Like conclusions for test suite with difficulty level 1
above. However, here one can see that the DFS starts to reach a depth that takes a nonnegligible amount of time (test 6, lowest choice of parameter).
Difficulty level 3: 7 rotations
Nodes developed
Test #
1
2
3
4
5
6
7
8
9
10
average
30635651
64412578
63830418
92827173
14571322
67146298
17957084
13161394
74898678
95113443
53455403
16223398
3400133
3369486
9286173
14580322
67155298
17966084
69280486
74907678
95122443
37129150
Time (msec)
1712398
278468
3459486
4985623
866917
3634020
1045099
3745854
4042007
5105950
2887582
49
103
101
148
23
107
28
2114
119
152
294
2
5
6
176
24
111
29
114
123
159
74
2
0
5
7
4
5
1
5
6
9
4
Sol.
Len.
7 7
7 7
7 7
7 7
7 7
7 7
7 7
7 7
7 7
7 7
7 7
Memory used
7
7
7
7
7
7
7
7
7
7
7
1137
1137
1137
1137
1137
1137
1137
1154
1137
1137
1137
10107
10107
10107
10124
10124
10124
10124
10124
10124
10124
10118
Conclusions for this test suite: Here we can see that the time invested in the BFS paysoff. When we invest 5 seconds in performing the deepest BFS (which only reaches a
distance of 2-3 rotations from the final states), we earn a lot more in the DFS time. Also,
one can see how much costly can deepening the IDFS be – test number 5 takes the same
amount of time for both BFS size of 1000 and 10000.
99912
99895
99912
99912
99912
99912
99912
99912
99912
99912
99910
Conclusions for the uninformed search: For the hard problems, the BFS time is wellinvested – it saves very expensive IDFS iterations. Also, the uninformed search is not
practical for harder problem (say, of difficulty levels above 13).
Informed search (beam)
The three choices for the maximal beam size are: 1000, 10000 and 100000.
The Memory used column was computed in this manner: if the number of nodes
developed was lower than the maximal beam size, then memory used is equal to nodes
developed. Otherwise, memory used is the beam size plus the close list size.
Difficulty level 1 : 3 rotations
Nodes developed
Time (msec)
Solution
length
Test #
1
2
57
Memory used
57
57
57
57
0.1
0
0.1
0
0.1
0
3
3
3
3
3
3
57
57
57
57
57
57
57
57
0
0
0
3
3
3
57
57
57
57
57
0
0
0
3
3
3
57
57
57
57
57
0
0
0
3
3
3
57
57
57
57
57
57
57
0
0
0
0
0
0
3
3
3
3
3
3
57
57
57
57
57
57
57
57
57
57
57
57
57
57
0
0
0
0
0
0
0
0
0
0
0
0
3
3
3
3
3
3
3
3
3
3
3
3
57
57
57
57
57
57
57
57
57
57
57
57
57
3
57
4
57
5
6
7
8
9
10
average
57
57
57
57
57
57
57
Conclusions for this test suite: The heuristic is quite effective for easy problems.
Difficulty level 2 : 5 rotations
Nodes developed
Time (msec)
Test #
1
2
133
4921
133
9196
133
0
9196 0.1
3 231097 207233 182153
8
0
0.1
Solution
Memory used
length
0
5 5 5
133
133
0.1
20 20 20 1259 1484
133
9196
11 24.7
77 33 39 13092 11802 109535
4
2953
8246
82466 0.1
0.1
0.1
25 25 25
1487
1434
8246
5
2348
5548
5548 0.1
0
0
23 33 25
1492
1292
5548
6
95
7 132107
95
70243
95
0
38988 2.3
0
2.5
0
0.7
5 5 5
82 74 40
95
6633
95
4644
95
38988
95
95
95
30097
95
95
95
31886
0
0
0
5 5 5
0
0
0
5 5 5
0
0
0
5 5 5
1 1.37 2.56 25.2 21 17
95
95
95
2447
95
95
95
2116
95
95
95
17202
8
9
10
average
95
95
95
37393
Conclusions for this test suite: Here the heuristic is still effective – we usually reach the
solution quite fast. However, we begin to see erratic behavior of the beam search – test
no. 3 takes a significantly larger amount of time.
Difficulty level 3 : 7 rotations
In this difficulty level, the beam search failed in the 9th test – we got an exception that the
VM was out of memory. The average are computed according to the 9 successful tests.
Nodes developed
Time (msec)
Solution length
Memory used
Test #
1 106343 85652 96938
2.3
3.4
4
72
72
72 6582 14499 96938
2
8208
0.1
0.1
1
15
25
25 1460 8208
8203
8740
8208
3
133
133
133
0
0
0
7
7
7 133
133
133
4 174173 257222 184756
3.5
12
23
88
84
82 8599 21015 107909
5 57931 46132 53580
1
1
1
61
55
61 4049 12428 53580
6 15238 12920 13015
0.2
0.2
0.1
32
32
32 1802 10680 13015
7
133
133
133
0
0
0
7
7
7 133
133
133
8 25194 43396 31217
0.3
1.3
0.6
43
43
43 2325 12264 31217
9
out- out- out- out- out- out- outoutout-of- out-of- out-ofofofofofofofofof- out-ofmem
mem
mem mem mem mem mem mem mem mem mem
mem
10 178410 85196 554014
3.5
3 133
84 114 100 8662 12541 122876
average 62921 59888 104666
1
2
18
45
48
47 3749 10211 48222
Conclusions for this test suite: Here we can see that the heuristic might be very good or
very bad. For some tests it's faster by far from the uninformed search. On the other hand,
it can fail for other tests, or just return a slower solution.
Conclusions for the informed search: The heuristic is erratic, but mostly it is successful.
In all difficulty levels, the smaller the beam size the faster the solution. This is because
the overhead for keeping a large beam doesn't worth the extra space, and all the extra
nodes we keep may "distract" us from our goal.
The solution path returned by the informed search can be far from optimal – sometimes
it's very long – up to 10 times the optimal solution path.
There's a great discrepancy in the memory used by the informed search for different tests.
Comparison
1. For problems with a low difficulty level, the beam search is better than the
uninformed search, even though the time both take is negligible (if the BFS time
is not taken into consideration). The BFS may develop many nodes which are not
used.
2. For problems with a medium difficulty level, the uninformed search takes a
negligible time, while the beam search, though quite fast, is relatively slow. This
is because the DFS doesn’t reach a depth that takes a significant time.
3. For problems with a high difficulty level, the beam search is usually much faster.
However, this is uncertain. There are problems that take the beam search more
time, and there are problems for which the beam search fails.
4. Also, from tests not shown above: The informed search may solve in a reasonable
time "very" hard problems – It managed to solve problems of difficulty level 13 in
a couple of minutes. This difficulty level is not feasible for the uninformed search.
Question 3:
a. The uninformed search is:
i. Complete – for every problem, we go over all of the possible paths
from it. Therefore, we are bound to reach a final state sooner or
later.
ii. Not optimal – the IDFS stops when it reaches one of the states
developed by the BFS, and not when it reaches a final state.
Because the number of nodes developed by the BFS is limited, the
leaves reached by the BFS (the nodes that don’t get developed)
may be in different distance from their final states ancestors.
Therefore, the IDFS may halt when it reaches one of the BFS
leaves that are farther from their final states ancestors, and the
resulting path is not necessarily optimal. However, from this
follows that the solutions path length is not bigger than the optimal
path by more than one.
b. The informed search is:
i. Not complete – some successors may be discarded before they get
developed (because their heuristic value is not good enough).
Therefore we have no guarantee that nodes that lead to a final state
will be developed.
ii. Not optimal – our tests contradicts it. There is no guarantee that the
optimal solution will be chosen. Ties (nodes with equal heuristic
value) are broken arbitrarily.