Artificial Intelligence, HW 1 – Search, External Documentation Prinz Amit 039668603 Rubin Oren 034107581 Please return to box number 39 1. General We’ve implemented the search in Java (version 1.5). The most “interesting” class is State, which represent a certain state in our search. For every type of search (Informed and Uninformed) there’s a class respectively that implements it in its main() method. The test suite is generated by the class Test, and the beam is implemented by class SortedList. For a detailed explanation on the structure and implementation of these classes, see the inner documentation. (This external documentation does not delve into implementation details). Please note that all code shown in this HW was written by yours truly. 2. Terminology a. Bead – A ball. A bead color is either one of azure, green, red or violet. b. Ring – Strictly speaking, a vector of 20 beads. c. State – composed mostly by two rings – a left ring and a right ring. d. Shared Bead – A bead that is shared by both the left and the right ring in a state. A state has two shared beads – the upper shared bead and the lower shared bead. 3. The Ring class a. General – The ring class is an inner class of the State class. It represents a ring. b. A Ring instance is not aware whether it represents the left or right ring of a state. A ring does hold which beads are its shared beads. 4. The State class a. General - The State class represent a certain state in the search – that is, a certain combination of beads in the two rings. b. A state may be an initial state, final state, or some intermediate state which is neither initial nor final. An intermediate state is achieved by applying a certain series of rotations on an initial or final state). c. Ring.lastRingMoved_ – this field can contain 3 values {LEFT, RIGHT, DON’T_CARE, NIETHER_LEFT_NOR_RIGHT} Since, by problem definition, there’s no point in rotating the same ring twice in succession, it is important that a state will remember which ring was rotated last. This helps us avoid visiting states we’ve already visited. The last ring to be rotated is kept in the lastRingMoved_. We should note that the successors are created by rotating the ring which is not specified in lastRingMoved_ (this accurse in successor () method). However, this creates some complication when testing for equality between states. See the inner documentation of the DONT_CARE value. d. State.h() – The State class hold our heuristic function. for a State s, h(s) == h1(s) – h2(s), where h1(s) == the number of maximal-length color sequences in both of the rings. A color sequence is a succession of beads in a ring of the same color. A maximal-length color sequence is a color sequence that is contained only in itself. A simple way to explain maximal-length color is this: when iterating sequentially over a ring (the 20 beads), count the times the color changed (add one when current color is different than the previous color). For example, if s is a final state, h1(s) = 6, since in every ring there’s 3 maximal-length color sequences. h2(s) == the discrepancy of both rings. The discrepancy of a ring is max {color of bead of color c1 + color of beads of color c2} . c1,c 2,c1c 2 In other words: On each ring we find which 2 colors have the maximal number of beads, we sum the two counts, that answer is denoted "the discrepancy" The minimal value of h1 is 6 and of h2 is 38. Note that only final state can get both of these values and get heuristic value of -32. The complexity of the h() is O(RING_SIZE). Strictly speaking, since RING_SIZE is constant, the complexity of h() is O(1). Our heuristic function has one more option. After calculating the heuristic value of a state s, s can get 10 "bad merit" points if : 1. On one of his rings there are exactly two beads of a certain color. 2. The distance between them is odd * our initial tests proved that this makes the beam search much more effective. 5. Uninformed Search a. The BFS open list allows duplicates. We believe the overhead for not allowing duplicates in the open list is not worth the time saved for the BFS. Instead of not allowing duplicates in open, we can simply increase the MAX_MEMORY_USED constant. At any rate, we end up with an amount of nodes which is very close to MAX_MEMORY_USED. b. A tricky thing to notice in the uninformed search is that for the state the IDFS returns as a solution x, we have a path from x to the initial state, and from x to the final state, which are in a reversed direction. The State.makeSolutionPath() method deals with that. 6. Informed Search a. The beam search is implemented with a close list, in which no duplicated are allowed. We believe that this is critical, since we are likely to return to nodes with a low heuristic value, and this might make the beam search enter a loop. Test Results: Here follows the results of our test suite (test seed = 0). For every type of search there are 3 tables, one for every difficulty level. The difficulty levels are 3, 5 and 7. In every table, for every metric computed there are 3 columns – one for every choice of parameter, from the lowest to the highest. Uninformed search: The time column for the uninformed search does not include the BFS time, since the BFS was computed once for every parameter of the test ( MAX_MEMORY_USED). The time, in milliseconds, it took to compute the BFS for every parameter choice: MAX_MEMORY_USED TIME 1000 62 10000 250 100000 5641 Difficulty level 1: 3 rotations Nodes developed Time (sec) Solution Memory used Test # length 1 1343 10343 100039 0 0 0 3 3 3 1069 10056 100000 2 1438 10438 100039 0 0 0 3 3 3 1069 10056 100000 3 1533 10039 100039 0 0 0 3 3 3 1069 10039 100000 4 1400 10400 100039 0 0 0 3 3 3 1069 10056 100000 5 3395 10172 100039 0 0 0 3 3 3 1086 10056 100000 6 4972 10248 100039 0 0 0 3 3 3 1086 10056 100000 7 5580 10267 100039 0 0 0 3 3 3 1086 10056 100000 8 1514 10514 100039 0 0 0 3 3 3 1069 10056 100000 9 1324 10324 100039 0 0 0 3 3 3 1069 10056 100000 10 1115 10115 100039 0 0 0 3 3 3 1069 10056 100000 average 2361 10286 100039 0 0 0 3 3 3 1074 10054 100000 Conclusions for this test suite: For low difficulty level problems, the max memory used is insignificant – it doesn't help us to get to a solution faster (and the BFS takes longer). The uninformed search solves these easy problems in negligible time. Difficulty level 2: 5 rotations Nodes developed Time Solution Memory used Test # (msec) length 1 135027 17088 107088 0 0 0 5 5 5 1103 10073 99878 2 238539 246019 112522 0 0 0 5 6 5 1103 10090 99878 3 239508 22560 112560 0 0 0 5 5 5 1103 10073 99878 4 58305 13060 103060 0 0 0 5 5 5 1103 10073 99878 5 216765 225765 111382 0 0 0 5 5 5 1103 10090 99878 6 5299151 288864 114688 8 0 0 5 5 5 1120 10090 99878 7 30451 39451 101597 0 0 0 5 5 5 1103 10090 99878 8 129669 16803 100381 0 0 0 5 5 5 1103 10073 99861 9 34897 11825 100134 0 0 0 5 5 5 1103 10073 99861 10 253131 23282 113282 0 0 0 5 5 5 1103 10073 99878 average 663544 90471 107669 0 0 0 5 5 5 1104 10079 99874 Conclusions for this test suite: Like conclusions for test suite with difficulty level 1 above. However, here one can see that the DFS starts to reach a depth that takes a nonnegligible amount of time (test 6, lowest choice of parameter). Difficulty level 3: 7 rotations Nodes developed Test # 1 2 3 4 5 6 7 8 9 10 average 30635651 64412578 63830418 92827173 14571322 67146298 17957084 13161394 74898678 95113443 53455403 16223398 3400133 3369486 9286173 14580322 67155298 17966084 69280486 74907678 95122443 37129150 Time (msec) 1712398 278468 3459486 4985623 866917 3634020 1045099 3745854 4042007 5105950 2887582 49 103 101 148 23 107 28 2114 119 152 294 2 5 6 176 24 111 29 114 123 159 74 2 0 5 7 4 5 1 5 6 9 4 Sol. Len. 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Memory used 7 7 7 7 7 7 7 7 7 7 7 1137 1137 1137 1137 1137 1137 1137 1154 1137 1137 1137 10107 10107 10107 10124 10124 10124 10124 10124 10124 10124 10118 Conclusions for this test suite: Here we can see that the time invested in the BFS paysoff. When we invest 5 seconds in performing the deepest BFS (which only reaches a distance of 2-3 rotations from the final states), we earn a lot more in the DFS time. Also, one can see how much costly can deepening the IDFS be – test number 5 takes the same amount of time for both BFS size of 1000 and 10000. 99912 99895 99912 99912 99912 99912 99912 99912 99912 99912 99910 Conclusions for the uninformed search: For the hard problems, the BFS time is wellinvested – it saves very expensive IDFS iterations. Also, the uninformed search is not practical for harder problem (say, of difficulty levels above 13). Informed search (beam) The three choices for the maximal beam size are: 1000, 10000 and 100000. The Memory used column was computed in this manner: if the number of nodes developed was lower than the maximal beam size, then memory used is equal to nodes developed. Otherwise, memory used is the beam size plus the close list size. Difficulty level 1 : 3 rotations Nodes developed Time (msec) Solution length Test # 1 2 57 Memory used 57 57 57 57 0.1 0 0.1 0 0.1 0 3 3 3 3 3 3 57 57 57 57 57 57 57 57 0 0 0 3 3 3 57 57 57 57 57 0 0 0 3 3 3 57 57 57 57 57 0 0 0 3 3 3 57 57 57 57 57 57 57 0 0 0 0 0 0 3 3 3 3 3 3 57 57 57 57 57 57 57 57 57 57 57 57 57 57 0 0 0 0 0 0 0 0 0 0 0 0 3 3 3 3 3 3 3 3 3 3 3 3 57 57 57 57 57 57 57 57 57 57 57 57 57 3 57 4 57 5 6 7 8 9 10 average 57 57 57 57 57 57 57 Conclusions for this test suite: The heuristic is quite effective for easy problems. Difficulty level 2 : 5 rotations Nodes developed Time (msec) Test # 1 2 133 4921 133 9196 133 0 9196 0.1 3 231097 207233 182153 8 0 0.1 Solution Memory used length 0 5 5 5 133 133 0.1 20 20 20 1259 1484 133 9196 11 24.7 77 33 39 13092 11802 109535 4 2953 8246 82466 0.1 0.1 0.1 25 25 25 1487 1434 8246 5 2348 5548 5548 0.1 0 0 23 33 25 1492 1292 5548 6 95 7 132107 95 70243 95 0 38988 2.3 0 2.5 0 0.7 5 5 5 82 74 40 95 6633 95 4644 95 38988 95 95 95 30097 95 95 95 31886 0 0 0 5 5 5 0 0 0 5 5 5 0 0 0 5 5 5 1 1.37 2.56 25.2 21 17 95 95 95 2447 95 95 95 2116 95 95 95 17202 8 9 10 average 95 95 95 37393 Conclusions for this test suite: Here the heuristic is still effective – we usually reach the solution quite fast. However, we begin to see erratic behavior of the beam search – test no. 3 takes a significantly larger amount of time. Difficulty level 3 : 7 rotations In this difficulty level, the beam search failed in the 9th test – we got an exception that the VM was out of memory. The average are computed according to the 9 successful tests. Nodes developed Time (msec) Solution length Memory used Test # 1 106343 85652 96938 2.3 3.4 4 72 72 72 6582 14499 96938 2 8208 0.1 0.1 1 15 25 25 1460 8208 8203 8740 8208 3 133 133 133 0 0 0 7 7 7 133 133 133 4 174173 257222 184756 3.5 12 23 88 84 82 8599 21015 107909 5 57931 46132 53580 1 1 1 61 55 61 4049 12428 53580 6 15238 12920 13015 0.2 0.2 0.1 32 32 32 1802 10680 13015 7 133 133 133 0 0 0 7 7 7 133 133 133 8 25194 43396 31217 0.3 1.3 0.6 43 43 43 2325 12264 31217 9 out- out- out- out- out- out- outoutout-of- out-of- out-ofofofofofofofofof- out-ofmem mem mem mem mem mem mem mem mem mem mem mem 10 178410 85196 554014 3.5 3 133 84 114 100 8662 12541 122876 average 62921 59888 104666 1 2 18 45 48 47 3749 10211 48222 Conclusions for this test suite: Here we can see that the heuristic might be very good or very bad. For some tests it's faster by far from the uninformed search. On the other hand, it can fail for other tests, or just return a slower solution. Conclusions for the informed search: The heuristic is erratic, but mostly it is successful. In all difficulty levels, the smaller the beam size the faster the solution. This is because the overhead for keeping a large beam doesn't worth the extra space, and all the extra nodes we keep may "distract" us from our goal. The solution path returned by the informed search can be far from optimal – sometimes it's very long – up to 10 times the optimal solution path. There's a great discrepancy in the memory used by the informed search for different tests. Comparison 1. For problems with a low difficulty level, the beam search is better than the uninformed search, even though the time both take is negligible (if the BFS time is not taken into consideration). The BFS may develop many nodes which are not used. 2. For problems with a medium difficulty level, the uninformed search takes a negligible time, while the beam search, though quite fast, is relatively slow. This is because the DFS doesn’t reach a depth that takes a significant time. 3. For problems with a high difficulty level, the beam search is usually much faster. However, this is uncertain. There are problems that take the beam search more time, and there are problems for which the beam search fails. 4. Also, from tests not shown above: The informed search may solve in a reasonable time "very" hard problems – It managed to solve problems of difficulty level 13 in a couple of minutes. This difficulty level is not feasible for the uninformed search. Question 3: a. The uninformed search is: i. Complete – for every problem, we go over all of the possible paths from it. Therefore, we are bound to reach a final state sooner or later. ii. Not optimal – the IDFS stops when it reaches one of the states developed by the BFS, and not when it reaches a final state. Because the number of nodes developed by the BFS is limited, the leaves reached by the BFS (the nodes that don’t get developed) may be in different distance from their final states ancestors. Therefore, the IDFS may halt when it reaches one of the BFS leaves that are farther from their final states ancestors, and the resulting path is not necessarily optimal. However, from this follows that the solutions path length is not bigger than the optimal path by more than one. b. The informed search is: i. Not complete – some successors may be discarded before they get developed (because their heuristic value is not good enough). Therefore we have no guarantee that nodes that lead to a final state will be developed. ii. Not optimal – our tests contradicts it. There is no guarantee that the optimal solution will be chosen. Ties (nodes with equal heuristic value) are broken arbitrarily.