From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
New Approaches
To Moving
Target
Search
Start
Melax
Department of Computing Science
University of Alberta
Edmonton, Alberta
Canada 114/,11 T6G2H1
melax@cs.ualberta.ca
Abstract
With a moving target, heuristic functions take
on the form h(a, b) -the predicted distance between
Newmethods for doing moving target search are pre- states a and b. The research presented here genersented. One algorithm, forgetful depth-first search, ally assumes the existence of a reasonably good and
attempts to adapt the well-known depth-first algo- admissible heuristic function. For the grid class of
rithm to this problem domain. Also, a search tech- graphs Manhattan distance is used. The Euclidean
nique called marking quickly acquires general knowl- distance can be used to break ties when multiple
edge about the search space. These methods are neighboring nodes have the same lowest heuristic.
One simple moving target search algorithm is for
discussed and compared with other known methods.
the solver to get on the target’s trail and follow it.
Experimental results show that forgetful depth-first
This only works for "slower" targets. At the opposearch and marking give good performance.
site end of the spectrum is Floyd’s algorithm[3] which
computes h*0 in O(n3) time. This algorithm can
be revised to work by increasing the working matrix
1 Introduction
starting from 0 or initial heuristic). This makes the
ntermediate computations useful (admissible). AfThe moving target search problem and solutions to ter this knowledge has been computed, the problem
it were initially introduced by Korf and Ishida in solver just f611owsthe direct path to the target.
their paper "MovingTarget Search" [1]. Ishida folThis paper reviews the work of Korf and Ishida,
lowed up this work in "Moving Target Search With introduces forgetful depth-first search and marking,
Intelligence"J2]. The problem is a variation of the and analyzes (with experimentation) the various alcommonsearch problem. There is a graph with a gorithms. This paper shows that the new methods
problem solver (3 looking for the goal node. What perform very well.
is different in this problem is that the goal node,
which we will refer to as the target ~, also moves.
Each entity knows the current position (node) of the
Other Work
other. Moves can only be made to adjacent nodes. 2
Whatmakes the problem interesting is that the solver
must make its decisions in a small constant amount of 2.1
Korf and Ishida
’91
time. To guarantee that it is possible to complete the
search, the target cannot always make optimal moves In the paper "Moving Target Search" [1], Korf and
Ishida introduce this search problem. In addition,
away from the solver. This condition could be satisfied with a slower moving target. Howeverthe search they present an algorithm by which a problem solver
problem is more interesting when the solver cannot can find the target.
depend on the target’s speed being slower than its
own. Instead the target makes "mistakes", such as 2.1.1
Original
Moving Target Search Algooccasionally making moves "toward" the solver.
rithm
Although the search problem and the algorithms
that are discussed are applicable to any graph, the Their basic moving target search algorithm (BMTS)
examples and experimentation presented in this pa- is an extension of the trivial "greedy" algorithm in
per uses grid type graphs with some of the nodes which the problem solver always moves to adjacent
blocked-out to prevent passage. Such graphs allow node with lowest heuristic value (ties broken ranfor much variation and are commonlyused for many domly). The reason it does not get stuck forever in
search problems.
a "local minimum"(Figure 1) is that it modifies its
3O
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
Figure 1: Local minimum(heuristic
4-6-
depression)
Figure 3: Adapting the heuristic function
n
solver will have to repeat the learning that previously
took place. This second problem is characterized as
"loss of information" due to target’s movement. In
a follow up paper, discussed later, Ishida addresses
these problems.
]
Q
2.1.3
Analysis
BMTSis complete under the assumption that the
target makes mistakes. The solver keeps increasing
his heuristic function (matrix) which, of course, cannot increase past the all-pairs shortest path matrix.
heuristic information using simple deduction. When Learning also depends on the rate in which the tarthe solver is at a non-goal node sl its true heuris- get makes "mistakes". The worst case is O(n3Te~r)
tic value (with respect to the target’s node t) must where T~r~ is the period of error (number of moves
be at least one greater than the least of its neigh- per mistake that the target makes).
BMTSdiffers from Floyd’s[3] because it concenbors s2. Therefore, when the actual heuristic does
trates its learning on correcting the heuristic values
not reflect this,it is updated (h(sl, t) h(s2, t) 1).
that pertain to the current positions of the target and
The solver does this each time it moves. Furthermore, the target’s moves (from tl to t2) are moni- the problem solver. In practice most of the heuristic
tored. If the heuristic value changes by more than 1 values are never used. Experimentation shows that
(h(s, tl) - h(s, t2) > 1) a similar heuristic update is actual search times are significantly better than the
made. The problem solver builds an n by n matrix worst case.
whose entries h(z, y) represent a lower bound on the
shortest path length between nodes x and y (Fig2.1.4
Completeness Questioned
ure 2). This representation of knowledgeis the same
The proof of completeness for BMTS
relies on the asas that used by Floyd.
sumption that occasionally the target moves in such
a way that the heuristic distance between the prob2.1.2
Heuristic
Depressions
lem solver and the target does not increase. Given
this, it seems reasonable for one to conclude that this
The BMTSproblem solver has great difficulty with criterion wouldbe satisfied if they use a target that,
heuristic depressions. Figure 3 shows how the prob- instead of occasionally skipping moves, would move
lem solver typically overcomes a misinformed heuris- semi-randomly or occasionally make moves that comtic situation such as a local minimum.The target is monsense dictates are "toward" the problem solver.
sitting still just on the other side of the wall at node However this is not so. In other words, the target
tl. The problem solver will moveback and forth along can move in such a way that reduces both the Manthe other side of the wall increasing the heuristic func- hattan distance (initial heuristic) and the shortest
tion h(X, tl) for states X on the problem solver’s
path distance (optimal heuristic h*0) but, accordside of the wall. O(k2) moves are required to get ing to the problem solver’s current heuristic informaout of a heuristic depression of k nodes. This ineffition, the heuristic disparity increases. Althoughthis
cient "thrashing" behavior occurs because the solver sounds possible, it seems likely that a target’s ranincrements heuristic information (usually by 1 or 2) dom move should have a good if not equal chance of
one node at a time. Figure 3 also illustrates another decreasing heuristic disparity for any given heuristic
problem. If just before the solver moves around the information. Unfortunately this is not the case. Exbarrier the target movesfrom state tl to t2, then the perimentation discovered a situation in which BMTS
Figure 2: All pairs shortest path matrix
31
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
bet of movestoward the goal) is called the degree of
commitment. The ideal value for the degree of commitment will depend on the search space. By experimentation on 100 by 100 search spaces with varying
densities, Ishida finds 10 to be a good value. Ishida
showsthat for the more difficult search spaces, (those
with 35 percent of the edges removed,) the performance improves by 5 to 10 times over the original
algorithm.
Figure 4: Stuck
2.2.2
builds up its heuristic information such that any legal
moveto an adjacent node that the target makes will
increase the heuristic disparity. Furthermore this is
not an unreasonable or isolated setup.
Figure 4 shows a situation where the solver is never
able to work its way out of a heuristic depression.
The target is programmed to move randomly but
only stay within the 3 by 2 area. Even after a very
long time the problem solver never moves out of the
heuristic depression marked by the rectangle. The
solver spends most of its time toward the left side of
the depression. In general, the size of a heuristic depression that a solver can tackle does depend on the
size of the area that the target movesabout.
Note that this problem only applies to bipartite
search spaces (no odd cycles). Possible remedies include using a slower target (in which case just following the target is better) or not using this class
of search spaces. Fortunately, the commitment enhancement introduced by Ishida (next section) will
restore completeness to searches with random targets
that do not skip moves.
2.2
Ishida
Even with commitment, the problem solver still
"thrashes" inside depressions as it incrementally updates its heuristic information. Ishida introduces a
mechanismcalled deliberation so that when the solver
enters a depression it switches to an off-line search
(similar to A*) in order to expand its search in all
directions. To maintain the real-time constraint, the
solver can only expand one node in off-line search per
regular move.
The solver limits the time spent deliberating to a
maximumnumber of nodes (called the degree of deliberation), unless it finds a way out of the depression
sooner. For easy search spaces the use of muchdeliberation is inappropriate since it allows the target to
get away as the solver remains stationary. Ishida’s
experiment with the value of 25 nodes seemed to
be the most promising with the more difficult search
spaces. The performance doubled in comparison to
using commitment alone. With respect to the original algorithm the improvement is between 10 and 20
times.
2.2.3
’92
In "Moving Target Search With Intelligence"[2],
Ishida presents improvementsto his and Korf’s original algorithm BMTS.Ishida introduces commitment
and deliberation to reduce the "information loss" and
"thrashing" problems discussed in the previous section.
Deliberation
Overall
Learning
Commitmentand deliberation significantly
improve
BMTS.However even with these revisions,
after
the problem solver’s immediate goal is satisfied, the
learning that has taken place is not likely to be beneficial again. For example consider a long grueling
search where the problem solver learns its way out
of a heuristic depression but later finds itself in the
same place. If at this time the target is in a differCommitment
2.2.1
ent position from where it was during the first time
that this happened, the problem solver will have to
In BMTS
the solver learns with respect to the target’s
current position. If the target did not move then learn its way out of this heuristic depression once
the solver could concentrate its learning and get out again. In short, learning seems to get lost in the
of heuristic depressions sooner. Ishida achieves this huge n by n heuristic matrix representing the probeffect using commitment. The solver ignores some lem solver’s knowledge. The essential reasoning proof the target’s moves, and instead concentrates on cess of Ishida’s revised problem solver remains the
a "goal" state which is occasionally updated to the same as its predecessor. Its knowledge domain still
deals with the collection of pairs of states. Ishida’s
target’s current position.
improvements are in the efficiency of the algorithm.
This goal state is updated whenever the solver
reaches it or when the solver makes a number of con- Learning is concentrated and localized, not generalsecutive moves toward it. This parameter (the num- ized.
32
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
~°
a__.o~
.h.S "kj’
List: b ab c gihijig cd
Figure 7: Problem solver’s
tree
Figure 5: Target wanders onto searched nodes
path and implicit
/::~i’~.:
h/
:::::::::::::::::::::::::::::::::::::: : : :
"\j’",
DFS
..~.,.
"~
=
List: c gihiji gc de fe
Figure 8: Samesearch 3 moveslater (with the first
moves removed)
Figure 6: Closed list inhibits movement
3
Forgetful
Search
Depth-First
This paper presents an algorithm with a radically different paradigm for searching for the moving target.
The objective is to capture the way depth-first search
(DFS) efficiently searches areas and quickly gets out
of heuristic depressions.
3.1
Problems
with
Traditional
that some previous piece of information must go. The
victim is the oldest piece of information, which is
the root of the search tree. The new root of the
search tree is the first child of the previous root. This
pruning occurs once for each move the solver makes
whether advancing onto a fresh node or backtracking over old ones. Figures 7 and 8 show the problem
solver’s path and the changes in the DFSsearch tree
as the search progresses.
DFS
Unfortunately, using normal depth-first search to
hunt a moving target would not be an effective strategy. One problem is that old information becomesinvalid. Nodes that have already been searched (those
on the solver’s closed list) are supposed to be places
where the target is not. However this is not true;
the target may move onto these nodes after they are
searched (Figure 5).
Another problem is that the closed list constructed
by the problem solver inhibits movement.This could
block efficient paths to the target (Figure 6).
3.3
Implementation
Such an algorithm seems complex and raises doubt as
to whether it can be done in real time. Fortunately
there is a simple implementation. The solver keeps
track of a list of visited nodes that allows duplicates.
Whenever the solver moves he adds the new node
onto the front of the list and removes the oldest node
at the back of the list. Whenthe solver is looking at
its neighboring nodes to determine which to moveto,
it selects one that is not currently in its list if possible. If the problem solver cannot moveto a node not
on its list, it movesto the predecessor of the oldest
3.2
Forgetful
Algorithm
occurrence on the list of the node it is currently at.
These setbacks are overcome using an algorithm
This mechanism for deciding which node to move to,
called forgetful depth-first search (FDFS). The solver along with maintaining the list of the last few nodes,
has limited memoryand cannot support a continually
achieves the desired forgetful depth-first search stratgrowing closed list. Whenthe solver moves onto a egy. Figures 7 and 8 also show the list and how it
node it will add it to his data structure. This means represents the implicit DFStree.
33
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
200
Search
Time
io:11oo~s
Figure 10: Marking nodes
Maximum
List Leagth300
Figure 9: Search times for varying list
100xl00
3.4
Target
tory
Trespasses
on Solver’s
maximums
His-
One special case to consider is when the target wanders onto any of the nodes that appear in the solver’s
list of nodes it has visited recently. Onceagain, a situation arises that is similar to the one shownback in
Figure 5. The problem solver must correct his current
information.
One method of dealing with this is to remove the
target’s node and all prior nodes from the solver’s list.
However,the list of nodes that remain nowrepresent
a path (not necessarily the best path) from the solver
to the target. Recall that the solver avoids nodes in
the list. The solution adopted is for the solver to
empty his list completely when the target wanders
onto any of the nodes on it.
Figure ll: Search space marked into regions
marking mechanism introduced in this section attempts to provide the problem solver with the ability
to learn efficiently and to maintain that knowledge
in a higher level representation than the traditional
all-pairs shortest path matrix.
4.1
Flat
Marking
The objective of marking is to factor the search space
into sections so that during the decision process the
An issue that arises with respect to using forgetful
solver can deal with collections of states instead of
depth-first
search is how much memoryshould the individual states. In particular, the problem solver
solver have, i.e. howlong should the list be. Exper- can mark a state with marking i if it is useless for
imentation has been done to determine a good max- the problem solver to move to that state unless the
imumlength for the solver’s list. Figure 9 shows re- target is at a state with the same marking i.
sults of normalized search times versus the maximum As stated, the prerequisite to marking a node is
allowable length. 100 by 100 search spaces were used that the solver’s ability to track the target is not inin the experimentation. In this case, as well as other hibited by doing so. Formally, a node can be marked
experiments (with other sizes), the search time tends if it does not lengthen the shortest path not involvto level out when the maximum
list length equals the ing such marked nodes between any two of the node’s
dimension of the search space. Therefore this was unmarked neighbors. For grid-type graphs, Figure 10
chosen as the default length for the forgetful depth- shows examples when nodes can be marked. A unique
first search algorithm.
marking number is distributed when a node is marked
and none of its neighbors are marked. The corner
node is marked because one can travel from one of
4 Marking
its neighbors to the other in two moveswithout going
through the corner node. Whenmarking a node with
Any real time learning technique strictly based on a marked neighbor normally the same marking value
using and updating the heuristic function falls vic- is used. The next node (to the right of the corner
tim to the huge n2 representation. Algorithms such node) can now be marked (with the same marking
as forgetful depth-first search mayhave an excellent as the corner node) because its non-marked neighstrategy for hunting down the target, however no bors have an alternate shortest path between them.
learning occurs. Consequently, the performance of Figure 11 shows how a search space would typically
the solver does not improve in the long term. The be marked after a problem solver explored it.
3.5
Amount
of
Memory
34
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
Advantage
of MultipleMarkings
===================================================
11.21
Hieaar~hie.al btar’~ag$yslcm
Figure 12: Multiple markings
UnmarkableSearch Space
Figure 13: Unmarkable search space
4.2
Hierarchical
Marking
Unfortunately there are some intuitive cases which
this flat marking system cannot handle. Figure 12
shows an example where a long "dead-end" hallway
cannot be conveniently marked off. The problem is
that adjacent nodes have different markings. The
solution is to mark the new node with a combination of its neighbors’ markings. In other words, each
node has a set of markings. Unmarkednodes can be
thought of as marked with all markings (complement
of the emptyset).
There is a potential danger in violating the realtime constraint of the search since a single node can
have many markings. Comparisons of large sets is
not a constant operation. Fortunately the size of such
multiple markings are typically small even for large
spaces. Even so, the real-time constraint can be guaranteed if the number of markings per node is limited
by a constant.
4.3
Marking
Limitations
functions or learning techniques. By itself, marking
would not be an effective strategy. Instead marking often reduces the number of neighboring nodes
that the solver must choose from. Because marking
is based only on deductive inferences, it is unlikely
that adding it will ever make a problem solver less
effective.
Even this marking strategy is not immune from
the fact that learning takes time. This means that
the solver using marking will still enter "markable"
heuristic depressions. However, it will only enter
them at most once. Consequently, marking will make
more of a difference in search times when the search
is long and difficult. In this setting the solver will
visit areas of the search space manytimes enabling it
to take advantage of previous learning.
5
Marking unfortunately does not solve every problem.
Figure 13 shows a search space for which this marking
strategy is helpless. The graph is two-connected and
every 2 by 2 collection of nodes has at least one of
them blocked out. Consequently the problem solver
cannot mark anything. On the other hand, simple
humanintuition would easily divide the space into
appropriate sections. A more advanced mechanism
that can look at more than just one node’s neighbors
is needed to do marking of this nature.
Experimentation
Experimentation of the various algorithms was done
to help understand and compare them.
5.1
Variance
in
Experimentation
One note concerning the experimentation is that
there are large amountsof variance in the time needed
to complete the search. One place where variance
shows up is in the difficulty of search spaces. Two
4.4
Using
Markings
graphs with the same percentage of nodes randomly
The problem solver uses marking information as it blocked out may differ by an order of magnitude in
decides where to move. It is useless for the solver the average time needed for the problem solver to
to move onto a neighboring node that is marked un- reach the target. Variance also occurs for different
less the target’s node is marked similarly under the runs using the same search space. Sometimes when
flat marking system. Whenthe hierarchical mark- search parameters are varied slightly there is a large
ing system is used, the solver should not moveonto a change in the search time; at other times, there is no
node unless its markings are a superset of the target’s difference. Such variance results from the high degree
node’s markings.
of sensitivity that can exist for individual moves. If
This marking technique may augment any moving the solver makes a "wrong turn" he may end up in a
target search strategy as easily as adding a heuristic large heuristic depression instead of on a direct path
function. Marking does not replace existing heuristic
to the target.
35
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
=
.......
.,
_
= ......
= =
RAND
BMTS
CMTS
IMTS
FDFS
Marking
AVOID
BMTS
CMTS
IMTS
FDFS
Marking
Figure 14: Stationary target search scenario
Algorithm
BMTS
FDFS
Marking
First
Round
105
37
37
Round 10
17
37
23
reed
rain
11174 10485
101
109
2110
915
1700
512
109
876
388
93
604
383
93
rain
mean
reed
9176
7300 135
123
2037 1162
512
112
1476
808
474
105
619
461
105
mean
max
>20000
>20000
>20000
>20000
>20000
max
>20000
>20000
>20000
>20000
>20000
over
2964
91
142
19
1
over
1844
20
77
4O
7
Figure 16: Algorithm Performance
5.3
Performance
Evaluation
The previous experiment illustrates
the behavior of
the various problem solvers. The experimentation in
this section attempts to measure the ability of these
algorithms.
100 by 100 toroidal (wrap-around) search spaces
5.2
Small Space
and Stationary
Tarwith a density of 35 percent were used. (Each node
has a 35 percent probability of being blocked out.)
get
Although not representative of all possible graphs,
these search spaces are challenging and appropriate
The first collection of experiments presented is based for this moving target search problem. At the beon the graph in Figure 14. The problem solver must ginning of each search the solver and the target are
work its way out of a heuristic depression in or- placed a maximumdistance (50 rows and 50 columns)
der to find a non-moving target. The main purpose apart.
Ten thousand randomly generated graphs (using
of this experimentation is to become familiar with
the nature of each of the algorithms. Algorithms the above parameters) were used for testing each of
tested include Korf and Ishida’s original movingtar- the problem solvers. The mean, median, and range
are recorded. Searches lasting longer than 20000
get search algorithm (BMTS), forgetful depth-first
search (FDFS), and the hierarchical marking tech- moves were halted and 20000 was recorded for the
nique combined with forgetful. The results are shown search. The number of searches exceeding this limit
is also recorded.
in Figure 15.
Two types of targets were tested against. One of
The second column of Figure 15 shows the time re- them (RAND)behaves completely randomly giving
quired for each of the algorithms to find the target.
equal probability to each neighboring node when deClearly BMTSdoes not do very well. This problem ciding its next move. The other target (AVOID)gives
solver keeps moving back and forth until it finally
higher probability to nodes away from the problem
builds up its heuristic information. With the stationsolver thus tending to avoid the solver.
ary target, adding marking to forgetful depth-first
Experimentation was done for BMTS(Ishida’s and
search does not improve performance.
Korf’s original algorithm), the same with Ishida’s
(CMTS),
The third column of Figure 15 shows the num- commitment technique but no deliberation
algorithm with commitmentand deliberation
ber of moves required a few rounds later (acquired Ishida’s
knowledge is maintained between rounds), which (IMTS), forgetful depth-first search (FDFS), and
helps illustrate the learning that occurs during these getful with the marking technique. The results are
searches. For every round after the first 10, BMTS shown in Figure 16. Note that the mean search times
for BMTSare deflated because of the large number
performs optimally heading directly from the start
state to the goal state. During later rounds FDFS (almost 30 per cent) of truncated searches.
For CMTSand IMTS the degree of committment
(which does not adapt)requires the same number
moves that it did in the first round. This is where used was 10 which is the same value that Ishida
marking pays off: after a few rounds the number of suggests in his paper. For the search spaces used
moves required drops but not quite as muchas it did in this experimentation, adding deliberation with a
maximumoff-line search of 25 nodes (as suggested
for BMTS.
Figure 15: Search Times
36
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
250£
Algorithm
Floyd
Follow
BMTS
Commitment
Deliberation
FDFS
Marking
Numbe
of
Moves
Knowledge Repn
all-pairs 2)
eO(n
target’s trail
all-pairs 2)
cO(n
all-pairs 2)
eO(n
all-pairs 2)
eO(n
own history
Regions cO(n)
Figure 18: Primary knowledge base of algorithms
Figure 17: Long term learning
Addable
Technique
by Ishida) yielded little improvement. Allowing for
more deliberation, up to 250 nodes, resulted in better
performance for IMTS. The results for IMTSshown
in Figure 16 are based on this. Using more deliberation was beneficial because the search spaces were
very dense and had huge heuristic depressions. Recall that the search spaces were generated by blocking
out nodes instead of just removing edges which is the
method Ishida used.
Floyd
Commitment
Deliberation
Marking
Long
Term
parameters
requiring
tuning
none
degree
degree
none
Figure 19: Comparison of algorithm enhancements
6
5.4
can be
combined
with
Any
BMTS
BMTS
Any
Discussion
Learning
Figures 18 and 19 summarize various points on the
Experimentation was done over a number of rounds algorithms discussed.
to observe general learning on the part of the solver
Clearly the original movingtarget search algorithm
for algorithms that acquire knowledge about the does not perform well compared to the other algosearch space. After a search the solver and target
rithms discussed in this paper. The sensitivity to
were placed back in their initial positions, the solver target’s movementsand the slow incremental learnwas allowed to retain any knowledge it had acquired ing hamper this algorithm. However the research by
from the previous search, and the search was carried
Korf and Ishida laid the foundation for this field of
out again.
study and also led to Ishida’s improved algorithm.
The hierarchical marking technique combined with
Adding commitment makes a substantial
improveforgetful depth-first search and Ishida’s movingtarget ment. The learning is focused for maximumbenesearch algorithm with commitment(no deliberation)
fit. The solver gets out of heuristic depressions much
were tested for improvement over 100 rounds. The quicker. Unfortunately, learning is "local" and helps
experimental setup is the same as the previous sec- little in the long term. In Section 5.4, which shows
tion, ten thousand times 100 by 100 spaces at density the benefit of learning over time, Ishida’s problem
of 35 percent. The random moving target was used.
solver improvements come slowly. Eventually (many
Figure 17 shows the performance of marking and rounds later) it will behave optimally.
Ishida’s algorithm (CMTS). The data on the CMTS Deliberation effectively reduces the thrashing probgraph is fitted (using least squares) with a straight
lem and therefore helps the solver get out of local
line. The progression along the x axis is the round minimums. Ishida’s techniques also require paramnumber which can be misleading because the amount eters, degree of commitment and deliberation, that
of time spent learning during a given round is propor- must be fine tuned for the current search setting.
tional to the number of moves taken in that round. Overall, deliberation and commitment are very good
Both algorithms show improvement over time. Note techniques. Unfortunately they are specifically dethat the scales on the y-axis differ. CMTS
has much signed to work with the original moving target search
more room for improvement than marking whose per- algorithm. They can not readily be added to other
formance is good from the beginning. Also, the data algorithms (forgetful depth-first search for example)
showsthat the two algorithms learn at different rates. with the guarantee of performance improvements.
For marking, the search time drops quickly in the first
Forgetful depth-first search is simple, requires little
few rounds and then show no improvement in later
overhead and memory, and seems to be a reasonable
rounds. On the other hand, CMTSwill likely main- approach to doing moving target search. The perfortain its slower rate of improvementuntil it approaches mance results from the experimentation done in this
optimal performance.
paper are very promising. For the search settings
37
From: AAAI Technical Report FS-93-02. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved.
used here, FDFS clearly beats CMTSand IMTS. A
criticism with FDFSis that it is not "O(n3) complete". (Although this is curable for any algorithm
by augmenting it with Floyd’s algorithm.) On a theoretical note, the forgetful depth-first search algorithm
(without any marking technique) would be usable
a dynamic search setting where edges are added and
deleted from the graph as the search progresses. Korf
and Ishida’s algorithms cannot handle edges being
added
Marking proves to be a beneficial addition to the
forgetful depth-first search algorithm. Although its
learning may not be as complete as knowing the true
distance between all pairs of nodes, it comes much
sooner and is very general. The marking technique
can be easily added to any algorithm, it requires no
parameters to be tuned, and it is very unlikely that
adding marking will have a negative effect on an algorithm’s performance.
7
Acknowledgements
The author wishes to thank his supervisor
Jonathan Schaeffer and colleague Sean tialliday
their helpful discussions during this research.
Dr.
for
References
[1] T. Ishida and R. E. Korf: "Moving Target
Search," IJCAI-91, pp. 204-210, 1991.
[2] T. Ishida: "Moving Target Search With Intelligence," AAAI-92, pp. 525-532, 1992.
[3] S. Baase: "Computer Algorithms, Introduction
to Design and Analysis," 2nd edition, Addison
Wesley, 1988.