Adversarial search occurs in competitive environments, in which other agents are
plotting against us. They have conflicting goals.
Game Theory
There are three stances toward multi-agent environments
1. There are a very large number of agent -> consider them in the aggregate as an
economy, and apply things like "increasing demand will cause prices to rise, without
considering each individual goal"
2. Treat other agents (even adversaries) as just part of the environment, making it
nondeterministic
We might treat the enemy's random next move as part of uncertainty of a
nondeterministic environment.
This is so simplistic, because adversaries have intentions of defeating us, while
other factors like random rain has no such intention
3. To explicitly model the adversarial agents with the techniques of adversarial gametree search. [Part of this chapter]
Two-player zero-sum games
The most commonly studied games are deterministic, two-player, turn-taking, perfectinformation (=fully observable), zero-sum (= what is good for a player is as much bad
for the other, no win-win situations) games.
Note
In such games, we might refer to state as position, and action as move. Sometimes a
move is a complete turn, and a ply is just one move for one player in one turn (=
bringing us one level deeper in the game tree)
Such games are Chess and Go
We call the players MAX and MIN. MAX moves first, and the players take turns moving
until the game is over.
: the initial state, which specifies how the game is set up at the start
To-Move(s): the player whose turn is is to move in state s
ACTIONS(s): the set of legal moves in state s
RESULT(s, a): the transition model, which defines the state resulting from taking
action a in state s
IS-TERMINAL(s): a terminal test, which is true when the game is over. States where
the game is over are called terminal states
UTILITY(s, p): a utility/objective/payoff function , which defines the final numeric
value to player p when the game ends in terminal state s.
In chess, the outcome is win, loss or draw, with values 1, 0, 1/2. Some games
have other possible outcomes
S0
Example for a tic-tac-toe game
Optimal Decisions in Games
MAX's strategy is consider all MIN's possible moves, MAX's strategy must be a
conditional plan. In games that have a binary outcome, we can use AND-OR search to
generate this conditional plan. For games with more outcome scores, we can use minimax
search.
The minimax value of a state in the tree, MINIMA(s), is the utility (for MAX) of being
in that state, assuming that both players play optimally from there to the end of the
game. The minimax of a terminal state is its utility. In non-terminal state, MAX
prefers to move to a state of maximum value when it's MAX's turn to move, and MIN
prefers a state of minimum values for MAX, and thus maximum to MIN.
Let's consider this tree
The minimax value for is 3 . The other two nodes, , and
have MINIMA value equal to 2.
The minimax decision at the root is the move a .
1
This is all under the assumption both players move optimally, but would we it's not the
best to play the minimax optimal move when facing a suboptimal opponent. If the optimal
move yields a draw, but we can have a move that would give more 10 actions, only 1
would yield success. If the opponent is sub-optimal, we can choose this risky move.
The minimax search algorithm
It's a recursive algorithm that proceeds all the way down to the leaves of the tree and
then backs up the minimax values through the tree as the recursion unwinds.
Optimal Decisions in multiplayer games
In case we have more than 2 players, we can extend the minimax idea to this case too.
We can replace the single value for each node with a vector of values. In a 3-player
game, , , , we can use the vector (, , ) associated with each node, this vector has the
utility of that state from each player's viewpoint.
in the node marked with , possible choices yield two different vectors. Since , then
we choose the left option.
Alpha-beta pruning
The number of game states is exponential in the depth of the tree. We can sometimes cut
in in half without examining every state by pruning large parts of the tree that make
no difference to the outcome
We can eliminate some steps like in here
At each point, we show the range of possible values for each node
1. The first node below (B) has the value 3. Hence, , which is a MIN NODE has a value
of at most 3.
2. The second leaf below B has a value of 12; MIN would avoid this move, so it's still
at most 3
3. The third leaf is 8, so the value of B is exactly (given there are no more states)
Now we also know that the root is at least 3
4. The first leaf below has the value 2. Hence, this node has a value of 2 at most,
but we know that is worth 3, thus MAX would never choose a node that would yield a
node with 2 value at most, thus we eliminate this node
5. The first leaf below node has the value 14, so is worth at most 14. This is still
higher than MAX's best alternative of 3, so we keep exploring
6. The second successor of is worth 5, so keep exploring, see 2, so 's worth is 2.
MAX's decision at the root is to move to
Summary
The general principle is this: Consider a node somewhere in the tree such that the
Player has a choice of moving to . If Player has a better choice either at the same
level (e.g. ) or at any point higher up in the tree (e.g. ), then Player will never
move to
Given that minimax search is depth-first, at any one time, we have to consdier the
nodes along a signle path in the tree. We define for every two nodes and , that
describe bounds on the backed-up values that appear anywhere along the path. We prune
when the condition (Notice that we start with , )
Another example on YT with this algorithm
Move Ordering
The effectiveness of alpha-beta pruning is dependent on the order in which the states
are examined. In the example above, we didn't prune any of D's successors because the
worst successors were generated first.
done perfectly, we will have to prune only ( ) nodes to pick the best move instead of ()
.
/2
The best moves are known as killer moves and to try them first is called the killer
move hueristic
Resources
Artificial Intelligence A modern approach