Game Playing

Kinds of Games
• Deterministic
• Turn-taking
• 2-player
• Zero-sum
• Perfect information
 Game as Search Problem
• Initial State: board position and player to
move
• Successor Function: returns a list of legal
(move, state) pairs
• Terminal Test: determines when the game
is over
• Utility function: Gives a numeric value for
the terminal state
 Game Trees
• Game trees are used to represent two-player
games.
• Alternate moves in the game are represented by
alternate levels in the tree (plies).
• Nodes in the tree represent positions.
• Edges between nodes represent moves.
• Leaf nodes represent won, lost or drawn positions.
Game Trees

This is an example of a
partial game tree for the
game tic-tac-toe.
Even for this simple game,
the game tree is very large.
 Assumptions
• In talking about game playing systems, we
make a number of assumptions:
• The opponent is rational – will play to win.
• The game is zero-sum – if one player wins, the
other loses.
• Usually, the two players have complete
knowledge of the game. For games such as
poker, this is clearly not true.
 Minimax
• Minimax is a method used to evaluate game
trees.
• A static evaluator is applied to leaf nodes,
and values are passed back up the tree to
determine the best score the computer can
obtain against a rational opponent.
 Minimax – Animated Example
Max 3 6 The computer can
obtain 6 by
choosing the right
Min 6 hand edge from the
5 3 first node.

Max 1 3 6 0 7
5

5 2 1 3 6 2 0 7
 Minimax Function
• MINIMAX-VALUE(n) =
• UTILITY(n)
if n is a terminal state
• maxs  Successors(n) MINIMAX-VALUE(s) if
n is a MAX node
• mins  Successors(n) MINIMAX-VALUE(s) if
n is a MIN node
 Searching Game Trees
• Exhaustively searching a game tree is not usually
a good idea.
• Even for a game as simple as tic-tac-toe there are
over 350,000 nodes in the complete game tree.
• An additional problem is that the computer only
gets to choose every other path through the tree –
the opponent chooses the others.
 Alpha-beta Pruning
• A method that can often cut off a half the
game tree.
• Based on the idea that if a move is clearly
bad, there is no need to follow the
consequences of it.
• alpha – highest value we have found so far
• beta – lowest value we have found so far
 Alpha-beta Pruning –
Example.
max 3 • In this tree,
having
min examined the
nodes with
values 7 and 1
there is no need
to examine the
final node.
 Bounded Lookahead
• For trees with high depth or very high branching
factor, minimax cannot be applied to the entire
tree.
• In such cases, bounded lookahead is applied:
• search is cut off at specified depth
• static evaluator applied.
• Horizon effect
 Static Evaluation Functions
• A static evaluator assigns a score to a position:
• High positive = computer is winning
• Zero = even game
• High negative = opponent is winning
• It is most important that a static evaluator will give
a better score to a better position – the actual
values are not so important.
 Creating Evaluation Function
• How likely are you to win from a given
position?
• Usually weighted linear functions
• Different scores are given to different position
and added together in a weighted fashion
• Possible chess function
• score = 9q + 5r + 3b + 3n + p
• q = # of queens, r = # of rooks, b = # of bishops,
n = # of knights, and p = # of pawns
 Checkers
• In 1959, Arthur Samuel creates a computer
program that could play checkers to a high level
using minimax and alpha-beta pruning.
• Chinook, developed in Canada defeated the world
champion:
• Uses alpha-beta pruning.
• Has a database of millions of end games.
• Also has a database of openings.
• Uses heuristics and knowledge about the game.
 Chess
• In 1997, Deep Blue defeated world champion,
Garry Kasparov.
• This has not yet been repeated.
• Current systems use parallel search, alpha-beta
pruning, databases of openings and heuristics.
• The deeper in a search tree the computer can
search, the better it plays.
 Go
• Go is a complex game played on a 19x19 board.
• Average branching factor in search tree around
360 (compared to 38 for chess).
• The best computer programs cannot compete yet
with the best human players.
• Methods use pattern matching or selective search
to explore the most appropriate parts of the search
tree.
 Games of Chance
• The methods described so far do not work
well with games of chance such as poker or
backgammon.
• Expectiminimax is a variant of minimax
designed to deal with chance.
• Nodes have expected values based on
probabilities.
Iterative Deepening
 Motivations
• BFS & A*
– good for optimality
– bad on memory, O(bd)
• DFS
– solutions not guaranteed optimal: dives and
misses good nodes
– good for memory O(bd)
• “Iterative Deepening” refers to a method that
tries to combine the best of the above
 Depth-Limited Search
• Simply put an upper limit on the depth (cost) of
paths allowed
• Motivation:
– e.g. inherent limit on range of a vehicle
• tell me all the places I can reach on 10 litres of petrol
– prevents search diving into deep solutions
– might already have a solution of known depth
(cost), but are looking for a shallower (cheaper)
one
 Depth-Limited Search

• Impose an upper limit on the depth (cost) of

• DepthLimitedDFS ( k ) :
DFS but only consider nodes with depth d ≤ k
 Trees: Depth-Limited
• Depth limit of 2 d=0 F

would mean that

ignored
H G
E d=2

A I d=3

D d=4
 Iterative Deepening Search
• Follow the BFS pattern of
“search all nodes of depth d before depth d+1”
• But do the search at depth d using Depth-Limited-DFS

• Schematically:

• IDS
:
k=
0;
while
( no
t
succ
ess
 Properties of IDS
• Memory Usage
– Same as DFS O(bd)

• Time Usage:
– Worse than BFS because nodes at each level will be
expanded again at each later level
– BUT often is not much worse because almost all the
effort is at the last level anyway, because trees are
“leaf –heavy”
– Typically might be at most a factor two worse
 Memory Usage of A*
• We store the tree in order to
– to return the route
– avoid repeated states
• Takes a lot of memory
• But scanning a tree is better with DFS
 IDA*
• Combine A* and iterative deepening
• f is the estimate of total path cost for start to
goal
• IDA*:
– Impose a limit on f
– Use DFS to search within the f limit
– Iteratively relax the limit
• Greatly reduces memory usage
• Can repeat far too much work, and so be
slow
 Summary
• Algorithm IDS:
– IDS : BFS plus DFS for tree search
• Algorithm IDA*:
– the basis of “state of the art” “complete &
optimal” algorithms
 Summary
• BFS & A*: good for optimality, but not memory
• DFS: good for memory O(bd), but not optimality
• “Iterative Deepening” refers to
– IDS “Iterative Deeepening Search”
• mix of DFS and BFS on trees
– a broad approach used for general search, with general aim to
combine optimality with low memory usage of DFS