Artificial Intelligence:
State Space Search
Russell & Norvig chap. 3 & section 4.1.1
Many slides from:
robotics.stanford.edu/~latombe/cs121/2003/home.htm
Motivation
moebius puzzle
Moebius puzzle
Play the 8-puzzle online
Rubiks cube
Tetris
2
Today
State Space Representation
State Space Search
Robotics
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Reasoning
Search
Learning
Informed search
Perception
Uninformed search
Agent
Hill climbing
Best-First
A*
Summary
Planning
Natural
language
KnowledgeConstraint
rep. satisfaction
...
Expert
Systems
Example: 8-Puzzle
8
Initial state
Goal state
State: Any arrangement of 8 numbered tiles and an empty tile on a 3x3 board
(n2-1)-puzzle
10 11 12
8-puzzle
....
13 14 15
15-puzzle
15-Puzzle
Invented in 1874 by Noyes Palmer Chapman
but Sam Loyd claimed he invented it!
15-Puzzle
Sam Loyd even offered $1,000 of his own
money to the first person who would solve the
following problem:
1
10
11
12
13
14
15
10
11
12
13
15
14
7
But no one ever won the prize
State Space
Many AI problems, can be expressed in terms of going
from an initial state to a goal state
Ex: to solve a puzzle, to drive from home to Concordia
Often, there is no direct way to find a solution to a
problem
but we can list the possibilities and search through them
Brute force search:
generate and search all possibilities (but inefficient)
Heuristic search:
only try the possibilities that you think (based on your
current best guess) are more likely to lead to good solutions
9
State Space
Problem is represented by:
1.
Initial State
2.
Set of operators
3.
Applied to a state to determine if it is a goal state
ex. solved puzzle, being at Concordia
Path cost function
actions responsible for transition between states
Goal test function
4.
starting state
ex. unsolved puzzle, being at home
Assigns a cost to a path to tell if a path is preferable to
another
Search space: the set of all states that can be reached
from the initial state by any sequence of action
Search algorithm: how the search space is visited
10
Example: The 8-puzzle
8
Initial state
Goal state
Set of operators:
blank moves up, blank moves down, blank moves left, blank moves right
Goal test function:
state matches the goal state
Path cost function:
each movement costs 1
so the path cost is the length of the path (the number of moves)
source: G. Luger (2005)
11
8-Puzzle: Successor Function
8
7
4
Search is about the exploration of alternatives
12
State Space Representation
S
1
3
State space S
Successor function:
xS SUCCESSORS(x)
Initial state s0
Goal states:
xS GOAL?(x) =T or F
Arc cost
13
State Graph
Each state is
represented by a
distinct node
An arc (or edge)
connects a node s
to a node s if
s SUCCESSOR(s)
The state graph may
contain more than one
connected component
14
State Space for the 8-puzzle
source: G. Luger (2005)
15
How big is the state space of the
(n2-1)-puzzle?
Nb of states:
8-puzzle --> 9! = 362,880 states
15-puzzle --> 16! ~ 2.09 x 1013 states
24-puzzle --> 25! ~ 1025 states
At 100 millions states/sec:
8-puzzle --> 0.036 sec
15-puzzle --> ~ 55 hours
24-puzzle --> > 109 years
16
Searching the State Space
It is often not
feasible (or too
expensive) to
build a complete
representation
of the state
graph
17
Just to make sure were clear
3
6
Initial state
Goal state
18
Problems with Graph Search
Several paths to same nodes
Cycles in path can prevent
termination
source: G. Luger (2005)
Need to select best path
according to needs of problem
Blind search without cycle
check will never terminate (ex.
see graph on the top)
There is no cycle problems
with trees
19
State Space as a Search Tree
Search tree
20
State Search of Travelling
Salesperson
Salesperson has
Goal: find shortest path for travel
source: G. Luger (2005)
to visit 5 cities
must return home afterwards
minimize cost and/or time of
travel
Nodes represent cities
Weighted arcs represent cost of
travel
Simplification: salesperson lives in
city A and will return there
Goal is to go from A to A with
minimal cost
21
Search of TSP problem
Each arc is marked with the total
weight of all paths from the start
node (A) to its endpoint (A)
source: G. Luger (2005)
22
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
23
Uninformed VS Informed Search
Uninformed search
We systematically explore the alternatives
aka: systematic/exhaustive/blind/brute force search
Breadth-first
Depth-first
Uniform-cost
Depth-limited search
Iterative deepening search
Bidirectional search
Informed search (heuristic search)
We try to choose smartly
Hill climbing
Best-First
A*
24
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
25
Breadth-first vs Depth-first Search
Determine order for examining states
Depth-first:
visit successors before siblings
Breadth-first:
visit siblings before successors (ie. visit level-bylevel
26
Data Structures
In all search strategies, you need:
open list (aka the frontier)
lists generated states not yet expanded
order of states controls order of search
closed list (aka the explored set)
stores all the nodes that have already been visited (to avoid
cycles).
ex:
Closed = [A, B, C, D, E]
Open = [K, L, F, G, H, I, J]
source: G. Luger (2005)
27
Data Structures
To trace back the entire path of the solution after
the search, each node is the lists contain:
STATE
REPRESENTATION
8 2
3 4 7
5 1
PARENT-NODE
6
BOOKKEEPING
CHILDREN
...
Action
Right
Depth
Path-Cost
Depth = length of path from root to node
28
Generic Search Algorithm
1.
Initialize the open list with the initial node so (top node)
2.
Initialize the closed list to empty
3.
Repeat
a)
If the open list is empty, then exit with failure.
b)
Else take the first node s from the open list.
c)
If s is a goal state, exit with success. Extract the path
from s to so
d)
Insert s in the closed list (s has been visited /expanded)
e)
Insert the successors of s in the open list in a certain
order if they are not already in the closed/open lists (to
avoid cycles)
Notes:
The order of the nodes in the open list depends on the search
strategy
29
DFS and BFS
DFS and BFS differ only in the way they order nodes in the
open
list:
DFS uses a stack:
nodes are added on the top of the list.
BFS uses a queue:
Nodes are added at the end of the list.
30
Breadth-First Search
a)
b)
c)
d)
e)
source: G. Luger (2005)
31
Breadth-First Search Example
BFS: (open is a queue)
source: G. Luger (2005)
32
Snapshot of BFS
source: G. Luger (2005)
Search graph at
iteration 6 of
breadth-first
search
States on open
and closed are
highlighted
33
BFS of the 8-puzzle
Shows order in
which states were
removed from
open
Goal
source: G. Luger (2005)
34
Function Depth-First Search
a)
b)
c)
d)
e)
source: G. Luger (2005)
35
Trace of Depth-First Search
source: G. Luger (2005)
36
Snapshot of DFS
source: G. Luger (2005)
Search graph at
iteration 6 of
depth-first
search
States on open
and closed are
highlighted
37
DFS on the 8-puzzle
Depth-first
search of the 8puzzle with a
depth bound of 5
Goal
source: G. Luger (2005)
38
Depth-first vs. Breadth-first
Breadth-first:
Complete: always finds a solution if it exists
Optimal: always finds shortest path
But:
inefficient if branching factor B is very high
memory requirements high -- exponential space for states
required: Bn
Depth-first:
Not complete (ex. may get stuck in an infinite branch)
Not optimal (will not find the shortest path)
But:
Requires less memory -- only memory for states of one path
needed: B
n
May find the solution without examining much of the search space
Efficient if solution path is known to be long
39
Depth-first vs. Breadth-first
If you want the shortest path --> breadth first
If memory is a problem --> depth first
Many variants of breadth & depth first:
Depth-limited Search
Iterative deepening search
Uniform-cost search
Bidirectional search
40
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
41
Depth-Limited Search
Compromise for DFS :
Do depth-first but with depth cutoff k (depth
at which nodes are not expanded)
Three possible outcomes:
Solution
Failure (no solution)
Cutoff (no solution within cutoff)
42
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breath-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
43
Iterative Deepening
Compromise between BFS and DFS:
use depth-first search, but
with a maximum depth before going to next level
Repeats depth first search with gradually increasing
depth limits
Requires little memory (fundamentally, its a depth first)
Finds the shortest path (limited depth)
Preferred search method when there is a large search
space and the depth of the solution is unknown
44
Iterative Deepening: Example
source: Russel & Norvig (2003)
45
Iterative Deepening: Example
source: Russel & Norvig (2003)
46
Iterative Deepening: Example
source: Russel & Norvig (2003)
47
Iterative Deepening: Example
source: Russel & Norvig (2003)
48
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breath-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
49
Search of TSP problem
Each arc is marked with the total
weight of all paths from the start
node (A) to its endpoint (A)
source: G. Luger (2005)
50
Size of Search Space
Complexity: (N - 1)! with N the number of cities
interesting problem size: 50 cities
simple exhaustive search becomes intractable
Several techniques can reduce complexity
--> heuristic search
ex: Nearest neighbor heuristic
go to closest unvisited city
highly efficient since only one
path is tried
might fail (see next example)
similar to hill climbing
51
Nearest Neighbour on TSP
nearest neighbour path
in bold
this path (A, E, D, B, C,
A) has a cost of 550
but it is not the
shortest path
the high cost of (C, A)
defeated the heuristic
nearest neighbour
52
Informed Search
(aka heuristic search)
Most of the time, it is not feasible to do an
exhaustive search, search space is too large
so far, all search algorithms have been
uninformed (general search)
so need an informed/heuristic search
Idea:
choose "best" next node to expand
according to a selection function (heuristic)
But: heuristic might fail
53
Heuristic - Heureka!
Heuristic:
Heuristic search:
a rule of thumb, a good bet
but has no guarantee to be correct whatsoever!
A technique that improves the efficiency of search,
possibly sacrificing on completeness
Focus on paths that seem most promising according to
some function
Need an evaluation function (heuristic function) to
score a node in the search tree
Heuristic function h(n) = an approximation to a
function that gives the true evaluation of the
node n
54
Why do we need heuristics?
A problem may not have an exact solution
1.
ex: ambiguity in the problem statement or the
data
The computational cost of finding the
exact solution is prohibitive
2.
ex: search space is too large
ex: state space representation of all possible
moves in chess = 10120
1075 = nb of molecules in the universe
1026 = nb of nanoseconds since the big bang
55
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breath-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
56
Example: Hill Climbing with Blocks World
Operators:
pickUp(Block)
putOnTable(Block)
stack(Block1,Block2)
Heuristic:
0pt if a block is sitting where it is supposed to sit
+1pt if a block is NOT sitting where it is supposed to sit
so lower h(n) is better
h(initial) = 2
h(goal) = 0
source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991.
57
Example: Hill Climbing with Blocks World
h(n) = 2
pickUp(A)+putOnTable(A)
h(n) = 1
pickUp(A)+stack(A,H)
pickUp(H)+putOnTable(H)
pickUp(H)+stack(H,A)
h(n) = 2
h(n) = 2
h(n) = 2
58
Hill Climbing
General hill climbing strategy:
as soon as you find a position that is better than the
current one, select it.
Is a local search
Does not maintain a list of next nodes to visit (an open list)
Similar to climbing a mountain in the fog with amnesia
always go higher than where you are now,
help!
but never go back
Steepest ascent hill climbing:
instead of moving to the first position that
is better than the current one
pick the best position out of all the next possible moves
59
Example: Hill Climbing with Blocks World
h(n) = 2
hill-climbing will stop,
because all successors are
higher than parent -->
local minimum
pickUp(A)+putOnTable(A)
h(n) = 1
pickUp(A)+stack(A,H)
pickUp(H)+putOnTable(H)
pickUp(H)+stack(H,A)
Dont be confused
a lower h(n) is better
So with lower h(n),
hill-climbing should really be
called deep-diving
h(n) = 2
h(n) = 2
h(n) = 2
60
Problems with Hill Climbing
Foothills (or local maxima)
reached a local maximum, not the global maximum
a state that is better than all its neighbors but is not better
than some other states farther away.
at a local maximum, all moves appear to make things worse.
ex: 8-puzzle: we may need to move tiles temporarily out of goal
position in order to place another tile in goal position
ex: TSP: "nearest neighbour" heuristic
source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991.
61
Problems with Hill Climbing
Plateau
a flat area of the search space in which the next states
have the same value.
it is not possible to determine the best direction in which
to move by making local comparisons.
can cause the algorithm to stop (or wander aimlessly).
source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991.
62
Some Solutions to Hill-Climbing
Random-restart hill-climbing
random initial states are generated
run each until it halts or makes no significant progress.
the best result is then chosen.
keep going even if the best successor has the same value as
current node
works well on a "shoulder"
but could lead to infinite loop
on a plateau
source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991. & Russel & Norvig (2003)
63
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
64
Best-First Search
problem with hill-climbing:
solution to hill-climbing:
one move is selected and all others are forgotten.
use "open" as a priority queue
this is called best-first search
Best-first search:
Insert nodes in open list so that the nodes are sorted in
best (ascending/descending) h(n)
Always choose the next node to visit to be the one with the
best h(n) -- regardless of where it is in the search space
65
Best-First: Example
Lower h(n) is better
source: Rich & Knight, Artificial Intelligence, McGraw-Hill College 1991.
66
Notes on Best-first
If you have a good evaluation function,
best-first can find the solution very quickly
The first solution may not be the best,
but there is a good chance of finding it quickly
It is an exhaustive search
will eventually try all possible paths
67
Best-First Search: Example (1)
3
4
Lower h(n) is better
source: G. Luger (2005)
68
Best-First Search: Example (2)
source: G. Luger (2005)
69
Designing Heuristics
Heuristic evaluation functions are highly
dependent on the search domain
In general: the more informed a heuristic is,
the better the search performance
Bad heuristics lead to frequent backtracking
70
Example: 8-Puzzle Heuristic 1
h1: Simplest heuristic
Hamming distance : count
number of tiles out of place
when compared with goal
3
6
STATE n
source: G. Luger (2005)
Goal state
h1(n) = 6
does not consider the
distance tiles have to be
moved
71
Example: 8-Puzzle Heuristic 2
h2: Better heuristic
Manhattan distance: sum up
all the distances by which
tiles are out of place
3
6
STATE n
source: G. Luger (2005)
Goal state
h2(n) = 2+3+0+1+3+0+3+1
= 13
72
Example: 8-Puzzle Heuristic 3
source: G. Luger (2005)
h3: Even Better
sum of permutation
inversions
See next slide
73
h3(N) = sum of permutation inversions
5
STATE n
For each numbered tile, count how many tiles on its right
should be on its left in the goal state.
1 2 3
h3(n) = n5 + n8 + n4 + n2 + n1 + n7 + n3 + n6
4 5 6
=4 +6 +3 +1 +0 +2 +0 +0
= 16
7 8
BTW:
Goal state
If this number is even, the puzzle is solvable.
If this number is odd, the puzzle is not solvable.
74
Remember this slide...
h3(n) = n1 + n2 + n3 + n4 + + n13 + n14 + n15
=0 +0 +0 +0+ +0 +1 +0
=1
the puzzle is not solvable.
75
Heuristics for the 8-Puzzle
5
3
6
STATE n
Goal state
h1(n) = misplaced numbered tiles
=6
h2(n) = Manhattan distance
= 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13
h3(n) = sum of permutation inversions
= n5 + n8 + n4 + n2 + n1 + n7 + n3 + n6
= 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16
76
Algorithm A
Heuristics might be wrong:
Solution:
so search could continue down a wrong path
Maintain depth count, i.e., give preference to shorter
paths
Modified evaluation function f:
f(n) = g(n) + h(n)
f(n) estimate of total cost along path through n
g(n) actual cost of path from start to node n
h(n) estimate of cost to reach goal from node n
77
Heuristic f(n) applied to 8-puzzle
source: G. Luger (2005)
78
State Space for 8-puzzle
source: G. Luger (2005)
79
State Space for 8-puzzle (cont)
source: G. Luger (2005)
80
State Space for 8-puzzle (cont)
source: G. Luger (2005)
81
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breath-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
82
Evaluating Heuristics
Admissibility:
Monotonicity:
optimistic
never overestimates the cost of reaching the goal
guarantees to find the shortest path to the goal, if it exists
e.g.: breadth-first is admissible -- it uses f(n) = g(n) + 0
local admissibility
guarantees to find the shortest path to each state
encountered in the search
Informedness:
measure for the quality of a heuristic
usually, the more informed, the better
83
Admissibility
Evaluation function f(n) = g(n) + h(n) for node n:
g(n) cost so far to reach n (eg. depth of n in graph)
h(n) heuristic estimate of distance from n to goal
f(n) estimate for total cost of solution path through n
Now consider f*(n) = g*(n) + h*(n):
g*(n) cost of shortest path from start to node n
h*(n) actual cost of shortest path from n to goal
f*(n) actual cost of optimal path from start to goal
through n
84
Algorithm A*
Problem: f* is usually not available
if g(n) g*(n) for all n
best-first used with such a g(n) is called algorithm A
if h(n) h*(n) for all n
i.e. h(n) never overestimates the true cost from n to a
goal
algorithm A used with such an h(n) is called A*
an A* algorithm is admissible (guarantees to find the
shortest path to the goal)
85
Example: 8-Puzzle
5
1 2 3
4 5 6
7 8
goal
h1(n) = Hamming distance = number of misplaced tiles = 6
--> admissible
h2(n) = Manhattan distance = 13
--> admissible
86
Monotonicity
(aka consistent)
An admissible heuristics may temporarily reach non-goal
states along a suboptimal path
A heuristic is monotonic if it always finds the minimal path to
each state the 1st time it is encountered
h is monotonic if for every node n and every successor n of n:
h(n) c(n,n') + h(n')
i.e. f(n) is non-decreasing along any path
87
Informedness
Intuition: number of misplaced tiles is less informed than
Manhattan distance
For two A* (ie. admissible) heuristics h1 and h2
if h1(n) h2(n), for all states n
then h2 is more informed than h1
h1(n) h2(n) h*(n)
More informed heuristics search smaller space to find
optimal solution
However, you need to consider the computational cost of
evaluating the heuristic
The time spent computing heuristics must be recovered
by a better search
88
Today
State Space Representation
State Space Search
Uninformed search
Informed search
Breath-first and Depth-first
Depth-limited Search
Iterative Deepening
Hill climbing
Best-First
A*
Summary
89
Summary
Search
Uses
h(n) ?
Open is a
Breadth-first
No
Queue
Depth-first
No
Stack
Depth-limited
No
Stack
Iterative Deepening
No
Stack
Hill climbing
Yes
none
Best-First
Yes
Priority queue sorted by h(n)
A*
Yes
Priority queue sorted by f(n)
f(n) = g(n) + h(n)
90
Today
State Space Representation
State Space Search
Uninformed search
Breadth-first and Depth-first
Depth-limited Search
Iterative Deepening
Informed search
Hill climbing
Best-First
A*
Summary
91
