Informed
Search
Chapter 4 (b)
Some material adopted from notes
by Charles R. Dyer, University of
Wisconsin-Madison
Todays class: local
search
Iterative improvement methods
Hill climbing
Simulated annealing
Local beam search
Genetic algorithms
Online search
Hill Climbing
Extended the current path with a
successor node which is closer to
the solution than the end of the
current path
If our goal is to get to the top of a
hill, then always take a step the
leads you up
Simple hill climbing take any
upward step
Steepest ascent hill climbing
consider all possible steps, and
Hill climbing on a
surface of states
Height
Defined by
Evaluation
Function
Hill-climbing search
If there exists a successor s for the current
state n such that
h(s) < h(n)
h(s) <= h(t) for all the successors t of n
then move from n to s. Otherwise, halt at n
Looks one step ahead to determine if a
successor is better than the current state; if so,
move to the best successor.
Like Greedy search in that it uses h, but doesnt
allow backtracking or jumping to an alternative
path since it doesnt remember where it has
been.
Is Beam search with a beam width of 1 (i.e., the
maximum size of the nodes list is 1).
Not complete since the search will terminate at
Hill climbing example
start
2 8 3
1 6 4
7
5
-5
h = -4
-5
2 8 3
1
4 h = -3
7 6 5
-3
h = -3
goal
1 2 3
8
4 h=0
7 6 5
-2
1 2 3
8 4 h = -1
7 6 5
-4
2
3
1 8 4
7 6 5
2 3
1 8 4 h = -2
7 6 5
-4
f(n) = -(number of tiles out of place)
Exploring the
Landscape
Local Maxima:
peaks that arent the
highest point in the
space
Plateaus: the space
has a broad flat
region that gives the
search algorithm no
direction (random
walk)
Ridges: flat like a
plateau, but with
drop-offs to the sides;
local maximum
plateau
ridge
Image from: http://classes.yale.edu/fractals/CA/GA/Fitness/Fitness.html
Drawbacks of hill
climbing
Problems: local maxima,
plateaus, ridges
Remedies:
Random restart: keep restarting
the search from random locations
until a goal is found.
Problem reformulation:
reformulate the search space to
eliminate these problematic features
Some problem spaces are great
for hill climbing and others are
Example of a local
optimum
start
2 5
1 7 4
8 6 3
1 2 5
7 4
8 6 3
1 2 5
7
4 -4
8 6 3
-3
-4
1 2 5
8 7 4 -4
6 3
goal
1 2 3
8
4 0
7 6 5
Hill Climbing and 8
Queens
Annealing
In metallurgy, annealing is a technique
involving heating and controlled cooling
of a material to increase the size of its
crystals and reduce their defects
The heat causes the atoms to become
unstuck from their initial positions (a local
minimum of the internal energy) and
wander randomly through states of higher
energy.
The slow cooling gives them more chances
of finding configurations with lower internal
energy than the initial one.
Simulated annealing
(SA)
SA exploits the analogy between how metal
cools and freezes into a minimum-energy
crystalline structure & search for a
minimum/maximum in a general system
SA can avoid becoming trapped at local
minima
SA uses a random search that accepts
changes increasing objective function f and
some that decrease it
SA uses a control parameter T, which by
analogy with the original application is
known as the system temperature
T starts out high and gradually decreases
SA intuitions
combines hill climbing (efficiency) with
random walk (completeness)
Analogy: getting a ping-pong ball into the
deepest depression in a bumpy surface
shake the surface to get the ball out of the local
minima
not too hard to dislodge it from the global
minimum
Simulated annealing:
start by shaking hard (high temperature) and
gradually reduce shaking intensity (lower the
temperature)
escape the local minima by allowing some bad
moves
Simulated annealing
A bad move from A to B is accepted
with
a
probability
-(f(B)-f(A)/T)
The higher the temperature, the more
likely it is that a bad move can be made
As T tends to zero, this probability tends
to zero, and SA becomes more like hill
climbing
If T is lowered slowly enough, SA is
complete and admissible
Simulated annealing
algorithm
Local beam search
Basic idea
Begin with k random states
Generate all successors of these
states
Keep the k best states generated
by them
Provides a simple, efficient way to
share some knowledge across a set
of searches
Stochastic beam search is a
variation:
Genetic algorithms
(GA)
A search technique inspired by evolution
Similar to stochastic beam search
Start with k random states (the initial
population)
New states are generated by mutating
a single state or reproducing
(combining) two parent states (selected
according to their fitness)
Encoding used for the genome of an
individual strongly affects the behavior
of the search
Genetic algorithms / genetic
programming are a large and active area
Ma and Pa solutions
8 Queens problem
Represent state
by a string of 8
digits in {1..8}
S = 32752411
Fitness function
= # of nonattacking pairs
f(Sol) = 8*7/2 =
28
F(S) = 24
Genetic algorithms
Fitness function: number of non-attacking pairs of
queens (min = 0, max = (8 7)/2 = 28)
24/(24+23+20+11) = 31%
23/(24+23+20+11) = 29% etc
Genetic algorithms
GA pseudo-code
Ant Colony Optimization
(ACO)
A probabilistic search technique for
problems reducible to finding good
paths through graphs
Inspiration
Ants leave
nest
Discover food
Return to nest,
preferring
shorter paths
Leave
pheromone trail
Shortest path
Tabu search
Problem: Hill climbing can get
stuck on local maxima
Solution: Maintain a list of k
previously visited states, and
prevent the search from revisiting
them
Online search
Interleave computation & action (search
some, act some)
Exploration: Cant infer outcomes of
actions; must actually perform them to
learn what will happen
Competitive ratio: Path cost found/ Path
cost that would be found if the agent knew
the nature of the space, and could use
offline search
On average, or in an adversarial scenario (worst
case)
*
Relatively easy if actions are reversible
(ONLINE-DFS-AGENT)
LRTA* (Learning Real-Time A*): Update h(s)
(in state table) based on experience
Other topics
Search in continuous spaces
Different math
Search with uncertain actions
Must model the probabilities of an actions
results
Search with partial observations
Acquiring knowledge as a result of
search
Summary: Informed
search
Hill-climbing algorithms keep only a
single state in memory, but can get
stuck on local optima.
Simulated annealing escapes local
optima, and is complete and optimal
given a long enough cooling
schedule.
Genetic algorithms can search a
large space by modeling biological
evolution.
Online search algorithms are useful in
