Intelligent Systems and Machine Learning Algorithms

Informed search strategies: Module 3

Sushma B.
Assistant Professor
Dept. of Electronics & Communication Engineering
CMRIT, Bangalore

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 1 / 67

 Module-3: Informed search strategies and Logical agents

Topics Covered
Informed search strategies : Greedy best first search, A* search,
Memory-bounded heuristic search, Learning to search better.
Heuristic functions: Effect of heuristic accuracy, Generating
admissible heuristics from relaxed problems, Generating admissible
heuristics from sub-problems: Pattern databases, Learning heuristics
from experience,
Local search algorithms and optimization problems, Hill-Climbing
search, Simulated annealing, Local beam search, Genetic algorithms,
Local search in continuous spaces
Logical agents: Knowledge based agents, Wumpus world, Logic,
propositional logic, propositional theorem proving,

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 2 / 67

 Informed (Heuristic) search strategies

Uninformed search strategies do not have any additional knowledge

about the problem domain other than the structure of the search
space.
Uses problem-specific knowledge : Informed search strategies are a
category of search algorithms that use additional information (usually
in the form of a heuristic function) to guide the search process.
Can find solution more efficiently
Informed search strategies:
I Greedy best-first search
I A* search
I Memory-bounded heuristic search

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 3 / 67

 Key-components of informed search

State Space: A representation of the problem you’re solving

(including initial state, goal state, and actions).
Heuristic Function h(n) : A function that estimates the cost or
distance from a given node n to the goal. It helps in guiding the
search toward the goal efficiently.
Evaluation Function: An evaluation of a state, often a combination of
the path cost and the heuristic. Depending on the search strategy,
this function may use different combinations of the path cost and
heuristic value.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 4 / 67

 Best first search

It is an instance of Tree-search or Graph-search algorithm in which a

node is selected for an expansion based on an evaluation function f(n).
Identical to uniform cost search, except for the use of f instead of g to
order the priority queue.
Choice of f determines the search strategy
h(n) - component of f a heuristic function denoted as h(n)
h(n) = estimated cost of the cheapest path from the state at node n
to a goal state.
Heuristic functions are the most common form in which additional
knowledge of the problem is imparted to the search algorithm.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 5 / 67

 Greedy best-first search

Tries to expand the node that is closest to the goal, so the solution
finding is quick
Heuristic based algorithm, simple. Used in various optimization and
path finding problems
It makes decisions at each step by choosing the option that seems the
most promising at that moment.
Evaluates each node by using : f(n)=h(n)
h(n)= Estimated cost from the state at node n to a goal state
Algorithm is called greedy - at each step it tries to get as close to the
goal as it can.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 6 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 7 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 8 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 9 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 10 / 67
 A* Search : Minimizing the total estimated solution cost

It evaluates nodes by combining g(n), the cost to reach the node, and
h(n), the cost to get from the node to the goal.
f(n)=g(n)+h(n)
g(n) —–> gives the path cost from the start node to node n, h(n) is
the estimated cost of the cheapest path from n to the goal.
f(n) = estimated cost of the cheapest solution through n
Algorithm is complete and optimal

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 11 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 12 / 67
 Conditions for optimality: Admissibility and consistency :
A* is considered optimal if it always finds the least-cost path to a goal
(when such a path exists), meaning that if there’s a shorter or cheaper
path, A* will find it. This property is true under two conditions:
First condition for optimality is h(n) be an admissible heuristic.
Second Condition : A second, slightly stronger condition called
consistency
Admissible heuristic : It refers to a heuristic function that never
overestimates the cost of reaching the goal from any node in the
search space.
In other words, an admissible heuristic is optimistic: it always
estimates the remaining cost as equal to or less than the true
minimum cost.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 13 / 67

 Admissible heuristics are by nature optimistic because they think the
cost of solving the problem is less than it actually is.
Example of an admissible heuristic is the straight-line distance hSLD
that we used in getting to Bucharest.
Straight-line distance is admissible because the shortest path between
any two points is a straight line, so the straight line cannot be an
overestimate.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 14 / 67

 Bucharest first appears on the frontier at step (e), but it is not
selected for expansion because its f-cost (450) is higher than that of
Pitesti (417).
Another way to say this is that there might be a solution through
Pitesti whose cost is as low as 417, so the algorithm will not settle for
a solution that costs 450.
Example: Stages in an A* search for Bucharest. Nodes are labeled
with f = g + h. The h values are the straight-line distances to
Bucharest

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 15 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 16 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 17 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 18 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 19 / 67
 A second, stronger condition called consistency (monotonicity)
A heuristic h(n) is consistent if, for every node n and every successor
n’ of n generated by any action a, the estimated cost of reaching the
goal from n is no greater than the step cost of getting to n’ plus the
estimated cost of reaching the goal from n’.
h(n) <= c(n, a, n0 ) + h(n0 )
Conclusion:
I Every consistent heuristic is also admissible.
I Consistency is therefore a stricter requirement than admissibility.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 20 / 67

 Proof of Optimality of A*

A* has the following properties:

Property-1: The tree-search version of A* is optimal if h(n) is
admissible,
Property-2: The graph-search version is optimal if h(n) is consistent.
Proof for property-2
I Claim-1 If h(n) is consistent, then the values of f(n) along any path are
non-decreasing.
I Claim-2: Whenever A* selects a node n for expansion, the optimal
path to that node has been found

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 21 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 22 / 67
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 23 / 67
 As long as there is a node n’ which belongs to the optimal path and
which is in the frontier list, node n will not be selected for expansion
along the sub-optimal path. Thus it cannot happen that a node is
selected through the sub-optimal path.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 24 / 67

 Memory-bounded heuristic search

Memory-bounded heuristic search techniques designed to handle the

memory limitations of traditional heuristic search algorithms like A*
Though A* guarantees optimality and completeness, it is
memory-intensive as it stores all generated nodes in the open and
closed lists.
Memory-bounded approaches aim to manage or restrict memory
usage while retaining as much of A*’s performance characteristics as
possible.
Memory bounded heuristic search algorithms:
I Iterative-Deepening A (IDA)*
I Recursive best-first search (RBFS)
I

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 25 / 67

 Iterative-Deepening A (IDA)*
IDA* combines the ideas of depth-first search (DFS) and A*. It
performs a series of depth-limited searches, where the limit is
determined by the f(n)-value rather than the depth.
At each iteration, nodes are expanded only if their f(n) value is less
than or equal to a threshold, which starts at f(start) and increases in
the next iteration to the smallest f(n) that exceeds the current
threshold.
Optimal and complete for admissible heuristics.
Re-expansion of nodes multiple times

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 26 / 67

 Recursive Best-First Search (RBFS)
Simple recursive algorithm that uses a limited amount of memory by
mimicking a best-first search without storing all nodes in memory.
It expands the most promising node (based on f(n) ) but maintains
only the current path in memory. It uses a backup value to remember
the second-best f(n) value in case backtracking is required.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 27 / 67

 Heuristic functions
Heuristics function for the 8-Puzzle
The objective of the puzzle is to slide the tiles horizontally or
vertically into the empty space until the configuration matches the
goal configuration.
3x3 board with 8 numbers and blank space
Blank space can be moved up, down,....
When the empty tile is in the middle, four moves are possible; when it
is in a corner, two; and when it is along an edge, three.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 28 / 67

 To find the shortest solutions by using A*, we need a heuristic
function that never overestimates the number of steps to the goal.
Two common candidates:
I h1 - The number of misplaced tiles. If all of the eight tiles are out of
position, so the start state would have h1 = 8. h1 is an admissible
heuristic because it is clear that any tile that is out of place must be
moved at least once.
I h2 - The sum of the distances of the tiles from their goal positions.
Because tiles cannot move along diagonals, the distance we will count
is the sum of the horizontal and vertical distances. This is sometimes
called the city block distance or Manhattan distance. h2 is also
admissible because all any move can do is move one tile one step
MANHATTAN DISTANCE closer to the goal.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 29 / 67

 The effect of heuristic accuracy on performance
Quality of the heuristic is to have effective branching factor b*.
The branching factor in the context of heuristics, particularly in
search algorithms, refers to the average number of child nodes
generated from a single node in the search tree. It is a critical
parameter in evaluating the efficiency of search algorithms, as it
directly impacts the time and space complexity of the search process.
Total number of nodes generated by A* for a particular problem is N
and the solution depth is d, then b* is the branching factor that a
uniform tree of depth d have to contain N+1 nodes.
N + 1 = 1 + b ∗ +(b∗)2 + − − − − (b∗)d
If A* finds a solution at depth 5 using 52 nodes then b*=1.92
A well-designed heuristic would have a value of b* close to 1,

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 30 / 67

 A good heuristic can reduce the effective branching factor by guiding
the search toward the goal state more efficiently.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 31 / 67

 Generating admissible heuristics from relaxed problems
Admissible heuristics can be derived by simplifying or relaxing the
original problem
The idea is that a heuristic is admissible if it never overestimates the
cost to reach the goal.
Relaxed problems, where certain constraints are removed, provides
admissible heuristics because solutions to these problems represent
lower bounds on the cost of solving the original problem.
A problem with fewer restrictions on the actions is called a relaxed
problem.
Steps to generate admissible heuristics from relaxed problems
I Identify rules or constraints that make the problem challenging to solve.
I Create a simpler version of the problem by removing few constraints.
I Compute the exact cost of solving this easier version of the problem.
This cost provides a heuristic estimate for the original problem.
I Ensure the relaxed problem cost is always less than or equal to the
actual problem cost. This guarantees admissibility.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 32 / 67

 Generating admissible heuristics from subproblems: Pattern databases
Admissible heuristics can also be derived from the solution cost of a
subproblem of a given problem.
This cost is a lower bound on the cost of the real problem.
Pattern databases store the exact solution for every possible
subproblem instance.
The complete heuristic is constructed using the patterns in the DB

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 33 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 34 / 67
 Learning heuristics from experience
A heuristic function h(n) is supposed to estimate the cost of a
solution beginning from the state at node n.
How could an agent construct such a function?
One such solution is "Learn from Experience".
“Experience” here means solving lots of 8-puzzles, for instance. Each
optimal solution to an 8-puzzle problem provides examples from
which h(n) can be learned.
Each example consists of a state from the solution path and the
actual cost of the solution from that point.
Use these examples and a learning algorithm to construct a function
h(n) that can predict solution costs for other states that arise during
search.
An inductive learning algorithm can be used to predict costs for other
states that arise during search (using neural networks, decision trees,
and other methods).

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 35 / 67

 Local Search Algorithms and Optimization Problems
The search algorithms are designed to explore search spaces
systematically.
Systematicity is achieved by keeping one or more paths in memory
and by recording which alternatives have been explored at each point
along the path.
When a goal is found, the path to that goal also constitutes a
solution to the problem.
Sometimes the final state is important, not the path how it is
achieved.
Integrated-circuit design, factory-floor layout, job-shop scheduling,
automatic programming, telecommunications network optimization,
vehicle routing, and portfolio management.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 36 / 67

 Different set of algorithms are considered, when the path to the goal
node is not important.
Local search algorithms operate using a single current node and
generally move only to neighbours of that node. Paths are not added
to memory
Not systematic. But they have two key advantages
They use very little memory-usually a constant amount.
They can often find reasonable solutions in large or infinite
(continuous) state spaces for which systematic algorithms are
unsuitable.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 37 / 67

 Local search algorithms are useful for solving pure optimization
problems.
Aim is to find the best state according to an objective function
To understand the local search algorithms consider state-space
landscape

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 38 / 67

 A landscape has both “location” (defined by the state) and
“elevation” (defined by the value of the heuristic cost function or
objective function).
If elevation corresponds to cost, then the aim is to find the lowest
valley–a global minimum.
If elevation corresponds to an objective function, then the aim is to
find the highest peak-a global maximum.
Local search algorithms explore this landscape. A complete local
search algorithm always finds a goal if one exists; an optimal
algorithm always finds a global minimum/maximum.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 39 / 67

 Hill climbing search
Simulated annealing
Local beam search
Genetic algorithms

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 40 / 67

 Hill-climbing search
Steepest-ascent version
It is a loop that continually moves in the direction of increasing
value—that is, uphill.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 41 / 67

 Limitations of Hill climb
Local maxima: a local maximum is a peak that is higher than each of
its neighboring states but lower than the global maximum.
Hill-climbing algorithms that reach the vicinity of a local maximum
will be drawn upward toward the peak but will then be stuck with
nowhere else to go.
Local Minima: Similarly, it can get stuck in a local minimum if the
problem is framed as a minimization task.
Plateaus: In flat regions of the search space where no immediate
improvement is apparent, the algorithm may stagnate because it lacks
directional information.
Greedy Behavior : Hill climbing evaluates only immediate neighbors in
the search space. This greedy approach can miss better solutions that
are farther away but accessible via intermediate steps.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 42 / 67

 Hill climbing variants
Stochastic Hill Climbing introduces randomness into the search
process. Instead of evaluating all neighbors or selecting the first
improvement, it selects a random neighboring node and decides
whether to move based on its improvement over the current state.
First-Choice Hill Climbing is a variant of the hill climbing algorithm
that introduces randomness into the selection of neighbors to
evaluate. Instead of examining all possible neighboring states to find
the best move (as in traditional or steepest-ascent hill climbing),
first-choice hill climbing evaluates neighbors in a random order and
selects the first neighbor that offers an improvement in the objective
or heuristic function.
Random-Restart Hill Climbing is a variant of the hill climbing
algorithm designed to overcome the limitations of getting stuck in
local maxima, minima, or plateaus. It achieves this by performing
multiple independent hill-climbing searches from randomly selected
starting points in the search space. By increasing the number of
restarts, the algorithm improves its chances of finding a global
optimum.
BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 43 / 67
 Simulate annealing

Simulated Annealing (SA) is an optimization algorithm inspired by

the annealing process in metallurgy, where a material is heated and
then slowly cooled to achieve a more stable structure. In
computational terms, simulated annealing is used to solve complex
optimization problems by exploring the search space in a way that
avoids getting trapped in local optima.
Idea: escape local minima (or local maxima experienced with
hillclimbing) by allowing some "bad” random moves
Bouncing ball analogy:
I Goal: get ball in deepest crevice of bumpy surface
I If let ball roll, might get stuck in local minimum
I If shake surface hard (high temperature), ball bounces out of LOCAL
min, but if shake too hard, ball will be dislodged from GLOBAL min
I Best start to shake hard, then gradually reduce intensity (lower the
temperature)

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 44 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 45 / 67
 Local beam search
Unlike traditional hill-climbing search, which maintains a single
current state, local beam search keeps track of multiple states
simultaneously, increasing its robustness against getting stuck in local
optima.
Start with k randomly generated initial states, For each state,
compute the evaluation function (e.g., heuristic value), Generate all
successor states for each of the k states, Out of all successors, select
the k best states based on their evaluation scores. Continue
expanding and selecting the k best states until a goal state is found or
another stopping condition is met.

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 46 / 67

 Genetic Algorithms
Genetic Algorithms are the heuristic search and optimization
techniques that mimic the process of natural evolution.
Select the best, Discard the rest.
A successor state is generated by combining two parent states (vs.
modifying a single state in local beam search)

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 47 / 67

BEC515A (Sushma, Dept of ECE, CMRIT) December 8, 2024 48 / 67