CS-323 Artificial Intelligence

Course Teacher: Dr. Maria Waqas

Solving Problems by Searching

Book Chapter: Chapter 3 Solving Problems by Searching [Artificial Intelligence – A Modern Approach, Russell and Norvig, 4/e (Global), 2022]

Uninformed Search Algorithms

Assumptions
 The path cost is the sum of individual step costs along the path.
 The environment is known, fully observable, discrete and deterministic.
 The solution to any problem is a fixed sequence of actions.
 An agent ignores any kind of percepts during the execution phase.

Breadth First Search (BFS)

 The order of expanding the states is such that all the nodes at a given level are expanded before any node at
the next level.
 Implemented by storing the frontier in a FIFO queue.
 Early goal testing makes the algorithm more efficient.
 Assume that the cost of all steps is equal.
 Example: Breadth-first search on a simple binary tree.
 At each stage, the node to be expanded next is indicated by the triangular marker.
 Green nodes: Frontier; Purple nodes: Explored; White nodes: Unexplored

Algorithm
Let S be the initial state and F be a FIFO queue to maintain frontier
If S == goal, return S
Initialize F ← S
Mark S as reached
While F is not empty:
Dequeue node N from the front of F and expand it
For each successor R of N not yet reached:
If R == goal, return R
Enqueue R to the rear of F
Mark R as reached
 For a more detailed pseudocode-style implementation, refer to page 95 of the book.

1
 CS-323 Artificial Intelligence
Course Teacher: Dr. Maria Waqas

Analysis
 For analysis in search problems, the state space graph is often represented as a search tree.
 The graph may contain cycles or multiple paths to the same state, but in the tree each path is shown separately.
This tree representation makes it easier to trace the steps of the search algorithm. However a cycle in a graph
results in an infinite branch when represented in a tree.
 Hence consider a uniform search tree with a branching factor 𝑏 assuming that goal node lies at depth 𝑑.
 Completeness: Complete if 𝑏 is finite.
 Optimality: Optimal if all step costs are equal.
 Time Complexity: Proportional to the number of nodes generated.
For the worst case analysis, it must be the last node generated.
Number of nodes generated at level 1 = 𝑏
Number of nodes generated at level 2 = 𝑏 2
Number of nodes generated at level 3 = 𝑏 3
---
Number of nodes generated at level d = 𝑏 𝑑
Therefore total number of nodes generated (ignoring depth 0) = ∑𝑑𝑖=1 𝑏 𝑖 → 𝑂(𝑏 𝑑 ) – Exponential complexity
 Space complexity: Proportional to the number of nodes stored in memory.
𝑂(𝑏 𝑑 ) – Exponential complexity

Scalability
Let 𝑏 = 10, 1M nodes can be generated per second, 1000 B of storage per node.

 Space complexity seems bigger concern than time complexity.

 Even time complexity becomes unbearable for problems having goal at a depth above a small threshold (> 10
in this example).

2
 CS-323 Artificial Intelligence
Course Teacher: Dr. Maria Waqas

Depth First Search (DFS)

 Depth-first search always expands the deepest node in the frontier first.
 Search proceeds immediately to the deepest level of the search tree, where the nodes have no successors. The
search then “backs up” to the next deepest node that still has unexpanded successors.
 Depth-first search is not cost-optimal; it returns the first solution it finds, even if it is not cheapest.
 Implemented by storing the frontier in a LIFO Queue (stack).
 Uses early goal testing.
 Assume that the cost of all steps is equal.
 Example: Depth-first search on a simple binary tree.
 Green nodes: Frontier; Purple nodes: Explored; White nodes: Unexplored

Algorithm
Let S be the initial state and F be a LIFO queue to maintain frontier
If S == goal, return S
Initialize F ← S
Mark S reached
While F is not empty:
Pop node N from top of F and expand it
For each successor R of N not reached
If R == goal, return R
Push R to the top of F
Mark R reached

3
 CS-323 Artificial Intelligence
Course Teacher: Dr. Maria Waqas

Analysis
 Completeness: Provided both d and b are finite.
 Optimality: Not optimal; may return the deepest goal node ignoring a shallower goal node on some earlier
level.
 Time Complexity: Proportional to the number of nodes generated.
For the worst case analysis, it must be the last node generated.
Total number of nodes generated (ignoring depth 0) = ∑𝑑𝑖=1 𝑏 𝑖 → 𝑂(𝑏 𝑑 ) – Exponential complexity.
 Space Complexity: Maximum size of F happens when goal is the deepest node of search tree. If d is the
maximum depth then then total nodes generated would be 𝑏𝑑, hence space complexity is 𝑂(𝑑) – Linear
complexity.
If the state space is a tree (no repeated states), then we don’t even need a reached list. DFS just goes down a
path and backtracks, so memory is minimal.

Depth Limited Search (DLS)

 DFS with a cutoff depth 𝑙.
 Nodes deeper than the limit are not expanded.
 Prevents infinite descent in infinite trees/graphs.
 The time complexity is 𝑂(𝑏 𝑙 ) and the space complexity is 𝑂(𝑏).
 If the limit is too small, the solution may be missed, making it incomplete again.
 Iterative deepening search solves the problem of picking a good value for 𝑙 by trying all values.

Algorithm
 DFS with depth limit L
Let S be the initial state, F be a LIFO queue to maintain frontier, and L be the depth limit
If S == goal, return S
Initialize F ← S
Mark S reached
While F is not empty:
Pop node N from top of F and expand it
If depth of N <= L:
For each successor R of N not reached
If R == goal, return R
Push R to the top of F
Mark R reached

Iterative Deepening Search (IDS)

 Runs DLS repeatedly with increasing depth limits: 0, 1, 2, … until solution found.
 By gradually increasing the depth limit, IDS avoids the risk of choosing a poor or incorrect depth
bound, while still combining the low memory usage of DFS with the completeness and optimality of
BFS (for uniform-cost problems).
 Widely used when depth of solution is unknown.

4
 CS-323 Artificial Intelligence
Course Teacher: Dr. Maria Waqas

 Example: Four iterations of iterative deepening search for goal 𝑀 on a binary tree, with the depth limit varying
from 0 to 3. [Green nodes: Frontier; Purple nodes: Explored; White nodes: Unexplored]

Algorithm
Let S be the initial state, and L be the depth limit
L←0
While True:
result ← DLS(S, L)
If result is solution, return.
L←L+1

Analysis
 Completeness: Provided if b is finite (like BFS).
 Optimality: Optimal if all step costs are equal (like BFS).
 Time Complexity: Proportional to the number of nodes generated.
For the worst case analysis, it must be the last node generated at maximum depth 𝑑.

5
 CS-323 Artificial Intelligence
Course Teacher: Dr. Maria Waqas

Total number of nodes generated (ignoring depth 0) = ∑𝑑𝐿=1 ∑𝐿𝑖=1 𝑏 𝑖 → 𝑂(𝑏 𝑑 ) – Exponential complexity.
Although nodes near the top levels are re-generated many times, this overhead is small compared to the last
level. Hence IDS works well for large state spaces.
Example:
Assume 𝑏 = 10 and 𝑑 = 5
BFS = ∑𝑑𝑖=1 𝑏 𝑖 = ∑5𝑖=1 10𝑖 = 10 + 100 + 1000 + 10000 + 100000 = 111110
IDS = ∑𝑑𝐿=1 ∑𝐿𝑖=1 𝑏 𝑖 = ∑𝑑𝐿=1 ∑𝐿𝑖=1 𝑏 𝑖
= 10 + 10 + 100 + 10 + 100 + 1000 + 10 + 100 + 1000 + 10000 + 10 + 100 + 1000 +10000 + 100000
= 50 + 400 + 3000 + 20000 + 100000 = 123450
Overhead of IDS vs BFS: 123450 / 111110 = 1.11 ⇒ 11.1% more node generations.
 Space Complexity: 𝑂(𝑑) – Linear complexity (like DFS).

Uniform Cost Search (UCS)

 Also called Dijkstra’s algorithm.
 While BFS spreads out in waves of uniform depth, UCS spreads out in waves of uniform path-cost.
 Returns least-cost path i.e. the optimal one.
 Maintains frontier as a priority queue ordered as per path costs.
 Used late goal testing.

Algorithm
 Same as BFS with the following changes:
 Priority queue is used, prioritizing the lowest cost.
 Doesn’t make use of reached list as it may result in overlooking the optimal path.

Let S be the initial state and F be a priority queue to maintain frontier

If S == goal, return S [can be omitted, checked at the start for convenience]
Initialize F ← S
While F is not empty:
Dequeue node N with lowest path cost from F
If N == goal, return N
Expand N
For each successor R of N:
Enqueue R

Analysis
 The complexity of uniform-cost search is characterized in terms of 𝐶 ∗ , the cost of the optimal solution, and 𝜖,
a lower bound on the cost of each action, with 𝜖 > 0 (cost must be positive).
 Completeness: Complete provided step cost > 𝜖 > 0 (cost must be positive).
 Optimality: Yes.
𝐶∗
1+⌊ ⌋
 Time Complexity: 𝑂 (𝑏 𝜖 ), ), which can be much greater than 𝑏 𝑑 .
 [𝐶 ∗ / 𝜖 is the average number of steps (depth), and 1 is added because of late goal testing.]
This is because uniform-cost search can explore large trees of actions with low costs before exploring paths
involving a high-cost and perhaps useful action.
𝐶∗
1+⌊ ⌋
 Space Complexity: 𝑂(𝑏 𝜖 )

 Although complexity is exponential, but the optimality feature makes it stand apart.
 If path costs are equal, UCS decomposes into BFS.