Lecture 02 – Part A

Problem Solving by Searching

Search Methods :
Uninformed (Blind) search
 Search Methods
 Once we have defined the problem space (state representation, the initial state, the goal state
and operators) is all done?

 Let’s consider the River Problem:

A farmer wishes to carry a wolf, a duck and corn across a river, from the south to the north
shore. The farmer is the proud owner of a small rowing boat called Bounty which he feels is
easily up to the job. Unfortunately the boat is only large enough to carry at most the farmer
and one other item. Worse again, if left unattended the wolf will eat the duck and the duck
will eat the corn.

River

boat
How can the farmer safely transport the wolf,
Farmer, Wolf,the duck and the corn to the opposite shore?
Duck and Corn

2
 Search Methods
The River Problem:
F=Farmer W=Wolf D=Duck C=Corn /=River

-/FWCD

FWCD/-

How can the farmer safely transport the wolf, the duck
and the corn to the opposite shore?
3
 Search Methods
 Problem formulation:

 State representation: location of farmer and items in both sides of river

[items in South shore / items in North shore] : (FWDC/-, FD/WC, C/FWD …)

 Initial State: farmer, wolf, duck and corn in the south shore FWDC/-

 Goal State: farmer, duck and corn in the north shore

-/FWDC

 Operators: the farmer takes in the boat at most one item from one side to the
other side
(F-Takes-W, F-Takes-D, F-Takes-C, F-Takes-Self [himself only])

 Path cost: the number of crossings

4
 Search Methods
 State space:
A problem is solved by moving from the initial state to the goal state by applying valid
operators in sequence. Thus the state space is the set of states reachable from a particular
initial state.

Initial state
F WD C

WD C D C W C WD
F F W F D F C

Dead ends
F W C F WD C

Illegal states
D

intermediate state
W C C W

repeated state F D F WD F D C

F C F W C F D C F W F WD F W C
WD D W D C C D

D C C D WD D W
F W F WD F W C F C F W C F D C

F D F WD F D C
W C C W

D
5
F W C F WD C
Goal state
 Search Methods
 Searching for a solution:

We start with the initial state and F WD C

keep using the operators to expand

the parent nodes till we find a goal WD C DC W C WD

state. F FW F D F C

FW C F WD C
D

 …but the search space might be

large… F
W C
D F WD
C
F
W
DC

 …really large… F
WD
C FW C
D
F
W
DC FW
DC
F WD
C
FW C
D

 So we need some systematic way to FW

DC
F WD
C D
FW C F
WD
C
D
FW C F
W
DC

search.
F D F WD F DC
W C C W

D
FW C F WD C
6
 Search Methods
 Problem solution:
F WD C

A problem solution is simply the set

of operators (actions) needed to reach WD C DC W C WD

the goal state from the initial state: F FW F D F C

FW C F WD C
D

F-Takes-D, F-Takes-Self, F-Takes-W, F

W C
D F WD
C
F
W
DC

F C FW C F DC FW F WD FW C
F-Takes-D, F-Takes-C, F-Takes-Self, WD D W DC C D

DC C D WD D W
FW F WD FW C F C FW C F DC
F-Takes-D.
F D F WD F DC
W C C W

D
FW C F WD C
7
 Search Methods
 Problem solution: (path Cost = 7)
While there are other possibilities here is one 7 step solution to the river problem

F D D F W D
F-Takes-D F-Takes-S F-Takes-W

F W D C W C F W C C
Initial State WC/FD FWC/D C/FWD

F-Takes-D

F W G C W C F W C W

F-Takes-D F-Takes-S F-Takes-C

F D D F D C
Goal State FD/WC D/FWC FDC/W

8
Problem Solving by searching

Basic Search Algorithms

 Basic Search Algorithms

 uninformed( Blind) search: breadth-first, depth-first, depth

limited, iterative deepening, and bidirectional search

 informed (Heuristic) search: search is guided by an evaluation

function: Greedy best-first, A*, IDA*, and beam search

 optimization in which the search is to find an optimal value of an

objective function: hill climbing, simulated annealing, genetic
algorithms, Ant Colony Optimization

 Game playing, an adversarial search: minimax algorithm, alpha-

beta pruning
10
 What Criteria are used to Compare different
search techniques ?

As we are going to consider different techniques to search the problem

space, we need to consider what criteria we will use to compare them.

 Completeness: Is the technique guaranteed to find an answer (if there

is one).

 Optimality/Admissibility : does it always find a least-cost solution?

- an admissible algorithm will find a solution with minimum cost

 Time Complexity: How long does it take to find a solution.

 Space Complexity: How much memory does it take to find a solution.

11
 Time and Space Complexity ?
Time and space complexity are measured in terms of:

 The average number of new nodes we create when expanding a new

node is the (effective) branching factor b.

 The (maximum) branching factor b is defined as the maximum nodes

created when a new node is expanded.

 The length of a path to a goal is the depth d.

 The maximum length of any path in the state space m.

12
 Branching factors for some problems

 The eight puzzle has a (effective) branching 2 1 3

factor of 2.13, so a search tree at depth 20 has
about 3.7 million nodes: O(bd) 4 7 6
5 8
 Rubik’s cube has a (effective) branching factor
of 13.34. There are 901,083,404,981,813,616
different states. The average depth of a solution
is about 18.

 Chess has a branching factor of about 35, there

are about 10120 states (there are about 1079
electrons in the universe).

13
 Uninformed search strategies (Blind
search)
Uninformed (blind) strategies use only the information available in the
problem definition. These strategies order nodes without using any
domain specific information

Contrary to Informed search techniques which might have additional

information (e.g. a compass).

 Breadth-first search
 Uniform-cost search
 Depth-first search
 Depth-limited search
 Iterative deepening search
 Bidirectional search

14
Basic Search Algorithms
Uninformed Search
Breadth First Search (BFS)
 Breadth First Search (BFS)

16
 Breadth First Search (BFS)

Main idea: Expand all nodes at depth (i) before expanding nodes at depth (i +
1)
Level-order Traversal.
Implementation: Use of a First-In-First-Out queue (FIFO). Nodes visited first
are expanded first. Enqueue nodes in FIFO (first-in, first-out) order.
• Complete? Yes.
• Optimal? Yes, if path cost is nondecreasing function of depth
• Time Complexity: O(bd)
• Space Complexity: O(bd), note that every node in the fringe is kept in the queue.
17
 Breadth First Search
 Application1:
Given the following state space (tree search), give the sequence
of visited nodes when using BFS (assume that the nodeO is the
goal state):
A

B C D E

F G H I J

K L M N

O
18
 Breadth First Search
 A,

B C D E

19
 Breadth First Search
 A,
 B,

B C D E

F G

20
 Breadth First Search
 A,
 B,C

B C D E

F G H

21
 Breadth First Search
 A,
 B,C,D

B C D E

F G H I J

22
 Breadth First Search
 A,
 B,C,D,E

B C D E

F G H I J

23
 Breadth First Search
 A,
 B,C,D,E,
 F,
A

B C D E

F G H I J

24
 Breadth First Search
 A,
 B,C,D,E,
 F,G
A

B C D E

F G H I J

K L

25
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H
A

B C D E

F G H I J

K L

26
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I
A

B C D E

F G H I J

K L M

27
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
A

B C D E

F G H I J

K L M N

28
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
 K, A

B C D E

F G H I J

K L M N

29
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
 K,L A

B C D E

F G H I J

K L M N

O
30
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
 K,L, M, A

B C D E

F G H I J

K L M N

O
31
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
 K,L, M,N, A

B C D E

F G H I J

K L M N

O
32
 Breadth First Search
 A,
 B,C,D,E,
 F,G,H,I,J,
 K,L, M,N, A
 Goal state: O
B C D E

F G H I J

K L M N

O
33
 Breadth First Search
 The returned solution is the sequence of operators in the path:
A, B, G, L, O

B C D E

F G H I J

K L M N

O
34
Basic Search Algorithms
Uninformed Search
Uniform Cost Search (UCS)
 Uniform Cost Search (UCS)

5 2
[5] [2]
1 4 1 7
[6] [9] [3] [9]
Goal state
4 5
[x] = g(n) [7] [8]
path cost of node n
36
 Uniform Cost Search (UCS)

5 2
[5] [2]

37
 Uniform Cost Search (UCS)

5 2
[5] [2]

1 7
[3] [9]

38
 Uniform Cost Search (UCS)

5 2
[5] [2]

1 7
[3] [9]
4 5

[7] [8]

39
 Uniform Cost Search (UCS)

5 2
[5] [2]
1 4 1 7
[6] [9] [3] [9]
4 5

[7] [8]

40
 Uniform Cost Search (UCS)

5 2
[5] [2]
Goal state
1 4 1 7
path cost
g(n)=[6] [3] [9]
[9]
4 5

[7] [8]

41
 Uniform Cost Search (UCS)

5 2
[5] [2]
1 4 1 7
[6] [9] [3] [9]
4 5

[7] [8]

42
 Uniform Cost Search (UCS)
 In case of equal step costs, Breadth First search finds
the optimal solution.

 For any step-cost function, Uniform Cost search

expands the node n with the lowest path cost.

 UCS takes into account the total cost: g(n).

 UCS is guided by path costs rather than depths.

Nodes are ordered according to their path cost.
43
 Uniform Cost Search (UCS)
 Main idea: Expand the cheapest node. Where the cost is the path cost g(n).

 Implementation:
Enqueue nodes in order of cost g(n).
QUEUING-FN:- insert in order of increasing path cost.
Enqueue new node at the appropriate position in the queue so that we
dequeue the cheapest node.

 Complete? Yes.
 Optimal? Yes, if path cost is nondecreasing function of depth
 Time Complexity: O(bd)
 Space Complexity: O(bd), note that every node in the fringe keep in the queue.

44
Basic Search Algorithms
Uninformed Search
Depth First Search (DFS)
 Depth First Search (DFS)

46
 Depth First Search (DFS)
 Application2:
Given the following state space (tree search), give the sequence
of visited nodes when using DFS (assume that the nodeO is the
goal state):
A

B C D E

F G H I J

K L M N

O
47
 Depth First Search
 A,

B C D E

48
 Depth First Search
 A,B,

B C D E

F G

49
 Depth First Search
 A,B,F,

B C D E

F G

50
 Depth First Search
 A,B,F,
 G,

B C D E

F G

K L

51
 Depth First Search
 A,B,F,
 G,K,

B C D E

F G

K L

52
 Depth First Search
 A,B,F,
 G,K,
 L,
A

B C D E

F G

K L

O
53
 Depth First Search
 A,B,F,
 G,K,
 L, O: Goal State
A

B C D E

F G

K L

O
54
 Depth First Search
The returned solution is the sequence of operators in the path:
A, B, G, L, O

B C D E

F G

K L

O
55
 Depth First Search

56
 Depth First Search (DFS)

Main idea: Expand node at the deepest level (breaking ties left to right).

Implementation: use of a Last-In-First-Out queue or stack(LIFO). Enqueue nodes in

LIFO (last-in, first-out) order.

• Complete? No (Yes on finite trees, with no loops).

• Optimal? No

• Time Complexity: O(bm), where m is the maximum depth.

• Space Complexity: O(bm), where m is the maximum depth.

57
 Mapping DFS to Real
World Problems
 Traversing a Maze

 Always left (right) hand on the wall

58
 Mapping DFS to Real
World Problems
 Searching for a Gift
 Number of shops, each have several floors

 What you naturally do?

59
Basic Search Algorithms
Uninformed Search
Depth-Limited Search (DLS)
 Depth-Limited Search (DLS)

61
Depth Bound = 3
 Depth-Limited Search (DLS)
 Application3:
Given the following state space (tree search), give the sequence
of visited nodes when using DLS (Limit = 2):

Limit = 0 A

Limit = 1 B C D E

Limit = 2 F G H I J

K L M N

O
62
 Depth-Limited Search (DLS)
 A,

B C D E

Limit = 2

63
 Depth-Limited Search (DLS)
 A,B,

B C D E

Limit = 2 F G

64
 Depth-Limited Search (DLS)
 A,B,F,

B C D E

Limit = 2 F G

65
 Depth-Limited Search (DLS)
 A,B,F,
 G,

B C D E

Limit = 2 F G

66
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,
A

B C D E

Limit = 2 F G H

67
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
A

B C D E

Limit = 2 F G H

68
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
 D, A

B C D E

Limit = 2 F G H I J

69
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
 D,I A

B C D E

Limit = 2 F G H I J

70
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
 D,I A
 J,
B C D E

Limit = 2 F G H I J

71
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
 D,I A
 J,
 E B C D E

Limit = 2 F G H I J

72
 Depth-Limited Search (DLS)
 A,B,F,
 G,
 C,H,
 D,I A
 J,
 E, Failure B C D E

Limit = 2 F G H I J

73
 Depth-Limited Search (DLS)
 DLS algorithm returns Failure (no solution)
 The reason is that the goal is beyond the limit (Limit =2): the
goal depth is (d=4)
A

B C D E

Limit = 2 F G H I J

K L M N

O
74
 Depth-Limited Search (DLS)
It is simply DFS with a depth bound.
 Searching is not permitted beyond the depth bound.
 Works well if we know what the depth of the solution is.
 Termination is guaranteed.
 If the solution is beneath the depth bound, the search cannot
find the goal (hence this search algorithm is incomplete).
 Otherwise use Iterative deepening search (IDS).

75
 Depth-Limited Search (DLS)
Main idea: Expand node at the deepest level, but limit depth to L.

Implementation:
Enqueue nodes in LIFO (last-in, first-out) order. But limit depth to
L

•Complete? Yes if there is a goal state at a depth less than L

• Optimal? No

• Time Complexity: O(bL), where L is the cutoff.

76
• Space Complexity: O(bL), where L is the cutoff.
Basic Search Algorithms
Uninformed Search
Iterative Deepening Search (IDS)
 Iterative Deepening Search (IDS)

function ITERATIVE-DEEPENING-SEARCH():

for depth = 0 to infinity do

if DEPTH-LIMITED-SEARCH(depth) succeeds
then return its result
end
return failure

78
Iterative deepening search L=0
Iterative deepening search L=1
Iterative deepening search L=2
Iterative Deepening Search L=3
Iterative deepening search
 Iterative Deepening Search (IDS)
 Key idea
 Iterative deepening search (IDS) applies DLS repeatedly
with increasing depth. It terminates when a solution is
found or no solutions exists.

 IDS combines the benefits of BFS and DFS

 Like DFS the memory requirements are very modest
(O(bd)).
 Like BFS, it is complete when the branching factor is
finite.

84
 Iterative Deepening Search (IDS)
 Key idea
 Seems wastefull becuase states are generated multiple times
 But not like that, it is as efficient as BFS or DFS; because majority of
the nodes are in the deepest level
 For BFS total number of generated nodes are
N(BFS) = 1 + b + b2 + b3 + … … + bd
= (1 – bd+1) / (1 – b )
 For Iterative Deepening Search
 Nodes at bottom level (depth d) are generated once
 Next to bottom generated twice
 The total number of generated nodes is
N(IDS)= db + (d-1)b2 + (d-2)b3 + … … + (1)bd + (d+1)

85
 Iterative Deepening Search (IDS)
 Coparison between DFS and IDS
 Let a tree of depth 4 and branching factor of 10
 For BFS total number of generated nodes are
N(BFS) = 1 + b + b2 + b3 + … … + bd
= (1 – bd+1) / (1 – b )
= (1 – 104+1) / (1 – 10)
= 11,111 nodes
 For Iterative Deepening Search (IDS)
 The total number of generated nodes is
N(IDS)= (d+1)b0 + db1 + (d-1)b2 + (d-2)b3 + … … + (1)bd
= (4 + 1) + 4 * 10 + 3 + 100 + 2 * 1000 + 10000
= 12,345 nodes
 In general, iterative deepening is the preferred uninformed search
method when there is a large search space and the depth of the
86
solution is not known.
Basic Search Algorithms
Uninformed Search
Bi-Directional Search (BDS)
 Bi-directional Search (BDS)

 Main idea: Start searching from both

the initial state and the goal state,
meet in the middle.

 Complete? Yes
 Optimal? Yes
 Time Complexity: O(bd/2), where d is
the depth of the solution.
 Space Complexity: O(bd/2), where d is
the depth of the solution.

88
Basic Search Algorithms

Comparison of search algorithms

 Comparison of search algorithms

b: Branching factor
d: Depth of solution
m: Maximum depth

l : Depth Limit
90
 Summary

 Search: process of constructing sequences of actions that achieve a goal given a problem.

 The studied methods assume that the environment is observable, deterministic, static and
completely known.
 Goal formulation is the first step in solving problems by searching. It facilitates problem
formulation.
 Formulating a problem requires specifying four components: Initial states, operators, goal
test and path cost function. Environment is represented as a state space.
 A solution is a path from the initial state to a goal state.

 Search algorithms are judged on the basis of completeness, optimality, time complexity
and space complexity.
 Several search strategies: BFS, DFS, DLS, IDS,…

 All uninformed searches have an exponential time complexity – hopeless as a viable

problem solving mechanism (unless you have a quantum computer!)
91
 References
Chapter 3 of “Artificial
Intelligence: A modern
approach” by Stuart Russell, Peter Norvig.
Chapter 4 of “Artificial Intelligence Illuminated”
by Ben Coppin

92