Unit 7

Backtracking and Branch

and Bound
Introduction
• In the backtracking method,
1. The desired solution is expressible as an n tuple (x1,x2,…xn ) where xi chosen
from some finite set Si
2. The solution maximizes or minimizes or satisfies a criterion function C (x1.x2,…
xn)
• The problem can be categorized into three categories.
• Finding whether there is any feasible solution? Is the decision problem.
• What is the best solution? Is the optimization problem.
• Listing of all the feasible solution Is the enumeration problem,
• The basic idea of backtracking is to build up a vector, one component at a time and
to test whether the vector being formed has any chance of success.
• The major advantage of backtracking algorithm is that we can realize the fact that
the partial vector generated does not lead to an optimal solution. In such situation
that vector can be ignored.
Cont…
• Backtracking algorithm determines the solution by systematically searching
the solution space (i.e. set of all feasible solutions) for the given problem.
• Backtracking is a depth first search with some bounding function.
• Backtracking algorithm solves the problem using two types of constraints.
Explicit constraint and Implicit constraints.
• Definition: Explicit constraints are the rules that restrict each element x, has
the chosen from given set only. Explicit constraints depends on particular
instance of the problem. All the tuples from solution set must satisfy the
explicit constraints.
• Definition: Implicit constraints are the rules that decide which tuples in the
solution space of I satisfy the criterion function. Thus the implicit constraint
represent by which x, in the solution set must be related with each other.
Terminologies used
• State space: All paths from root to other nodes define the state space of the
problem
• Explicit constraints: Explicit constraints are rules, which restrict, each
vector element to be chosen from given set
• Implicit constraints: Implicit constraints are rules which determine which
of the tuples in the solution space satisfy the criterion function
• Problem states: Each node in the state space tree is called problem state.
Terminologies used
• Solution states: The solution states are those problem states for which the
path from root to s defines a tuple in the solution space. In some trees the
leaves define solution states.
• Answer states: These are the leaf nodes which correspond to an element in
the set of solutions. These are the states which satisfy the implicit
constraints.
• Live node: A node which is generated and whose children have not yet
been generated is called live node.
• E-node: The live node whose children are currently being expanded is
called E-node
• Dead node: A dead node is a generated node which is not to be expanded
further or all of whose children have been generated.
Applications of backtracking
• Various applications that are based on backtracking method are
1. 8 Queen's problem: This is a problem based on chess games. By this
problem, it is stated that arrange the queens on chessboard in such a way
that no two queen scan attack each other.
2. Sub of subset problem.
3. Graph colouring problem.
4. Finding Hamiltonian cycle.
5. Knapsack problem.
The Eight Queens problem
• The n-queen's problem can be stated as follows.
Consider a n x n chessboard on which we have to place n queens so
that no two queens attack each other by being in the same row or in the same
column or on the same diagonal.

The next queen- if is placed on

the paths marked by dotted lines
then they can attack each other
The Eight Queens problem
• The n-queen's problem can be stated as follows.
Consider a n x n chessboard on which we have to place n queens so
that no two queens attack each other by being in the same row or in the same
column or on the same diagonal.
Q Q Q Q Q
Q Q Q Q Q
illegal illegal illegal illegal illegal
But 4 queen’s problem is solvable
Q
Q
Q
Q
The Eight Queens problem
• Example,
• Solve 8 queen’s problem for feasible sequence (6,4,7,1)
6 4 7 1 ? ? ? ?

1 2 3 4 5 6 7 8
1 Q
2 Q
3 Q
4 Q
5
6
7
8
The Eight Queens problem
• Diagonal conflict can be checked using following formula,
• Let, P1=(i, j), P2=(k, l)
• Here P1 and P2 positions are on same diagonal, if
• i+j = k+l OR i-j =k-l
1 2 3 4 5 6 7 8
1 Q
2 Q
3 Q
4 Q
5 Q
6 Q
7 Q
8 Q
The knapsack problem using backtracking

• Problem Statement:
• If we are given n objects a knapsack or a bag in which the object i with
weight wi is to be placed.
• The knapsack has capacity M.
• The profit value that can be earned is Vi.
• Then objective is to obtain filling up of knapsack with maximum profit
earned, but it should not exceed the capacity m of knapsack.
• Here also we can not divide objects in different parts.
Steps to solve knapsack problem using backtracking
• Arrange objects m non-decreasing order (i.e. Vi/Wi)
• Find upper bound value.
• Draw state space tree.
• Follow DFS (Depth First Search)
• Explore node based on bounding function (upper bound)

Note: Upper bound is a value which can be generated by applying greedy

approach.
Example
Object x1 x2 X3 X4 Capacity of Knapsack
M=16
V 45 30 45 10
W 3 5 9 5
V/W 15 6 5 2
Solution cv=0 cw=0 UB x1 x2 x3
UB=115 V 45 30 (45/9)*8=40
x1=1 W 3 5 8 (8 out of 9)
x1=0

cv=45 cw=3 cv=0 cw=0

UB=115 UB=79
x2=0
x2=1
UB x1 x2 x3 x4
cv=75 cw=8 cv=45 cw=3 V 45 30 45 (10/5)*2=4
UB=115 UB=98 W 3 5 9 2(2out of 5)
x3=0
x3=1 x3=1 x3=0

cv=75 cw=8 cv=90 cw=12 cv=45 cw=3

cv=120 cw=17
UB=85 UB=98 UB=55

x4=1 x4=0
1 0 1 0
cv=100 cw=17 cv=90 cw=12
Branch and Bound method
• In branch and bound method a state space tree is built and all the children of E
nodes (a live node whose children are currently being generated) are generated
before any other node can become a live node.
• For exploring new nodes either a BFS or D-search technique can be used.
• In Branch and bound technique, BFS-like state space search will be called FIFO
(First In First Out) search. This is because the list of live node is First In First
Out list (queue). On the other hand the D-search like state space search will be
called LIFO search because the list of live node is Last in First out list (stack).
• In this method a space tree of possible solutions is generated. Then
partitioning(called as branching) is done at each node of the tree. We compute
lower bound and upper bound at each node. This computation leads to selection
of answer node.
• Bounding functions are used to avoid the generation of subtrees that do not
contain an answer node.
Travelling salesman problem
• Problem Statement:
• If there are n cities and cost of travelling from any city to any other city is
given.
• Then we have to obtain the cheapest round trip such that each city is visited
exactly once and then returning to starting city, complete the tour.
Condition:
1. No city is to be visited twice before the
tour of all cities is completed
2. Going from city i to i is not permitted.

1 2 3
1 ∞ 20 30
2 15 ∞ 16
3 3 5 ∞
Example
• Find the solution of the following travelling salesman problem using
dynamic approach and branch and bound approach.
Consider an instance for TSP is given by G as.

∞ 20 30 10 11
15 ∞ 16 4 2
3 5 ∞ 2 4
19 6 18 ∞ 3
16 4 7 16 ∞
Example
• State space tree
Dynamic Reduction
• Steps in dynamic reduction
1. Draw a state space tree with optimum cost at root node.
a) Full reduction (row minimization and column minimization)

entries as ∞, also set M[j][i]=∞.

2. Obtain the cost of matrix for path ij by making ith row and jth column

3. Cost of corresponding node x with i,j is

=optimum cost+ reduced cost+M[i][j].
4. Set node with minimum cost as E-node and generate its
children. Repeat step 1 to 4 for completing tour with
optimum cost.
Step-1:Draw a state space tree with optimum cost at root node.
• To find optimum cost we must know row minimization, column
minimization and full reduction.
• Row minimization
∞ 20 30 10 11 10
15 ∞ 16 4 2 2
∞
Total Reduction=21
3 5 2 4 2
19 6 18 ∞ 3 3
16 4 7 16 ∞ 4
∞ 10 20 0 1
13 ∞ 14 2 0
1 3 ∞ 0 2
16 3 15 ∞ 0
12 0 3 12 ∞
Step-1:Draw a state space tree with optimum cost at root node.
• To find optimum cost we must know row minimization, column
minimization and full reduction.
• column minimization
∞ 10 20 0 1
13 ∞ 14 2 0
1 3 ∞ 0 2 Total Reduction=04
16 3 15 ∞ 0
12 0 3 12 ∞

1 0 3 0 0 ∞ 10 17 0 1
12 ∞ 11 2 0
0 3 ∞ 0 2
15 3 12 ∞ 0
11 0 0 12 ∞
Step-1:Draw a state space tree with optimum cost at root node.
• To find optimum cost we must know row minimization, column
minimization and full reduction.
• Full reduction
∞ 20 30 10 11 ∞ 10 17 0 1
15 ∞ 16 4 2 12 ∞ 11 2 0
∞ ∞
Fully
3 5 2 4 0 3 0 2
∞ ∞
Reduction
19 6 18 3 Matrix 15 3 12 0
16 4 7 16 ∞ 11 0 0 12 ∞

Total Reduced cost is:

= Cost(Red_Row(M)) + Cost(Red_Col(M))
= 21+4
= 25 (Optimum cost)
Step-1:Draw a state space tree with optimum cost at root node.
• To find optimum cost we must know row minimization, column
minimization and full reduction.

column entries as ∞, also set M[j][i]= ∞.

• Step-2: obtain the cost of matrix for path ij by making ith row and jth

• Step-3: cost of 2 is
=optimum cost + reduced cost + old value of M[1][2]
 column entries as ∞, also set M[j][i]= ∞.
• Step-2: obtain the cost of matrix for path ij by making ith row and jth

∞ 10 17 0 1 • Consider path 1,2. make 1st row and 2nd column ∞.

12 ∞ 11 2 0 And set M[2][1]=∞.
0 3 ∞ 0 2
15 3 12 ∞ 0 Step-3: cost of 2 is
11 0 0 12 ∞ = optimum cost + reduced cost + value
of M[1][2] from original matrix
=25+0+10
=35
25

2 1

2
35
 column entries as ∞, also set M[j][i]= ∞.
• Step-2: obtain the cost of matrix for path ij by making ith row and jth

∞ 10 17 0 1 • Consider path 1,3. make 1st row and 3rd column ∞.

12 ∞ 11 2 0 And set M[3][1]=∞.
0 3 ∞ 0 2
15 3 12 ∞ 0
11 0 0 12 ∞
Step-3: cost of 3 is
25
= optimum cost + reduced cost + value
2 1 of M[1][3] from original matrix
3 =25+11+17
2 =53
35 3
53
 column entries as ∞, also set M[j][i]= ∞.
• Step-2: obtain the cost of matrix for path ij by making ith row and jth

∞ 10 17 0 1 • Consider path 1,4. make 1st row and 4th column ∞.

12 ∞ 11 2 0 And set M[4][1]=∞.
0 3 ∞ 0 2 Step-3: cost of 4 is
15 3 12 ∞ 0 = optimum cost + reduced cost + value of
∞
M[1][4] from original matrix
11 0 0 12
=25+0+0
=25

25

2 1
4
3
2
35 4
3
53
25
 column entries as ∞, also set M[j][i]= ∞.
• Step-2: obtain the cost of matrix for path ij by making ith row and jth

∞ 10 17 0 1 • Consider path 1,5. make 1st row and 5th column ∞.

12 ∞ 11 2 0 And set M[5][1]=∞.
0 3 ∞ 0 2
∞
Step-3: cost of 5 is
15 3 12 0
∞
= optimum cost + reduced cost + value of
11 0 0 12 M[1][5] from original matrix
=25+5+1
=31
25

2 1
5
3 4
2
35 5
3 4
53
25 31
E-node
 • Consider path 1,4,2 for node 6. make 1 st row and 4th
row to ∞. Set 2nd column to ∞.
And set M[4][1]=∞, M[2][1]= ∞.

Step-3: cost of 6 is
25 = optimum cost + reduced cost +
1 value of M[4][2] from original matrix
2
5 =25+0+3
3 4 =28
2
35 5
3 4
53
25 31
2

28 6
 • Consider path 1,4,3 for node 7. make 1 st row and 4th
row to ∞. Set 3rd column to ∞.
And set M[4][1]=∞, M[3][1]= ∞.

Step-3: cost of 7 is
25 = optimum cost + reduced cost +
1 value of M[4][3] from original matrix
2
5 =25+13+12
3 4 =50
2
35 5
3 4
53
25 31
2 3

6 7
28 50
 • Consider path 1,4,5 for node 8. make 1 st row and 4th
row to ∞. Set 5th column to ∞.
And set M[4][1]=∞, M[5][1]= ∞.

Step-3: cost of 8 is
25 = optimum cost + reduced cost +
1 value of M[4][5] from original matrix
2
5 =25+11+0
3 4 =36
2
35 5
3 4
53
25 31
2 3 4

6 7 7
28 50 36
E-node
 • Consider path 1,4,2,3 for node 9. make 1 st row and 4th
row and 2nd row to ∞. Set 3rd column to ∞.
And set M[4][1]=∞, M[2][1]= ∞, M[3][1]= ∞.

Step-3: cost of 9 is
25 = optimum cost + reduced cost +
1 value of M[2][3] from original matrix
2
5 =28+13+11
3 4 =52
2
35 5
3 4
53
25 31
2 3 4

6 7 7
28 50 36

52 9
 • Consider path 1,4,2,5 for node 9. make 1 st row and 4th
row and 2nd row to ∞. Set 5th column to ∞.
And set M[4][1]=∞, M[2][1]= ∞, M[5][1]= ∞.

Step-3: cost of 10 is
25 = optimum cost + reduced cost +
1 value of M[2][5] from original matrix
2
5 =28+0+0
3 4 =28
2
35 5
3 4
53
25 31
2 3 4

As for node 10 only child being

6 7 7
28
generated is node 11. we set path as
5 50 36
3 1,4,2,5,3. to complete the tour we
return to 1. hence the state space tree
52 9 10 28 is
E-node
 • As for node 10 only child being generated is node 11. we set path as
1,4,2,5,3. to complete the tour we return to 1. hence the state space tree is:
25

2 1
5
3 4
2
35 5
3 4
53
25 31
2 3 4

6 7 7
28 5 50 36
3

52 9 10 28 Optimum path is:

142531
3
Optimum cost is:28
11
GTU Questions
• Explain Backtracking Method. What is N-Queens Problem? Give
solution of 4- Queens Problem using Backtracking Method
Thank You