Amity School of Engineering and Technology

Amity School of Engineering & Technology

B.Tech CSE , 6th Semester
Analysis & Design of Algorithms
Anant Kr Jayswal
MTech(CSE)-JNU, PhD*(CSE)-JNU
GATE(CS), UGC-NET(CS)

1 1
 Amity School of Engineering and Technology

Backtracking Approach
• The term backtracking was coined by American Mathematician D. H.
Lehmer in 1950’s.
• Backtracking is the approach of finding the solution of any problem by
constructing a state-space tree of a given problem.
• Backtracking uses the brute-force approach, and this approach is used when
the problem has many solution and we want to explore all the solution.
• Let we have 3 kids (2 Boys, 1 Girl) and we have 3 Chairs. There are 3!=6
arrangements of these kids, in a given 3 chairs. We can maintained these
arrangements using state-space tree as follows:
• State-space tree represent all possible
Solution of the given problem.

2
 Amity School of Engineering and Technology

Backtracking Approach
Backtracking algorithm try each possibility, until it finds the right one.
During the search if alternate does not work, the search backtrack to the choice
point, and try for the next alternatives, and so on.
If all the alternatives are exhausted (at all choice point), and no solution found
then search fails.

State-space tree:
In Backtracking, while solving a problem, a tree is constructed based on the
choices made. Such a tree with all possible solution is called a SST.

3
 Amity School of Engineering and Technology

Backtracking Process Dead End

?
Dead End
Dead End
START ? ? ?

Dead End
?

Dead End
Success

? Represents the multiple options or paths 4

 Amity School of Engineering and Technology

Backtracking Approach
• Problems searching for a set of solutions or which require an optimal
solution can be solved using the backtracking method .
• Backtracking is used to solve problem in which a sequence of objects is
chosen from a specified set so that the sequence satisfies some criterion.
• The desired solution is expressed as an n-tuple (x1, . . . . , xn) where each xi
Є S, S being a finite set.
• The solution is based on finding one or more vectors that maximize,
minimize, or satisfy a criterion function P (x1, . . . . . , xn).
• Form a solution and check at every step if this has any chance of success. If
the solution at any point seems not promising, ignore it.
• All solutions requires a set of constraints divided into two categories:
explicit and implicit constraints.

5
 Amity School of Engineering and Technology

Problem Statement (N Queen’s Problem)

• N Queen is the problem of placing N chess queens on an N˟N chessboard so that no two
queens attack to each other.
• Two queen’s attack to each other, when they are in same row/column or diagonal.

• For example, the following is the one of the solution for 4 Queen Problem.
Each queen is placed such that it does not attack any other queen on the board.

A Queen can attack

vertically, horizontally
and diagonally.

So, no two queen

“attack”, means, no two
of them are on the same
row, column, or diagonal.

6
 Amity School of Engineering and Technology
How to solve N=4 Queen’s problem(𝑄1 , 𝑄2 , 𝑄3 , 𝑄4 ) using
Backtracking? and a state space tree for 4 Queen’s:
We place queen row-by-row (i.e. 𝑄1 in row 1, 𝑄2 in row 2 and so on).
Backtracking gives “all possible solution”. If you want optimal solution, then
go for Dynamic programming.
Let’s see Backtracking method, there are 164 ways to place a queen on a
𝐶
4x4 chess board. So to reduce the size (not any where on chess board, since
there are 164𝐶 Possibilities), we place queen row-by-row, and no same
column.

7
 Amity School of Engineering and Technology
State-space Tree
• This tree is called a permutation tree (here we avoids same row or same columns
but allowing diagonals)
• Total nodes=1 + 4 + 4 × 3 + 4 × 3 × 2 + 4 × 3 × 2 × 1 = 65
• The edges are labeled by possible values of xi. Edges from level 1 to level 2 nodes
specify the values for x1. Edges from level i to level i+1 are labeled with the values
of xi.
• The solution space is defined by all paths from root node to leaf node. There are 4!
= 24 leaf nodes are in the tree.
• Nodes are numbered as depth first Search.

8
 Amity School of Engineering and Technology
State-space Tree
• Now we solve with the idea (Avoid same row, avoid same columns and avoid
same diagonals)

Two solutions:
2 4 1 3

3 1 4 2 Mirror image

9
 Amity School of Engineering and Technology
• Given a 4 *4 chessboard and number the rows and column of the chessboard
from 1 through 4.
• Without loss of generality, assume that ith queen can be placed in ith row,
because no two queen can place in the same row.

• All solution can represented as 4-tuple 1 2 3 4

(x1,x2,x3,x4) where xi is the column
number of the ith row of ith queen placed. 1
• Here explicit constraints are Si = {1,2,3,4},
1 ≤ i ≤ 4 and the solution space will consist 2
of 44 4-tuple.
3
• But, according to the implicit constraints no
two queen can on the same row. So, all 4
solution are permutation of 4-tuple(1, 2, 3,
4)
• Thus the searches is reduce to 44 4-tuple to
4! tuple.
10
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
3

11
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

12
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3
B

13
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8
B

14
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8
B
9
B

15
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8
B
9 11

B B

16
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8
B
9 11

B B

17
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11
B B

18
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14
B B

19
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14
B B
15

20
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14 16
B B
15

21
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14 16
B B B
15

22
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14 16
B B B
15

23
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2 18
x2=2 x2=4
3 x 2= 3

4 3 8 13
B
9 11 14 16
B B B
15

24
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2 18
x2=2 x2=4
3 x 2= 3

4 3 8 13 19
B B
9 11 14 16
B B B
15

25
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2 18
x2=2 x2=4
3 x 2= 3

4 3 8 13 19 24
B B B
9 11 14 16
B B B
15

26
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2 18
x2=2 x2=4 x2= 4
3 x 2= 3

4 3 8 13 19 24 29
B B B
x3 = 1

9 11 14 16 30
B B B x4 = 3

15
31
B

27
 Amity School of Engineering and Technology
1 2 3 4
1
1 x1=1 x1= 2

2
2 18
x2=2 x2=4 x2= 4
3 x 2= 3

4 3 8 13 19 24 29
B B B
x3 = 1

9 11 14 16 30
B B B x4 = 3

x1 x2 x3 x4 15 31
Solution vector:
2 4 1 3
B

28
 Amity School of Engineering and Technology
N Quuen’s Problem Formulation

29
 Amity School of Engineering and Technology
N Quuen’s Problem Formulation

30
Amity School of Engineering and Technology

31
Amity School of Engineering and Technology

32
Amity School of Engineering and Technology

33
Amity School of Engineering and Technology

34
Amity School of Engineering and Technology

35
Amity School of Engineering and Technology

36
Amity School of Engineering and Technology

37
Amity School of Engineering and Technology

38
Amity School of Engineering and Technology

39
Amity School of Engineering and Technology

40
Amity School of Engineering and Technology

41
Amity School of Engineering and Technology

42
 Amity School of Engineering and Technology
8 Queen Problem
8-Queens Problem is to place eight queens on an 8 x 8 chessboard so
that no two “attack”, i.e., no two of them are on the same row, column,
or diagonal.

Two queens lie on the same diagonal if and only if:

𝑗−𝑙 = 𝑖−𝑘

where, j be the column of object in row i for the ith queen and l be the
column of object in row k for the kth queen.

43
Amity School of Engineering and Technology

44
 Amity School of Engineering and Technology
Algorithm (N-Queen’s problem)

45
 Amity School of Engineering and Technology

1 2 3 4 5 6 7 8
Let us consider for the case whether the queens
1 on 3rd row and 8th row are conflicting or not.
2 In this case (i, j) = (3, 1) and (k, l) = (8, 6).
3 Therefore:
4 𝑗−𝑙 = 𝑖−𝑘
5 6−5 = 3−8
6
5 = 5
7
8 Thus, the condition is true i.e both the queens
are diagonally conflict each other.

46
 Amity School of Engineering and Technology

Exercise

Solve 8-Queen Problem by using backtrack approach

and draw the state space tree.

47
 Amity School of Engineering and Technology
Problem Statement
The problem is to find a subset of a given set 𝑆 =
𝑛1 , 𝑛2 , … . . , 𝑛𝑛 of n positive integers, whose sum is
equal to a given positive integer ‘d’.
𝑛1 ≤ 𝑛2 ≤ ⋯ ≤ 𝑛𝑛

For Example:
Given a set S ={3, 4, 5, 6} and d =9. Obtain the subset
sum(s) using Backtracking approach.

48
 Amity School of Engineering and Technology

Solution

• The Subset-Sum Problem can be solved by using the

backtracking approach.

• We assume that the elements of the given set are arranged

in increasing order.
𝑛1 ≤ 𝑛2 ≤ ⋯ ≤ 𝑛𝑛

49
 Amity School of Engineering and Technology
• In this implicit tree is a binary tree.

• The root node represents that no decision is

yet taken on any input. For Example:
Given a set S = (3, 4, 5, 6)

• The left child indicates if element of set i.e.

'S1' is included.
0
• The right child indicates if 'S1‘ is not IN 3 EX 3
included i.e. excluded.

• Each node stores the total of the partial 3 0

solution elements.

• If at any stage the sum equals to 'X' then the

search is successful and terminates. 50
 Amity School of Engineering and Technology

• The dead end in the tree appears only when either of the two
inequalities exists:

– The sum of s' is too large i.e.

𝑠 ′ + 𝑠𝑖+1 > 𝑑

– The sum of s' is too small i.e.

51
 Example
Amity School of1 Engineering and Technology
Given a set S = (3, 4, 5, 6) and d =9. Obtain the subset sum using
Backtracking approach.

0
EX 3
IN 3

3 0
IN 4 EX 4 EX 4
IN 4

3 4 0
7
EX 5
EX 5 IN 5 EX 5 IN 5
IN 5

12 7 8 3 9 4

IN 6 EX 6
EX 6 EX 6 IN 6
IN 6

11
8 9 3
13 7 10 4

52
Amity School of Engineering and Technology
Example2

53
 Amity School of Engineering and Technology
Example3
Given W[1…6]={5,10,12,13,15,18},
n=6, M=30

54
Amity School of Engineering and Technology
Algorithm

55
 Amity School of Engineering and Technology

Algorithm

56
 Example 2
Amity School of Engineering and Technology
Given a set S = (4,6,7,8) and X =18. Obtain the subset sum using Backtracking
approach.
0
EX 4
IN 4

4 0
IN 6 EX 6 EX 6
IN 6

4 6 0
10
IN 7 EX 7 EX 7
IN 7 EX 7 IN 7

17 10 11 4 13 6

IN 8 EX 8
EX 8 EX 8 IN 8
IN 8

12 4
18 10 14 6
57
 Amity School of Engineering and Technology

0/1 Knapsack problem (Using Backtracking)

• Lets there are n number of items are given
– weight of these items is given as W = ( w1, w2,….,wn ), where wi is positive
weight of ith item.
– value of profits corresponding to each item is represented as ( p1, p2,….,pn ),
where pi represents profit of ith item, and
– The capacity of knapsack is m, where m > 0.

The 0-1 knapsack problem is

To fill the knapsack in such a manner that total

weight of the items should not exceed the
knapsack capacity and maximum possible
value can be obtained.

58
 Amity School of Engineering and Technology

Problem statement

• The 0-1 knapsack problem chooses

subset of weight set W such that

is maximized

where xi € { 0, 1 } i.e ith item is

selected then xi = 1 or otherwise 0.

59
 Amity School of Engineering and Technology

Methodology to Solve
• Solve this problem using a state space tree exactly like the one in the Sum-
of-Subsets problem.
• This problem is different from the others as it is an optimization problem.
This means that we do not know if a node contains a solution until the
search is over.
• Therefore, we backtrack a little differently. If the items included up to a
node have a greater total profit than the best solution so far, we change the
value of the best solution so far.
• However, we may still find a better solution at one of the node's
descendants (by stealing more items).
• Therefore, for optimization problems we always visit a promising node's
children.
60
 Amity School of Engineering and Technology
0/1 Knapsack Problem

Ex: Knapsack Capacity (M)=8 Kg

i 1 2 3 4
p 3 5 6 10
w 2 3 4 5

For solving any problem using

Backtracking, we always construct a state-space tree. Here every node of a
tree contains w P and left branch of a tree represents “inclusion of
items” right branch of a tree represent “ exclusion of item”

61
 Amity School of Engineering and Technology
0/1 Knapsack Problem: Backtracking

62
Amity School of Engineering and Technology

63
 Amity School of Engineering and Technology

Example2:

i 1 2 3 4
p 4 6 7 11
w 3 4 5 6

Capacity M=9 Kg

64
 Amity School of Engineering and Technology
Branch and Bound
❑ Branch and Bound is similar to backtracking method, which is
also uses state space tree to solve a problem. That is a
solution is represented by a state space tree.
❑ But B&B is used to solve optimization problem and only used
for minimization problem (not for maximization problem).
❑ To solve any maximization problem (like 0/1 Knapsack
problem) , we need to first convert it into minimization
problem, solve it and then at the last get the solution of
maximization problem.
❑ In Backtracking, we use DFS, and in B&B we use BFS approach.
❑ Problem can be solved 2 ways: By using LC B&B and FIFO B&B

65
 Amity School of Engineering and Technology
0/1 Knapsack problem (using B&B)

66
 Amity School of Engineering and Technology
TSP (using B&B)
▪ Suppose we have a complete graph (𝐾4 ). A salesperson start from vertex 1
and passing from rest of the vertices and returning back to vertex ‘1’. We
want a tour whose cost is minimum. There are lots of path or sequences.
Let us make a state space tree starting from vertex ‘1’.
▪ Backtracking approach is just like a DFS or pre-order traversal,
▪ In Backtracking, every end node is a solution(not optimal). By comparing all
permutation solution, we can get optimal solution.
▪ But here we don’t want all possible tour, we want
Only optimal tour (minimum cost tour).
• This is the reason for optimization problem (max. or
Min problem) we avoid backtracking, since this takes
More time to solve a problem.
• Backtracking is good for permutation problem (not for optimization).
• So we use Branch and bound approach.
67
 Amity School of Engineering and Technology
TSP (using B&B)-Method
Given a directed connected weighted graph G(V,E).
Step1: The graph is represented by its cost matrix 𝐶𝑖𝑗 , where
𝐶𝑖𝑗 = cost of edge , if edge exist between i and j 𝐶𝑖𝑗 = ∞ , other wise.
Step2: Convert the cost matrix into Reduced matrix ( i.e. every row and every column
contains at least one 0’s)
Step3: Find the cost (L) of Reduced matrix, R (i.e. sum of elements subtracted from
row’s and column’s of cost matrix to make it reduced.
Step4: Make the state-space search tree for the reduced matrix.
Step5: To find the next E-Node, find the least cost value node by calculating reduced
cost matrix for each node.
Step6: If (𝑖, 𝑗) edge is to be included, then there are 3 coditions:
a) Change all the entries in row 𝑖 and column 𝑗 of R to ∞. b) Set 𝑗, 1 = ∞
c) Reduced all rows and columns in resulting matrix, except for rows and columns
containing ∞.
Step6: Calculate the cost of each node using formula: 𝐶 = 𝐿 + 𝑐𝑜𝑠𝑡(𝑖, 𝑗) + 𝑟
Step7: Repeat the above steps for all the nodes generated and we get a path.
68
Amity School of Engineering and Technology
TSP (using B&B)

69
Amity School of Engineering and Technology
TSP (using B&B)
In LC-B&B node with min
value is explored but in FIFO-
B&B all node explored

70
Amity School of Engineering and Technology
TSP (using B&B)

71
Amity School of Engineering and Technology
TSP (using B&B)

72
 Amity School of Engineering and Technology
TSP (using B&B)

Cost of the tour is 28, at root node ∞ is modified to 28 (since no leaf

node is less than 28 so all rest of the leaf node is killed. And Path of
the tour is 1→4→2→5→3→1
73
Amity School of Engineering and Technology

74
 Amity School of Engineering and Technology
0/1 knapsack (using B&B)

75
 Amity School of Engineering and Technology
0/1 knapsack (using B&B)

76
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

77
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

78
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

79
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

80
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

81
 Amity School of Engineering and Technology
TSP (Using Greedy Method)

82
Amity School of Engineering and Technology
Graph representation

83
Amity School of Engineering and Technology
Graph representation

Pros: Representation is easier to implement and follow. Removing an edge

takes O(1) time. Queries like whether there is an edge from vertex ‘u’ to vertex
‘v’ are efficient and can be done O(1).
Cons: Consumes more space O(V^2). Even if the graph is sparse(contains less
number of edges), it consumes the same space. Adding a vertex is O(V^2)
time.
Please see this for a sample Python implementation of adjacency matrix.

84
 Amity School of Engineering and Technology
Graph representation

Pros: Saves space O(|V|+|E|) . In the worst case, there can be C(V, 2) number of
edges in a graph thus consuming O(V^2) space. Adding a vertex is easier.
Cons: Queries like whether there is an edge from vertex u to vertex v are not
efficient and can be done O(V).
85
Amity School of Engineering and Technology
BFS and DFS

86
Amity School of Engineering and Technology
BFS and DFS

87
Amity School of Engineering and Technology
BFS and DFS

88
 Amity School of Engineering and Technology
BFS and DFS- Time and Space Complexity

89