PERFECT ENGINEERING CLASSES

Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Introduction to
Design and Analysis of Algorithm

Abu Jafar Muhammad ibn Musa Al-Khwarizmi

[Born: about 780 in Baghdad (now in Iraq). Died: about 850]

An algorithm, named after the ninth century scholar Abu Jafar Muhammad Ibn Musa Al-
Khowarizmi, is defined as follows: Roughly speaking:

 An algorithm is a set of rules for carrying out calculation either by hand or on a machine.
 An algorithm is a finite step-by-step procedure to achieve a required result.
 An algorithm is a sequence of computational steps that transform the input into the output.
 An algorithm is a sequence of operations performed on data that have to be organized in data
structures.
 An algorithm is an abstraction of a program to be executed on a physical machine (model of
Computation).

The most famous algorithm in history dates well before the time of the ancient Greek s: this is the
Euclid's algorithm for calculating the greatest common divisor of two integers. This theorem
appeared as the solution to the Proposition II in the Book VII of Euclid's "Elements." Euclid's
"Elements" consists of thirteen books, which contain a total number of 465 propositions.

We start with defining the model of computation(TOC). Once the model of computation has been
defined, an algorithm can be describe using a simple language (or pseudo language) whose syntax is
close to programming language such as C or java.

Algorithmic is a branch of computer science that consists of designing and analyzing computer
algorithms

Page 1
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

In this course students are learn various procedures and strategies to design algorithm and analysis of
design algorithms.

It helps to developed software in well manner. In software companies developers mainly performed
following tasks-

1. Design algorithm
2. Implementation
3. Testing
4. Analysis of algorithm

1. Design Algorithm:

To design the algorithm we have to used various strategies-

a. Divide and conquer methods (DAC)

b. Greedy methods
c. Dynamic programming (DP)

2. Implementation:

Expression of designed algorithm into program by using programming languages is called as

implementation.

3. Testing:
There are two phases-

 Debugging:

The process of executing program on sample data sets

 Profiling:

Executing correct program on data sets and measuring time and space it takes to compute the results.

4. Analysis of algorithm:

Determining the computing time and storage requires.

Page 2
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Course Objectives:
To provide a solid foundation in algorithm design and analysis. Specifically, the student learning
outcomes include, Upon completion of this course, students will be able to do the following:

 Basic knowledge of graph and matching algorithms.

 Ability to understand and design algorithms using greedy strategy, divide and conquer
approach, dynamic programming.
 Ability to analyze asymptotic runtime complexity of algorithms including formulating
recurrence relations.
 Basic knowledge of computational complexity, approximation and randomized algorithms.

 Write rigorous correctness proofs for algorithms.

 Apply important algorithmic design paradigms and methods of analysis.
 Synthesize efficient algorithms in common engineering design situations.

Page 3
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

GATE 2022 SYLLABUS

1. Searching
2. Sorting
3. Hashing
4. Asymptotic Worst Case Time and Space Complexity.
5. Algorithm Design Techniques:
 Divide and Conquer (DAC)
 Greedy Method
 Dynamic Programming
6. Graph Search
7. Minimum Spanning Trees
8. Shortest Paths.

Page 4
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT-I
Recurrence Relation

TOPICS:
1. Mathematical foundation
2. Summation of arithmetic and geometric series, Σn, Σn2
3. Bounding summation using integrations
4. Recurrence relations
5. Solutions of recurrence relations using technique of characteristic equation
6. Recursion tree method and master theorem
7. Complexity calculation of various standard functions
8. Principles of designing algorithms.

Page 5
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Mathematical foundations:

The analysis of algorithms often requires us to draw upon a body of mathematical tools. Some of
these tools are as simple as high-school algebra, but others, such as solving recurrences, may be new
to you referred as mathematical foundation.

2. Summation of arithmetic and geometric series:

Summation:
When an algorithm contains an iterative control construct such as a while or for loop, its running
time can be expressed as the sum of the times spent on each execution of the body of the loop. By
adding up the time spent on each iteration, we obtained the summation or series

Evaluating this summation yielded a bound of (n2) on the worst-case running time of the
algorithm. This example indicates the general importance of understanding how to manipulate and
bound summations.

Summation formulas and properties:

1. Given a sequence a1, a2, . . . Of numbers, the finite sum a1 + a2 + . . . +an can be written

2. Given a sequence a1, a2, . . . Of numbers, the infinite sum a1 + a2 + can be written

Which is interpreted to mean

Arithmetic series:

The summation

Which came up when we analyzed insertion sort, is an arithmetic series and has the value

Page 6
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Geometric series:

For real x 1, the summation

Is a geometric or exponential series and has the value

When the summation is infinite and |x| < 1, we have the infinite decreasing geometric series

3. Bounding summation using integrations:

Approximation by integrals:
When a summation can be expressed as

where â(k) is a monotonically increasing function, we can approximate it by integrals:

The summation is represented as the area of the rectangles in the figure, and the integral is the
shaded region under the curve. When â(k) is a monotonically decreasing function, we can use a
similar method to provide the bounds

The integral approximation gives a tight estimate for the nth harmonic number.
For a lower bound, we obtain

Page 7
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

For the upper bound, we derive the inequality

which yields the bound

Figure shows Approximation of by integrals. The area of each rectangle is shown

within the rectangle, and the total rectangle area represents the value of the summation.

Page 8
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

4. Recursion and Induction:

Recurrence Relations:
A recurrence relation is an equation which is defined in terms of itself is called as recurrence
relation. Recurrence relation can be classified into following types-
 Homogeneous recurrence relation
 Non-homogeneous recurrence relation
 Logarithmic recurrence relation

Solutions of Recurrence Relations:

There are various methods are used to solve the recurrence relation are-
 Characteristic equation method
 Substitution method
 Iteration methods
 Recursion tree method
 Master theorem’s

5. Characteristic equation method:

In this methods to solve the RR first find the roots of characteristic equations, and then generate the
solution according to following cases-

1. Roots are real and distinct:

𝑇 (𝑛 ) = 𝐶1 𝑟1 𝑛 + 𝐶2 𝑟2 𝑛 + 𝐶3 𝑟3 𝑛 ….

2. Roots are Real and same:

𝑇 (𝑛 ) = 𝐶1 𝑟1 𝑛 + 𝐶2 𝑛𝑟2 𝑛 + 𝐶3 𝑛 2 𝑟3 𝑛 ….

3. Roots are Real distinct and same:

𝑇 (𝑛 ) = 𝐶1 𝑟1 𝑛 + 𝐶2 𝑟2 𝑛 + (𝐶3 𝑟3 𝑛 + 𝐶4 𝑛𝑟4 𝑛 + 𝐶5 𝑛 2 𝑟5 𝑛 … . )

Page 9
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Homogeneous Recurrence Relation:

If within the RR the input is constantly operated with count function then such RR is called as
Homogenous RR.

𝑡(𝑛 ) ± 𝑎0 𝑡(𝑛 − 1) ± 𝑎1 𝑡(𝑛 − 2) ± 𝑎2 𝑡(𝑛 − 3) ± ⋯ = 0

Where
𝑡(𝑛 ) − time complexity of n inputs
𝑎𝑛 − 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑒𝑟𝑚

Example : solve the following RR.

𝑛 , 𝑖𝑓 𝑛 = 0 𝑜𝑟 𝑛 = 1
𝑡 (𝑛 ) = {
𝑡(𝑛 − 1) + 𝑡(𝑛 − 2) , 𝑜𝑡𝑒𝑟𝑤𝑖𝑠𝑒

Solution:
𝑡(𝑛 ) = 𝑡(𝑛 − 1) + 𝑡(𝑛 − 2)
𝑡(𝑛 ) − 𝑡(𝑛 − 1) − 𝑡(𝑛 − 2) = 0

1. Generate Characteristic Equitation (CE) and Find Roots:

𝑥2 − 𝑥 − 1 = 0
𝐹𝑖𝑛𝑑 𝑅𝑜𝑜𝑡𝑠
𝑟1 = 1.6180
𝑟2 = −0.6180

2. Roots are real and distinct using Standard solution:

𝑡(𝑛 ) = 𝐶1 𝑟1 𝑛 + 𝐶2 𝑟2 𝑛
∴ 𝑡(𝑛 ) = 𝐶1 (1.6180)𝑛 + 𝐶2 (−0.6180)𝑛 …… (1)

3. Using initial condition:

Put n = 0 in basic equation
𝑡(0) = 0
Put n = 0 in equation (1)
𝑡(0) = 𝐶1 + 𝐶2
∴ 𝐶1 + 𝐶2 = 0 …… (2)
Put n = 1 in basic equation
𝑡(1) = 1
Put n = 1 in equation (1)
𝑡(1) = 1.6180𝐶1 + (−0.6180)𝐶2
∴ 1.6180𝐶1 − 0.6180𝐶2 = …… (3)
Solve eqn (2) and (3) we get
𝐶1 = 0.447

Page 10
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

𝐶1 = −0.447
From eqn (1)

∴ 𝒕(𝒏) = (𝟎. 𝟒𝟒𝟕)(𝟏. 𝟔𝟏𝟖𝟎)𝒏 + (−𝟎. 𝟒𝟒𝟕)(−𝟎. 𝟔𝟏𝟖𝟎)𝒏

2. Non Homogenous Recurrence Relation:

If the recurrence is not constant and if contain different term vary differently according to time is
called as non homogenous recurrence relation.
Standard form
𝑡(𝑛 ) ± 𝑎0 𝑡(𝑛 − 1) ± 𝑎1 𝑡(𝑛 − 2) ± 𝑎2 𝑡(𝑛 − 3) ± ⋯ = 𝑏 𝑛 𝑝(𝑛)

Where
𝑡(𝑛 ) − time complexity of n inputs
𝑎𝑛 − 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑡𝑒𝑟𝑚
𝑏 𝑛 − 𝑒𝑥𝑝𝑜𝑛𝑒𝑛𝑡𝑖𝑎𝑙 𝑡𝑒𝑟𝑚
𝑝(𝑛 ) − 𝑝𝑜𝑙𝑦𝑛𝑜𝑚𝑖𝑎𝑙 𝑡𝑒𝑟𝑚

Example:
𝑡(𝑛 ) − 2𝑡(𝑛 − 1) − 3𝑡(𝑛 − 2) = 3𝑛 𝑛

Logarithmic Recurrence Relation:

In algorithm index variable is operated by either multiplication or division operator then that
recurrence relation is known as logarithmic recurrence relation.
For example:
𝑛
𝑇 (𝑛 ) = 𝑎𝑇 ( ) + 𝑓(𝑛)
𝑏

Substitution method:

The substitution method for solving recurrences involves guessing the form of the solution and then
using mathematical induction to find the constants and show that the solution works. The name
comes from the substitution of the guessed answer for the function when the inductive hypothesis is ;

P0l./’; nbcnnmjhkol,,l2121papplied to smaller values. This method is powerful, but it obviously can
be applied only in cases when it is easy to guess the form of the answer.

The substitution method can be used to establish either upper or lower bounds on a recurrence. As an
example, let us determine an upper bound on the recurrence

T(n) = 2T( n/2 ) + n

Page 11
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Iteration method:

The method of iterating a recurrence doesn't require us to guess the answer, but it may require more
algebra than the substitution method. The idea is to expand (iterate) the recurrence and express it as a
summation of terms dependent only on n and the initial conditions. Techniques for evaluating
summations can then be used to provide bounds on the solution.

6. Recursion tree method and master theorem:

Recursion Trees method:

A recursion tree is useful for visualizing what happens when a recurrence is iterated. It diagrams the
tree of recursive calls and the amount of work done at each call.
For instance, consider the recurrence

T(n) = 2T(n/2) + n2.

The recursion tree for this recurrence has the following form:

In this case, it is straightforward to sum across each row of the tree to obtain the total work done at a
given level:

Page 12
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

This a geometric series, thus in the limit the sum is O(n2). The depth of the tree in this case does not
really matter; the amount of work at each level is decreasing so quickly that the total is only a
constant factor more than the root. Recurrence trees can be a good method of guessing.

Let's consider another example,

T(n) = T(n/3) + T(2n/3) + n.
Expanding out the first few levels, the recurrence tree is:

Note that the tree here is not balanced: the longest path is the rightmost one, and its length is log3/2 n.
Hence our guess for the closed form of this recurrence is O(n log n).

Master theorems:
The master theorem applies to recurrence of the following form
𝑛
𝑇 (𝑛 ) = 𝑎𝑇 ( ) + 𝑓(𝑛)
𝑏
Where 𝑎 ≥ 1 , 𝑏 > 1 𝑎𝑟𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝑎𝑛𝑑 𝑓(𝑛 ) 𝑖𝑠 𝑎𝑛 𝑎𝑠𝑦𝑚𝑝𝑡𝑜𝑡𝑖𝑐𝑎𝑙𝑙𝑦 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛.
There are 3 cases-
𝑎−𝜖 𝑎
1. 𝑖𝑓 𝑓 (𝑛 ) = Ο( 𝑛 𝑙𝑜𝑔𝑏 ), 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑐𝑜𝑛𝑡𝑎𝑛𝑡 𝜖 > 0 𝑡ℎ𝑒𝑛 𝑇 (𝑛 ) = θ( 𝑛 𝑙𝑜𝑔𝑏 )
𝑎 𝑎
2. 𝑖𝑓 𝑓 (𝑛 ) = θ( 𝑛 𝑙𝑜𝑔𝑏 ), 𝑡ℎ𝑒𝑛 𝑇(𝑛 ) = θ( 𝑛 𝑙𝑜𝑔𝑏 𝑙𝑜𝑔𝑛)
𝑎+𝜖
3. 𝑖𝑓 𝑓 (𝑛 ) = Ω( 𝑛 𝑙𝑜𝑔𝑏 ), 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑐𝑜𝑛𝑡𝑎𝑛𝑡 𝜖 > 0 𝑡ℎ𝑒𝑛 𝑇 (𝑛 ) = θ(𝑓(𝑛) )
Example: find the complexity by using master theorems
𝑛
1. 𝑇 (𝑛 ) = 4𝑇 ( ) + 𝑛
2
𝑛
2. 𝑇 (𝑛 ) = 4𝑇 ( ) + 𝑛 2
2
𝑛
3. 𝑇 (𝑛 ) = 4𝑇 ( ) + 𝑛 3
2
𝒏
𝟏. (𝒏) = 𝟒𝑻 ( ) + 𝒏
𝟐
Solution:
𝑎=4
𝑏=2
𝑓(𝑛) = 𝑛
𝑎 4
𝑛 𝑙𝑜𝑔𝑏 = 𝑛 𝑙𝑜𝑔2 = 𝑛 2
𝑎
𝑛 𝑙𝑜𝑔𝑏 > 𝑓(𝑛 ) 𝑐𝑎𝑠𝑒1
𝑎
𝑇 (𝑛 ) = θ( 𝑛 𝑙𝑜𝑔𝑏 )
∴ 𝑻(𝒏) = 𝛉( 𝒏𝟐 )
Page 13
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

𝒏
𝟐. 𝑻(𝒏) = 𝟒𝑻 ( ) + 𝒏𝟐
𝟐
Solution:
𝑎=4
𝑏=2
𝑓(𝑛) = 𝑛 2
𝑎 4
𝑛 𝑙𝑜𝑔𝑏 = 𝑛 𝑙𝑜𝑔2 = 𝑛 2
𝑎
𝑛 𝑙𝑜𝑔𝑏 = 𝑓(𝑛 ) 𝑐𝑎𝑠𝑒2
𝑎
𝑇 (𝑛 ) = θ( 𝑛 𝑙𝑜𝑔𝑏 log 𝑛)
∴ 𝑻(𝒏) = 𝛉( 𝒏𝟐 𝒍𝒐𝒈𝒏 )

𝒏
𝟑. 𝑻(𝒏) = 𝟒𝑻 ( ) + 𝒏𝟑
𝟐
Solution:
𝑎=4
𝑏=2
𝑓(𝑛) = 𝑛 3
𝑎 4
𝑛 𝑙𝑜𝑔𝑏 = 𝑛 𝑙𝑜𝑔2 = 𝑛 2
𝑎
𝑛 𝑙𝑜𝑔𝑏 < 𝑓(𝑛 ) 𝑐𝑎𝑠𝑒3
𝑇 (𝑛 ) = θ( 𝑓(𝑛))
∴ 𝑻(𝒏) = 𝛉( 𝒏𝟑 )

7. Complexity calculation of various standards functions:

Complexity is nothing but finding loop in the program. There are following loop are used as follow-
for loop:
Consider the following for loop
1. for( i=0 ; i<n ; i++)
{
a = b+c;
}
Complexity: O(n)
2. for( i=0 ; i<n ; i++)
{
for( j=0 ; i<m ; j++)
{
a = b+c;
}
}

Complexity: O(nm)

Page 14
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

3. 𝑙 = 0

for j=1 to n do
{
for i=1 to n2 do
{
for k=1 to n3 do
{
l =l+1;
}
}
}
Complexity : O( n *n2 *n3)

8. Principal of Designing Algorithm:

Algorithm:
It is the well defined computational procedure that takes some values as input and produce
some values as output.

Algorithm is a finite step of procedure that accomplished a particular task

Sequence of
Input Output
instructions

Characteristics of an Algorithm:
There are following characteristics are
Input:
Zero or more quantities are externally supplied into algorithm are called input.

Outputs:
At least one quantity must be produced are called as outputs.

Definiteness:
Each instruction must be clear and unambiguous.

Finiteness:
It can be defined as the algorithms must be terminated after finite number of steps. Time
Of termination should be short

Effectiveness:

Page 15
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Every instruction must be very basic.

Program:

A program is the expression of an algorithm in programming languages.

Area of Algorithm:

The main active areas of research on algorithms are

1. Devise The Algorithm:

The creation of an algorithm is called as devising the algorithm. There are various design
strategies are used

o Divide and conquer (DAC)

o Greedy methods
o Dynamic Programming

2. Validate the Algorithm:

Check if the algorithm working correctly is called the algorithm validation.

3. Analysis the Algorithm:

The process of determining its complexity in term of time or space is known as analysis of
algorithm.

4. Testing:

Testing is the process of determining the performance of the program

Testing a program consist of 2 parts
− Debugging
− profiling

Expressing Algorithm:

There are following methods are used to express an algorithm are

o English
o Flow chart
o Pseudo code

Page 16
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Writing Algorithm:

An algorithm consist of heading and body

The heading takes the form

Algorithm Name of Algorithm( list of parameter)

Body has one or more statements written within braces {.....}

An algorithm may or may not return any value

Example: algorithm to find sum of array value A' and 'n' is the size of array

Algorithm Sum(A, n)

{
S=0

For i=1 to n do

S=S+A[i];

Return S;
}

Page 17
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT-I
Asymptotic Notation

TOPICS
1. Asymptotic notations of Analysis of Algorithms
2. Analyzing Control Structures
3. Worst case and Average case Analysis
4. Amortized analysis
5. Sorting networks
6. Comparison networks
7. Bio-tonic sorting network
8. Advanced data structures like
– Fibonacci heap
– Disjoint set representation

Page 18
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Asymptotic Notations of Analysis of Algorithms:

Asymptotic Notations:
Asymptotic Notation are mathematical tools to represent the time complexity of algorithm for
asymptotic analysis. It is also known as rate of growth.
There are following three asymptotic notation are mostly used to represent time complexity of
algorithm are-
Big-oh Notation(𝑂)
Omega Notation(Ω)
Theta Notation (𝜃)

Big-oh Notation(𝑶):
The 𝑂(𝑛) is the formal way to express the upper bound of an algorithm running time, it measures
the worst case time complexity or longest amount of time an algorithm can possibly take to
complete shown in fig.

𝑓(𝑛 ) = 𝑂(𝑔(𝑛 )), 𝑖𝑓 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑡 𝑐 > 0 𝑎𝑛𝑑 𝑛0

𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑓(𝑛 ) ≤ 𝑐𝑔(𝑛 ), 𝑓𝑜𝑟 ∀ 𝑛 > 𝑛0

Omega Notation(𝛀):
The Ω(𝑛) is the formal way to express the lower bound of an algorithm running time, it measures
the best case time complexity or best amount of time an algorithm can possibly take to complete
shown in fig.

𝑓(𝑛 ) = Ω(𝑔(𝑛 )), 𝑖𝑓 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑡 𝑐 > 0 𝑎𝑛𝑑 𝑛0

𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑓(𝑛 ) ≥ 𝑐𝑔(𝑛 ), 𝑓𝑜𝑟 ∀ 𝑛 > 𝑛0
Page 19
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Theta Notation(𝜽):
The 𝜃(𝑛) is the formal way to express the both lower bound and upper bound of an algorithm
running time, it measures the average case time complexity an algorithm can possibly take to
complete shown in fig.

𝑓(𝑛 ) = 𝜃(𝑔(𝑛 )), 𝑖𝑓 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑡 𝑐1 & 𝑐2 > 0 𝑎𝑛𝑑 𝑛0

𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑐1 𝑔(𝑛) ≤ 𝑓 (𝑛 ) ≥ 𝑐2 𝑔(𝑛 ), 𝑓𝑜𝑟 ∀ 𝑛 > 𝑛0

2. Analyzing Control Structures:

The analysis of an algorithm refers as calculating individual instruction running time, instruction
running time can be calculated by analyzing control structures, there are following control structures
are-
 Sequencing
 If then else
 For loop
 While loop
 Recursion
Sequencing:
Suppose an algorithm consist of two parts A and B.
A takes time 𝑡𝐴 and B takes time 𝑡𝐵 for computation. The total computation (𝑡𝐴 +𝑡𝐵 ) time is
calculated according to the sequencing rules is (max (𝑡𝐴 , 𝑡𝐵 ) )
Example:
Suppose 𝑡𝐴 = 𝑂(𝑛 ) 𝑎𝑛𝑑 𝑡𝐵 = 𝑂(𝑛 2 ) , calculate the total time computation as,
Computation time = 𝑡𝐴 +𝑡𝐵
= max (𝑡𝐴 , 𝑡𝐵 )
= max (𝑂(𝑛 ), 𝑂 (𝑛 2 ))
= 𝑂 (𝑛 2 )
If then else:
The total computation time is calculated according to the conditional rule “if-then else” is
max (𝑡𝐴 , 𝑡𝐵 )., shown in fig.

Page 20
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Condition

𝑡𝐴 = 𝑂(𝑛 2 ) A A 𝑡𝐵 = 𝜃(𝑛 3 )

Total computation time = max (𝑡𝐴 , 𝑡𝐵 )

= max (𝑂(𝑛 2 ), 𝜃(𝑛 3 ) )
= 𝜃(𝑛 3 )
For loop:
Consider the following for loop
𝐹𝑜𝑟 𝑖 = 1 𝑡𝑜 𝑚
{
P(i)
}
If the computation time 𝑡𝑖 for P(i) varies as a function of ‘i’ then total computation time for the “for
loop” is given by
𝑚 𝑚 𝑚 𝑚

𝑇𝑜𝑡𝑎𝑙 𝐶𝑜𝑚𝑝𝑢𝑡𝑎𝑡𝑖𝑜𝑛 𝑡𝑖𝑚𝑒 = ∑ P(i) = ∑ ti = ∑ θ(1) = 𝜃 ∑ 1 = θ(𝑚)

𝑛=1 𝑛=1 𝑛=1 𝑛=1

If Algorithm consist of nested “for” loops then computational time is:

𝐹𝑜𝑟 𝑖 = 1 𝑡𝑜 𝑚
{
𝐹𝑜𝑟 𝑗 = 1 𝑡𝑜 𝑛

{
P(ij)
}
}
𝑚 𝑛 𝑚 𝑛

𝑇𝑜𝑡𝑎𝑙 𝐶𝑜𝑚𝑝𝑢𝑡𝑎𝑡𝑖𝑜𝑛 𝑡𝑖𝑚𝑒 = ∑ ∑ 𝑝(𝑖𝑗 ) = 𝜃 ∑ ∑(1) = 𝜃(𝑚𝑛)

𝑖=1 𝑗=1 𝑖=1 𝑗=1

Page 21
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

While loop:
Consider the example:
While ( n > 1)
{
P(i)
𝑛
𝑛=
2
}

𝑇𝑜𝑡𝑎𝑙 𝐶𝑜𝑚𝑝𝑢𝑡𝑎𝑡𝑖𝑜𝑛 𝑡𝑖𝑚𝑒 = 𝑂 log2 𝑛

Recursion:
A procedure that calls itself is called as recursive.
Consider the example
𝐹𝑎𝑐𝑡 (n)
{
If( n = 0 or n = 1 )
Return 1;
else
Return (n× 𝐹𝑎𝑐𝑡(𝑛 − 1));
}

3. Worst case and Average case Analysis:

Average Case vs. Worst Case Running Time of an Algorithm:

• An algorithm may run faster on certain data sets than on others,
• Finding the average case can be very difficult, so typically algorithms are measured by the worst-
case time complexity.
• Also, in certain application domains (e.g., air traffic control, surgery) knowing the worst-case time
complexity is of crucial importance.

Page 22
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

4. Amortized analysis:
:
Amortized analysis means finding the average running time per operation over a worst case sequence
of operations.
Amortized analysis assumes worst case input and typically does not allow random choice.
There are several techniques used in amortized analysis-
Aggregate methods:
In this method, we compute an upper bound T(n) on the total cost of a sequence of n operations.
Accounting method:
In this ,overcharge some operations early in the sequence this overcharge is used later in the
sequence for pay operation that charge less than they actually cost.
Potential method:
In this, maintain the credit as the potential energy to pay for future operation.
Example:
Define amortized analysis. Explain the amortized complexity for 4 bit binary incrementor from 0to 8
,and also write algorithm:
Solution:
Consider binary no. between 0 to 8

0 0000 0 cycle
1 0001 1
2 0010 2
3 0011 1
4 0100 3
Are consuming more cycle i.e.
5 0101 1
potential input
6 0110 2
7 0111 1
8 1000 4

Total time = 4×8 = 32 cycle

Algorithm Binary Incrementor
{
i=n
while ( i > = 1 and a[ i] = 1 )
{
a[ i ] = 0
i = i -1
}

a[ i ] = 1

Page 23
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

5. Sorting networks:

A :sorting networks is a comparison network for which output sequence is monotonically increasing
for every input sequence.
Sorting network is capable of sorting thousands of items in the order of microseconds and can be
constructed with present day hardware.
Advantage:
o It is a fast sorting capability can be used to manipulate large set of data.
o Capable of sorting thousands of item in order of microseconds.

6. Comparison networks:

Comparison network is made of wires and comparators.

A comparator is a device with two inputs x and y and two outputs x’ and y’ where
x’:= min ( x, y )
y’ = max (x, y)
x x’ = min (x, y)
Comparator
y
y’ = max (x, y)

7. Bio-tonic sorting network:

:
A sequence that either monotonically increases and then monotonically decreases or monotonically
decreases then monotonically increases is called as bio-tonic sequence
Example:
: sequence < 3, 4, 6, 9, 7, 5, 2, 1 >
The
There are two components are-
The half cleaner:
A bio-tonic sorter is comprises of several stages, each of which is called a half cleaner.
Bio-tonic sorter:
By recursively combining half cleaner we can build a bio-tonic sorter which is a network that sorts
bio-tonic sequence.
Example:
Sort the following data using bio-tonic sorting technique.
1, 3, 5, 9, 8, 7, 4, 2
Solution:
The bio-tonic sequence

Page 24
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1, 3, 5, 9,| 8, 7, 4, 2

1 1
3 2
5 3
9 4

8 5

7 7
4 8
2 9

The sorted sequence is: 1, 2, 3, 4, 5, 7, 8, 9.

Page 25
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

8. Advanced data structures like:

:
Fibonacci heap:
Fibonacci heap history.
Fredman and Tarjan (1986)
:
 In genious data structure and analysis.
 Original motivation: O(m + n log n) shortest path algorithm. also led to faster algorithms for
MST, weighted bipartite matching
 Still ahead of its time.
Fibonacci heap intuition:
 Similar to binomial heaps, but less structured.
 Decrease-key and union run in O(1) time.
 "Lazy" unions.

Delete Min:
In this , we show how to decrease the key of a node in a Fibonacci heap in O(1) amortized time and
how to delete any node from an n-node Fibonacci heap in O(D(n)) amortized time. These operations
do not preserve the property that all trees in the Fibonacci heap are unordered binomial trees. They
are close enough, however, that we can bound the maximum degree D(n) by O(1g n). Proving this
bound will imply that FIB-HEAP-EXTRACT-MIN and FIB-HEAP-DELETE run in O(1g n)
amortized time.

Page 26
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Disjoint set representation:

A disjoint-set data structure maintains a collection S = {S1, S2, . . . ,Sk} of disjoint dynamic sets.
Each set is identified by a representative, which is some member of the set. In some applications, it
doesn't matter which member is used as the representative; we only care that if we ask for the
representative of a dynamic set twice without modifying the set between the requests, we get the
same answer both times. In other applications, there may be a prespecified rule for choosing the
representative, such as choosing the smallest member in the set (assuming, of course, that the
elements can be ordered).
As in the other dynamic-set implementations we have studied, each element of a set is represented by
an object. Letting x denote an object, we wish to support the following operations.
MAKE-SET(x) creates a new set whose only member (and thus representative) is pointed to by x.
Since the sets are disjoint, we require that x not already be in a set.
UNION(x, y) unites the dynamic sets that contain x and y, say Sx and Sy, into a new set that is the
union of these two sets. The two sets are assumed to be disjoint prior to the operation. The
representative of the resulting set is some member of Sx Sy , although many implementations
of UNION choose the representative of either Sx or Sy, as the new representative. Since we require

Page 27
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

the sets in the collection to be disjoint, we "destroy" sets Sx and Sy , removing them from the
collection S.
FIND-SET(x) returns a pointer to the representative of the (unique) set containing x.
Throughout this chapter, we shall analyze the running times of disjoint-set data structures in terms of
two parameters: n, the number of MAKE-SET operations, and m, the total number of MAKE-
SET, UNION, and FIND-SET operations. Since the sets are disjoint, each UNION operation reduces
the number of sets by one. After n - 1 UNION operations, therefore, only one set remains. The
number of UNION operations is thus at most n - 1. Note also that since the MAKE-SET operations
are included in the total number of operations m, we have m n.

Page 28
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT -III
Divide and Conquer Strategies & Greedy
Approach

TOPICS:

 Divide and conquer basic strategy

1. Binary search
2. Min max
3. Merge sort
4. Quick sort
5. Multiplication algorithm

 Greedy method basic strategy

1. Knapsack Problem
2. Application to Job Sequencing with Deadlines Problem
3. Minimum Cost Spanning Trees
4. Single Source Shortest Path
5. Optimal Merge Patterns.

Page 29
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Divide and Conquer (DAC) Strategies:

In divide and conquer approach, the problem is divided into smaller sub-problems and then each
problem is solved independently. The solution of all sub-problems is finally merged in order to
obtain the solution of an original problem.
We can understand divide-and-conquer approach in a three-step process.

 Divide/Break:

This step involves breaking the problem into smaller sub-problems. This step generally takes a
recursive approach to divide the problem until no sub-problem is further divisible.

 Conquer/Solve:

This step receives a lot of smaller sub-problems to be solved. Generally, at this level, the problems
are considered 'solved' on their own.

 Merge/Combine:

When the smaller sub-problems are solved, this stage recursively combines them until they formulate
a solution of the original problem.
There are following Algorithms are-
1. Binary search

2. Min max Algorithm

3. Merge Sort

4. Quick Sort

5. Matrix Multiplication

Page 30
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Binary search:
These Analysis is used to find average no. of comparison required for successful or unsuccessful
search.
To find average no of comparison by using
o Average no. of comparison for successful search

𝐼𝑃𝐿
𝐴𝑆 (𝑛 ) = +1
𝑛
o Average no. of comparison for unsuccessful search

𝐸𝑃𝐿
𝐴𝑢 ( 𝑛 ) =
𝑛+1
Where,
IPL (Internal Path Length):
It is the length of path for all node in a binary search node and search will be successful.
EPL (External Path Length):
It is the length of path for all node in a binary search node and search will be unsuccessful.
𝐄𝐏𝐋 = 𝐈𝐏𝐋 + 𝟐𝐧
𝐧 − no. of elements

Assumption:
A - Array of ‘n’ elements and sorted in ascending order
key – element to be search
low – lower index =1
high – upper index = n

Page 31
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm:
Algorithm Binsearch(A, key, low, high)
{
if ( low = = high )
{
if ( key = = A[low] )
return low;
else
return 0;
}
else
{
mid = ( low + high) /2
if ( key = = A[mid] )
return mid;
else
{
if ( key < A[mid] )
Binsearch(A, key, low, mid-1)
else
Binsearch(A, key, mid+1,high)
}
}
}
Complexity:
Best case: 𝑇 (𝑛 ) = 1, 𝑓𝑜𝑟 𝑛 = 1
Average and worst case:
𝑛
𝑇 (𝑛 ) = 𝑇 ( ) + 1, 𝑓𝑜𝑟 𝑛 ≥ 1
2
Recurrence Relation:
𝟏 , 𝒊𝒇 𝒏 = 𝟏
𝑻(𝒏) = { 𝑛
𝑇( )+ 1 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
2

𝑻(𝒏) = 𝑶(𝐥𝐨𝐠𝟐 𝒏)

Page 32
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Min max Algorithm:

This method is used to find the minimum and maximum element from a given array with divide and
conquer approach. Array contains with following information shown in fig.

Where, Low, high, min, max

low - lower index
high - upper index
min - minimum element
max - maximum element
Assumption:
There are three cases:
1. Array contain only one element then (low = high)
2. Array contain two elements then check min and max
3. Array contains more than two elements then divide array using DAC

Algorithm:
Algorithm Min_max( A, min, max, low, high)
{
if ( low = = high) then
{
min = max = A[low]
}
else
{
if ( low = high -1) then
{
if ( A[low] < A[high]) then
{
min = A[low]
max = A[high]
}
else
{
min = A[high]
max = A[low]
}
}
else
{
mid =( low + high) /2
Min_max( A, low, mid, min, max)
Min_max(A, mid+1, high, min, max)
}
}
Complexity: log2n

Page 33
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

3. Merge Sort:
In this algorithm, array divided into small size with single element
The divided array will be merge into single array by performing comparison between two elements.
Assumption:
Input: unsorted array A
Output: sorted array A
Algorithm:
Algorithm Mergesort( A, low, high)
{
if ( low < high)
{
mid =( low + high) /2
Mergesort( A, low, mid)
Mergesort( A, mid+1, high)
Merge ( A, low, mid, high)
}
}
Complexity: O(n log2n)
Where ‘n’ is the no of comparisons and log2 n is the height of tree.

Page 34
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

4. Quick Sort:
This sorting method is based on dividing the array on a key value and generating the position of key
value such that array will be divided into two parts and after divide the array position of key is fixed.
There are two conditions:
1. If (i < j) then interchange A[i] and A[j]
2. If (i ≥ j ) then interchange A[key] and A[j].

Algorithm Quick Sort(A, low, high)

{
if (low < high )
mid = Partition ( A, low, high)
Quick Sort ( A, low, mid-1)
Quick Sort ( A, mid+1, high)
}
Algorithm Partition ( A, low, high)
{
i ,k = low, j = high
while ( A[i] < = k && i < high )
i++
while ( A[j] > key && j > low )
j- -
if( i< j) then
interchange A[i] & A[j]
else
interchange A[j] & A[k]
}

Page 35
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Complexity:
1. Best case & Average case - O (n log2n):
In quick sort when the elements is placed at proper position automatically. It most of the cases few
more elements is replace proper position. This will reduce the complexity of algorithm. Hence best
and average case complexity of quick sort is O (n log2n) where ‘n’ is the maximum no. of
comparison and log2n is no. of passes.
2. Worst case - O (n2 ):
in Quick sort ,if array is already sorted in descending order but require to arrange in ascending order
then after each passes the array generated will be (n-1) value is one part and only one value is
another part
with this approach the divide array size n,(n-1) operations will be required. Hence the complexity of
Quick sort in worst case is O (n2).

5. Matrix Multiplication Algorithm:

Stassen’s Matrix Multiplication:
A A 𝟏𝟐 B𝟏𝟏 B𝟏𝟐
A = ( 𝟏𝟏 ) B= ( )
A 𝟐𝟏 A 𝟐𝟐 B𝟐𝟏 B𝟐𝟐

P = (A11 + A22) ( B11 + B22 )

Q = (A21 + A22) B11
R = A11 (B12 – B22 )
S = A22 ( B21 – B11 )
T = (A11 + A22 ) B22
U = (A21 – A11 ) ( B11 + B 12 )
V = (A12 – A12 ) ( B21 + B 22 )
C11 = P + S – T + V
C12 = R + T
C21 = Q + S
C11 = P + R – Q + U
A A 𝟏𝟐 B B𝟏𝟐 C C𝟏𝟐
( 𝟏𝟏 ) . ( 𝟏𝟏 ) = ( 𝟏𝟏 )
A 𝟐𝟏 A 𝟐𝟐 B𝟐𝟏 B𝟐𝟐 C𝟐𝟏 C𝟐𝟐

Page 36
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Greedy Methods:
Among all the algorithmic approaches, the simplest and straightforward approach is the Greedy
method. In this approach, the decision is taken on the basis of current available information without
worrying about the effect of the current decision in future.

Greedy algorithms build a solution part by part, choosing the next part in such a way, that it gives an
immediate benefit. This approach never reconsiders the choices taken previously. This approach is
mainly used to solve optimization problems. Greedy method is easy to implement and quite efficient
in most of the cases. Hence, we can say that Greedy algorithm is an algorithmic paradigm based on
heuristic that follows local optimal choice at each step with the hope of finding global optimal
solution.

In many problems, it does not produce an optimal solution though it gives an approximate (near
optimal) solution in a reasonable time.

Components of Greedy Algorithm:

Greedy algorithms have the following five components −

 A candidate set − A solution is created from this set.

 A selection function − Used to choose the best candidate to be added to the solution.
 A feasibility function − Used to determine whether a candidate can be used to contribute to
the solution.
 An objective function − Used to assign a value to a solution or a partial solution.
 A solution function − Used to indicate whether a complete solution has been reached.

Areas of Application:

Greedy approach is used to solve many problems, such as

 Finding the shortest path between two vertices using Dijkstra’s algorithm.
 Finding the minimal spanning tree in a graph using Prim’s /Kruskal’s algorithm, etc.

Where Greedy Approach Fails

In many problems, Greedy algorithm fails to find an optimal solution, moreover it may produce a
worst solution. Problems like Travelling Salesman and Knapsack cannot be solved using this
approach.

Page 37
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Knapsack Problem:
 Knapsack is a bag of fixed capacity capable of storing certain objects.
 The objects can be stored either completely or partially.
 There are three different approaches for knapsack problem.
1. Select the objects with maximum Profit (P i).
2. Select the objects with minimum Weigh t(Wi).
3. Select the objects with maximum (Pi /Wi).

Assumption:
1. P[1….n] profit array
2. W[1…n] weight array
Capacity = total capacity
Weight = 0
x[1…n] is the array that contains three types of values
1 = complete
0 = reject
Fraction = partially
Algorithm Knapsack(P, W, capacity, weight)
Step1: For i = 1 to n do
x[i] = 0
weight = 0
Step2: while ( weight < capacity) do
{
Select the best object with max (Pi/ Wi) ratio, let its index = i
if (weight + W[i] <= capacity ) then
{
x[i] = 1
weight = weight +W[i]
}
else
{
𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦−𝑤𝑒𝑖𝑔ℎ𝑡
x[i] =
𝑊[𝑖]
weight = capacity
}
Step3: Profit = 0
For i = 1 to n do
Profit = profit + [x[i]×P[i]]
}
Complexity: O( nlog n)

Page 38
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Application to job sequencing with deadlines problem:

Job scheduling:
In jobs scheduling there are various jobs to be processed using single processor.
There are two methods of jobs scheduling-
1. Job scheduling without deadline
2. Job scheduling with deadline

1. Job scheduling without deadline:

There is no deadline associated with jobs but the jobs should be scheduled that their processing time
should be minimum. For exp. FCFS, SJF, SRTF, RR etc.

2. Job scheduling with deadline:

– In this , with each jobs a deadline and profit value is associated.
– The profit value is valid if the jobs is executed before or at the deadline.
– Each jobs will require one cycle for execution.
– The objective of required to achieve the maximum profit.

Algorithm Job_scheduling(d[0….n] : j[0…n])

{
d[0] = 0
j[0] = 0
k =1 , j[1] = 1
for i = 2 to n do
{
r=k
while( (d[ j[r]) > max (d[i],r))
r = r-1
if( d[i] > r ) then
for m = k to r + 1 do
j[r +1] = i
k ++
}
}

Page 39
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

3. Minimum Cost Spanning Trees (MCST):

It is a methods of generating tree from graph with following properties-
o Method to generate Spanning Tree from Graph.
o All the vertices present in the Graph will be present in the Tree.
o There should be no cycle ie. Called Acyclic.
o Edge selected should be minimum cost.
o The number of edges in tree should be equal to n -1 vertices

There are two methods to generate minimum cost spanning tree-

1. Kruskal's Algorithm
2. Prim’s Algorithm
 Kruskal’s Algorithm:
Procedure:
o Select the edges with minimum cost and arrange the edges in ascending order of their
weight/cost.
o Continue the process and should be remember that there is no cycle, till the all the vertices
present in the graph.
o If n is the number of vertices then the number of edges in a spanning tree and n -1.

Assumption:
1. G is a graph which represented in the form of cost matrix of size n×n
2. Minimum cost = 0
3. There are two functions used in algorithm
1. Find:
These function will find the component in which the vertex are present.
2. Merge:
These function will merge the two component if they are different.

Page 40
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm Kruskal(cost, G, n, mincost: t)

{
Step1. Create component as per no of vertices in the graph
// n: component
Step2. i=1
mincost = 0
while( i < = n-1) do
select the best edge and it be (u,v)
x = Find(u)
y = Find(v)
if (x ≠ y)
t[i, 1] = u
t[i, 2] = v
t[i, 3] = cost[u,v]
mincost = cost[u,v] + mincost
Merge[x,y]
i = i+1
}
Complexity: Elog V

 Prim’s algorithm:

o In this algorithm the Spanning Tree will be generated by selecting the edges with minimum
cost.
o The process is continue from one of vertices of the selected edge which leads to minimum
cost.

Page 41
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm prims(G, n, cost, near, mincost:t)

{
Step1. Consider t
Let cost[k, l] be the min value in cost matrix
t[1,1] = k
t[1, 2] = l
t[1,3] = cost[k, l]
// consider near array
For i = 1 to n do
{
if( cost [k, i] < cost[l, i] )
near[i] = k
else
near[i] = l
}
near[k] = 0
near[l] = 0
step2. // refer t
for i = 2 to n-1
{
Let j be index in near array such that near[j] ≠0 & cost[ near[j], j] is min
t[i, 1] = near[j]
t[i, 2] = j
t[i, 3] = cost[near[j], j]
mincost = mincost + cost[near[j], j]
near[j] = 0
for k = 1 to n do
{
if(near[k] ≠ 0) and ( cost[k, j] < cost[k.near[k])
near[k] = j
}
}
Complexity: E + log V

Page 42
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Difference between Kruskal’s and Prim’s algorithm:

S.N. Kruskal’s Prim’s

1. In this, edge is selected with minimum In this, edge is selected with minimum value
value. and connectivity.

2. Graph generated with isolated and will Graph generated will be connected but no
required connecting edge inter path require.

3. Complexity is E log V Complexity is E+ log V

4. Single Source Shortest path (SSSP):

o In Single Source Shortest Path algorithm will find the shortest path from one given vertex to all
possible vertices in a graph.
o Path may be directed or undirected.
o Default vertex is '1' otherwise given vertex is a source vertex

Algorithm SSSP ( cost, n, dist)

Step1. For i = 1 to n do
{
S[i] = 0
dist[i] = cost[v,i]
}
Step2. S[v] = 0
dist[v] = 0
Step3. for i =2 to n do
{
Determine n-1 path from vertex ‘v’ and let the vertices ‘u’ select a vertex such that
S[i] ≠ 1 and dist[u] = min
for ( each ‘w’ adjacent to vertex ‘u’)
{
if( dist[w] > dist[u] + cost[u, w]) then
dist[w] = dist[u] + cost[u,w]
}
}
Complexity: O(n 2)

Page 43
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

5. Optimal Merge Patterns (OMP):

In this, Merge a set of sorted files of different length into a single sorted file. We need to find an
optimal solution, where the resultant file will be generated in minimum time.

If the number of sorted files are given, there are many ways to merge them into a single sorted file.
This merge can be performed pair wise. Hence, this type of merging is called as 2-way merge
patterns.

Page 44
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT -IV

Dynamic Programming

TOPICS:

Dynamic Programming basic strategy:

1. Multistage graphs
2. All pairs shortest path
3. Single source shortest paths
4. Optimal binary search trees
5. Traveling salesman problem
6. Longest Common Subsequence problem
7. 0/1 Knapsack Problem
8. Chained Matrix Multiplication

Dynamic Programming basic strategy:

 Dynamic Programming applied to Optimization problem.

 This technique is invented by Richard Bellman in 1950.
 Dynamic programming is a technique for solving problem with overlapping sub problem.
 In this method each sub problem is solved only once and the result of each sub problem is stored
in a table, from which we can obtain solution to the original problem.

Page 45
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Multistage Graphs:
Principal:
In this method, Graph is divided into different stages that is stage-1 to stage-n and will have only one
vertex for each.
The main objective is to find shortest path between source and destination.
There are two methods are used-
 Backward algorithm
 Forward algorithm

 Backward Algorithm:
In this method calculation for shortest path from source to destination the information about previous
stage is considered.
To find backward cost by using-
𝑏𝑐𝑜𝑠𝑡 (𝑖, 𝑗 ) = min { 𝑏𝑐𝑜𝑠𝑡 (𝑖 − 1, 𝐿 ) + 𝑐𝑜𝑠𝑡(𝐿, 𝑗 )}
Where, 𝑖 − 𝑠𝑡𝑎𝑔𝑒 𝑛𝑜.
𝑗 − 𝑁𝑜𝑑𝑒 /𝑉𝑒𝑟𝑡𝑒𝑥 𝑛𝑜.
𝑏𝑐𝑜𝑠𝑡 ( ) − 𝑛𝑜𝑑𝑒 𝑐𝑜𝑠𝑡
𝑐𝑜𝑠𝑡( ) − 𝑝𝑎𝑡ℎ 𝑐𝑜𝑠𝑡
Assumption:
1. Graph is represented in form of cost matrix size n×n
2. bcost[1…n] is used to store cost value
3. d[1…n] is used to store intermediate vertices
4. P[1…k] is path array and k is no of edges

Algorithm Backward(G, k, n, d:P)

{
Step1: bcost[1] = 0
Step2 : For j = 2 to n do
Let ‘r’ be the vertex such that <r, j> is on edge and bcost[r] + cost[r, j] is min
bcost[j] = bcost[r] + cost[r, j]
d[j] = r
Step3:
P[1] = 1
P[k] = n
For i = k-1 to 2 do
{
P[i] = d[P[i+1]]
}
}

Page 46
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

 Forward Algorithm:
In this method calculation of shortest path from destination to source the information about next
stage is consider
To find shortest path by using-
𝐹𝑐𝑜𝑠𝑡 (𝑖, 𝑗 ) = min{ 𝐹𝑐𝑜𝑠𝑡 (𝑖 + 1, 𝐿 ) + 𝑐𝑜𝑠𝑡(𝐽, 𝐿 )}
Where, 𝑖 − 𝑠𝑡𝑎𝑔𝑒 𝑛𝑜.
𝑗 − 𝑁𝑜𝑑𝑒 /𝑉𝑒𝑟𝑡𝑒𝑥 𝑛𝑜.
𝐹𝑐𝑜𝑠𝑡 ( ) − 𝑛𝑜𝑑𝑒 𝑐𝑜𝑠𝑡
𝑐𝑜𝑠𝑡( ) − 𝑝𝑎𝑡ℎ 𝑐𝑜𝑠𝑡

Algorithm Forward(G, k, n, d:P)

{
Step1: Fcost[n] = 0
Step2:
For j = n-1 to 1 do
Let r be the vertex such that <r, j> is on edge Fcost[r] + cost[j, r] is min
Fcost[j] = Fcost[r] + cost[ j, r]
d[j] = r
Step3:
P[1] = 1
P[k] = n
For i = 2 to k-1 do
{
P[i] = d[P[i-1]]
}
}

Page 47
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. All pair shortest path:

This algorithm is used to find shortest path from one source to all destination present in a graph.
The distance are calculated by considering vertex as intermediate and increase the number of
intermediate vertex in each step by one. It is also known as Floyd Warshall algorithm
Assumption:
1. Graph is represented in the form of matrix of size n×n
2. Algorithm will generate distance matrix n×n
Algorithm APSP ( cost, n: dist)
{
Step:1 For i = 1 to n do
For j = 1 to n do
dist[i, j] = cost[i, j]
Step:2 For k =1 to n do
For i =1 to n do
For j=1 to n do
dist[i, j] = min[dist[i,j] , cost[i, k] + cost[k,j]]
}

Page 48
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

3. Single Source Shortest path algorithm:

The Single Source Shortest Path Algorithm is to find shortest distance from one designated vertex to
other vertex present in a Graph.
There are two methods-
 Dijktra’s algorithm
 Bellman Ford algorithm

 Dijktra’s Algorithm:

This Algorithm works with directed and undirected Graph with non negative edge weight.

Algorithm Dijkstra(G, s)
{
For each vertex v
Mark v as unvisited
and set d(v) = 1
Set d(s) = 0
while (there is unvisited vertex)
{
v = unvisited vertex with smallest d ;
Visit v, and Relax all its outgoing edges;

}
return d ;
}
Complexity: O (V2)

Page 49
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Bellman Ford Algorithm:

This Algorithm works with directed and undirected Graph with Negative/ Non negative edge
weight.

Algorithm Bellman-Ford(G, s)
{
For each v
Set d(v) = 1
Set d(s) = 0
for (k = 1 to |V|-1)

Relax all edges in G in any order ;

//check if s reaches a neg-weight cycle
for each edge (u,v)
if (d(v) > d(u) + weight(u,v))
return “something wrong !!”
return d ;
}
Complexity: O (VE)
4. Optimal Binary Search Tree (OBST):
It is a method by which BST is modified by selecting proper value as Root Node and balancing BST
such that entire information is implemented in minimum value.
The cost of OBST is associated with two values of probability successful and unsuccessful.
Procedure:
1. Construction of Probability Matrix:
𝑾 [ 𝟏 … 𝒏 , 𝟎 … 𝒏 − 𝟏]
By using:
𝑾[𝒊, 𝒋] = { 𝒘[𝒊, 𝒋 − 𝟏] + 𝒑𝒊 + 𝒒𝒊 }

2. Construction of Evaluation Matrix:

𝑬 [ 𝟏 … 𝒏 , 𝟎 … 𝒏 − 𝟏]
By using:
𝑬[𝒊, 𝒋] = 𝐦𝐢𝐧 { 𝑬[𝒊, 𝒓 − 𝟏] + 𝑬[𝒓 + 𝟏, 𝒋] + 𝒘[𝒊, 𝒋]}
𝒊 ≤ 𝒓≤ 𝒋

Page 50
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm OPTIMAL-BST(p, q, n: W, E, root)

{
for i = 1 to n + 1 do
{
E[i, i - 1] = q[i-1]
W[i, i - 1] = q[i-1]
}
for l = 1 to n do
for i =1 to n - l + 1 do
{
j=i+ l-1
E[i, j] = ∞
W[i, j] =W[i, j - 1] + p[j] + q[j]

for r = i to j do
{
t = E[i, r - 1] + E[r + 1, j] + W[i, j]
if (t < E[i, j])then
{
E[i, j] = t
root[i, j] = r
}
}
}
}
return E and root

5. Traveling Salesman Problem(TSP):

o The main objective of this problem is to find shortest path from one source to all vertices and
travelling back to source.
o The path should be such that all the vertices are covered in minimum distance.
o The concept is applicable on directed and complete graph.

To find shortest distance by using -

𝒈(𝒊, 𝒔) = 𝐦𝐢𝐧{ 𝑪𝒊𝒋 + 𝒈( 𝒋, 𝒔 − { 𝒋 }}

Page 51
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

6. Longest Common Subsequence (LCS):

If a set of sequences are given, the longest common subsequence problem is to find a common
subsequence of all the sequences that is of maximal length.

The longest common subsequence problem is a classic computer science problem, the basis of data
comparison programs such as the diff-utility, and has applications in bioinformatics. It is also widely
used by revision control systems.

Let X = < x1, x2, x3,…, xn > and Y = < y1, y2 , y3,…, ym > be the sequences. To compute the length of
an element the following algorithm is used.

Algorithm LCS ( cost)

{
for i = 0 to n
for j = 0 to m
if (i = 0 or j = 0) then
{
c[i,j] = 0
}
else
{
if ( i > 0 and j > 0 )then
if( a[i] = b[j]) then
c[i, j] = c[ i-1, j-1] + 1
else
c[i, j] = max( c[i,j-1], c[i-1, j])
}
}
Complexity: O(mn)

Page 52
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

7. Chain Matrix Multiplication:

Chain matrix multiplication is a method of find out best possible sequence which will give least of
multiplication.
Given a sequence of matrices, find the most efficient way to multiply these matrices together. The
problem is not actually to perform the multiplications, but merely to decide in which order to
perform the multiplications.
Objective:
The main objective is to find sequence generating minimum number of multiplication.
The minimum number of multiplication required to multiply the matrix.
Procedure:
1. Construction of Dimension Matrix:

𝐷 = [𝑑0 , 𝑑1 , 𝑑2 , … , 𝑑𝑛 ]
2. Construction of Cost Matrix:

To construct the cost matrix of 𝑛 × 𝑛 ( where ‘n’ is the no of Matrices)

o Assign all diagonal value to 0.
o Generate lower triangular matrix
o Calculate cost by using-
1. 𝑖𝑓 𝑗 = 𝑖 + 1 𝑜𝑟 𝑖 = 𝑗 − 1 𝑡ℎ𝑒𝑛

𝑴𝒊𝒋 = 𝒅𝒊−𝟏 ∗ 𝒅𝒊 ∗ 𝒅𝒊+𝟏

2. 𝑖𝑓 𝑗 > 𝑖 + 1 𝑡ℎ𝑒𝑛

𝒎𝒊𝒏
𝑴 𝒊,𝒊+𝒔 = 𝒊≤𝒌<𝑖+𝑠 { 𝑴𝒊𝒌 + 𝑴𝒌+𝟏,𝒊+𝒔 + 𝒅𝒊−𝟏 ∗ 𝒅𝒌 ∗ 𝒅𝒊+𝒔 }
3. Find optimal sequence

Page 53
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm Chain_Matrix(array p[1..n])

{
for i = 1 to n do m[i,i] = 0; // initialize
for l = 2 to n do
{
// l = length of subchain
for i = 1 to n-l+1 do
{
j=i+l-1;
m[i,j] = INFINITY;
for k = i to j-1 do
{
// check all splits
q = m[i, k] + m[k+1, j] + p[i-1]*p[k]*p[j]
if (q < m[i, j])
{
m[i,j] = q;
s[i,j] = k;
}
}
}
}
return m[1,n] (final cost) and s (splitting markers);

Page 54
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Difference between Divide and conquer and Dynamic Programming:

1. Difference between Greedy method and Dynamic Programming:

Page 55
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT V

Basic Traversal and Backtracking Technique

Syllabus
 Basic traversal and search technique
1. Breadth first search (BFS)
2. Depth first search (DFS)
 Backtracking basic strategy
1. N- Queen’s problem
2. Graph coloring
3. Hamiltonian cycle
4. Sum of subset
5. Introduction to approximation algorithms

Page 56
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Basic traversal and search technique:

Introduction:
Graph and trees required searching for a node with some specific properly. When the search require
necessarily visiting every node in the structure, then it called as traversal
One of the most fundamental graph problems is to traverse every edge and vertex in the graph.
There are following techniques are used-
1. Breadth first search (BFS)
2. Depth first search (DFS)

1. Breadth First Search (BFS):

BFS traverse a connected components of a graph and defines spanning tree.
BFS starts at a given vertex which is at level 0
In this stage, we visit all vertices at level 1, and then level 2 so on.
In BFS traversal terminates when every vertex has been visited.
Procedure:
1. Insert the starting vertex in the queue
2. Delete the front of queue and insert all adjacent vertex into queue.
3. Repeat step 2 until queue becomes empty

Page 57
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm BFS (V)

{
// BFS of a graph G is carried out at begin vertex V. for any node I visited [i] = 1
If i has been already being visited.
U= V
Repeat
{
For all vertex w adjacent to u do
{
If (visited[w] = 0) then
Add w to queue
Visited [w] = 1;
}
}
If queue is empty then return
else
delete U from queue.
}
until ( false )
Complexity O(V + E)
Application of BFS:
1. Testing whether graph is connected
2. Computing a spanning forest of graph.

Page 58
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Depth First Search (DFS):

In DFS, exploration of the node is suspended as soon as new unexplored node is reach. We go in the
depth first manner
Procedure:
1. Push the starting vertex into stack
2. Pop the stack top and push all its adjacent vertex onto stack
3. Repeat procedure until stack is empty
Algorithm DFS ( V )
{
Visited [v] = 1
For each vertex w adjacent from u do
{
If (visited[w] = 0)
{
DFS( w);
}
}
}
Complexity 𝜽 (V + E)
Application of DFS:
1. Testing whether the graph is connected.
2. Computing a spanning forest of graph.

Page 59
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Bi-connectedGraph:
Bi-connected graph has edges that are “two way” ie. We can go from one vertex to another and vice-
versa.
A Bi-connected graph is a connected and “non seprable” graph meaning that if any vertex to be
removed the graph will remain connected
Bi-connected graph has no articulation point
Bi- connected graph, if in a graph there is no AP then it is called is connected graph. ie. it is by
connected component which is entire graph.

1 2

5 4

Fig Bi- connected graph

Page 60
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Articulation point:
Represent a vertex in a graph such that if it is remove then graph is divided into 2 or more sub graph
this vertex is called as articulation point.
It is require to find AP then the principle of DFS is used.

1 AP 5

4 2 7

3 8

10
9

Method:
To find out articulation point on the graph perform DFS & generate a sequence representing the
order in which nodes are visited.

4 6 7

3 5 8

10 9 2

Sequence: 1-4-3-10-9-2-5-6-7-8
From the generated sequence draw a tree called DFS Tree in which the vertex is visited will decide
the position of vertex in the tree.

Page 61
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Tree edge:
If the edge is given in the graph & also present in generated tree then it is called as tree edge.
Back edge:
If the edge is present in the graph but not present in the generated tree then it is called as back edge
shown as (…………)
Forward edge:
Forward edges are those edge non tree edge (u,v) connecting a vertex u to descendent v in depth first
tree
Cross edge:
Cross edges are other edges they can go between vertices in the same DFS, as long as on vertex is
not an ancestor of the other, or they can go between verttices in different DFS
Rules:
If a vertex act as an intermediate in a connection using back edge ancestor, parent & child then
vertex must be an articulation point

A
1

P
10
9 2

6
7

8
C

The root node can be consider as AP if the degree of root node is 2 or more in the graph.
Page 62
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Backtracking:
Backtracking is a systematic way to go through all possible configuration of search space.
: basic idea of backtracking is build up a vector one component at a time and to test whether the
The
vector being formed has any chance of success.
:
Backtracking algorithm is that if it is realized that the partial vector generated does not lead to an
:
optimal solution then that vector may be ignored.
In:this method input is provided in the form of different component present in one set

: example:
For
Input X = { x1, x2, x3,………… xn}
X = Card deck then xn =52
The solution is generated by the reorganize in the given i/p with respect to certain constraints.
For example:
Output Y = { y1, y2, y3,………… yn}.
There are two types of constraints
1. Explicit constraints.
2. Implicit constraints
1. Explicit constraints:
If any condition is specifically specified in a given problem then it is called as explicit constraints
(given constraints)
2. Implicit constraints:
If the condition is derived from the given problem it is called as implicit constraints.
Note that:
The most important aspect of backtracking is to define the constraints. There is no process involve
between input and output, we have organize all input to get the output
There are following problem based on backtracking:
1. N- Queen’s problem
2. Graph coloring
3. Hamiltonian cycle
4. Sum of subset
5. Introduction to approximation algorithm

Page 63
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. N- Queen’s problem:
In N-Queen’s problem to define the constraints are-
Explicit constraints:
Chess board n× n is given in the problem where ‘n’ is the number of Queen’s
Implicit constraints:
1. No two Queen’s should be in same row
2. No two Queen’s should be in same column
3. No two Queen’s should be in same diagonally
There are two types of problems:
 4- Queen’s problem
 8- Queen’s problem

1. 4- Queen’s problem:
First define the constraints
Explicit constraints:
Chess board 4× 4 is given in the problem where ‘4’ is the number of Queen’s
Implicit constraints:
1. No two Queen’s should be in same row
2. No two Queen’s should be in same column
3. No two Queen’s should be in same diagonally
Possible solutions:
Solution 1

- Q1 - -

- - - Q2

Q3 - - -

- - Q4 -

Sequence x[2, 4, 1, 3 ]

Page 64
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Solution 2

- - Q1 -

Q2 - - -

- - - Q3

- Q4 - -

Sequence x[3, 1, 4, 2 ]

2. 8- Queen’s problem:
First define the constraints
Explicit constraints:
Chess board 8× 8 is given in the problem where ‘8’ is the number of Queen’s
Implicit constraints:
1. No two Queen’s should be in same row
2. No two Queen’s should be in same column
3. No two Queen’s should be in same diagonally
Possible solutions:
Solution 1
- - - Q1 - - - -
- - - - - Q2 - -
- - - - - - - Q3
- Q4 - - - - - -
- - - - - - Q5 -
Q6 - - - - - - -
- - Q7 - - - - -
- - - - Q8 - - -

Sequence x[4, 6, 8, 2, 7, 1, 3, 5]

Page 65
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Solution 2
- - - - Q1 - - -
- - Q2 - - - - -
Q3 - - - - - - -
- - - - - Q4 -
- Q5 - - - - - -
- - - - - - - Q6
- - - - Q7 - -
- - - Q8 - - - -

Sequence x[5, 3, 1, 7, 2, 8, 6, 4]

Page 66
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm N-queen( k, n)
{
For i = 1 to n do
{
if Place (k, i) then
X[k] = i
if ( k= n) then
Write ( x[1…..n])
else
N-queen(k +1, n)
}
}
Algorithm Place(k, i)
{
it is proposed to place k th queen in i th column
for j = 1 to k -1 do
{
if (x[j] = i) or
(abs (x[j] = abs( j – k))
return (false)
else
return (true)
}
}
k- row no or queen no
i- column
Complexity : O(k).

Page 67
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Graph coloring:
In a given graph G (V, E). It is required to color vertices to the fixed number of color. Such that
1. No two adjacent vertices should be same color.
2. Number of color used should not exceed the given number of color.
Constraints:
1. Explicit constraints:
Fixed number of color
2. Implicit constraints:
Connected vertices should not be the same color.
Chromatic number (K):
The chromatic number of a graph is the smallest number of colors needed to color the vertices so that
no two adjacent vertices be the same color.
Chromatic number is represent the minimum number of color used to color the graph
Solution space tree:
These tree will define all the possible solution which can exist for a given graph and number of color
Example:
Consider a graph and implement graph coloring of all possible condition

1 2

4 3

Solution:
Number of vertices = 4
Number of color used =
Chromatic number (K) = d + 1 = 2 +1 = 3

Page 68
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Solution space tree:

G
C1 C3
C2
1 1 1

C2
C3
2 2
C2
C1 C1
C3
3 3 3 3
C2 C2 C2 C3 C3
C3
4 4 4 4 4 4

Interchange c1 c2

C2

C1
C3
2 2
C1
C2 C2
C3
3 3 3 3
C1 C1 C1 C3 C3
C3
4 4 4 4 4 4

Page 69
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Interchange c2 c3

C3

C1
C2
2 2
C1
C3 C3
C2
3 3 3 3
C1 C1 C1 C2 C2
C2
4 4 4 4 4 4

Total number of solution = 6 *3 = 18

Page 70
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm:
Algorithm Mcoloring(k)
{
do
{
x[k] = Getnode color(k)
if( x[k] == 0)
return
if( k == n) then
write( x[ 1-n])
else
Mcoloring(k+1)
}
while(true)
}
Algorithm Getnode color(k)
{
do
{
x[k] = [x[k] + 1] mod (m + 1)
//next higher color
if (x[k] == 0)
then return // all color restricted
for( j = 1; j< n; j++)
{
if (G[k, j] ≠ && x[k] ==x[j])
// same color
break
}
}
}

Page 71
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Plannar Graph:
A graph is plannar if it can be drawn such that no edges cross when drawn on plane surface. A map,
used by geographers can be easily converted to plannar graph.

3. Hamiltonian cycle:
It is a cycle generated on undirected graph and it is required to start from particular vertex and end
that same vertex and visiting all vertices exactly once. If edge is used is not repeated again.
Application:
1. Hamiltonian cycle is used to cryptography method.
2. It is used for circuit testing
Constraints:
Explicit constraints:
Starting and ending vertex is same
Implicit constraints:
1. No repetition of edge is allowed.
2. If the vertex is already visited then it will be available in previously generated sequence and it
cannot be used again.
Condition:
1. In last vertex should have connected to first vertex.
2. If the vertex is referred and position k then it not should present 1 to k-1.
3. There should be edge between vertex k – 1 and k

Page 72
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Example:
Apply the Hamiltonian cycle for the following graph

1 2 3 4

8 7 6 5

Solution:
Cycle 1:
1-2-8-7-6-5-4-3-1
Cycle 2:
1-7-8-6-5-4-3-2-1
Cycle 3:
1-3-4-5-6-7-8-2-1

Page 73
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm:
Algorithm HCycle(k)
{
do
{
Next value (k)
if ( x[k] = 0) then
Return
if ( k = n)
write (1-----n)
else
HCycle(k + 1)
}
while(true)
}

Algorithm Next value(k)

{
do
{
x[k] = ( x[k] + 1)mod (n + 1)
if (x[k] = 0) then return
if (G(x[k-1),x[k]) != 0 ) then
for (j=1 to k -1) do
{
if( x[j] = x[k]) then
break;
}
If (j=k) then
return;
If(k=n) then
// last vertex
If(G(x[n], x[1]=1) then
return
}
while(true)
}

Complexity: O(nn)

Page 74
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

4. Sum of Subset:
It is a method by which a given array and a given integer operation is performed such that elements
are extracted from the array whose sum is equal to the given integer.
Constraints:
Explicit constraints:
Only given value should be used and summation should be equal to given sum.
Implicit constraints:
If an element ai is used then next element should be ai+1
There are two methods:
1. Variable size
2. Fixed size

5. Introduction to approximation algorithms:

Many problems are NP-complete, it is fast algorithm but that are not guarantee to give best solution,
so unlikely find efficient algorithm.
Three ways to get around:
• If input size is small, exponential algorithm is OK.

• Find near-optimal solutions in poly time.

So approximate algorithms returning a near-optimal solution is called approximate algorithm.

In approximation algorithm, we remove the requirement that the algorithm solves the optimization
problem must always generate an optimal solution.
Example of approximation algorithms are-
1. Vertex cover problem
2. Travelling salesman problem
3. Set covering problem

Page 75
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1. Vertex cover problem:

Vertex cover: given an undirected graph G=(V,E), then a subset V'V such that if (u,v)E, then
uV' or v V' (or both).
Size of a vertex cover: the number of vertices in it.
Vertex-cover problem: find a vertex-cover of minimal size.
APPROX-VERTEX-COVER(G)
1 C←Ø
2 E′ ← E[G]
3 while E′ ≠ Ø
4 do let (u, v) be an arbitrary edge of E′
5 C ← C U {u, v}
6 remove every edge in E′ incident on u or v
7 return C

Page 76
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Page 77
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

2. Travelling salesman problem:

Given an undirected weighted Graph G we are to find a minimum cost Hamiltonian cycle.
Satisfying triangle inequality or not this problem is NP-Complete.
We can solve Hamiltonian path.
Exact exponential solution:
APPROX-TSP-TOUR(G, c)

1 select a vertex r Є V[G] to be root.

2 compute a MST for G from root r using Prim Alg.
3 L=list of vertices in preorder walk of that MST.
4 return the Hamiltonian cycle H in the order L.

Example

a d

b f g

c h

Fig. (a) given set of points

Page 78
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

a d

b f g

c h

Fig.(b) MST

a d

b f g

c h

Fig. (c) full tree walk on T

a d

b f g

c h
Fig.(d) preorder sequence gives a tour

3. Set covering problem:

Generalization of vertex-cover problem.

Page 79
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

We have given (X,F) :

X : a finite set of elements.
F : family of subsets of X such that every element of X belongs to at least one subset in F.
Solution C : subset of F that Includes all the members of X.

RTMNU QUESTIONS:

1. Let W = {11,13,7,24} and m = 31. Find all possible subset of W that generate sum = m and draw
fixed size and variable size tuple solution space tree for same [w-10].
2. For the following graph, design Hamiltonian cycle. Write the algorithm for same. Explain how
: solution may result in multiple solutions.[w-10]
one

:
1 2 3 4
:
:
6
:
9 8 7 5

3. What is planner graph? implement graph coloring on the following graph and generate the solution
space tree if number of permitted colors = 3 give algorithm for graph coloring
[w-10, s-11,s-12]

1 2

4 3

4.Write an algorithm to solve 8 queens problem. Explain explicit constraints and implicit constraints
associated with this problem. Give at least 2 solutions for this problem.
[s-11,w-11,s-12,w-12,w-13, s-14]
5. What is backtracking? Explain the application in which backtracking principal can be used to
design solution. [s-11]
6. What is use of Hamiltonian Cycle? Implement cycle on following graph.[s-11,w-11,w-13]

Page 80
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1 2 3 4

8 7 6 5

7. What is bi-connected graph? find articulation point for the following graph.[w-11]

1 7

2 3 8

4 9

5
6

8. Implement Hamiltonian cycle on the following graph. generate at least two cycle. Write algorithm
for same.[w-12,s-12]

Page 81
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

1 2 3 4

8 7 6 5

9. Draw the solution space tree for fixed size tuple and variable size tuple for following array and
positive integer.
a = [12,16,9,25,10] sum = 35.
Find out solution node. [w-12]
9. Find out articulation point in the following graph. explain the procedure to find articulation point.
[w-12,s-14]

1 5

4 2 7

3 8

10
9

10. What is optimal vertex cover problem? Implement the problem on following graph to find out the
suitable vertex cover write algorithm.[s-13]

Page 82
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

A C E

D F
B

11. With respect to DFS, define Tree edge, Back edge, Forward edge, Cross edge. In DFS of an
undirected graph G. prove that every edge of G is either a free edge a back edge.[s-12]
12. Explain articulation point in DFS and give the complete algorithm for finding articulation point
of undirected graph.[w-13]
13. Explain approximation algorithm. Write an algorithm for vertex cover problem using
approximation principle.[w-13]
14. Design a solution for Hamiltonian path, explain how the solution can be used to solve
Hamiltonian cycle problem.[s-14].

Page 83
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

UNIT-VI
NP problems
Introduction to NP problem:
Many computer science problems are covered by the class NP. In particular, the decision versions of
many interesting search problems and optimization problems are contained in NP.
NP principle based on the concepts of Problem and Algorithm.

1. Classification of problems:
In computer science problems are classified in to following types-
1. Decision problem
2. Optimization problem
3. Search problem
4. Counting problem
5. Function problem
1. Decision problem:
Any problem for which the answer is either yes or no depending on the values of some input
parameter is called decision problem. Instead of yes/no sometimes it is also uses true/false,
success/failure as a output. Output of decision problem are Boolean. The input parameters can be
natural numbers as well as strings o formal languages.

Input

Algorithm

Yes No

Example: Give two n umbers x and y , does x divided by y is a decision problem. Because the
answer can be either yes or no depending upon value of x and y.

2. Optimization problem:
The problem of finding the best solution from all feasible solution is an optimization problem, it
involves the identification of optimal solution ie. Either maximum or minimum.
Example:
Finding chromatic number is the optimization problem.

Page 84
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

3. Search problem:
The process of finding particular elements from given problem domain is referred as search problem
Example:
Linear and binary search problems.

4. Counting problem:
The task of finding the number of elements of some set with a particular property.
Example:
Such counting problems are usually encountered in combinatorics.

5. Function problem:
In computational complexity theory, a function problem is a computational problem where a single
output (of a total function) is expected for every input, but the output is more complex than that of a
decision problem. For function problems, the output is not simply 'yes' or 'no'.
Example: Travelling seals man problem.

2. Classification of Algorithms:
Result return by the algorithm may be defined or may not be defined uniquely based on the
following factors-
1. Deterministic Algorithm
2. Non deterministic Algorithm

1. Deterministic Algorithm:
A Deterministic algorithm behaves as expected about the result of every operation of algorithm.
Deterministic algorithm always produces the same output and the machine will always pass through
the same sequence of states.

2. Non deterministic algorithm:

A non deterministic algorithm is an algorithm which has one or more choices and without any
specification any one if them can be chosen. So, non deterministic algorithm produces an answer by
a series of “correct guess”. It consist of following concepts.
1. Guessing stage:
An arbitrary string S is generated which we can think as a candidate proposed solution to the
problem instances.

2. Verifying stage:
An deterministic algorithm takes both instances and input S, and checks whether S satisfied or not. It
returns success or failure.

Page 85
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Algorithm of Non-deterministic sorting:

Assumption:
Sorting array A[1:n] of positive integers in ascending order
Algorithm NSort(A, n)
{
for i=1 to n do B[i] = 0 // initialize
for i=1 to n do
{
j= choose(1,n) // guessing
if ( B[j] !=0) then fail( )
B[j] = A[i]
}
for i=1 to n-1 do // verifying
{
if ( B[i] > B[i+1] then fail( )
write(B[1:n])
Success( )
}
Classes of problems:
There are mainly four classes of problems-
1. P- Class
2. NP- Class
3. NP- Complete Class
4. NP- Hard Class
P- class:
All decision problem that can be solved in polynomial time is called as P- class problem.
This class of problem are solved in O(nk) time where k is constant and n is the size of input value.
Example: shortest path problem
NP class:
NP refers to Non deterministic polynomial time. NP class problem are decision problem that are
solvable in polynomial time by Non deterministic algorithms and their solution is verifiable using a
deterministic algorithm. There are certain problem which are only NP type not p- type.
Example:

Page 86
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Graph coloring problem.

NP- complete class:
A decision problem A is said to be NP-complete if it satisfies the following properties-
1. It belongs to NP class
2. Every problem in NP is polynomially reducible to A.
In NP complete if solution to this Acan be verified in polynomial time then it can be shown to be in
NP.
If the problem satisfies only case 2 then it is called in NP- hard problem.
Some well known example that are NP complete when expressed as decision problems-
o Knapsack problem
o Circuit satisfiability problem (C-SAT)
o Clique decision problem(CDP)
NP- Hard class:
NP hard problem are both decision and optimization problem. To prove a problem A is NP hard,
reduce a known NP hard problem to A. if NP hard problem can be solved in polynomial time. As a
result, all NP complete problem are NP hard, but all NP hard problem are not NP complete.
Example: The Travelling salesman problem is example of an optimization problem of NP hard type.
The Turing Machine Halting Problem is decision problem that is NP hard type but not complete.

Relationship Among Classes of problem:

Page 87
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Cook’s Theorem:
The theorem can be states that-
o Any NP problem can be converted to SAT in polynomial time.
o C-SAT is NP- complete.
o Satisfiability is NP complete.
Prove that P ⊆ NP , Also explain any three difference between P and NP.
ANS:
o The class of P problem is sub- class of NP problems because a problem in P can be solved in
polynomial time without given any certificate.
o NP-type problem is trying to solve (verifiable) in polynomial time thus P ⊆ NP
Difference between P and NP
P NP

1. In polynomial time the input is fixed In NP the input is not fixed

2. P problem can be efficient NP problem which gives proposed solution.

3. Process and time is fixed Process and time is not fixed

4. P is a set of all decision problem NP is a set of all decision problem solvable

solvable by deterministic algorithm in by Non deterministic algorithm in polynomial
polynomial time time

5. Example: bubble sort O(n2) Example: Graph coloring

Page 88
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Clique Decision Problem (CDP):

In an undirected graph, a clique is a complete sub-graph of the given graph. Complete sub-graph
means, all the vertices of this sub-graph is connected to all other vertices of this sub-graph.
The Max-Clique problem is the computational problem of finding maximum clique of the graph.
Max clique is used in many real-world problems.
Algorithm: Max-Clique (G, n, k)
S := Φ
for i = 1 to k do
t = choice (1…n)
if t Є S then
return failure( )
S := S ∪ t
for all pairs (i, j) such that i Є S and j Є S and i ≠ j do
if (i, j) is not a edge of the graph then
return failure
return success

Analysis
Max-Clique problem is a non-deterministic algorithm. In this algorithm, first we try to determine a
set of k distinct vertices and then we try to test whether these vertices form a complete graph.
There is no polynomial time deterministic algorithm to solve this problem. This problem is NP-
Complete.
Example:
Take a look at the following graph. Here, the sub-graph containing vertices 2, 3, 4 and 6 forms a
complete graph. Hence, this sub-graph is a clique. As this is the maximum complete sub-graph of the
provided graph, it’s a 4-Clique.

Page 89
Notes Prepared By. Ahmad Sir
Contact: 9867597554
PERFECT ENGINEERING CLASSES
Contact: S-54,Utkarsh Nirmaan Complex, Mangalwari, Sadar, Nagpur

Polynomial time reduction:

Before we have seen that sometimes one problem can be solved by using another problem.
Consider two decision problems A and B.

 If problem B can be solved in polynomial time then problem A can be solved in polynomial time
 If there is no polynomial-time algorithm to solve A then there is no polynomial-time algorithm to
solve B
We have reduced problem A to problem B.
Definition:
We can say that a decision problem A is polynomially many-one reducible to problem B (A<=B) if
there exists a function f computable in polynomial time that:
 maps every instance x of A to an instance f(x) of B and
 x is a yes-instance of A if and only if f(x) is a yes-instance of B.

Page 90
Notes Prepared By. Ahmad Sir
Contact: 9867597554