BCSE204L- Design and

Analysis of Algorithms

Dr. Iyappan Perumal

Assistant Professor Senior Grade 2
School of Computer Science & Engineering
VIT,Vellore.
BCSE204L- Design and Analysis of Algorithms-

Course Objective

1. To provide mathematical foundations for analysing

the complexity of the algorithms

2. To impart the knowledge on various design

strategies that can help in solving the real world
problems effectively

3. To synthesize efficient algorithms in various

engineering design situations
 BCSE204L- Design and Analysis of Algorithms-
Course outcome

1. Apply the mathematical tools to analyse and derive

the running time of the algorithms

2. Demonstrate the major algorithm design paradigms.

3. Explain major graph algorithms, string matching and

geometric algorithms along with their analysis.

4. Articulating Randomized Algorithms.

5. Explain the hardness of real-world problems with

respect to algorithmic efficiency and learning to

cope with it.

BCSE204L- Design and Analysis of
Algorithms- Syllabus
 BCSE204L- Design and Analysis of
Algorithms- References

Text Books:
1. Thomas H. Cormen, C.E. Leiserson, R L.
Rivest and C. Stein, Introduction to
Algorithms, 2009, 3rd Edition, MIT Press.

Reference Books:
1. Jon Kleinberg, ÉvaTardos ,Algorithm Design,
Pearson education, 2014.
2. Rajeev Motwani, Prabhakar Raghavan;
Randomized Algorithms, Cambridge
University Press,1995 (Online Print – 2013)
3. Ravindra K.Ahuja, Thomas L.Magnanti and
James B.Orlin, “ Network Flows: Theory,
Algorithms and Applications”, Pearson
Education, 2014.
Module 1: Design Paradigms: Greedy, Divide
and Conquer Techniques(6 Hours)
 Overview and Importance of Algorithms
◦ Stages of algorithm development:
 Describing the problem
 Identifying a suitable technique
 Design of an algorithm
 Derive Time Complexity,
 Proof of Correctness of the algorithm
 Illustration of Design Stages
◦ Greedy techniques:
 Fractional Knapsack Problem
 Huffman coding
◦ Divide and Conquer:
 Maximum Subarray
 Karatsuba faster integer multiplication
algorithm.
 Time Complexity
 The amount of time taken by an
algorithm to run to completion
 It is measured by number of primitive
operations required to produce output.
 Space complexity
 The amount of space required by an
algorithm to run to completion.
 Mostly, time and space complexity is
computed as a function of input size
 Frequency Count/Step Count
Method
 Number of times a statement is executed.
 RULES:
◦ For comments, declarations, STEP COUNT=0
◦ For return, assignment statements, STEP COUNT=1
◦ Ignore lower order exponents when higher order
exponents are present.
◦ Ignore constant multipliers
 Example: Sum of N Numbers(Iterative )
Algorithm Step Count
--
Alg sum(a[],n)
--
{
1
sum=0;
N+1
for(i=1 to n) do
N
sum=sum+a[i];
1
return sum;
}
2N+3
TOTAL
O(N)
 Example: Sum of N Numbers( Recursive)

Algorithm N=0 N>0

Alg Rsum(a,n) -- --

{ -- --

if(n<=0) then 1 1

return 0; 1 --

else return Rsum(a,n-1)+a[n]) -- 1+x

} --

TOTAL 2 2+x

X=tRsum(n-1)
 Recurrences equation

 tRsum(n)= 2 if n=0
2+tRsum(n-1) if n>0

= 2+tRsum(n-1)

= 2+2+tRsum(n-2)

= 2+2+2+tRsum(n-3)

= n(2)+tRsum(0)

= 2n+2 = O(n)
Example: Sum of N Numbers(Space Complexity)
Algorithm
Alg sum(a[],n) A[]– n words
{ n=one word
sum=0; Sum=one word
for(i=1 to n) do i= one word
sum=sum+a[i];
return sum;
}
TOTAL N+3
O(n)
 Recurrences and Running Time

 An equation that describes a function in terms of

its value on smaller inputs.

Example: T(n) = T(n-1) + n

 Recurrences arise when an algorithm contains

recursive calls to itself.

 What is the actual running time of the algorithm?

◦ Need to solve the recurrence

 Example Recurrences
 T(n) = T(n-1) + n Θ(n2 )
Recursive algorithm that loops through the input
to eliminate one item
• T(n) = T(n/2) + c Θ(logn)
Recursive algorithm that halves the input in one
step
• T(n) = T(n/2) + n Θ(n)
Recursive algorithm that halves the input but must
examine every item in the input
• T(n) = 2T(n/2) + 1 Θ(n)
Recursive algorithm that splits the input into 2
halves and does a constant amount of other work
 Orders of growth

1<logn<√n<n<nlogn<n2<n3<……2n < 3n<nn

Example: Binary Search
Algorithm BINARY-SEARCH (A, low, high, x)
{
if (low > high) ------------ Constant time: C1
return FALSE
mid=(low+high)/2 ------------ Constant time: C2
if x = A[mid] ------------ Constant time: C3
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, low, mid-1, x) --Same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, high, x) --Same problem of size n/2

} T(n) = C+T(n/2)

T(n) – running time for an array of size n

Iteration Method
 Convert the recurrence into a summation and try to
bound it using known series.

◦ Iterate the recurrence until the initial condition is

reached.

◦ Use back-substitution to express the recurrence in

terms of n and the initial (boundary) condition.
 Iteration Method
==> T(n) = C+T(n/2)
==> T(n) = C+C+T(n/4) Where T(n/2) = c + T(n/4)
==> T(n) = C+C+C+T(n/8)where T(n/4) = c + T(n/8)
…………………..
 T(n) = c + c + … + c + T(n/2k)
k times

= K.C + T(1)
= C.logn +1
Note: Where
n/2k = 1 : Place not
divided further
= Θ(log n) so, n = 2k
 Iteration Method - Example
T(n) = n + 2T(n/2)
T(n) = n + 2T (n/2)
= n + 2(n/2 + 2T(n/4))where T(n/2) = n/2 + 2T(n/4)
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
Pattern is repeated for i times
… = i(n) + 2i T(n/2i )
= i(n) + 2i T(1)
= nlogn + nT(1)
= Θ(nlogn)
 Master Method
 Master Theorem is the easiest way of
obtaining runtime of recursive algorithms.
 General formula for the Master Theorem
is: Any other
instructions/
operation time
Number of taken
times done
Division halves
Where a ≥ 1, b ≥ 1 and f(n)= Θ(nklogp n) ,K ≥ 1
 Once, we have a, b and f(n) we can
determine the runtime of the work done
by the recursion.
 It is given by
3 CASES
 CASE 1: Logb a > k T(n)= Θ(n logb a )
 CASE 2: Logb a = k
◦ If p>-1, T(n)= Θ(nklogp+1 n)
◦ If p= -1, T(n)= Θ(nklog(log n))
◦ If p<-1, T(n)= Θ(nk)
 CASE 3: Logb a < k
◦ If p>=0, T(n)= Θ(nklogp n)
◦ If p<0, T(n)= Θ(nk)
 Solve the recurrence
T(n)= 16T(n/4)+n3
 CASE 1: Logb a > k 1. a=16, b=4, f(n)= n3
T(n)= Θ(n logb a ) 2. Find the values of k and p
 CASE 2: Logb a = k 3. f(n)= Θ(nklogp n)
4. In the problem f(n)= n3
 If p>-1, T(n)= Θ(nklogp+1 n) Compute and find k and p
 If p= -1, T(n)= Θ(nklog(log 5. k=3, p=0
n)) 6. Find Logb a = Log4 16 = 2
 If p<-1, T(n)= Θ(nk) 7. Compare Logb a with k
 CASE 3: Logb a < k Here it (2<3) falls on case
3
 If p>=0, T(n)= Θ(nklogp n)
8. Check with p value it is >=0
 If p< 0T(n)= Θ(nk) 9. Θ(n3log0 n)
10. Θ(n3)
T(n)= 4T(n/2)+n2 log n
 Try yourself….
T(n) = T(n-1) + n ---- Θ(n2 )
 Recursive algorithm that loops through the input to
eliminate one item- Bubble sort using Recursion
Algorithm Bubble sort(A,N)
{
if (n==1) 1
return;

// One Pass of Bubble sort, Largest element placed in end

for (int i=0; i<n-1; i++) N
{
if (arr[i] > arr[i+1]) N-1
swap(arr[i], arr[i+1]);
}

// Recursion for remaining array

Bubble sort(A,N-1)
}
T(n) = T(n-1) + n
= T(n-2) + n-1 + n
= T(n-3) + n-2 + n-1 + n
// Generalize to k
= T(n-k) + n-k+1 + n-k+2 + ... + n-1 + n
// since n-k = 1 so T(1) = 1
= 1 + 2 + ... + n
//Here = n(n-1)/2 = n2/2 - n/2
// By Taking the dominating term which is n2*1/2
= O(n2)
Finding Max & Min – General solution

Alg Generalmaxmin(a, n, max, min)

{
max=min=a[1];
for i=2 to n do
{
if(a[i]>max) then max =a[i];
if(a[i]<min) then min=a[i];
}
}

Time : 2(n-1) comparisons in best, average and worst cases

Finding Max & Min – Improvement
Alg Generalmaxmin(a, n, max, min)
{
max=min=a[1];
for i=2 to n do
{
if(a[i]>max) then max =a[i];
else if(a[i]<min) then min=a[i];
}
}

Best case occurs when elements are in increasing order -(n-1)

comparisons
Worst case occurs when elements are in decreasing order -2(n-1)
comparisons
Average case occurs greater than max half the time- (3n/2 - 1)
Divide and Conquer Strategy- Finding Max & Min
 Analysis A[1] 2 3 4 5 6 7 8 9
of Max & 20 3 -15 -8 25 80 14 51 27
Min
80 -15 Problem is
Problem is big
big 1 9 Max Min
9
25 -15 80 14
1 5 Max Min 6 9 Max Min
5 8

20 -15 25 -8 80 14 51 27
1 3 Max Min 4 5 Max Min 6 7 Max Min 8 9 Max Min

3 7
4 6

20 3 -15 -15 Problem

1 2 Max Min 3 3 Max Min size : 2
elements
1 2 Problem size : 1
element
Analysis of Max & Min

 What is the number of element comparisons

required??- The recurrence relation is

 By solving this, we get (3n/2 – 2)Comparisons

required for Best, Worst and average cases

 In case of storage, this is not better because it requires

stack space for (i,j,max, min, max1, min1)