Unit: 1

 Introduction:-
What is an Algorithm?
Performance Analysis:
-Space Complexity & Time Complexity
Asymptotic notations
 Divide and Conquer :-
General method
Binary search
Finding Maximum and Minimum
Merge sort
Quick sort
Strassen’s Matrix Multiplication
 What is an Algorithm?
 An algorithm is a finite set of instructions that, if followed, accomplishes a
particular task.
 Characteristics of an Algorithm:
1) Input: Zero or more quantities are externally supplied.
2) Output: At least one quantity is produced.
3) Definiteness: Each instruction is clear and unambiguous.
4) Finiteness: If we trace out the instructions of an algorithm, then for all
cases, the algorithm terminates after a finite number of steps.
5) Effectiveness: Every instruction must be basic enough to be carried out,
in principle, by a person using only pencil and paper. It must be feasible.
 What is Pseudo code?
 Pseudo code is an informal way of programming description that
does not require any strict programming language syntax or
underlying technology considerations.
 It is used for creating an outline or a rough draft of a program.
 Pseudo code summarizes a program's flow, but excludes
underlying details.
What is Pseudo code?
[Pseudo code for printing sum of n natural numbers]

begin
Integer counter, sum=0
for counter=1 to 100 step by 1 do
sum=sum + counter
end for
print sum
end
Performance Analysis
The performance analysis is a process of estimating the
efficiency of an algorithm.
There are two fundamental parameters based on which we can
analyze the algorithm:
 Space Complexity: The total amount of memory required by
an algorithm to complete its execution expressed as a function
of number of inputs.
 Time Complexity: The total amount of time required by an
algorithm to complete its execution expressed as a function of
number of inputs.
Space Complexity
 When we design an algorithm to solve a problem, it needs some
computer memory to complete its execution.
 For any algorithm, memory is required for the following
purposes...
 Memory required to store program instructions
 Memory required to store constants
 Memory required to store variables
 And for few other things (functions, dynamic allocation,
arguments of recursive calls, return address)

[Memory -> Instruction space+ Environmental stack+ data space]

Space Complexity
#include<stdio.h> Here 3 integer variables are
int main() used. The size of the integer
{ data type is 4. So, the total
space occupied by the program
int a = 5, b = 5, c; is 4 * 3 = 12 bytes. Since no
c = a + b; additional variables are used,
printf("%d", c); no extra space is required.
} Hence, space complexity for
the above-given program is
O(1), or constant.
Space Complexity
#include <stdio.h> In this code, the array consists
int main() of n integer elements. So, the
{ space occupied by the array is
4 * n. Also we have integer
int n, i, sum = 0; variables such as n, i and sum.
scanf("%d", &n); Assuming 4 bytes for each
int arr[n]; variable, the total space
for(i = 0; i < n; i++) occupied by the program is 4n
{ + 12 bytes. Since the highest
order of n in the equation 4n +
scanf("%d", &arr[i]); 12 is n, so the space complexity
sum = sum + arr[i]; is O(n) or linear.
}
printf("%d", sum);
}
Space Complexity
int sum(int A[], int n) 'n*2' bytes of memory to store
{ array variable 'a[]'
int sum = 0, i; 2 bytes of memory for integer
parameter 'n'
for(i = 0; i < n; i++) 4 bytes of memory for local
sum = sum + A[i]; integer variables 'sum' and 'i'
return sum; (2 bytes each)
} 2 bytes of memory for return
value.

Totally it requires 2n+8

(Consider int capacity as 2B) bytes of memory to complete
its execution.
Space Complexity
 If any algorithm requires a fixed amount of space for all input
values then that space complexity is said to be Constant Space
Complexity.
 O(1) Constant Space Complexity occurs when the program
doesn’t contain any loops, recursive functions or call to any
other functions.
 If the amount of space required by an algorithm is increased
with the increase of input value, then that space complexity is
said to be Linear Space Complexity.
 O(n) Linear space complexity occurs when the program
contains any loops.
Time Complexity
#include <stdio.h> Here, 1 unit of time for
int main() executing every statement. So
{ total 4 units of time,
irrespective of inputs.
int a = 4;
int b = 6;
The time complexity of the
int c; given program is O(1), as this
c = a + b; program consists of only
printf(%d, c); assignment and arithmetic
} operations and all those will
be executed only once.
Time Complexity
 Time Complexity
int count(int arr[], int n)  Here we have a control
{ statement which executes for ‘n’
int sum = 0,i; times. Along with that we also
have operations like
for(i = 1; i < n; i++) assignment, arithmetic and a
{ return statement. Hence, the
sum = sum + arr[i]; time complexity is (2n + 3)
} units.
return sum;  For larger values of n, the
} constant values become
negligible.
 Hence, the final time
complexity of the code is O(n).
Time Complexity
int main() Here, the first & the second
{ for loops get executed n times
int i,j, n = 8; individually.
for (i = 1; i <= n; i++) Here, the time complexity is
expressed as T(n)= 2n2+2n+3
{
So the time complexity will be
for (j = 1; j <= n; j++) n*n = O(n2)
{
printf(“Hello CSE");
}
}
printf(“How are you?");
}
 Time Complexity
while(low<=high) Here, the logic breaks the
{ given set of elements into two
mid=(low+high)/2; halves and then searches for a
if(n<arr[mid]) particular element. Further, it
keeps dividing these two
high=mid-1; halves into further halves until
else if(n>arr[mid]) each individual element is
low=mid+1; searched.
else All such algorithms which
break; work on the principle of
} recursive division of elements
into halves will have O(Log n)
complexity.
Time Complexity
Time Complexity
Time Complexity
 O(1) Constant Time Complexity O(1) occurs when the program doesn’t
contain any loops, recursive functions or call to any other functions. The
run time, in this case, won’t change no matter what the input value is.
 O(n) Linear Time Complexity O(n) occurs when the run time of the
code increases at an order of magnitude proportional to n. Here n is the
size of the input.
 O(log n) Logarithmic Time Complexity O(log n) occurs when at each
subsequent step in the algorithm, the time is decreased at a magnitude
inversely proportional to n. This generally happens in Binary Search
Algorithm.
 O(n log n) Time Complexity. Example of algorithms that runs with this
time complexity is Quick Sort, Heap Sort, Merge Sort
 O(n²) Quadratic Time Complexity
 O(2^n) Exponential Time Complexity
 O(n!) Factorial Time Complexity
Time Complexity
Time complexity: Y axis is growth of a function f and X axis is n values

O(1) < O(log n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n)
 Asymptotic Notations
 The word Asymptotic means approaching a value or curve
arbitrarily closely.
 Asymptotic notations are the mathematical notations used to describe
the running time of an algorithm when the input tends towards a
particular value or a limiting value.
 For example: In bubble sort, when the input array is already sorted, the
time taken by the algorithm is linear i.e. the best case.
 But, when the input array is in reverse condition, the algorithm takes the
maximum time (quadratic) to sort the elements i.e. the worst case.
 When the input array is neither sorted nor in reverse order, then it takes
average time. These are denoted using asymptotic notations.
There are mainly three asymptotic notations:
 Big-O notation
 Omega notation
 Theta notation
 Asymptotic Notations
 Asymptotic analysis is a technique of representing limiting
behavior. The methodology has the applications across
science. It can be used to analyze the performance of an
algorithm for some large data set.

 The simplest example is a function ƒ (n) = n2+3n, the term 3n

becomes insignificant compared to n2 when n is very large.

 The function "ƒ (n) is said to be asymptotically equivalent

to n2 as n → ∞", and here is written symbolically as ƒ (n) ~ n2.
Asymptotic Notations
Asymptotic Notations
Definition [Big "oh"] The function f(n) = O(g(n)) iff there
exist positive constants c and no such that f(n) ≤ c.g(n) for all
n, n ≥ no.
Example functions
 3n+2 = O(n) as 3n+2 ≤ 4n for all n ≥ 2.
 3n+3 = O(n) as 3n + 3 ≤ 4n for all n ≥ 3.
 100n+ 6 = O(n) as 100n + 6 ≤ 101 n for all n ≥ 6.
 10n2+4n +2 = O(n2) as 10n2+4n +2 ≤ 11n2 for all n ≥ 5.
 3n+3 = O(n2 ) as 3n+3 ≤ 3 n2 for n ≥ 2.
 10 n2 +4n +2 = O(n4) as 10 n2 +4n +2 ≤ 10n4 for n ≥ 2.
 3n +2 ≠ O(1) as 3n +2 is not less than or equal to ‘c’ for any
constant ‘c’ and all n ≥ n0
 10n2+4n +2 ≠ O(n).
Asymptotic Notations
Asymptotic Notations
Definition [Omega] The function f(n) = Ω(g(n)) iff there
exist positive constants c and no such that f(n) ≥ c.g(n) for all
n, n ≥ no.
Example functions
 3n+2 = Ω(n) as 3n+2 ≥ 3n for all n ≥ 1.
 3n+3 = Ω(n) as 3n + 3 ≥ 3n for all n ≥ 1.
 100n+ 6 = Ω(n) as 100n + 6 ≥ 100 n for all n ≥ 1.
 10n2+4n +2 = Ω(n2) as 10n2+4n +2 ≤ n2 for all n≥1.
 6*2n +n2 = Ω(2n) as 6*2n +n2 ≥ 2n for n ≥ 1.
 Also 3n+3 = Ω(1). 10 n2 +4n +2 = Ω(1) or Ω(n)
 6*2n +n2 = Ω(n100), Ω(n50.2) , Ω(n2), Ω(n), Ω(1).
Asymptotic Notations
Asymptotic Notations
Definition [Theta] The function f(n) = Θ(g(n)) iff there exist
positive constants c1 c2 and no such that c1 g(n) ≤ f(n) ≤ c2 g(n)
for all n, n ≥ no.
Example functions
 3n+2 = Θ(n) as 3n+2 ≥ 3n for all n ≥ 2 and
3n+2 ≤ 4n for all n ≥ 2, so c1=3, c2=4 and no = 2.
 3n+3 = Θ(n)
 10n2+4n +2 = Θ(n2)
 6*2n +n2 = Θ (2n)
 10*logn + 4 = Θ(logn)
 3n+2 ≠ Θ(1), 3n+3 ≠ Θ(n2), 10n2+4n +2 ≠ Θ(n),
6*2n +n2 ≠ Θ(n2), Θ(n100) , Θ(n2)
 The General Method
 Divide and Conquer algorithm consists of a dispute using the
following three steps.
Divide the original problem into a set of sub problems.
Conquer: Solve every sub problem individually, recursively.
Combine: Put together the solutions of the sub problems to
get the solution to the whole problem.

Examples: The specific computer algorithms are based on the

Divide & Conquer approach:
 Binary Search
 Maximum and Minimum Problem
 Sorting (merge sort, quick sort)
 Tower of Hanoi.
The General Method
 Binary Search - Analysis
Let T (n) = number of comparisons of an item with n elements
in a sorted array.
• Set beg= 1 and end= n
• Find mid = (beg + end)/2
• Compare the search item with the mid item.
• Case 1: item = A[mid], then LOC = mid, but it the best case
and T (n) = 1
• Case 2: item ≠A [mid], then we will split the array into two
equal parts of size n/2.
• And again find the midpoint of the half-sorted array and
compare with search element.
• Repeat the same process until a search element is found.
 Binary Search - Analysis
T(n)= T(n/2) + 1 (if n>1)
=1 (if n=1)

T(n)= T(n/2) + 1
T(n/2)=T(n/4)+1
T(n)=T(n/4)+2
T(n)=T(n/2k)+k
Recursion will stop If n=2k then T(1)=1
T(n)=T(1)+k
T(n)=1+log2n
T(n)=log2n
 Finding Max and Min in Array
 Method 1: if we apply the general approach to the array of size
n, the number of comparisons required are 2n-2.
Algorithm: Max and Min Element (a [])
Max= a [1]
Min= a [1]
For i= 2 to n do
if a[i]> max then max = a[i]
else if a[i] < min then min= a[i]
print(max, min)
 Method-2: In another approach, we will divide the problem
into sub-problems and find the max and min of each group.
Now max of each group will compare with the only max of
another group and min with min.
Divide and Conquer Method
 Max1=Min1=a[mid+1];

return(Max,Min);
Max and Min Problem - Analysis
T(n)= T(n/2) + T(n/2) + 2 (if n>2)
=1 (if n=2)

T(n)= 2T(n/2) + 2
T(n/2)=2T(n/4)+2
T(n)=2[2T(n/4)+2 ]+2
T(n)=22T(n/22)+22+2
T(n)=2kT(n/2k)+2k+2k-1+.....+2
Recursion will stop if n/2k =2 then T(2)=1
T(n)=2kT(2)+2(2k-1)/(2-1) [GP: a(rn-1)/(r-1)]
T(n)=2k+2(2k-1)=n/2+n-2=3n/2-2
T(n)=(3n/2)-2
 Merge Sort
 The Merge Sort function keeps on splitting an array into two
halves until a condition is met where we try to perform Merge
Sort on a sub array of size 1.
 And then, it combines the individually sorted sub arrays into
larger arrays until the whole array is merged.
Algorithm MERGESORT (low, high)
{ // a (low : high) is an array to be sorted.
if (low < high)
{
mid = (low + high)/2 //finds where to split the set
call MERGESORT(low, mid) //sort one subset
call MERGESORT(mid+1, high) //sort the other subset
call MERGE(low, mid, high) // combine the results
}
}
 Merge Sort
Algorithm MERGE (low, mid, high)
{ k=low; i := low; j:= mid + 1;
while ((i < mid) and (j < high)) do
{ if (a[i] < a[j]) then { b[k] := a[i]; i := i + 1;}
else { b[k] :=a[j]; j := j + 1; }
k := k + 1;
}
while(i<mid) do
{ b[k] := a[i]; i := i + 1; k:=k+1}
while(j<high) do
{ b[k] := a[j]; j := j + l; k:=k+1 }
for k := low to high do
a[k] := b[k]; }
Merge Sort - Example
Merge Sort - Analysis
T(n)= T(n/2) + T(n/2) + n (if n>1)
=1 (if n=1)

T(n)= 2T(n/2) + n
T(n/2)=2T(n/4)+n/2
T(n)=2[2T(n/4)+n/2 ]+n
T(n)=22T(n/22)+2n
T(n)=2kT(n/2k)+kn (kth step)
Recursion will stop if n/2k =1 then T(1)=1
T(n)=2kT(1) + kn
T(n)=n+ (log2n)n
T(n)= O(nlog2n)
 Quick Sort
 The quick sort algorithm partitions the original array by rearranging
it into two groups. The first group contains those elements less than
some chosen value taken from the set, and the second group
contains those elements greater than or equal to the chosen value.
 The chosen value is known as the pivot element. Once the array has
been rearranged in this way with respect to the pivot, the same
partitioning is recursively applied to each of the two subsets. When
all the subsets have been partitioned and rearranged, the original
array is sorted.

 A general algorithm based on a recursive approach as follows:

1. Partition the current array into two parts
2. Invoke the quick sort to sort the left sub array.
3. Invoke the quick sort to sort the right sub array.
 Quick Sort
Algorithm QUICKSORT(a[ ], lb, ub)
{
if (lb < ub) then
{
j := PARTITION(a, lb, ub); // J is the position of the partitioning element
Call QUICKSORT(lb, j – 1);
Call QUICKSORT(j + 1 , ub);
}
}
Algorithm INTERCHANGE(a[ ], i, j)
{
t:=a[i];
a[i] := a[j];
a[j] := t;
}
 Quick Sort
Algorithm PARTITION(a, lb, ub)
{ p:=a(lb); i :=lb; j:= ub;
Repeat
{
Repeat
i := i + 1
until a(i) >p
Repeat
j := j – 1
until a(j) < p
if (i < j) then INTERCHANGE(a, i, j)
} Until (i > j);
a[lb] :=a[j]; a[j] :=p;
return j;
}
 1 2 3 4 5 6 7 8 9 10 11 12 13 Remarks
(38 08 16 06 79 57 24 56 02 58 04 70 45)
pivot i j swap i & j
04 79
i j swap i & j
02 57
j i
(24 08 16 06 04 02) 38 (56 57 58 79 70 45) swap pivot
&j
pivot j, i swap pivot
&j
(02 08 16 06 04) 24
pivot, i swap pivot
j &j
02 (08 16 06 04)
pivot i j swap i & j
04 16
j i
(06 04) 08 (16) swap pivot
&j
pivot, i
j

(04) 06 swap pivot

&j
04
pivot,
j, i
16
pivot,
j, i
(02 04 06 08 16 24) 38
(56 57 58 79 70 45)
 (56 57 58 79 70 45)

pivot i j swap i & j

45 57
j i
(45) 56 swap pivot
&j
(58 79 70 57)
45 swap pivot
pivot,
j, i &j

swap i & j
(58 79 57)
pivot i 70 j
57 79
j i
(57) 58 (70 79) swap pivot
&j
57
pivot,
j, i

(70 79)
swap pivot
pivot, j &j
i
70

79
pivot,
j, i
(45 56 57 58 70 79)
02 04 06 08 16 24 38 45 56 57 58 70 79
 Quick Sort - Analysis
• T(n) is the total time required to sort the n elements using quick sort .
• T(n) = P(n) + T(j-lb) + T(ub-j)
• P(n) is the time for partitioning the array into two parts and T(j-lb) is the time
required to sort the left sub array and T(ub-j) is the time to sort the right sub
array.
• The time to partition the array P(n) is equal to O(n).

• Worst Case: The worst case occurs when at each invocation, the array is
partitioned into two sub tables such that one of them is empty. i.e j=lb or j=ub
T(n) = P(n) + T(0) + T(n-1)
= c*n + T(n-1)
= c*n + c*(n-1) + T(n-2)
= c*n + c*(n-1) + c*(n-2) + T(n-3)
= c*n + c*(n-1) + c*(n-2) + ..........+ c*1 +T(0) and so on
= c( n+ (n-1) + (n-2) + (n-3) + .....+ 1) + 0
= c (n(n+1)/2) = c(n2 + n)/2
T(n) = O(n2 )
 Quick Sort - Analysis
• Best Case: The best case occurs when at each invocation, the array is partitioned
into two equal arrays . i.e j = (lb +ub)/2
T(n) = P(n) + T(j-lb) + T(ub-j)
= c*n + 2T(n/2)
= c*n + 2[c*(n/2) + 2 T(n/4)]
= 2cn + 4T(n/4)
= 2cn + 4[c* n/4) +2T(n/8)]
= 3cn + 8T(n/8)
= 3cn + 23 T(n/23)
= 4cn + 24 T(n/24) and so on
substitute k in place of 4
= kcn + 2kT(n/2k)
If n is a power of 2 then substitute 2k = n
= log n *cn + n T(n/n)
= n(c*log n +1)
T(n) = Ω(nlogn)

• For average case also the complexity is θ(nlog n)

 Strassen’s Matrix Multiplication
 Traditional way of multiplying 2 matrices is as below:
 Algorithm Multiplication(A,B,C,n)
{
for i:=1 to n do
for j:=1 to n do
C[i,j]:=0
for i:=1 to n do
for j:=1 to n do
for k:=1 to n do
C[i,j]:=C[i,j] + A[i,k] * B[k,j]
}
T(n)= O(n3)
 Apply Divide-and-Conquer technique:
1. Assume that n is a power of 2. n=2k i.e Square Matrix
2. Divide the two matrices A and B into four equal matrices of size
n n
 2  2 

C11 C12   A11 A12   B11 B12 

 C  A  B 
     
C 21 C 22   A21 A22   B21 B22 

C11  A11  B11  A12  B21

C12  A11  B12  A12  B22
C 21  A21  B11  A22  B21
C 22  A21  B12  A22  B22
If n=2 then directly perform multiplication. If n>2 then again divide into four parts.
Repeat this process until the size is 2 X 2.
51
Example:
1 1 1 1 1 1
1 1 1  1 1 1
   
1 1 1 1 1 1
use Divide-and-Conquer way to solve it as follows:
1 1 1 0 1 1 1 0 3 3 3 0
1 1 1 0 1 1 1 0 3 3 3 0
  
1 1 1 0 1 1 1 0 3 3 3 0
     
0 0 0 0  0 0 0 0  0 0 0 0

1 1 1 1 1 0 1 1 3 3
C11          
1 1 1 1 1 0 0 0 3 3
1 1 1 0 1 0 1 0 3 0
C12      
1 1 1 0 1 0 0 0 3 0
1 1 1 1 1 0 1 1 3 3
C 21       
0 0 1 1 0 0 0 0 0 0
  

1 1 1 0 1 0 1 0 3 0
C 22        
0 0 1 0 0 0 0 0 0 0
 

52
The Time Complexity :
In order to compute the product, we need 8 multiplications of (n/2 X n/2)
matrices and 4 additions of (n/2 X n/2) matrices. Generally for addition , it is
O(n2).
If the matrix multiplications are done recursively then the running time is:
T(n) = 8T(n/2) + O(n2)
Recurrence Theorem of equation:-
T(n)= a T(n/b) + O(nk) a>=1 & b>1
= O(nlogba ) if a>bk
= O(nk logn) if a=bk
= O(nk) if a<bk So , T(n) = O(n3)
so, there is no improvement compared with the traditional method .
Strassen’s had derived some equations called strassen’s equations and
these are applied after Dividing the problem into sub matrices.
53
Strassen’s matrix multiplication equations:
P  ( A11  A22 )( B11  B22 )
Q  ( A21  A22 ) B11
R  A11 ( B12  B22 )
S  A22 ( B21  B11 )
T  ( A11  A12 ) B22
U  ( A21  A11 )( B11  B12 )
V  ( A12  A22 )( B21  B22 )
C11  P  S  T  V
C12  R  T
C 21  Q  S
C 22  P  R  Q  U

Here it requires 7 multiplications of n/2 X n/2 size matrices,

18 additions and subtractions.
54
 Therefore T(n)=7T(n/2) + O(n2)

As per the Recurrence relation:-

T(n)= a T(n/b) + O(nk) a>=1 & b>1

= O(nlogba ) if a>bk
= 7 > 22
= O(nlog27 )

Therefore T(n) = O(n2.81)

This is an improvement compared with the general method

55
56