Space and Time Complexity

Space and Time Complexity

• To analyze an algorithm means:

• developing a formula for predicting how fast an algorithm is, based on the
size of the input (time complexity),
and/or
• developing a formula for predicting how much memory an algorithm requires,
based on the size of the input (space complexity).

• Usually, time is our biggest concern

• Most algorithms require a fixed amount of space

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Size of the input ?

Problem Size of the Input

Searching an element in an n
array of size n
Sorting n elements n
Merging two arrays Sum of size of two array
Adding two matrices of order m and n
mxn
Multiplying two matrices or m, n and p
order m x n and n x p

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Space Complexity
Depends on
1. Data Space:
• Space needed by the constants and simple variables
• Space needed by dynamically allocated objects

2. Environmental Stack Space: used to save information needed to resume execution of

partially completed methods and functions.
• Example: if function f() invokes function g(), then we must save at least a pointer to the instruction of
f() to be executed when g() terminates.

3. Instruction Space: Space needed to store the compiled version of the program instruction
• Compiler Dependent

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Space Complexity Contd…

The space requirement of any program P can be written as:

• S = C + Sp(size of the input), where c denotes fixed part and Sp denotes the
variable component

• We are interested in Variable Component.

03/23/2022 ICT 2257 Design and Analysis of Algorithms

In C++ data size

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Space Complexity Example: Sequential or Linear Search
Int SequentialSearch(int a[], int n, const int& x)
{
int i;
for (i = 0; i < n && a[i] != x; i++);
if (i == n) return -1;
else return i;
}
Space needed by parameters a, n and x = 4 + 4 + 4 = 12 bytes (integer type)
Space needed by variable i = 4 bytes
Space needed by constants 0 and -1 = 4 + 4 = 8 bytes
Total Space needed = 24 bytes (Constant ) therefore, Sp(n)=0
Complexity
03/23/2022
= O(1) ICT 2257 Design and Analysis of Algorithms
Space Complexity Example: Recursive Sequential Search
Int RSequentialSearch(int a[], int n, const int& Environment Stack Space:
x) RSequentialSearch (a, n, x)
{ RsequentialSearch (a, n-1, x)
…..
if ( n < 1) return -1; ……
if (a[n-1] == x) return n-1; RSequentialSearch(a, 1, x)
return RSquentialSearch(a n-1, x); RSequentialSearch (a, 0, x)
Therefore, depth of recursion: n + 1
}
Each time for recursive calls, a, n, x and return
address stored in stack. Since all are integers,
4 * 4 = 16 bytes needed.
Ignore the constant space needed and find
Environmental stack space. Therefore, total environmental stack space =
16 (n+1) bytes= O(n).
03/23/2022 ICT 2257 Design and Analysis of Algorithms
• Write a C++ function for finding sum of array elements. Write both
iterative and recursive program and find its space complexity.

int sum (int a[], int n) int Rsum (int a[], int n)
{ {
int result; if (n > 0)
for(int i=0; i<n; i++) return Rsum(a, n-1) + a[n-1];
result= result + a[i];
return result; return 0;
} }
Rsum(a, n) Rsum(a, n-1)
….Rsum(a,1), Rsum(a, 0)
therefore, depth= n+1
Sp(n) = 0 constant time
O(1) Sp(n) = 12 (n+1) bytes
03/23/2022
=O(n)
ICT 2257 Design and Analysis of Algorithms
Data Space Examples
i. Int *a;
a = new int [n]; Space needed 4n Bytes (n integers, each need 4 Bytes)

ii. Int a[30]; 4 x 30 = 120 Bytes

iii. int **A;

A = new int * [m];
for(int i=0; i<m; i++)
A[i] = new int [n];

Space allotted for matrix A is 4mn bytes. O(mn) space complexity.

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Time Complexity

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Time complexity
• Operation Count Method
• Step Count Method

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Operation Count Method
• Identify one or more key operations in the algorithm/program and count how
many times each key operation is performed in the algorithm.

• Examples
• Matrix multiplication:
• Key operations are addition and multiplications
• Sorting:
• Key operations are comparison and swapping.
• Searching:
• Comparison

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Matrix Multiplication
void matrixMultiply(int **a, int **b, int **c, int m, int n, int p)
{
// Multiply the m x n matrix a and the n x p matrix b to get c.
for (int i = 0; i < m; i++)
for (int j = 0; j < p; j++)
{
int sum = 0; • Key operations are addition
for (int k = 0; k < n; k++)
and multiplications
sum += a[i][k] * b[k][j]; • Additions mnp times
c[i][j] = sum; • Multiplications mnp times
} • Complexity O(mnp)
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Polynomial Evaluation: p(x)=cnxn+ cn-1xn-1 + . . . + c1 x + c0

double polyEval(double c[], int n, const double& x)

{// Evaluate the degree n polynomial with • No. of additions = n
// coefficients coeff[0:n] at the point x.
• No. of Multiplications = 2n
double y = 1, value = c[0];
for (int i = 1; i <= n; i++) • Complexity = O(n)
{// add in next term
4x^3 + 5x + 3
y *= x;
value += y * c[i]; C3 = 4
C2=0
} C1=5
C0=3
return value;
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Horner’s Rule for Polynomial Evaluation
p(x)=cnxn+ cn-1xn-1 + . . . + c1 x + c0
Normal way: value = c0 , y=1 Horner’s way: value = cn
y=y*x = x value = value *x + cn-1 = cn x + cn-1
value = value + c1y = c1x+ c0 p(x)= value *x + cn-2 = = cn x2 + cn-1 x + cn-2
y=y*x = x2 ……
value = value + c2y = c2x2 + c1x + c0 …..
……
p(x)= cnxn+ cn-1xn-1 + . . . + c1 x + c0
…..

value = cnxn+ cn-1xn-1 + . . . + c1 x + c0

5x3 - 4x2 + 2x + 7 = ((5x – 4)*x + 2)x +7
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Horner’s Rule for Polynomial Evaluation
double horner(double c[], int n, const double& x)
{// Evaluate the degree n polynomial with
// coefficients c[0:n] at the point x. • No. of additions = n
double value = c[n];
for (int i = 1; i <= n; i++) • No. of Multiplications = n
value = value * x + c[n - i];
• Complexity = O(n)
return value;
}

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Rank of an Element in a Sequence
• The rank of an element in a sequence is the number of smaller
elements in the sequence plus the number of equal elements that
appears to its left.
• example: a = [4, 3, 9, 3, 7]
Rank r = [2, 0, 4, 1, 3]

Rank Sort: Sorting elements in an array based on their ranks

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Rank of an Element in a Sequence

void rank(int a[], int n, int r[]) Key Operation: Comparison

{// Rank the n elements a[0:n-1].
// Element ranks returned in r[0:n-1] Number of times a[j] <= a[i] occurs
for (int i = 0; i < n; i++) is
r[i] = 0; // initialize
i=1 j takes the value 0
i=2 j takes the value 0, 1
// compare all element pairs
….
for (int i = 1; i < n; i++)
i=n-1 j takes the value 0, 1, …, n-2
for (int j = 0; j < i; j++)
if (a[j] <= a[i]) r[i]++; Total Comparisons = 1 + 2 + … + n-1
else r[j]++; = n (n – 1 )/2
} = O(n2)
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Rank Sort
void rearrange(T a[], int n, int r[])
{
int *u = new int [n]; // create additional array

// move to correct place in u

for (int i = 0; i < n; i++) Finding Rank : O(n2)
u[r[i]] = a[i]; Rearrangement takes O(n) time

// move back to a Total Complexity of Rank Sort = O(n2)

for (i = 0; i < n; i++)
a[i] = u[i];

delete [] u;
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Selection and Bubble Sort

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Selection Sort

void selectionSort(int a[], int n) int indexOfMax(int a[], int n)

{// Sort the n elements a[0:n-1]. {// Locate the largest element in a[0:n-1].
int indexOfMax = 0;
for (int size = n; size > 1; size--) for (int i = 1; i < n; i++)
{ {
int j = indexOfMax(a, size); if (a[indexOfMax] < a[i])
swap(a[j], a[size - 1]); indexOfMax = i;
} }
} return indexOfMax;
}
0 1 n-1
a=
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Selection Sort
void selectionSort(int a[], int n)
{ Swap operation (n – 1) times
for (int size = n; size > 1; size--) Comparisons:
{ when size = n , k = 1, 2, … , n-1
int j = 0, temp; when size = n-1, k = 1, 2, …,, n-2
for (int k = 1; k < size; k++) …..
{ ……
if (a[j] < a[k]) when size = 2, k= 1
j = k;
} Total comparison = 1 + 2 + … + n-1
temp=a[j]; = n (n-1)/2 = O(n2)
a[j]=a[size-1];
a[size-1]=temp;
}
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Bubble Sort
void bubble(int a[], int n)
void bubbleSort(int a[], int n)
{// Bubble largest element in a[0:n-1] to
right. {
for (int i = 0; i < n - 1; i++) for (int i = n; i > 1; i--)
if (a[i] > a[i+1]) swap(a[i], a[i + 1]); {
}
for (int j = 0; j < i - 1; j++)
{
void bubbleSort(int a[], int n) if (a[j] > a[j+1]) swap(a[j], a[j + 1]);
{// Sort a[0:n - 1] using bubble sort. }
for (int i = n; i > 1; i--)
}
bubble(a, i);
} }
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Insert An Element into Sorted Array
• Given a sorted list/sequence, insert a new element
• Given 3, 6, 9, 14
• Insert 5
• Result 3, 5, 6, 9, 14
Insert an Element into Sorted Array
• 3, 6, 9, 14 insert 5
• Compare new element (5) and last one (14)
• Shift 14 right to get 3, 6, 9, , 14
• Shift 9 right to get 3, 6, , 9, 14
• Shift 6 right to get 3, , 6, 9, 14
• Insert 5 to get 3, 5, 6, 9, 14
 Insert an Element into Sorted Array

void insert(int a[], int n, int x)

{

for (int i = n - 1; i >= 0 && x < a[i]; i--)

a[i+1] = a[i];

a[i+1] = x; // insert x
}
Key operation Comparison Total comparison = n Complexity: O(n)
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Insertion Sort
void insert(int a[], int n, const int& x)
{
for (int i = n-1; i >= 0 && x < a[i]; i--)
a[i+1] = a[i];
a[i+1] = x;
}

void insertionSort(int a[], int n)

{// Sort a[0:n-1] using the insertion sort method.
for (int i = 1; i < n; i++)
{ \\ insert a[i] into a[0:i-1]
int t = a[i];
insert(a, i, t);
}
} 03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Insertion Sort
void insertionSort(int a[], int n) i =1, j=0, 1 comparison
{ i=2, j = 0, 1 2 comparisons
for (int i = 1; i < n; i++)
{// insert a[i] into a[0:i-1] ….
int t = a[i]; …..
i = n-1, j= 0, 1,…, n-2 n-1 comparisons
for (int j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t; Total Comparisons = 1 + 2 + …. + (n-1)
} = n(n-1)/2 = O(n2)
}

03/23/2022 ICT 2257 Design and Analysis of Algorithms

Time Complexity : Step Count Method
• A step is an amount of computing that does not depend on the
instance characteristic (input size)

• 10 adds, 100 subtracts, 1000 multiplies can all be counted as a single

step

• n adds or n/2 adds, p+q addition, p-q additions cannot be counted as

1 step

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Complexity of Insertion Sort: Step Count Method

s/e frequency Total Steps

void insertionSort(int a[], int n)
{
for (int i = 1; i < n; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];

for (int j = i - 1; j >= 0 && t < a[j]; j--)

a[j + 1] = a[j];
a[j + 1] = t;
}
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
 Complexity of Insertion Sort: Step Count Method
s/e frequency Total Steps
void insertionSort(int a[], int n) 0 0 0
{ 0 0 0
for (int i = 1; i < n; i++) 1 n n
{// insert a[i] into a[0:i-1]
int t = a[i]; 1 n-1n-1
𝑛 −1

for (int j = i - 1; j >= 0 && t < a[j]; j--) 1 ∑ ( 𝑖+1 -) 1

n(n+1)/2
𝑖 =1
𝑛 −1

a[j + 1] = a[j]; 1 n(n-1)/2 ∑ (𝑖 )

𝑖 =1

a[j + 1] = t; 1 n-1n-1
}
} Total Steps = n2 – 3n – 3 = O(n2)
03/23/2022 ICT 2257 Design and Analysis of Algorithms
void selectionSort(int a[], int n)
{
for (int size = n; size > 1; size--)
{
int j = 0, temp;
for (int k = 1; k < size; k++)
{
if (a[j] < a[k])
j = k;
}
temp=a[j];
a[j]=a[size-1];
a[size-1]=temp;
}
}

03/23/2022 ICT 2257 Design and Analysis of Algorithms

void bubble(int a[], int n)
{// Bubble largest element in a[0:n-1] to right.
for (int i = 0; i < n - 1; i++)
if (a[i] > a[i+1])
{ //swap
temp=a[i];
a[i]=a[i+1];
a[i+1]=temp;
}
}

void bubbleSort(int a[], int n)

{// Sort a[0:n - 1] using bubble sort.
for (int j = n; j > 1; j--)
bubble(a, j);
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Step Count Method: Few more examples

1. int n; \\ assume n as power of 2

while (n>1)
n=n/2;

2. int x=0;
for(int i=0; i<m; i++)
for (int j=0; j<n; j++)
for(int k=0; k<p; k++)
x = x+1;
What is the final value of x? how many times for loop for k will execute?
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Step Count Method: Few more examples

1. double n; \\ assume n =2k and k as power of 2

while (n>=2)
n=sqrt(n);

i. int a = 0;
for (i = 0; i < N; i++) {
for (j = N; j > i; j--) {
a = a + i + j;
}
}
03/23/2022 ICT 2257 Design and Analysis of Algorithms
Reference:
• Sartaj Sahani, Data Structures Algorithms and Applications in C++,
Second Edition, Universities Press (India) Private Ltd, 2005.

03/23/2022 ICT 2257 Design and Analysis of Algorithms

 Thank You

03/23/2022 ICT 2257 Design and Analysis of Algorithms