Computational Complexity
Chnoor M. Rahman
Spring 2023
A priori analysis and A posteriori analysis
• A priori analysis of algorithms
• It means we do analysis (space and time) of an algorithm prior to running it
on a specific system.
• That is, we determine time and space complexity of algorithm by just seeing
the algorithm rather than running it on a particular system (with different
processor and compiler).
• A posteriori analysis of algorithms
• It means we analyze the algorithm only after running it on a system.
• It directly depends on the system and it changes from system to system.
A priori analysis
• Algorithms
• Independent of language
• Hardware independent
• Time and Space function
• Results do not change
Posteriori Testing
• Program
• Language dependent
• Hardware dependent
• Results might not be the same
Analyzing algorithms
To analyze algorithms, their complexity should be calculates.
The most popular technique for computing complexity of algorithms
is Big oh notation.
Computational Complexity
• Computational complexity is a continuum, in that some algorithms require
linear time (that is, the time required increases directly with the number of
items or nodes in the list, graph, or network being processed).
an algorithm is a series of contained steps, which you follow in order to
achieve some goal, or to produce some output.
• Another group of algorithms require quadratic or even exponential time to
complete (that is, the time required increases with the number of items
squared or with the exponential of that number).
7
Big O notation
• Time complexity analysis in programming is just an extremely simplified
mathematical way of analyzing how long an algorithm with a given number of
inputs (n) will take to complete it’s task. It’s usually defined using Big-O notation.
It tells you the growth of an algorithm
• Big O Notation in Data Structure tells us how well an algorithm will perform in a
particular situation.
8
Big O notation
Assume we have the following program:
Instead of:
How much time does it take to run
array = [2, 3, 4, 5, ……, 8] this function? This depends on the
type of the machine
int findSum(array){
int total = 0; Use:
for(int i=0, i<array.lemgth; i++) How does the run time of this
function grow?
totatl+=i;
return total; To answer this use:
} Big O notation
9
The general steps for Big-O runtime analysis are as
follows:
1.Figure out what the input is and what n represents.
2.Express the maximum number of operations, the algorithm performs
in terms of n.
3.Eliminate all excluding the highest order terms.
4.Remove all the constant factors.
10
Eliminate all excluding the highest order terms
Regular Big-O
2 O(1) --> It's just a constant number
2n + 10 O(n) --> n has the largest effect
5n^2 O(n^2) --> n^2 has the largest effect
11
Common Time complexities
1. O(1) — Constant Time: Given an input of size n, it only takes a single step for
the algorithm to accomplish the task.
2. O(log n) — Logarithmic time: given an input of size n, the number of steps it
takes to accomplish the task are decreased by some factor with each step.
3. O(n) — Linear Time: Given an input of size n, the number of of steps required
is directly related (1 to 1)
12
Common Time complexities cont..
4. O(n²) — Quadratic Time (polynomial): Given an input of size n, the number of
steps it takes to accomplish a task is square of n.
5. O(C^n) — Exponential Time: Given an input of size n, the number of steps it
takes to accomplish a task is a constant to the n power (pretty large number).
13
14
Example:
let n = 16;
O (1) = 1 step "(awesome!)"
O (log n) = 4 steps "(awesome!)" -- assumed base 2
O (n) = 16 steps "(pretty good!)"
O(n^2) = 256 steps "(uhh..we can work with this)"
O(2^n) = 65,536 steps "(...)“ (an n increases by 1 -> count doubles roughly by 2)
15
Big O Analysis
Required time
No. of inputs
16
Example
algorithm change_Position (X , Y){
Store:=X; (1)
X:=Y; (1)
Y:=Store; (1)
}
F(n) =3
Time Complexity = O(1)
O(1) – Example
//If I know the persons name, I only have to take one step to check:
function isFriend(name){ //similar to knowing the index in an Array
return friends[name]; (1)
}
isFriend('Mark’) // returns True and only took one step
F(n) = 1; Time Complexity: O(1)
________________________________________________________________________
function add(num1,num2){ // I have two numbers, takes one step to return the value
return num1 + num2; (1)
} Time Complexity: O(1)
18
O(1) – Example
void constantTimeComplexity(int arr[])
{
printf("First element of array = %d",arr[0]);
}
Answer: O(1)
Here, the input array could be 1 item or 1,000 items, but this function 8istill just
require one step.
19
Example
For (i=1; i<n;i=i*2){ i
Statement; ---
} 1*2=2
2*2=4
O(log2n) 4*2=8
8*2=16
Any time the loop is increased by 16*2=32
multiplication then the time complexity is 32*2=64
O(log2n) .
2^k
O(log n) - Example
//You decrease the amount of work you have to do with each step
function thisOld(num, array){
var midPoint = Math.floor( array.length /2 );
if( array[midPoint] === num) return true;
if( array[midPoint] < num ) --> only look at second half of the array
if( array[midpoint] > num ) --> only look at first half of the array
//recursively repeat until you get the solution
}
When the input is divided with each iteration, it’s O(log n). Example: Binary Search
21
O(n) – Example
//The number of steps you take is directly correlated to the input size
function addAges(array){
var sum = 0;
for (let i=0 ; i < array.length; i++){ //has to go through each value
sum += array[i]
}
return sum;
}
22
O(n) – Example
void linearTimeComplexity(int arr[], int size)
{
for (int i = 0; i < size; i++)
{
printf("%d\n", arr[i]);
}
}
Answer: O(n)
This function runs in O(n) time (or "linear time"), where n is the number of items in the
array. If the array has 10 items, we have to print 10 times. If it has 1000 items, we have to
print 1000 times.
23
O(n²) – Example 1 Here we're nesting two loops. If
our array has n items, our outer
function addedAges(array){ loop runs n times, and our inner
var addedAge = 0; loop runs n times for each iteration
for (let i=0 ; i < array.length; i++){ of the outer loop, giving us n^2
for(let j=0 ; j < array.length ; j++){ total prints. If the array has 10
addedAge += array[i][j];
items, we have to print 100 times.
If it has 1000 items, we have to
}
print 1000000 times. Thus this
} function runs in O(n^2) time (or
return addedAge; "quadratic time").
}
Note: If one for loop is linear time (n) Then two nested for loops are (n * n) or (n^2)
Quadratic!
24
O(n²) – Example 2
void quadraticTimeComplexity(int arr[], int size)
{
for (int i = 0; i < size; i++)
{
for (int j = 0; j < size; j++)
{
printf("%d = %d\n", arr[i], arr[j]);
}
}
}
Answer: O(n^2)
25
O(2^n) – Example 1
The number of steps it takes to accomplish a task is a constant to the n power. For
example, trying to find every combination of letters for a password of length n.
26
O(2^n) – Example 2
int fibonacci(int num)
{
if (num <= 1) return num;
return fibonacci(num - 2) + fibonacci(num - 1);
}
Answer: O(2^n)
An example of an O(2^n) function is the recursive calculation of Fibonacci
numbers. O(2^n) denotes an algorithm whose growth doubles with each addition to
the input data set. The growth curve of an O(2^n) function is exponential - starting
off very shallow, then rising meteorically.
27
Example Since
A is two dimension array, B is two
Algorithm Sum(X,Y,n) dimension array and C is two dimension
array
{
Then
for (i=0;i<n;i++) X n^2
{ Y n^2
for(j=0;j<n;j++) Z n^2
{ n 1
i 1
Z[i, j]=X[I,j]+Y[I,j];
j 1
}
} f(n)=3 n^2+3
} O(n^2)
Example
For (i=1, i<n; i=i+20)
{
statement; n/20
}
--------
f(n)=n/20
Example
Algorithm Sum(X,Y,n)
{
for (i=0;i<n;i++) n+1
{
for(j=0;j<n;j++) n * (n+1)
{
Z[i, j]=X[I,j]+Y[I,j]; n * n
} ---------
} 2n^2+ 2n+1
} f(n)=(n^2)
O(n^2)
Example
For (i=0, i<n; i++) n+1
{
statement; n
}
_____________
O(n)
Example
For (i=n, i>0; i--) n+1
{
statement; n
}
_____________
O(n)
Example
Algorithm Adding (X , n)
{
a=0 ; 1
for( i=0 ; i<n ; i++) n+1
{
a= a+ X[i]; n
} 1
return a;
f(n)=2n +3
O(n)=n
}
Example
public static void main(String[] args){
int a = 0, b = 0;
int N = 5, M = 5;
for (int i = 0; i < N; i++)
a += 5;
for (int i = 0; i < M; i++)
b += 10;
System.out.println(a + " " + b);
}
34
More Examples:
Logarithmic algorithm – O(logn) – Binary Search.
Linear algorithm – O(n) – Linear Search.
Superlinear algorithm – O(nlogn) – Heap Sort, Merge Sort.
Polynomial algorithm – O(n^c) – Strassen’s Matrix Multiplication, Bubble Sort,
Selection Sort, Insertion Sort, Bucket Sort.
Exponential algorithm – O(c^n) – Tower of Hanoi.
Factorial algorithm – O(n!) – Determinant Expansion by Minors, Brute force
Search algorithm for Traveling Salesman Problem.
35
Example - HW
int f(int n){
If(n==1)
return 1;
Else
return f(n-1) + f(n-1);
}
36
References
• https://www.freecodecamp.org/news/time-is-complex-but-priceless-f0abd01506
3c/#:~:text=O(n%C2%B2)%20%E2%80%94%20Quadratic%20Time%3A%20The%2
0number%20of%20steps,power%20(pretty%20large%20number)
.
• https://www.vegaitglobal.com/media-center/knowledge-base/fundamental-data
-structures-computational-complexity
• Data Structures and algorithms – 4th Edition – Chapter 4
37
