Design & Analysis of Algorithms:

An Introduction

Dr. Muhammad Sarwar Ehsan

[email protected]

Department of Computer Science,

University Central of Punjab

September 29, 2025

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 1 / 42

Outline

1 Introduction
Course Outcomes

2 Course Outline
Books

3 Schedule
4 Grading Policy
5 What is an Algorithm?
6 Analysis of Algorithms

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 2 / 42

Introduction
Instructor
Name: Prof. Dr. Muhammad Sarwar Ehsan
Professor / Associate Dean
email: [email protected]
Office: Building A (F-201 Cabin # 03)
Teaching (Undergrad): DAA, Probability and Statistics,
Signals & Systems, Digital Signal Processing
Teaching (Grad): Stochastic Processes, Advanced Digital
Signal processing, Multimedia Systems
Research Interests: Multimedia Processing & Artifiacial
Intelligence, Information Theory

Course
CSAL 3233: Design & Analysis of Algorithms
Course Learning Outcomes

CLO1: Analyze the time complexity of different algorithms

using asymptotic notation. [C4- Synthesize]
CLO2: Apply different strategies (brute-force, greedy, divide
and conquer, and dynamic programming) to solve a
computing problem . [C3-Application]
CLO3: Solve problems using graph algorithms, including
single-source, Spanning Tree, and Graph traversing
algorithms. [C3-Application]

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 4 / 42

Studying at the University

You have an opportunity to learn on the lectures. I can not

force you to learn.
You have more responsibility on your learning than you had
in vocational school or senior high school.
1 cr ≈ 30 hours of work. 3 cr = 90 hours of work. You will
spend 45 hours on the lectures.
Which means that you should use about 45 hours of your
own time for studying!
If I proceed too fast or too slow, please interject me (or tell
me by email).
Do not hesitate to ask.

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 5 / 42

Course Outline

Introduction to the course and course objectives

What is an algorithm? Fundamental of the algorithmic
problem solving, important problem types fundamental data
structures
Fundamentals of the analysis of algorithm efficiency
Brute force and exhaustive search
Divide-and-Conquer
Dynamic Programming
Greedy Techniques
Graph algorithms
Different complexity classes (P, N P, N P -complete)

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 6 / 42

Text & Reference Books

Text Book
T. H. Corman, C. E. Leiserson, R. . Rivest and C. Stein,
Introduction to Algorithms. 4th Edition, MIT Press, 2009.

Reference Books
M. T. Goodrich, and R. Tamassia: Algorithm Design and
Applications. Wiley Inc., 2015.
R. Sedgewick and K. Wayne: Algorithms. Addison-Wesley,
4th edition, 2011.
S. Dasgupta, C. Papadimitriou and U. Vazirani:
Algorithms. McGraw-Hill, 2006.

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 7 / 42

Lecture Schedule

Section E1
Tuesday 12:00 - 13:55 A - 112
Thursday 16:00 - 16:55 A - 105

Section E7
Monday 13:00 - 14:55 A - 103
Thursday 13:00 - 13:55 A - 117

Office Hours (Building A F201 cabin # 3)

▶ Monday 11:00 - 12:00
▶ Tuesday 10:00 - 11:00
▶ Wednesday 12:00 - 13:00
▶ Thursday 11:00 - 12:00

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 8 / 42

Grading Policy

Quizzes: 15%
Assignments: 10%
Class Participation: (5%)
Mid Term: 20%
Final: 50%

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 9 / 42

Important Note

No body will be allowed to enter the class after the

arrival of the instructor
No relaxation in attendance, once it is marked in the
class
There will be a participatory work in every class
Dates for quizzes are tentative. There will be surprise
quizzes

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 10 / 42

What is an Algorithm?

Algorithm
▶ Named after, Muhammad ibn Musa al-Khwarizmi
Definition
▶ An algorithm is a definite procedure for solving a problem in
finite number of steps
▶ Algorithm is a well defined computational procedure that
takes some value (s) as input, and produces some value (s) as
output.
▶ Algorithm is finite number of computational statements that
transform input into the output

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 11 / 42

Properties of Algorithms

Has input values zero or more inputs

Produces output values at least one output
Definiteness clear, precise, and unambiguous
Finiteness terminate after a finite number of steps
Effectiveness Each individual step must be basic, feasible,
and doable in finite time
Generality general class of problems

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 12 / 42

Euclid’s algorithm for greatest
common divisor

Second Algorithm
First Algorithm 1: A ← First integer
1: A ← First integer 2: B ← Second integer
2: B ← Second integer 3: while (1) do
3: while (B ̸= 0) do 4: REM ← A%B
4: while (A > B) do 5: if (REM == 0) then
5: A←A−B 6: Break
6: end while 7: end if
7: B ←B−A 8: A←B
8: end while 9: B ← REM
9: return A 10: end while
11: return B

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 13 / 42

Algorithm – Examples

The rules of how to play a game

Directions for driving from A to B
Finding the largest number in a list of numbers
Finding greatest common divisor between two numbers
Matrix Multiplication
The Traveling Salesman Problem
Clique Problem
Knapsack Problem

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 14 / 42

What kinds of problems are solved by
algorithms?

The Human Genome Project

The internet quickly access and retrieve large amounts of
information
Electronic commerce
Resource allocation
Shortest path search
Data compression
Machine learning

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 15 / 42

Important Designing Techniques

Brute Force: Straightforward, naive approach–Mostly

expensive
Divide-and-Conquer: Divide into smaller sub-problems, e.g.,
merge sort
Greedy Approach: Select the best option in a given scenario
Dynamic programming: Programming–Dependent
sub-problems, reuse results

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 16 / 42

Analysis of Algorithms

Two essential approaches to measuring algorithm efficiency:

Empirical analysis:
▶ Program the algorithm and measure its running time on
example instances
Theoretical analysis
▶ Employ mathematical techniques to derive a function which
relates the running time to the size of instance
In this course our focus will be on Theoretical
Analysis

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 17 / 42

Analysis of Algorithms

Worst case: The worst outcome for any possible input We

often concentrate on this for our analysis as it
provides a clear upper bound of resources an absolute
guarantee
Average case: Measures performance over the entire set of
possible instances
Very useful, but treat with care: what is
“average”?
Random (equally likely) inputs vs. real-life
inputs
Best Case: The best outcome for any possible input
provides lower bound of resources

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 18 / 42

Empirical Analysis

Write a program implementing the algorithm

Run the program with inputs of varying size and
compositions
Use timing routines to get an accurate measure of the actual
running time, e.g., System.currentTimeMillis()
Implementation dependent
Platform dependent
Data dependent

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 19 / 42

Theoretical Analysis

Data independent
Platform independent
Language independent
Implementation independent
Characterizes running time as a function of input size, n.
Easy to extrapolate without risk

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 20 / 42

Why Algorithm Analysis is Important?
Algorithm analysis is an important part of computational
complexity theory, which provides theoretical estimation for
the required resources of an algorithm to solve a specific
computational problem
To predict the behavior of an algorithm without
implementing it on a specific computer
It is much more convenient to have simple measures for the
efficiency of an algorithm than to implement the algorithm
and test the efficiency every time a certain parameter in the
underlying computer system changes
It is impossible to predict the exact behavior of an algorithm.
There are too many influencing factors
The analysis is thus only an approximation; it is not perfect
More importantly, by analyzing different algorithms, we can
compare them to determine the best one for our purpose
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 21 / 42
What do we analyze about
Algorithms?
Algorithms are analyzed to understand their behavior and to
improve them if possible
Correctness
▶ Does the input/output relation match algorithm
requirement?
Amount of work done
▶ Basic operations to do task
Amount of space used
▶ Memory used
Simplicity, clarity
▶ Verification and implementation
Optimality
▶ Is it impossible to do better?
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 22 / 42
Problem Solving Process

Problem
Strategy
Algorithm
▶ Input
▶ Output
▶ Steps
Analysis
▶ Correctness
▶ Time & space optimality
Implementation
Verification

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 23 / 42

Computation Model for Analysis
To analyze an algorithm is to determine the amount of
resources necessary to execute it. These resources include
computational time, memory and communication bandwidth
Analysis of the algorithm is performed with respect to a
computational model called RAM (Random-Access
Machine)
A RAM is an idealized uni-processor machine with an infinite
large random-access memory
▶ Instruction are executed one by one
▶ All memory equally expensive to access
▶ No concurrent operations
▶ Constant word size
▶ All basic operations take unit time
▶ Does not account for the memory hierarchy such as cache or
virtual memory
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 24 / 42
Complexity of an Algorithm

The complexity of an algorithm is the amount of work the

algorithm performs to complete its task. It is the level of
difficulty in solving mathematically posed problems as
measured by:
▶ Time (time complexity)
▶ No. of steps or arithmetic operations (computational
complexity)
▶ Memory space required (space complexity)
Complexity is a function T (n) which yields the time (or
space) required to execute the algorithm of a problem of size
“n”

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 25 / 42

Pseudo-code
High-level description of an algorithm
More structured than English prose but Less detailed than a
program
Preferred notation for describing algorithms
Hides program design issues
Example
1: i ← 10
2: if i ≥ 5 then
3: i←i−1
4: else
5: if i ≤ 3 then
6: i←i+2
7: end if
8: end if
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 26 / 42
Elementary Operation

An elementary operation is an operation which takes

constant time regardless of problem size
Example of elementary operations include
▶ variable assignmen
▶ arithmetic operations (+, −, ×, /) on integer
▶ comparison operations (a < b)
▶ boolean operation
▶ accessing an element of an array
We will measure number of steps taken in term of size of
input

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 27 / 42

Components of an Algorithm

Variables and values

Instructions
Sequences
Selections
Repetitions

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 28 / 42

Instruction and Sequence

A linear sequence of elementary operations is also performed

in constant time
More generally, given two program fragments P1 and P2
which run sequentially in times t1 and t2
▶ use the maximum rule which states that the larger time
dominates
▶ complexity will be max(t1 , t2 )

Example
1: x←a ▷1
2: x ← a + b ∗ c/h − u ▷1
3: a>b ▷1
T (n) = 1 + 1 + 1 = 3

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 29 / 42

Selection
If < test > then P1 else P2 structures are a little harder;
conditional loops
The maximum rule can be applied here too: max(t1 , t2 ),
assuming t1 , t2 are times of P1 , P2

Example
1: if i ≥ maxval then
2: i←0
3: else
4: if i + k ≤ maxval then
5: i←i+k
6: end if
7: end if
T (n) = 1 + max(2, 1) = 3
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 30 / 42
Repetition (Loops)
Analyzing loops: Any loop has two parts:
▶ How many iterations are performed?
▶ How many steps per iteration?
Running time of a for-loop is at most the running time of the
statements inside the for-loop times number of iterations
This approach is reasonable, provided n is positive
For nested loops time is mn

Example
1: for <1 to n> do
2: <do stuff>
3: end for
T (n) = nt

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 31 / 42

Analysis Example

Example
1: n←0 ▷1
2: sum ← 0 ▷1
3: i←0 ▷1
4: while i < n do ▷n
5: number ← read input from user ▷ n−1
6: sum ← sum + number ▷ n−1
7: i←i+1 ▷ n−1
8: end while
9: mean ← sum/n ▷1

The computing time for this algorithm is

T (n) = 1 + 1 + 1 + n + (n − 1) + (n − 1) + (n − 1) + 1 = 4n + 1

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 32 / 42

Asymptotic Growth Rate

Changing the hardware/software environment

▶ Affects T (n) by constant factor, but does not alter the
growth rate of T (n)
The growth of the complexity functions is what is more
important for the analysis and is a suitable measure for the
comparison of algorithms with increasing input size n
Asymptotic notations are used to compute the complexity
because different implementations of algorithm may differ in
efficiency
Helps to determine best-case, worst-case and average case
behavior

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 33 / 42

Asymptotic Growth Rate

Two reasons why we are interested in asymptotic growth

rates
Practical purposes: For large problems, when we expect to
have big computational requirements
Theoretical purposes: concentrating on growth rates frees
us from some important issues
▶ fixed costs (e.g. switching the computer on!), which may
dominate for a small problem size but be largely irrelevant
▶ machine and implementation details
▶ The growth rate will be a compact and easy to
understand the function

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 34 / 42

Example: Linear Search Algorithm
Given a linear array A containing n elements, locate the
position of an Item ‘x’
Comparing ‘x’, one by one, with each element in A.
1: LinearSearch (A, x)
2: i←1 ▷1
3: n ← Sizeof Array ▷1
4: while (i ≤ n) do ▷ n+1
5: if (A[i] == x) then ▷n
6: return TRUE ▷1
7: else
8: i←i+1 ▷ n−1
9: end if
10: return FALSE ▷1
11: end while
T (n) = 3n + 4
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 35 / 42
Example: Linear Search Algorithm
Best Case: ‘x’ is located in the first position of the array A

T (n) = 1 + 1 + 1 + 1 + 1
= 5
= O(1)

Worst Case: ‘x’ is located in last location of the array or is

not there at all

T (n) = 1 + (n + 1) + n + (n − 1) + 1 + 1 + 1
= 3n + 4
= O(n)
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 36 / 42
 Average Case: Assume that it is equally likely for ‘x’ to
appear at any position in array A,. Accordingly, the number
of comparisons can be any of the numbers 1, 2, 3, · · · , n, and
each number occurs with probability p = 1/n.

1 1 1
T (n) = 1 · + 2 · + ··· + n ·
n n n
1
= (1 + 2 + 3 + · · · + n) ·
n
n(n + 1) 1
= ·
2 n
n+1
=
2
= O(n)

This agrees with our intuitive feeling that the average number of
comparisons needed to find the location of ‘x’ is approximately
equal to half the number of elements in the list A
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 37 / 42
Example: Iterative Binary Search

1: while low ≤ high do

2: mid = (low + high)/2
3: if (x == arr[mid]) then
4: return mid
5: else if (x > arr[mid]) then
6: low = mid + 1
7: else
8: high = mid − 1
9: end if
10: end while

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 38 / 42

Example: Recursive Binary Search

1: function binarySearch(arr, x, low, high)

2: if low > high then
3: return F alse
4: else
5: mid = (low + high)/2
6: if x == arr[mid] then
7: return mid
8: else if x > arr[mid] then
9: return binarySearch(arr, x, mid + 1, high)
10: else
11: return binarySearch(arr, x, low, mid − 1)
12: end if
13: end if
14: end function
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 39 / 42
Operation Count Example # 1
int x = 1, y = 10, z = 20;

for (int x = 1; x < N; x = x + 4)

for (int y = 1; y < M; y = y * 4)

Task a;

c1 · 1
c2 · (⌈ N4−1 ⌉ + 1) ≈ c2 N4
c3 · (⌈ N4−1 ⌉) · (⌈log4 M ⌉ + 1) ≈ N
4
log4 M
c4 · (⌈ N4−1 ⌉) · (⌈log4 M ⌉) ≈ N
4
log4 M
T (n) = c1 + c2 N4 + c3 N4 log4 M + c4 N4 log4 M
= O(N log4 M )
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 40 / 42
Operation Count Example # 2
for (i = 0; i <= N; i = i + 2){

for (j = 1; j <= 2 * i; j = j + 2){

a = a * 2 - 3;

⌊ N2 ⌋ + 2 ≈ N
2
N2
(⌊ N2 ⌋ + 1)( N2 + 1) = N +2
2
· N +2
2
≈N+ 4
N2
(⌊ N2 ⌋ + 1)( N2 ) ≈ 4
N N2 N2
T (n) = 2
+N + 4
+ 4
= O(N 2 )
Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 41 / 42
 for (int x = 1; x < N; x = x + 2)
{
for (int y = 1; y < M; y = y * 3)
{
a = a + b;
}
b = a + b;
}
c1 · (⌊ N2 ⌋ + 1) ≈ N
2

c2 · ⌊ N2 ⌋ · (⌈log3 M ⌉ + 1) ≈ N
2
log3 M
c3 · ⌊ N2 ⌋ · ⌈log3 M ⌉ ≈ N
2
log3 M
N
c4 · 2
T (n) = c1 ( N2 ) + c2 N2 (log3 M ) + c3 N2 (log3 M ) + c4 N2
= O(N log3 M )

Dr. M. Sarwar Ehsan (FoIT&CS) DAA: An Introduction September 29, 2025 42 / 42