Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basic Efficiency Classes (DAA) BCA SEP SEM-III

Design and Analysis of Algorithms(DAA) Unit-II BCA Semester III
1
Notes by Dr. Chandrakantha T S, Vagdevi College of BCA, Melinakuruvalli, Thirthahalli-577 432
For more notes and resources, visit: https://sites.google.com/view/chandrakanthats/bca
Unit-2: Asymptotic Notations and Basic Efficiency Classes
Introduction
• When analyzing algorithms, we need a standard way to describe their efficiency
without getting lost in machine-specific details or exact execution times.
• This is done using asymptotic notations, which describe how the running time or space
requirement of an algorithm grows with the size of the input n.
Key Idea:
• Focus on growth rates as n becomes very large.
• Ignore constants and lower-order terms because they have little effect for large
inputs.
Informal Introduction
Imagine two algorithms:
• Algorithm A takes 5n + 10 steps.
• Algorithm B takes n² + 2n steps.
For small n, Algorithm A may be slower than Algorithm B, but for large n, the n² term
dominates Algorithm B, making it slower.
Asymptotic notations help us classify algorithms based on their growth rates, rather than
exact counts.
Big O Notation (O-notation)
Definition:
• Big O notation describes the upper bound of an algorithm’s running time.
• It tells us the worst-case growth rate.
Formal Definition:
An algorithm is O(f(n)) if there exist constants c > 0 and n₀ > 0 such that:
T(n) ≤ c * f(n) for all n ≥ n₀
Example:
• If T(n) = 3n² + 5n + 2, then T(n) = O(n²)
• Reason: For large n, the n² term dominates.
Interpretation:
• Big O gives a safe estimate of maximum time needed by an algorithm

2
Omega Notation (Ω-notation)
Definition:
• Omega notation describes the lower bound of an algorithm’s running time.
• It tells us the best-case growth rate.
Formal Definition:
An algorithm is Ω(f(n)) if there exist constants c > 0 and n₀ > 0 such that:
T(n) ≥ c * f(n) for all n ≥ n₀
Example:
• If T(n) = 3n² + 5n + 2, then T(n) = Ω(n²)
• Reason: The n² term ensures that the algorithm cannot be faster than n²
asymptotically.
Theta Notation (θ-notation)
Definition:
• Theta notation describes the tight bound of an algorithm’s running time.
• It tells us both the upper and lower bounds.
Formal Definition:
An algorithm is θ(f(n)) if there exist constants c₁, c₂ > 0 and n₀ > 0 such that:
c₁ * f(n) ≤ T(n) ≤ c₂ * f(n) for all n ≥ n₀
Example:
• If T(n) = 3n² + 5n + 2, then T(n) = θ(n²)
• Reason: The algorithm grows exactly like n² for large n.
Interpretation:
• Theta gives a precise idea of the growth rate.

3
Basic Efficiency Classes
Algorithms are commonly classified by their growth rates:
Efficiency
Class
Notation Example Characteristics
Constant O(1) Accessing an array
element
Time does not depend on input
size
Logarithmic O(log n) Binary search Time grows slowly with n
Linear O(n) Linear search Time grows proportionally with
n
Linearithmic O(n log
n)
Merge Sort, Quick Sort Combination of linear and
logarithmic growth
Quadratic O(n²) Bubble Sort, Insertion
Sort
Time grows with square of n
Cubic O(n³) Matrix multiplication Time grows with cube of n
Exponential O(2^n) Tower of Hanoi,
recursive Fibonacci
Time doubles with each
additional input element
Factorial O(n!) Permutations Time grows factorially with
input size
Key Points:
• Algorithms with lower growth rates are preferred for large inputs.
• Big O, Omega, and Theta help compare algorithms without machine dependence.
Mathematical Analysis of Non-Recursive Algorithms
Non-recursive algorithms are those that do not call themselves. They typically use loops or
sequential instructions.
Steps for Analysis:
1. Identify Basic Operations:
o Count comparisons, assignments, arithmetic operations, or key steps in the
algorithm.
2. Express Running Time:
o Represent the running time T(n) as a function of input size n.
3. Simplify Using Asymptotic Notation:
o Use Big O, Omega, or Theta to describe efficiency.

4
Example 1: Linear Search
LinearSearch(array, key):
for i = 1 to n:
if array[i] == key:
return i
return -1
Analysis:
• Loop runs at most n times.
• One comparison per iteration → total comparisons ≤ n.
• Worst-case Time Complexity: O(n)
• Best-case Time Complexity: Ω(1)
• Average-case Time Complexity: θ(n/2) ≈ θ(n)
Example 2: Simple Sum of Array
Sum(array):
sum = 0
for i = 1 to n:
sum = sum + array[i]
return sum
Analysis:
• Loop executes n times → n additions.
• Time Complexity = θ(n)
Key Points:
• Non-recursive analysis mainly involves loops and sequential statements.
• Count the number of times each operation executes relative to input size.

5
Mathematical Analysis of Recursive Algorithms
Recursive algorithms call themselves, and their running time depends on the number of
recursive calls.
Steps for Analysis:
1. Formulate a Recurrence Relation:
o Express the total running time T(n) in terms of smaller inputs.
2. Solve the Recurrence:
o Methods:
▪ Substitution method
▪ Recursion tree method
▪ Master theorem
Example 1: Factorial (Simple Recursion)
Factorial(n):
if n == 0 or n == 1:
return 1
else:
return n * Factorial(n-1)
Analysis:
• Recurrence relation:T(n) = T(n-1) + O(1)
• Solve: T(n) = O(n)
• Linear recursion → Time Complexity = O(n)
Example 2: Fibonacci Sequence (Naive Recursion)
Fibonacci(n):
if n <= 1:
return n
else:
return Fibonacci(n-1) + Fibonacci(n-2)
Analysis:
• Recurrence relation:T(n) = T(n-1) + T(n-2) + O(1)
• This grows exponentially, because each call generates two more calls.
• Time Complexity = O(2^n)

6
Observation:
• Recursive algorithms can be inefficient if overlapping subproblems exist.
• Optimization: Use dynamic programming or memoization to reduce repeated
calculations.
Comparison of Mathematical Analysis: Non-Recursive vs Recursive
Algorithms
Non-Recursive Algorithms Recursive Algorithms
1. Do not call themselves; use loops or
sequential statements.
1. Call themselves with smaller input to solve
a problem.
2. Running time is calculated by
counting loops and basic operations.
2. Running time is calculated by forming a
recurrence relation.
3. Easy to analyze; mainly depends on
number of iterations.
3. Analysis is more complex; depends on
number of recursive calls.
4. Example: Linear Search, Array Sum. 4. Example: Factorial, Fibonacci.
5. Time complexity is usually linear,
quadratic, or logarithmic depending on
loops.
5. Time complexity can be linear,
polynomial, or exponential, depending on
recursion structure.
6. Memory usage is mostly constant or
small, depends on variables used.
6. Memory usage includes stack space for
recursive calls, can be large.
Brute Force Method
Definition
The Brute Force method is a simple problem-solving technique in which all possible
solutions are tried one by one until the correct solution is found.
• It is also called the exhaustive search method.
• Brute Force is easy to understand and implement, but may be inefficient for large
inputs.

7
Algorithm: Brute Force Search.
1. Start with the given problem or input.
2. Generate all possible candidate solutions.
3. For each candidate solution, check if it satisfies the problem conditions.
4. If a solution satisfies the condition, return it.
5. If no solution satisfies, report that no solution exists.
Example
Problem: Find an element key in an array of size n.
Brute Force Algorithm: Linear Search
LinearSearch(array, key):
for i = 0 to n-1:
if array[i] == key:
return i // element found
return -1 // element not found
The algorithm checks each element one by one until the key is found.
Analysis
Time Complexity:
• Best Case: O(1) → key is at the first position.
• Worst Case: O(n) → key is at the last position or not present.
• Average Case: θ(n) → key may be anywhere in the array.
Space Complexity:
• O(1) → no extra memory is required except for variables.
Advantages of Brute Force Method
1. Simple and easy to implement.
2. Guarantees a solution if one exists.
3. Works for any type of problem, without special techniques.
Limitations of Brute Force Method
1. Inefficient for large input sizes.
2. Takes too much time for problems with many possible solutions.
3. Not suitable for optimization problems where faster methods exist.

8
Sequential Search (Linear Search)
Definition
Sequential Search, also called Linear Search, is a simple search technique in which each
element of a list or array is checked one by one until the desired element (key) is found or
the list ends.
• It is the simplest searching method.
• Works on both sorted and unsorted lists.
Algorithm: Sequential Search
Input: An array A[0..n-1] and key element K
Output: Index of K if found, otherwise -1
1. Start from the first element i = 0
2. Repeat until i < n:
a. If A[i] == K, return i (element found)
b. Else, move to the next element i = i + 1
3. If end of array reached, return -1 (element not found)
Example
Problem: Find 7 in the array [3, 5, 2, 7, 9].
Step-by-Step Search:
1. Check 3 → not 7
4. Check 7 → found! Return index 3
Analysis
Case Number of Comparisons Time Complexity
Best Case 1 (first element) O(1)
Worst Case n (last element or not present) O(n)
Average Case n/2 θ(n)
Space Complexity: O(1) → only a few variables are used.

9
Advantages of Sequential Search (Linear Search)
1. Simple and easy to implement.
2. Works on unsorted data.
3. No preprocessing of data is required.
Limitations of Sequential Search (Linear Search)
1. Inefficient for large lists.
2. Takes linear time even if the element is near the end.
3. Not suitable for frequent searches in large datasets.
Comparison of Brute Force Method and Sequential (Linear) Search
Brute Force Method Sequential Search (Linear Search)
1. Tries all possible solutions to find the
correct one.
1. Checks each element of the list/array
one by one until the key is found.
2. General approach used for many types of
problems (searching, optimization, etc.).
2. Specific searching technique for finding
an element in a list or array.
3. Simple to understand and implement. 3. Simple to implement and easy to
understand.
4. Can be inefficient for large problem sizes
due to exhaustive search.
4. Inefficient for large arrays, time
complexity grows linearly with input size.
5. Works for any problem where all
possibilities can be enumerated.
5. Works for unsorted or sorted lists.
6. Time Complexity: Best case O(1), Worst
case O(n), Average case θ(n).
6. Time Complexity: Best case O(1), Worst
case O(n), Average case θ(n).
7. Space Complexity: Usually O(1)
(depends on problem).
7. Space Complexity: O(1).
8. Advantage: Guarantees a solution if it
exists.
8. Advantage: Simple and works without
preprocessing.
9. Limitation: Not suitable for large input
sets or optimization problems.
9. Limitation: Slow for large arrays, not
suitable for frequent searches in big
datasets.
*** ** ***

Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basic Efficiency Classes (DAA) BCA SEP SEM-III

More Related Content

Similar to Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basic Efficiency Classes (DAA) BCA SEP SEM-III

More from Kuvempu University

Recently uploaded

Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basic Efficiency Classes (DAA) BCA SEP SEM-III