Sorting Algorithms
Course Instructor: Maheen Zulfiqar
Insertion sort
Cont.
• Best Case: O(n):
• When the input array is already sorted, insertion sort only needs to iterate through
the array once to confirm its sorted order. This results in a linear time complexity of
O(n).
• Average Case: O(n^2):
• On average, with randomly ordered data, each element needs to be compared and
potentially swapped with elements before it in the sorted portion of the array. This
leads to a quadratic number of operations, hence O(n^2).
• Worst Case: O(n^2):
• The worst-case scenario occurs when the input array is sorted in reverse order. In
this case, every element needs to be compared and shifted to its correct position,
resulting in a quadratic number of operations (O(n^2)).
Quick Sort
• Pick a Pivot: Choose one item from the list. It can be any item, but people often pick the middle
item or the first one.
• Divide the List: Compare all the other items to the pivot. Put the items smaller than the pivot on
the left side and the items larger than the pivot on the right side.
• Repeat: Now, do the same thing with the left side and the right side separately. Pick new pivots
and keep dividing the list until each group has only one item left.
• Combine: Once all the small groups are sorted, you put them back together to get the full, sorted
list.
Cont.
Cont.
Best Case:
• O(n log n) - This occurs when the pivot element consistently divides the
array into roughly equal halves, leading to a balanced partitioning.
Average Case:
• O(n log n) - This is the typical performance when the pivot selection is
random or uniformly distributed.
Worst Case:
• O(n^2) - This happens when the pivot consistently results in highly
unbalanced partitions, such as when the array is already sorted or when
the pivot is always the smallest or largest element.
• The worst case scenario for Quicksort is O(n2). This is when the pivot element is
either the highest or lowest value in every sub-array, which leads to a lot of
recursive calls. With our implementation above, this happens when the array is
Cont.
• Space Complexity:
• Average and Best Case:
• O(log n) - This is due to the recursive nature of the algorithm.
• Worst Case:
• O(n) - In the worst-case scenario, the recursion depth can become
linear to the input size, leading to a space complexity of O(n).
Merge Sort
The algorithm needs to split the array and merge it back together whether it is already sorted or
completely shuffled.
Cont.
• Time Complexity:
• Merging: The algorithm then merges these sorted subarrays to produce a
final sorted array.
• O(n log n): The time complexity arises from the logarithmic number of
divisions (log n) and the linear time (n) required for each merge operation.
• Consistent Performance: Merge Sort consistently performs at O(n log n) in all
cases, making it a reliable sorting algorithm.
• Space Complexity:
• O(n): Merge Sort requires O(n) space to store the temporary arrays during the
merging process.
Bubble Sort
Code
• The Bubble Sort algorithm loops through every value in the array,
comparing it to the value next to it. So for an array of n values, there
must be n such comparisons in one loop.
• And after one loop, the array is looped through again and
again n times.
• This means there are n⋅n comparisons done in total, so the time
complexity for Bubble Sort is: O(n^2)
Cont.
• Time Complexity:
• Best Case: O(n) - occurs when the array is already sorted, requiring only
one pass.
• Average Case: O(n²) - occurs when the array is in a random order.
• Worst Case: O(n²) - occurs when the array is sorted in reverse order.
• Space Complexity:
• O(1) - Bubble sort is an in-place algorithm, meaning it sorts the array
directly without needing a lot of extra memory. It only requires a few
extra variables to hold values temporarily while swapping elements.
Linear Search Algorithm
Time Complexity
Best Case Complexity
• The element being searched could be found in the first position.
• In this case, the search ends with a single successful comparison.
• Thus, in the best-case scenario, the linear search algorithm
performs O(1) operations.
Worst Case Complexity
• The element being searched may be at the last position in the array
or not at all.
• In the first case, the search succeeds in ‘n’ comparisons.
Cont.
• In the next case, the search fails after ‘n’ comparisons.
• Thus, in the worst-case scenario, the linear search algorithm performs O(n)
operations.
Average Case Complexity
• When the element to be searched is in the middle of the array, the average case of the
Linear Search Algorithm is O(n).
The linear search algorithm takes up no extra space; so, its space complexity is O(1).
Linear Search
python
def linear_search(arr, key):
for i in range(len(arr)):
if arr[i] == key:
return i
return -1
•Best Case: Key at index 0 → O(1)
•Worst Case: Key not found or last
element → O(n)
•Average Case: O(n)
Binary Search
• Binary search is a fast search algorithm with run-time complexity of
(log n). This search algorithm works on the principle of divide and
conquer, since it divides the array into half before searching. For this
algorithm to work properly, the data collection should be in the
sorted form.
• Binary Search algorithm is an interval searching method that performs
the searching in intervals only. The input taken by the binary search
algorithm must always be in a sorted array since it divides the array
into subarrays based on the greater or lower values.
Cont.
Step 1 − Select the middle item in the array and compare it with the key value
to be searched. If it is matched, return the position of the median.
Step 2 − If it does not match the key value, check if the key value is either
greater than or less than the median value.
Step 3 − If the key is greater, perform the search in the right sub-array; but if
the key is lower than the median value, perform the search in the left sub-
array.
Step 4 − Repeat Steps 1, 2 and 3 iteratively, until the size of sub-array becomes
1.
• Step 5 − If the key value does not exist in the array, then the algorithm
returns an unsuccessful search.
Key value=31
Time Complexity
Best-case
The best-case scenario occurs when the target element is the middle element of the
array.
• In this case, the algorithm will find the element on the first iteration itself.
• Thus, the time complexity for the best case is O(1).
Average – worst - case
• The time complexity of the binary search algorithm is O(log n)
Space Complexity: The iterative version has better space complexity O(1) compared
to the recursive version O(log n) because the latter uses additional memory.
Cont.
