2. ../Introduction, Internal & External Sort
../Comparison Sorting Algorithms: Bubble,
Selection and Insertion Sort
../Divide and Conquer Sorting: Merge and
Quick Sort
../Efficiency of Sorting Algorithms
2
3. ./
3
.,
-
.,
-
Sorting is a technique to rearrange the elements of a list in ascending or descending order,
which can be numerical, lexicographical, or any user-defined order.
Sorting is a process through which the data is arranged in ascending or descending
order The complexity of a sorting algorithm can be measured in terms of
number of algorithm steps to sort n records
number ofcomparisons between keys (appropriate when thekeys are long character
strings)
.l number of times records must be moved (appropriate when record size is large)
.,- Example:
l, 3, 4, 6, 8, 9 arc in increasing order, as every next clement is greater than the previous
clement.
❒ 9. 8. 6, 4, 3, I are in decreasing order, as every next element is less than the previous
element.
For example, 9, 8, 6, 3, 3, I arc in non-increasing order, as every next clement is less than or
equal to (in case of3) but not greater than any previous element.
For example, I, 3, 3, 6, 8, 9 are in non-decreasing order. as every next element is greater than or
equal to (in case of 3) but not less thanthe previous one.
4. ./
4
./
./
Sorting algorithms 1nay require some extra space for comparison and temporary storage of
few data elements. These algorithms do not require any extra space and sorting is said to
happen in-place, or for example, within the array itself. This is called in-place sorting.
Bubble sort is an example of in-place sorting.
Sorting which uses equal or n1ore space is called not-in-place sorting. Merge-sort is an
example of not-in-place sorting.
If a sorting algorithm, after sorting the contents, does not change the sequence of si,nilar
content in which thev aooear. it is called stahle sortinl!.
-r � l-[ � [ 4 :
-
=
-
=JG= : 1( -.: TG: J=G
,= �[� )["2
,
,
_:.1c
� l
1..o ) [ _ , . . :i -
.
g
H 1
2 a H 2 6 H a - . II = 1 = I[
"""'H4 4 L
" 3 5 .
.
7 a
./ If a sorting algorithm, after sorting the contents, changes the sequence of sin1ilar content in
which they
appear.
4 & & 7 e 9
It is calledunstablcsorting. [ =-
-,
]
[
2 3
4 2 ] [ -.o
J(-.,4 ]
o
3 s ] [
-
- '- , e . , ( 2 6 ] [ - ] ( 2 6 ]
( 3 , J
r ..o H ..
4 ll"
"
'
"
'
.
.
.
_.
.,
,
,
.
.
.
.
,
2 6
0 -, 2 3
H II 2 6 l
3 -,
4 G
5. Internal sorting
5
./
'
.,
An internal sort is any data sorting process that takes place entirely within the main memory of a
computer.
This is possible whenever the data to be sorted is small enough to all be held in the main
memory.There are3 types ofinternal sorts.
(i) SELECTION SORT :- Ex:- Selection sort algoritbn1, Heap Son algorithm
,.., (ii) INSERTION SORT:- Ex:- Insertion sort algorithm, Shell Sort algorithm
(iii) EXCHANGE SORT:- Ex:- Bubble Sort Algoritlun, Quick sort algorithm
External
sorting
., Sorting large amount ofdata requires external or secondary memory. This process uses external
memory such as HDD, to store the data which is not fit into the main memory. So, primary
memory holds the currently being sorted data only. All external sorts are based on process of
merging. Different parts ofdata are sorted separately and merged together.
./' Ex:- ltlerge Sort
NOTE: Wewill only consider internal
sorting
6. .,
,
6
.,
,
.,
,
Insertion sort is a simple sorting algorithm that works similar to the lfay you sort
playing cards in your hands.
The array is virtually split into a sorted and an unsorted part.
Values from the unsorted part are picked and placed at the correct position in the
sorted part.
E
J
E
J
1 st card: 1
o
�
2nd card: 5
�
3rd card: K
�
B
EJB
7. 1
.
7
2.
3.
We start with an empty left hand [sorted array] and the cards face dovn on the
table [unsorted array].
T hen remove one card (key] at a time from the table [unsorted array], and
insert it into the correct position in the left hand [sorted array].
To find the correct position for the card, we compare it with each of the cards
already in the hand, from right to left.
Start
Iteration
1
Iteration 2
Iteration 3
I_r
:
-
i......_ . , . , ___.,_,.,,
1� Sorted
j
I0 Unsorted
I r:7
Unsorted
I
I
! � To be inserted ,
8. void insertionSort(int a[], int n)
{
8
ocxt is the item to be
=
for (int i 1; i < n; i++)
{
- - - -
inserted
int next = a[i];
int j;
for (j= i-1; j>= 0 && au]> next; j--)
au+1] = au]; - -
au+ 1] = next;
-
}
}
Shift sorted items to make place for next
Insert next to the correct location
9. ../ Outer-loop executes (n-1) times
9
../ Number of times inner-loop is executed depends on the input
1 Best-case: the array is already sorted and (a[j] > next) is always false
� No shifting of data is necessary
:. Worst-case: the array is reversely sorted and (a[j] > next) is always
hue Insertion always occur at the fr
ont
Average case: wehave to look at all possible intial data organization
../ Therefore,
* best-case titne is O(n)
' worst-case time is O(n2)
,., average- case time is O(n2)
10. ./ The selection sort algorithm sorts an array by
repeate dly fi nding the mi n imum element
(considering ascending order) from unsorted part
and putting it at the beginning.
Algorithm:
10
1. Find tl1e minimum value in the list
,
.
, Swap it with the value in the first position
J. Repeat tl1is process for all the
elements unti l the entire array is sorted
11. §
[
E
l
[
E
(
C
1
J
�
(
Q
)
� §
(
Q
)
�
u
�
[E�[F)l[E
11
T h e array, b e f o r e th e sete-ction Sort
42 16
8 4 12 7 7 26 53 ope-ration b e g i n s ¥
2
:
1 2 T h e smallest numt>er (12) Is s w a p p e d
6 4
I I I:4
I
16 7 7 2 6 53 in1o the fir-sl element in the stl"Jcture.
I I
12
16 8 4 42 77 2 6 53
In the data that ren,ains. 16 is the
smallest; a n d it d o e s not n e e d to
b e moved.
2
6 4 26 is the next smallest number. anct
12 16 42 7 7
I I :I 1:
I
8 53 it is s w a p p e d into the third position.
42 is the next smauest number"; it is
I I I
12 16 26 42 77 84 53 already in the correct position.
5
3
7
7
12 16 26 42 8 4
I I I :I 1:
I
5 3 is t h e s m a l l e s t numbe-..-. in t h e
d a t a t n a l r e m a i n s : a n d 1
1ts s w a p p e d
t o the appropriate posHion.
12 16 26 42 5 , 3
I I I I I I7 7 : I : 8 4
I
o r the two remaining data items� 7 7 is
the smaller:; the item s are s w a pp e d .
The selection sort is nolV cornplete.
12. ../Selecting the lowest element requires scanning
all
12
n elements (this takes n-1 co1nparisons) and then
swapping it into the first position.
../ Finding the next lowest element requires
scanning
all n-1 elements and so on, for (n-1) + (n-2) + ...
+
2 + 1 (O(n2)) comparisons. Each of these
scans requires one swap for n-1 elements.
../ Therefore,
;: Worst case performance: O(n2)
') Best case performance: O(n2)
o Average case performance:
O(n2)
13. ./ Bubble Sort is the simplest sorting algorithm that
works by repe atedly swapping the adjacent
elements if they are in wrong order
Algorithm:
13
Given an array of n items
1. Compare each pair of adjacent elements in the list
2. Swap two element if necessary
J. Repeat this process for all the elements until the
entire array is sorted
4. Reduce n by Iand go to Step I
14. [2J[2JG
J[2JQ+unsorted list
14
[2JGJ
[2J0[2J
2<6,NoSwap
QGJ0 0 [
2
J 6>3, S w ap
0 [2JGJ0 0 7>2, S w a p
[2J[2JGJ[2J 0 7>6, S w a p
0 G
J 0 0 0
7>3,Sw a
p
D GJ 0 0
0
7>1, S w a p
• 0 0 0 0 0 6>1,Swap
0 0 0 0 0
+ - End Of Round 2
0 G
J '2J[!] l2] 2>1, S w ap
12] GJ GJ GJ l 2 J - endOfRound4 •
Final Answer
Bubble
Sort
�
0 0 0 0 0
0 0 0 0 0
2<3,NoSwap
3>1,
Swap
15. ./
15
./
./
./
./
1 iteration of the inner loop (test and swap) requires time
bounded by a constant c
Two nested loops Two nested loops
1 Outer loop: exactly n iterations
c Inner loop:
o when i:=O, (n-
1) iterations wben i""L,
(u-2) iterations
0
when i::(n- l), 0 iterations
Total number of iterations = O+I+...+(n-1) = n(n-
1)/2 Total time = c n(n-1)/2 = O(n2)
Therefore,
rJ worst case:O(n2)and best case:O(n)
16. ,,
-
,,
-
2
1
v'
Divide and Conquer is an algorithmic pattern.
In algorithmic methods,the design is to take a dispute on a huge input, break the
input into minor pieces, decide the problem on each of the small pieces, and then
merge the piecewise solutions into a global solution. This 1nechanism of solving
the problem is called the Divide & Conquer Strategy.
Divide and Conquer algo1ithm consists of a dispute using the following three
steps.
Divide the original problen1 into a set ofsubproblems.
Conquer: Solve every subproblcn1 individually, recursively.
' Con1bine: Put together the solutions of the subproble1ns to get the solution
to the whole
problem.
,,
-
Examples: The specific computer algorithms are based on the Divide & Conquer
approach:
, Maxi1num and Mini1num Problen1
..., Binary Search
.: Sorting (merge sort, quick sort)
0 Tower of Hanoi.
18. .,,- Merge Sort is a divide-and-conquersorting
algorithm
24
.,,-This is a simple and very efficient algorithm for sortinga
list of numbers.
.,,-We are given a sequence of nnumberswhich we will
assume is stored in an array A [1 ...n].
r Divide step
• Divide the array into two (equal) halves
Recursively sort the two halves
J Conquer step
Merge the two halves toform a sorted
array
19. ../ Conceptually, amerge sort works as
follows:
25
1. If the list is of length O or 1, the11it is already
sorted. Otherwise:
2. Divide the unsorted list into two sublists of about
half tl1e size.
3. Sort each sublist recursively by re-applying merge
sort.
4. Merge the two sublists back into one sorted list.
21. Level O:
mergeSort n items
Total time complexity= O(n
lg(n))
27
Level 1:
mergeSort n/2 items n/2 n/2
Level 2:
mergeSort n/2 2 items n/22
�
n
/
22
Level (lg n):
mergeSort 1 item
n/(2k) = 1 -+ n = 2k -+ k = lg n
Level O:
1call to mergeSort
Level 1:
2 calls to mergeSort
Level 2:
22
calls to mergeSort
Level (lg n): 2•e
n(= n) calls to
mergeSort
22. ./ Qu i ck Sort is on e of the di fferent Sorti ng
Technique which is based on the concept of
Divide and Conquer
./ It is also called partiti on-exch ange sort. This
algorithm divides the list into three main parts:
2.8
>Elements less than the Pivot element
o Pivot element(Central element)
.) Eleme11ts greater than the pivot element
./ Pivot element can be any element from the array,
it can be the first element, the last element or
any random element.
23. Following are the steps involved in quick sort algorithm:
2.9
1. After selecting an element as pivot, which is the last
index of the array in our case, wedivide the array for the
3
.
first time.
2. Define two variables i and j. Set i and j to first
and last elements of the list respectively.
Increment i until list[i] > pivot then stop.
4
.
Decrement j until list[j] < pivot then
stop.
5. If i < j then exchange list[i] and list[j].
6. Repeat steps 3,4 & 5until i > j.
7. Exchange the pivot element with list[j]
element.
24. Consider the following unsorted list of
elements...
30
Define pivot, left & right Set pivot= 0,left= 1& right= 7. Here '7' indicates 'size-1'.
List I 5 I 3 I 8 I 1 I4 I 6 I
2 I 7 I
pivot
Compare List[left] with List[pivot]. If List[left] is greater than List[plvot] then stop leftotherwise move left
to the next.
Compare List[right) with List[pivot]. If List[right] is smaller than List[pivot] then stop right otherwise move
right to the previous.
Repeat the same until left> =right.
If both left & right are stoped but left<nght thenswap List[left] withlist[right] andcountinue the
process. If left>= right thenswap List[pivot] with List(right].
25. left
31
right
L
�
l
i
s
l
3
l
lsl11416
12 1I
plvot
Compare List[left]<List{pivot] as it is true increment left by one and repeat the same, left will stop at 8.
Compare List[right)> List[pivot] as it is true decrement right by one and repeat the same, right will stop at 2.
- right
L
i
s
t
I5 I I 8 I
I
3 146
2
I I I 7
I
pivot
Here left & right both are stoped and left is not greater than right so we need to swap List[left] and list[right]
left right
i
L
s
t
Is
1
3
1
4
6
s 1
2111 1
1 1
1
pivot
Compare List[left:I <List[pivot] as it is true increment left by one and repeat the same. left will stop at 6.
Compare List[right]>List[pivot) as it is true decrement right by one and repeat the same, right will stop at 4.
right
left
u
�
l
5 8
7
131211141
611 1
pivot
Here left & right both are stoped and left is greater than right so we need to swap List[pivotJ and List[right)
26. @
�
�
(� §(Q)�lf�
[E� [p)[L[E
32
list
left
Here wecan observe that all the numbers to the left sideof 5are smaller and right side are greater. That
means S is placed in its correct position.
Repeat the same processon the left sublist and rightsublist to the number 5.
right
ldt right
list
I4 I 3 I2 111s I 6
1s 11I
I4 I 3 I2 I 1 I s I 6 Is
I1 I
pivot pivot
In the left sublist astherearenosmallernumberthanthepivot leftwillkeeponmovingtothenextandstops at last number. As
the List[right] is smaller, right stops at same position. Now left and right both are equal so weswap pivot with
right.
left right
list
11
I 3
I 2
I 4
I 5
I 6
1s11
I
pivo
t
27. left right
Lidl1l3l2l4lsl6lsl1I
pivot
In the right sublist left is grester than the pivot, left will stop at same position.
As the List[right] is greater than List[pivot), right moves towards left and stops at pivot number position. Now left > right
so weswap pivot with right. (6 is swap by itself).
left right
pivot
Repeat the same recursively on both left and right sublists until all the numbers are sorted.
The final sorted list will be as follows...
List I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8
I 3
3
28. ./ Best-case: Pivot is always the median
)T(0)=T(l)= l
rT(n)=2T(n/2 ) + n -- linear-time partition
r Same recurrence as mergesort: O(n log 11)
./ Worst-case: Pivot is always smallest or largest element
oT(0)=
T(l)=l
OT(n) = lT(n-1) + n
''"' Basically same recun·ence as selection sort: O(n2)
./Average-case (e.g., with random pivot)
T(n) = + + Cn:t)' !Il�1 T(i - l) + T(n - i)]
() O(n log n),
36
29. .,,-The co mplexity of a sorting algorithm mea sures the
running time of a function in which n number of items
are to be sorted.
.,,- The choice of sorting method depends on effici
ency considerations for different problems.
.,,- Tree most important of these considerations are:
CThe length of ti1ne spent by programmer in
coding a particular sorting program
11 Amount of machinetime necessary for running
theprogram OThe amount of memory necessary for running
theprogram
52
30. Algorithm Data Structure
Qulcksor
t
Time Complexity
Best
jo(n log(n))I
Average
1ocn l o g ( >
Worst
locnAZ)I
Array
Array
loc
n
0
1
�.
n
.
221
[o
c
}og&n 2
1
Array I
jo(n log(n))I
jocn log(n))I
lo<n>I
1ocn l o g (
j o(n"Z>I
locnAZ)I
IO(n"2)I
locnAZ)I
Mergesort
Heapsort
Bubble Sort
Insertion Sort
Select Sort
Array
Array
Array
lo
jo(nAZ)I
jo(n"Z)I lo(n"Z)j
54