Sorting I

Chapter 8
Kruse and Ryba
 Introduction
• Common problem: sort a list of values, starting
from lowest to highest.
– List of exam scores
– Words of dictionary in alphabetical order
– Students names listed alphabetically
– Student records sorted by ID#
• Generally, we are given a list of records that have
keys. These keys are used to define an ordering of
the items in the list.
C++ Implementation of Sorting
• Use C++ templates to implement a generic sorting
function.
• This would allow use of the same function to sort items
of any class.
• However, class to be sorted must provide the following
overloaded operators:
– Assignment: =
– Ordering: >, <, ==
• Example class: C++ STL string class
• In this lecture, we’ll talk about sorting integers; however,
the algorithms are general and can be applied to any class
as described above.
 Quadratic Sorting Algorithms
• We are given n records to sort.
• There are a number of simple sorting
algorithms whose worst and average case
performance is quadratic O(n2):
– Selection sort
– Insertion sort
– Bubble sort
 Sorting
Sorting an
an Array
Array of
of Integers
Integers
• Example: we
are given an 70
array of six 60
integers that 50
we want to 40
sort from 30
smallest to 20
largest
10
0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• Start by
finding the 70
smallest 60
entry. 50
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• Swap the
smallest 70
entry with 60
the first 50
entry. 40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• Swap the
smallest 70
entry with 60
the first 50
entry. 40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side
70
• Part of the 60
array is now 50
sorted. 40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• Find the
smallest
element in
the unsorted
side.

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• Swap with
the front of
the unsorted
side.

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• We have
increased the
size of the
sorted side
by one
element.

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• The process
continues... Smallest
Smallest
from
from
unsorted
unsorted

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• The process
continues... w aapp
SSw th
wwi ith t
nt
frfroon

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
Sorted
Sortedside
side
is
isbigger
bigger
Sorted side Unsorted side

• The process
continues...

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• The process Sorted side Unsorted side
keeps adding
one more
number to the
sorted side.
• The sorted side
has the smallest
numbers,
arranged from
small to large. [0] [1] [2] [3] [4] [5]
 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• We can stop Sorted side Unsorted sid
when the
unsorted side
has just one
number, since
that number
must be the
largest number.

[0] [1] [2] [3] [4] [5]

 The
The Selection
Selection Sort
Sort Algorithm
Algorithm
• The array is now
sorted.
• We repeatedly
selected the
smallest
element, and
moved this
element to the
front of the
unsorted side. [0] [1] [2] [3] [4] [5]
template <class Item>
void selection_sort(Item data[ ], size_t n)
{
size_t i, j, smallest;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 0; i < n-1 ; ++i)

{
// find smallest in unsorted part of array
smallest = i;
for(j = i+1; j < n; ++j)
if(data[smallest] > data[j]) smallest = j;

// put it at front of unsorted part of array (swap)

temp = data[i];
data[i] = data[smallest];
data[smallest] = temp;
}
}
 Selection Time Sort Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 1 to n-1
find smallest key in unsorted part of array

swap smallest item to front of unsorted array

decrease size of unsorted array by 1
 Selection Time Sort Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 1 to n-1 O(n)
find smallest key in unsorted part of array O(n)
swap smallest item to front of unsorted array
decrease size of unsorted array by 1
• Selection sort analysis: O(n2)
template <class Item>
void selection_sort(Item data[ ], size_t n)
{
size_t i, j, smallest;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 0; i < n-1 ; ++i) Outer loop:

{ O(n)
// find smallest in unsorted part of array
smallest = i;
for(j = i+1; j < n; ++j)
if(data[smallest] > data[j]) smallest = j;

// put it at front of unsorted part of array (swap)

temp = data[i];
data[i] = data[smallest];
data[smallest] = temp;
}
}
template <class Item>
void selection_sort(Item data[ ], size_t n)
{
size_t i, j, smallest;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 0; i < n-1 ; ++i) Outer loop:

{
// find smallest in unsorted part of array O(n)
smallest = i;
for(j = i+1; j < n; ++j) Inner loop:
if(data[smallest] > data[j]) smallest = j; O(n)
// put it at front of unsorted part of array (swap)
temp = data[i];
data[i] = data[smallest];
data[smallest] = temp;
}
}
 The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
• The Insertion
Sort algorithm 70
also views the 60
array as having 50
a sorted side 40
and an 30
unsorted side. 20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side
• The sorted 70

side starts 60

with just the 50

first 40
element, 30
which is not 20
necessarily 10
the smallest 0
element. [1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• The sorted 70

side grows 60

by taking the 50

front 40

element 30
from the 20
unsorted 10
side... 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• ...and 70

inserting it 60

in the place 50

that keeps 40

the sorted 30
side 20
arranged 10
from small 0
[1] [2] [3] [4] [5] [6]
to large. [0] [1] [2] [3] [4] [5]
The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side
70

60

50
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Insertion
Insertion Sort
Sort Algorithm
Algorithm
Sorted side Unsorted side

• Sometimes 70

we are lucky 60

and the new 50

inserted item 40

doesn't need 30
to move at 20
all. 10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Insertionsort
Insertionsort Algorithm
Algorithm
Sorted side Unsorted side

• Sometimes 70

we are lucky 60

twice in a 50

row. 40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Copy the Sorted side Unsorted side
new element 70
to a separate
60
location.
50
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Shift
elements in 70
the sorted
60
side,
50
creating an
40
open space
30
for the new
20
element.
10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Shift
elements in 70
the sorted
60
side,
50
creating an
40
open space
30
for the new
20
element.
10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Continue
shifting 70
elements...
60

50
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Continue
shifting 70
elements...
60

50
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 ...until you
reach the 70
location for
60
the new
50
element.
40

30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
 Copy the Sorted side Unsorted sid
new element 70
back into the
60
array, at the
50
correct
40
location.
30

20

10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 How
How to
to Insert
Insert One
One Element
Element
• The last Sorted side Unsorted sid
element
must also be
inserted.
Start by
copying it...

[0] [1] [2] [3] [4] [5]

Sorted
Sorted Result
Result

[0] [1] [2] [3] [4] [5]

template <class Item>
void insertion_sort(Item data[ ], size_t n)
{
size_t i, j;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 1; i < n; ++i)

{
// take next item at front of unsorted part of array
// and insert it in appropriate location in sorted part of array
temp = data[i];
for(j = i; data[j-1] > temp and j > 0; --j)
data[j] = data[j-1]; // shift element forward

data[j] = temp;
}
}
 Insertion Sort Time Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 1 to n-1
take next key from unsorted part of array

insert in appropriate location in sorted part of array:

for j = i down to 0,
shift sorted elements to the right if key > key[i]
increase size of sorted array by 1
 Insertion Sort Time Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 1 to n-1 Outer loop:
take next key from unsorted part of array O(n)

insert in appropriate location in sorted part of array:

for j = i down to 0,
shift sorted elements to the right if key > key[i]
increase size of sorted array by 1
 Insertion Sort Time Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 1 to n-1 Outer loop:
take next key from unsorted part of array O(n)

insert in appropriate location in sorted part of array: Inner loop:

for j = i down to 0, O(n)
shift sorted elements to the right if key > key[i]
increase size of sorted array by 1
template <class Item>
void insertion_sort(Item data[ ], size_t n)
{
size_t i, j;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 1; i < n; ++i)

O(n)
{
// take next item at front of unsorted part of array
// and insert it in appropriate location in sorted part of array
temp = data[i];
for(j = i; data[j-1] > temp and j > 0; --j)
O(n)
data[j] = data[j-1]; // shift element forward

data[j] = temp;
}
}
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Swap? 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Yes! 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Swap? 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Swap? 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Swap? 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Yes! 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Swap? 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• The Bubble
Sort algorithm 70
looks at pairs 60
of entries in 50
the array, and 40
swaps their 30
order if 20
needed.
10

Yes! 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Repeat.
70
60

50

40
30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Loop over
array n-1 70
times, 60
swapping pairs 50
of entries as 40
needed. 30

20
10

Swap? No. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
 The
The Bubble
Bubble Sort
Sort Algorithm
Algorithm
• Continue
looping, until 70
done. 60

50

40
30

20
10

Swap? Yes. 0
[1]
[0] [2]
[1] [3]
[2] [4]
[3] [5]
[4] [6]
[5]
template <class Item>
void bubble_sort(Item data[ ], size_t n)
{
size_t i, j;
Item temp;

if(n < 2) return; // nothing to sort!!

for(i = 0; i < n-1; ++i)

{
for(j = 0; j < n-1;++j)
if(data[j] > data[j+1]) // if out of order, swap!
{
temp = data[j];
data[j] = data[j+1];
data[j+1] = temp;
}
}
}
template <class Item>
void bubble_sort(Item data[ ], size_t n)
{
size_t i, j;
Item temp;
bool swapped = true;

if(n < 2) return; // nothing to sort!!

for(i = 0; swapped and i < n-1; ++i)
{ // if no elements swapped in an iteration,
// then elements are in order: done!
for(swapped = false, j = 0; j < n-1;++j)
if(data[j] > data[j+1]) // if out of order, swap!
{
temp = data[j];
data[j] = data[j+1];
data[j+1] = temp;
swapped = true;
}
}
}
 Bubble Sort Time Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 0 to n-1
for j =0 to n-2
if key[j] > key[j+1] then swap
if no elements swapped in this pass through array, done.
otherwise, continue
 Bubble Sort Time Analysis
• In O-notation, what is:
– Worst case running time for n items?
– Average case running time for n items?
• Steps of algorithm:
for i = 0 to n-1 O(n)
for j =0 to n-2 O(n)
if key[j] > key[j+1] then swap
if no elements swapped in this pass through array, done.
otherwise, continue
 Timing
Timing and
and Other
Other Issues
Issues
• Selection Sort, Insertion Sort, and Bubble Sort all
have a worst-case time of O(n2), making them
impractical for large arrays.
• But they are easy to program, easy to debug.
• Insertion Sort also has good performance when the
array is nearly sorted to begin with.
• But more sophisticated sorting algorithms are
needed when good performance is needed in all
cases for large arrays.
• Next time: Merge Sort, Quick Sort, and Radix Sort.