Scribd
Upload
16 views7 pages

Introduction and Insertion Sort

Algorithm updated notes

Uploaded by

ushashangka
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
16 views7 pages

Introduction and Insertion Sort

Algorithm updated notes

Uploaded by

ushashangka
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Introduction and Insertion Sort

An algorithm is a well-defined computational procedure that takes


some value, or set of values, as input and produces some value, or
set of values, as output.

An algorithm is a sequence of computational steps that transform the


input into the output.

An algorithm is a tool for solving a well-specified computational


problem.

The sorting Problem: The input is sequence of numbers a1 , a2 ,..., an  .


The output is a permutation (reordering)  a1' , a2' ,..., an'  of the input
sequence such that a1'  a2'  ...  an' .

Example: If the input is 31,41,59,26,41,58 a sorting algorithm should


return the sequence  26,31,41,41,58,59 as output.

The above input sequence is called an instance of the sorting


problem. In general, an instance of a problem consists of the input
(satisfying whatever constraints are imposed in the problem
statement) needed to compute a solution to the problem.

An algorithm is said to be correct if, for every input instance, it halts


with the correct output. We say that a correct algorithm solves the
given computational problem. An incorrect algorithm might not halt at
all on some input instances, or it might halt with an answer other than
the desired one.

Analyzing an algorithm: Predicting the resources required such as


memory, communication bandwidth, or computational time. In most
cases, the computational time is really what we want to measure.

Efficiency: Algorithms devised to solve the same problem often


differ dramatically in their efficiency. These differences can be much
more significant than differences due to hardware and software.
Example: Suppose we have two computers, computer A, a fast
computer running insertion sort. Computer B, a slow computer
running merge sort. Each must sort an array of ten million numbers.
Suppose that computer A executes ten billion instructions per second
and computer B executes only ten million instructions per second
( Notice that computer A is 1000 times faster than computer B ).
Suppose Computer A requires 2n 2 instructions to sort n numbers and
computer B requires 50n lg n ( lg n = log 2 n ) instructions to sort the n
numbers. Then
Computer A takes

2.(10 7 ) 2 instructions
= 20,000 sec onds (more than 5.5 hours)
1010 instructions / sec ond

As for computer B, it takes

50.107. lg 107 instructions


 1163 sec onds (less than 20 minutes)
107 instructions / sec ond

This means that computer B solves the problem at hand 20 times


faster than computer A!

Insertion Sort: This algorithm will help us solve the sorting problem
introduced above. It is an efficient algorithm for sorting a small number of
elements.

Input: A sequence of n numbers  a1 , a2 ,..., an  .


Output: A permutation (reordering)  a1' , a2' ,..., an'  of the input sequence such
that a1'  a2'  ...  an'

The numbers that we wish to sort are known as the keys

Example: Suppose we have A = 5,2,4,6,1,3 . To sort this array of numbers,


first 2 need to be compared with 5, since 2 is less than 5, we exchange the
two numbers, and hence our array becomes: A =  2,5,4,6,1,3 . Next we must
compare 4 with 5 and 2, since 4 is smaller than 5, we exchange 4 and 5, then
4 is compared with 2, but 4 is larger than 2, so nothing to be done, so we get:
A =  2,4,5,6,1,3 . Next we compare 6 with 5. Since 6 is the larger than 5, no
exchange is done. Now, 1 is to be compared with 6, 5, 4, and 2. Since 1 is
the smallest, it is to be exchanged until we get the following
array: A = 1,2,4,5,6,3 . Finally, 3 is exchanged with 6, then 3 is exchanged
with 5, then it is exchanged with 4. Hence, we have the following sorted
array: A = 1,2,3,4,5,6 .

INSERTION-SORT(A)
1 for j = 2 to A.length
2 key = A[ j ]
3 // Insert A[ j ] into the sorted sequence A[1.. j − 1]
4 i = j −1
5 while i  0 and A[i]  key
6 A[i + 1] = A[i ]
7 i = i −1
8 A[i + 1] = key

A loop invariant is an invariant used to prove properties of loops.


Informally, a loop invariant is a statement of the conditions that should be
true on entry into a loop and that are guaranteed to remain true on every
iteration of the loop.
We use loop invariants to help us understand why an algorithm is correct.
We must show three things about a loop invariant:

Initialization: It is true prior to the first iteration of the loop


Maintenance: If it is true before an iteration of the loop, it remains true
before the next iteration.
Termination: When the loop terminates, the invariant gives us a useful
property that helps show that the algorithm is correct.

Now, let us see how these properties hold for insertion sort

Initialization: When j = 2 , the subarray A[1.. j − 1] consists of just the single


element A[1] , which is in fact the original element in A[1] . But, this subarray
is sorted (trivially, of course), which shows that the loop invariant holds
prior to the first iteration of the loop.
Maintenance: We will show this step informally, the body of the for loop
works by moving A[ j − 1] , A[ j − 2] , A[ j − 3] , and so on by one position to the
right until it finds the proper position for A[ j ] (lines 4-7), at which point it
inserts the value of A[ j ] (line 8). The subarray A[1.. j ] then consist of the
elements originally in A[1.. j ] , but in sorted order. Incrementing j for the
next iteration of the for loop then preserves the loop invariant.

Termination: The condition causing the for loop to terminate is that


j  A.length = n . Because each loop iteration increases j by 1, we must have
j = n + 1 at that time. Substituting n = 1 for j in the wording of the loop
invariant, we have that the subarray A[1..n] consists of the elements
originally in A[1..n] , but in sorted order. Observing that the subarray A[1..n]
is the entire array, we conclude that the entire array is sorted. Hence, the
algorithm is correct.

Analysis of Insertion Sort: The time taken by the Insertion-Sort procedure


depends on the input: sorting a 1000 numbers takes longer than sorting 3
numbers. In general, the time taken by an algorithm grows with the size of
the input. The running time is a function of the input size. The running
time also depends on how well sorted the input sequence is.

Input Size: In most cases the input size is the number of items in the input.
But in some cases, like multiplying large numbers, the total number of bits
in the input is a better measure.

Running Time: The running time of an algorithm on a particular input is the


number of primitive operations or “steps” executed. For now, a constant
amount of time is required to execute each line of our pseudocode. One
line may take a different amount of time than another line, but we shall
assume that each execution of the ith line takes time c i , where c i is a
constant.

Running Time of Insertion Sort: Let n = A.length . Let t j be the number of


times line 5 is executed for that value of j. The number of times each line is
executed is shown below: (Notice that for while and for loops the condition
is executed once more than the body of the loop)
cost times

1 for j = 2 to A.length c1 n
2 key = A[ j ] c2 n −1
3 // Insert A[j] into the sorted sequence
// A[1 .. j – 1] 0 n −1
4 i = j −1 c4 n −1
5 while i  0 and A[i]  key c5 t 2 + t 3 + ... + t n
6 A[i + 1] = A[i ] c6 (t 2 − 1) + (t 3 − 1) + ... + (t n − 1)
7 i = i −1 c 7 (t 2 − 1) + (t 3 − 1) + ... + (t n − 1)
8 A[i + 1] = key c8 n −1

The running time of the algorithm is:

 (cost of statements). (number of times statement is executed). The sum

is taken over all statements.

The total running time on the given machine is:

T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 (t 2 + t 3 + ... + t n ) + c6 [(t 2 − 1) + (t 3 − 1) + ... + (t n − 1)]


+ c7 [(t 2 − 1) + (t 3 − 1) + ... + (t n − 1)] + c8 (n − 1)

The running time depends on the values of t j . These vary according to the
input.

Best-Case: Line 5 is executed just once for each j. This happens if A is


already sorted. For all j, t j = 1 and

T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 (n − 1) + c8 (n − 1)
= (c1 +c 2 +c4 + c5 + c8 )n − (c2 + c4 + c5 + c8 )

T (n) can be expressed as an + b for constants a and b that depend on the given
machine. It is a linear function of n.
Worst-Case: Line 5 is executed j times for each j . This can happen if A
starts out in reverse order. For all j , t j = j,
(n + 2)(n − 1)
t 2 + t 3 + ... + t n = 2 + 3 + ... + n = and
2

n(n − 1)
(t 2 − 1) + (t 3 − 1) + ...(t n − 1) = 1 + 2 + ... + n − 1 = . Hence, T(n) could be
2
expressed as:

(n + 2)(n − 1) n(n − 1) n(n − 1)


T (n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 + c6 + c7 + c8 (n − 1)
2 2 2
Upon simplification, one gets:

T (n) = an 2 + bn + c , where a, b, and c are constants that depend on the


statement costs c i . Notice that T (n) is a quadratic function of n.

Average-Case: We assume that on the average, half of the elements in


A[1.. j − 1] are greater than the key and the other half are less. Line 5 is
j j
executed times for each j so that t j = . Then
2 2

2 3 n (n + 2)(n − 1)
t 2 + t 3 + ... + t n = + + ... + = and
2 2 2 4
n
(t2 − 1) + (t3 − 1) + ... + (tn − 1) = (0) + (0.5) + (1) + ... + ( − 1)
2

(n 2 − 3n + 2)
If we simplify the above expression, we get:
4

Hence, we have:

(n + 2)(n − 1) (n 2 − 3n + 2) (n 2 − 3n + 2)
T (n) = c1 n + c 2 (n − 1) + c 4 (n − 1) + c5 + c6 . + c7 . + c8 (n − 1)
4 4 4

Upon simplification, one gets that:

T (n) = an 2 + bn + c , where a, b, and c are constants that depend on the given


machine. Notice that T(n) is a quadratic function of n.
Now, another important concept one likes to consider is the running time for
Insertion-Sort for small values of n or for large values of n.

For small values of n, Insertion-Sort will probably run very quickly that we
do not care how long it takes.

For large values of n, Insertion-Sort will probably run very slowly, and it
will take a long time and its running time is very important.

Consider the worst-case running time for Insertion-Sort:


T (n) = an 2 + bn + c , where a, b, and c are constants that depend on the given
machine. As n gets larger and larger the an 2 term grows faster than the bn
and c terms. As n gets large

T ( n) b c
2
= a + + 2 approaches the constant a and the other two terms go
n n n
toward 0.

When we analyze any algorithm, we are interested in the running time for
large input sizes (large n) and we just consider the dominant term (the term
that grows the fastest). Our analysis of algorithms should not depend on any
particular machine, so we ignore the constant in front of the dominant term.
We only look at the rate of growth as the input size gets larger and larger.

For Insertion-Sort we say the worst-case running time is (n 2 ) , and the best-
case running time is (n) .

In general, when analyzing an algorithm one has to concentrate on the


average-case or worst-case. This is because:

a) The worst-case running time of an algorithm is an upper bound on the


running time for any input
b) For some algorithms, the worst case occurs fairly often.
c) The average-case is often roughly as bad as the worst case. (See the
average-case analysis for the Insertion-sort, it was as bad as the worst-
case as seen above)

You might also like