0% found this document useful (0 votes)
11 views44 pages

Lecture 3 Insertion Sort

The document explains the Insertion Sort algorithm, detailing its implementation, time complexity, and correctness through mathematical induction. It highlights the best and worst-case scenarios, concluding that the overall time complexity is Θ(n²). Additionally, it provides a step-by-step example of how the algorithm sorts an array.

Uploaded by

usamaayaz475
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
11 views44 pages

Lecture 3 Insertion Sort

The document explains the Insertion Sort algorithm, detailing its implementation, time complexity, and correctness through mathematical induction. It highlights the best and worst-case scenarios, concluding that the overall time complexity is Θ(n²). Additionally, it provides a step-by-step example of how the algorithm sorts an array.

Uploaded by

usamaayaz475
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 44

Insertion Sort

Aimal Rextin

Department of Computing
Insertion Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1
Insertion Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Is Insertion-Sort correct?

What is the time complexity of Insertion-Sort?

Can we do better?
Complexity of Insertion-Sort
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1
Complexity of Insertion-Sort
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Outer loop (lines 1–5) runs exactly n − 1 times (with n = length(A))

What about the inner loop (lines 3–5)?


◮ best and worst case?
Complexity of Insertion-Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Best case:
Complexity of Insertion-Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Best case: the inner loop is never executed


◮ what case is this?
Complexity of Insertion-Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Best case: the inner loop is never executed


◮ what case is this?

Worst case:
Complexity of Insertion-Sort (2)
Insertion-Sort(A)
1 for i = 2 to length(A)
2 j =i
3 while j > 1 and A[j − 1] > A[j]
4 swap A[j] and A[j − 1]
5 j = j −1

Best case: the inner loop is never executed


◮ what case is this?

Worst case: the inner loop is executed so that j goes from its
original location to A[2] (start)
◮ what case is this?
Complexity of Insertion-Sort (3)
The worst-case complexity is when the inner loop is executed exactly
j − 1 times, so

T (n) = 1 + 2 + 3 + · · · + j − 1

Where T (n) is number of steps taken by inner loop


Complexity of Insertion-Sort (3)
The worst-case complexity is when the inner loop is executed exactly
j − 1 times, so

T (n) = 1 + 2 + 3 + · · · + j − 1

Where T (n) is number of steps taken by inner loop


T (n) is similar to the sum of first n natural numbers. So with some
basic arithmetic it solves to

n(n − 1)
T (n) =
2

T (n) = Θ(n2 )
Complexity of Insertion-Sort (3)
The worst-case complexity is when the inner loop is executed exactly
j − 1 times, so

T (n) = 1 + 2 + 3 + · · · + j − 1

Where T (n) is number of steps taken by inner loop


T (n) is similar to the sum of first n natural numbers. So with some
basic arithmetic it solves to

n(n − 1)
T (n) =
2

T (n) = Θ(n2 )

The outer loop contributes Θ(n) operations only


Complexity of Insertion-Sort (3)
The worst-case complexity is when the inner loop is executed exactly
j − 1 times, so

T (n) = 1 + 2 + 3 + · · · + j − 1

Where T (n) is number of steps taken by inner loop


T (n) is similar to the sum of first n natural numbers. So with some
basic arithmetic it solves to

n(n − 1)
T (n) =
2

T (n) = Θ(n2 )

The outer loop contributes Θ(n) operations only

So overall time complexity becomes T (n) = Θ(n2 )


Correctness
Correctness
Does Insertion-Sort terminate for all valid inputs?
Correctness
Does Insertion-Sort terminate for all valid inputs?

If so, does it satisfy the conditions of the sorting problem?


◮ A contains a permutation of the initial value of A
◮ A is sorted: A[1] ≤ A[2] ≤ · · · ≤ A[length(A)]
Correctness
Does Insertion-Sort terminate for all valid inputs?

If so, does it satisfy the conditions of the sorting problem?


◮ A contains a permutation of the initial value of A
◮ A is sorted: A[1] ≤ A[2] ≤ · · · ≤ A[length(A)]

We want a formal proof of correctness


◮ does not seem straightforward. . .
Insertion Sort: How It Works
Goal: Sort the array by building a sorted subarray incrementally.
Analogy: Like sorting a hand of playing cards.
Insertion Sort: How It Works
Goal: Sort the array by building a sorted subarray incrementally.
Analogy: Like sorting a hand of playing cards.

Steps:
1. Start with the first element as a ”sorted” subarray.
Insertion Sort: How It Works
Goal: Sort the array by building a sorted subarray incrementally.
Analogy: Like sorting a hand of playing cards.

Steps:
1. Start with the first element as a ”sorted” subarray.
2. For each subsequent element (from position 2 to end):
Insertion Sort: How It Works
Goal: Sort the array by building a sorted subarray incrementally.
Analogy: Like sorting a hand of playing cards.

Steps:
1. Start with the first element as a ”sorted” subarray.
2. For each subsequent element (from position 2 to end):

◮ Slide it backward until it is in the correct position.

5 3 4 1 2
Example: Inserting 3, then 4, then 1, then 2.
Formal Proof
Mathematical proofs, show why mathematical hypothsis e.g. ( there
are ∞ prime numbers) are true.
Formal Proof
Mathematical proofs, show why mathematical hypothsis e.g. ( there
are ∞ prime numbers) are true.

They work by using basic mathematical axioms and previously


proved facts.
A number of techniques for it:
Formal Proof
Mathematical proofs, show why mathematical hypothsis e.g. ( there
are ∞ prime numbers) are true.

They work by using basic mathematical axioms and previously


proved facts.
A number of techniques for it:
◮ Proof by construction
Formal Proof
Mathematical proofs, show why mathematical hypothsis e.g. ( there
are ∞ prime numbers) are true.

They work by using basic mathematical axioms and previously


proved facts.
A number of techniques for it:
◮ Proof by construction

◮ Proof by induction ( we will use today)


Formal Proof
Mathematical proofs, show why mathematical hypothsis e.g. ( there
are ∞ prime numbers) are true.

They work by using basic mathematical axioms and previously


proved facts.
A number of techniques for it:
◮ Proof by construction

◮ Proof by induction ( we will use today)

◮ Proof by contradiction aka reductio ad absurdum (Latin: reducing to


absurdity)
Todays technique: Proof by Induction
Proof by Induction (Part 1)
Theorem: For all positive integers n,

X
n
n(n + 1)
i=
2
i=1
Proof by Induction (Part 1)
Theorem: For all positive integers n,

X
n
n(n + 1)
i=
2
i=1

Base Case (n=1):


LHS: 1
1(1+1)
RHS: 2 =1
Proof by Induction (Part 1)
Theorem: For all positive integers n,

X
n
n(n + 1)
i=
2
i=1

Base Case (n=1):


LHS: 1
1(1+1)
RHS: 2 =1

Base case verified: LHS = RHS X


Proof by Induction (Part 2)
Inductive Hypothesis:

Assume true for n = k:


X
k
k(k + 1)
i=
2
i=1
Proof by Induction (Part 2)
Inductive Hypothesis:

Assume true for n = k:


X
k
k(k + 1)
i=
2
i=1

Goal: Prove for n = k + 1:

X
k+1
(k + 1)(k + 2)
i=
2
i=1
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1

k(k + 1)
= + (k + 1) (by hypothesis)
2
k(k + 1) + 2(k + 1)
=
2
(k + 1)(k + 2)
=
2
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1

k(k + 1)
= + (k + 1) (by hypothesis)
2
k(k + 1) + 2(k + 1)
=
2
(k + 1)(k + 2)
=
2
∴ LHS = RHS for n = k + 1 X
Proof by Induction (Part 3)
Inductive Step:

Starting with LHS:


X
k+1
i = (1 + 2 + 3 + 4 + · · · + k) + (k + 1)
i=1

k(k + 1)
= + (k + 1) (by hypothesis)
2
k(k + 1) + 2(k + 1)
=
2
(k + 1)(k + 2)
=
2
∴ LHS = RHS for n = k + 1 X
Conclusion: Formula holds for all n ∈ N
◮ as induction step and base case both are TRUE
Back to Insertion Sort
Correctness: Inductive Argument
1. Base Case:
◮ When processing i = 2, the subarray A[1..1] is trivially sorted.

2. Inductive Step:
◮ Assume A[1..i − 1] is sorted before processing i.
Correctness: Inductive Argument
1. Base Case:
◮ When processing i = 2, the subarray A[1..1] is trivially sorted.

2. Inductive Step:
◮ Assume A[1..i − 1] is sorted before processing i.
◮ Insert A[i] by swapping it leftward until:

◮ It is larger than its predecessor, or


Correctness: Inductive Argument
1. Base Case:
◮ When processing i = 2, the subarray A[1..1] is trivially sorted.

2. Inductive Step:
◮ Assume A[1..i − 1] is sorted before processing i.
◮ Insert A[i] by swapping it leftward until:

◮ It is larger than its predecessor, or

◮ It reaches the start of the array.

◮ Now A[1..i] is sorted.


Correctness: Inductive Argument
1. Base Case:
◮ When processing i = 2, the subarray A[1..1] is trivially sorted.

2. Inductive Step:
◮ Assume A[1..i − 1] is sorted before processing i.
◮ Insert A[i] by swapping it leftward until:

◮ It is larger than its predecessor, or

◮ It reaches the start of the array.

◮ Now A[1..i] is sorted.

3. Conclusion: After processing all i, the entire array is sorted.


Example: Step-by-Step Sorting
Initial Array: [5, 3, 4, 1, 2]
1. Insert 3 (at i = 2): [3, 5, 4, 1, 2]
2. Insert 4 (at i = 3): [3, 4, 5, 1, 2]
3. Insert 1 (at i = 4): [1, 3, 4, 5, 2]
4. Insert 2 (at i = 5): [1, 2, 3, 4, 5]

Key Observation: Each insertion maintains the sortedness of the left


subarray.

You might also like