Insertion Sort

Insertion-Sort(A)
Chapter 2: Getting Started for j ← 2 to length(A) do
key ← A[j]
Insertion Sort i←j−1
while i > 0 and A[i] > key do
Mergesort A[i + 1] ← A[i]
Recurrences i←i−1
Integer Multiplication A[i + 1] ← key
CS 5633 Analysis of Algorithms Chapter 2: Slide – 2

Insertion Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Loop Invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Mergesort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Divide and Conquer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Recurrences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Loop Invariants
Multiplying Big Integers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7  A loop invariant specifies what is true just before the loop condition is
Recursive Version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 tested.
Faster Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
– The loop variable(s) should be used to describe the loop invariant.
– The loop invariant specifies the progress of the algorithm up to that
point.
 Insertion-Sort outer loop, loop variable j
Loop Invariant: A[1..j − 1] has been sorted.
 Insertion-Sort inner loop, loop variable i
Loop Invariant: key = old value of A[j],
A[i + 1..j − 1] has been shifted to A[i + 2..j], and key < all elements in
A[i + 2..j]
CS 5633 Analysis of Algorithms Chapter 2: Slide – 3

1 2
 Mergesort Recurrences
 The running time of a recursive algorithm can often be described by
Merge-Sort(A, p, r) recurrences.
if p < r then  T (n) is the running time on a problem of size n.
q ← ⌊(p + r)/2⌋  Merge Sort: 2 subproblems of size n/2 and linear time to merge.
Merge-Sort(A, p, q) 
Θ(1) if n = 1
Merge-Sort(A, q + 1, r) T (n) =
2T (n/2) + Θ(n) if n > 1
Merge(A, p, q, r)
 (exercise) Recursion tree for merge sort.
 (exercise) Insertion sort as a recursive algorithm.
CS 5633 Analysis of Algorithms Chapter 2: Slide – 4
CS 5633 Analysis of Algorithms Chapter 2: Slide – 6

Multiplying Big Integers

Divide and Conquer ✄ Bit-Multiply multiplies bit arrays A and B

 Divide the problem into subproblems. ✄ + and − denotes adding/subtracting bit arrays.
Divide n elements into 2 sets of n/2 elements. ✄ It is assumed that n is a power of 2.
 Conquer the subproblems by recursion. ✄ Multiplying times 2i is a bit shift.
Call Merge-Sort on the 2 sequences. Bit-Multiply(n, A, B)
 Combine the solutions to the subproblems. C ← an array of 2n zero bits
Merge 2 sorted sequences into 1 sorted sequence. for i ← 0 to n − 1 do
CS 5633 Analysis of Algorithms Chapter 2: Slide – 5
if B[i] = 1 then
C ← C + 2i · A
return C
CS 5633 Analysis of Algorithms Chapter 2: Slide – 7

3 4
 Recursive Version

Bit-Multiply(n, A, B)
C ← an array of 2n zero bits
if n = 1 then
C[0] ← A[0] xor B[0]
C[1] ← A[0] ∧ B[0]
return C
Divide A and B into 4 n/2 bit arrays.
✄ E.g., A = A1 + 2n/2 · A2
C ← C + Bit-Multiply(n/2, A1, B1)
C ← C + 2n/2 · Bit-Multiply(n/2, A1, B2)
C ← C + 2n/2 · Bit-Multiply(n/2, A2, B1)
C ← C + 2n · Bit-Multiply(n, A2, B2)
return C
CS 5633 Analysis of Algorithms Chapter 2: Slide – 8

Faster Multiplication
Bit-Multiply(n, A, B)
C ← an array of 2n zero bits
if n = 1 then C[0] ← A[0] xor B[0]
C[1] ← A[0] ∧ B[0]; return C
Divide A and B into 4 n/2 bit arrays.
✄ E.g., A = A1 + 2n/2 · A2
P1 ← Bit-Multiply(n/2, A1, B1)
C ← C + P1
P2 ← Bit-Multiply(n/2, A2, B2)
C ← C + 2n · P2
P3 ← Bit-Multiply(n/2, A1 − A2, B2 − B1 )
C ← C + 2n/2 · (P3 − P1 − P2 )
return C
CS 5633 Analysis of Algorithms Chapter 2: Slide – 9