Divide and Conquer

Divide and Conquer

 Divide: Break the problem into smaller
subproblems
 Conquer: Solve the subproblems. Generally,

this involves waiting for the problem to be

small enough that it is trivial to solve (i.e. 1 or
2 items)
 Combine: Given the results of the solved

subproblems, combine them to generate a

solution for the complete problem
Divide and Conquer: some
thoughts
 Often, the sub-problem is the same as the
original problem
 Dividing the problem in half frequently, does

the job
 May have to get creative about how the data

is split
 Splitting tends to generate run times with log

n in them
Divide and Conquer: Sorting
 How should we split the data?
 What are the subproblems we need to solve?

 Howdo we combine the results from these

subproblems?
MergeSort
MergeSort: Merge
 Assuming L and R are sorted already, merge
the two to create a single sorted array
Merge
L: 1 3 5 8 R: 2 4 6 7
Merge
L: 1 3 5 8 R: 2 4 6 7

B:
Merge
L: 1 3 5 8 R: 2 4 6 7

B:
Merge
L: 1 3 5 8 R: 2 4 6 7

B:
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5 6
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5 6
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5 6 7
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5 6 7
Merge
L: 1 3 5 8 R: 2 4 6 7

B: 1 2 3 4 5 6 7 8
Merge
 Does the algorithm terminate?
Merge
 Is it correct?
 Loop invariant: At the end of each iteration of the for loop
of lines 4-10 the subarray B[1..k] contains the smallest k
elements from L and R in sorted order.
Merge
 Is it correct?
 Loop invariant: At the end of each iteration of the for loop
of lines 4-10 the subarray B[1..k] contains the smallest k
elements from L and R in sorted order.
Merge
 Running time?
Merge
 Running time? Θ(n) - linear
Merge-Sort: Divide
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4
 Merge-Sort: Divide
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

8 5 1 3
 Merge-Sort: Divide
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

8 5 1 3

8 5
 Merge-Sort: Conquer
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

8 5 1 3

Sorting a list of
8 5 one element is
easy
 Merge-Sort: Combine
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

5 8 1 3

8 5
 Merge-Sort: Divide
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

5 8 1 3

1 3
 Merge-Sort: Divide
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

5 8 1 3

1 3
 Merge-Sort: Merge
8 5 1 3 6 2 7 4

8 5 1 3 6 2 7 4

5 8 1 3

1 3
 Merge-Sort: Merge
8 5 1 3 6 2 7 4

1 3 5 8 6 2 7 4

5 8 1 3
Merge-Sort: …
8 5 1 3 6 2 7 4

1 3 5 8 6 2 7 4

…
Merge-Sort: …
8 5 1 3 6 2 7 4

1 3 5 8 2 4 6 7
Merge-Sort: Merge
1 2 3 4 5 6 7 8

1 3 5 8 2467

Done!
MergeSort
 Running time?

 c if n is small
T (n) 
2T (n / 2)  D(n)  C (n) otherwise

D(n): cost of splitting (dividing) the data

C(n): cost of merging/combining the data
MergeSort
 Running time?

 c if n is small
T (n) 
2T (n / 2)  D(n)  C (n) otherwise

D(n): cost of splitting (dividing) the data - linear Θ(n)

C(n): cost of merging/combining the data – linear Θ(n)
MergeSort
 Running time?

 c if n is small
T (n) 
2T (n / 2)  cn otherwise

Which is?
MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn

T(n/2) T(n/2)
MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8)

MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8

…

c c c c c … c c c c c c
MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn cn

cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn

cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn

…

c c c c c … c c c c c c cn
MergeSort 
T (n) 
c if n is small
2T (n / 2)  cn otherwise
cn cn

cn/2 cn/2 cn

cn/4 cn/4 cn/4 cn/4 cn

cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn/8 cn

…

c c c c c … c c c c c c cn

Depth?
MergeSort
 We can calculate the depth, by determining when the
recursion is c
 At each level, we divide by 2

n
d
1
2
2 d n
log 2 d log n

d log 2 log n

d log 2 n
  c if n is small
T (n) 
MergeSort 2T (n / 2)  cn otherwise

 Running time?

 Each level costs cn

 log n levels

 cn log n = Θ(n log n )

Binary Search
 Problem: Find element ‘x’ in an Array A.
 What is the simplest way to do this?
 Just scan the list, examining each element one by
one.
 Takes Θ(n) time
 Canwe do better?
 What if the list is sorted?
Binary Search
 Assume the list is sorted. (e.g. lets say we are
searching for 11 in array A shown below).

1 3 4 9 10 11 12 18
20
1. Look at the middle element of the list i.e. A[5] = 10
2. Is 10 what we are searching for? (NO!!)
 We want 11, which is larger than 10
3. Repeat on A[6…9]
Binary Search
 The Binary Search Algorithm returns the
index of ‘x’ if it is present in A.
 Otherwise returns null.

BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p q

BinSearch(A,1,9,11
)
p=1
q=9

BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,1,9,11
)
p=1
q=9
k=5
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,1,9,11
)
p=1
q=9
k=5
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,1,9,11
)
p=1
q=9
k=5
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p q

BinSearch(A,6,9,11
)
p=6
q=9

BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,6,9,11
)
p=6
q=9
k=7
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,6,9,11
)
p=6
q=9
k=7
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

p k q

BinSearch(A,6,9,11
)
p=6
q=9
k=7
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

pq

BinSearch(A,6,6,11
)
p=6
q=6

BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

pkq

BinSearch(A,6,6,11
)
p=6
q=6
k=6
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

pkq

BinSearch(A,6,6,11
)
p=6
q=6
k=6
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
 Binary Search
1 3 4 9 10 11 12 18 20

pkq

BinSearch(A,6,6,11
)
p=6
q=6
k=6 FOUND !!
Return 6 as the index of 11
BinSearch(A,p,q,x)
1. if p≤q
2. k (p+q)/2
3. if A[k] = x
4. return k
5. else if A[k] < x
6. return BinSearch(A,k+1,q,x)
7. else return BinSearch(A,p,k-1,x)
8. else return null
Binary Search
 Running time recurrence?
 T(n) = T(n/2) + O(1)
 Solution?
 T(n) = Θ (logn)
 You can do the solution using the tree
methods we studied.
 Formal Solution techniques coming up in the

next chapter.