Mansoura University

Faculty of Computers and Information

Sciences
Department of Computer Science
First Semester- 2020-2021

Analysis and Design of Algorithms

Grade: THREE
Prof. Samir Elmougy
Chapter 4
Divide and Conquer (Part II)

Based on: Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest, “Introduction to Algorithms”, 3rd Edition, The MIT
Press, 2009.
 Last Lecture
◼ Review for Recurrence Relations
◼ Divide and Conquer Design Approach
◼ Binary Search
◼ Merge Sort
◼ Recurrence Relation for Merge Sort
◼ Analysis of Merge Sort Recurrence

Page 2
 Contents
◼ Recurrence Tree for Solving Recurrences
◼ Master Method for Solving Recurrences
◼ Iteration Method for Solving Recurrences
◼ Matrix Multiplication: Simple Divide and Conquer
◼ Strassen’s Algorithm for Matrix Multiplication

Page 3
 Recurrence Tree for Solving
Recurrences
• The number of nodes at depth d in
a perfect binary tree = 2d

• A perfect binary tree of height h has:

2h+1• −1 nodes
• Number of leaf nodes in a perfect binary
tree of height h = 2h
• Number of internal nodes in a perfect binary
tree of height h = 2h − 1

4
 Tree Termonology

◼ A=Root: No parent for this node

(Green Color) • Degree of a tree: the
◼ Siblings: the nodes that have the maximum # of this node.
same parent • Degree of a node: # of this
◼ leaf (External node): havs no node children
child (Orange Color)
◼ Internal node: a node with at least
one child (Green and Yellow) A
◼ Ancestors of a node: parent,
grandparent, grand-grandparent,
…. B C
◼ Descendant of a node: child,
grandchild, grand-grandchild, …..
◼ Depth of a node: # of ancestors D E
◼ Height of a tree: maximum depth
of any node in the tree (here it is 3)
I
 Recursion tree
•T(n) = 2T(n/2) + c.n where (T(1) = c)

Θ(n. log n).

 Page 38: Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

L1.7
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

T( n )

L1.8
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

T(n/2) T(n/2)

L1.9
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

L1.10
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

Q(1)

L1.11
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn

cn/2 cn/2

h = log n cn/4 cn/4 cn/4 cn/4

Q(1)

L1.12
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

cn/2 cn/2

h = log n cn/4 cn/4 cn/4 cn/4

Q(1)

L1.13
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

cn/2 cn/2 cn

h = log n cn/4 cn/4 cn/4 cn/4

Q(1)

L1.14
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

cn/2 cn/2 cn

h = log n cn/4 cn/4 cn/4 cn/4 cn

Q(1)

L1.15
 Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn

cn/2 cn/2 cn

h = log n cn/4 cn/4 cn/4 cn/4 cn

Q(1) Number of leaves=n Q(n)

L1.16
 Recursion tree
Solve T(n) = 2T(n/2) + c n, where c > 0 is constant.

cn cn

cn/2 cn/2 cn

h = log n cn/4 cn/4 cn/4 cn/4 cn

c Number of leaves = n cn

Or T(1) Total = c n log n

L1.17
= Q(n log n)
 Recursion Tree
A recursion tree for T(n)=T(n/3) + T(2n/3) + c n

Note that the tree here is not balanced: the longest path is
the rightmost one, and its length is log3/2 n 18
 4.5 P. 93- The Master Method for Solving Recurrences
◼ Theorem 4.1. (The Master Theorem): Let a ≥ 1 and b > 1
be constants, let f(n) be a function, and
- T(n) be defined on the nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n)
where we interpret n/b as ⌊n/b⌋ or ⌈n/b⌉.
Then T(n) can be bounded asymptotically as follows:

19
 The Master Theorem

Example P. 95: T(n) = 9 T(n/3)+ n

Answer: a =9, b =3, and f (n) = n
Is n = i.e. Is n = O (n 2 -  )
Yes (take ), so Case 1 implies that T(n) =  (n 2)
20
 The Master Theorem

Example P. 95: T(n) = T(2n/3)+ 1

Answer: a =1, b =3/2, and f (n) = 1
= f (n)
Then Apply Case 2 → T(n) =  (lg n)
21
 The Master Theorem

Example P. 96: T(n) = 2T(n/2)+  (n)

Answer: a =2, b =2, and f (n) =  (n)
=  (n) = f (n)
Then Case 2 could be applied, and so T(n) =  ( n lg n )
22
 The Master Theorem

Example P. 95: T(n) = 3T(n/4)+ n lg n

Answer: a =3, b =4, and f (n) = n lg n

i.e. Is f(n) =O (n 0.793 -  ) i.e. Is n lg n = O (n 0.793 -  )

23
 The Master Theorem

Example P. 95: T(n) = 3T(n/4)+ n lg n

Answer: a =3, b =4, and f (n) = n lg n

T(n) =  ( n lg n )
Then Case 1 and Case 2 can not be applied
Try Case 3
Case 3

24
 The Master Theorem

Example:
 The Master Theorem
T(n) = aT(n/b) + f (n) where f(n)  (nd), d  0

Master Theorem: If a < bd, T(n)  (nd)

If a = bd, T(n)  (nd log n)
If a > bd, T(n)  (nlog b a )
◼ Examples:
T(n) = T(n/2) + n  T(n)  (n )
Here a = 1, b = 2, d = 1, a < bd

T(n) = 2T(n/2) + 1  T(n)  (n log 2 2 ) = (n )

Here a = 2, b = 2, d = 0, a > bd

26
Solving Recurrence Relations: Iteration Method
◼ Convert the recursive relation into a summation
and try to bound it using known series

◼ Example:

Solve T(n) = T(n/2) + c, where T(1)=1

27
 Iteration Method

Solve T(n) = T(n/2) + c , where T(1)=1

Solution
T(n) = T(n/2) + c How long does it continue?
Ans: When T(n/2k) = T(1) i.e. when
T(n) = [T(n/4) + c ] + c n/2k =1
i.e. when k = log n
T(n) = T(n/4) + 2c

T(n) = [T(n/8) + c ] + 2 c T(n) = T(n/2k) + k c

T(n) = T(n/23) + 3c
T(n) = T(1) +( log n ). c
T(n) = [T(n/24) + c ] + 3 c
T(n) = Q (log n)
T(n) = T(n/24) + 4 c
 Iteration Method

Solve T(n) = 2 T(n/2) + n, where T(1)=1

Solution
T(n) = 2 T(n/2) + n How long does it continue?
Ans: When T(n/2k) = T(1) i.e.
T(n) =2 [2 T(n/4) + n/2] + n when n/2k =1
i.e. when k = log n
T(n) =22 T(n/4) + 2n

T(n) = 22[2T(n/8) + n/4] + 2 n T(n)=2k T(n/2k) + k n

T(n) = 23T(n/23) + 3n
T(n)=n.T(1)+( log n ). n
T(n) = 23[2T(n/24) + n/23] + 3 n
T(n) = Q n.(log n)
T(n) = 24T(n/24) +4n
 Iteration Method:
Analysis of Merge Sort Recurrence
T(n) = 2T(n/2) + cn for n >1 and T(1) = b
T(n) = 2T(n/2)+cn
T(n) = 2 [ 2T [(n/4) +c (n/2)] +cn
T(n) = 4T(n/4) +cn +cn
How long does it
T(n) = 8T(n/8)+cn+cn+cn
continue?
T(n) = 23T(n/(23))+3 cn Answer:
When T(n/2k) = T(1)
T(n) = 24T(n/(24))+4 cn
i.e. when n/2k =1
T(n) = 2kT(n/2k)+kcn i.e. when k = log n
When n/2k=1,
T(n) = n T(1) + cn log n
T(n) =  (n log n)
30
 4.2 Strassen’s algorithm for matrix multiplication P.75

Matrix Multiplication
◼ Compute the matrix product C = A x B, for two n x n

matrices

31
 Summary
◼ Recurrence Tree for Solving Recurrences
◼ Master Method for Solving Recurrences
◼ Iteration Method for Solving Recurrences
◼ Matrix Multiplication

Page 32
 Next Lecture
◼ Matrix Multiplication: Simple Divide and Conquer
◼ Strassen’s Algorithm for Matrix Multiplication
◼ Heap Sort Algorithms

Page 33