Lecture three

Recurrences
 Recurrences and Running
Time
An equation or inequality that describes a function

in terms of its value on smaller inputs. Such as

T(n) = T(n-1) + n
Recurrences arise when an algorithm contains

recursive calls to itself

What is the actual running time of the algorithm?

Need to solve the recurrence

Find an explicit formula of the expression
Bound the recurrence by an expression that

2 involves n
 Example Recurrences
T(n) = T(n-1) + n Θ(n2)
Recursive algorithm that loops through the
input to eliminate one item
T(n) = T(n/2) + c Θ(lgn)
Recursive algorithm that halves the input in
one step
T(n) = T(n/2) + n Θ(n)
Recursive algorithm that halves the input but
must examine every item in the input
T(n) = 2T(n/2) + 1 Θ(n)
Recursive algorithm that splits the input into 2
halves and does a constant amount of other
work
3
Recurrent Algorithms
BINARY-SEARCH
 for an ordered array A, finds if x is in the array
A[lo…hi]

Alg.: BINARY-SEARCH (A, lo, hi, x)

1 2 3 4 5 6 7 8

2 3 5 7 9 10 11 12
if (lo > hi)
return FALSE mid
lo hi
mid  (lo+hi)/2
if x = A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
4
Example
 A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
lo = 1 hi = 8 x=7
1 2 3 4 5 6 7 8

1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8

5 6 7 8

1 2 3 4 5 7 9 11 mid = 6, A[mid] = x
Found!

5
Another Example
 A[8] = {1, 2, 3, 4, 5, 7, 9, 11}

– lo = 1 hi = 8 x=6
1 2 3 4 5 6 7 8

1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8
low high

1 2 3 4 5 7 9 11 mid = 6, A[6] = 7, lo = 5, hi = 8
low high
1 2 3 4 5 7 9 11 mid = 5, A[5] = 5, lo = 5, hi = 5
NOT FOUND!

1 2 3 4 5 7 9 11
high low

6
 Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi) constant time: c1
return FALSE
mid  (lo+hi)/2 constant time: c2
if x = A[mid] constant time: c3
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi,same
x) problem of size n/2

 T(n) = c +T(n/2)
 T(n) – running time for an array of size n
7
 Methods for Solving Recurrences

Iteration method

Substitution method

Recursion tree method

Master method

8
 The Iteration Method
Convert the recurrence into a summation

and try to bound it using known series

Iterate the recurrence until the initial

condition is reached.
Require more algebraic knowledge.
In the iteration method we iteratively

the recurrence until we see the pattern .

9
 The Iteration Method
T(n) = c + T(n/2)
T(n) = c + T(n/2) T(n/2) = c + T(n/4)
= c + c + T(n/4) T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + … + c + T(1)
k times
= clgn + T(1)
= Θ(lgn)

10
 Iteration Method – Example
T(n) = n + 2T(n/2) Assume: n =
T(n) = n + 2T(n/2) 2k
T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
… = in + 2iT(n/2i)
= kn + 2kT(1)
= nlgn + nT(1) = Θ(nlgn)

11
 The substitution method

1- Guess a solution

2- Use induction to prove that the

solution works

12
 The substitution method

 n
T (n) aT    f (n)
b

13
 Substitution method
 Guess a solution
 T(n) = O(g(n))
 Induction goal: apply the definition of the

asymptotic notation
(strong induction)
 T(n) ≤ c g(n), for some c > 0 and n ≥ n0

 Induction hypothesis: T(k) ≤ c g(k) for all k < n

 Prove the induction goal

 Use the induction hypothesis to find some

values of the constants c and n0 for which the

induction goal holds
14
 Example 1: Binary Search

T(n) = c + T(n/2)
 Guess: T(n) = O(lgn)
 Induction goal: T(n) ≤ d lgn, for some d

and n ≥ n0
 Induction hypothesis: T(n/2) ≤ d lg(n/2)

 Proof of induction goal:

T(n) = T(n/2) + c ≤ d lg(n/2) + c

= d lgn – d + c ≤ d lgn
15 if: – d + c ≤ 0, d ≥ c
 Example 2
T(n) = T(n-1) + n
 Guess: T(n) = O(n2)

 Induction goal: T(n) ≤ c n2, for some c and n ≥

n0
 Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k <

n
 Proof of induction goal:

T(n) = T(n-1) + n ≤ c (n-1)2 + n

= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0  c ≥ n/(2n-1)  c ≥ 1/(2 – 1/n)
16  For n ≥ 1  2 – 1/n ≥ 1  any c ≥ 1 will work
 Example 3
T(n) = 2T(n/2) + n
 Guess: T(n) = O(nlgn)
 Induction goal: T(n) ≤ cn lgn, for some c and n

≥ n0
 Induction hypothesis: T(n/2) ≤ cn/2 lg(n/2)

 Proof of induction goal:

T(n) = 2T(n/2) + n ≤ 2c (n/2)lg(n/2) + n

= cn lgn – cn + n ≤ cn lgn
if: - cn + n ≤ 0  c ≥ 1
17
Changing variables
T(n) = 2T( n ) + lgn
Rename: m = lgn  n = 2m

T (2m) = 2T(2m/2) + m
Rename: S(m) = T(2m)

S(m) = 2S(m/2) + m  S(m) = O(mlgm)

(demonstrated before)
T(n) = T(2m) = S(m) = O(mlgm)=O(lgnlglgn)
Idea: transform the recurrence to one that
you have seen before
18
The recursion-tree method

Convert the recurrence into a tree:

 Each node represents the cost incurred

at various levels of recursion

 Sum up the costs of all levels

Used to “guess” a solution for the recurrence

19
 Example
W(n) = 2W(n/2) + n2

20
 Example
W(n) = 2W(n/2) + n2

21
 Example

22
Master’s method
 “used for solving recurrences of the form:

 n
T (n) aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Idea: compare f(n) with nlogba

f(n) is asymptotically smaller or larger than

nlogba by a polynomial factor n


f(n) is asymptotically equal with
23 nlogba
 Master’s method
 n
 T (n) aT    f (n) where, a ≥ 1, b > 1,
and f(n) > 0  b 

Case 1: if f(n) = O(nlogba -) for some  > 0, then:

T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all

regularity condition
sufficiently large n, then:T(n) = (f(n))
24
Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

 f(n) = O(n1-) Case 1

 T(n) = (n)
25
Examples (cont.)

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)
26
Examples (cont.)
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
 f(n) = (n1+) Case 3  verify regularity
cond.
a f(n/b) ≤ c f(n)
 2 n2/4 ≤ c n2  c = ½ is a solution (c<1)
 T(n) = (n2)
27