COMP 3711 Design and Analysis of Algorithms

Tutorial 2: Divide & Conquer

 Divide-and-Conquer Revision
Main idea of D&C: Solve a problem of size n by breaking it into one or more
smaller problems of size less than n. Solve the smaller problems recursively and
combine their solutions, to solve the large problem
Idea of binary search : Set q ← middle of the array. If x = A[q], return q.
If x < A[q], search A[1..q-1]. If x > A[q], search A[q+1..n]

2
 Solving Recurrence
Expansion method: Recursion tree method:
𝑇(𝑛) = 𝑇 𝑛/2 +2
= 𝑇 𝑛/22 + 2 +2
= 𝑇 𝑛/22 +4
= .….
= 𝑇 𝑛/2𝑖 + 2i
= ….
= 𝑇 𝑛/2log2 𝑛 + 2log 2 𝑛
= 𝑇 1 + 2log 2 𝑛
= 𝟏 + 𝟐𝐥𝐨𝐠𝟐 𝐧

For 𝑛 > 1, 𝑇 𝑛 = 𝑇 𝑛/2 + 2, 𝑎𝑛𝑑 𝑇 1 =1

3
 Finding a “fixed point”
(Binary Search Application)

4
 Question
Input: a sorted array 𝐴[1. . 𝑛] of 𝑛 distinct integers
(Distinct means that there are no repetitions among the integers.
The integers can be positive, negative or both).

Problem:
Design an 𝑂(log 𝑛) algorithm to return an
index 𝑖 such that 𝐴 𝑖 = 𝑖, if such an 𝑖 exists.
Otherwise, report that no such index exists.

As an example, in the array immediately below, the algorithm returns 4

𝑖 1 2 3 4 5 6 7 8
𝐴[𝑖] −3 0 1 4 12 17 20 22

while in the array below, the algorithm returns that no such index exists.

𝑖 1 2 3 4 5 6 7 8
𝐴[𝑖] −3 0 1 7 12 17 20 22
5
 Solution
The main observation is that

(*) If 𝐴 𝑚 > 𝑚, then 𝐴 𝑖 > 𝑖 for all 𝑖 > 𝑚.

Follows directly from facts that the array is sorted and all integers are distinct.

More specifically, those two facts imply that 𝐴 𝑚 + 1 ≥ 𝐴 𝑚 + 1.

So, if 𝐴 𝑚 > 𝑚 => 𝐴 𝑚 + 1 ≥ 𝐴 𝑚 + 1 > 𝑚 + 1

Statement (*) follows by mathematical induction.

A similar proof shows that

(**) If 𝐴[𝑚] < 𝑚, then 𝐴[𝑖] < 𝑖 for all 𝑖 < 𝑚.

6
 Solution
Use observations that
(*) If 𝐴 𝑚 > 𝑚, then 𝐴 𝑖 > 𝑖 for all 𝑖 > 𝑚.
(**) If 𝐴[𝑚] < 𝑚, then 𝐴[𝑖] < 𝑖 for all 𝑖 < 𝑚.

• If 𝐴 𝑚 = 𝑚, we have found solution and can stop

• If 𝐴 𝑚 > 𝑚, from first observation,
if solution exists, it must be in 𝐴 1, 𝑚 − 1
• If 𝐴 𝑚 < 𝑚, from second observation
if solution exists, it must be in 𝐴 𝑚 + 1. . 𝑛

So, after checking 𝐴 𝑚 against 𝑚 (choose m to be middle item), can either

(i) stop or
(ii) throw away half of the array and solve the problem on the other half.

The running time of the algorithm has the recurrence

𝑇 𝑛 = 𝑇(𝑛Τ2) + 𝑂(1), which solves to 𝑇 𝑛 = 𝑂(log 𝑛).
7
 Solution from another perspective

Define array 𝐵[1 . . 𝑛] by setting 𝐵[𝑖] = 𝐴[𝑖] − 𝑖 for all 𝑖. For every
index 𝑖 we have

𝐵𝑖 = 𝐴𝑖 − 𝑖 ≤ 𝐴𝑖 + 1 − 1 − 𝑖 = 𝐴𝑖 + 1 − 𝑖 + 1 = 𝐵𝑖 + 1.

, where B is also in increasing order. Clearly, 𝐴[𝑖] = 𝑖 if and only if

𝐵[𝑖] = 0. By doing so, we convert the problem of finding 𝐴[𝑖] = 𝑖 to
performing a binary search of 0 in B.

𝑖 1 2 3 4 5 6 7 8
𝐴[𝑖] -3 0 1 4 12 17 20 22
𝐵[𝑖] -4 -2 -2 0 7 11 13 14

8
 Solution (Code)
INDEX-SEARCH(A,s,t) \\ first call is INDEX-SEARCH(A,1,n)
if 𝑠 > 𝑡 \\ 𝑂(1) termination condition (empty subarray)
return “no such index exists";
else \\ recurse
𝑠+𝑡
𝑚← 2
;

if 𝐴 𝑚 = 𝑚
return 𝑚; \\ 𝑂 1 only one item
else if 𝐴 𝑚 > 𝑚 \\ go left
𝑛
return INDEX-SEARCH(A,s,m-1) ; \\𝑇 2
else \\ go right
𝑛
return INDEX-SEARCH(A,m+1,t); \\𝑇 2

Note that algorithm ``assumes’’ 𝑠 ≤ 𝑡, i.e., that array 𝐴[𝑠 … 𝑡] is nonempty. Termination condition
(𝑠 > 𝑡) implies that search has ended without finding solution. Convince yourself that this is valid
by working through what happens when 𝑡 − 𝑠 = 0,1 in both successful and unsuccessful cases.
9
Majority Element

10
 Question
Let A[1. . n] be an array of n elements. A majority element of A is any element
occurring more than n/2 times (e.g., if n = 8, then a majority element should
occur at least 5 times). Your task is to design an algorithm that finds a majority
element, or reports that no such element exist

a) Suppose that you are not allowed to order the elements; the only way you
can access the elements is to check whether two elements are equal or
not. Design an 𝑂(𝑛 log 𝑛) time algorithm for this problem using divide and
conquer

The array below, 2 is the majority item

i 0 1 2 3 4 5 6 7
A[i] 2 3 2 1 2 5 2 2

The array below has no majority item

i 0 1 2 3 4 5 6 7
A[i] 2 3 5 1 2 5 2 2

11
 Solution

If A contains a majority element e,

e must be a majority element in at least one of A[1. . n/2] and A[n/2 + 1. . N].

Algorithm:
1. Find majority e1, if it exists, in A[1. . n/2]
2. Find majority e2, if it exists, in A [n/2 + 1. . n]
3. If e1 exists, count how many times it occurs in A
If more than n/2, report e1.
If e2 exists, count how many times it occurs in A.
If more than n/2, report e2.
If neither exist, report, “no majority”.

The base case is when n = 1, and we can simply return the only element as the
majority.

12
Closest Pair of Points

13
 Question

Given n points in the plane, find a pair with smallest Euclidean distance
between them.
Fundamental geometric primitive.
• Graphics, computer vision, geographic information systems, molecular
modelling, air traffic control.
• Special case of nearest neighbour, Euclidean MST, Voronoi.

Brute force: Check all pairs of points p and q with O 𝑛2 comparisons.

1-D version: O(n log n) easy if points are on a line.

Assumption: No two points have same x coordinate

14
 Solution
Algorithm.
• Divide: draw vertical line L so that roughly ½n points on each side.
• Conquer: find closest pair in each side recursively.
• Combine: find closest pair with one point in each side.
• Return best of 3 solutions.

15
 Solution

Find closest pair with one point in each side, assuming that distance < 𝛿.
• Observation: only need to consider points within 𝛿 of line L.
• Sort points in 2 𝛿 -strip by their y coordinate.
• Only check distances of those within 11 positions in sorted list

16
 Solution

𝑛
𝑇 𝑛 ≤ 2𝑇 + 𝑂(𝑛 log 𝑛) ⇒ 𝑇 𝑛 = 𝑂(𝑛 𝑙𝑜𝑔2 𝑛)
2
17
 O(n log n) Solution
Don't sort points in strip from scratch each time.
• Each recursive returns two lists: all points sorted by y coordinate, and all
points sorted by x coordinate.
• Sort by merging two pre-sorted lists.
ClosestPair (p1, p2, . . . , pn):
// The points are in sorted order by x.
// We also have the points in a different list ordered by y
If n ≤ 3 then solve and return the answer.
Let m = 𝑛/2
Let δ = min(ClosestPair(p1,……, pm), ClosestPair(pm+1,……, pn))
Form a list q of the points (sorted by increasing y) that are
within δ of the x coordinate of pm. Call these points q1, q2… qk

Now we compute di , the minimum distance between qi and all the qs

below it, for each 1 ≤ i ≤ k as follows:
di = min distance from qi to qi−1, qi−2,…, qj , where we stop when
j gets to 1, or the y coordinate gets too small: qj · (0, 1) < qi
· (0, 1) − δ

Return min(d1,……, dk, δ)

Reference: https://www.cs.cmu.edu/~15451-s20/lectures/lec23-closest-pair.pdf 18
Using Black Box Median
Algorithms

19
Black-box means that you can call the algorithm and use its result,
but can’t peer inside of it.

20
 Outline

(a) Give a simple linear-time algorithm that solves the selection

problem for an arbitrary order statistic.
(b) Solve the weighted median problem in 𝑂(𝑛) time.

21
(a) Give a simple linear-time algorithm that solves the selection
problem for an arbitrary order statistic.

Solution: Modify the randomized selection algorithm so that it is deterministic.

At every step, instead of choosing the pivot at random from the current subarray, it uses
the O(n) median finding algorithm to find the median and then uses the median as pivot.

At each step, the algorithm will reduce the size of the array in which it is searching by at
least one- half so the running time satisfies

22
 Outline

(a) Give a simple linear-time algorithm that solves the selection

problem for an arbitrary order statistic.
(b) Solve the weighted median problem in 𝑂(𝑛) time.

23
(b)Let 𝑥1, 𝑥2, … , 𝑥𝑛 be n distinct (unsorted) elements with
associated positive weights 𝑤1, 𝑤2, … , 𝑤𝑛 such that Σ𝑤𝑖 = 1.

Example:
1 2 3 4 5 6 7 8

4 8 5 12 11 10 15 6

.05 .1 .1 .2 .1 .15 .1 .2

24
(b)Let 𝑥1, 𝑥2, … , 𝑥𝑛 be n distinct (unsorted) elements with
associated positive weights 𝑤1, 𝑤2, … , 𝑤𝑛 such that Σ𝑤𝑖 = 1.

Show that if the items are not sorted you can still solve the problem in linear time
using the linear –time black box median finding algorithm as a subroutine.

Intuition: We will develop a linear time test (using the black box) that reduces the size of
the problem by a half each step. This immediately implies a linear time algorithm

25
Major observation is

26
Observation:
Proof:

27
From observation, none of the removed items were solution to the weighted median problem.

28