Design and Analysis

of
Advanced Algorithm

Md. Manowarul Islam

Assistant Professor
Dept. Of CSE
Jagannath University, Bangladesh
 Greedy Algorithm
• Greedy algorithms make the choice that looks
best at the moment.
• This locally optimal choice may lead to a globally
optimal solution (i.e. an optimal solution to the
entire problem).

2
When can we use Greedy algorithms?

We can use a greedy algorithm when the following are true:

1) The greedy choice property: A globally optimal solution

can be arrived at by making a locally optimal (greedy) choice.

2) The optimal substructure property: The optimal solution

contains within its optimal solutions to subproblems.

3
 Example of Greedy Algorithm
• Fractional Knapsack
• Huffman Coding
• Activity Selection Problem
• Minimum Spanning Tree
• Dijkstra’s Shortest Path Algorithm

4
 Example: Making Change
• Instance: amount (in cents) to return to customer
• Problem: do this using fewest number of coins
• Example:
– Assume that we have an unlimited number of coins of
various denominations:
– 1c , 5c, 10c, 25c, 1$
– Objective: Pay out a given sum $5 with the smallest
number of coins possible.

5
 Example: Making Change
• Given 30 cents, and coins {1, 5, 10, 25}
• Here is what a casher will do: always go with coins of
highest value first
• Choose the coin with highest value 25
–1
• Now we have 5 cents left
–1
– The solution is: 2 coins

6
 The Coin Changing Problem
• Assume that we have an unlimited number of coins of various
denominations
• Objective: Pay out a given sum S with the smallest number of
coins possible.

• The greedy coin changing algorithm:

while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;

7
 Example: Making Change

Do your self : Change 97 taka

Available coin: {1,2,5,10,20 }
How many coins?
while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;
S Condition C num Results
S=97 1 20 97/20=4 4 of 20

8
 Example: Making Change

Do your self : Change 97 taka

Available coin: {1,2,5,10,20 }
How many coins?
while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;
S Condition C num Results
S=97 1 20 97/20=4 4 of 20
S=97-80=17 1 10 17/10=1 1 of 10

9
 Example: Making Change

Do your self : Change 97 taka

Available coin: {1,2,5,10,20 }
How many coins?
while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;
S Condition C num Results
S=97 1 20 97/20=4 4 of 20
S=97-80=17 1 10 17/10=1 1 of 10
S=17-10=7 1 5 7/5=1 1 of 5

10
 Example: Making Change

Do your self : Change 97 taka

Available coin: {1,2,5,10,20 }
How many coins?
while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;
S Condition C num Results
S=97 1 20 97/20=4 4 of 20
S=97-80=17 1 10 17/10=1 1 of 10
S=17-10=7 1 5 7/5=1 1 of 5
S=7-5=2 1 2 2/2=1 1 of 2

11
 Does Greedy Always Work?
• US postal denominations: 1, 10, 21, 34, 70, 100,
350, 1225, 1500.
– Ex. 140¢.
– Greedy: 100, 34, 1, 1, 1, 1, 1, 1.
– Optimal: 70, 70.

12
Example: Making Change

13
 Making Change – A big problem
• Example 2: Coins are valued $.30, $.20, $.05,
$.01
– Does not have greedy-choice property, since $.40 is
best made with two $.20’s, but the greedy solution will
pick three coins (which ones?)

14
 An Activity Selection Problem
(Conference Scheduling Problem)

• Input: A set of activities S = {a1,…, an}

• Each activity has start time and a finish time
– ai=(si, fi)
• Two activities are compatible if and only if their
interval does not overlap
• Output: a maximum-size subset of mutually
compatible activities

15
 Compatible

• Two activities are compatible if and only if their

interval does not overlap

16
 The Activity Selection Problem
Input: list of time-intervals L

Output: a non-overlapping subset S of the intervals

Objective: maximize |S| 3,7

2,4
5,8
6,9
1,11
10,12
0,3 17
 The Activity Selection Problem
Input: list of time-intervals L

Output: a non-overlapping subset S of the

intervals
3,7
Objective: maximize |S| 2,4
5,8
6,9
1,11
Answer = 3 10,12
0,3 18
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity a
3. remove all activities conflicting with a
4. repeat

19
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity “a”
3. remove all activities conflicting with “a”
4. repeat

20
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity “a”
3. remove all activities conflicting with “a”
4. repeat

21
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

22
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

23
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

24
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

25
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

26
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

27
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

28
 The Activity Selection Problem
• Here are a set of start and finish times

• What is the maximum number of activities that can be

completed?
• {a3, a9, a11} can be completed
• But so can {a1, a4, a8’ a11} which is a larger set
• But it is not unique, consider {a2, a4, a9’ a11}

29
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Interval Representation

Not Observed yet

Added in optimal Solution

Removed from the list

34
 Early Finish Greedy
• Select the activity with the earliest finish
• Eliminate the activities that could not be
scheduled
• Repeat!
Not Observed yet

Added in optimal Solution

Removed from the list
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3615
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3715
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3815
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3915
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 4015
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 4115
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 4215
 Algorithm

• Input: A set of activities S = {a1,…, an}

• Each activity has start time and a finish time
– ai=(si, fi)
• Two activities are compatible if and only if their interval does not overlap
• Output: a maximum-size subset of mutually compatible activities

43
 Why it is Greedy?
• Greedy in the sense that it leaves as much
opportunity as possible for the remaining
activities to be scheduled
• The greedy choice is the one that maximizes the
amount of unscheduled time remaining
 The Fractional Knapsack Problem
A thief robbing a store finds n items.
ith item: worth bi dollars
wi pounds
W, wi, bi are integers.
He can carry at most W pounds.
He can take fractions of items.

?
 The Fractional Knapsack Problem
Problems exhibits the optimal-substructure property:

Consider the most

valuable load that
weighs at most W
pounds.

If jth item is the remaining load must be

removed the most valuable load
from his weighting at most W-wj that
load, he can take from the n-1
original items excluding j.

=> Can be solved by dynamic programming

 The Fractional Knapsack Problem
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with weight at
most W.
• If we are allowed to take fractional amounts, then this is the fractional
knapsack problem.
– In this case, we let xi denote the amount we take of item i

– Objective: maximize

b (x / w )
iS
i i i
– Constraint:

x
iS
i W ,0  xi  wi

47
 The Fractional Knapsack Algorithm
• Greedy choice: Keep taking item with highest value (benefit to
weight ratio)
– Since  b ( x / w )  (b / w ) x
iS
i i i
iS
i i i

Algorithm fractionalKnapsack(S, W)
Input: set S of items w/ benefit bi and weight wi; max. weight W
Output: amount xi of each item i to maximize benefit w/ weight at most W

for each item i in S

xi  0
vi  bi / wi {value}
w0 {total weight}
while w < W
remove item i with highest vi
xi  min{wi , W - w}
w  w + min{wi , W - w} 48
 Example
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with total weight at
most W.

“knapsack”

Solution:
Items: P
1 2 3 4 5 • 1 ml of 5
50$
Weight: 4 ml 8 ml 2 ml 6 ml 1 ml
• 2 ml of 3
Benefit: $12 $32 $40 $30 $50 10 ml 40$
Value: 3 4 20 5 50 • 6 ml of 4
($ per ml) 30$
• 1 ml of 49
2
4$
 Example
W=50.

Itemitem 1 2 3
Weight 10 20 30
benifit 60 100 120
Value

for each item i in S

xi  0
vi  bi / wi {value}
w0 {total weight}
while w < W
remove item i with highest vi
xi  min{wi , W - w}
w  w + min{wi , W - w}
50
 Example-solution
W=50.

Itemitem 1 2 3
Weight 10 20 30
benifit 60 100 120
Value 6 5 4

for each item i in S

xi  0 Item weight value

vi  bi / wi {value} 1 10 60
w0 {total weight} 2 20 100
while w < W 3 20(2/3) 80
remove item i with highest vi Total 240
xi  min{wi , W - w}
w  w + min{wi , W - w}
51