Design & Analysis

of Algorithms
EID305

Module-2: Greedy Method

1
 Agenda of Today’s
Session
1.The General Method
2. Knapsack problem
3. Job sequencing with deadlines
4. Minimum cost spanning trees
5. Optimal storage on tapes
6. Single source shortest paths

2
Greedy Method
The greedy method is perhaps the most straight forward design
technique
 The problems have n inputs and require to obtain a subset that
satisfies some constraints.
 A solution that satisfy the constraints of a problem is known as
feasible solution
 A feasible solution that either maximize or minimize a given
objective function is known as optimal solution.

3
General idea:

Given a problem with n inputs, we are required to obtain a

subset that maximizes or minimizes a given objective function
subject to some constraints.

Feasible solution — any subset that satisfies some

constraints

Optimal solution — a feasible solution that maximizes

or minimizes the objective
function

4
procedure Greedy (A, n)
begin
solution  Ø;
for i  1 to n do
x  Select (A); // based on the objective
// function
if Feasible (solution, x),
then solution  Union (solution, x);
end;
Select: A greedy procedure, based on a given objective function,
which selects input from A, removes it and assigns its value to x.

Feasible: A boolean function to decide if x can be included into

solution vector (without violating any given constraint).
5
General Algorithm

6
 The function ‘Select’ selects an
input from a[ ] and removes it.
 The selected input's value is
assigned to x.
 Feasible is a Boolean-valued
function that determines whether
x can be included into the
solution vector.
 The function Union combines x
with the solution and updates the
7

objective function.
KNAPSACKPROBLEM

 There are given n objects and a knapsack or bag.

Object i has a weight wi and the Knapsack has a
capacity m.
 If a fraction xi, 0 <xi < 1,of object i is placed into
the knapsack, then a profit of pixi is earned.
 The objective is to obtain a filling of the knapsack
that maximizes the total profit earned.
 Since the knapsack capacity is m, we require the
total weight of all chosen objects to be at most m.

8
An optimal solution is a feasible
solution for which is maximized.

9
Algorithm Greedy Knapsack(m,n)

10
 Example
Let n = 3, M = 20,
(p1 p2 p3) = (25, 24, 15)
(w1 w2 w3) = (18, 15, 10)
Feasible Solutions

(x1, x2, x3) ∑ni=1 wixi ∑ni=1 pixi

1 (1/2, 1/3,1/4) 16.5 24.25

2 (1, 2/15,0) 20 28.2
3 (0, 1,1/2) 20 31.5

11
Strategy 1 : maximise objective function

Put the object with the greatest profit in the knapsack.

Then use a fraction of the last object to fill the knapsack to

capacity.
n

Strategy does not yield an optimal solution.

Capacity was quickly exhausted which con- strained the profit

attained.

12
Strategy 2 : maximise capacity
Choose objects according to least weight.
The idea is that we will get more objects into the knapsack and
potentially more profit !

 Solution is still not optimal

 Rate of increase of profit was not high enough

13
Strategy 3: balancing profit and capacity
Find the object to include by maximum profit per unit of capacity, i.e.,
compute pi/wi
Then choose objects starting with the largest and working to smallest
ratio.

Solution is optimal !
Achieves a balance between rate at which profit increases with the
rate at which the capacity is used.

14
JOBSEQUENCINGWITH
DEADLINES
a set of n job are given.
 Associated with job i is an integer deadline di >0 and a profit pi >0.
 For any job i the profit pi is earned iff the job is completed by its
deadline.
 To complete a job, one has to process the job on a machine for one
unit of time.
 A feasible solution for this problem is a subset J of jobs such that
each job in this subset can be completed by its deadline.

15
16
17
High level description of Job sequencing
Algorithm.

18
19
Let n=5, {p1,p2,p3,p4,p5} = {60,100,20,40,20} and
(d1,d2,d3,d4,d5} = {2,1,3,2,1}. Find the Optimal solution.

Jobs J2 J1 J4 J3 J5
Profits 100 60 40 20 20
Deadlines 1 2 2 3 1

20
J Assigned Slots Jobs Considered Action
Profit
Φ none 2 assign to[0,1] 0
{2} [0,1] 1 assign to[1,2] 100
{2, 1} [0,1][1,2] 4 assign fit, reject
100+60
{2, 1} [0,1][1,2] 3 assign to[2,3]
100+60
{2, 1,3} [0,1][1,2][2,3] 5 Reject
100+60+20

The optimal solution is J={2,1,3} with a profit of 180.

21
 TREE

A
 Connected acyclic graph
B C

E  Tree with n nodes

D F
contains exactly n-1
edges.
GRAPH
 Graph with n nodes
contains less than or equal
to n(n-1)/2 edges.
22
 SPANNING TREE...
Suppose you have a connected undirected
graph
 Connected: every node is reachable from
every other node
 Undirected: edges do not have an
associated direction
...then a spanning tree of the graph is a
connected subgraph in which there are no
cycles

A connected, Four of the spanning trees of the graph

undirected 23
graph
EXAMPLE.
.

24
 MINIMUM SPANNING TREE

Let G = (N, A) be a connected, undirected graph

where N is the set of nodes and A is the set of edges.
Each edge has a given nonnegative length. The
problem is to find a subset T of the edges of G such
that all the nodes remain connected when only the
edges in T are used, and the sum of the lengths of
the edges in T is as small as possible possible. Since
G is
connected, at least one solution must exist.

25
 Finding Spanning Trees
Applications
• They are helpful in routing applications.
• Communication Networks.
• They can be used to obtain circuit equations for an
electric network.

• There are two basic algorithms for finding minimum-

cost spanning trees, and both are greedy algorithms

• Kruskal’s algorithm:
Created in 1957 by Joseph Kruskal

• Prim’s algorithm
Created by Robert C. Prim 26
 Kruskal’s Algorithm
List the edges
in order of
B 5 size:
C

3 ED 2
4 AB 3
8 6
AE 4
CD 4
8
BC 5
A D
7 F EF 5
CF 6
5 AF 7
4 BF 8
2
CF 8

E 27
 Kruskal’s Algorithm

B 5
C Select the
shortest edge in
3 the network
4
8 6
ED 2
8
A D
7 F

5
4
2

E 28
 Kruskal’s Algorithm

B 5
C
Select the
next
3
4 shortest
8 6 edge which does
not create a cycle
8 ED
A D 2
7 F
AB
5 3
4
2

E 29
Kruskal’s Algorithm
B 5
C
Select the
next
3
4 shortest
8 6
edge which does
not create a cycle
8
ED 2
A D
7 F AB 3
CD 4 (or AE
5 4)
4
2

E 30
 Kruskal’s Algorithm

B 5
C Select the next
shortest edge which
3 does not create a
4
8 6 cycle

8 E
A D
7 F

5
A
4
2

E 31
C
 Kruskal’s Algorithm

B 5
C
Select the next
shortest edge which
3
4 does not create a
8 6 cycle
ED 2
8 AB 3
A D CD
7 F
4
5 AE 4
4 BC 5 – forms a
2 cycle EF 5

E 32
 Kruskal’s Algorithm

B 5 All vertices have

been connected.
C
3 The solution
4
8 6 is
ED
8 2
A D AB
7 F
3
5 CD
4 4
2 AE
4
Total weight of
EF tree:
5 18
E 33
34
Example

35
36
 Prim’s Algorithm

B 5
C Select any
vertex
3 A
4
8 6
Select the
8 shortest edge
A D connected to that
7 F
vertex
5
AB 3
4
2

E 37
 Prim’s Algorithm

B 5
C
Select the
3 shortest edge
4 connected to any
8 6
vertex already
connected.
8
A D
7 F AE 4

5
4
2

E 38
 Prim’s Algorithm

B 5
C

3 Select the
4
8 6 shortest edge
connected to any
8 vertex already
A D connected.
7 F
ED 2
5
4
2

E 39
 Prim’s Algorithm

B 5
C
Select the
3 shortest edge
4 connected to any
8 6
vertex already
connected.
8
A D
7 F DC 4

5
4
2

E 40
 Prim’s Algorithm

B 5
C
Select the
3 shortest edge
4 connected to any
8 6
vertex already
8 connected.
A EF
F D
7 5

5
4
2

E 41
 Prim’s Algorithm

B 5 All vertices have

been connected.
C
3
4 The solution
8 6 is

8 AB 3
A D AE 4
7 F
ED 2
5 DC 4
4 EF 5
2
Total weight of
tree: 18
E 42
43
Example

44
 Example

8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

45
 Solution

8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

46
 Minimum Connector Algorithms
Kruskal’s algorithm
Prim’s algorithm
1. Select the shortest edge
1. Select any vertex
in a network
2. Select the shortest
edge connected to
that vertex 2. Select the next shortest
edge which does not create
a cycle
3. Select the shortest
edge connected to
any vertex already 3. Repeat step 2 until
connected all vertices have
been connected
4. Repeat step 3 until
all vertices have
been connected 47
OPTIMAL STORAGE ON TAPES

 There are n programs that are to be stored on a computer tape of

length I
 Associated with each program i is a length li, 1<= i < =n.
 All programs can be stored on the tape if and only if the sum of
the lengths of the programs is at most I.
 In tape, each program is retrieved sequentially.
 Let Tj be the time required to retrieve program.
 If all programs are retrieved equally often, then the expected or
mea retrieval time (MRT) is given by
which should be minimum.

48
 Optimal Storage on Tapes

The problem:

Given n programs to be stored on tape, the lengths of

these n programs are l1, l2 , . . . , ln respectively.
Suppose the programs are stored in the order
of i1, i2 , . . . , in

Let tj be the time to retrieve program ij.

Assume that the tape is initially positioned at the beginning.

tj is proportional to the sum of all lengths of programs stored in

front of the program ij. 49
 1 n
The goal is to minimize MRT (Mean Retrieval Time),  t j
n j
n j 1
i.e. want to minimize   li
j 1 k 1
k

Ex: n 3, (l1 , l 2 , l 3 ) (5,10,3)

There are n! = 6 possible orderings for storing them.
orde total retrieval MRT
r time
1 1 2 35+(5+10)+(5+10+3 38/3
2 1 3 2 )=38 31/3
3 2 1 35+(5+3)+(5+3+10) 43/3
4 2 3 1 =31 41/3
5 3 1 210+(10+5)+(10+5+ 29/3 Smallest
6 3 2 1 3)=43 34/3
10+(10+3)+(10+3+
Note: The problem can be solved using greedy strategy,
5)=41
just always let the shortest program goes first.
3+(3+5)+(3+5+10)
( Can simply get the right order by=29using any sorting algorithm)
3+(3+10)+(3+10+5 50

)=34
 Analysis:
Try all combination: O( n! )
Shortest-length-First Greedy method: O (nlogn)

Shortest-length-First Greedy method:

Sort the programs s.t. l1 l2 . . . ln
and call this ordering L.

Next is to show that the ordering L is the best

Proof by contradiction:
Suppose Greedy ordering L is not optimal, then there
exists some other permutation I that is optimal.
I = (i1, i2, … in)  a < b, s.t. lia  lib (otherwise I = L)
51
Interchange ia and ib in and call the new list I :
I x

… ia ia+1 ia+2 … ib …

I SWAP
… ib ia+1 ia+2 … ia …

x
In I, Program ia+1 will take less (lia- lib) time than in I to be retrieved
In fact, each program ia+1 , …, ib-1 will take less (lia- lib) time
For ib, the retrieval time decreases x + lia
For ia, the retrieval time increases x + lib
totalRT ( I )  totalRT ( I ) (b  a  1)(lia  lib )  ( x  lia )  ( x  lib )
(b  a )(lia  lib )  0 ( )
Contradiction!!
Therefore, greedy ordering L is optimal 52
53
 When they are stored on the tape in this order the MRT is
minimized.
The time taken to build the required permutation is O(n!).

54
 Assigning Programs to tapes
Find an optimal placement for 13 programs on three tapes To,T1,and
T2,where the program are of lengths 12,5,8,32,7,5,18,26,4,3,11,10
and 6.
Step1: Arrange all the files in ascending order according to their
length.
If ascending order is:
3,4,5,5,6,7,8,10,11,12,18,26,32.
By using the algorithm store the following programs are assigned to
the tapes.

55
Tape-0 : 3,5,8,12,32 =
3+(3+5)+(3+5+8)+(3+5+8+12)+(3+5+8+12+32)
Tape-1: 4,6,10,18 = 4+(4+6)+(4+6+10)+(4+6+10+13)
Tape-2: 5,7,11,26 = 5+(5+7)+(5+7+11)+(5+7+11+26)

56
SINGLE SOURCE SHORTEST PATHS

Graph is used to represent the distance between two cities.

In single source shortest path problem the shortest distance from a
single vertex called source is obtained.
Let G(V,E) be a graph, the in single source shortest path, the
shortest paths from vertex V0 to all remaining vertex is determined.
The vertex V0 is then called as source and the last vertex is called
destination

57
 Single-Source Shortest Paths in DAGs

In a dag:
• Every path is a subsequence of the topologically sorted
vertex order
• If we do topological sort and process vertices in that
order
• We will process each path in forward order
 Never relax edges out of a vertex until have
processed all edges into the vertex
• Thus, just 1 pass is sufficient
58
Example

59
Example

60
Example

61
Example

62
Example

63
Example

64
 Single-Source Shortest Paths in DAGs
DAG-SHORTEST PATHS(G, s)
TOPOLOGICALLY-SORT the vertices of G
INIT(G, s)
for each vertex u taken in topologically sorted order
do
for each v  Adj[u] do
RELAX(u, v)

65
66
Greedy Algorithm to generate Shortest
paths

67
68
69
70
 ou!
n k y
T h a

71