Greedy Paradigm

Greedy Paradigm
• Greedy approaches are applicable to optimization problems
• An optimization problem is the problem of finding the best solution from all
feasible solutions
• An optimization problem is associated with an objective function and a set of
constraints. The solution to a problem is to create a subset of solutions that
satisfies constraints associated with the problem.
• Any subset solution that satisfies constraints is called a feasible solution
• Any feasible solution that maximizes or minimizes the given objective function
is called the optimal solution

Examples:
• Finding the shortest path : we wish to find the path with lowest weight
• Making coin change
• Storing files in a compressed form in a disk
• Optimally loading a suitcase with items of different weights
• Optimally arranging a shelf
 Greedy Paradigm
• The greedy approach works in stages.
• At every stage, a decision or choice is made
• These decisions are locally optimal
• Finally, the global solution of the problem is
obtained by combining locally optimal
decisions.
 Greedy Paradigm
• A greedy algorithm always makes the choice
that looks best at the moment.
• it makes a locally optimal choice in the hope
that this choice will lead to a globally optimal
solution
• The greedy method is quite powerful and
works well for a wide range of problems.
Comnponents of greedy algorithm
• Objective function
• Generating multiple candidate solutions
• Selection procedure
• Feasibility check
• Solution check
 Greedy Paradigm
Applications of the greedy method, including
• minimum-spanning-tree algorithms
• Dijkstra’s algorithm for shortest paths from a
single source ,
• Chv´atal’s greedy set-covering heuristic
 Algorithm greedy(A,n)
Input: Array a[1…n]
Output: solution of the problem
SolutionSet=Null
while(Solution is not complete) do
Select a best candidate solution x
If x is a feasible solution then
Add the solution x to the Solutionset
end if
%%Check if the solution of the given problem is obtained
If(solution obtained) then
return Solutionset
end if
end while
end
Important properties of a greedy algorithm

• Optimum substructure
A problem has an optimal substructure if an
optimal solution to the entire problem contains
the optimal solutions to the sub-problems
• Greedy Choice property
A global (overall) optimal solution can be reached
by choosing the optimal choice at each step
 An Activity Selection Problem
• The problem is to schedule several competing activities that require exclusive use of a
common resource, with a goal of selecting a maximum-size set of mutually compatible
activities.
• Suppose we have a set of n proposed activities that wish to use a
resource, such as a lecture hall, which can serve only one activity at a time.
• Each activity has a start time and a finish time , where
• If selected, activity takes place during the half-open time interval

• Activities and are compatible if the intervals

• do not overlap. That is, and are compatible if and

• In the activity-selection problem, we wish to select a maximum-size subset of mutually

compatible activities.

• Assume the activities are sorted in monotonically increasing order of finish time:
 Greedy Paradigm
• An algorithm to solve the activity-selection problem does
not need to work bottom-up, like a table-based dynamic-
programming algorithm.
• It works top-down, choosing an activity to put into the
optimal solution and then solving the sub problem of
choosing activities from those that are compatible with
those already chosen.
• Greedy algorithms typically have this top-down design:
make a choice and then solve a sub problem, rather than
the bottom-up technique of solving subproblems before
making a choice
The Activity Selection Problem
A recursive greedy algorithm

Time complexity O(n)

An iterative greedy algorithm

The maximum finish time of any activity in A

 Greedy Paradigm
• Time complexity O(n)
 Elements of the greedy strategy

• Determine the optimal substructure of the problem.

• Develop a recursive solution
• Show that if we make the greedy choice, then only one
subproblem remains
• Prove that it is always safe to make the greedy choice.
(Steps 3 and 4 can occur in either order.)
• Develop a recursive algorithm that implements the
greedy strategy.
• Convert the recursive algorithm to an iterative
algorithm.
 0-1 knapsack problem
• A thief robbing a store finds n items
• The ith item is worth vi dollars and weighs wi
pounds.
where vi and wi are integers
• The thief wants to take as valuable a load as
possible,
• but he can carry at most W pounds in his knapsack,
for some integer W
• Which items should he take?
0-1 knapsack problem
 0-1 knapsack problem
• This example has 3 items and a knapsack that
can hold 50 pounds.
• Item 1 weighs 10 pounds and is worth 60
dollars
• Item 2 weighs 20 pounds and is worth 100
dollars
• Item 3 weighs 30 pounds and is worth 120
dollars
 0-1 knapsack problem
• For the 0-1 problem, consider the most
valuable load that weighs at most W pounds.
• If we remove item j from this load, the
remaining load must be the most valuable
load weighing at most W- wj that the thief can
take from the n-1 original items excluding j
 fractional knapsack problem
• the thief can take fractions of items, rather
than having to make a binary (0-1) choice for
each item
• Both knapsack problems exhibit the optimal-
substructure property
 fractional knapsack problem
• For the comparable fractional problem,
consider that if we remove a weight w of one
item j from the optimal load, the remaining
load must be the most valuable load weighing
at most W- w that the thief can take from the
n- 1 original items plus wj - w pounds of item j.
 fractional knapsack problem

• The fractional knapsack problem can be solved by a greedy

strategy, but the 0-1 problem cannot solved by greedy
strategy.
To solve the fractional problem,
• we first compute the value per pound vi / wi for each item
• According to greedy strategy, the thief begins by taking as
much as possible of the item with the greatest value per
pound
• If the supply of that item is exhausted and he can still carry
more, he takes as much as possible of the item with the next
greatest value per pound, and so forth, until he reaches his
weight limit W
Thus, by sorting the items by value per pound, the greedy
algorithm runs in O(n logn) time.
• This example has 3 items and a knapsack that can hold 50 pounds.
• Item 1 weighs 10 pounds and is worth 60 dollars
• Item 2 weighs 20 pounds and is worth 100 dollars
• Item 3 weighs 30 pounds and is worth 120 dollars

• Thus, the value per pound of item 1 is 6 dollars per pound, which is
greater than the value per pound of either item 2 (5 dollars per pound)
or item 3 (4 dollars per pound)

• The greedy strategy, therefore, would take item 1 first.

• however, the optimal solution takes items 2 and 3, leaving item 1
behind.
• The two possible solutions that take item 1 are both suboptimal
• To see that this greedy strategy does not work
for the 0-1 knapsack problem,
• consider the problem instance illustrated in
Figure (a)
• For the comparable fractional problem, however, the
greedy strategy, which takes item 1 first, does yield an
optimal solution(fig.c)
• Taking item 1 doesn’t work in the 0-1 problem because the
thief is unable to fill his knapsack to capacity, and the empty
space lowers the effective value per pound of his load.
• In the 0-1 problem, when we consider whether to include
an item in the knapsack, we must compare the solution to
the subproblem that includes the item with the solution to
the subproblem that excludes the item before we can make
the choice
Matroids and greedy methods
 Matroids and greedy methods
• This theory describes many situations in which
the greedy method yields optimal solutions.
• It involves combinatorial structures known as
“matroids.”
 Matroids
A matroid is an ordered pair M=(S, I) satisfying the following
conditions
 Matroids
• The word “matroid” is due to Hassler
Whitney. He was studying matric matroids,
in which the elements of S are the rows of a
given matrix and a set of rows is
independent if they are linearly independent
in the usual sense.
 Matroids
• As another example of matroids, consider the
graphic matroid defined in terms
of a given undirected graph G =(V,E) as follows
 Graphic Matroid
• Graphic Matroid

The graphic matroid is closely related to

the minimum-spanning-tree problem
 Graphic Matroid
• Given a matroid M =(S,I), an element an
extension of A, that is, x is an extension of A if

• In a graphic matroid , If A is an
independent set of edges, then edge e is an
extension of A if and only if e is not in A (
) and the addition of e to A does not create a
cycle
 Maximal Independent Set
• If A is an independent subset in a matroid M,
A is said to be maximal if it has no extensions.
That is, A is maximal if it is not contained in
any larger independent subset of M.
• All maximal independent subsets in a matroid
have the same size
 Spanning Tree
• is a graphic matroid for a connected, undirected graph
G,
• Every maximal independent subset of must be a free
tree with exactly |V |-1 edges that connects all the vertices
of G
• Such a tree is called a spanning tree of G
• A matroid M =(S,I) is weighted if it is associated with a
weight function w that assigns a strictly positive weight
w(x) to each element
• The weight function w extends to subsets of S by
summation:

For example, if w(e) denote the weight of an edge e in a graphic matroid , then
w(A) is the total weight of the edges in edge set A
Greedy algorithms on a weighted matroid

finding a maximum-weight independent subset in a

weighted matroid
– For a weighted matroid M =(S, I), aim is to find an
independent set such that w(A) is maximized
– such a subset is independent and has maximum
possible weight, is an optimal subset of the matroid.
Because the weight w(x) of any element is
positive,
– an optimal subset is always a maximal independent
subset—it always helps to make A as large as possible.
 Minimum-spanning-tree problem
• G =(V,E) is a connected undirected graph, and a
length function w such that w(e) is the (positive)
length of edge e.
• Aim is to find a subset of the edges that connects all
of the vertices together and has minimum total length
• This problem can be viewed as finding an optimal
subset of a matroid,
• ,the weighted matroid with weight function ,
where and is larger than the
maximum length of any edge.
Minimum-spanning-tree problem
• In weighted matroid, all weights are positive
and an optimal subset is a spanning tree of
minimum total length in the original graph
• More specifically, each maximal independent
subset A corresponds to a spanning tree with
|V|-1 edges, and since
Minimum-spanning-tree problem
Minimum-spanning-tree problem
• for any maximal independent subset A, an
independent subset that maximizes the
quantity must minimize w(A)
• Thus, any algorithm that can find an optimal
subset A in an arbitrary matroid can solve the
minimum-spanning-tree problem.
 a greedy algorithm that works for any
weighted matroid
• The algorithm takes as input a weighted matroid
M =(S,I) with an associated positive weight
function w, and it returns an optimal subset A
• In pseudocode, the components of M are denoted
by M.S and M.I and the weight function by w.
• The algorithm is greedy because it considers in
turn each element , in order of monotonically
decreasing weight, and immediately adds it to the
set A being accumulated if is independent
 MST greedy algorithm for any weighted
matroid

The running time of GREEDY is easy to analyze.

Let n denote |S|.
The sorting phase of GREEDY takes time O(n lg n)
Line 4 executes exactly n times, once for each element of S.
Each execution of line 4 requires a check on whether or not
the set is independent.
If each such check takes time O(f (n)), the entire
algorithm runs in time O(n lg n+ nf (n))