Design and Analysis of Algorithm

Greedy Methods
(Activity Selection Algorithm, Task
Scheduling, Huffman Coding )

Lecture – 24 - 26
 Overview

• A greedy algorithm always makes the choice that

looks best at the moment. (i.e. it makes a locally
optimal choice in the hope that this choice will
lead to a globally optimal solution).

• The objective of this section is to explores

optimization problems that are solvable by greedy
algorithms.
 Greedy Algorithm
• In mathematics, computer science and
economics, an optimization problem is the
problem of finding the best solution from all
feasible solutions.
• Algorithms for optimization problems typically go
through a sequence of steps, with a set of
choices at each step.
• Many optimization problems can be solved using
a greedy approach.
• Greedy algorithms are simple and
straightforward.
 Greedy Algorithm
• A greedy algorithm always makes the choice that
looks best at the moment.
• That is, it makes a locally optimal choice in the
hope that this choice will lead to a globally
optimal solution.
• Greedy algorithms do not always yield optimal
solutions, but for many problems they do.
• This algorithms are easy to invent, easy to
implement and most of the time provides best
and optimized solution.
 Greedy Algorithm
• Application of Greedy Algorithm:
• A simple but nontrivial problem, the
activity-selection problem, for which a greedy
algorithm efficiently computes a solution.
• In combinatorics,(a branch of mathematics), a
‘matroid’ is a structure that abstracts and
generalizes the notion of linear independence in
vector spaces. Greedy algorithm always produces
an optimal solution for such problems. Scheduling
unit-time tasks with deadlines and penalties is an
example of such problem.
 Greedy Algorithm
• Application of Greedy Algorithm:
• An important application of greedy techniques is
the design of data-compression codes (i.e.
Huffman code) .
• The greedy method is quite powerful and works
well for a wide range of problems. They are:
• Minimum-spanning-tree algorithms
(Example: Prims and Kruskal algorithm)
• Single Source Shortest Path.
(Example: Dijkstra's and Bellman ford algorithm)
 Greedy Algorithm
• Application of Greedy Algorithm:
• A problem exhibits optimal substructure if an
optimal solution to the problem contains within it
optimal solutions to subproblems.
• This property is a key ingredient of assessing the
applicability of dynamic programming as well as
greedy algorithms.
• The subtleties between the above two techniques
are illustrated with the help of two variants of a
classical optimization problem known as knapsack
problem. These variants are:
• 0-1 knapsack problem (Dynamic Programming)
• Fractional knapsack problem (Greedy Algorithm)
Greedy Algorithm
Greedy Algorithm
 Greedy Algorithm
• Problem 1: An activity-selection problem
Example 1: Given 10 activities with their start and
finish time compute a schedule where the largest
number of activities take place.

Activity
1 2 3 4 7 8 9 9 11 12
3 5 4 7 10 9 11 13 12 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
First arraigning the following activities in increasing
order on their finishing
Activity
1 2 3 4 7 8 9 9 11 12
3 5 4 7 10 9 11 13 12 14

Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
1 3 2 4 8 7 9 11 9 12
3 4 5 7 9 10 11 12 13 14

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Example 2: Find the optimal set in the given activity
selection problem.
Activity
1 2 3 4 7 8 9 9 11 12
5 3 4 6 7 8 11 10 12 13
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
First arraigning the following activities in increasing
order on their finishing
Activity
1 2 3 4 7 8 9 9 11 12
5 3 4 6 7 8 11 10 12 13

Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 1: An activity-selection problem
Solution:
Activity
2 3 1 4 5 6 8 7 9 10
3 4 5 6 7 8 10 11 12 13

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Greedy Algorithm
Greedy Algorithm
Greedy Algorithm
Greedy Algorithm
 Greedy Algorithm
• Problem 2: Task Scheduling problem
• An interesting problem that are solving using
matroids is the problem of optimally scheduling
unit-time tasks on a single processor, where each
task has a deadline, along with a penalty paid if
the task misses its deadline.
• This problem looks complicated, when it was
solved in a surprisingly simple manner by casting
it as a matroid and using a greedy algorithm.
 Greedy Algorithm
• Problem 2: Task Scheduling problem
• A unit-time task is a job, such as a program to be
run on a computer, that requires exactly one unit
of time to complete.
• Given a finite set S of unit-time tasks, a schedule
for S is a permutation of S specifying the order in
which to perform these tasks.
• The first task in the schedule begins at time 0 and
finishes at time 1, the second task begins at time
1 and finishes at time 2, and so on..
Greedy Algorithm
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 1: Find an optimal schedule from the
following table, where the tasks with
penalties(weight) and deadlines are given.
Task
4 2 4 3 1 4 6
70 60 50 40 30 20 10
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 1:
Solution: As per the greedy algorithm, first sort the
tasks in descending order of their penalties. So that
minimum penalties will be charged.

4 2 4 3 1 4 6
70 60 50 40 30 20 10

(Note: In this problem the tasks are already sorted)

 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

Can’t place

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

Can’t place

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Solution:

4 2 4 3 1 4 6
70 60 50 40 30 20 10

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 Greedy Algorithm

4 2 4 3 1 4 6
70 60 50 40 30 20 10

Gantt Chart

0 1 2 3 4 5 6
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 2: Find an optimal schedule from the
following table, where the tasks with
penalties(weight) and deadlines are given.
Task
2 1 2 1
100 10 15 27
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 2:
Solution: As per the greedy algorithm, first sort the
tasks in descending order of their penalties. So that
minimum penalties will be charged.
Task
2 1 2 1
100 10 15 27
Task
2 1 2 1
After Sorting
100 27 15 10
 Greedy Algorithm
Task
2 1 2 1
100 27 15 10

Gantt Chart Can’t place

0 1 2 3 4 5 6
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 3: Find an optimal schedule from the
following table, where the tasks with
penalties(weight) and deadlines are given.
Task
1 3 3 4 1 2 1
3 5 18 20 6 1 38
 Greedy Algorithm
• Problem 2: Task Scheduling problem
Example 3:
Solution: As per the greedy algorithm, first sort the
tasks in descending order of their penalties. So that
minimum penalties will be charged.
Task
1 3 3 4 1 2 1
3 5 18 20 6 1 38

Task
1 4 3 1 3 1 2
38 20 18 6 5 3 1
 Greedy Algorithm

Task
1 4 3 1 3 1 2
38 20 18 6 5 3 1

Can’t place
Gantt Chart

0 1 2 3 4
 Greedy Algorithm
• Problem 3: Huffman Coding (A solution to
encoding problem)
Problem:
Suppose we have a 58,000 characters data file that we
wish to store compactly. The characters occurred with
frequencies on that file is given below:
Character a e i o u s t
Frequency
10 15 12 3 4 13 1
(in thousand)
Fixed length
000 001 010 011 100 101 110
codeword (3-bit)
It was observed that if we assigned 3-bit to each
character, we required 1,74,000 bit to encode the file.
 Greedy Algorithm
• Problem 3: Huffman Coding
• Huffman Coding is a famous Greedy Algorithm.
• It is used for the lossless compression of data.
• It uses variable length encoding.
• It assigns variable length code to all the characters.
• The code length of a character depends on how
frequently it occurs in the given text.
• The character which occurs most frequently gets the
smallest code.
• The character which occurs least frequently gets the
largest code.
• It is also known as Huffman Encoding
 Greedy Algorithm
• Problem 3: Huffman Coding
• Huffman Coding implements a rule known as a prefix
rule.
• This is to prevent the ambiguities while decoding.
• It ensures that the code assigned to any character is
not a prefix of the code assigned to any other
character.
 Greedy Algorithm
• Problem 3: Huffman Coding
• Major Steps in Huffman Coding-
• There are two major steps in Huffman Coding-
1. Building a Huffman Tree from the input
characters.
2. Assigning code to the characters by traversing the
Huffman Tree.
 Greedy Algorithm
• Problem 3: Huffman Coding
1. Building a Huffman Tree from the input characters.
The steps involved in the construction of Huffman Tree
are as follows-
Step-01:
• Create a leaf node for each character of the text.
• Leaf node of a character contains the occurring
frequency of that character.
Step-02:
• Arrange all the nodes in increasing order of their
frequency value.
 Greedy Algorithm
• Problem 3: Huffman Coding
1. Building a Huffman Tree from the input characters.
Step-03:
• Considering the first two nodes having minimum
frequency,
• Create a new internal node.
• The frequency of this new node is the sum of
frequency of those two nodes.
• Make the first node as a left child and the other
node as a right child of the newly created node.
Step-04:
• Keep repeating Step-02 and Step-03 until all the
nodes form a single tree.
The tree finally obtained is the desired Huffman Tree.
 Greedy Algorithm
• Problem 3: Huffman Coding
1. Building a Huffman Tree from the input characters.
Time Complexity-
The time complexity analysis of Huffman Coding is as follows-
• extractMin( ) is called 2 x (n-1) times if there are n
nodes.
• As extractMin( ) calls minHeapify( ), it takes O(logn)
time.

Thus, Overall time complexity of Huffman Coding becomes

O(nlogn).
[Note: Here, n is the number of unique characters in the given text.]
 Greedy Algorithm
• Problem 3: Huffman Coding
2. Assigning code to the characters by traversing the
Huffman Tree.
• Assign weight to all the edges of the constructed Huffman
Tree.
• Let us assign weight ‘0’ to the left edges and weight ‘1’ to
the right edges.
 Greedy Algorithm
• Problem 3: Huffman Coding
2. Assigning code to the characters by traversing the
Huffman Tree.
• Assign weight to all the edges of the constructed Huffman
Tree.
• Let us assign weight ‘0’ to the left edges and weight ‘1’ to
the right edges.
• Rule
• If you assign weight ‘0’ to the left edges, then assign weight ‘1’ to the
right edges.
• If you assign weight ‘1’ to the left edges, then assign weight ‘0’ to the
right edges.
• Any of the above two conventions may be followed.
• But follow the same convention at the time of decoding that is
adopted at the time of encoding.
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
A file contains the following characters with the frequencies as shown. If
Huffman Coding is used for data compression, determine-
• Huffman Code for each character
• Average code length
• Length of Huffman encoded message (in bits)

Characters Frequencies
a 10
e 15
i 12
o 3
u 4
s 13
t 1
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
• First let us construct the Huffman Tree.
• Huffman Tree is constructed in the following steps-

Step 1:

1 3 4 10 12 13 15
t o u a i s e
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 2:

4 4 10 12 13 15
u a i s e

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 3:

8 10 12 13 15
a i s e

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 4:

18 12 13 15
i s e

8 10
a

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 6:

12 13 15 18
i s e

8 10
a

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 7:
25 15 18
e

12 13 8 10
i s a

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 8:
15 18 25
e

8 10 12 13
a i s
4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 9: 33 25

15 18 12 13
e i s
8 10
a

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
Step 10: 25 33

12 13 15 18
i s e

8 10
a

4 4
u

1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
58
Step 10:
25 33

12 15 18
13
i e
s
8 10
a
4 4
u
1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Now, as per
the rule assign
Solution weight ‘0’ to
58 the left edges
Step 10:
and weight ‘1’
25 33 to the right
edges.
12 15 18
13
i e
s
8 10
a
4 4
u
1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution
0 58 1
Step 10:
25 33
0 1 0 1

12 15 18
13
i e 0 1
s
8 10
0 1 a
4 4
0 1 u
1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1-
Solution Now, answer
0 58 1 all the
Step 10:
questions
25 33
0 1 0 1

12 15 18
13
i e 0 1
s
8 10
0 1 a
4 4
0 1 u
1 3
t o
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Question 1: Huffman Code for each character.
Solution Characters Frequencies Huffman Code
58 a 10 111
0 1

25 33 e 15 10
0 0 1
1 i 12 00
12 15 18
13
e 1
o 3 11001
i s 0
u 4 1101
8 10
0 1 a s 13 01
4 4 t 1 11000
0 1 u

1 3
t o

Huffman Tree
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Question 1: Huffman Code for each character.
Solution Characters Frequencies Huffman Code
58 a 10 111
0 1

25 33 e 15 10
0 0 1
1 i 12 00
12 15 18
13
e 1
o 3 11001
i s 0
u 4 1101
8 10
0 1 a s 13 01
4 4 t 1 11000
0 1 u
Observation:
1 3 • Characters occurring less frequently in the text are
t o assigned the larger code.
• Characters occurring more frequently in the text are
Huffman Tree assigned the smaller code.
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Question 2: Average code length
Solution .
Characters Frequencies Huffman Code
58 a 10 111
0 1
e 15 10
25 33
0 1 i 12 00
0
1 o 3 11001
12 15 18
13 u 4 1101
e 0 1
i s
s 13 01
8 10
t 1 11000
0 1 a

4 4
0 1 u

1 3
t o

Huffman Tree
 Greedy Algorithm
• Problem 3: Huffman Coding
Question 3: Length of Huffman encoded
Example 1-
message (in bits)
Solution Characters Frequencies Huffman Code

0
58
1
. a 10 111
e 15 10
25 33
0 1 i 12 00
0
1 o 3 11001
12 15 18
13 u 4 1101
e 0 1
i s
s 13 01
8 10
t 1 11000
0 1 a

4 4
Total number of bits in Huffman encoded message
0 1 u = Total number of characters in the message x Average code
length per character
1 3 = 58 x 2.52
t o = 146.16
Huffman Tree ≅ 147 bits
 Greedy Algorithm
• Problem 3: Huffman Coding (Algorithm)
HUFFMAN(C)
1 n ← |C|
2 Q←C
3 for i 1 to n - 1
4 do allocate a new node z
5 left[z] ← x ← EXTRACT-MIN (Q)
6 right[z] ← y ← EXTRACT-MIN (Q)
7 f [z] ← f [x] + f [y]
8 INSERT(Q, z)
9 return EXTRACT-MIN(Q)
 Greedy Algorithm
• Problem 3: Huffman Coding (Algorithm)
HUFFMAN(C)
1 n ← |C|
2 Q←C
3 for i 1 to n - 1
4 do allocate a new node z
5 left[z] ← x ← EXTRACT-MIN (Q)
6 right[z] ← y ← EXTRACT-MIN (Q)
7 f [z] ← f [x] + f [y]
8 INSERT(Q, z)
9 return EXTRACT-MIN(Q)

The total running time of HUFFMAN on a set of n characters is O (n lg n).

 Greedy Algorithm
• Problem 3: Huffman Coding
Example 2-
A file contains the following characters with the frequencies as shown. If
Huffman Coding is used for data compression, determine-
• Huffman Code for each character
• Average code length
• Length of Huffman encoded message (in bits)

Characters Frequencies
a 45
b 13
c 12
d 16
e 9
f 5
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 2- Question 1: Huffman Code for each character.
Solution Characters Frequencies Huffman Code
0
100
1
a 45 0
55
b 13 101
45 0 1
a c 12 100
25 30
d 16 111
0 0 1
1
e 9 1101
12 13 14 16
c b 1 d f 5 1100
0
5 9
e
f

Huffman Tree
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Question 1: Huffman Code for each character.
Solution Characters Frequencies Huffman Code
0
100
1
a 45 0
55
b 13 101
45 0 1
a c 12 100
25 30
d 16 111
0 0 1
1
e 9 1101
12 13 14 16
c b 1 d f 5 1100
0
5 9
e Observation:
f
• Characters occurring less frequently in the text are
Huffman Tree assigned the larger code.
• Characters occurring more frequently in the text are
assigned the smaller code.
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 1- Question 2: Average code length
Solution .
Characters Frequencies Huffman Code
0
100
1
a 45 0
55
b 13 101
45 0 1
a c 12 100
25 30
d 16 111
0 0 1
1
e 9 1101
12 13 14 16
c b 1 d f 5 1100
0
5 9
e
f

Huffman Tree
 Greedy Algorithm
• Problem 3: Huffman Coding
Question 3: Length of Huffman encoded
Example 1-
message (in bits)
Solution Characters Frequencies Huffman Code
.
0
100
1
a 45 0
55
b 13 101
45 0 1
a c 12 100
25 30
d 16 111
0 0 1
1
e 9 1101
12 13 14 16
c b 1 d f 5 1100
0 Total number of bits in Huffman encoded message
5 9 = Total number of characters in the message x Average code
e
f length per character
= 100 x 2.24
Huffman Tree
= 224 bits
 Greedy Algorithm
• Problem 3: Huffman Coding
Example 3- (Practice yourself)
A file contains the following characters with the frequencies as shown. If
Huffman Coding is used for data compression, determine-
• Huffman Code for each character
• Average code length
• Length of Huffman encoded message (in bits)

Characters a b c d e f g h

Frequencies 1 1 2 3 5 8 13 21