Scribd
Upload
39 views2 pages

ASSIGNMENT

The document contains 23 questions related to information theory concepts like self information, average information, mutual information, entropy, information rate, redundancy, and probability. It asks the student to calculate entropy values and information rates for various communication sources and channels. It also contains probability questions about events like coin flips, ball selections from urns, and people being divided into groups.

Uploaded by

Mahesh J2000
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
39 views2 pages

ASSIGNMENT

The document contains 23 questions related to information theory concepts like self information, average information, mutual information, entropy, information rate, redundancy, and probability. It asks the student to calculate entropy values and information rates for various communication sources and channels. It also contains probability questions about events like coin flips, ball selections from urns, and people being divided into groups.

Uploaded by

Mahesh J2000
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

ASSIGNMENT

1. Explain the terms (i) Self information (ii) Average information (iii) Mutual
Information.
2. Discuss the reason for using logarithmic measure for measuring the amount of
information.
3. Explain the concept of amount of information associated with message. Also explain
what infinite information is and zero information.
4. A binary source emitting an independent sequence of 0‟s and 1‟s with probabilities p
and (1-p) respectively.
5. Explain the concept of information, average information, information rate and
redundancy as referred to information transmission.
6. Let X represents the outcome of a single roll of a fair dice. What is the entropy of X?
7. A code is composed of dots and dashes. Assume that the dash is 3 times as long as the
dot and has one-third the probability of occurrence. (i) Calculate the information in dot
and that in a dash; (ii) Calculate the average information in dot-dash code; and (iii)
Assume that a dot lasts for 10 ms and this same time interval is allowed between
symbols. Calculate the average rate of information transmission.
8. What do you understand by the term extension of a discrete memory less source? Show
that the entropy of the nth extension of a DMS is n times the entropy of the original
source.
9. A card is drawn from a deck of playing cards. A) You are informed that the card you
draw is spade. How much information did you receive in bits? B) How much
information did you receive if you are told that the card you drew is an ace? C) How
much information did you receive if you are told that the card you drew is an ace of
spades? Is the information content of the message “ace of spades” the sum of the
information contents of the messages ”spade” and “ace”?
10. A block and white TV picture consists of 525 lines of picture information. Assume that
each consists of 525 picture elements and that each element can have 256 brightness
levels.
11. Pictures are repeated the rate of 30/sec. Calculate the average rate of information
conveyed by a TV set to a viewer.
12. A zero memory source has a source alphabet S= {S1, S2, S3} with P= {1/2, 1/4, 1/4}.
Find the entropy of the source. Also determine the entropy of its second extension and
verify that H (S2) = 2H(S).
13. Show that the entropy is maximum when source transmits symbols with equal
probability.
14. The output of an information source consists of 128 symbols, 16 of which occurs with
15. probability of 1/32 and remaining 112 occur with a probability of 1/224. The source
emits 1000 symbols/sec. assuming that the symbols are chosen independently; find the
rate of information of the source.
16. In how many ways can 8 people be seated in a row if (a) there are no restrictions on the
seating arrangement? (b) persons A and B must sit next to each other? (c) there are 4
women and 4 men and no 2 men or 2 women can sit next to each other? (d) there are 5
men and they must sit next to each other? (e) there are 4 married couples and each
couple must sit together?
17. An elevator starts at the basement with 8 people (not including the elevator operator)
and discharges them all by the time it reaches the top floor, number 6. In how many
ways could the operator have perceived the people leaving the elevator if all people
look alike to him? What if the 8 people consisted of 5 men and 3 women and the
operator could tell a man from a woman?
18. Suppose that 5 percent of men and .25 percent of women are color blind. A color-blind
person is chosen at random. What is the probability of this person being male? Assume
that there are an equal number of males and females. What if the population consisted
of twice as many males as females?
19. There are 3 coins in a box. One is a two-headed coin, another is a fair coin, and the third
is a biased coin that comes up heads 75 percent of the time. When one of the 3 coins is
selected at random and flipped, it shows heads. What is the probability that it was the
two-headed coin?
20. An urn contains 5 white and 10 black balls. A fair die is rolled and that number of balls
is randomly chosen from the urn. What is the probability that all of the balls selected
are white? What is the conditional probability that the die landed on 3 if all the balls
selected are white?
21. Given 20 people, what is the probability that, among the 12 months in the year, there
are 4 months containing exactly 2 birthdays and 4 containing exactly 3 birthdays?
22. A group of 6 men and 6 women is randomly divided into 2 groups of size 6 each. What
is the probability that both groups will have the same number of men?
23. At time 0 a coin that comes up heads with probability p is flipped and falls to the ground.
Suppose it lands on heads. At times chosen according to a Poisson process with rate λ,
the coin is picked up and flipped. (Between these times the coin remains on the ground.)
What is the probability that the coin is on its head side at time t?

You might also like