Scribd
Upload
45 views

Information Theory Coding

1. The capacity of an AWGN channel is given by a formula involving bandwidth W, received power P, and noise power spectral density σ2. For fixed P/σ2 = 1000, the capacity is approximately 1.44 kbps. 2. The entropy H(L) for the number of tosses L needed to get the first head in a series of fair coin tosses is approximately 2 bits. 3. The capacity of a binary symmetric channel (BSC) with a crossover probability of 0.5 is 0 bits.

Uploaded by

Roshdy AbdelRassoul
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
45 views

Information Theory Coding

1. The capacity of an AWGN channel is given by a formula involving bandwidth W, received power P, and noise power spectral density σ2. For fixed P/σ2 = 1000, the capacity is approximately 1.44 kbps. 2. The entropy H(L) for the number of tosses L needed to get the first head in a series of fair coin tosses is approximately 2 bits. 3. The capacity of a binary symmetric channel (BSC) with a crossover probability of 0.5 is 0 bits.

Uploaded by

Roshdy AbdelRassoul
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Information Theory and Coding

1. The capacity of a band-limited additive white Gaussian (AWGN) channel


is given by
= 2 (1 +

) bits per second(bps), where W is the channel

bandwidth, P is the average power received and 2 is the one-sided power


spectral density of the AWGN.

For a fixed

= 1000, the channel capacity (in kbps) with infinite

bandwidth ( ) is approximately
(a) 1.44
(b) 1.08

(c) 0.72
(d) 0.36
[GATE 2014: 1 Mark]

Soln.
= ( +

+
(
)

=
( +
)

= [ ( + )]

= .

= . = .
Option (a)

2. A fair is tossed repeatedly until a Head appears for the first time. Let L
be the number of tosses to get this first Head. The entropy H(L) in bits
is _________
[GATE 2014: 2 Marks]
Soln. If 1 toss is required to get first head, then probability =

If 2 tosses are required to get first head then = =

If 3 tosses are required to get first head then = =


Entropy

=
=

+ + +

+ + +

3. The capacity of a Binary Symmetric Channel (BSC) with cross-over


probability 0.5 is ________
[GATE 2014: 1 Mark]
Soln.

p
(0)

(0)

(1 p)

(1 p)

(1)

Given cross over probability of 0.5

(1)

( ) =
( ) =
Channel capacity for BSC

() = [ ( ) ( )]
=

+ + ( ) ( )

= + (/) + (/)

=
=

4. In a digital communication system, transmission of successive bits


through a noisy channel are assumed to be independent events with error
probability p. The probability of at most one error in the transmission of
an 8-bit sequence is
(a) 7(1 )/+/8
(c) (1 )8 + (1 )7
(b) (1 )8 + 8(1 )7
(d) (1 )8 + (1 )7
[GATE 1988: 2 Marks]
Soln. Getting almost one error be success
Probability of at most one error = p
Say, success
Failure

=1p

P (X = at most 1 error)
= ( = ) + ( = )

Note that probability that event A occurs r times is given by


bionomical probability man function defined as
( = ) nCr ( )
= 8C0 () ( ) + 8C1 () ( )
= ( ) + ( )
Option (b)

5. Consider a Binary Symmetric Channel (BSC) with probability of error


being p. To transmit a bit say 1, we transmit a sequence of three sequence
to represent 1 if at least two bits bit will be represent in error is
(a) 3 + 32 (1 )
(c) (1 )3
(d) 3 + 2 (1 )
(b) 3
[GATE 2008: 2 Marks]
Soln. () = () =
() = () =
Reception with error means getting at the most one 1.
P(reception with error)
= ( = ) + ( = )
Using the relation of Binomial probability man function
( = ) = nCr ( )
For r = 0, 1, 2, -------- n
= 3C0 ( ) + 3C1 ( )
= + ( )
Option (a)
6. During transmission over a certain binary communication channel, bit
errors occur independently with probability p. The probability of at most
one bit in error in a block of n bits is given by
(a)
(c) (1 )1 + (1 )
(b) 1
(d) 1 (1 )
[GATE 2007: 2 Marks]

Soln. Probability of at most one bit is error


P = P (non error)+P(one bit error)
Using the relation of Binomial probability man function
= nC0 () ( ) + nC1 () ( )
= ( ) + ( )
Note, nC0 = 1

and nC1 = n

Option (c)

7. Let U and V be two independent and independent and identically


1

distributed random variables such that ( = +1) = ( = 1) = .


2
The entropy H(U+V) in bits is
(a) 3/4
(c) 3/2
(b) 1
(d) log2 3
[GATE 2013: 2 Marks]
Soln. U and V are two independent and identically distributed random
variables
( = +) = ( = ) =
( = +) = ( = ) =

So, random variables U and V can have following values


= +, ;

= +,

= =
+ { = , = = , = ,

= =
+ =

( + ) = = =

+=

( + ) = = + =

+=

( + ) = = =

Entropy of ( + ) = ( + )

= ( + )

(+)

= + +


+ + =

Option (c)

8. A source alphabet consists of N symbols with the probability of the first


two symbols being the same. A source encoder increases the probability
of the first symbol by a small amount e. After encoding, the entropy of
the source
(a) increases
(c) increases only if N = 2
(b) remains the same
(d) decreases
[GATE 2012: 1 Mark]
Soln. Entropy is maximum, when symbols are equally probable, when
probability changes from equal to non-equal, entropy decreases
Option (d)

9. A communication channel with AWGN operating at a signal at a signal to


noise ratio SNR >>1 and bandwidth B has capacity C1. If the SNR is
doubled keeping B constant, the resulting capacity C2 is given by
(a) 2 21
(c) 2 1 + 2
(b) 2 1 +
(d) 2 1 + 0.3
[GATE 2009: 2 Marks]
Soln. When SNR >>1, channel capacity C

= ( + )

( )

When SRN is doubled

( ) = + ( )

= ( ) +

= +
Option (b)

10.A memoryless source emits n symbols each with a probability p. The


entropy of the source as a function of n
(a) increases
(c) increases as n
(b) decreases as log n
(d) increases as n log n
[GATE 2008: 2 Marks]
Soln. Entropy H(m) for the memoryless source

() =

Pi = Probability of individual symbol


= = =

() =

Entropy H(m) increases as a function of


Option (a)

11. A source generates three symbols with probability 0.25, 0.25, 0.50 at a
rate of 3000 symbols per second. Assuming independent generation of
symbols, the most efficient source encoder would have average bit rate of
(a) 6000 bits/sec
(c) 3000 bits/sec
(b) 4500 bits/sec
(d) 1500 bits/sec
[GATE 2006: 2 Marks]

Soln. Three symbols with probability of 0.25, 0.25 and 0.50 at the rate of
3000 symbols per second.

= .

+ .
+ .
.
.
.

= . + . + .
= .
Rate of information R = r.H
R = 3000 symbol/sec
= .
= /
Option (b)

12.An image uses 512 512 picture elements. Each of the picture elements
can take any of the 8 distinguishable intensity levels. The maximum
entropy in the above image will be
(a) 2097152 bits
(c) 648 bits
(b) 786432 bits
(d) 144 bits
[GATE 1990: 2 Marks]
Soln. For 8 distinguishable intensity levels
=
= =
Maximum entropy =
=
=
1 1 1

2 4 8

13. A source produces 4 symbols with probability , ,

. For this

source, a practical coding scheme has an average codeword length of 2


bits/symbols. The efficiency the code is
(a) 1
(c) 1/2
(b) 7/8
(d) 1/4
[GATE 1989: 2 Marks]

Soln. Four symbol with probability , ,

= = ( )
=

= [ ( ) + ( ) + ( ) + ( )]

+ + +

=+

Code efficiency

Option (b)

You might also like