Scribd
Upload
127 views3 pages

EE602 Assignment 5 PDF

The document contains 6 problems related to estimation and Bayesian statistics: 1. Find the MMSE estimator of θ given observations from a conditional PDF and prior on θ. 2. Determine the number of measurements needed for a MMSE estimator to achieve a given accuracy, with and without prior knowledge of tolerances. 3. Find the MMSE estimator of A and minimum Bayesian MSE given observations of a linear model with Gaussian noise and prior on A. 4. Relate the entropy of a Gaussian distribution to its concentration and show that the information gain from observing data is always non-negative. 5. Find the MAP estimator of θ given a conditional PDF and prior on θ. 6. Prove the

Uploaded by

Ankit Bansal
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
127 views3 pages

EE602 Assignment 5 PDF

The document contains 6 problems related to estimation and Bayesian statistics: 1. Find the MMSE estimator of θ given observations from a conditional PDF and prior on θ. 2. Determine the number of measurements needed for a MMSE estimator to achieve a given accuracy, with and without prior knowledge of tolerances. 3. Find the MMSE estimator of A and minimum Bayesian MSE given observations of a linear model with Gaussian noise and prior on A. 4. Relate the entropy of a Gaussian distribution to its concentration and show that the information gain from observing data is always non-negative. 5. Find the MAP estimator of θ given a conditional PDF and prior on θ. 6. Prove the

Uploaded by

Ankit Bansal
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

EE 602-Assignment 5

1. The data x [n] forn = 0, 1, , N 1 are observed, each sample having the conditional PDF { exp((x[n] )) x[n] > p (x [n] | ) = 0 x[n] < and conditioned on the observations are independent. The prior PDF { exp() > 0 p (x [n] | ) = 0 < 0. Find the MMSE estimator of . 2. A quality assurance inspector has the job of monitoring the resistance values of manufactured resistors. He does so by choosing a resistor from a batch and measuring its resistance with an ohmmeter. He knows that ohmmeter is of poor quality and imparts an error to the measurement which he models as a N (0, 1) random variable. Hence, he takes N independent measurements. Also, he knows that the resistance should be 100 ohms. Due to manufacturing tolerances,however, they are generally in error by , where N (0, 0.011). If the inspector chooses a resistor, how many ohmmeter measurements are necessary to ensure that a MMSE estimator of the resistance R yields the correct resistance to 0.1 ohms on the average or as he continuous to choose the resistors throughout the day? How many measurements would he need if he did not have prior knowledge about the manufacturing tolerances? 3. The data x [n] = Arn + w [n] 1 n = 0, 1, , N 1

2 where r is known, w [n] is WGN with variance 2 , and A N (0, A ) independent of w [n] are observed. Find the MMSE estimator of A as well as the minimum Bayesian MSE.

4. A measure of the randomness of a random variable is its entropy dened as H () = E ( ln p ()) = ln p () p () d.


2 If N ( , ), nd the entropy and relate it to the concentration of the PDF. After observing the data, the entropy of posterior PDF can be dened as H (|x) = E ( ln p (|x)) = ln p (|x) p (x, ) dxd

and should be less than H (). Hence a measure of the information gained by observing the data is I = H () H (|x) Prove that I 0. Under what conditions will I=0? Hint: Express H () as H ( ) = ln p () p (x, ) dxd You will also need the inequality p1 (u) ln p1 (u) du p2 (u)

for PDFs p1 (u),p2 (u). Equality holds if and only if p1 (u) = p2 (u). 5. Assume that { exp(x[n]) x[n] > 0 p (x [n] | ) = 0 x[n] < 0

where x[n]s are conditionally IID, or p (x|) = 2


N 1 n=0

p (x [n]|)

and the prior PDF is { exp(x[n]) > 0 p () = 0 <0 Find the MAP estimator of . 6. Prove the Bayesian Gauss-Markov theorem below Bayesian Gauss-Markov theorem: If the data are described by Bayesian linear model form x = H + w where x is an N1 data vector, H is a known Np observation matrix, is a p 1 random vector of parameters whose realization is to be estimated and has mean E () and covariance matrix C , and w is N1 random vector with zero mean and covariance Cw and is uncorrelated with , then the LMMSE estimator of is, ( ) = E () + C H T HC H T + Cw 1 (x HE ()) ( ) 1 = E () + C 1 + H T Cw 1 H H T Cw 1 (x HE ()) The performance of the estimator is measured by the error = whose mean is zero and whose covariance matrix is ( ) C = Ex, T ( ) 1 = C C H T HC H T + Cw HC ( ) 1 = C 1 + H T Cw 1 H

You might also like