CH5115 State Parameter Estimation
ED 23 D 008
Assignment 3
Lakshmi S
1 a
Yen r Poisson distribution
PUN x N observations
given
Means variance poisson distribution
of
is X Y E ly Van y X
Mc estimate fishes information by
is given
flyn x
IT pre x
NET
i o
hog likelihood function is computed as
L lmfao Eg Kiba na
In x 22 kn
when
estimating o
of yen
i Vas
Ey Cu
e
x
E
I Ix 8 2
i I 17
2
The lower hound variance of X is given
by
var X Tn
estimation of X i e
am is s
t
omit
É
This the
is an
efficient estimator as
estimator is
independent of X
% Define the lambda values
lambda_values = [0.1, 1, 10];
% Define the number of observations
N = 1000; % You can adjust this as needed
% Initialize arrays to store results
variance_theta_ML = zeros(1, length(lambda_values));
fisher_information = zeros(1, length(lambda_values));
% Generate random samples from Poisson distribution for each lambda
for i = 1:length(lambda_values)
lambda = lambda_values(i);
samples = poissrnd(lambda, N, 1); % Generate Poisson-distributed samples
% Maximum Likelihood Estimation for lambda (theta)
theta_ML = sum(samples) / N;
% Calculate the Fisher Information
fisher_information(i) = N / lambda;
% Calculate the variance of the ML estimator
variance_theta_ML(i) = var(samples) / N;
end
% Display the results
for i = 1:length(lambda_values)
fprintf('Lambda = %.1f:\n', lambda_values(i));
fprintf(' Variance of ML estimator: %.4f\n', variance_theta_ML(i));
fprintf(' Fisher Information: %.4f\n\n', fisher_information(i));
end
Lambda = 0.1:
Variance of ML estimator: 0.0001
Fisher Information: 10000.0000
Lambda = 1.0:
Variance of ML estimator: 0.0010
Fisher Information: 1000.0000
Lambda = 10.0:
Variance of ML estimator: 0.0092
Fisher Information: 100.0000
% Define parameters
N = 300;
sigma_e2 = 1.5;
a = 4;
b = 5;
% Generate data with known parameters
X = randn(N, 1); % Random X values
epsilon = sqrt(sigma_e2) * randn(N, 1); % Random noise
1
Y = a * X + b + epsilon; % Observed Y values
% Likelihood functions for a and b
likelihood_a = @(a) -sum((Y - a*X - b).^2) / (2*sigma_e2);
likelihood_b = @(b) -sum((Y - a*X - b).^2) / (2*sigma_e2);
% Grid of values for a and b
a_values = linspace(0, 8, 100);
b_values = linspace(0, 10, 100);
% Calculate likelihood values for a and b
likelihood_values_a = arrayfun(likelihood_a, a_values);
likelihood_values_b = arrayfun(likelihood_b, b_values);
% Plot likelihood functions for a
subplot(2, 1, 1);
plot(a_values, likelihood_values_a);
xlabel('Parameter a');
ylabel('Log Likelihood');
title('Likelihood Function for Parameter a');
% Plot likelihood functions for b
subplot(2, 1, 2);
plot(b_values, likelihood_values_b);
xlabel('Parameter b');
ylabel('Log Likelihood');
title('Likelihood Function for Parameter b');
2
% Find ML estimates for a and b
[~, idx_a] = max(likelihood_values_a);
[~, idx_b] = max(likelihood_values_b);
% ML estimates
a_ML = a_values(idx_a);
b_ML = b_values(idx_b);
% Display results
fprintf('ML Estimate for a: %.4f\n', a_ML);
ML Estimate for a: 4.0404
fprintf('ML Estimate for b: %.4f\n', b_ML);
ML Estimate for b: 5.0505
3
Ib Linear regression problem
Y ax b te X follows
normal distribution
Fisher information for parameter a
8
Lia
It Egg gth
Log likelihood
In Ca
1k KITE
E Y
g7tbl
galnkia E secret
E E x
Ica Hash 261 Iz
Fisher information Joe parameter b
in Icb 72h kite
glnLCb
E lycut.gg
YzI1b7 Yz
3 a To determine with what degree of
confidence we can assert that the average
thepolymer is between
molecular weight of
a confidence
23000 15500 perform
we can
based on the sample data
interval calculation
Given
Sample mean 14678
Sample Std deviation 1445
Sample size 80
interval used 195 11
confidence typically
A 0 05
For 951 CI 2 1096
the margin error is computed by
E Zeit
fr
CI E E J t E
E 1 96
1g
325 04
CI 6 78 325 04 14678
325 043
14
I 14352 96 150030043
we can assert
hence with 95.1 confidence
the average modular weight of the polymer
that
14352096 15003 04
is in between
3b
Define hypothesis
o the mean scores
of the students from
institution A MD institution B Nz are
equal
score students from
417 The mean exam
of
less than the mean enam
institution A pre is
institution B Ms
Score of students from
195 1 Confidence Interval
Let CI 0005
ti
IEEE
t 1.224
853ft
To find the degreesof freedom
101 69 I 101
1.660
for t table we
get 2
Sh Ce 2 1 660
E z 10224
We can the null hypothesis Ho
reject to
This means there is sufficient evidence
no
conclude that 95.1 confidence interval
the exam
worse is
that intuition A performed
Than institution B
La
Given VCR is
gaussian
Ev
ven fn EJVGJeapf jztf.tl
fo Yr
Pen E Pin
WitI anti
1258k
n
If verge
tjÉÉVCk sinkhole
ÉIVCK cos
Etf k
w
Real part
imaginary
part
lunch Unenlfn Egf
I I
a RUG bae IUCN
II
an
very coset for
bn Gtfo k
GI
sin
very
Gaussian
that VCD follows
we know
distribution
pours
the DFT
that
en
implies are dependent
on Ucr
an bn
coefficients distribution
Gaussian
an bn follow
bn o
E
also Elan o
Vas Can 03 2,4
Van bn
tea
VCD eck
off bn are
Thus theoritically proving an
Gaussian distributed
2b
PIL
IIE anfbi
Druffn Ifn
Prr Cfn
IN
Yuu'tn jaitf.nl
jg.fnuceye
its
he consistent check for
For estimate to
bias variance
E Pruyn Éingforweepeiwn
inal
II z
Ier oyyage
To realise bias in the estimate
introduce the sequence
Ill een D
are
I
I otherwise
O
Pur fun
GE g felonyCD éJwnl
I Alun 8 Con 7 da
This the periodogram is the biased
the spectral density grew
version of
E pawn
V Wn
MMTEI.me aces I 51 1
as N a
hug A Cw 21 8 Wn
Applying
in we get
Eleven Durin
Is
UNIT crown
CIN
t
7 8 Inn
Van f Pur fun
as
fig D wn
consistent estimates
not
Periodogram is
an
the no of
U Because increasing
of PSD fn the variance or the
Samplesdoes not decrease
error in the estimates
% Parameters
N = 1000; % Number of samples
M = 10000; % Number of MC simulations
sigma = 1; % Variance of random process v[k]
% Initialize arrays to store results
real_parts_an = zeros(M, N);
imaginary_parts_bn = zeros(M, N);
% Perform MC simulations
for m = 1:M
% Generate a random sample of v[k] (assumed to be zero-mean)
v = sigma * randn(1, N);
% Compute DFT coefficients
F = fft(v);
% Extract real and imaginary parts of F
real_parts_an(m, :) = real(F);
imaginary_parts_bn(m, :) = imag(F);
end
% Plot histograms of real and imaginary parts of a_n and b_n
figure;
subplot(1, 2, 1);
histogram(real_parts_an(:), 100, 'Normalization', 'pdf');
title('Histogram of Real Parts of a_n');
subplot(1, 2, 2);
histogram(imaginary_parts_bn(:), 100, 'Normalization', 'pdf');
title('Histogram of Imaginary Parts of b_n');
1
% subplot(2, 2, 3);
% qqplot(real_parts_an(:));
% title('QQ Plot of Real Parts of a_n');
% subplot(2, 2, 4);
% qqplot(imaginary_parts_bn(:));
% title('QQ Plot of Imaginary Parts of b_n');
We can see that the DFT coefficients follow Gaussian distributions in MC simuations
% Parameters
M = 10000; % Number of realizations
N_values = [100, 200, 500, 1000, 10000]; % Different values of N
% Initialize arrays to store mean and variance
mean_Pfn = zeros(length(N_values), 1);
variance_Pfn = zeros(length(N_values), 1);
for i = 1:length(N_values)
N = N_values(i);
Pfn_realizations = zeros(M, N); % Store P[f_n] for each realization
for m = 1:M
% Generate a realization of the Gaussian process v[k]
vk = sigma * randn(1, N);
2
% Compute DFT coefficients and estimate PSD
V = fft(vk);
Pfn = abs(V).^2 / N;
Pfn_realizations(m, :) = Pfn;
end
% Compute mean and variance of P[f_n]
mean_Pfn(i) = mean(Pfn_realizations(:));
variance_Pfn(i) = var(Pfn_realizations(:));
end
% Plot mean and variance as functions of N
figure;
subplot(2, 1, 1);
plot(N_values, mean_Pfn);
xlabel('N');
ylabel('Mean of P[f_n]');
title('Mean of Estimated PSD vs. N');
subplot(2, 1, 2);
plot(N_values, variance_Pfn);
xlabel('N');
ylabel('Variance of P[f_n]');
title('Variance of Estimated PSD vs. N');
3
Mean and variance don't go to zero but remains constant after large values of N , thus showing its not a
consistent estimator
4
44
Give N observations determine CRB for
amplitude frequency
A sin 21Tf k te Cr
Y Cr
D CA f
distribution
Given noise follows gaussian
114,0
pigment 1 13 4
hog Likelihood for
Lfo atte
y Elm
EIagct.nl
Zig
I Sco y I
00
Aif
mygt E.ly
l
food
N
Yen Layer sinkitfok
Ém
7
2Ejyer sink't Jok
n.tn
IIgi2kittn
HALEY
Ck Sin ftp.k 5j 2Asm2f2itfokl
A ÉÉ y a Sinezitfok
it
Ilo E
Ig Etsikittok
CCRB_for Amplitude
n
É
Similarly
min J d Asian f 6
ÉyCr
fo Joy 2 XRCoskafok
E 41IEEIKcosaf.ie
Ilo
I
of
CHILI gaff
K o
various
Effects.nl
var logo
Tiffins't't Jok
IVmlontvonfoj.FI
4b BLUE for 02
MB ive ly Cr m
In
M O
we
I É ly Cn
02 This estimator is
the properties of
unbiased for 022 given BLUE
they are also
sample variance
using Transformed
data
obunatins
Consider taking Logarithm of squared
FCK to linearize the problem
Infy CK Z k
É
E
Ch
I
1293 272
T
I É
Tere emp 25 53
domain the problem
In this transformed
becomes linear in h 1027
however estimates is already lineage
unbiased there is 1s
the sample variance
datataftion
described above is the
as
estimates as
optimal choice
