Property of Simple Random Sample Sample mean Expectation of sample mean

Example
Consider population has three numbers: 1, 2, and 3.
Expectation of sample mean
Consider a population with mean µ and variance ‡ 2 . Sample mean Population mean: µ = 1+23+3 = 2.
Expectation of sample mean is population mean First consider sample of size 1.
Property of Simple Random Sample Sample mean is the mean of a sample. Simple random sample implies that the sample mean will be one of
the three possibilities with equal probability:
qn µX̄ = E (X̄ ) = µ
If X1 , . . . , Xn is a simple random sample, then i = 1 Xi X1 + . . . + Xn X̄ 1 2 3
X̄ = = P (X̄ ) 13 13 13
I E (Xi ) = µ, n n
I Var (Xi ) = ‡ 2 , and Expectation of sample mean is
I If we sample many times, average of all sample
I Cov (Xi , Xj ) = 0 for i ”= j. I This varies sample by sample. means is the population mean 1 1 1
µX̄ = E (X̄ ) = (1) + (2) + (3) = 2
I Hence, it is also a random variable. I This nice property is known as unbiasedness (see 3 3 3

合cpmf
点的
— ×
next chapter)

Chiu Yu KO 6 / 61 Chiu Yu KO 10 / 61 Chiu Yu KO 12 / 61 Chiu Yu KO 14 / 61

Example Dispersion of sample mean Standard error of sample mean Example

Now consider sample of size 2. Consider population has three numbers : 1, 2, and 3.
Simple random sample implies that the sample mean will be one of Population mean: µ = 1+23+3 = 2.
the nine possibilities with equal probability: Variance of sample mean Standard deviation of a statistics is often called standard Population variance:
X1 \X2 1 2 3
1 1 1.5 2 Population variance divided by sample size: error: (1 ≠ 2)2 + (2 ≠ 2)2 + (3 ≠ 2)2 2
‡2 = =
2 1.5 2 2.5 Standard error of the mean 3 3
3 2 2.5 3 ‡2
‡X̄2 = Var (X̄ ) = Standard deviation of the sample mean First consider sample of size 1. We know that E (X̄ ) = 2. Hence,
Sample mean is X̄ = X1 +X2
2 : n X̄ 1 2 3
X̄ 1 1.5 2 2.5 3 ‡ (X̄ ≠ E (X̄ ))2 1 0 1
P (X̄ ) 19 2 3 2 1 ‡X̄ = Ô P (X̄ ) 1 1 1
9 9 9 9 Note that spread of sample mean is proportional to the n 3 3 3
Expectation of the sample mean is Variance of sample mean is
popolution variance but inversely proportional to the
1 2 3 2 1 sample size. 1 1 1 2
µX̄ = E (X̄ ) = (1) + (1.5) + (2) + (2.5) + (3) = 2 ‡X̄2 = Var (X̄ ) = (1) + (0) + (1) =
9 9 9 9 9 3 3 3 3

Chiu Yu KO 15 / 61 Chiu Yu KO 18 / 61 Chiu Yu KO 19 / 61 Chiu Yu KO 20 / 61

Law of large numbers Central Limit Theorem

I sample mean approximately follows a normal Example
Example
Let X1 , . . . , Xn be a random sample from a distribution distribution with a large enough sample. Consider a population with mean 5 and variance 64.
Now consider sample of size 2. We know that E (X̄ ) = 2. Hence, with mean µ and variance ‡ 2 . Denote X̄ = X1 +...n +Xn . Consider a sample with size 100.
X̄ 1 1.5 2 2.5 3
Central Limit Theorem What is the probability that the sample mean is no more than 4?
(X̄ ≠ E (X̄ ))2 1 0.25 0 0.25 1 Law of Large Numbers 64
By central limit theorem, we have X̄ ≥ N (5, 100 ):
P (X̄ ) 1
9
2
9
1
3
2
9
1
3 When n gets large, we have
Variance of sample mean is For any ‘ > 0, when n is sufficientlly large, we have X̄ ≠ 5 4≠5
Q R Pr(X̄ Æ 4) = Pr( Ô Æ Ô )
‡2 b 8/ 100 8/ 100
1 2 3 2 1 Pr(µ ≠ ‘ < X̄ < µ + ‘) ¥ 1 X̄ ≥ N aµ, 4≠5
‡X̄2 = Var (X̄ ) = (1) + (0.25) + (0) + (0.25) + (1) n = Pr(Z Æ Ô
9 9 9 9 9 )
2/3 8/ 100
= or = Pr(Z Æ ≠1.25)
2 I Loosely speaking, when sample size is large, X̄ ≠ µ X̄ ≠ µ
Z = = Ô ≥ N (0, 1) ¥ 0.1057
variation disappers. ‡X̄ ‡/ n
I the sample mean becomes population mean
Rule of thumb: sample size n is at least 35.
Chiu Yu KO 21 / 61 Chiu Yu KO 25 / 61 Chiu Yu KO 32 / 61 Chiu Yu KO 33 / 61

Graphics for CLT left-tail Special case: binary data Sample proportion Expectation of sample proportion
Expectation of sample proportion is population
Each element in the population is ether 0 or 1 For binary data, we call the sample mean as sample proportion:
I The proportion with one is population proportion fi. proportion:
Expectation of sample proportion
I Mean of population: µ = fi Sample proportion
I Variance of population is ‡ 2 = fi (1 ≠ fi ) µp = E ( p ) = fi
≠z– = ≠1.25
– = 0.1057 Consider a random sample of size one: Special case for sample average
I Bernoulli distribution with probability of success fi qn Binary data is a special case:
z i = 1 Xi
0 2 4 I Mean is µ = fi p = X̄ = I sample mean is sample proportion (X̄ = p)
≠4 ≠2 n I population mean is poplulation proprotion (µ = fi)
I Variance is ‡ 2 = fi (1 ≠ fi )
I expectation of sample mean is popluation mean
(E (X̄ ) = µ)

Chiu Yu KO 34 / 61 Chiu Yu KO 35 / 61 Chiu Yu KO 37 / 61 Chiu Yu KO 38 / 61

Dispersion of sample proportion Central limit theorem Graphics for normal approximation Normality assumption
Central limit theorem for binary data
Variance of sample proportion X̄ ≠µ
When n gets large, we have In general, we do not know the distribution of s/ Ô .
n
‡ 2
fi (1 ≠ fi ) Q R We need an extra assumption on the population:
‡p2 = = qn
i = 1 Xi fi (1 ≠ fi ) b
n n p = X̄ = ≥ N afi, Normality assumption
n n ≠z– = ≠1.02
– = 0.1539 Population follows normal distribution with mean µ and
Standard error of the proportion or
p≠fi p≠fi
z variance ‡ 2 :
Z = =Ú ≥ N (0, 1)
‡
ı̂
ı fi (1 ≠ fi ) ‡p fi (1≠fi ) ≠4 ≠2 0 2 4 X ≥ N (µ, ‡ 2 )
n
‡p = Ô = Ù
n n
Rule of thumb: good approximation when nfi and
n (1 ≠ fi ) are at least 5.
Chiu Yu KO 39 / 61 Chiu Yu KO 40 / 61 Chiu Yu KO 44 / 61 Chiu Yu KO 47 / 61

Sample mean under normality assumption t-distribution Example Example: t table

Consider a population follows normal distribution with mean 5. Recall that we need to find
We have a simple random sample with size 100.
We know that sample variance 8.
Sample mean under normality assumption Pr (t99 < ≠1.768)
Distribution of sample mean (unknown variance) What is the probability that the sample mean is less than 4.5?
Under normality, sample mean is normally distributed: Note that ÔX̄8/100
≠5
≥ t99 . Then From t-table, we can either use n = 90 or n = 100. Usually we
Under normality assumption, it is t distribution: prefer to use one.
Q R
qn
‡2 Pr(X̄ < 4.5) = Pr(X̄ ≠ 5 < 4.5 ≠ 5) I Pr(≠1.662 < t90 < 1.662) = 0.9 and
i = 1 Xi
X̄ = ≥ N aµ, b X̄ ≠ µ X̄ ≠ µ X̄ ≠ 5 4.5 ≠ 5 Pr(≠1.987 < t90 < 1.987) = 0.95.
n n = Ô ≥ tn≠1 = Pr( Ô <Ô
sX̄ s/ n 8/100 8/100
) I By symmetry, we have Pr(t90 < ≠1.662) = 0.01 and
4.5 ≠ 5 Pr(t90 < ≠1.987) = 0.05.
that is, t distribution with n ≠ 1 degrees of freedom = Pr(t99 < Ô )
Note that same as CLT but it is not appriomation and 8/100 I Hence, 0.025 Æ Pr(t90 < ≠1.768) Æ 0.05.
there is no need for a large sample. ¥ Pr(t99 < ≠1.768)
¥ 0.0401. Note that Pr(Z < ≠1.768) ¥ 0.0386 (if we assume
population variance is 8).
Chiu Yu KO 48 / 61 Chiu Yu KO 50 / 61 Chiu Yu KO 51 / 61 Chiu Yu KO 52 / 61

Graphics for t-distribution

≠t99,– = ≠1.768

– = 0.0401
t99
≠4 ≠2 0 2 4

Chiu Yu KO 53 / 61