CHAPTER TWO

2. Estimation and Sampling Distribution of Mean, Proportion and Variance

2.1 Introduction
Inference, specifically decision-making and prediction, is centuries old and plays a very important
role in our lives. Each of us faces daily personal decisions and situations that require predictions
concerning the future. The objective of statistics is to make inferences about a population based
on information contained in a sample. Populations are characterized by numerical descriptive
measures called parameters. Typical population parameters are the mean u, the median M, the
standard deviation s, and a proportion Π. Most inferential problems can be formulated as an
inference about one or more parameters of a population.
Statistical inference is the process by which we acquire information and draw conclusions about
populations from samples. In order to do inference, we require the skills and knowledge of
descriptive statistics, probability distributions, and sampling distributions. There are two types of
inference: estimation and hypothesis testing. The objective of estimation is to determine the
approximate value of a population parameter on the basis of a sample statistic.
E.g., the sample mean (x̅ ) is employed to estimate the population mean (u).
2.2 Statistical Estimation
2.2.1 Point and Interval Estimation
Statistical estimation is one way of making inference about the population parameter where the
investigator does not have any prior notion about values or characteristics of the population
parameter. The objective of estimation is to determine the approximate value of a population
parameter on the basis of a sample statistic.
There are two ways estimation:
i. Point Estimation: The first step in statistical inference is point estimation, in which we
compute a single value (statistic) from the sample data to estimate a population parameter.
Suppose that we are interested in estimating a population means and that we are willing to
assume the underlying population is normal. One natural statistic that could be used to
estimate the population mean is the sample mean, but we also could use the median and the
trimmed mean. Which sample statistic should we use?
It is a single value or number of sample information that is used to estimate a parameter.
The best point estimate of the population mean  is the sample means X . It is a procedure
that results in a single value as an estimate for a parameter.
ii. Interval estimation:
Interval estimation draws inferences about a population by estimating the value of an
unknown parameter using an interval. That is we say (with some ___% certainty) that the
population parameter of interest is between some lower and upper bounds. It is the
procedure that results in the interval of values as an estimate for a parameter, which is
interval that contains the likely values of a parameter. It deals with identifying the upper
and lower limits of a parameter. For example, suppose we want to estimate the mean
summer income of a class of business students. For n=25 students, is calculated to be 400
$/week is a point estimate. An alternative statement is: The mean income is between 380
and 420 $/week.

1|Page
Estimator and Estimate
Estimator is the rule or random variable that helps us to approximate a population parameter.
However, estimate is the different possible values, which an estimator can assume. For example
n

X
is an estimator for the population mean and X = 10 is an estimate,
i
the sample mean X = i =1

n
which is one of the possible values of X .
Properties of Best Estimator
The following are some qualities of an estimator
o It should be unbiased.
o It should be consistent.
o It should be relatively efficient.
To explain these properties let ˆ be an estimator of θ
1. Unbiased Estimator: An estimator whose expected value is the value of the parameter being
()
estimated i.e. E ˆ =  . Example: the sample mean 𝑋̅ is an unbiased estimator of the population
mean µ, since: E(X) =µ
2. Consistent Estimator: An estimator which gets closer to the value of the parameter as the
sample size increases. I.e. ˆ gets closer to θ as the sample size increases. Example ̅
X is a consistent

̅) is
2
estimator of µ because: V(X . That is, as n grows larger, the variance of X grows smaller.
n
3. Relatively Efficient Estimator: The estimator for a parameter with the smallest variance. This
actually compares two or more estimators for one parameter.
Example both the sample median and sample mean are unbiased estimators of the population
mean, however, the sample median has a greater variance than the sample mean, so we choose
𝑋̅ since it is relatively efficient when compared to the sample median.

2.2.2 Sampling Distribution of the Sample Mean

A sampling distribution is created by, as the name suggests, sampling. The method we will employ
on the rules of probability and the laws of expected value and variance to derive the sampling
distribution. Sampling distribution of the sample mean is a theoretical probability distribution that
shows the functional relationship between the possible values of a given sample mean based on
samples of size n and the probability associated with each value, for all possible samples of size
n drawn from that particular population. There are commonly three properties of interest of a
given sampling distribution.
✓ Its Mean
✓ Its Variance, and
✓ Its Functional form.
Steps for the construction of Sampling Distribution of the mean

1. From a finite population of size N , randomly draw all possible samples of size n .
2. Calculate the mean for each sample.
3. Summarize the mean obtained in step 2 in terms of frequency distribution or relative
frequency distribution.
Example: Suppose we have a population of size N = 5 , consisting of the age of five children:
6, 8, 10, 12, and 14

2|Page
  Population mean =  = 10
population Variance =  2 = 8
Take samples of size 2 with replacement and construct sampling distribution of the sample mean.
Solution:
N = 5, n = 2
➔ We have N = 5 = 25 possible samples since sampling is with replacement.
n 2

Step 1: Draw all possible samples:

6 8 10 12 14
6 (6, 6) (6, 8) (6, 10) (6, 12) (6, 14)
8 (8,6) (8,8) (8,10) (8,12) (8,14)
10 (10,6) (10,8) (10,10) (10,12) (10,14)
12 (12,6) (12,8) (12,10) (12,12) (12,14)
14 (12,6) (14,8) (12,10) (12,12) (12,14)
Step 2: Calculate the mean for each sample:
6 8 10 12 14
6 6 7 8 9 10
8 7 8 9 10 11
10 8 9 10 11 12
12 9 10 11 12 13
14 10 11 12 13 14
Step 3: Summarize the mean obtained in step 2 in terms of frequency distribution.
X 6 7 8 9 10 11 12 13 14

Frequency 1 2 3 4 5 4 3 2 1

a) Find the mean of X , say  X

X =  i i =
X f 250
= 10 = 
 fi 25
Find the variance of X , say  X
2
b)

 ( X i −  X ) 2 f i 100
X 2
= = = 42
 fi 25
Remark:
1. In general, if sampling is with replacement
2
X = 2

n
2. If sampling is without replacement

3|Page
 2  N −n
X = 2
 
n  N −1 
3. In any case the sample mean is unbiased estimator of the population mean. i.e.
 X =   E (X ) =  (Show!)
Sampling may be from a normally distributed population or from a non-normally distributed
population. When sampling is from a normally distributed population, the distribution of X
will possess the following property.
1. The distribution of X will be normal
2. The mean of X is equal to the population mean , i.e.  X = 
3. The variance of X is equal to the population variance divided by the sample size, i.e.
 2
X2 =
n
2
 X ~ N ( , )
n
X −
Z = ~ N (0,1)
 n
Central Limit Theorem
The sampling distribution of the mean of a random sample drawn from any population is
approximately normal for a sufficiently large sample size. The larger the sample size, the more
closely the sampling distribution of 𝑋̅ will resemble a normal distribution. Given a population of
any functional form with mean  and finite variance  , the sampling distribution of X ,
2

computed from samples of size n from the population will be approximately normally distributed
2
with mean  and variance , when the sample size is large. In many practical situations, a
n
sample size of 30 may be sufficiently large to allow us to use the normal distribution as an
approximation for the sampling distribution of X .
Note: the definition of “sufficiently large” depends on the extent of non-normality of X (e.g.
heavily skewed; multimodal)

2.2.3 Point and Interval Estimation of Population Mean

Point estimation of the population mean: μ

Another term for statistic is point estimate, since we are estimating the parameter value. A point
estimator is the mathematical way we compute the point estimate. For instance, sum of X i over

4|Page
n is the point estimator used to compute the estimate of the population means,  . That is,

X =
 X i is a point estimator of the population mean.
n
Confidence interval estimation of the population mean
Although X possesses nearly all the qualities of a good estimator, because of sampling error, we
know that it's not likely that our sample statistic will be equal to the population parameter, but
instead will fall into an interval of values. We will have to be satisfied knowing that the statistic is
"close to" the parameter. That leads to the obvious question, what is "close"?

We can phrase the latter question differently: How confident can we be that the value of the
statistic falls within a certain "distance" of the parameter? Or, what is the probability that the
parameter's value is within a certain range of the statistic's value? This range is the confidence
interval. A confidence interval is a specific interval estimate of a parameter determined by using
data obtained from a sample and the specific confidence level of the estimate.

The following terms are of great importance in interval estimation:

The confidence level is the probability that the value of the parameter falls within the range
specified by the confidence interval surrounding the statistic. There are different conditions to be
considered to construct confidence intervals of the population mean,  .

There are different cases to be considered to construct confidence intervals.

5|Page
Case 1:
If sample size is large or if the population is normal with known variance
Recall the Central Limit Theorem, which applies to the sampling distribution of the mean of a
sample. Consider samples of size n drawn from a population, whose mean is  and standard
deviation is  with replacement and order important. The population can have any frequency
distribution. The sampling distribution of X will have a mean  x =  and a standard deviation

x = , and approaches a normal distribution as n gets large. This allows us to use the normal
n
distribution curve for computing confidence intervals.
The Z test is a statistical test for the mean of the population. It can be used when n ≥ 30, or when
the population is normally distributed and 𝜎 is unknown. The formula for the Z test is:
X̅ −𝜇
𝑍=𝜎 , where
⁄ 𝑛
√
X = is Sample mean,
𝜇 = is hypothesized population mean,
σ = population standard deveiation
n = sample size
X −
Z = has a normal distributi on with mean = 0 and var iance = 1
 n
  = X  Z n
= X , where  is a measure of error.
 = Z n
For the interval estimator to be good the error should be small. How it be small?

• By making n large
• Small variability
• Taking Z small
To obtain the value of Z, we have to attach this to a theory of chance.

6|Page
 That is, there is an area of size 1 −  such
P ( − Z 2  Z  Z 2 ) = 1 − 
Where  = is the probability that the parameter lies outside the int erval
Z 2 = s tan ds for the s tan dard normal var iable to the right of which
 2 probability lies , i.e P( Z  Z 2 ) =  2
X −
 P(− Z 2   Z 2 ) = 1 − 
 n
 P( X − Z 2  n    X + Z 2  n) = 1 − 
 ( X − Z 2  n , X + Z 2  n ) is a100(1 −  )% conifidence int erval for 

But usually 2 is not known, in that case we estimate by its point estimator S
 ( X − Z 2 S n , X + Z 2 S n ) is a100(1 −  )% conifidence int erval for 

Here are the z values corresponding to the most commonly used confidence levels.

100(1 −  ) %   2 Z 2
90 0.10 0.05 1.645
95 0.05 0.025 1.96
99 0.01 0.005 2.58

Case 2: If sample size is small and the population variance,  2 is not known.
X −
t= has t distributi on with n − 1 deg rees of freedom.
S n

 ( X − t 2 S n , X + t 2 S n ) is a100(1 −  )% conifidence int erval for 

The unit of measurement of the confidence interval is the standard error. This is just the standard
deviation of the sampling distribution of the statistic.
Examples: From a normal sample of size 25 a mean of 32 was found. Given that the population
standard deviation is 4.2. Find
a) A 95% confidence interval for the population mean.
b) A 99% confidence interval for the population mean.
Solution:

7|Page
 a)
X = 32,  = 4.2, 1 −  = 0.95   = 0.05,  2 = 0.025
 Z 2 = 1.96 from table.
 The required int erval will be X  Z 2  n
= 32  1.96 * 4.2 25
= 32  1.65
= (30.35, 33.65)
b)
X = 32,  = 4.2, 1 −  = 0.99   = 0.01,  2 = 0.005
 Z 2 = 2.58 from table.
 The required int erval will be X  Z 2  n
= 32  2.58 * 4.2 25
= 32  2.17
= (29.83, 34.17)
2. A drug company is testing a new drug which is supposed to reduce blood pressure. From the
six people who are used as subjects, it is found that the average drop in blood pressure is 2.28
points, with a standard deviation of 0.95 points. What is the 95% confidence interval for the
mean change in pressure?
Solution:

X = 2.28, S = 0.95, 1 −  = 0.95   = 0.05,  2 = 0.025

 t 2 = 2.571 with df = 5 from table.
 The required int erval will be X  t 2 S n
= 2.28  2.571 * 0.95 6
= 2.28  1.008
= (1.28, 3.28)
That is, we can be 95% confident that the mean decrease in blood pressure is between 1.28 and
3.28 points.

2.2.4 Sampling Distribution of the Sample Proportion

8|Page
In situations where it is not possible to measure the characteristic under study, but is possible to
classify the whole population in various categories with respect to the attributes they possess,
consideration is usually given to estimating the population elements that belong to a defined
category of class. The objective of studying the sampling distribution of proportions will be to
guess or estimate the number of one of two types of qualitative values in the population based on
the proportion of the same type we have in the sample. Since proportions are usually expressed in
terms of percentages, we also refer to the above procedure as sampling distributions of percentages.
Suppose that we have two complementary and mutually exclusive class, C and C' such that every
unit in the population falls into either of them.
In order to know how many of the units fall in class C, we define a counting variable as
1, if the unit fall in class C
Xi = {
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
If the number of units falling in C is denoted by A for the population and by a for the sample, then
𝐴
∑𝑁𝑖 𝑋𝑖 = 𝐴 and hence the population proportion denoted by P is given by P = . 𝑁
𝑎 ∑𝑛
𝑖 𝑋𝑖
Given a simple random sample of n units, the sample proportion denoted by p= 𝑛 = from the
𝑛
formula, we see that X and p are essentially identical. In fact, p is special case of X , the case
where possible values of Xi are only 0 and 1. Consequently, p possesses all properties of X . p is
an estimate of P, with variance
2  N −n 𝑖 (𝑋𝑖 −  )
∑𝑁 2
var(p) = var( X ) =   where  2
= = PQ
n  N −1  𝑁

PQ  N − n 
var(P) =  
n  N −1 
Where Q=1-P is proportion of units falling in class C'.
PQ  N − n 
var(P) =   is estimated by using sample values as
n  N −1 
pq  N − n 
var(𝑝̂ ) =  =
pq
(1 − f )
n −1  N  n −1
Where sampling fraction, f = n/N.
npq
This expression is obtained by replacing  2 by its estimator S2 = .
n −1
The sampling fraction can be ignored, when N is large relative to sample size n, n/N<0.05.
pq pq
var(𝑝̂ ) = and the standard error of p is √ .
n −1 n −1
PQ  N − n 
Sample proportion p is normally distributed with mean P and variance var (p) =  
n  N −1 
Example: In a simple random sample of size 100, from a population of size 500, there are 37
employed persons in the sample.
a) Estimate proportion of employed persons in the population.
b) Calculate the standard error of p.
Solution:

9|Page
a) Population proportion P is estimated by p= a/n = 37/100 = 0.37. 37% of the population is
employed.
pq(1 − f ) (0.37)(0.63)(1 − 0.2)
b) Standard error of p is√ =√ = 0.0434.
n −1 99
2.2.5 Point and Interval Estimation of Population Proportion
Point estimation of the population Proportion: 𝜋
In a random sample of n from a population in which the proportion of elements classified as
successes is 𝜋 𝑜𝑟 𝑃, the best estimate of the parameter 𝜋 𝑜𝑟 𝑃 is the sample proportion of
successes. Letting X denote the number of successes in the n sample trials, the sample proportion
𝑋 X
is 𝑝̂ = . The distribution of p̂ = can be approximated by a normal distribution with a mean and
𝑛 n
𝑝(1−𝑝)
a standard error as given here. 𝜇𝑝̂ = 𝜋 𝑜𝑟 𝑃 and 𝜎𝑝̂ = √
𝑛
Confidence Interval for Population Proportion
The confidence interval for the population proportion is performed in the same manner as the
population mean. We have discussed that the sampling distribution of sample proportion is normal.
The sample estimate of population proportion P is sample proportion p and sample estimate of
𝑝𝑞 𝑝𝑞
variance of sample proportion is 𝑉𝑎𝑟(𝑝̂ ) 𝑛−1 for large sample 𝑉𝑎𝑟(𝑝̂ ) = 𝑛 .

A (1-α) 100% confidence interval for proportion p is given by (for large n):
𝑝̂ ± 𝑍α⁄2 √𝑝̂𝑛𝑞̂
Example: The Human Resource director of a large organization wanted to know what proportion
of all persons who had ever been interviewed for a job with his organization had been hired. He
was willing to settle for 95% confidence interval. A random sample of 500 interview records
revealed that 76 or 0.152 of the persons in the sample had been hired.
Solution:

Given: 𝑝̂ = 0.152, 𝑞̂ = 1 − 𝑝̂ = 0.848, n = 500, α = 0.05, 𝑍0.025 = 1.96

The 95% confidence interval for the population proportion is given by

𝑝̂ ± 𝑍α⁄2 √𝑝̂𝑛𝑞̂ = 0.152 ± 1.96√0.152×0.848

500
= 0.152 ± 0.031
= (0.121, 0.183)
Hence, the required proportion varies between 0.121 and 0.183.

10 | P a g e