Probability and Statistics

Dr. Marwa Mostafa

 Lecture 3

Conditional Probability
Theorem of Total Probability
Bayes’ Theorem
Independent Events

Dr. Marwa Mostafa

 Conditional Probability
The probability of an event B occurring when it is known that
some event A has occurred is called a conditional probability
and is denoted by P(B A).

The symbol is usually read

"B occurs given the probability that A occurs"
or simply
"the probability of B, given A "

Definition : The conditional probability of B, given A,

denoted by is defined by

if P(A) > 0
Dr. Marwa Mostafa
Example (1): A coin is flipped twice . If we assume that all
four points in the sample space
S = {(H, H), (H, T), (T, H), (T, T)} are equally likely,
what is the conditional probability that both flips result in
heads, given that the first flip does?

Solution: If B = {(H, H)} denotes the event that both flips

land heads, and
A = {(H, H), (H, T)}
the event that the first flip lands heads, then the desired
probability is given by

Dr. Marwa Mostafa

Example (2): Page 68
If the probability that a communication system will have high
fidelity is 0.81 and the probability that it will have high
fidelity and high selectivity is 0.18 , what is the probability
that a system with high fidelity will also have high selectivity?

Solution: If A = event that the system will have high fidelity

B = event that the system will have high selectivity
P(A) = 0.81 , P(A  B)  0.18
Then, the required probability is

Dr. Marwa Mostafa

 Note that
A general formula for that is valid for all events A
and B is derived in the same manner as just described .
Namely , if the event A occurs , then in order for B to occur it
is necessary that the actual occurrence be a point in both B and
A ; that is , it must be in A∩B .
Now, since we know that A has occurred , it follows that A
becomes our new (reduced ) sample space and hence the
probability of A∩B occurs will equal the probability of A∩B
relative to the probability of A . That is

Dr. Marwa Mostafa

Example (3):Suppose that an office has 100 calculating
machines . Some of these machines are electric (E) while
others are manual (M) . And some of the machines are new
(N) while others are used (U) . The following table gives the
number of machines in each category . A person enters the
office , picks a machine at random , and discovers that it is
new . What is the probability that it is electric ?

E M Total

N 40 30 70
U 20 10 30
Total 60 40
Dr. Marwa Mostafa
100
Solution :
We observe that
n(S) = 100, n(E) = 60, n(M) =40 , n(N) = 70 , n(U) = 30.
In terms of the notation introduced we wish to compute P(E N)

Dr. Marwa Mostafa

Example(4) : A bin contains 5 defective (that immediately fail
when put in use ), 10 partially defective ( that fail after a
couple of hours of use ), and 25 acceptable transistors .
A transistor is chosen at random from the bin and put into
use . If it does not immediately fail, what is the probability it is
acceptable ?

Dr. Marwa Mostafa

Solution : Since the transistor did not immediately fail , we
know that it is not one of the 5 defectives and so the desired
probability is :
P(acceptable, not defective)
P(acceptable not defective) 
P(not defective)

P(acceptable)

P(not defective)
We obtain that

25 40 25 5
P(acceptable not defective)   
35 40 35 7
Dr. Marwa Mostafa
Note that :

By multiplying both sides of the equation by P(A) we obtain

that

In words, the last equation states that the probability that

both A and B occur is equal to the probability that A occurs
multiplied by the conditional probability of B given that A
occurred . This equation is often quite useful in computing
the probability of the intersection of events . This is
illustrated by the followingDr.example
Marwa Mostafa.
Example (5) : Ms. Perez figures that there is a 30 percent
chance that her company will set up a branch office in Phoenix
. If it dose , she is 60 percent certain that she will be made
manager of this new operation . What is the probability that
Perez will be a Phoenix branch office manager ?
Solution : Let
B = the event that the company sets up a branch in Phoenix
M = the event that Perez is made the Phoenix manager
The desired probability is P(B∩M) , which is obtained as
follows :
P( B  M )  P( B) P( M | B)  (0.3)(0.6)  0.18
Hence, there is an 18 percent chance that Perez will be the
Dr. Marwa Mostafa
Phoenix manager .
Example (6): Box 1 contains 4 defective and 16 nondefective
light bulbs. Box 2 contains 1 defective and 1 nondefective
light bulb. We roll a fair die 1 time. If we get a 1 or a 2, then
we select a bulb at random from box 1 . Otherwise we select a
bulb from box 2. What is the probability that the selected bulb
will be defective?

Solution: Define
E1: We select a bulb from box 1 .
E2 : We select a bulb from box 2 .
D : The selected bulb is defective .
2 1
Then P( E1)  P( we get face 1 or face 2)   and
6 3

P(E 2 )  4  2 P ( D | EDr.  20
1) Marwa
4
 1
P ( D | E 2)  1
6 3 Mostafa
5 2
Since,
D  ( E1  D)  ( E 2  D)

and
( E1  D)  ( E 2  D)  
Then,
P( D)  P( E1  D)  P( E 2  D)

 P ( E1 ) P ( D | E1)  P ( E2 ) P ( D | E 2)

 1.1  2.1  6  2
3 5 3 2 15 5
Dr. Marwa Mostafa
 Theorem of Total Probability
The events E1 , E2 , …, En are called a partition of the sample

space S if Ei ∩ Ej = φ for all i ≠ j , and E1  E 2 ...  E n  S.

Thus a partition cuts the whole sample space into mutually

exclusive pieces. Figure 1, gives a Venn diagram with n = 7

events in the partition.

Dr. Marwa Mostafa

Figure 1

Dr. Marwa Mostafa

If A  S , is any event and E1 , E2 , …, En is a partition of S ,
then E1 , E2 , …, En also partition A ;
that is
A  (A  E1)  (A  E 2 ) ...  (A  E n )
and of course,
( A  Ei )  ( A  Ej)   for all i ≠ j

Figure 2, pictures this partitioning of the event A, again with

n = 7 events in the partition.

Dr. Marwa Mostafa

Figure 2
Dr. Marwa Mostafa
It then follows that we can write

P(A)  P(A  E1)  P(A  E 2 )  ...  P(A  E n )

a result known as the theorem of total probability . This

result was actually used earlier in example (6) and is
illustrated once more in the following example .

Dr. Marwa Mostafa

Example (7): A calculator manufacturer buys the same
integrated circuit from three different suppliers. Call them I,
II, III. From past experience, 1 percent of the circuits
supplied by I have been defective, 3 percent of those
supplied by II have been defective, and 4 percent of those
supplied by III have been defective. Granted that this
manufacturer buys 30 percent of his circuits from I,
50 percent from II, and the rest from III.
What is the probability that an integrated circuit, checked just
before final assembly into a calculator is found to be
defective.

Dr. Marwa Mostafa

Solution: Let
A : The event that the chip is found defective , and let

E1 , E2 , E3 be the events that the chip selected was

manufactured by I , II , III , respectively , and we have

P(E1 )  0.3 , P ( E 2 )  0 .5 , P ( E 3 )  0 .2 ,
, ,

Thus

= (0.01)(0.3)+(0.03)(0.5)+(0.04)(0.2)

= 0.003 + 0.015 + 0.008 = 0.026

Dr. Marwa Mostafa
 Bayes' Rule
The theorem of total probability can be used to easily
establish Bayes' theorem, named after the Reverend T.
Bayes; the result is commonly credited with first being
published in 1764 in Bayes' posthumous memoirs.

Theorem (1): Let E1 , E2 , …, En be a partition of S . Then

for any event A  S

Dr. Marwa Mostafa , i = 1, 2, …, n

Proof: By definition

and since,
and

Then the result follows immediately, then

Dr. Marwa Mostafa

Example (8): Let us assume the same situation as
Example (7). If one circuit is selected from the box and tested,
and found defective. Find the probability that it came from

(a) box I (b) box II (c) box III

Solution:

(a)

(0.3)(0.01)
  0.115
(0.3)(0.01)  (0.5)(0.03)  (0.2)(0.04)
Dr. Marwa Mostafa
(b)

(0.5)(0.03)
  0.577
(0.3)(0.01)  (0.5)(0.03)  (0.2)(0.04)

(c)

(0.2)(0.04)
  0.308
(0.3)(0.01)  (0.5)(0.03)  (0.2)(0.04)
Dr. Marwa Mostafa
Example (9):
Four technicians regularly make repairs when breakdowns
occur on an automated population line. Janet, who services
20% of the breakdowns , makes an incomplete repair 1 time
in 20 ; Tom who services 60% of the breakdowns , makes an
incomplete repair 1 time in 10 ;Georgia, who services 15%
of the breakdowns , makes an incomplete repair 1 time in
10 ;and Peter who services 5% of the breakdowns , makes an
incomplete repair 1 time in 20 . For the next problem with
the production line diagnosed as being due to an initial repair
that was incomplete , what is the probability that this initial
repair was made by Janet?

Dr. Marwa Mostafa

Solution: Let
A : The event that the initial repair that was incomplete ,
and let
E1 , E2 , E3 and E4 be the events that the initial repair was
made by Janet , Tom , Georgia , and Peter, respectively , and
we have
P(E1 )  0.2 , P(E 2 )  0.6 , P(E 3 )  0.15, P(E 3 )  0.05

Thus , the probability that the initial repair was made by

Janet is given by

Dr. Marwa Mostafa

 (0.2)(0.05)
(0.2)(0.02)  (0.6)(0.1)  (0.15)(0.1)  (0.05)(0.05)  0.114

Dr. Marwa Mostafa

 Independent Events
Consider the situation where we have events A and B such
that
.

In other words, the occurrence of B had no impact on the odds

of occurrence of A. Here the occurrence of A is independent
of the occurrence of B.

Definition : Two events A and B are independent if and only

if

and .
Dr. Marwa Mostafa
Otherwise, A and B are dependent.
In other words, A and B are independent if and only if
P( A  B)
P( A | B)   P ( A) then, P( A  B )  P ( A) P ( B )
P( B)
and
P( A  B)
P ( B | A)   P ( B ) then, P( A  B )  P ( A) P ( B )
P( A)
We thus have the following definition.
Definition: Two events A and B are said to be independent iff

P(A  B)  P(A) P(B)

Dr. Marwa Mostafa

Example (10): Suppose that we toss 2 fair dice. Let A denote
the event that the sum of the dice is 6 and B denote the event
that the first die equals 4. Is A and B independent?
Solution: S = { (i , j ) : i , j = 1,2,3,4,5,6 } , and n(S) = 36 .
A = { (1, 5) , (5,1) , (2,4) , (4,2) , (3,3) } ,
n(A) = 5 , P(A) = 5/36 , and
B = { (4,1) , (4,2) , (4,3) , (4,4) , (4,5) , (4,6) } ,
n(B) =6 , P(B) = 6/36 = 1/6 ,
and
A ∩ B = { (4,2)} , n(A ∩ B ) = 1 ,P(A ∩B) = 1/36
5 1 5
Now, P(A ∩B) = 1/36 and P(A) P(B)= ( 36 )( 6 )  216
then . P ( A  B)  P ( A ) P ( B) Dr. Marwa Mostafa
. Hence A and B are not independent.
Example (11): If A and B are independent events in a sample
space S . Show that
(a) A and B are independent
(b) A and B are independent
Solution:
(a) P(A  B)  P(A  B)  1  P(A  B)  1  [P(A)  P(B)  P(A B)]

Since P(A  B)  P(A)P(B)

Then P(A  B)  1  [P(A)  P(B)  P(A)P(B)]

 [1  P(A)].[1  P(B)]  P(A).P(B)
Then , A and B are independent
Dr. Marwa Mostafa
(b)

P(A  B)  P(B)  P(A  B)  P(B)  P(A)P(B)

 P(B)[1  P(A)]  P(B)P(A)
Then A and B are independent

Dr. Marwa Mostafa

Example (12):page 70
If P(C) = 0.65 , P(D) = 0.40 , and P(C ∩ D) = 0.24 , are the
events C and D independent ?

Solution:

P(C∩D)= 0.24 ≠ P(A)P(B) = (0.65) (0.40) = 0.26.

Then the two events C and D are not independent .

Dr. Marwa Mostafa

Definition : The three events A, B, and C are said to be
independent if:

1- P(A∩B∩C) = P(A) P(B) P(C )

2- P(A∩B) = P(A) P(B)
3- P(A∩C) = P(A) P(C )
4- P(B∩C) = P(B) P(C )

Dr. Marwa Mostafa

Example (13): Assume a fair coin is flipped two times and
define the three events.
A: Head on first flip.
B: Head on second flip.
C: Same face on both flips.
Are A , B , and C independent ?

Dr. Marwa Mostafa

Solution:
A = {(H, H), (H, T)}
B = {(H, H), (T, H)}
C ={(H, H) , (T, T)}
P(A) = P(B) = P(C) = 1/2 and that
P(A∩B) = P(A∩C) = P(B∩C) =1/4
so equations 2,3, and 4 of the last definition are satisfied . But
since P(A∩B∩C ) = ¼ ≠ (1/2 )3 .
Then A, B, and C are called pairwise independent events

Dr. Marwa Mostafa