EEF 271E

PROBABILITY AND
STATISTICS
FALL 2022

Lecture 03
Conditional Probability, Bayes' Theorem

Dr. Ramazan Çağlar

Lecture Notes 03 1
Conditional Probability, Bayes' Theorem
 Lesson Overview

Conditional Probability

• Conditional Probability
• Total Probability
• Bayes’ Theorem

2
Lecture Notes 03 Conditional Probability, Bayes' Theorem
 Lesson Overview

Conditional Probability

Let A, B  F.
Then the
‘‘conditional probability of A given B’’
denoted is defined as

(It is undefined if )

3
Lecture Notes 03 Conditional Probability, Bayes' Theorem
 has the interpretation of the likelihood of
event A occurring, given that B has
occurred. Why?

Intuitive explanation 1:
If B occurs, the sample

space is ‘‘reduced’’ to .
A
A B B Then A also occurs if and
only if AB occurs. We then
renormalize P(AB) by
1/P(B)
4
Lecture Notes 03 Conditional Probability, Bayes' Theorem
Intuitive explanation 2:
(Using relative frequency approach)

Say an experiment is performed N times,

and B occurs NB times. Of this NB trials, A

also occurs NAB times. Then

P(A occurs given that B occurs)

5
Lecture Notes 03 Conditional Probability, Bayes' Theorem
Intuitive explanation 3:

The evet AB can be thought of as

‘’B occurs ‘’ B occurs ‘’ A occurs

and A ’’ given that B
occurs ’’ occurs ’’

P( A ∩ 𝐵)=𝑃 ( 𝐵)∙ 𝑃 ( 𝐴| 𝐵 )

Lecture Notes 03 Conditional Probability, Bayes' Theorem 6

 Special cases for A and B

(1) (i.e., A and B are ‘’disjoint’’

or ‘’mutually exclusive’’)

Then ( If P(B) > 0 )

(2) If B occurs, A also occurs.

(3) Then

Lecture Notes 03 Conditional Probability, Bayes' Theorem 7

 Check that conditional Probability satisfies
axioms of Probability

For any event with ,

(1) ? (For any A (YES)

(2) ? (YES, since
(special case 2 on previous
page)
(3) If

Conditional Probability, Bayes' Theorem 8

Lecture Notes 03
 Check that conditional Probability satisfies
axioms of Probability

(3) If ?
YES,
because and disjoint events

Lecture Notes 03 Conditional Probability, Bayes' Theorem 9

 Example 1:
Expt. Roll a fair die
Sample space: Ω = {1, 2, 3, 4, 5, 6}
Define ,

(from Classical
arguments)

Lecture Notes 03 Conditional Probability, Bayes' Theorem 10

Example 1 (cont.): Expt. Roll a fair die

Sample space: Ω = {1, 2, 3, 4, 5, 6}

Define ,

i.e. probability of rolling 1 or 2, given that an

even number was gotten is 1/3.

Lecture Notes 03 11
Conditional Probability, Bayes' Theorem
Example 1 (cont.): Exp. Roll a fair die
Sample space: Ω = {1, 2, 3, 4, 5, 6}

Define ,

Next

i.e. if the outcome was 1 or 2, the probability

that the outcome was given is 1/2.

Lecture Notes 03 12
Conditional Probability, Bayes' Theorem
MULTIPLICATION AND TOTAL PROBABILITY RULES

Multiplication Rule:
The definition of conditional probability

can be rewritten to provide a general expression for

the probability of the intersection of two events.

This formula is referred to as a multiplication rule for

probabilities.

Lecture Notes 03 13
Conditional Probability, Bayes' Theorem
Total Probability Rule

The multiplication rule is useful for

determining the probability of an event
that depends on other events.

Lecture Notes 03 14
Conditional Probability, Bayes' Theorem
 Probability Level of Probability
Example 2: of Failure Contamination of Level
0.1 High 0.2
0.005 Not High 0.8

Suppose that in semiconductor manufacturing the probability

is 0.10 that a chip that is subjected to high levels of
contamination during manufacturing causes a product
failure.

The probability is 0.005 that a chip that is not subjected

to high contamination levels during manufacturing causes a
product failure.

In a particular production run, 20% of the chips are

subject to high levels of contamination. What is the
15
probability
Lecture Notes 03 that a product using
Conditional oneTheorem
Probability, Bayes' of these chips fails?
Clearly, the requested probability depends on
whether or not
the chip was exposed to high Levels of contamination.

We can solve this problem by the following reasoning.

For any event
B, we can write B as the union of the part of B in A and
the part of B in A’ .

Lecture Notes 03 16
Conditional Probability, Bayes' Theorem
This result is shown in the Venn diagram Because A
and A’ are mutually exclusive, AB and A’B are
mutually exclusive. Therefore, from the probability of
the union of mutually exclusive events and the
Multiplication Rule, the
following total probability rule is obtained.

Total Probability Rule (two events)

For any events A and B,

Lecture Notes 03 17
Conditional Probability, Bayes' Theorem
Solution of the Example 2 Probability Level of Probability
of Failure Contamination of Level
0.1 High 0.2
0.005 Not High 0.8

Let F denote the event that the product fails.

Let H denote the event that the chip is exposed to high

contamination during manufacture. Then

P(F|H)=0.100 and P(H)=0.2, so P(F∩H) = P(F|H).P(H) =

0.020

P(F|Hc)=0.005 and P(Hc)=0.8, so P(F∩Hc) = P(F|Hc).P(Hc) =

0.004

P(F) = P(F∩ H) + P(F∩ Hc) (Using Total Probability rule)

Lecture Notes 03 18
= 0.020 + 0.004 = 0.024
Conditional Probability, Bayes' Theorem
 Conditional probability and total probability

Suppose are mutually exclusive events such

that

Then

(disjoint union)

Lecture Notes 03 Conditional Probability, Bayes' Theorem 19

 Conditional probability and total probability

So

‘’weighted sum of
‘’total probability’’
conditional
probabilities’’
20
Lecture Notes 03 Conditional Probability, Bayes' Theorem
Example 3:
Bin numbers
Ohms
1 2 3 Total
10 Ω 500 0 200 700
100 Ω 300 400 600 1300
1000 Ω 200 600 200 1000
Total Ω 1000 1000 1000 3000

Experiment:
A bin selected at random,
and a resistor from that bin is selected at
random.
Let A = { Resistor has resistance 10 },
Find P(A)
Lecture Notes 03 21
Conditional Probability, Bayes' Theorem
Solution of Example 3:
Let be the event ‘’ ith bin is chosen’’
Then

Next

Lecture Notes 03 22
Conditional Probability, Bayes' Theorem
Solution of the Example 3 (continued):

Thus

Lecture Notes 03 Conditional Probability, Bayes' Theorem 23

 Solution of the example 3 (a shown in single
page):
Let be the event ‘’ ith bin is chosen’’. Then

Next

Thus

24
Lecture Notes 03 Conditional Probability, Bayes' Theorem
Example 4:
Let us ask another
question. Suppose we are
given that the resistor
drawn has resistance 10
.
What is the Probability that it came from bin 1
(or bin 2, or bin 3).
In other word we want
P(B1|A) , P(B2|A) , P(B3|A)

These probabilities are also called ‘’a posteriori’’

probabilities (after the fact)
25
Lecture Notes 03 Conditional Probability, Bayes' Theorem
Recall that

(or in general ) so

or in general

This is called BAYES’ THEOREM

Lecture Notes 03 26
Conditional Probability, Bayes' Theorem
Example 4 and its solution
(cont.):
Suppose we are given that the
resistor drawn has resistance 10 .
What is the Probability that it came
from bin 1 (or bin 2, or bin 3).
Applying Bayes’ theorem to this example,

Note that (why?)

Lecture Notes 03 Conditional Probability, Bayes' Theorem 27

Example 5:
Suppose that an Integrated Supplier Failure probability Number
P(F|supplier) ordered
Circuits (IC) is used in a
A 0,10 1000
device. The producer orders B 0,05 2000
ICs from 3 different C 0,08 3000
suppliers, named A, B and
C respectively.
The number of ordered IC’s and the conditional
failure probability are given in a table above.
Expt: ICs mixed together, one picked at random.
a) Find P(failure)
b) Given that the device fails, find P ({supplier was
A}).
Lecture Notes 03 28
Conditional Probability, Bayes' Theorem
 Solution of the Example 5:

Total Probability formula:

From classical definition:

So

Lecture Notes 03 29
Conditional Probability, Bayes' Theorem
b) Given that the device fails, find P ({supplier was
A}).

So we want

Lecture Notes 03 30
Conditional Probability, Bayes' Theorem
Example 6:
Suppose the longevity of an Integrated
Circuits (IC) is described by:

( is a parameter that is usually measured

empirically. This is called an “exponential
law” )

Lecture Notes 03 31
Conditional Probability, Bayes' Theorem
Example 6 (cont.):

Suppose that an IC manufacturing

process produces both “good” and
“bad” ICs with probabilities (1-p) and p
respectively.
(1-p) + p = 1.0
Let the lifetime of these IC follows the
exponential law with parameters
respectively.
, and )
Experiment: A chip is selected at
random.
Lecture Notes 03 32
Conditional Probability, Bayes' Theorem
Example 6 (cont.):

a) Find the probability that IC still

function after t second usage.
Event A: IC chip still function after t
seconds.
Event G: IC is a good chip
Event B: IC is a bad chip

Lecture Notes 03 33
Conditional Probability, Bayes' Theorem
Example 6 (cont.):

b) Suppose a chip is found to be still working

after t second; what is the probability that it
is a “good” chip?

(Interpret this result for very small t, and very

large t... )

Lecture Notes 03 34
Conditional Probability, Bayes' Theorem
Example 7:
A producer wants to maximize its sale of special
product which influenced directly weather
conditions. In this sense, it must predict temperature
and wind. Let's denote the event "warm day" as
event A and the event "windy day" as event B.
The weather forecast indicates that the probability of
a windy day is 0.80, the probability of a warm day is
0.30, and the probability of a warm and windy day is
0.28
a) What is the probability of the event "windy day
will be warm"?
b) What is the probability of the event "warm day
will be windy"?
c) What is the probability of the event "windy and
not warm"?
Lecture Notes 03 35
Conditional Probability, Bayes' Theorem
Solution of the Example 7:

a) Denote the event "not warm" as.

Then
since A and are complements.
Therefore,

Lecture Notes 03 36
Conditional Probability, Bayes' Theorem
Example 8:
A manufacturing firm puts into service three machines (A, B, C)
to produce the Integrated Circuits (ICs).
• 40% of the firm’s daily ICs production comes from machine A,
• 35% of the firm’s daily ICs production comes from machine B
and,
• 25% of the firm’s daily ICs production comes from machine C.
• 1% of ICs produced on machine A are defective, while the
defective rates for machines B and C are 2% and 3%
respectively.
• If one IC is selected at random from a day’s production and
is found to contain a defect,
a) What is the probability that the selected IC is found to
contain a defect?
b) What is the probability that it was produced on machine B?

Lecture Notes 03 37
Conditional Probability, Bayes' Theorem
Solution of the Example 8:
Let use the symbol D for the defective IC.

a)

b)

Lecture Notes 03 38
Conditional Probability, Bayes' Theorem
Example 9:
Consider the system below, and assume that each
component functions independently of the others with
probability p = 0.8.
What is the probability that the entire system works?

2 3
1 5
4

Lecture Notes 03 39
Conditional Probability, Bayes' Theorem
Answer for the example 9:
Since each component functions independently of the
others, the probability of the system success that is the
system functions to achieve their duty is given by following
equations.
C2 C3
C1 C5
C4

P(S) =P (the system functions) = P(C1).P[(C2C3)C4)].P(C5)

Let’s rename (C2C3) as D, and (DC4) as E then,

D
C1 C5 =
C1 E C5
C4

Lecture Notes 03 40
Conditional Probability, Bayes' Theorem
Answer for example 9 (continued):

Lecture Notes 03 41
Conditional Probability, Bayes' Theorem
Example 9:
Two persons have a date at a given time, and each will
arrive at the meeting place with a delay between 0 and 1
hour. The first to arrive will wait for 15 minutes and will
leave if the other has not yet arrived.
What is the Probability that they will meet?
Solution:
Let the persons be Romeo & Juliet.
Assume that they will meet at 0 o’clock.
Romeo delays for x minutes
Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.

Lecture Notes 03 42
Conditional Probability, Bayes' Theorem
Solution:
Romeo delays for x minutes
Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.
y
15 minutes 1/4 hours
1
Ÿ M event

M = { (x, y) | |x-y| ”1/4 , 0 ”x < 1, 0 ”y < 1 }

Ÿ = [(0,1)x(0,1)] : square area ( Sample Space)

1/4

x
1/4 1

Lecture Notes 03 43
Conditional Probability, Bayes' Theorem
Solution:
Romeo delays for x minutes, Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.
y
15 minutes 1/4 hours
1
Ÿ M event

M = { (x, y) | |x-y| ”1/4 , 0 ”x < 1, 0 ”y < 1 }

Ÿ = [(0,1)x(0,1)] : square area ( Sample Space)

1/4

x
1/4 1

Lecture Notes 03 44
Conditional Probability, Bayes' Theorem