Probability and Statistics

Lecture: Conditional probability

1
Example

Alice and Bob toss two fair coins separately. Denote the event that Alice gets a
H by EA and the event that Bob gets a H by EB .
We know that P(EA ) = P(EB ) = 1/2.

2
Example

Alice and Bob toss two fair coins separately. Denote the event that Alice gets a
H by EA and the event that Bob gets a H by EB .
We know that P(EA ) = P(EB ) = 1/2.
 The sample space you are considering is: HT, TH, TT, HH. So EA = {HT,
HH} and EB = { TH , HH} .

2
Example

Alice and Bob toss two fair coins separately. Denote the event that Alice gets a
H by EA and the event that Bob gets a H by EB .
We know that P(EA ) = P(EB ) = 1/2.
 The sample space you are considering is: HT, TH, TT, HH. So EA = {HT,
HH} and EB = { TH , HH} .
 So the union is really
P(EA ∪ EB ) = P(EA ) + P(EB ) − P(EA ∩ EB ) = 3/4.

2
Example

Alice and Bob toss two fair coins separately. Denote the event that Alice gets a
H by EA and the event that Bob gets a H by EB .
We know that P(EA ) = P(EB ) = 1/2.
 The sample space you are considering is: HT, TH, TT, HH. So EA = {HT,
HH} and EB = { TH , HH} .
 So the union is really
P(EA ∪ EB ) = P(EA ) + P(EB ) − P(EA ∩ EB ) = 3/4.
 May be what you are really thinking about is independent and not
disjoint events.

2
Partial information

 So far, we have assumed we know nothing about the outcome of our

experiment.
 Sometimes, however, we have partial information that may affect the
likelihood of a given event.
 The experiment involves rolling a die. You are told that the number is
odd.
 The experiment involves the weather tomorrow. You know that the
weather today is rainy.
 The experiment involves the presence or absence of a disease. A
blood test comes back positive.
 In each case, knowing about some event B (e.g. “it is raining
today”) changes our beliefs about event A (“Will it rain
tomorrow?”).
 We want to update our probability law to incorporate this new
knowledge.

3
Conditional Probability
Original problem:
What is the probability of some event A.
e.g. What is the probability that we roll a number less than 4?
In other words, what is P(A)?
This is given by our probability law.

4
Conditional Probability

Original problem:
 What is the probability of some event A.
e.g., What is the probability that we roll a number less than 4?
 In other words, what is P(A)?
 This is given by our probability law.
New problem:
 Assuming event B (equivalently given event B ), what is the
probability of event A?
e.g., Given that the number rolled is an odd number, what is the
probability that it is less than 4?
 We call this the conditional distribution of A given B .
 We write this as P(A|B)
 Read | as “given” or “conditioned on the fact that”.
 Our conditional probability is still describing “the probability of
something”, so we expect it to behave like a probability distribution.

4
Conditional Probability

 Consider rolling a fair 6-sided die (uniform, discrete probability

distribution).
 Let A be the event “outcome is equal to 1’.
 What is P(A)?
 Let’s now assume that the number rolled is an odd number.
 What is the set, B, that we are conditioning on?

 What do you think P(A|B) should be?

5
Conditional Probability

 You are conditioning on event B . So your new sample space is

{ 1, 3, 5} instead of { 1, 2, 3, 4, 5, 6} .

6
Conditional Probability

 You are conditioning on event B . So your new sample space is

{ 1, 3, 5} instead of { 1, 2, 3, 4, 5, 6} .
 Each of these are equally likely. So P(A|B) = 1/3.

6
Conditional Probability

 You are conditioning on event B . So your new sample space is

{ 1, 3, 5} instead of { 1, 2, 3, 4, 5, 6} .
 Each of these are equally likely. So P(A|B) = 1/3.
Formally, if all outcomes are equally likely, we have
# elements in A ∩ B
P(A|B) =
# elements in B

6
Conditional Probability

 You are conditioning on event B . So your new sample space is

{ 1, 3, 5} instead of { 1, 2, 3, 4, 5, 6} .
 Each of these are equally likely. So P(A|B) = 1/3.
Formally, if all outcomes are equally likely, we have
# elements in A ∩ B
P(A|B) =
# elements in B

P(A ∩ B)
More generally, P(A|B) = .
P(B)

6
Conditional Probability

 You are conditioning on event B . So your new sample space is

{ 1, 3, 5} instead of { 1, 2, 3, 4, 5, 6} .
 Each of these are equally likely. So P(A|B) = 1/3.
Formally, if all outcomes are equally likely, we have
# elements in A ∩ B
P(A|B) =
# elements in B

P(A ∩ B)
More generally, P(A|B) = .
P(B)
A conditional probability is only defined if P(B) > 0.

6
Conditional Probability Axioms

 Nonnegativity – Check.
 Normalization –

 Additivity –

7
Conditional Probability Axioms

 Nonnegativity – Check.
 Normalization – Your new universe is now B , and we know that
P(B|B) = 1.

 Additivity –

7
Conditional Probability Axioms

 Nonnegativity – Check.
 Normalization – Your new universe is now B , and we know that
P(B|B) = 1.

 Additivity – P(A 1 ∪ A2|B) = P(A 1|B) + P(A 2|B) for two disjoint sets,
A1 and A2.

7
Conditional Probability Axioms

 Nonnegativity – Check.
 Normalization – Your new universe is now B , and we know that
P(B|B) = 1.

 Additivity – P(A 1 ∪ A2|B) = P(A 1|B) + P(A 2|B) for two disjoint sets,
A1 and A2.

Ω
B Conditioning on B
— − − − − − − −→ B
A1 A2
A2 ∩ B A1 ∩ B

7
Conditional Probability Axioms

 Nonnegativity – Check.
 Normalization – Your new universe is now B , and we know that
P(B|B) = 1.

 Additivity – P(A 1 ∪ A2|B) = P(A 1|B) + P(A 2|B) for two disjoint sets,
A1 and A2.

Ω
B Conditioning on B
— − − − − − − −→ B
A1 A2
A2 ∩ B A1 ∩ B

Using additivity, P((A1 ∪ A2 ) ∩ B) = P(A1 ∩ B) + P(A2 ∩ B) , so

P(A1 ∩ B) + P(A2 ∩
P(A1 ∪ A |B)
B) P(B) = P (A1 |B ) + P (A2 |B )
= 2

7
Properties of conditional probability

If P(B) > 0,

If Ai for i ∈ {1, . . . , n} are all pairwise disjoint, then

𝑛
𝑃 ¿ 𝑖=1 ¿ 𝑛 𝐴 𝑖∨ 𝐵=∑ 𝑃 ( 𝐴¿ ¿ 𝑖∨ 𝐵) ¿
𝑖 =1

8
Properties of conditional probability

If P(B) > 0,

If Ai for i ∈ {1, . . . , n} are all pairwise disjoint, then

𝑛
𝑃 ¿ 𝑖=1 ¿ 𝑛 𝐴 𝑖∨ 𝐵=∑ 𝑃 ( 𝐴¿ ¿ 𝑖∨ 𝐵) ¿
𝑖 =1

If A1 ⊆ A2, then P(A1 |B) ≤ P(A2|B).

8
Properties of conditional probability

If P(B) > 0,

If Ai for i ∈ {1, . . . , n} are all pairwise disjoint, then

If A1 ⊆ A2, then P(A1 |B) ≤ P(A2|B).

P(A1 ∪ A2 |B) = P(A1 |B) + P(A2 |B) − P(A1 ∩ A2 |B)

8
Properties of conditional probability

If P(B) > 0,

If Ai for i ∈ {1, . . . , n} are all pairwise disjoint, then

𝑛
𝑃 ¿ 𝑖=1 ¿ 𝑛 𝐴 𝑖∨ 𝐵=∑ 𝑃 ( 𝐴¿ ¿ 𝑖∨ 𝐵) ¿
𝑖 =1

If A1 ⊆ A2, then P(A1 |B) ≤ P(A2|B).

P(A1 ∪ A2 |B) = P(A1 |B) + P(A2 |B) − P(A1 ∩ A2 |B)

Union bound: P(A1 ∪ A2 |B) ≤ P(A1 |B) + P(A2 |B)

8
Properties of conditional probability

If P(B) > 0,

If Ai for i ∈ {1, . . . , n} are all pairwise disjoint, then

𝑛
𝑃 ¿ 𝑖=1 ¿ 𝑛 𝐴 𝑖∨ 𝐵=∑ 𝑃 ( 𝐴¿ ¿ 𝑖∨ 𝐵) ¿
𝑖 =1

If A1 ⊆ A2, then P(A1 |B) ≤ P(A2|B).

P(A1 ∪ A2 |B) = P(A1 |B) + P(A2 |B) − P(A1 ∩ A2 |B)

Union bound: P(A1 ∪ A2 |B) ≤ P(A1 |B) + P(A2 |B)

8
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?

9
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?
 Sample space for three coin tosses
{ HHH, HHT , HTH , HTT , THH , THT , TTH , T T T } .

9
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?
 Sample space for three coin tosses
{ HHH, HHT , HTH , HTT , THH , THT , TTH , T T T } .
 A = { HTH , THT } and A ∩ B = { HTH } .

9
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?
 Sample space for three coin tosses
{ HHH, HHT , HTH , HTT , THH , THT , TTH , T T T } .
 A = { HTH , THT } and A ∩ B = { HTH } .
 We have P(B) = 4/8 and P(A ∩ B) = 1/8 and so P(A|B) = 1/4.

9
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?
 Sample space for three coin tosses
{ HHH, HHT , HTH , HTT , THH , THT , TTH , T T T } .
 A = { HTH , THT } and A ∩ B = { HTH } .
We have P(B) = 4/8 and P(A ∩ B) = 1/8 and so P(A|B) = 1/4.

 Your new world/sample space when you condition is

B = {HHT, HTH, HTT, HHH}. Each of these are equally likely. Out of
these 1 event satisfies alternating heads and tails.

9
Example: Coin toss

Consider the experiment of tossing a fair coin three times. What is the probability
of getting alternating heads and tails conditioned on the event that your first
toss gives a head?
 Notation: Let A := {Tosses yield alternating tails and heads} and
B := { The first toss is a head}.
 We want P(A|B) .
 What if you wanted to follow the formula?
 Sample space for three coin tosses
{ HHH, HHT , HTH , HTT , THH , THT , TTH , T T T } .
A = { HTH , THT } and A ∩ B = { HTH } .
 We have P(B) = 4/8 and P(A ∩ B) = 1/8 and so P(A|B) = 1/4.

 Your new world/sample space when you condition is

B = {HHT, HTH, HTT, HHH}. Each of these are equally likely. Out of
these 1 event satisfies alternating heads and tails.
 So, P(A|B) = 1/4.