PROBABILITY THEORY

R.P.G.AMITHANI (BSC (HONS) USJ, MSC UOM)

R.P.G.AMITHANI 5/6/2024 1
WHY LEARN PROBABILITY?

 Nothing in life is certain. In everything we do, we measure the chances of successful

outcomes, from business to medicine to the weather
 It provides a bridge between descriptive and inferential statistics

Probability
Population Sample
Statistics
 Know about the Population and predict about the sample :- probabilistic reasoning
R.P.G.AMITHANI 5/6/2024 2
 Don’t know about the population :- statistical reasoning
WHAT IS MEANT BY PROBABILITY?

 Probability is a field of mathematics that gives us the language and tools to quantify the uncertainty of events and
reason in a principal manner.
 Handing uncertainty is typically described using everyday words like chance, luck and risk

 0≤Probability≤1;
 various grades of uncertainty Impossible (probability = 0) No way May be possibly Probably No worries Certain
(Probability = 1)

R.P.G.AMITHANI 5/6/2024 3
 Impossible (probability = 0)
No way
May be possibly
Probably
No worries
Certain (Probability = 1)

R.P.G.AMITHANI 5/6/2024 4
 0 ≤ Probability ≤ 1

R.P.G.AMITHANI 5/6/2024 5
BASIC CONCEPTS

 Experiment is the process by which an observation (or measurement)

is obtained.
 The set of all simple events of an experiment is called the sample
space, S.
 An event is an outcome of an experiment, usually denoted by a capital
letter.

R.P.G.AMITHANI 5/6/2024 6
 Examples: Experiment and Event
Experiment: Number of tomato production in a plant
Event:
A: observe more than 10 tomatoes in a plant
B: observe less than 2 tomatoes in a plant
Experiment: Toss a die
Event:
A: observe an odd number
B: observe a number greater than 2

R.P.G.AMITHANI 5/6/2024 7
 Experiment Sample Space
 Toss a Coin, Note Face {Head, Tail}
 Toss 2 Coins, Note Faces {HH, HT, TH, TT}
 Select 1 Card, Note Kind {2♥, 2♠, ..., A♦}
 Select 1 Card, Note Color {Red, Black}
 Play a Football Game {Win, Lose, Tie}
Inspect a Part, Note Quality {Defective, Good}

R.P.G.AMITHANI 5/6/2024 8
 Simple event
(Elementary event)
• A simple event includes one and only one
outcome.
• Denoted by 𝐸1, 𝐸2, 𝐸3 and so forth.

EVENT

Compound event
(Composite event)

• Consists of more than one outcome.

• Denoted by 𝐴, 𝐵, 𝐶 and so forth.
R.P.G.AMITHANI 5/6/2024 9
COMPUTING PROBABILITY

 Probability, which gives the likelihood of occurrence of an event, is

denoted by 𝑃.

 The probability of a simple event 𝐸 𝑖is denoted by 𝑷(𝑬𝒊), and the

probability of a compound event 𝐴 is denoted by 𝑃(𝐴).

R.P.G.AMITHANI 5/6/2024 10
PROPERTIES OF PROBABILITY
 The probability of an event always lies in the range 0 to 1.

 The probability of a simple or compound event is never less than 0 or greater than 1.
 𝟎 ≤ 𝐏 𝑬𝒊 ≤ 𝟏 𝐚𝐧𝐝 𝟎 ≤ 𝐏 𝐀 ≤ 𝟏

 An event that cannot occur has zero probability; such an event is called an impossible event. For

impossible event 𝐴 ∶ 𝑃(𝐴) = 0

 An event that is certain to occur has a probability equal to 1 and is called a sure event. For sure event

𝐵 ∶ 𝑃(𝐵) = 1

The sum of the probabilities of all simple events (or final outcomes) for an experiment, denoted by
𝑷(𝑬𝒊 )is always 𝟏
R.P.G.AMITHANI 5/6/2024 11
𝑷 𝑬𝟏 + 𝑷 𝑬𝟐 + 𝑷 𝑬𝟑 + … = 𝟏
 Example 1
Consider the experiment of flip of a coin at once. The sample space of this
experiment is 𝑆 = {𝐻, 𝑇}. Thus,
𝑃 𝐻 +𝑃 𝑇 =1

R.P.G.AMITHANI 5/6/2024 12
 Example 2

For the experiment of flip of a coin at twice. Then the sample space
is 𝑆 = 𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇 . Thus,
𝑃 𝐻𝐻 + 𝑃 𝐻𝑇 + 𝑃 𝑇𝐻 + 𝑃 𝑇𝑇 = 1

R.P.G.AMITHANI 5/6/2024 13
RULES OF EVENT OPERATIONS

What is Set
Well- defined collection of distinct objects

R.P.G.AMITHANI 5/6/2024 14
 INTERSECTION OF EVENTS

 The intersection of two events, A and B, is the event that both A and B occur when the experiment
is performed. We write A B.
 Which contains all elements that are common to all events

R.P.G.AMITHANI 5/6/2024 15
EXAMPLE

 Consider the experiment of rolling a single die and find the intersection of two
events.
A:The number rolled is odd
B:The number rolled is greater than three

R.P.G.AMITHANI 5/6/2024 16
Consider the following sets and look at the Venn diagram to identify
intersections.
𝑆 = {𝐴, 𝐵, 𝐶, 𝐷, 𝐸, 𝐹, 𝐺, 𝐻} 𝑀 = {𝐴, 𝐶, 𝐷, 𝐸, 𝐹}
𝑁 = {𝐴 , 𝐵, 𝐹} 𝐾 = {𝐵, 𝐶 , 𝐹, 𝐺}

R.P.G.AMITHANI 5/6/2024 17
UNION OF EVENTS

 The union of two events, A and B, is the event that either A or B or both occur
when the experiment is performed.
 We write
A B
 Which contains all elements that belong to all events

R.P.G.AMITHANI 5/6/2024 18
 𝑀 = {𝐴, 𝐶, 𝐷, 𝐸, 𝐹} 𝑀 = {1,3,5,8}
𝑁 = {𝐴 , 𝐵, 𝐹} 𝑁 = {2,5,9,11,71}

𝑀 ∪ 𝑁 = {𝐴 , 𝐵, 𝐶, 𝐷, 𝐸, 𝐹} 𝑀 ∪ 𝑁 = {1,2,3,5,8,9,11,71}

All the elements in Set M and set N

R.P.G.AMITHANI 5/6/2024 19
DIFFERENCE OF TWO EVENTS (COMPLEMENT)

 The complement of an event 𝐴 in a sample space 𝑆, denoted as 𝐴𝑐, is the

collection of all outcomes in 𝑆 that are not elements of the set 𝐴

Probability Rule for Complements

𝑷 ( 𝑨𝒄)=𝟏 − 𝑷 ( 𝑨 )
R.P.G.AMITHANI 5/6/2024 20
 Example

Consider the experiment of rolling a single die and find the complement
of the following events.
𝐸 : “the number rolled is prime”.
𝐸 = {2,3,5} 𝑡ℎ𝑒𝑛 𝑡ℎ𝑒 𝑐𝑜𝑚𝑝𝑙𝑒𝑚𝑒𝑛𝑡 𝑜𝑓 𝐸, 𝐸𝑐 = 𝑆− 𝐸 = {1,4,6}

R.P.G.AMITHANI 5/6/2024 21
• Difference of an event A from another event B denoted by A \ B (A-B)
• The event consisting of those elements of A which does not belong to B

R.P.G.AMITHANI 5/6/2024 22
Mutually Exclusive Events

 Let 𝐴1 , 𝐴2, … , 𝐴𝑛 are events of sample space ‘𝑆’

 If two or more above events can not occur simultaneously in a single
trial of an experiment. (Events do not occur at the same time.)
 Then those events are mutually exclusive events
 If sets A and B are mutually exclusive events 𝑃 𝐴 ∩ 𝐵 = 0.

R.P.G.AMITHANI 5/6/2024 23
 𝐴 ∩𝐵 =∅
R.P.G.AMITHANI 5/6/2024 24
 Example:
Three coins are tossed at the same time. We say A as the event of
receiving at least 2 heads. Likewise, B denotes the event of getting no heads
and C is the event of getting heads on the second coin. Which of these is
mutually exclusive?
A = {HHT, HTH, THH, HHH}.
B = {TTT}
C = {THT, HHH, HHT, THH}
B & C and A & B are mutually exclusive

R.P.G.AMITHANI 5/6/2024 25
 Experiment:- Tossing adice
S= {1,2,3,4,5,6}

Event 1 (A1) :- Having an odd number

A1= {1, 3,5}

Event 2 ( A 2) :- Having an even number

A2= {2,4,6}

A1 ∩ A 2 = 0
R.P.G.AMITHANI 5/6/2024 26
Collectively Exhaustive Events

 Let 𝐴1 , 𝐴2, … , 𝐴𝑛 are events of Sample space ‘𝑆’

 If A1  A2 ... An  S
 Then 𝐴1 , 𝐴2, … , 𝐴𝑛 are collectively exhaustive events

R.P.G.AMITHANI 5/6/2024 27
 Collectively Exhaustive Events and Mutually Exclusive Events
Example
S = {1, 2, 3, 4, 5, 6}
A = {1, 3, 5} , B = {2, 4, 6} , C = {1, 2, 6}
A  B  S ; A and B are collectively exhaustive
AC  S ; A and C are not collectively exhaustive
A  B  ; A and B are mutually exclusive

A  C  ; A and C are not mutually exclusive

R.P.G.AMITHANI 5/6/2024 28
Equally Likely Events

 Let 𝐴1 , 𝐴2, … , 𝐴𝑛 are events of a random experiment

 If 𝐴1 , 𝐴2 have the same chance to occur (equal probability).

 Then 𝐴1 & 𝐴2 are equally likely events

R.P.G.AMITHANI 5/6/2024 29
 1 1
Equal probability - Equal probability -
2 6

R.P.G.AMITHANI 5/6/2024 30
Three Conceptual ApproachesTo Probability

Conceptual
Approaches to
Probability

R.P.G.AMITHANI 5/6/2024 31
Axiomatic approach
 Classical Probability

• The classical probability rule is applied to compute the probabilities of

events for an experiment for which all outcomes are equally likely.

• According to the classical probability rule, the probability of a simple

event is equal to,
𝟏
𝑷 𝑬𝒊 =
𝑻𝒐𝒕𝒂𝒍 𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒔𝒊𝒎𝒑𝒍𝒆 𝒆𝒗𝒆𝒏𝒕𝒔 𝒇𝒐𝒓 𝒕𝒉𝒆 𝒆𝒙𝒑𝒆𝒓𝒊𝒎𝒆𝒏𝒕
𝟏
= ; 𝑺 𝒊𝒔 𝒂 𝒔𝒂𝒎𝒑𝒍𝒆 𝒔𝒑𝒂𝒄𝒆
𝒏(𝑺)
R.P.G.AMITHANI 5/6/2024 32
• The probability of a compound event(A) is equal to,

𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒐𝒖𝒕𝒄𝒐𝒎𝒆𝒔 𝒊𝒏 𝒆𝒗𝒆𝒏𝒕 𝑨

𝑷 𝑨 =
𝑻𝒐𝒕𝒂𝒍 𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒖𝒕𝒄𝒐𝒎𝒆𝒔 𝒇𝒐𝒓 𝒕𝒉𝒆 𝒆𝒙𝒑𝒆𝒓𝒊𝒎𝒆𝒏𝒕
𝒏(𝑨)
= ; 𝑺 𝒊𝒔 𝒂 𝒔𝒂𝒎𝒑𝒍𝒆 𝒔𝒑𝒂𝒄𝒆
𝒏(𝑺)

R.P.G.AMITHANI 5/6/2024 33
 Example : Consider the experiment, that rolling a fair die.

Here, 𝑆 = {1, 2, 3, 4, 5, 6}.

1. Probability of obtaining an odd number.
It is a compound event (𝐴) = {1,3,5}. Then,

𝑛(𝐴) 3 1
𝑃 𝐴 = = = .
𝑛(𝑆) 6 2

2. Probability of obtaining number 3 It is a simple event (𝐸). Then,

1 1
𝑃 𝐸 = = .
R.P.G.AMITHANI 𝑛(𝑆) 6 5/6/2024 34
 What is the probability that a dice with 12 faces returns a value 9 if each
face has a number starting from 1.

𝑆 = {1, 2, 3, 4, 5, 6,7,8,9,10,11,12}.
It is a simple event (𝐸).
1 1
𝑃 𝐸 = =
𝑛 𝑠 12

What is the probability that a dice with 12 faces returns a number 1 if

each face has a number starting from 1.

It is compound event (A).

𝑛(𝐴) 4
R.P.G.AMITHANI 𝑃 𝐴 = = ; 𝐴 = {1,10,11,12} 5/6/2024 35

𝑛 𝑠 12
Relative Frequency Concept Of Probability
• Consider a random experiment
• If the number of trails is n and the number of times the event (A) occurs is r,
then,

• The probability of the event A occurs,

r
P A : lim
n→∞ n
• Theoretically can be obtained
r
• In practice; P(A) = ,for large n
n
R.P.G.AMITHANI 5/6/2024 36
 Example : A number of families of a particular type were measured by the
number of children they contain. Consider the following table.

Use this information to calculate the probability that another family of this type
of will have,
a) 2 : P(A)=22/74
b) 3 or more : P(B)=12/74
c) Less than 2 children : P(C)=40/74

R.P.G.AMITHANI 5/6/2024 37
 Example : Approximate the probability of getting a 4 from this spinner?
P(A)=40/200

R.P.G.AMITHANI 5/6/2024 38
Subjective Probability
• Subjective probability is a type of probability derived from an individual's personal
judgment or own experience about whether a specific outcome is likely to occur.
• Probabilities are based on the knowledge, past experience, beliefs of the person
• Subjective probabilities differ from person to person and contain a high degree of
personal bias.
• It does not make use of mathematical calculation or data analysis; it rather depends upon
your gut feeling to predict the outcome.
Example;
1. A doctor may feel a patient has a 90% chance of full recovery
2. If someone claims that their favorite sports team has an 80% chance of winning the next
R.P.G.AMITHANI 5/6/2024 39

game, they are expressing a subjective probability.

 Axiomatic Approach
Axioms of Probability
Consider the sample space (S) of a random
experiment and let A be any defined event. Then Pr(A)
satisfies following axioms
 Pr(A) ≥ 0

 P(S) = 1

 If A1, A2,…An are mutually exclusive events, then,

P(A1 U A2 …U An ) = P (A1) + P(A2) + …+P(An)

R.P.G.AMITHANI 5/6/2024 40
 Axiomatic Theorems

 Theorem (1)
If  is the empty set; Pr() = 0

 Theorem (2)
If A is the complement of an event
P r( A)  1  P r( A)
R.P.G.AMITHANI 5/6/2024

41
  Theorem (3)
If A and B are any two events then,

Pr(A \ B)  Pr(A)  Pr(A  B)

Pr(𝐴 ∩ 𝐵′)  Pr(A)  Pr(AB)

 Theorem (4) (Addition Low of Probability)

If A and B are any two events then,

Pr(AB)  Pr(A)Pr(B) Pr(AB)

R.P.G.AMITHANI 5/6/2024 42
 Example
The results of an examination conducted in two parts for 50
candidates were recorded as follows.
20 passed in part I; and 15 passed in part II; and 18 failed in both part I and part
II. If out of these candidates one is selected at random, find the probability that
the candidate :
i) Passed in both part I and part II.
ii) Failed only in part I.
iii) Failed only in part I or part II.

R.P.G.AMITHANI 5/6/2024 43
Joint Probability & Marginal Probability

Joint Probability

 Probability of joint occurrence of two or more events

Marginal Probability
 Probability of one event, ignoring any information about the other event

R.P.G.AMITHANI 5/6/2024 44
R.P.G.AMITHANI 5/6/2024 45
Conditional Probabilities

 Conditional probability refers to the chances that some outcome occurs

given that another event has also occurred.
 Let 𝑃(𝐵) ≠ 0. Given an event 𝐵, the probability that the event 𝐴 happens given
that event 𝐵 has occurred is defined and denoted by,

R.P.G.AMITHANI 5/6/2024 46
R.P.G.AMITHANI 5/6/2024 47
 Independence of Two Events
 The two events 𝐴 and 𝐵 are independent, if the occurrence or nonoccurrence of
either one does not affect the probability of the occurrence of the other.
 We can redefine independence in terms of conditional probabilities:
Two events A and B are independent if and only if
P(A|B) = P(A) or P(B|A) = P(B)
Otherwise, they are dependent.
• If the events A and B are independent, then the probability that both A
and B occur is
P(A B) = P(A) P(B)
R.P.G.AMITHANI 5/6/2024 48
R.P.G.AMITHANI 5/6/2024 49
The Law of Total Probability

 Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive events (that is, one and only
one must happen). Then the probability of any event A can be written as

P(A) = P(A  S1) + P(A  S2) + … + P(A  Sk)

= P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk)

R.P.G.AMITHANI 5/6/2024 50
  Let B is any random event. Then

R.P.G.AMITHANI 5/6/2024 51
Bayes’ Rule

 Let A1, A2, A3, …, An are mutually exclusive and exhaustive

events, where Pr(Ai) ≠ 0 for all i =1, 2, …, n
and
 Let B is any random event (for which Pr(B) ≠ 0), thus for all i,

R.P.G.AMITHANI 5/6/2024 52
 Example 1
A container contains 24 bulbs which are 8 defectives, 6 partially defective and 10
acceptable. A bulb is chosen at random from the container and put in to use. If it
does not immediately fail, what is the probability it is acceptable ?

Answer
First step : Identify the sample space.

Let 𝐴: Acceptable, 𝑇𝐷: Totally defective,

𝑃𝐷: Partially defective
Then, sample space 𝑆 = {𝐴, 𝑇𝐷, 𝑃𝐷}.

R.P.G.AMITHANI 5/6/2024 53

5
Second step : find the probabilities of the outcome.

𝑁𝑜 𝑜𝑓 𝑎𝑐𝑐𝑒𝑝𝑡𝑎𝑏𝑙𝑒 𝑏𝑢𝑙𝑏𝑠 10
𝑃 𝐴 = =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑜 𝑜𝑓 𝑏𝑢𝑙𝑏𝑠 24

𝑁𝑜 𝑜𝑓 𝑡𝑜𝑡𝑎𝑙 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒 𝑏𝑢𝑙𝑏𝑠 8

𝑃 𝑇𝐷 = =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑜 𝑜𝑓 𝑏𝑢𝑙𝑏𝑠 24

𝑁𝑜 𝑜𝑓 𝑝𝑎𝑟𝑡𝑖𝑎𝑙 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒 𝑏𝑢𝑙𝑏𝑠 6

𝑃 𝑃𝐷 = =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑜 𝑜𝑓 𝑏𝑢𝑙𝑏𝑠 24
R.P.G.AMITHANI 5/6/2024 54

5
 Third step : define the events of the interest.
It does not immediately fail →It is not defective (i.e. it may be partially
defective or acceptable one).
𝑬𝟏: 𝑰𝒕 𝒊𝒔 𝒏𝒐𝒕 𝒂 𝒅𝒆𝒇𝒆𝒄𝒕𝒊𝒗𝒆 𝒃𝒖𝒍𝒃
𝑬𝟐: 𝑰𝒕 𝒊𝒔 𝒂𝒏 𝒂𝒄𝒄𝒆𝒑𝒕𝒂𝒃𝒍𝒆 𝒃𝒖𝒍𝒃
So, the probability that the chosen bulb is acceptable given that it is not a defective bulb.
It gives,

𝑃(𝐸1 ∩ 𝐸2 ) 𝑃(𝑛𝑜𝑡 𝑎 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒 ∩ 𝑎𝑐𝑐𝑒𝑝𝑡𝑎𝑏𝑙𝑒)

𝑃 𝐸2 𝐸1 ) = =
𝑃(𝐸1 ) 𝑃(𝑛𝑜𝑡 𝑎 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒)
10
𝑃(𝑛𝑜𝑡 𝑎 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒 ∩ 𝑎𝑐𝑐𝑒𝑝𝑡𝑎𝑏𝑙𝑒) 24 10
= = = = 0.625
𝑃(𝑛𝑜𝑡 𝑎 𝑑𝑒𝑓𝑒𝑐𝑡𝑖𝑣𝑒) 10 6 16
+
R.P.G.AMITHANI 24 24 5/6/2024 55

5
 Example 2

Using the conditional probability formula,

what’s the probability?

1. P(A|D) = ?
2. P(C|B) = ?

2 1
𝑃(𝐴 ∩ 𝐵) 10 2 𝑃(𝐶 ∩ 𝐵) 10 2
1. 𝑃 𝐴 𝐷 = = = 2. 𝑃 𝐶 𝐵 = = =
𝑃(𝐷) 5 5 𝑃(𝐵) 4 5
10 10
R.P.G.AMITHANI 5/6/2024 56
EXAMPLE

In a certain population , we know that 49% of the population are female. Also
of the female patients, 8% are high risk for heart attack, while 12% of the male
patients are high risk. A single person is selected at random and found to be
high risk for heart attack. What is the probability that it is a male?
Define H: high risk F: female M: male

We know: P( M ) P( H | M )
.49 P( M | H ) 
P(F) = P( M ) P( H | M )  P( F ) P( H | F )
P(M) = .51
.51 (.12)
P(H|F) = .08   .61
R.P.G.AMITHANI .51 (.12)  .49 (.08) 5/6/2024 57
P(H|M) = .12
EXAMPLE
A lamp produced by a Company was found to be defective. There are three factories (𝐴, 𝐵
, 𝐶) where such lamps are manufactured. A Quality Control Manager is responsible for
investigating the source of found defects. This is what he knows about the company’s
lamp production and the possible source of defects:

Factory % of total production Probability of

defective lamps

A 0.35 0.015
B 0.35 0.010
C 0.30 0.020
If a randomly selected lamp is defective, what is the probability that the lamp was manufactured in
R.P.G.AMITHANI
factory B? 5/6/2024 58
 𝐹1: Lamp produces in Factory 𝐴.
𝐹2: Lamp produces in Factory 𝐵.
𝐹3: Lamp produces in Factory 𝐶.
𝐷: Lamp is Defective

Then,

𝑃 𝐹1 = 0.35 𝑃 𝐹2 = 0.35 𝑃 𝐹3 = 0.30

𝑃 𝐷|𝐹1 = 0.015 𝑃 𝐷|𝐹2 = 0.010 𝑃 𝐷|𝐹3 = 0.020
R.P.G.AMITHANI 5/6/2024 59
 If a randomly selected lamp is defective, the probability that the
lamp was P F2|D ,
Using Bayes’ rule,

𝑃 𝐷 𝐹2 𝑃(𝐹2)
𝑃 𝐹2 𝐷 = 𝑛
𝑖=1 𝑃 𝐷 𝐹𝑖 𝑃(𝐹𝑖)
𝑃 𝐷 𝐹2 𝑃(𝐹2)
=
𝑃 𝐷 𝐹1 𝑃 𝐹1 + 𝑃 𝐷 𝐹2 𝑃 𝐹2 + 𝑃 𝐷 𝐹3 𝑃(𝐹3)
(0.010)(0.35)
=
0.015 0.35 + 0.010 0.35 + (0.020)(0.30)
= 0.2373
R.P.G.AMITHANI 5/6/2024 60
R.P.G.AMITHANI 5/6/2024 61