Introduction to

Probability
Com 201
By
Dr Muhammad Hafsat Ahmad
Introduction
• Probability is a measure of the likelihood of a
random phenomenon or chance behavior.
Probability describes the long-term proportion
with which a certain outcome will occur in
situations with short-term uncertainty.
• Probability deals with experiments that yield
random short-term results or outcomes, yet
reveal long-term predictability.
• The long-term proportion with which a certain
outcome is observed is the probability of that
outcome in subsequent experience.
Introduction
• Roughly, PROBABILITY is how frequently we expect different
outcomes to occur if we repeat the experiment over and over
(“frequentalist” view)
• Definitions of probability
• Classical
• Frequentist
• Subjective
Classical definition
• If a random experiment can result in ‘n’ mutually exclusive and
equally likely outcomes, and if nA of these outcomes has an attribute
A, then the probability of A is the fraction nA/n
• Examples: the toss of two coins, roll of an unbiased die, drawing a
card from a deck
Frequentist definition
• Replaces the classical definition when ‘n’, the number of trials is not
finite and the outcomes not equally likely
• Observations have to be made (e.g by taking a random sample)
unlike the classical case
• Probability of an event denotes the relative frequency of occurrence
of an experiment’s outcome, when repeating the experiment
• Considers probability to be the relative frequency ‘in the long run’ of
outcomes
Subjective definition
• Numbers are assigned as a degree of belief

• Most popular version is the Bayesian probability (combines expert

knowledge and experimental data to produce probabilities)
Operational definition: probability of
an Event E.
Suppose that the sample space S = {o1, o2, o3, … oN} has
a finite number, N, of oucomes.
Also each of the outcomes is equally likely (because of
symmetry).
Then for any event E

n E  n E  no. of outcomes in E
P E =  
n S  N total no. of outcomes
Note : the symbol n  A  = no. of elements of A
Thus this definition of P[E], i.e.

n E  n E  no. of outcomes in E
P E =  
n S  N total no. of outcomes

Applies only to the special case when

1. The sample space has a finite no.of outcomes,
and
2. Each outcome is equi-probable
If this is not true a more general definition of
probability is required.
The sample Space, S
The sample space, S, for a random
phenomena is the set of all possible
outcomes.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}

2. Rolling a die – outcomes

S ={ , , , , , }

={1, 2, 3, 4, 5, 6}
An Event , E
The event, E, is any subset of the sample space, S. i.e.
any set of outcomes (not necessarily all outcomes) of
the random phenomena

Venn
S diagram
E
The event, E, is said to have occurred if after the
outcome has been observed the outcome lies in E.

S
E
Examples

1. Rolling a die – outcomes

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}

E = the event that an even number is

rolled
= {2, 4, 6}
={ , , }
Special Events
The Null Event, The empty event - f

f = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S
S = the event that contains all outcomes
The empty event, f , never occurs.
The entire event, S, always occurs.
Set operations on Events
Union
Let A and B be two events, then the union of A
and B is the event (denoted by AB) defined by:
A  B = {e| e belongs to A or e belongs to B}

AB

A B
The event A  B occurs if the event A occurs or
the event and B occurs .

AB

A B
Intersection

Let A and B be two events, then the intersection

of A and B is the event (denoted by AB) defined
by:
A  B = {e| e belongs to A and e belongs to B}

AB

A B
The event A  B occurs if the event A occurs and
the event and B occurs .

AB

A B
Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
IN PROBLEMS YOU WILL RECOGNIZE THAT YOU ARE
WORKING WITH:

1. Union if you see the word or ( + )

2. Intersection if you see the word and ( X )

3. Complement if you see the word not.
Definition: mutually exclusive
Two events A and B are called mutually
exclusive if:
A  B 

A B
If two events A and B are mutually exclusive
then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
Rules of Probability
Rule The additive rule
(Mutually exclusive events)
P[A  B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]

if A  B = f
(A and B mutually exclusive)
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome of the
random experiment can not belong to both A and B.

A B
P[A  B] = P[A] + P[B]
i.e

P[A or B] = P[A] + P[B]

A B
Rule The additive rule
(In general)

P[A  B] = P[A] + P[B] – P[A  B]

or
P[A or B] = P[A] + P[B] – P[A and B]
Logic AB
A B

AB

When P[A] is added to P[B] the outcome in A  B

are counted twice
hence
P[A  B] = P[A] + P[B] – P[A  B]
 P  A  B   P  A  P  B   P  A  B 

Example:
Diabetes and Hypertension are two of the diseases
expected to be prevalent in a community based study for
. (There are also many other diseases). The researchers
are narrowing the findings 5 most prevalent disease.
There is a 20% chance that Diabetes will be amongst 5
most prevalent diseases and 35% chance Hypertension
will make the list while 8% chance that both diseases
will make the final list. What is the probability that
Diabetes or Hypertension will be amongst the 5 most
prevalent diseases in the community.
Solution:
Let A = the event that Diabetes is amongst the final 5.
Let B = the event that Hypertension is amongst the final
5.
Given P[A] = 0.20, P[B] = 0.35, and P[A  B] = 0.08
What is P[A  B]?
Note: “and” ≡ , “or” ≡  .

P  A  B   P  A  P  B   P  A  B 
0.20  0.35  0.08 0.47
RULE FOR COMPLEMENTS

2. P  A  1  P  A

or
P  not A 1  P  A
Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
Logic:
A and A are mutually exclusive.
and S  A  A

A
A

thus 1 P  S  P  A  P  A 
and P  A  1  P  A
Conditional Probability
Conditional Probability
• Frequently before observing the outcome of a random experiment
you are given information regarding the outcome
• How should this information be used in prediction of the outcome.
• Namely, how should probabilities be adjusted to take into account
this information
• Usually the information is given in the following form: You are told
that the outcome belongs to a given event. (i.e. you are told that a
certain event has occurred)
Definition
Suppose that we are interested in computing the
probability of event A and we have been told event B
has occurred.
Then the conditional probability of A given B is defined
to be:

P  A  B if P  B  0
P  A B  
P  B
 Rationale:
If we’re told that event B has occurred then the sample
space is restricted to B.
The probability within B has to be normalized, This is
achieved by dividing by P[B]
The event A can now only occur if the outcome is in of
A ∩ B. Hence the new probability of A is:
A
P  A  B B
P  A B  
P  B A∩B
An Example
The depression study.
For a specific married couple the probability that the
husband suffers depression is 80%, the probability that
his wife had same problem is 65%, while the probability
that they both had depression is 60%.
If the husband is had depression, what is the
probability that his wife is also having the problem
Solution:
The depression study.
Let B = the event that the husband had depression
P[B]= 0.80
Let A = the event that his wife also having the problem
P[A]= 0.65 and P[A ∩ B]= 0.60

P  A  B 0.60
P  A B    0.75
P  B 0.80
Independence
Definition
Two events A and B are called independent if
P  A  B   P  A P  B 
Note if P  B  0 and P  A 0 then
P  A  B P  A P  B 
P  A B     P  A
P  B P  B
P  A  B P  A P  B 
and P  B A   P  B 
P  A P  A
Thus in the case of independence the conditional probability of
an event is not affected by the knowledge of the other event
Difference between independence
and mutually exclusive

mutually exclusive
Two mutually exclusive events are independent only in
the special case where
P  A 0 and P  B  0. (also P  A  B  0
Mutually exclusive events are
A highly dependent otherwise. A
B
and B cannot occur
simultaneously. If one event
occurs the other event does not
occur.
Independent events
P  A  B   P  A P  B 

P  A  B P  A
or  P  A 
P  B PS
S

A B
The ratio of the probability of the
AB set A within B is the same as the
ratio of the probability of the set
A within the entire sample S.
The multiplicative rule of probability

 P  A P  B A if P  A 0
P  A  B  
 P  B  P  A B  if P  B  0

and
P  A  B   P  A P  B 

if A and B are independent.