Basic Probability

Dr. K. M. Salah Uddin

Associate Professor, Dept. of

MIS, DU
 Contents

 Basic probability concepts and definitions

 Conditional probability
 To use Bayes’ Theorem
 Various counting rules
 Important Terms

 Probability – the chance that an uncertain event

will occur (always between 0 and 1)
 Event – Each possible outcome of a variable
 Simple Event – an event that can be described
by a single characteristic
 Sample Space – the collection of all possible
events
 Assessing Probability
 There are three approaches to assessing the probability
of an uncertain event:
1. a priori classical probability
X number of ways the event can occur
probability of occurrence  
T total number of elementary outcomes

2. empirical classical probability

number of favorable outcomes observed
probability of occurrence 
total number of outcomes observed

3. subjective probability
an individual judgment or opinion about the probability of occurrence
 Sample Space
The Sample Space is the collection of all
possible events
e.g. All 6 faces of a die:

e.g. All 52 cards of a bridge deck:

 Events
 Simple event
 An outcome from a sample space with one
characteristic
 e.g., A red card from a deck of cards
 Complement of an event A (denoted A’)
 All outcomes that are not part of event A
 e.g., All cards that are not diamonds
 Joint event
 Involves two or more characteristics simultaneously
 e.g., An ace that is also red from a deck of cards
 Visualizing Events
 Contingency Tables
Ace Not Ace Total

Black 2 24 26
Red 2 24 26

Total 4 48 52
 Tree Diagrams Sample
Ace 2 Space
Sample
C ar d
Space
Bla ck 24
Full Deck No t a n A c e
of 52 Cards Ac e
Red C 2
ard
Not an 24
Ace
 Visualizing Events
 Venn Diagrams
 Let A = aces
 Let B = red cards A ∩ B = ace and red

B
A U B = ace or red
 Mutually Exclusive Events
 Mutually exclusive events
 Events that cannot occur together

example:

A = queen of diamonds; B = queen of clubs

 Events A and B are mutually exclusive

 Collectively Exhaustive Events
 Collectively exhaustive events
 One of the events must occur
 The set of events covers the entire sample space

example:
A = aces; B = black cards;
C = diamonds; D = hearts
 Events A, B, C and D are collectively exhaustive
(but not mutually exclusive – an ace may also be
a heart)
 Events B, C and D are collectively exhaustive and
also mutually exclusive
 Probability
 Probability is the numerical measure
of the likelihood that an event will 1 Certain

occur
 The probability of any event must be
between 0 and 1, inclusively
0 ≤ P(A) ≤ 1 For any event A 0.5
 The sum of the probabilities of all
mutually exclusive and collectively
exhaustive events is 1
P(A)  P(B)  P(C)  1
0 Impossible
If A, B, and C are mutually exclusive and
collectively exhaustive
 Computing Joint and
Marginal Probabilities

 The probability of a joint event, A and B:

number of outcomes satisfying A and B
P( A and B) 
total number of elementary outcomes

 Computing a marginal (or simple) probability:

P(A)  P(A and B1 )  P(A and B 2 )    P(A and Bk )


Where B1, B2, …, Bk are k mutually exclusive and collectively
exhaustive events
 Joint Probability Example

P(Red and Ace)

number of cards that are red and ace 2
 
total number of cards 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
 Marginal Probability Example

P(Ace)
2 2 4
 P( Ace and Re d)  P( Ace and Black )   
52 52 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
 General Addition Rule

General Addition Rule:

P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

For mutually exclusive events A and B
 General Addition Rule Example

P(Red or Ace) = P(Red) +P(Ace) - P(Red and Ace)

= 26/52 + 4/52 - 2/52 = 28/52

Don’t count
the two red
Color aces twice!
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
 Computing Conditional
Probabilities
 A conditional probability is the probability of one
event, given that another event has occurred:
P(A and B) The conditional
P(A | B)  probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A)  probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal probability of A
P(B) = marginal probability of B
 Conditional Probability Example

 Of the cars on a used car lot, 70% have air

conditioning (AC) and 40% have a CD player
(CD). 20% of the cars have both.

 What is the probability that a car has a CD

player, given that it has AC ?

i.e., we want to find P(CD | AC)

 Conditional Probability Example
(continued)
 Of the cars on a used car lot, 70% have air conditioning
(AC) and 40% have a CD player (CD).
20% of the cars have both.
CD No CD Total
AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(CD and AC) 0.2

P(CD | AC)    0.2857
P(AC) 0.7
 Conditional Probability Example
(continued)
 Given AC, we only consider the top row (70% of the cars). Of these,
20% have a CD player. 20% of 70% is about 28.57%.

CD No CD Total
AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(CD and AC) 0.2

P(CD | AC)    0.2857
P(AC) 0.7
 A box contains 3 white balls and 5 black balls.
We extract 2 balls from the box, one after the
other. Give a sample space for this experiment
and the probabilities of the elementary events
of this sample space.
 A bag contains 5 white marbles, 3 black
marbles and 2 green marbles. In each draw, a
marble is drawn from the bag and not replaced.
In three draws, find the probability of obtaining
white, black and green in that order.
 What is the probability that the total of two dice
will be greater than 8, given that the first die is a
6?

 Three coins are tossed. Show that the events

“heads on the first coin” and the event “tails on
the last two” are independent.
 Statistical Independence
 Two events are independent if and only
if:

P(A | B)  P(A)
 Events A and B are independent when the probability
of one event is not affected by the other event
 Multiplication Rules

 Multiplication rule for two events A and B:

P(A and B)  P(A | B) P(B)

Note: If A and B are independent, then P(A | B)  P(A)

and the multiplication rule simplifies to

P(A and B)  P(A) P(B)

 Some examples of Independence

 Two coins are tossed, consider the following

events: A1 ="obtain heads on the first coin", A2
="obtain tails on the second coin", A3 ="obtain
heads and tails". Show that the events A1,A2, A3
are not 3-independent.
 If the events A and B are independent, and Ac
and Bc are respectively their complementary
events, then the following pairs of events are
also independent:
 A and Bc (ii) Ac and B and (iii) Ac and Bc
 Some examples of Independence

 Two coins are tossed. Show that the events A=

"obtain heads on the first coin" and B ="obtain
tails on the second coin" are independent.
 Three coins are tossed. Show that the events
“heads on the first coin” and the event “tails on
the last two” are independent.
 Marginal Probability

 Marginal probability for event A:

P(A)  P(A | B1 ) P(B1 )  P(A | B2 ) P(B2 )    P(A | Bk ) P(Bk )

 Where B1, B2, …, Bk are k mutually exclusive and

collectively exhaustive events
 Bayes’ Theorem

P(A | B i )P(Bi )
P(B i | A) 
P(A | B 1)P(B1)  P(A | B 2 )P(B 2 )    P(A | B k )P(Bk )

 where:
Bi = ith event of k mutually exclusive and collectively
exhaustive events
A = new event that might impact P(Bi)
 Bayes’ Theorem Example

 A drilling company has estimated a 40%

chance of striking oil for their new well.
 A detailed test has been scheduled for more
information. Historically, 60% of successful
wells have had detailed tests, and 20% of
unsuccessful wells have had detailed tests.
 Given that this well has been scheduled for a
detailed test, what is the probability
that the well will be successful?
 Bayes’ Theorem Example
(continued)

 Let S = successful well

U = unsuccessful well
 P(S) = 0.4 , P(U) = 0.6 (prior probabilities)
 Define the detailed test event as D
 Conditional probabilities:
P(D|S) = 0.6 P(D|U) = 0.2
 Goal is to find P(S|D)
 Bayes’ Theorem Example
(continued)

Apply Bayes’ Theorem:

P(D | S)P(S)
P(S | D) 
P(D | S)P(S)  P(D | U)P(U)
(0.6)(0.4)

(0.6)(0.4)  (0.2)(0.6)
0.24
  0.667
0.24  0.12

So the revised probability of success, given that this well

has been scheduled for a detailed test, is 0.667
 Bayes’ Theorem Example
(continued)

 Given the detailed test, the revised probability

of a successful well has risen to 0.667 from
the original estimate of 0.4

Prior Conditional Joint Revised

Event Prob.
Prob. Prob. Prob.
S (successful) 0.4 0.6 (0.4)(0.6) = 0.24 0.24/0.36 = 0.667
U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12 0.12/0.36 = 0.333

Sum = 0.36
 Suppose that Mr. Zafar becomes sick in the middle of
the night and asks his sleepy wife to get some drug for
him from the medicine cabinet. Two kinds of tablets are
available T1 and T2. There are only four bottles in the
cabinet; three bottles contain T1, and one bottles
contains T2. If T1 is taken, there is a 90 percent chance
of dizziness, but if T2 is taken, the probability of
dizziness is only 10 percent. Infuriated by being
awakened from a sound sleep, Mr. Zafar’s wife grabs
some tablets and hands them to her husband without
actually knowing whether they are T1 or T2. A few
minutes later, Mr. Zafar becomes very dizzy. What is the
probability that the tablet was T1 or T2?
 Counting Rules

 Rules for counting the number of possible

outcomes

 Counting Rule 1:
 If any one of k different mutually exclusive and
collectively exhaustive events can occur on each of
n trials, the number of possible outcomes is equal to

kn
 Counting Rules
(continued)

 Counting Rule 2:
 If there are k1 events on the first trial, k2 events on
the second trial, … and kn events on the nth trial, the
number of possible outcomes is
(k1)(k2)…(kn)

 Example:
 You want to go to a park, eat at a restaurant, and see a
movie. There are 3 parks, 4 restaurants, and 6 movie
choices. How many different possible combinations are
there?
 Answer: (3)(4)(6) = 72 different possibilities
 Counting Rules
(continued)

 Counting Rule 3:
 The number of ways that n items can be arranged in
order is
n! = (n)(n – 1)…(1)

 Example:
 Your restaurant has five menu choices for lunch. How many
ways can you order them on your menu?
 Answer: 5! = (5)(4)(3)(2)(1) = 120 different possibilities
 Counting Rules
(continued)

 Counting Rule 4:
 Permutations: The number of ways of arranging X
objects selected from n objects in order is
n!
n Px 
(n  X)!

 Example:
 Your restaurant has five menu choices, and three are
selected for daily specials. How many different ways can
the specials menu be ordered?
n! 5! 120
nPx     60
(n  X)! (5  3)! 2
 Answer: different possibilities
 Counting Rules
(continued)

 Counting Rule 5:
 Combinations: The number of ways of selecting X
objects from n objects, irrespective of order, is
n!
n Cx 
X!(n  X)!

 Example:
 Your restaurant has five menu choices, and three are
selected for daily specials. How many different special
combinations are there, ignoring the order in which they are
selected?
n! 5! 120
n Cx     10
X!(n  X)! 3!(5  3)! (6)(2)
 Answer: different possibilities