Bayes Classification
Method
By,
Umakanth N
Session Outcomes
Identify the domain of application, where one can
use Bayes classifier method
Classify the data set using Naïve Bayesian
classification
February 10, 2025 Bayes Classification Method 2
Bayes Classification
Method
Bayesian classifiers are statistical classifiers
Can able to provide high accuracy and speed
when classifying the large database
Naïve Bayesian classifier is the simple Bayesian
classifier, which assumes the attribute value on a
given class is independent of the values of other
attributes. ie., class-conditional
independence
February 10, 2025 Bayes Classification Method 3
Bayes’ Theorem
Let X be the data tuple and H be some hypothesis
such that X belongs to specific class C.
P(H/X) is the probability that H hold given the tuple X.
ie., probability of tuple X belongs to class C, given that
we know the attribute description of X
P(H/X) & P(X/H) is the posterior probability.
Eg1., X is 30 years old customer with income 40 K & H
=> customer purchases computer. P(H/X) = ?
Eg2., P(X/H) => probability of customer with age 30 &
income 40K, given that customer will buy a computer
P(H) is the prior probability
Eg., probability that any customer will purchase the
computer
February 10, 2025 Bayes Classification Method 4
Bayes’ Theorem (Contd.,)
Let X be the data tuple and H be some
hypothesis such that X belongs to specific class
C.
Bayes’ theorem provides a way to calculate the
Posteriori probability by,
February 10, 2025 Bayes Classification Method 5
Naïve/Simple Bayesian
Classification
Assumption: all features are independent for the
given the class label Y
Let D be the training set of tuples X=(x1, x2, …,
xn) represents the values of the attributes A1,
A2, …, An. Totally m class labels represented by
C1, C2, …, Cm.
Naïve bayesian classifier predicts that tuple X
belongs to the class Ci iff,
P(Ci |X) > P(Cj |X), for 1 <=j<= m, j !=
I
ie., we need to maximize P(Ci|X)
where P(Ci | X) can be calculated by Bayes’ theorem
February 10, 2025 Bayes Classification Method 6
Naïve/Simple Bayesian
Classification (Contd.,)
As P(X) is constant for all classes => P(X|Ci) P(Ci)
needs to be maximized
If class prior probabilities are not known, then we
can assume they are likely equal ie., P(C1) = P(C2)
= … = P(Cm)
Thus P(X|Ci) is enough to maximize
Else P(X|Ci) P(Ci) is needed to maximize
For complete dataset, finding P(X|Ci) is
computationally expensive. Thus by Naïve
assumption of class-conditional independence, it
can be computed by,
February 10, 2025 Bayes Classification Method 7
Naïve/Simple Bayesian
Classification (Contd.,)
How to compute P(Xk | Ci)?
If Ak is categorical, then P(Xk | Ci) = ( |Ci,D|
having Xk ) / |Ci,D|
If Ak is continuous valued, it is assumed to have
Gaussian distribution with mean µ and standard
deviation σ, defined by
where µ is the mean and σ is the standard deviation of
the values of attribute Ak for the training tuple of class
February 10, 2025 Bayes Classification Method 8
Naïve/Simple Bayesian
Classification (Contd.,)
How to predict the class label of X?
Evaluate P(X|Ci) P(Ci) for every Ci
Find the maximum value of P(X|Ci) P(Ci) and make
that Ci as the class label of X
ie., class label of X = Ci of MAX ( P(X|Ci) P(Ci) ) , for
0<=i<=m
February 10, 2025 Bayes Classification Method 9
Naïve Bayesian
Classification (Example)
Consider buys_computer as the class label
attribute and classify the dataset based on the
attributes age=youth, income=medium,
student=yes, credit_rating=fair
February 10, 2025 Bayes Classification Method 10
Naïve Bayesian
Classification (Example)
Consider C1 => buys_computer=yes & C2 => no
We need to maximize P(X|Ci) P(Ci), for i=1,2
P(C1) = 9/14 = 0.643
P(C2) = 5/14 = 0.357
February 10, 2025 Bayes Classification Method 11
Naïve Bayesian
Classification (Example)
To compute P(X|Ci), need to compute conditional
probabilities as,
P(age=youth | C1) = 2 / 9 = 0.222
P(age=youth | C2) = 3 / 5 = 0.6
P(income = medium | C1) = ?
P(income = medium | C2) = ?
February 10, 2025 Bayes Classification Method 12
Naïve Bayesian
Classification (Example)
To compute P(X|Ci), need to computer conditional
probabilities as,
P(student = yes | C1) = ?
P(student = yes | C2) = ?
P(credit_rating = fair | C1) = ?
P(credit_rating = fair | C2) = ?
February 10, 2025 Bayes Classification Method 13
Naïve Bayesian
Classification (Example)
Compute P(X|Ci):
P(X|C1) = P(age=youth | C1) * P(income = medium
| C1)
* P(student = yes | C1) *
P(credit_rating = fair | C1)
P(X|C1) = 0.044
P(X|C2) = 0.019
February 10, 2025 Bayes Classification Method 14
Naïve Bayesian
Classification (Example)
Compute P(X|Ci) P(Ci):
P(X|C1) P(C1) = 0.044 * 0.643 = 0.028
P(X|C2) P(C2) = 0.019 * 0.357 = 0.007
Thus Naïve classifier predicts
buys_computer=yes for given attribute values
February 10, 2025 Bayes Classification Method 15
Check Yourself..!!!
Is Bayesian Classifier effective???
How to perform the classification if Age is
numerical attribute in previous example?
What happened if some P(Ci) = 0?
Can use Laplacian correction / Laplacian estimation
Among 100 tuples, o tuples with age=senior, 90
tuples with age=youth & 10 tuples with
age=middle; such case we can add negligible value,
say 1 to all cases as follows:
1 / 103 for senior
91 / 103 for youth
11 / 103 for medium
February 10, 2025 Bayes Classification Method 16
Rule Based classification
Learned model is represented as a set of IF-THEN
rules
Topics:
Using IF-THEN Rules for Classification
Rules extraction from the decision tree
Rule Induction Using a Sequential Covering
Algorithm
February 10, 2025 Bayes Classification Method 17
? ?
. . ?
ie s
e r
Q u
