Scribd
Upload
66 views

Artificial Intelligence: Binary Classifiers For Multi-Class Classification Problems

The document discusses using binary classifiers for multi-class classification problems. It describes several approaches for transforming a multi-class problem into multiple binary classification problems, including one-vs-one, one-vs-rest, hierarchical classification, and binary coding. It also notes some advantages and disadvantages of each approach, such as the number of models required and scaling issues for large datasets.

Uploaded by

Thành Cao Đức
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
66 views

Artificial Intelligence: Binary Classifiers For Multi-Class Classification Problems

The document discusses using binary classifiers for multi-class classification problems. It describes several approaches for transforming a multi-class problem into multiple binary classification problems, including one-vs-one, one-vs-rest, hierarchical classification, and binary coding. It also notes some advantages and disadvantages of each approach, such as the number of models required and scaling issues for large datasets.

Uploaded by

Thành Cao Đức
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Artificial Intelligence

Binary Classifiers for Multi-


class Classification Problems
Pham Viet Cuong
Dept. Control Engineering & Automation, FEEE
Ho Chi Minh City University of Technology
Binary Classifiers for Multi-class Classification Problems

ü Classification: A predictive modeling problem that involves assigning a


class label to an example
ü Binary classification: Classification tasks with two classes
ü Multi-class classification: Classification tasks with more than two classes
ü Binary classifiers:
v Support Vector Machines
v Perceptron
v Logistic Regression

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 2
Binary Classifiers for Multi-class Classification Problems

ü Multi-class classifications problems:


v Handwritten digit (0, 1, . . ., 9)
v Image classification ILSVRC2012
v ...

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 3
Binary Classifiers for Multi-class Classification Problems

ü Binary classifiers for multi-class classifications problems:


v One-vs-One
v One-vs-Rest (One-vs-All)
v Hierachical
v Binary coding

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 4
Binary Classifiers for Multi-class Classification Problems

ü One-vs-Rest:
v Split a multi-class classification into one binary classification problem
per class
v A binary classifier is trained on each binary classification problem
v Predictions are made using the most confident model
v Example
§ Three class: sun, moon, star
§ Three binary classification problem
• Binary Classification Problem 1: sun vs [moon, star]
• Binary Classification Problem 2: moon vs [sun, star]
• Binary Classification Problem 3: star vs [sun, moon]

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 5
Binary Classifiers for Multi-class Classification Problems

ü Each model predicts a class membership probability (probability-like


score)
ü Class index with largest score (argmax) is used to predict a class
ü Common used for Logistic Regression, Perceptron
ü Cons:
v Requires one model for each class (issue for large datasets, slow
models, or very large numbers of classes)
v Scale of confidence values may differ between binary classifiers
v Unbalanced distributions between the sets of positives and negatives

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 6
Binary Classifiers for Multi-class Classification Problems

ü One-vs-One:
v Split a multi-class classification into one binary classification problem
per each pair of classes
v Fit a binary classifier on each binary classification problem
v Example:
§ Three class: sun, moon, star
§ Three binary classification problem
• Binary Classification Problem 1: sun vs moon
• Binary Classification Problem 2: sun vs star
• Binary Classification Problem 3: moon vs star

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 7
Binary Classifiers for Multi-class Classification Problems

ü Each data point: classified according to a majority vote


ü If each model predicts a class membership probability: class index with
largest sum score (argmax) is predicted the class label
ü Cons:
v Number of binary models: C*(C-1)/2, C: # classes
v More models than the one-vs-rest
v Requires one model for each pair of class (issue for large datasets,
slow models, or very large numbers of classes)
v Suffers from ambiguities (some regions of input space may receive
the same number of votes)

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 8
Binary Classifiers for Multi-class Classification Problems

ü Hierarchical classification:
v Divide the output space i.e. into a tree
v Each parent node is divided into multiple child nodes until each child
node represents one class
v Example
Flat classification

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 9
Binary Classifiers for Multi-class Classification Problems

ü Binary coding:
v Encode class label by binary numbers
v # binary classifiers = log2(C) (C: # classes)
x = ceil(x): smallest integer greater than or equal to x
v Least binary classifiers required
v What if a bit incorrectly determinded?
v What if # classes not a power of 2?

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 10
Binary Classifiers for Multi-class Classification Problems

ü Linearly separable data:

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 11
Artificial Neural Networks
ü References
v https://machinelearningmastery.com/one-vs-rest-and-one-vs-one-for-
multi-class-classification/
v https://en.wikipedia.org/wiki/Artificial_neural_network

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 12

You might also like