LEARNING AGENTS

[email protected]

1
 Contents to be
covered
• What is learning
- Components of learning Agent

• Data set preparation for learning

• Classification tasks
- Steps for classification
- Classifier performance measure
- Factors affecting classifier model performance

• Types of learning

• Learning approaches/methods
2
 What is Learning?
• Learning is one of the keys to human intelligence. Do
you agree?
• what learning is? Learning is:
- Memorizing something
- Knowing facts through observation and exploration
- Improving motor and/or cognitive skills through practice
• The idea behind learning is that percepts should not
only be used for acting now, but also for improving
the agent’s ability to act in the future to achieve
the goal.
- Learning is essential for unknown environments,
i.e. when the agent lacks knowledge.
- It enables to organize new knowledge into
general,
- effective representations
Learning modifies the agent's decision making mechanisms
3
to improve performance
 Learning Examples
– Learning is memorizing and remembering
• Telephone number
• Reading textbook
• Understanding Language (memorizing
grammar &
practicing)
• Recognize face, signature & fraudulent credit
card transactions
– Learning is improving motor skills
• Riding a bike
• Exercising and practicing the idea
– Learning is understanding the strategy & rule of the
game
• Playing chess and football
– Learning is abstraction and exploration
• Develop scientific theory
• Undertaking research
– Learning is nothing but
• Feature extraction 4
• Classification
 Feature extraction

Male Hair Fema Hair

length le length y
y =0
=Task: to extract features which are=good for = 30 cm
classification.
Good features:
• Objects from the same class have similar feature values.
• Objects from different classes have different values.

“Good” “Bad” 5
Classification
• Applications
–Character recognition:
Different printing styles.
–Speech recognition:
• Use of a dictionary or
the syntax of the
language.
• Example: Credit scoring
–Differentiating between
low-risk & high-risk
customers from their
income & savings
Discriminant: IF income > θ1 AND savings > θ2 THEN
low-risk
ELSE
high-risk 6
 Face Recognition
• Learning: it is training (adaptation) from data set
• Training examples of a person –
• Face recognition is
challenging because
of the effect of
facial expression,
• Test images lighting, occlusion,
make-up,
hair style, etc.

7
Learning Agents
Percepts

Actions 8
 The Basic Learning Model
• A computer program is said to learn from experience
E with respect to some class of tasks T and
performance measure P,
– if its performance at tasks T, as measured by P, improves
with experience E.
• Learning agents consist of four main components:
– learning element -- the part of the agent responsible
for improving its performance
– performance element -- the part that chooses the
actions
to take
– critic – provides feedback for the learning element
how the agent is doing with respect to a performance
standard
training
- problem thegenerator -- suggests problems or actions that 9
system further.
 Data sets preparation for learning
• Data Sets also k.as Samples, Examples or Instances
• Good data is a prerequisite for
producing effective models of any types of
problem.
• Usually, the given data set is divided into training
and test sets.
• Training set
– Used in supervised learning, a training set is a set
of problem instances (described as a set of
properties and their values), together with a
classification of the instance.
• Test set
– A set of instances and their classifications used
 How to Split Data Sets into Training
 Holdout Method and Testing Sets?
– Given data is randomly partitioned into
two independent sets
o Training set (2/3) for model construction
o Test set (1/3) for accuracy estimation
- If many (thousands) of examples are available, including
several hundred examples from each class.

 Cross-Validation Method
-Randomly partition the data into k mutually
exclusive
subsets, each approximately equal size.
-Where k = 10 is most popular.
At i-th iteration, use Di as test set and others as
-set 11
- training
Use Cross-validation for small data
Cross Validation Examples

12
 Example: 3-Fold Cross-Validation

 Data is partitioned into 3 sets

 3 experiments: Each with a different holdout set
for testing
Hold
Out Tra ining Performance=67%
(test)

Average Performance=
Hold
out Tra ining Performance=60%
(67+60+81)/3=69.3

Hold
Tra ining Performance=81%
out
 Data sets preparation for learning…
- It is important that the test data is not used in any
way to create the classifier.

Generally,
– The larger the training data the better the
classifier

– The larger the test data the more accurate

the error estimate

14
 Classification tasks- A two step process
 Model construction: (Learning step or training
step)
– Construct the classification model based on the training
data.
• Training Data
- A set of tuples
- Each tuple is assumed to belongs to a predefined class
- Labeled data(ground truth)
• How a classification model looks like?
-A classification model can be represented by one of the
ff forms:-
 classification rules,
 decision trees, or
 mathematical formulae
o Thus, model construction refers to describing a set15
 Step 1: Model Construction

Classification
Algorithms
Trainin
g
Data

NAME RANK YEARS TENURED Classifier

Mike Assistant Prof 3 no (Model)
Mary Assistant Prof 7 yes
Bill Professor 2 yes
Jim Associate Prof 7 yes IF rank = ‘professor’
Dave Assistant Prof 6 no OR years > 6
THEN tenured = 1
Anne Associate Prof 3 no
‘yes’ 6
 Classification tasks- A two step process…

 Model usage:
- Before using the model, we first need to test its
accuracy.
• Measuring model accuracy
- To measure the accuracy of a model we need
test
data.
- Test data is similar in its structure to training
data(labeled data).
• How to test?
• The known label of test samples is compared with
the classified result from the model.
17
 Classification tasks- A two step process…
• Accuracy rate is the percentage of test set samples
that are correctly classified by the model

Number of correctclassifications
Accuracy  ,
Total number of test cases
• Important:- test data should be independent of training
set, otherwise over-fitting will occur.

• Using the model- if the accuracy is acceptable, use the

model to classify data tuples whose class labels are not
known

18
 Step 2: Using the Model in
Prediction

Classifier
model

Testin
g Unseen Data
Data
(Jeff, Professor, 4)
NAM E RANK YEARS TENURED
Tom Assistant Prof 2 no Tenured?
M erlisa Associate Prof 7 no
George Professor 5 yes
Joseph Assistant Prof 7 yes 19
 Classification:- Step1
Split data into train and test sets

20
 Classification:- Step2
Build a model on a training set

21
 Classification:- Step3
Evaluate on test set (Re-train?)

22
 Confusion matrix
• A confusion matrix is useful tool for analyzing how well u’r classifier
can recognize tuples of different classes
• A confusion matrix displays the number of correct and incorrect
predictions made by the model compared with the actual
classifications in the test data.
Observe the following Confusion Matrix

Classified as Cancer Not Cancer

Actual Cancer 7(# of TP) 2(# of FN) To tal Dataset=14
Class Not Cancer 3(# of FP) 2(# of TN)

• The matrix is n-by-n, where n is the number of classes.

• The above confusion matrix can be used to calculate TP, FP,
Precision, Recall and F-Measure, and also Accuracy of the sy2s3
tem
 Classifier performance Measure
Where:c1=“cancer”,c2=“not_cancer”

P=?????
N=????

24
Error Rate=1-accuracy
 Performance Measure

Table-1 Confusion Matrix

for a 2-class problem

25
Example

2
6
 Classifier performance Measure Examples

Compute the following performance measures from the

confusion matrix given below.
- Sensitivity, Specificity, Accuracy, Recall and Precision

Classes Yes NO
NO How many
Yes
90 210
210 data sets
Yes
NO 140 9560 are there in
140 9560
Sensitivity =
this
TP/P = 90/300 = 30%
Specificity = TN/N = 9560/9700 = 98.56% example?
Accuracy = (TP + TN)/(P+N) = 9650/10,000 =
96.50%
Error Rate= 1-Accuracy= 1- 0.965= 3.5%
Precision = TP/(TP+FP) = TP/N = 90/230 = 39.13% 27

Recall = TP/(TP+FN) = TP/P= 90/300 = 30.0%

 Exercise
Compute the following performance measures from
the confusion matrix given below.

Sensitivity =TP/TP+FP
Specificity =
Accuracy =
Error Rate=
Precision =
Recall 28
 Performance Measure
 Accuracy
- classifier accuracy: predicting class label
- predictor accuracy: guessing value of predicted
attributes
 Speed
- time to construct the model (training time)
- time to use the model (classification/prediction time)
 Robustness: handling noise and missing values

 Scalability: efficiency in disk-resident databases

 Interpretability
- understanding and insight provided by the model 29
 Purpose of Evaluation
• The objective of learning classifications from
sample data is to classify and predict
successfully on new data

• The aim of evaluation is to estimate the true

error rate using a finite amount of data.

• The true error rate is defined as the error rate

of a classifier on an asymptotically large number
of new cases that converge in the limit to the
actual population distribution (i.e. it is an
inherently statistical measure). 30
 Factors Affecting Performance of
Classifier
• There are several factors affecting the
performance:
- Types of training provided
- The learning algorithms used
- The type of feedback provided
- The form and extent of any initial background
knowledge

31
 Sum-up
• What is learning?
• Discuss the learning agent Components?
• What is dataset? What is the need of splitting data
into training and test samples/instances?
• What are the methods to split dataset into training
and testing dataset?
• What are the two classification tasks and discus it.
• What is the purpose of confusion matrix?
• What are the commonly used measures for evaluating
the classification(Classifier model) performance
• Which factors affect the performance of classifier
model?
32
 Types of learning

• Supervised learning
- classification, regression

• Unsupervised learning
- clustering
• Semi- Supervised learning
• Reinforcement learning

33
 Types of learning
 Supervised learning – Classification
-Isthelearning process when the outcome variable
is
known.

-Data and corresponding labels are given.

-The outcome datasets are provided w/c are used to train

the machine and get the desired output.

-Occurs where a set of input/output pairs are explicitly

presented to the agent by a teacher
– The teacher provides a category label for each pattern in
a training set, then the learning algorithm finds a rule
that adoes
with newainput.
good job of predicting the output associated
34
 Cont…
Classification:- Given a collection of records (training set ),
with each record containing a set of attributes, and one of
the attribute being the class.
-Goal: previously unseen records should be assigned a class
as accurately as possible.
- A test set is used to determine the accuracy of the model.

3
5
 Supervised Learning

Example:-

36
Supervised learning
An example: data (loan application)

37
 The learning Process
• Learn a classification model from the data
• Use the model to classify future loan applications
into
– Yes (approved) and
– No (not approved)
• What is the class for following case/instance?

38
 Example

label
Classification: a finite set of
apple labels

apple

banana

banana

Super
Example: Digit Recognition

40
 Ranking example

Given a query
and a set of
web pages,rank
them
according
to relevance

41
Cont…

42
 Applications
 Medical Diagnosis:- Predicting tumor
cells as benign or malignant

 Fraud Detection:- Classifying credit

card transactions
as legitimate or fraudulent

 Credit Approval
 Classifying secondary structures of
protein
as alpha-helix, beta-sheet, or
random
coil

 Categorizing news stories as finance, 43

 Classification Examples
• In classification, we predict labels y (classes) for
inputs x
• Examples:
– OCR (input: images, classes: characters)
– Medical diagnosis (input: symptoms, classes: diseases)
– Automatic essay grader (input: document,
classes: grades)
– Fraud detection (input: account activity, classes:
fraud
/ no fraud)
– DNA and protein sequence identification
– Categorization and identification of astronomical
images
– Customer service email routing
– books
Recommended articles in a newspaper, 44

… recommended
many more
 Types of learning…
• Unsupervised learning – Clustering
- The class labels of training data is unknown.
-Learning when there is no information about what the
correct outputs are.
– In unsupervised learning or clustering there is no explicit
teacher, the system forms clusters or natural groupings
of the input patterns.
- A form of learning by observation rather than learning by
examples
– Clustering is a technique for finding similarity
groups in data.
4
5
 Cont…
• Thus Cluster Analysis
– Finding groups of objects such that the objects in a
group will be similar (or related) to one another and
different from (or unrelated to) the objects in
other groups
Inter-cluster
Intra-cluster distances
distances are are
minimized maximized
 Example

Unsupervised learning: given data, i.e. examples, but

no labels 47
 Examples
Example 1: Given a collection of text documents,
we want to organize them according to their
content similarities.

Example 2: groups people of similar sizes

together to make “small”, “medium” and “large”.

49
 Unsupervised Learning

Example:-
Unsupervised learning 50
 Types of learning
• Reinforcement learning (RL): an agent interacting with
the world makes observations, takes actions, & is
rewarded or punished; it should learn to choose actions
in order to obtain a lot of reward.

• Examples
– Game playing: player knows whether it win or lose, but
not know how to move at each step
– Control: a traffic system can measure the delay of cars,
but not know how to decrease it.

51
RL is learning from interaction

52
 Example

Backgammon

WIN!

LOSE!

Given sequences of moves and whether or not

the player won at the end, learn to
moves
make good 53
 Summary about types of Learning

• Clustering is a machine learning technique

that finds similarities between data
according to the characteristics found in the
data & groups similar data objects into one
cluster
 Learning methods
• There are various learning methods.
Popular learning techniques include the following.
– Decision tree : divide decision space into
piecewise constant regions.
– K- Nearest Neighbour:- classify based on
similarity measurements
– Neural networks: partition by non-linear boundaries
– Bayesian network: a probabilistic model
– Regression: (linear or any other polynomial)
– Support vector machine: solves non-linearly
separable problems.
– Expectation maximization algorithm
55
The End

አመሰግናለሁ !!!
56