Scribd
Upload
4 views18 pages

TransferLearning

Uploaded by

as.business.023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
4 views18 pages

TransferLearning

Uploaded by

as.business.023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

Performance Metrics

Confusion Matrix
Sensitivity / True Positive Rate / Recall
AUC-ROC curve
– AUC (Area Under The Curve)
– ROC (Receiver Operating Characteristics)
• AUC - ROC curve is a performance measurement for the
classification problems at various threshold settings.
• ROC is a probability curve and AUC represents the degree or
measure of separability.
• It tells how much the model is capable of distinguishing
between classes.
• Higher the AUC, the better the model is at predicting 0 classes
as 0 and 1 classes as 1.
• The ROC curve is plotted with TPR against the FPR where TPR
is on the y-axis and FPR is on the x-axis.
AUC-ROC curve
Transfer Learning
Traditional ML
Transfer Learning
• Transfer learning is a machine learning method
where a model developed for a task is reused as
the starting point for a model on a second task.
• Transfer learning make use of the knowledge
gained while solving one problem and applying
it to a different but related problem.
• For example, knowledge gained while learning
to recognize cars can be used to some extent to
recognize trucks.
Transfer Learning
Pre-Training

• When we train the network on a large dataset(for example:


ImageNet) , we train all the parameters of the neural network
and therefore the model is learned. It may take hours on your
GPU.
Fine Tuning
• We can give the new dataset to fine tune the pre-trained CNN.
Consider that the new dataset is almost similar to the orginal
dataset used for pre-training. Since the new dataset is similar, the
same weights can be used for extracting the features from the new
dataset.

• If the new dataset is very small, it’s better to train only the final
layers of the network to avoid overfitting, keeping all other layers
fixed. So remove the final layers of the pre-trained network. Add
new layers . Retrain only the new layers.

• If the new dataset is very much large, retrain the whole


network with initial weights from the pretrained model.
Fine Tuning
• How to fine tune if the new dataset is very different from the orginal
dataset ?

• The earlier features of a ConvNet contain more generic features (e.g. edge
detectors or color blob detectors), but later layers of the ConvNet becomes
progressively more specific to the details of the classes contained in the
original dataset.

• The earlier layers can help to extract the features of the new data. So it will
be good if you fix the earlier layers and retrain the rest of the layers, if you
got only small amount of data.

• If you have large amount of data, you can retrain the whole network with
weights initialized from the pre-trained network.
Transfer Learning
Benefits
• The benefits of Transfer Learning are that it can speed up the
time it takes to develop and train a model by reusing these
pieces or modules of already developed models.

• This helps speed up the model training process and accelerate


results.
Approaches
Two common approaches are as follows:
– Develop Model Approach
– Pre-trained Model Approach
Develop Model Approach

• Select Source Task. You must select a related predictive modeling problem with
an abundance of data where there is some relationship in the input data, output
data, and/or concepts learned during the mapping from input to output data.

• Develop Source Model. Next, you must develop a skillful model for this first
task. The model must be better than a naive model to ensure that some feature
learning has been performed.

• Reuse Model. The model fit on the source task can then be used as the starting
point for a model on the second task of interest. This may involve using all or
parts of the model, depending on the modeling technique used.

• Tune Model. Optionally, the model may need to be adapted or refined on the
input-output pair data available for the task of interest.
Pre-trained Model Approach
• Select Source Model. A pre-trained source model is chosen from
available models. Many research institutions release models on
large and challenging datasets that may be included in the pool of
candidate models from which to choose from.

• Reuse Model. The model pre-trained model can then be used as


the starting point for a model on the second task of interest. This
may involve using all or parts of the model, depending on the
modeling technique used.

• Tune Model. Optionally, the model may need to be adapted or


refined on the input-output pair data available for the task of
interest.

You might also like