Scribd
Upload
26 views12 pages

ResNet Deep Residual Learning for Image Recognition (1)

ResNet, introduced in 2015 by Kaiming He and colleagues, revolutionized deep convolutional neural networks by addressing challenges in training ultra-deep models through the use of residual connections. This architecture significantly improved image recognition performance, winning the ILSVRC 2015 challenge and influencing subsequent advancements in computer vision. ResNet's design has paved the way for further innovations and remains a foundational model in deep learning.

Uploaded by

tacej76706
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
26 views12 pages

ResNet Deep Residual Learning for Image Recognition (1)

ResNet, introduced in 2015 by Kaiming He and colleagues, revolutionized deep convolutional neural networks by addressing challenges in training ultra-deep models through the use of residual connections. This architecture significantly improved image recognition performance, winning the ILSVRC 2015 challenge and influencing subsequent advancements in computer vision. ResNet's design has paved the way for further innovations and remains a foundational model in deep learning.

Uploaded by

tacej76706
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

ResNet: Deep Residual

Learning for Image


Recognition
Introduced in 2015 by Kaiming He and colleagues, ResNet
revolutionized the field of deep convolutional neural networks
(CNNs). This architecture dramatically improved image
recognition performance and became a breakthrough in
training ultra-deep neural networks. ResNet won the
prestigious ILSVRC 2015 challenge and has since amassed
over 110,000 citations, underpinning countless advancements
in computer vision and beyond.
YS
by Yash Shrivastava
The Problem: Training Deep
CNNs
Depth Challenges
Training very deep CNNs with 20 or more layers revealed critical
challenges such as vanishing and exploding gradients. These
issues limited the ability to optimize weights effectively,
restricting model depth and complexity.

Degradation Problem
Adding more layers unexpectedly caused accuracy degradation,
not just overfitting. This problem occurs even when the deeper
model is theoretically capable of achieving the same or better
performance than a shallower one. It highlights an optimization
challenge, not overfitting or underfitting.
Why Previous Approaches Fell Short
Conventional Deep Models :-

Earlier networks like VGG and GoogleNet increased depth by stacking layers but faced optimization
bottlenecks, causing training errors to increase.

Limitations :-

• Simple stacking didn’t guarantee


improved model capacity
• Higher layers led to more errors during training
• Overfitting was not the sole issue;
optimization difficulty was key

CIFAR-10 with 20-layer and 56-layer “Plain” networks.


The ResNet Breakthrough: Residual Learning
Residual Connections Training Ultra-Deep Nets
ResNet introduced skip connections allowing This approach allowed networks with over 100
layers to learn residual functions added to layers to be trained efficiently without
inputs, effectively setting up identity shortcuts. degradation in accuracy.
The Architecture: Residual
Blocks
Residual Unit Structure Variants

The fundamental unit • Basic block for ResNet-34


features two or three • Bottleneck block for
convolutional layers plus a deeper models like
shortcut skip connection, ResNet-50/ 101/152.
enabling information to
• All units use batch
bypass intermediate layers.
normalization and ReLU
activation
How Skip Connections Work
1 Gradient Flow 2 Prevents Degradation
Improvement
The network can learn
Skip connections identity mappings if
enable gradients to additional layers do not
bypass several layers improve the solution,
during maintaining or
backpropagation, improving accuracy.
combating vanishing
gradients.

3 Optimization Simplification
Allows deeper networks to be optimized as residual
functions instead of direct mappings, leading to easier
convergence during training.
Basic Structure of a Residual Block in ResNet34

Layer 1
• Convolution
• Batch Normalization
• Activation - ReLU

Layer 2
• Convolution
• Batch Normalization

Adder
• Adding input through Skip connection
• Activation - ReLU
ResNet Variants and
Scalability
Layer Depth Bottleneck Design
ResNet architectures This design reduces
come in different depths: computation by
18, 34, 50, 101, and 152 compressing intermediate
layers to balance feature maps, enabling
accuracy and practical training of very
computational demand. deep networks.

Scalable Efficiency
Even the deepest ResNet-152 models train efficiently on
modern hardware, validating the approach for scaling
depth.
Empirical Results: ImageNet & COCO
ImageNet Performance COCO Detection and Very Deep Models
Segmentation
ResNet-152 achieved a top-5 Networks beyond 1000 layers
error of just 3.57%, securing first The model showed superior were also successfully optimized,
place at ILSVRC 2015. generalization in object demonstrating robustness of the
detection and segmentation, residual design.
outperforming previous
architectures significantly.
Counterintuitive Insights
and Impact
Unexpected Findings Deep Nets
Outperform Shallow
Without skip
connections, training Residual design enables
error increases with deeper networks to
depth, a surprising outperform or match
behavior reversed by shallower ones
residual learning. consistently, defying
earlier limitations.
Broad Influence
The principles from ResNet have influenced
architectures beyond CNNs, including transformers and
AlphaGo’s policy networks.
Scope for Improvement & Legacy
Advancements on ResNet Future Directions

• Wide ResNets for improved width over depth Regularization, pruning, and efficiency
• ResNeXt for aggregated transformations improvements continue to evolve, building on
residual learning foundations.
• SENet introducing channel-wise attention
ResNet remains a foundational model and a core
inspiration across deep learning domains even
years after its introduction.
THANK YOU!

You might also like