ResNet Deep Residual Learning for Image Recognition (1)
ResNet Deep Residual Learning for Image Recognition (1)
Degradation Problem
Adding more layers unexpectedly caused accuracy degradation,
not just overfitting. This problem occurs even when the deeper
model is theoretically capable of achieving the same or better
performance than a shallower one. It highlights an optimization
challenge, not overfitting or underfitting.
Why Previous Approaches Fell Short
Conventional Deep Models :-
Earlier networks like VGG and GoogleNet increased depth by stacking layers but faced optimization
bottlenecks, causing training errors to increase.
Limitations :-
3 Optimization Simplification
Allows deeper networks to be optimized as residual
functions instead of direct mappings, leading to easier
convergence during training.
Basic Structure of a Residual Block in ResNet34
Layer 1
• Convolution
• Batch Normalization
• Activation - ReLU
Layer 2
• Convolution
• Batch Normalization
Adder
• Adding input through Skip connection
• Activation - ReLU
ResNet Variants and
Scalability
Layer Depth Bottleneck Design
ResNet architectures This design reduces
come in different depths: computation by
18, 34, 50, 101, and 152 compressing intermediate
layers to balance feature maps, enabling
accuracy and practical training of very
computational demand. deep networks.
Scalable Efficiency
Even the deepest ResNet-152 models train efficiently on
modern hardware, validating the approach for scaling
depth.
Empirical Results: ImageNet & COCO
ImageNet Performance COCO Detection and Very Deep Models
Segmentation
ResNet-152 achieved a top-5 Networks beyond 1000 layers
error of just 3.57%, securing first The model showed superior were also successfully optimized,
place at ILSVRC 2015. generalization in object demonstrating robustness of the
detection and segmentation, residual design.
outperforming previous
architectures significantly.
Counterintuitive Insights
and Impact
Unexpected Findings Deep Nets
Outperform Shallow
Without skip
connections, training Residual design enables
error increases with deeper networks to
depth, a surprising outperform or match
behavior reversed by shallower ones
residual learning. consistently, defying
earlier limitations.
Broad Influence
The principles from ResNet have influenced
architectures beyond CNNs, including transformers and
AlphaGo’s policy networks.
Scope for Improvement & Legacy
Advancements on ResNet Future Directions
• Wide ResNets for improved width over depth Regularization, pruning, and efficiency
• ResNeXt for aggregated transformations improvements continue to evolve, building on
residual learning foundations.
• SENet introducing channel-wise attention
ResNet remains a foundational model and a core
inspiration across deep learning domains even
years after its introduction.
THANK YOU!