ResNet Explained - Vanishing Gradients, Skip Connections, and Code Implementation | Computer Vision
ResNet Explained - Vanishing Gradients, Skip Connections, and Implementation from Scratch
In this lecture from our "Computer Vision from Scratch" series, we dive deep into ResNet (Residual Networks) - a revolutionary architecture that changed the course of deep learning forever.
We begin by understanding the vanishing gradient problem that plagued deep neural networks before 2015. You will see why deeper models like VGG and Inception often failed to train effectively, even though they should have performed better.
Then we explore how ResNet solves this with a simple but powerful idea - residual or skip connections. We discuss:
Why shallow layers stop learning in deep networks
The intuition behind skip connections
Mathematical explanation of gradient decay
Why ResNet made training 100+ layer networks possible
How ResNet performs compared to AlexNet, VGG, Inception, and SqueezeNet on the same flower dataset
How to implement ResNet using PyTorch and transfer learning
Why ResNet is not just good at classification but also powerful in tasks like image enhancement
We also compare the top models across parameters, accuracy, and innovations, and finally implement ResNet-50 on a five-class flower dataset using PyTorch and track the results using Weights and Biases.
Final accuracy:
Training: 98 percent
Validation: 95 percent
The best result in the course so far.
Links
🧠 Miro Notes: https://miro.com/app/board/uXjVIoT5zH...
💻 Colab Notebook: https://colab.research.google.com/dri...