Contrastive Learning in PyTorch - Part 1: Introduction
▬▬ Notes ▬▬▬▬▬▬▬▬▬▬▬
Two small things I realized when editing this video
SimCLR uses two separate augmented views as positive samples
Many frameworks have separate projection heads on the learned representations
which transforms them additionally for the contrastive loss
▬▬ Papers/Sources ▬▬▬▬▬▬▬
Intro: https://sthalles.github.io/a-few-word...
Survey: https://arxiv.org/ftp/arxiv/papers/20...
Supervised Contrastive Learning: https://arxiv.org/abs/2004.11362
Contrastive Loss: / losses-explained-contrastive-loss
Triplet Loss: https://towardsdatascience.com/triple...
NT-Xent Loss: https://medium.datadriveninvestor.com...
SimCLR
▬▬ Used Icons ▬▬▬▬▬▬▬▬▬▬
All Icons are from flaticon: https://www.flaticon.com/authors/freepik
▬▬ Used Music ▬▬▬▬▬▬▬▬▬▬▬
Music from Uppbeat (free for Creators!):
https://uppbeat.io/t/t-check/lemon-limes
License code: KJ7PFP0HB9BWHJOF
▬▬ Used Images ▬▬▬▬▬▬▬▬▬▬▬
All Images are from pixabay.com (Cats, Dogs, ...)
and royalty-free.
▬▬ Timestamps ▬▬▬▬▬▬▬▬▬▬▬
00:00 Introduction
00:22 Overview
01:35 Supervised vs. Self-Supervised CL
03:25 Applications
04:24 Popular Papers
06:17 Metric Learning
07:21 Loss 1
09:39 Loss 2
10:54 Loss 3
13:22 Variations between Losses
13:42 Part 2 Outlook
▬▬ Support me if you like 🌟
►Website: https://deepfindr.github.io/
►Support me on Patreon: https://bit.ly/2Wed242
►Buy me a coffee on Ko-Fi: https://bit.ly/3kJYEdl
►Coursera: https://imp.i384100.net/b31QyP
►Link to this channel: https://bit.ly/3zEqL1W
►E-Mail: [email protected]
▬▬ My equipment 💻
Microphone: https://amzn.to/3DVqB8H
Microphone mount: https://amzn.to/3BWUcOJ
Monitors: https://amzn.to/3G2Jjgr
Monitor mount: https://amzn.to/3AWGIAY
Height-adjustable table: https://amzn.to/3aUysXC
Ergonomic chair: https://amzn.to/3phQg7r
PC case: https://amzn.to/3jdlI2Y
GPU: https://amzn.to/3AWyzwy
Keyboard: https://amzn.to/2XskWHP
Bluelight filter glasses: https://amzn.to/3pj0fK2