Positional Encoding in Transformers | Deep Learning

Опубликовано: 24 Апрель 2026
на канале: Learn With Jay
13,440
892

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

Timestamps:
0:00 Intro
0:42 Problem with Self-attention
2:30 Positional Encoding Derivation
11:32 Positional Encoding Formula
13:04 How it capture relative positions?
19:06 Concatenate or Add positional Encoding?
21:39 How Positional Encodings do not interfere with Word Embeddings?
25:04 Outro

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

Transformation Matrix resource - https://blog.timodenk.com/linear-rela...

Code - https://github.com/Coding-Lane/Positi...

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Follow my entire Transformers playlist :

📕 Transformers Playlist:    • Transformers in Deep Learning | Introducti...  

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

✔ RNN Playlist:    • What is Recurrent Neural Network in Deep L...  

✔ CNN Playlist:    • What is CNN in deep learning? Convolutiona...  

✔ Complete Neural Network:    • How Neural Networks work in Machine Learni...  

✔ Complete Logistic Regression Playlist:    • Logistic Regression Machine Learning Examp...  

✔ Complete Linear Regression Playlist:    • What is Linear Regression in Machine Learn...  

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖