Pay Attention to MLPs - Paper Explained
In this video, I explain the paper “Pay Attention to MLPs”. This article claims that self-attention blocks used in transformer architecture aren't necessary in many applications, and offers the gMLP architecture, which delivers outcomes comparable to transformers without using attention. In addition, the article suggests that adding a small single-head self-attention into gMLP will result in an even better architecture called aMLP.
📑 Chapters:
0:00 Abstract
0:25 Introduction
1:37 Model (gMLP)
4:13 Results (gMLP)
5:46 Model (aMLP)
6:03 Resutls (aMLP)
6:16 Conclusion
📝 Link to the paper:
https://arxiv.org/abs/2105.08050
👥 Authors:
Hanxiao Liu, Zihang Dai, David R. So, and Quoc V. Le.
🔗 Helpful Links:
My Video on the Paper "Attention is All you Need"
• Attention Is All You Need - Paper Explained
My Video on the Paper "An Image Is Worth 16x16 Words"
• An Image Is Worth 16x16 Words - Paper Expl...
🙋♂️ Find me on:
Find me on: halflingwizard.me
🎁 Support the Channel:
If you’d like to support my work, you can check out my wishlist here: https://www.amazon.com/registries/gl/...
Your support helps me keep creating content like this. Thank you for being part of this journey!
#computer_vision #NLP #MLP #gMLP