MiniMax-01: Scaling Foundation Models with Lightning Attention
MiniMax-01: https://arxiv.org/abs/2501.08313
Lightning attention: https://arxiv.org/abs/2405.17381
Lightning attention v2: https://arxiv.org/abs/2401.04658
Lightning attention code: https://github.com/OpenNLPLab/lightni...
Huggingface page with models: https://huggingface.co/papers/2501.08313
Notes (lightning attention): https://drive.google.com/file/d/1DMPL...
Notes (Minimax): https://drive.google.com/file/d/1DINk...
00:00 Intro
01:30 Linear attention
15:04 Lightning attention
29:11 Lightning attention code and some remarks
34:20 MiniMax