Large Multimodal Models Are The Future - Text/Vision/Audio in LLMs
Vision and auditory capabilities in language models bring AI one step closer to human cognitive capabilities in a digital world
Code & Notebook: https://github.com/ALucek/multimodal-...
Chapters:
00:00 - Multimodal Understanding
02:34 - Image: Introduction
04:01 - Image: Vision Transformer
06:37 - Image: CLIP
09:36 - Image: Flamingo
13:32 - Image: BLIP-2
16:43 - Image: Modern Techniques
17:39 - Image: Example
19:02 - Video: Introduction
20:48 - Video: TimeSFormer
22:31 - Video: VideoMAE
23:35 - Video: InternVideo2
26:01 - Video: Apollo
27:39 - Video: Example
30:07 - Audio: Introduction
31:34 - Audio: Speech Aside
32:48 - Audio: Audio Spectrogram Transformer
35:06 - Audio: Audio Flamingo
36:28 - Audio: GAMA
37:17 - Audio: Example
38:33 - Large Multimodal Models
#ai #coding #datascience