LLaMA 3 Deep Dive! (Thomas Scialom - Meta)
Become a Patreon: / theaiepiphany
👨👩👧👦 Join our Discord community: / discord
Thomas joined us for the second time to talk about their latest work: LLaMA 3! We cover synthetic data for pre/post training, why didn't they go with MoE, privacy (was it trained on Facebook user data?), and much more.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
LLaMA 3: https://ai.meta.com/research/publicat...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 - 00:27 Intro
00:27 - 02:08 Hyperstack GPUs platform! (sponsored)
02:08 - 06:40 What is new in new Llama?
06:40 - 13:30 Synthetic data
13:30 - 15:35 Privacy - training on Facebook user data?
15:35 - 19:10 Scaling and distillation
19:10 - 25:35 MoE, new architectures?
25:35 - 37:15 Upper boundary for the quality of SX data?
37:15 - 45:10 Context length
45:10 - 46:40 What framework does Meta use for Llama
46:40 - 51:20 Playing with smaller Llamas
51:20 - 53:20 Multilingual capabilities
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 SPONSOR
The AI Epiphany - / theaiepiphany
One-time donation - https://www.paypal.com/paypalme/theai...
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💼 LinkedIn - / aleksagordic
🐦 Twitter - / gordic_aleksa
👨👩👧👦 Discord - / discord
📺 YouTube - / theaiepiphany
📚 Medium - / gordicaleksa
💻 GitHub - https://github.com/gordicaleksa
📢 AI Newsletter - https://aiepiphany.substack.com/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#llama3 #llms #meta #opensource