Orca LLM - bridging the gap between ChatGPT and opensource?!
Orca is the first model that addresses the very problem of reasoning and logical thinking prevalent in small open-source models like Vicuna and Alpaca. Its the reasoning ability that puts them behind ChatGPT and GPT-4.
Orca with just 13 Billion parameter seems to do as good as ChatGPT! So what is the secret recipe here? Lets find out in this video.
📝 Orca Paper: https://arxiv.org/abs/2306.02707
📝 Vicuna Blog: https://lmsys.org/blog/2023-03-30-vic...
📝 Stanford Alpaca: https://crfm.stanford.edu/2023/03/13/...
⌚️⌚️⌚️ TIMESTAMPS ⌚️⌚️⌚️
0:00 - Intro
1:56 - Instruction Tuning
2:38 - Problems with Instruction Tuning
3:45 - Explanation Tuning
4:32 - System Instructions
5:10 - Dataset Collection and FLAN-v2
7:04 - Sampling Algorithm
7:28 - Training Strategy
9:06 - Evaluation
11:15 - Limitations and Conclusion
🧸🧸🧸 TOYS OF THE TRADE 🧸🧸🧸
📝 XP-Pen for my scribbles - https://amzn.to/3HNOpPH
⌨️🐭 My Keyboard & Mouse - https://amzn.to/3XKZ3wF
🖥 My Monitor - https://amzn.to/3YdVpvT
💻 My GPUs - https://amzn.to/3YE2aXX
🎤 My Mic - https://amzn.to/3YCdDHA
My work desk - https://amzn.to/3GWF1IP
🛠 🛠 🛠 MY SOFTWARE TOOLS 🛠 🛠 🛠
✍️ Notion - https://affiliate.notion.so/aibites-yt
✍️ Notion AI - https://affiliate.notion.so/ys9rqzv2vdd8
📹 OBS Studio for video editing - https://obsproject.com
📼 Manim for some animations - https://www.manim.community
🎵 My music - https://www.bensound.com and
📚 📚 📚 BOOKS I HAVE READ, REFER AND RECOMMEND 📚 📚 📚
📖 Deep Learning by Ian Goodfellow - https://amzn.to/3Wnyixv
📙 Pattern Recognition and Machine Learning by Christopher M. Bishop - https://amzn.to/3ZVnQQA
📗 Machine Learning: A Probabilistic Perspective by Kevin Murphy - https://amzn.to/3kAqThb
📘 Multiple View Geometry in Computer Vision by R Hartley and A Zisserman - https://amzn.to/3XKVOWi
MY KEY LINKS
Website: https://www.ai-bites.net
YouTube: / @aibites
Twitter: / ai_bites
Patreon: / ai_bites
Github: https://github.com/ai-bites
WHO AM I?
I am a Machine Learning Researcher / Practioner who has seen the grind of academia and start-ups equally. I started my career as a software engineer 15 years back. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the now happening AI revolution just started. Life has changed for the better ever since.
#machinelearning #deeplearning #aibites