I Reverse Engineered Deepseek R1: Here Is The Code and Explanation Of The Method
Colab Notebook: https://colab.research.google.com/dri...
Richard Aragon explores the Deepseek R1 language model, focusing on its unique use of reinforcement learning. He begins by explaining the model's architecture, which uses a combination of a smaller model and a hyperbolic model. Richard then dives into the model's training process, highlighting its use of reinforcement learning without relying on supervised fine-tuning. He explains the two reward functions used in the training process: an accuracy reward and a format reward. Richard also points out an "aha moment" that emerges during training, where the model learns to step back and re-evaluate its responses.
Richard then provides the code for the model and explains how to implement it. He discusses the distillation process, where a larger "teacher" model's knowledge is transferred to a smaller "student" model. Richard emphasizes the simplicity and effectiveness of this process, noting how it can lead to significant performance improvements in the student model.
Throughout the video, Richard provides his insights and opinions on the model and its implications. He expresses concerns about the potential for gaming benchmarks and the financial incentives surrounding large language models. Richard concludes by encouraging viewers to like and subscribe to his channel for more content on AI and related topics.