OLMo
OLMo is a state-of-the-art, truly Open Language Model and its framework to build and study the science of language modeling is public. Unlike most prior efforts that have only released model weights and inference code, OLMo and the whole framework, including training data and training and evaluation code is released.
In this video, I talk about the following: What is OLMO’s architecture and how is it trained? How does OLMO perform?
For more details, please look at https://arxiv.org/pdf/2402.00838 and https://github.com/allenai/OLMo and https://github.com/allenai/OLMo-Eval and https://huggingface.co/allenai/OLMo-7B
Groeneveld, Dirk, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha et al. "Olmo: Accelerating the science of language models." arXiv preprint arXiv:2402.00838 (2024).