The Allen Institute for Artificial Intelligence, or Ai2, is shaking things up with their latest release: the OLMo 2 32B. This open-source language model is going toe-to-toe with big names like GPT-3.5-Turbo and GPT-4o mini. But here’s the kicker—OLMo 2 32B is all about transparency. You can dive right into its code, data, and training details, which is a big win for the open-source community.
One of the standout features of OLMo 2 32B is its efficiency. It only needs a third of the computational power that similar models, like Qwen2.5-32B, require. This means even if you’re working with limited resources, you can still get in on the action.
The team behind OLMo 2 32B took a three-step approach to training. First, they fed it 3.9 trillion tokens to learn the basics of language patterns. Next, it was all about quality, with the model learning from top-notch documents and academic sources. Finally, they used the Tulu 3.1 framework to teach it how to follow instructions, blending supervised and reinforcement learning techniques.
To make all this training possible, Ai2 developed OLMo-core, a platform that keeps everything running smoothly across multiple machines. The training itself happened on Augusta AI, a supercomputer setup with 160 machines using H100 GPUs, cranking out over 1,800 tokens per second per GPU.
Now, it’s true that models like Qwen2.5 and Gemma 3 score higher on average benchmarks. But what sets OLMo 2 32B apart is its fully open-source nature, a stark contrast to the partial open-source efforts from companies like Alibaba and Google.
Nathan Lambert from Ai2 remarks, “With just a bit more progress, everyone can pretrain, midtrain, post-train, whatever they need to get a GPT 4 class model in their class. This is a major shift in how open-source AI can grow into real applications.”
This release builds on Ai2’s earlier work with Dolma in 2023, further solidifying their commitment to open-source AI. They’ve released several checkpoints for OLMo 2 32B, so you can really dig into the details and see how it all works.
A paper published in December, along with the 7B and 13B versions of OLMo 2, provides more technical insights. Lambert’s analysis suggests that the gap between open and closed-source AI systems is narrowing, with just about an 18-month difference now.
While OLMo 2 32B is on par with Google’s Gemma 3 27B in basic training, Gemma 3 pulls ahead after fine-tuning. This highlights some areas where open-source models can still improve, especially in post-training methods. Ai2 is already working on boosting the model’s logical reasoning and its ability to handle longer texts.
If you’re curious to see OLMo 2 32B in action, you can test it out on Ai2’s Chatbot Playground. And while Ai2 did roll out the larger Tülu-3-405B model in January, Lambert notes that it’s not fully open-source since Ai2 wasn’t involved in its pretraining.
