Google’s latest Gemini 2.5 Flash model is already sparking debate. Internal assessments suggest that while the model is more adept at following user instructions, it’s also slipping up on safety more often than its predecessor, Gemini 2.0 Flash. Automated tests indicate a 4.1% drop in text-to-text safety and a 9.6% decline in image-to-text safety, meaning it’s more likely to produce content that contravenes Google’s guidelines.
A Google spokesperson confirmed these findings amid a broader industry shift. Many tech companies are loosening restrictions to allow more open conversation on controversial topics. For example, Meta has adjusted its Llama models to maintain neutrality in political discussions, while OpenAI is preparing to roll out future models that offer a wider range of perspectives.
Yet, such flexibility isn’t without risks. An earlier report from TechCrunch highlighted how a bug in OpenAI’s ChatGPT allowed inappropriate content generation for minors. Similarly, Google’s Gemini 2.5 Flash sometimes takes instructions too literally—even when those instructions push familiarity with sensitive topics. Although some issues are chalked up to false positives, there are clear cases where the model generates content that breaches safety policies.
The technical report lays out this tension plainly. While striving to follow every instruction, the model struggles to keep controversial queries in check. Benchmark tests like SpeechMap reveal that Gemini 2.5 Flash is less evasive when confronted with challenging or controversial subjects. In fact, tests on the OpenRouter platform even saw it suggesting ideas like replacing human judges with AI or backing warrantless surveillance.
Commenting on the report, Thomas Woodside from the Secure AI Project stresses the need for more transparency. He points out that without detailed examples of the policy breaches, it’s hard for independent analysts to fully gauge the risk. It’s a reminder that balancing clear instruction-following with robust safety measures remains a significant challenge.
Google has faced similar criticism before. The delayed launch and initial lack of detailed safety metrics in the Gemini 2.5 Pro report only added fuel to the fire. With this more comprehensive report, the tech giant aims to shed more light on its safety processes—even though some questions still linger.
If you’ve ever struggled to reconcile performance with safety in tech, these developments offer a timely reminder: the path to smarter AI is lined with trade-offs. As these systems evolve, keeping a close eye on how safety measures adapt is more crucial than ever.
