More than two years after Google was caught flat-footed by the release of OpenAI’s ChatGPT, the company has dramatically picked up the pace.
In late March, Google launched an AI reasoning model, Gemini 2.5 Pro, that leads the industry on several benchmarks measuring coding and math capabilities. That launch came just three months after the tech giant debuted another model, Gemini 2.0 Flash, that was state-of-the-art for the time.
Google’s Director and Head of Product for Gemini, Tulsee Doshi, told TechCrunch in an interview that the increasing cadence of the company’s model launches is part of a concerted effort to keep up with the rapidly evolving AI industry.
“We’re still trying to figure out what the right way to put these models out is — what the right way is to get feedback,” said Doshi.
But the ramped-up release time frame appears to have come at a cost. Google has yet to publish safety reports for its latest models, including Gemini 2.5 Pro and Gemini 2.0 Flash, raising concerns that the company is prioritizing speed over transparency.
Today, it’s fairly standard for frontier AI labs — including OpenAI, Anthropic, and Meta — to report safety testing, performance evaluations, and use cases whenever they launch a new model. These reports, sometimes called system cards or model cards, were proposed years ago by researchers in industry and academia. Google was actually one of the first to suggest model cards in a 2019 research paper, calling them “an approach for responsible, transparent, and accountable practices in machine learning.”
Doshi told TechCrunch that the company hasn’t published a model card for Gemini 2.5 Pro because it considers the model to be an “experimental” release. The goal of these experimental releases is to put an AI model out in a limited way, get feedback, and iterate on the model ahead of a production launch, she said.
Google intends to publish Gemini 2.5 Pro’s model card when it makes the model generally available, according to Doshi, adding that the company has already done safety testing and adversarial red teaming.
In a follow-up message, a Google spokesperson told TechCrunch that safety continues to be a “top priority” for the company, and that it plans to release more documentation around its AI models, including Gemini 2.0 Flash, moving forward. Gemini 2.0 Flash, which is generally available, also lacks a model card. The last model card Google released was for Gemini 1.5 Pro, which came out more than a year ago.
System cards and model cards provide useful — and unflattering, at times — info that companies don’t always widely advertise about their AI. For example, the system card OpenAI released for its o1 reasoning model revealed that the company’s model has a tendency to “scheme” against humans, and secretly pursue goals of its own.
By and large, the AI community perceives these reports as good-faith efforts to support independent research and safety evaluations, but the reports have taken on additional importance in recent years. As Transformer previously noted, Google told the U.S. government in 2023 that it would publish safety reports for all “significant,” public AI model releases “within scope.” The company made a similar commitment to other governments, promising to “provide public transparency.”
There have been regulatory efforts at the federal and state levels in the U.S. to create safety reporting standards for AI model developers. However, they’ve been met with limited adoption and success. One of the more notable attempts was the vetoed California bill SB 1047, which the tech industry vehemently opposed. Lawmakers have also put forth legislation that would authorize the U.S. AI Safety Institute, the U.S.’ AI standard-setting body, to establish guidelines for model releases. However, the Safety Institute is now facing possible cuts under the Trump Administration.
From all appearances, Google is falling behind on some of its promises to report on model testing while at the same time shipping models faster than ever. It’s a bad precedent, many experts argue — particularly as these models become more capable and sophisticated.
Add comment