Big names in artificial intelligence are struggling as advanced AI models become increasingly expensive.
OpenAI is close to a new milestone. In September, the startup completed the first round of training for Orion, an artificial intelligence model that it hopes will outperform previous versions and even humans.
However, Bloomberg sources said Orion has not performed well. In a test run late last summer, the model did not respond well to a request to write untrained code. Overall, Orion has not outperformed GPT-4 or GPT-3.5.
After years of launching cutting-edge AI products at a breakneck pace, many major companies in the industry are seeing falling profits and skyrocketing model development costs.
It’s not just OpenAI, Google’s upcoming Gemini version hasn’t met internal expectations, while Anthropic has also delayed the release of its Claude 3.5 Opus model.
“The AGI bubble is bursting a little”
AI companies face a number of challenges. It’s increasingly difficult to find high-quality, untapped human-generated training data to build advanced AI systems, while incremental improvements are not enough to justify the enormous costs of building and operating models.
According to insiders, OpenAI has put Orion into a post-training process, a period of feedback and refinement before it is released to the public. However, OpenAI is not yet satisfied to release the model widely until early next year.
This has been a new problem in Silicon Valley in recent years, especially after the launch of ChatGPT. AI companies are struggling to keep up with the “scaling laws,” in which more computing power, data, and models are expected to pave the way for increasingly powerful AI.
The predicament also calls into question the goal of creating general AI that companies are pursuing.
The term refers to hypothetical AI systems that could match or surpass humans in mental tasks. OpenAI and Anthropic leaders have previously suggested that AGI could be just a few years away.
“The AGI bubble is bursting a little bit,” said Margaret Mitchell, an ethics scientist at the startup Hugging Face, who stressed the need for “different training methods” for AI models to perform well on a variety of tasks.
Responding to Bloomberg, a Google DeepMind spokesperson said the company is “happy with the progress of building Gemini” and will “share more when we are ready.” Meanwhile, OpenAI and Anthropic both declined to comment.
Still, AI companies are pursuing a “more is better” strategy. To build human-level AI products, they seek to increase computing power, data volume, and training time, which drives up costs.
According to Amodei, it might cost companies $100 million to train the most advanced models this year, but in the next few years that could be as high as $100 billion.
Another approach
Another question mark is taking center stage in Silicon Valley. In March, Anthropic released three new models, claiming that its most powerful option (Claude Opus) outperformed GPT-4 and Gemini on a number of benchmarks, such as graduate-level reasoning.
Over the next few months, Anthropic updated the two Claude models, but not the Opus. By October, some analysts noticed that information about the 3.5 Opus, including its release date, had been removed from the Anthropic website.
Like its competitors, Anthropic is having trouble developing Claude 3.5 Opus. After training, the company found that the model performed better than the previous version, but not by as much as expected given the scale and cost of development.
Anthropic representatives said the reason for removing some information about the 3.5 Opus comes from changes in marketing, only showing released and benchmarked models.
When asked if the Opus 3.5 would launch this year, an Anthropic spokesperson quoted CEO Dario Amodei in a November 11 podcast, saying the company still plans to release the model but declined to disclose a specific time frame.