AI models' slowdown spells end of gold rush era

FE Team | Published: December 16, 2024 00:07:10

NEW YORK, Dec 15 (Reuters Breakingviews): Amid the many uncertainties in the artificial intelligence field, one thing has always seemed clear: bigger and more expensive systems produce better results. Hence the relentless fundraising of model developers like $157 billion OpenAI and the mammoth capital expenditures of Big Tech groups.
Now, however, that kernel of certainty seems to be disintegrating. Having run out of novel data on which to train the software, researchers are struggling to get smarter outputs simply by throwing more resources at the problem. The gold rush phase may be ending, opening the field for nimbler new competitors.
Technologists until recently agreed that AI "scales", meaning that bigger is better. In 2020, researchers at Sam Altman's OpenAI showed that so-called large language models improved, at steady rates when trained using more data, computing power and parameters, which are like the knobs and dials of the system. That provoked an arms race for chips and data centers, with analysts expecting Microsoft alone to spend $64 billion on capex in 2025, or six times as much as General Motors. Investors buy it: Alphabet, Amazon.com, Meta Platforms, Microsoft and Nvidia's combined market capitalization is $8 trillion higher than in November 2022, when OpenAI released, ChatGPT.
The snag is that evidence for the AI scaling law seems to be unraveling. Cutting-edge systems have already sucked up most of the world's useful and available training data. Multiple AI labs have had problems wringing out improvements in the next generation of models. Sundar Pichai, Alphabet's CEO, said at a recent event that leading models had converged on similar performance levels, and the hill for further improvements was now steeper. OpenAI's Altman said "there is no wall" at the same conference, while acknowledging that the easier gains from AI scaling have faded.
Some researchers hope that future advances will come from better algorithms, instead of the historic brute-force approach. A technique known as "test-time compute" focuses on enhancing the inference process, which refers to when a customer uses the AI system. Giving the models extra time to spot patterns or use new data could yield better results, perhaps allowing the machine to break big problems into smaller ones. While promising, that's a step-down from the vision of exponentially improving software that AI proponents had been pushing. And once a model has thought through all the possible answers to a problem, adding more time doesn't necessarily help. Users may also look elsewhere for an answer if an AI system takes too long.
Whispers of a slowdown haven't troubled the share prices of Alphabet, Amazon, Meta, Microsoft and Nvidia. It's likely that a new era will affect the players in different ways.
Nvidia arguably has the most to lose. Jensen Huang's company benefited from a mad rush to secure access to its silicon, epitomized by recent plans from Elon Musk's xAI to build a supercomputer housing 1 million graphics processing unit chips, or 10 times its already enormous current level. In the future, companies might instead prefer to use more specialized, cheaper semiconductors, putting Nvidia's $3.3 trillion equity value at risk. Model developers like OpenAI and Dario Amodei's Anthropic probably feel ambivalent.
On the positive side, their financial statements will benefit if they no longer have to train ever-bigger systems. On the other hand, the end of AI scaling would undermine part of the bull case for these companies, which is that OpenAI or Anthropic's proprietary models will get ever smarter and eventually supersede much of the world's existing software.
The picture is similarly mixed for the behemoths. The good news is that Microsoft's Satya Nadella and his peers may no longer face the existential risk of watching a rival company develop a superintelligent, giant model capable of performing any task, which might have been conceivable if AI scaled indefinitely. Losing that race would have meant missing out on possibly the greatest wealth-generating technology of all time. So while the jackpot may now be smaller, so is the risk of getting squashed by a more powerful rival.
Freed from that concern, Nadella and Pichai could take their foot off the gas and wait for more revenue to justify all the spending. Shareholders will be happy: past booms in railroads and telecoms show the danger of over-enthusiasm surrounding new technologies. Even Meta's Mark Zuckerberg has acknowledged that companies have probably overinvested. Computing costs for the largest models have been doubling every eight months, and the power consumption for training doubled every year, according to research group Epoch.ai.
Yet an end to the capex arms race may also mean lower barriers to entry. If gargantuan computing power is no longer the order of the day, new startups should be able to produce competitive AI products at minimal cost, perhaps by basing their designs on open-source models provided by Zuckerberg's Meta. It's possible to imagine a new wave of enterprise-software businesses that tweak the widely available systems to serve specific industries, like the legal profession or coders. Y Combinator, which funds and incubates fledgling Silicon Valley firms, is teeming, opens new tab with hundreds of these potential new challengers.
Regardless of which companies win, it's probably welcome news for investors overall if AI training costs stops spiralling upwards.
That would follow a recent precipitous decline in the price of inference, or the expense incurred when a customer uses an already trained model. Processing a million tokens, which is a unit of data, cost $60 three years ago. Now it costs 6 cents, according to venture firm Andreessen Horowitz,. Cost deflation should aid adoption, allowing the early signs of progress to proliferate. At Meta, for example, quarterly ad revenue is up 46 per cent since the pre-ChatGPT period, perhaps because of better advertising targeting, while operating costs are only up 5 per cent. After the gold rush comes the hard task of proving a return on investment - and justifying investors' giddy expectations.