In the three weeks since Nvidia shocked the tech world with its forecast of unprecedented revenue growth, Wall Street has been on the hunt for other chip companies that could benefit from the latest AI boom.
But as the search progressed, the gap between Nvidia and the rest of the chip industry only widened.
In one of the most anticipated attempts to catch up with Nvidia, rival AMD this week showed off a new AI chip, the MI300X. The chip contains a GPU – a product originally designed for video games and at the heart of Nvidia’s success – as well as a more general purpose CPU and onboard memory to provide data to both processors.
The design reflects efforts by chipmakers to bundle different technologies in the search for the most efficient way to process the large amounts of data required to train and apply the large models used in generative AI.
AMD claimed impressive performance for its new chip, which it claims would surpass Nvidia’s flagship H100 on several counts. But it failed to pinpoint potential customers considering the chip and only emphasized the product’s ability to handle AI inference – by applying pre-trained AI models – rather than the more demanding task of training, which is behind the rising sales of Nvidia. It also said it will not begin ramping up production of the new chip until the last quarter of this year.
By the time AMD’s new chip becomes generally available in the first half of next year, Nvidia’s H100 will have been on the market for 18 months, giving it a huge head start, said Stacy Rasgon, an analyst at Bernstein. AMD is “far behind. They could get the dregs (of the AI market) — though perhaps even that is enough to justify Wall Street’s recent enthusiasm for the company’s stock, he said.
“Nvidia is free and clear in this round” of the chip wars that have erupted around AI, added Patrick Moorhead, an analyst at Moor Insights & Strategy.
Wall Street has singled out a few chip companies that could get a boost from generative AI. The combined market capitalization of AMD, Broadcom and Marvell rose $99 billion, or 20 percent, in the two days following Nvidia’s stunning sales forecast last month. But their AI-related sales are not expected to come from the market dominated by Nvidia.
For example, Broadcom can benefit from rising demand for its data communications products and from collaborating with Google to design an internal data center chip, known as TPU. Earlier this month, Broadcom predicted that AI-related activities would account for about a quarter of revenue by 2024, compared to just 10 percent last year.
However, the processors used to train and run large AI models are getting the biggest surge in demand and generating the most excitement in the stock market. As AMD’s new chip disappointed Wall Street, Nvidia’s stock market value soared back above $1 trillion, a level it first reached two weeks ago.
“There is no doubt that AI will be the main driver of silicon consumption for the foreseeable future,” with data centers the main focus of the investment, said AMD chief executive Lisa Su. She predicted that the market for AI accelerators — the GPUs and other specialized chips designed to speed up the data-intensive processing needed to train or run — would grow from $30 billion this year to more than 150. billion dollars by 2027.
As they struggle to compete with Nvidia for the most advanced AI chips, companies such as AMD and Intel are banking on an evolution in the generative AI market to drive demand for other types of processors. Large-scale models like OpenAI’s GPT-4 dominated the early stages of the technology, but a recent explosion in the use of smaller and more specialized models could lead to higher sales of less powerful chips, they argue.
Many customers looking to train models using their business data will also want to keep their information close to home rather than risk handing it over to the companies that provide large AI models, said Kavitha Prasad, vice president at Intel. Along with all the computing required to prepare data for training, that will create a lot of work for Intel’s CPUs and AI accelerators, she said.
However, with the rapidly changing demands on data centers caused by the proliferation of services like ChatGPT, chipmakers are struggling to anticipate how their markets will evolve. CPU sales could actually decline in the coming years as data center customers put their spending into AI accelerators, Rasgon said.
Rivals looking to take a bite out of Nvidia’s growing AI business face an equally daunting challenge when it comes to software. The widespread use of Nvidia’s chips in AI and other applications is largely due to the ease with which the GPUs, originally designed for video games, can be programmed for other tasks using the Cuda software.
In an effort to attract more developers to its AI chips, AMD this week highlighted its efforts to work with PyTorch, a widely used AI framework. Still, it has a long way to go to match the many software libraries and applications already developed for Cuda, Rasgon said. “It will be a decade” before rivals can match Nvidia’s software — a period in which Nvidia will still be quick to extend its lead, he said.
“No one wants an industry with one dominant player,” says Moorhead. But for now, the booming market for chips that can handle generative AI belongs to Nvidia.
