Nvidia CEO Jensen Huang unveiled a new inference system at the company’s annual GTC conference on Monday — the company’s most decisive move yet to defend its dominance as inference becomes AI’s next battleground.
Loading audio narration…
The new push into inference comes as Huang said Nvidia projects massive demand. The company expects at least $1 trillion in demand for its Blackwell and Rubin AI systems through 2027 — up from about $500 billion in projected demand through 2026, he said.
The AI chip giant announced the new Nvidia Groq 3 LPX, which Huang said can speed up inference workloads by up to 35 times. It integrates technology from AI chip startup Groq and pairs it with Nvidia’s Vera Rubin architecture.
Samsung manufactures the new Groq chip, and Nvidia expects the system to ship in the second half of this year.
“The inflection point of inference has arrived,” Huang said at the keynote.
Nvidia’s new system builds on the roughly $20 billion deal it struck with Groq in December, which saw it license Groq’s technology and hire its top engineers.
Huang had previously hinted at a collaboration with startup Groq during Nvidia’s latest earnings call. The Wall Street Journal earlier reported that the company was preparing a new inference system incorporating Groq technology.
Nvidia’s graphics processing units (GPUs) still dominate the AI field and can be used for both training AI models and inference, or how AI models make decisions or predictions.
Now, a growing number of Nvidia competitors — from hyperscalers to chip startups — are developing specialized systems that are cheaper and more efficient for the repetitive and cost-sensitive work of inference.
The rise of AI agents — or tools that conduct tasks on behalf of humans — could dramatically increase inference demand.
To this end, AI companies like OpenAI have explored alternatives to Nvidia hardware. Reuters previously reported that it was dissatisfied with the company’s inference chips. In January, OpenAI signed a reported $10 billion compute deal with inference chip startup Cerebras.
Have a tip? Contact this reporter via email at gweiss@businessinsider.com or Signal at @geoffweiss.25. Use a personal email address, a nonwork WiFi network, and a nonwork device; here’s our guide to sharing information securely.
