Nvidia Unveils $20B Groq Racks for AI 'Talking' Chips

Nvidia is expanding its presence in the AI chip market with the delivery of its Groq 3 LPX racks, which are expected to start this year. The $20 billion acquisition of Groq technology last December has enabled Nvidia to move into a new category of chips that focus on the 'talking' part of AI workloads, specifically the decode phase.

The Groq chips are designed for low-latency applications and can deliver 3,400 tokens per second, according to benchmarks from Artificial Analysis. They will be used in the Nebius data center alongside Nvidia's Vera CPUs and Rubin GPUs. This marks a new revenue stream for Nvidia, targeting customers who prioritize speed over raw training power.

Nvidia is not alone in this market, with AMD combining its systems with Cerebras chips for similar low-latency work. The split between training and inference has become a defining theme in the AI hardware market, with Nvidia's acquisition of Groq technology giving it a way to cover both sides.