Nvidia Groq Racks Enter Full Production, $20 Billion Acquisition Pays Off

Nvidia's $20 billion acquisition of Groq assets has taken a significant step forward. The company announced that its Groq 3 LPX rack is in full production, marking the commercialization of the technology.

The Groq 3 LPX rack will be deployed at neocloud Nebius alongside Vera central processors and Rubin graphics processors. It's expected to be online later this year, with Nvidia senior director Dion Harris noting that it unlocks premium tiers of service for users demanding low-latency inference.

Nvidia's focus on low-latency chips is a competitive space, with Advanced Micro Devices (AMD) integrating its rack-scale systems with Cerebras' chips. OpenAI's Ultrafast mode also promises 750 tokens per second, powered by Cerebras.

However, Harris emphasized that low-latency chips like Groq are not meant to replace GPUs, but rather focus on a specific part of the workload called the 'decode' phase.