Nvidia's Nemotron 4 Targets One Trillion Parameters, Still Trails China
Nvidia is working on Nemotron 4, an open-weight model that aims to compete with the best freely available models worldwide. The largest version of Nemotron 4 will have at least one trillion parameters, more than double the size of its predecessor Nemotron 3 Ultra.
This scale is already surpassed by Chinese labs, including Moonshot AI's Kimi K3, which has 2.8 trillion parameters, and DeepSeek V4 Pro, with 1.6 trillion. At launch in June, Nemotron 3 Ultra was the strongest open US model on the Artificial Analysis Intelligence Index, but still trailed Kimi K2.6.
Nvidia has tripled its cloud spending on in-house model training to $28 billion through 2031. The earliest possible release of Nemotron 4 would be this fall. Nvidia is also among the signatories of a petition against regulating open models, which could put it in direct competition with major customers like OpenAI.