Nvidia Tries to Catch Up with China in Open-Source Language Models

Nvidia has released Nemotron 3.5 Lightning, a 30 billion parameter mixture-of-experts model that uses a hybrid architecture of Mamba-2, MoE, and attention layers with a one million token context window.

The model's weights are available on Hugging Face and ModelScope under the OpenMDW-1.1 license, which allows for commercial use without permission or payment to Nvidia.

Nvidia claims that Nemotron 3.5 Lightning achieves output speeds of up to four times that of similar-sized models in the lab, but only reaches a modest 30% speedup over Alibaba's Qwen 3.6 35B model on actual tasks.

The company has also released NeMo Switchyard, an open-source library that routes each step of an agent workflow to the best-suited model, with plans to route up to a frontier model and execution down to Lightning.