Nvidia Publishes Day-0 Inference Results for Alibaba's 2.4 Trillion Parameter Qwen3.8 Model

Nvidia has published Day-0 inference results for Alibaba's open-weight Qwen3.8-2.4T-A95B model, showing it running across its GB300 NVL72 rack-scale system.

The model has 2.4 trillion parameters in total but activates only 95 billion for each token, using a fine-grained mixture-of-experts (MoE) architecture to reduce computation required during inference.

Nvidia says its initial FP8 implementation exceeds 4,000 tokens per second per GPU and 350 tokens per second per user on GB300 NVL72.

The Qwen3.8 model combines full attention with linear attention, keeping memory and compute requirements under control as context lengths grow.

Nvidia's rack-scale approach differs from conventional GPU servers connected primarily through external networking, reducing the communication bottleneck created when MoE expert traffic has to move between accelerators.