NVIDIA Unleashes Cosmos 3, Revolutionizing Robotics with Physics-Driven Decision-Making

NVIDIA has revolutionized robotics with its Cosmos 3 platform, which enables robots to transition from vision-based to physics-driven decision-making. This shift is made possible by World Action Models (WAMs), a cutting-edge approach that combines physical world modeling and action prediction.

Cosmos 3 is built on NVIDIA's Mixture-of-Transformers (MoT) architecture, allowing for multimodal learning across video, text, audio, and action. The platform comes in three configurations, Edge (4B parameters), Nano (16B), and Super (64B), to cater to both on-device and workstation deployments.

The benefits of Cosmos 3 are numerous, including reduced task-specific data requirements, improved generalization to unseen environments, and accelerated adaptation to new robotic platforms. A recent study showed that policies trained on Cosmos 3's omni-model architecture outperformed traditional methods, improving task success rates from 28.1% to 36.8% on the RoboLab benchmark.