Nvidia-Backed Startup Teaches Robots New Tasks with Single Video

Nvidia-backed startup Skild AI has developed a general-purpose physical AI system that can learn new tasks from a single video. The technology, called S1, is being used in an Nvidia factory in Houston to assemble Blackwell GPU systems.

S1 was built on top of Nvidia's AI infrastructure and uses various tools such as Cosmos, Isaac Lab, Isaac Sim, and Omniverse for data generation, training, simulation, and deployment. According to the company, S1 can learn previously unseen tasks from a single video demonstration and adjust when objects move or recover from errors.

Amit Goel, director of product management for autonomous machines at Nvidia, said that S1 shows how quickly an operator can teach a robot new work. He also mentioned that collecting experience on real robots is slow and expensive, but simulation and synthetic data can help developers expand that experience more quickly.

The goal of the project is to connect training and testing to real operating conditions so that improvements translate into dependable performance on the factory floor. Goel said that Nvidia sees the industry moving toward an inflection point where shared foundation models can provide intelligence for different types of robots.