Nvidia's Next Act: Riding the Inference Wave

Nvidia has already dominated the market for training AI models, but its next enormous opportunity lies in inference - the process of deploying trained models to perform tasks.

Think of training as a student learning at school, while inference is what happens after graduation, when the now-educated individual applies their knowledge on the job.

Nvidia's GPUs have been the backbone of AI training, but once a model is ready, it requires significant computing power to perform tasks such as answering questions, generating images, or searching for information. This demand for inference workloads could become enormous if AI agents become widespread in everyday life and business.

The company is positioning itself to benefit from this trend with its newest Vera Rubin architecture, which combines GPUs and CPUs to handle long-running inference workloads. However, the competition among chipmakers in this space will be intense, and Nvidia's market share in this growing segment of AI is not guaranteed.