Nvidia Faces Challenge from Hyperscaler Rivals in AI Chip Market

Nvidia's dominance in the AI chip market is being challenged by its biggest customers - Amazon, Google, and Microsoft.

The hyperscalers are trying to commoditize Nvidia's layer of the stack, which includes chip design and data center infrastructure. They want to capture more value from AI spending, which could reach trillions of dollars annually.

Nvidia's 75% gross margin business is seen as unsustainable by its customers, who have strong operating cash flows and are developing their own ASICs (Application-Specific Integrated Circuits) for inference workloads. Inference is becoming a dominant market, with companies like Microsoft processing over 100 trillion tokens in a quarter.

Google's TPU (Tensor Processing Unit) and Amazon's Trainium are already competitive with Nvidia chips on cost-per-token basis. Microsoft is also developing its own chip design capabilities, with the Maia 300 expected to be unveiled soon.