AWS CEO Sees Inference Dominating AI Compute Demand

AWS CEO Matt Garman highlighted a significant shift in AI demand during his recent address at re:Invent 2025. Specifically, he noted that AI inference now accounts for roughly two-thirds of all AI compute demand, up from about one-third in 2023.

Inference is the process of putting trained models to work, whereas training is the initial teaching phase. This shift has significant implications for various industries, including crypto mining, which has been pivoting towards offering AI and high-performance computing services.

AWS is betting big on inference, with 80-90% of enterprise AI value expected to come from inference-powered agents in the future. These agents will automate tasks, run AI agents, and fundamentally change how businesses operate.

The company has also showcased its Trainium3 chips, purpose-built for inference workloads rather than training-optimized GPUs. This move highlights a fundamental change in hardware requirements, with inference needing different optimization than training.