AWS Cracks Down on Internal CPU Use Amid Agentic AI Demand
Amazon Web Services (AWS) is cracking down on internal use of EC2 instances among its engineers. The company has reportedly told engineers to reduce CPU waste, citing a surge in demand for CPUs due to agentic AI workloads.
The traditional eight-to-one or four-to-one ratio of GPUs to CPUs is moving closer to parity, with CPUs taking center stage in the agentic era. Agentic workloads involve complex tool calls that run on the CPU, as well as orchestration of inference on GPUs.
AWS engineers are now waiting days to access EC2 instances, which was previously a process that took hours. The increased demand for CPUs has led to shortages, particularly for spot instances.
Amazon deploys several types of CPUs in EC2 instances, including AMD and Intel options and its relatively new Graviton5 chip. The major players are capitalizing on the demand, with AMD unveiling its portfolio of Zen 6 'Venice' CPUs for the data center.