Google Cloud has announced that dynamic slicing is now generally available on Google Kubernetes Engine (GKE), offering a more flexible and efficient way to manage large-scale AI workloads on Tensor Processing Units (TPUs). This feature decouples logical TPU slice formation from physical nodepool provisioning, allowing for dynamic assembly of custom TPU slice topologies on demand.
Dynamic slicing reduces AI workload startup times by up to 5x and workload recovery by 4.5x, significantly improving TPU goodput, efficiency, and resilience. Previously, managing TPU capacity required separate, fixed nodepools for each TPU slice, increasing startup times and complexity. Dynamic slicing eliminates this rigid approach by using TPU optical circuit switching (OCS) technology to stitch together TPU cubes dynamically, forming larger slices (super-slicing) or partitioning cubes into smaller slices (sub-slicing) for multiple workloads within a nodepool.
The feature offers several key improvements, including zero-downtime topology agility, up to 5x faster job startup, improved fleet utilization, and up to 4.5x faster recovery and fault tolerance. It also provides enhanced visibility and control over TPU partitions, with real-time health and usage metrics for optimized workload scheduling.
Dynamic slicing is available in two forms: super-slicing, which creates larger TPU slices by connecting multiple TPU cubes, and sub-slicing, which partitions a single TPU cube into smaller, independent slices. This flexibility supports a wide range of AI workloads, from large-scale training to inference deployments, and enhances operational efficiency across thousands of accelerators.