NVIDIA's EPD Disaggregation Revolutionizes Multimodal AI Speed

NVIDIA has introduced an optimization technique for multimodal AI models called EPD disaggregation in its open-source framework, Dynamo. This method separates vision encoding, prefill, and decode stages into independently scalable components, resulting in up to 7x faster end-to-end response times.

EPD builds on the concept of prefill-decode (PD) disaggregation, which splits large language model inference into two stages: prefill, where the input is processed, and decode, where output tokens are generated. By adding a third stage, encoding, EPD further decouples workloads for multimodal inputs like images or video.

NVIDIA's lab testing showed that separating encoder workloads can support 70% more traffic at the same latency threshold in image-heavy use cases. The company's Dynamo implementation offers a practical solution for scenarios like video analysis, content moderation, and interactive AI applications requiring low-latency responses.