AMD's MI355X Beats Nvidia B300 in Cost-Effective AI Inference

A recent test by InferenceX shows that AMD's Instinct MI355X GPU outperforms Nvidia's B300 in terms of cost-effectiveness, specifically when running Moonshot AI's Kimi K3 model.

The Kimi K3 model has a massive 2.8 trillion parameters and requires 288 GB of high-bandwidth memory per GPU, limiting its deployment to newer platforms such as the Nvidia B300 and AMD MI355X.

The results show that the MI355X runs at less than half the hourly cloud rental cost of the B300, at $2.50 per GPU hour compared to $6 for the B300.

This lower rental price allows the MI355X to deliver a lower cost per generated token, despite Nvidia retaining an advantage in raw processing speed in some tests.