Majestic Labs Challenges Nvidia's Dominance with Memory-Focused AI Server

Majestic Labs has unveiled its server, Prometheus, which claims to do the work of a rack of Nvidia GPUs by addressing a different bottleneck in AI inference - memory. The company's founders, former Google and Meta engineers, argue that pairing pricey GPUs with scarce high-bandwidth memory is a dead end.

Majestic Labs has designed its server to swap graphics chips for Ignite AI Processing Units, which blend Arm cores with RISC-V vector and tensor engines. Each unit sits in one server, sharing a single pool of 8TB to 128TB of LPDDR6 memory. This is the cheap memory found in phones, not the costly high-bandwidth memory that GPUs depend on.

The trick behind Prometheus is its custom aggregation chiplets, which pool memory through copper cables up to a metre long. The result is one coherent pool far larger than a GPU box can address. According to Majestic Labs, an Nvidia DGX B300 with eight Blackwell GPUs carries 2.3TB of high-bandwidth memory, while Prometheus offers more than 50 times as much fast memory at 1.7 times the interconnect bandwidth.