Banks Bet Big on On-Prem AI with NVIDIA

NVIDIA's DGX Spark machines are gaining popularity among banks due to their ability to run AI workloads on-premises, disconnected from the cloud. This allows banks like Morgan Stanley and JPMorgan to maintain control over their sensitive data.

According to Perplexity CEO Aravind Srinivas, big banks want a hybrid approach that combines local hardware with cloud access for frontier models. He noted that 'there's a lot of ram in our own devices, there's a lot of power in our own offices, in our own homes that we're not actually tapping into for AI inference today.'

NVIDIA's on-prem revenue has hit $8 billion in the automotive vertical and $7 billion in financial services, manufacturing, and healthcare combined. The company expects non-hyperscaler categories to account for roughly half of its data center business.