China's AI Industry Struggles with Compute Constraints Amid Surging Demand

China's AI industry is facing significant compute constraints as demand for inference surges. According to Guan Jiawei, vice-president of inference optimisation start-up Approaching.AI, high-tier tasks require stringent performance metrics that domestic processors cannot yet reliably deliver.

Guan noted that advanced Chinese models place high demands on chips, especially in scenarios like coding where users are willing to pay a premium.

The bottleneck is largely due to restricted access to Nvidia processors. High-quality tokens, which are the basic units of data that models process and generate, still rely heavily on these imported chips.