Cache-to-Cache Framework Revolutionizes Large Language Model Communication

A team of researchers from Tsinghua University's THU-NICS group has developed a framework called Cache-to-Cache, or C2C, that allows large language models to communicate directly with each other without using text. This approach significantly reduces latency and errors in model communication.

C2C uses the key-value caches of LLMs as the communication medium itself, eliminating the need for text-based communication between models. The system designates one model as the 'Sharer' and another as the 'Receiver', with a lightweight neural component called the 'Cache Fuser' projecting and fusing the Sharer's KV-cache directly into the Receiver's representation space.

The research team tested C2C across multiple model families, including Qwen, Llama, and Gemma, achieving accuracy improvements of up to 14.2% compared to individual models working alone. In comparison to traditional text-to-text communication between models, C2C delivered accuracy improvements of 3.1% to 5.4%. On the latency front, C2C reduced communication time by a factor of 2 to 2.5 times in standard configurations and up to 14 times faster in certain setups.