Lemonflow, a company specializing in AI-driven solutions for electric vehicle (EV) charging, has developed a real-time agent system to address the high failure rates of EV charging attempts in Europe. The company reports that between 15% and 25% of charging attempts end in failure, leaving drivers stranded or stuck in long call center queues. To tackle this issue, lemonflow created AI agents that provide support in over 90 languages, assisting an EV driver every 90 seconds.
The challenge for lemonflow was to deliver ultra-low-latency voice interactions and ensure dedicated compute capacity. Initially relying on managed LLM APIs, the company faced latency deviations and HTTP 429 rate-limit errors during peak hours. To overcome these issues, lemonflow migrated to self-hosting the open Gemma 4 model on Google Cloud TPU v6e hardware in the EU. This move allowed for more complete architectural control and eliminated latency bottlenecks.
The solution involved developing an intelligent routing gateway to prioritize workloads and implementing custom optimizations to the vLLM serving engine. These changes reduced prompt update and test cycle times from 2-3 hours to about 10 minutes and lowered serving latency to around 350 milliseconds for P50 and 600 milliseconds for P90. The platform now supports more than 40 charging networks in 17 countries, covering over 250,000 charge points.
Looking ahead, lemonflow plans to move to Google’s seventh-generation TPU, Ironwood, which promises six times the high-bandwidth memory per chip. The company’s ultimate goal is to ensure no driver is ever left stranded at a charger, providing instant, expert answers in the driver’s own language and remotely rebooting chargers if necessary.