AI System Demonstrates Expert-Level Performance in Real-Time Video Consultations
A new AI system, called AMIE (Video), has demonstrated expert-level performance in real-time video consultations. This system, developed by Google researchers, addresses a fundamental constraint in medical AI research: text-based interfaces that discard visual and auditory dimensions of clinical practice.
AMIE (Video) is an extension of the Articulate Medical Intelligence Explorer (AMIE), which previously showed expert-level performance in text-based diagnostic dialogue. The new system uses an asynchronous multi-agent architecture to divide labor across three specialized agents: a talker agent, a planner agent, and a perception agent.
The perception agent continuously reviews audio and visual streams, identifying clinically relevant non-verbal cues and contextualizing these observations within the ongoing conversation. This decoupled design allows AMIE (Video) to maintain natural conversational latency while performing diagnostic reasoning and audio-visual perception.