IBM Research has introduced a new plugin library called vLLM-Hook MIA, designed to enable fast runtime monitoring and control of large language models (LLMs). The tool allows developers to observe and steer a model’s evolving states during inference, addressing issues like malicious prompts, hallucinated facts, or tone and accuracy adjustments, all with minimal impact on performance.
The plugin targets the inference engine vLLM, the primary open-source engine for enterprise workloads. According to a new technical report by IBM researchers, vLLM-Hook MIA is up to 20 times faster than its predecessor, vLLM-Hook, and up to 14 times faster than similar methods that insert code into the model. “We show that you can observe and intervene in an AI model’s generation process without a major lag in performance,” says Irene Ko, an IBM researcher leading the project.
The tool addresses a key challenge in AI inference: balancing speed with transparency. Inference engines like vLLM prioritize speed through features like graph capture and replay, which reduce CPU bottlenecks. However, this efficiency makes runtime monitoring difficult, as many interpretability tools require disabling these features. IBM’s solution involves capturing and steering model internals before the graph is recorded, using a “scheduler” and a “courier” to manage access without interrupting processing.
IBM researchers demonstrated that vLLM-Hook MIA can delay inference by just 2% for smaller models and 6% for larger models like Llama 70B. The tool outperformed other methods, including vLLM-Lens and EasySteer, which either disable graph replay or require constant repository synchronization. The development was motivated by the need to bring interpretability tools from research environments into production, where speed and safety are critical.
To encourage further development, IBM and other researchers have launched The Steerability Challenge, connected to the upcoming NeurIPS 2026 conference. Participants can compete for $25,000 in prizes by developing steering interventions that suppress “dishonest” behavior without degrading model capabilities. The new plugin aims to provide enterprises with a practical way to ensure fast, transparent, and controllable AI deployment.