EvoHarness-RL Boosts AI Performance by 50% with Trainable Coordination Layer

Meta AI and the University of Illinois Urbana-Champaign have collaborated on a groundbreaking project called EvoHarness-RL, a trainable coordination layer that enables large language model agents to create, access, and manage their own external state. This innovation has led to a significant improvement in performance, with a 96.9% success rate on a standard benchmark, up from 47.9% for the baseline model running without it.

The framework introduces a structured external state built around three components: Belief, Progress, and Experience (BPE). The BPE representation captures the agent's current understanding of its environment, tracks completed and pending subgoals, and stores lessons from errors to adapt its approach. Training happens in two stages: supervised fine-tuning using expert demonstrations and cost-aware optimization through Group Relative Policy Optimization.

The researchers tested their approach on the ALFWorld benchmark for embodied AI, achieving impressive results in both familiar and unseen environments. The 86.6% success rate in unseen environments is particularly notable, as it demonstrates the model's ability to generalize and adapt to new tasks. The framework also exhibits two emergent phenomena: 'harness annealing,' where the agent internalizes routine harness operations, and 'harness evolution,' where the BPE representation becomes leaner and more specialized.