TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI

Enterprise AI deployments often route each LLM call independently to balance cost and quality, but agentic workflows only get evaluated by delayed, task-level outcomes, misaligning per-call routing with actual performance signals. TRACE-Router fixes this by assigning an entire task to one model at the start via a contextual bandit, then updating its policy using the task's terminal reward balancing accuracy and latency. Across agentic benchmarks like tau2-Bench and Terminal-Bench, it achieved notable accuracy and latency improvements over single-model baselines. This has clear applications in enterprise LLM infrastructure, optimizing cost-quality tradeoffs for long-horizon, multi-step agentic applications. Authors: Ritik Raj, Souvik Kundu, Sarbartha Banerjee, Dheemanth Joshi, Ishita Vohra, Tushar Krishna Paper: https://arxiv.org/abs/2607.22465v1