Learning to Route: Static Embedding Models as Self Improving LLM Routers for Coding Tasks

Frontier coding models cost 5x to 55x more per token than small models that solve most real coding tasks just as well. This paper describes a router where routing is literally embedding search: a task is embedded by a static embedding model (a token table lookup and mean pool, about 0.15ms on CPU from a 16MB file) and its k nearest previously seen tasks vote on which model to use, weighted by observed pass rates. Router state is vectors plus outcome counters, so it learns online from every completed agent task with no training loop. On a 27 problem coding benchmark spanning medium to hard exact tasks and optimization tasks scored by solution quality, a verify and escalate cascade over four cheap models solves 100% of tasks at 26% of the cost of the best single model and 4% of the cost of a frontier model (gpt-5.5), which ties a $0.03 model at 77.8%. The full experimental history cost about four dollars. Router, benchmark and serving code in Python, Go and Zig are MIT licensed.