{"ID":5937133,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-09T11:06:26.522512815Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04951","arxiv_id":"2607.04951","title":"When Words Predict Workload","abstract":"Standard distributed \\ac{llm} schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On \\ac{epo} claims governed by Article 84 \\ac{epc}, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving this ambiguity mid-flight forces dynamic multi-model ensemble expansion, triggering unpredictable KV-cache and weight-allocation spikes that saturate consumer-grade edge GPU VRAM and cause severe \\ac{oom} crashes. To prevent hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway. The gateway extracts a 16-dimensional text-structure vector and applies an XGBoost predictor to forecast trap-band membership. The resulting escalation probability ($\\Pesc$) is evaluated against a dynamic, closed-form routing threshold ($\\Tauroute(t)$) computed via real-time latency telemetry. Requests are safely routed to either a local Qwen2.5-7B edge worker or a remote contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 \\emph{before} any edge GPU memory is allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction ($R_{\\mathrm{mis}}$) to $0.087$--$0.095$, an order of magnitude below the token-count baseline ($0.849$). Peak edge VRAM remained safely bounded at $\\SI{4.82}{\\gibi\\byte}$ (under the $\\SI{8}{\\gibi\\byte}$ ceiling) across a $27\\times$ variation in \\ac{wan} delay. The predictor achieved a live-trial AUROC of $0.84$, and the dynamic $\\Tauroute(t)$ controller yielded an $8.2\\%$ relative reduction in misroutes compared to an equivalent static threshold.","short_abstract":"Standard distributed \\ac{llm} schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On \\ac{epo} claims governed by Article 84 \\ac{epc}, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving thi...","url_abs":"https://arxiv.org/abs/2607.04951","url_pdf":"https://arxiv.org/pdf/2607.04951v1","authors":"[\"Anubhab Banerjee\"]","published":"2026-07-06T11:27:29Z","proceeding":"cs.DC","tasks":"[\"cs.DC\",\"cs.CL\"]","methods":"[\"Large Language Model\"]","has_code":false}