{"ID":6138104,"CreatedAt":"2026-07-09T01:07:32.349475501Z","UpdatedAt":"2026-07-11T06:21:26.878598377Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.07046","arxiv_id":"2607.07046","title":"Voltron: Enabling Elastic Multi-Device Execution of LLM Inference for Empowered Edge Intelligence","abstract":"Large language models (LLMs) are widely used in intelligent services due to their remarkable capability in generative tasks. Typically, LLM-based services process the inference requests of the users in a centralized data center. Unfortunately, such centralized execution has limitations for end-users, such as increased response latency with communication overhead and privacy leakage risk. To alleviate the aforementioned limitations, there have been increasing pushes to execute LLM inference locally on user-end devices. However, the limited resources of a single edge device impose restrictions on achievable accuracy of LLMs. To overcome the issue, we first propose to leverage multiple user-end devices available at the edge for LLM inference, enabling the execution of larger models. Specifically, we propose Voltron, a novel on-device LLM inference framework that elastically utilizes multiple user-end devices for LLM inference execution while adapting to diverse real-world edge environments. In our evaluation, Voltron achieves up to 16.5% higher accuracy than state-of-the-art LLMs that can be executed on a single edge device, satisfying user QoS requirements.","short_abstract":"Large language models (LLMs) are widely used in intelligent services due to their remarkable capability in generative tasks. Typically, LLM-based services process the inference requests of the users in a centralized data center. Unfortunately, such centralized execution has limitations for end-users, such as increased...","url_abs":"https://arxiv.org/abs/2607.07046","url_pdf":"https://arxiv.org/pdf/2607.07046v1","authors":"[\"Chanwoo Cho\",\"Wooseok Kim\",\"Yonglak Son\",\"Young Seo Lee\",\"Young Geun Kim\"]","published":"2026-07-08T06:23:04Z","proceeding":"cs.DC","tasks":"[\"cs.DC\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}