{"ID":5443844,"CreatedAt":"2026-07-01T02:07:11.383974684Z","UpdatedAt":"2026-07-03T16:03:33.891311754Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31903","arxiv_id":"2606.31903","title":"Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference","abstract":"Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation. Existing acceleration methods usually remove visual tokens or skip visual-token updates in entire layers, but these coarse strategies may discard fine-grained evidence or suppress useful operators together with redundant ones. In this paper, we study visual-token computation from an answer-observable perspective and find that late visual-token updates can remain large while having little effect on answer-token representations. Motivated by this answer-silent redundancy, we decompose each Transformer layer into attention and FFN operators and show that useful visual computation is often operator-dominant and layer-dependent. We propose an operator-level visual-token skipping framework that preserves the full visual-token sequence while selectively bypassing redundant attention, FFN, or both. Experiments across three MLLM architectures and 10 VQA benchmarks show that our method achieves strong efficiency-accuracy trade-offs, reducing \\textbf{33.7\\%} TFLOPs on Qwen3-VL while retaining \\textbf{99.5\\%} of the vanilla model performance.","short_abstract":"Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation. Existing acceleration methods usually remove visual tokens or skip visual-token updates in entire layers, but these coarse strategies may discard fine-grained evidence or suppress use...","url_abs":"https://arxiv.org/abs/2606.31903","url_pdf":"https://arxiv.org/pdf/2606.31903v1","authors":"[\"Zhaoyang Luo\",\"Runmin Dong\",\"Miao Yang\",\"Fan Wei\",\"Yushan Lai\",\"Bin Luo\",\"Haohuan Fu\"]","published":"2026-06-30T16:08:29Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false}