{"ID":5551831,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-04T08:00:54.702513071Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.00580","arxiv_id":"2607.00580","title":"Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers","abstract":"Vision Transformers (ViTs) commonly rely on injected positional mechanisms to address self-attention's permutation invariance. Motivated by the spatial regularities of natural images, we ask whether spatial organization can be induced from data rather than explicitly injected. Under controlled, matched from-scratch training, we propose Active Spatial Guidance (Guidance), a training-only objective that disables positional injection and applies an auxiliary 2D coordinate-regression loss to the final-layer patch tokens. The guidance head is used only during training and removed for inference; the deployed model consists of a positional-injection-free ViT encoder and the task-specific prediction module. Using DINOv3 ViT backbones, Guidance consistently improves performance on ImageNet-100 classification, ADE20K semantic segmentation, and Hypersim monocular depth estimation, outperforming strong injected baselines such as learned absolute positional embeddings and rotary positional embeddings under identical training protocols. On ImageNet-100, broader comparisons against representative injected positional designs further support Guidance's effectiveness. Guidance also improves robustness under resolution transfer, and multi-resolution training further strengthens accuracy across input sizes. Overall, our results suggest that spatial inductive bias in ViTs need not be architecturally injected, but can be shaped through training-time supervision. The code used for training and evaluation is publicly available in https://github.com/cloudlc/asg.","short_abstract":"Vision Transformers (ViTs) commonly rely on injected positional mechanisms to address self-attention's permutation invariance. Motivated by the spatial regularities of natural images, we ask whether spatial organization can be induced from data rather than explicitly injected. Under controlled, matched from-scratch tra...","url_abs":"https://arxiv.org/abs/2607.00580","url_pdf":"https://arxiv.org/pdf/2607.00580v1","authors":"[\"Cong Liu\",\"Xiaofang Li\",\"Simon X. Yang\"]","published":"2026-07-01T08:02:46Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Vision Transformer\",\"Transformer\",\"Generative Adversarial Network\"]","has_code":false,"code_links":[{"ID":613847,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-02T01:54:51.863792489Z","DeletedAt":null,"paper_id":5551831,"paper_url":"https://arxiv.org/abs/2607.00580","paper_title":"Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers","repo_url":"https://github.com/cloudlc/asg","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}