{"ID":3006011,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-04T17:52:58.968687531Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.02737","arxiv_id":"2606.02737","title":"Attention Calibration for Position-Fair Dense Information Retrieval","abstract":"Dense retrieval models exhibit positional bias: retrieval effectiveness degrades when relevant information appears later in a passage (Zeng et al., 2025). We ask whether this bias can be reduced at inference time, without retraining and without sacrificing overall retrieval effectiveness. To this end, we adapt inference-time attention calibration (Schuhmacher et al., 2026) to downstream retrieval and extend it with a strength coefficient lambda that interpolates between the original and fully calibrated attention distributions. Across three embedding models on SQuAD-PosQ and FineWeb-PosQ, we examine how basket size, calibrated layer set, and strength affect the trade-off between positional fairness and retrieval effectiveness, finding that partial calibration frequently outperforms full calibration. A single configuration (B=128, lambda=0.5, 50% layer depth) improves the harmonic mean of nDCG@10 across positional groups on FineWeb-PosQ for all three models without per-model tuning, and applies to both \u003cs\u003e-pooled and last-token-pooled architectures. This default configuration transfers without modification to PosIR, which spans 10 languages and 31 domains, reducing the Position Sensitivity Index in all 16 length-quartile x model x retrieval-setting combinations, while preserving or improving aggregate nDCG@10. We release our extended codebase at https://github.com/impresso/fair-sentence-transformers","short_abstract":"Dense retrieval models exhibit positional bias: retrieval effectiveness degrades when relevant information appears later in a passage (Zeng et al., 2025). We ask whether this bias can be reduced at inference time, without retraining and without sacrificing overall retrieval effectiveness. To this end, we adapt inferenc...","url_abs":"https://arxiv.org/abs/2606.02737","url_pdf":"https://arxiv.org/pdf/2606.02737v1","authors":"[\"Andrianos Michail\",\"Elias Schuhmacher\",\"Juri Opitz\",\"Simon Clematide\",\"Rico Sennrich\"]","published":"2026-06-01T18:04:26Z","proceeding":"cs.IR","tasks":"[\"cs.IR\",\"cs.AI\",\"cs.CL\"]","methods":"[\"Transformer\"]","has_code":false,"code_links":[{"ID":612743,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-03T03:09:48.883664427Z","DeletedAt":null,"paper_id":3006011,"paper_url":"https://arxiv.org/abs/2606.02737","paper_title":"Attention Calibration for Position-Fair Dense Information Retrieval","repo_url":"https://github.com/impresso/fair-sentence-transformers","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}