{"ID":2878030,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.18877","arxiv_id":"2508.18877","title":"Optimization of Latent-Space Compression using Game-Theoretic Techniques for Transformer-Based Vector Search","abstract":"Vector similarity search plays a pivotal role in modern information retrieval systems, especially when powered by transformer-based embeddings. However, the scalability and efficiency of such systems are often hindered by the high dimensionality of latent representations. In this paper, we propose a novel game-theoretic framework for optimizing latent-space compression to enhance both the efficiency and semantic utility of vector search. By modeling the compression strategy as a zero-sum game between retrieval accuracy and storage efficiency, we derive a latent transformation that preserves semantic similarity while reducing redundancy. We benchmark our method against FAISS, a widely-used vector search library, and demonstrate that our approach achieves a significantly higher average similarity (0.9981 vs. 0.5517) and utility (0.8873 vs. 0.5194), albeit with a modest increase in query time. This trade-off highlights the practical value of game-theoretic latent compression in high-utility, transformer-based search applications. The proposed system can be seamlessly integrated into existing LLM pipelines to yield more semantically accurate and computationally efficient retrieval.","short_abstract":"Vector similarity search plays a pivotal role in modern information retrieval systems, especially when powered by transformer-based embeddings. However, the scalability and efficiency of such systems are often hindered by the high dimensionality of latent representations. In this paper, we propose a novel game-theoreti...","url_abs":"https://arxiv.org/abs/2508.18877","url_pdf":"https://arxiv.org/pdf/2508.18877v1","authors":"[\"Kushagra Agrawal\",\"Nisharg Nargund\",\"Oishani Banerjee\"]","published":"2025-08-26T09:51:02Z","proceeding":"cs.IR","tasks":"[\"cs.IR\",\"cs.AI\",\"cs.LG\"]","methods":"[\"Transformer\",\"Large Language Model\"]","has_code":false}