{"ID":2871113,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.12384","arxiv_id":"2509.12384","title":"Exploring Distributed Vector Databases Performance on HPC Platforms: A Study with Qdrant","abstract":"Vector databases have rapidly grown in popularity, enabling efficient similarity search over data such as text, images, and video. They now play a central role in modern AI workflows, aiding large language models by grounding model outputs in external literature through retrieval-augmented generation. Despite their importance, little is known about the performance characteristics of vector databases in high-performance computing (HPC) systems that drive large-scale science. This work presents an empirical study of distributed vector database performance on the Polaris supercomputer in the Argonne Leadership Computing Facility. We construct a realistic biological-text workload from BV-BRC and generate embeddings from the peS2o corpus using Qwen3-Embedding-4B. We select Qdrant to evaluate insertion, index construction, and query latency with up to 32 workers. Informed by practical lessons from our experience, this work takes a first step toward characterizing vector database performance on HPC platforms to guide future research and optimization.","short_abstract":"Vector databases have rapidly grown in popularity, enabling efficient similarity search over data such as text, images, and video. They now play a central role in modern AI workflows, aiding large language models by grounding model outputs in external literature through retrieval-augmented generation. Despite their imp...","url_abs":"https://arxiv.org/abs/2509.12384","url_pdf":"https://arxiv.org/pdf/2509.12384v2","authors":"[\"Seth Ockerman\",\"Amal Gueroudji\",\"Song Young Oh\",\"Robert Underwood\",\"Nicholas Chia\",\"Kyle Chard\",\"Robert Ross\",\"Shivaram Venkataraman\"]","published":"2025-09-15T19:25:49Z","proceeding":"cs.DC","tasks":"[\"cs.DC\",\"cs.DB\"]","methods":"[\"RAG\",\"Language Model\"]","has_code":false}