{"ID":2880451,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.13415","arxiv_id":"2508.13415","title":"MAVIS: Multi-Objective Alignment via Inference-Time Value-Guided Selection","abstract":"Large Language Models (LLMs) are increasingly deployed across diverse applications that demand balancing multiple, often conflicting, objectives -- such as helpfulness, harmlessness, or humor. Many traditional methods for aligning outputs to user-specific preferences require fine-tuning models for each objective or for specific preference configurations, which is computationally expensive and inflexible. We introduce \\textbf{MAVIS} -- \\textit{Multi-Objective Alignment via Inference-Time Value-Guided Selection} -- a lightweight inference-time alignment framework that enables dynamic control over LLM behavior without modifying the base model's weights. MAVIS trains a set of small value models, each corresponding to a distinct objective. At inference time, these value models are combined using user-specified weights to produce a tilting function that adjusts the base model's output distribution toward desired trade-offs. The value models are trained using a simple iterative algorithm that enables monotonic improvement of the KL-regularized policy. We show empirically that MAVIS achieves a superior pareto front compared to baselines which fine-tune per-objective models and combine them post hoc or train a single preference-conditioned value model for guidance. Our code is available at https://github.com/5-Jeremy/MAVIS/tree/main.","short_abstract":"Large Language Models (LLMs) are increasingly deployed across diverse applications that demand balancing multiple, often conflicting, objectives -- such as helpfulness, harmlessness, or humor. Many traditional methods for aligning outputs to user-specific preferences require fine-tuning models for each objective or for...","url_abs":"https://arxiv.org/abs/2508.13415","url_pdf":"https://arxiv.org/pdf/2508.13415v3","authors":"[\"Jeremy Carleton\",\"Debajoy Mukherjee\",\"Srinivas Shakkottai\",\"Dileep Kalathil\"]","published":"2025-08-19T00:26:07Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":610677,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2880451,"paper_url":"https://arxiv.org/abs/2508.13415","paper_title":"MAVIS: Multi-Objective Alignment via Inference-Time Value-Guided Selection","repo_url":"https://github.com/5-Jeremy/MAVIS","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}