{"ID":2891830,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.21132","arxiv_id":"2507.21132","title":"Can You Trust an LLM with Your Life-Changing Decision? An Investigation into AI High-Stakes Responses","abstract":"Large Language Models (LLMs) are increasingly consulted for high-stakes life advice, yet they lack standard safeguards against providing confident but misguided responses. This creates risks of sycophancy and over-confidence. This paper investigates these failure modes through three experiments: (1) a multiple-choice evaluation to measure model stability against user pressure; (2) a free-response analysis using a novel safety typology and an LLM Judge; and (3) a mechanistic interpretability experiment to steer model behavior by manipulating a \"high-stakes\" activation vector. Our results show that while some models exhibit sycophancy, others like o4-mini remain robust. Top-performing models achieve high safety scores by frequently asking clarifying questions, a key feature of a safe, inquisitive approach, rather than issuing prescriptive advice. Furthermore, we demonstrate that a model's cautiousness can be directly controlled via activation steering, suggesting a new path for safety alignment. These findings underscore the need for nuanced, multi-faceted benchmarks to ensure LLMs can be trusted with life-changing decisions.","short_abstract":"Large Language Models (LLMs) are increasingly consulted for high-stakes life advice, yet they lack standard safeguards against providing confident but misguided responses. This creates risks of sycophancy and over-confidence. This paper investigates these failure modes through three experiments: (1) a multiple-choice e...","url_abs":"https://arxiv.org/abs/2507.21132","url_pdf":"https://arxiv.org/pdf/2507.21132v1","authors":"[\"Joshua Adrian Cahyono\",\"Saran Subramanian\"]","published":"2025-07-22T14:11:13Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CY\",\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}