{"ID":2873734,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.07022","arxiv_id":"2509.07022","title":"Preventing Another Tessa: Modular Safety Middleware For Health-Adjacent AI Assistants","abstract":"In 2023, the National Eating Disorders Association's (NEDA) chatbot Tessa was suspended after providing harmful weight-loss advice to vulnerable users-an avoidable failure that underscores the risks of unsafe AI in healthcare contexts. This paper examines Tessa as a case study in absent safety engineering and demonstrates how a lightweight, modular safeguard could have prevented the incident. We propose a hybrid safety middleware that combines deterministic lexical gates with an in-line large language model (LLM) policy filter, enforcing fail-closed verdicts and escalation pathways within a single model call. Using synthetic evaluations, we show that this design achieves perfect interception of unsafe prompts at baseline cost and latency, outperforming traditional multi-stage pipelines. Beyond technical remedies, we map Tessa's failure patterns to established frameworks (OWASP LLM Top10, NIST SP 800-53), connecting practical safeguards to actionable governance controls. The results highlight that robust, auditable safety in health-adjacent AI does not require heavyweight infrastructure: explicit, testable checks at the last mile are sufficient to prevent \"another Tessa\", while governance and escalation ensure sustainability in real-world deployment.","short_abstract":"In 2023, the National Eating Disorders Association's (NEDA) chatbot Tessa was suspended after providing harmful weight-loss advice to vulnerable users-an avoidable failure that underscores the risks of unsafe AI in healthcare contexts. This paper examines Tessa as a case study in absent safety engineering and demonstra...","url_abs":"https://arxiv.org/abs/2509.07022","url_pdf":"https://arxiv.org/pdf/2509.07022v1","authors":"[\"Pavan Reddy\",\"Nithin Reddy\"]","published":"2025-09-07T08:43:39Z","proceeding":"cs.CY","tasks":"[\"cs.CY\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}