{"ID":2923647,"CreatedAt":"2026-06-02T04:05:25.881865328Z","UpdatedAt":"2026-06-04T13:12:39.622923895Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.02282","arxiv_id":"2606.02282","title":"POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems","abstract":"Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation. Existing evaluation paradigms share a common flaw: centralised judgment creates single points of failure and demands domain-specific expertise. Here we present POIROT, a protocol that repurposes a system's own agents as its diagnostic layer, leveraging the epistemic diversity already present in the architecture. Across evaluated settings, POIROT outperforms single-LLM evaluator baselines, with gains that scale with problem complexity (OR = 1.60, $p = 0.008$), agent count, and fault dimensionality, persisting under compound fault conditions. These results demonstrate that safety oversight need not be externalised: the agents executing a role carry sufficient collective intelligence to audit it. We release POIROT as an open-source library alongside BLAME, a benchmark for fault attribution in safety-critical multi-agent systems.","short_abstract":"Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation. Existing evaluation pa...","url_abs":"https://arxiv.org/abs/2606.02282","url_pdf":"https://arxiv.org/pdf/2606.02282v1","authors":"[\"Iñaki Dellibarda Varela\",\"R. Sendra-Arranz\",\"Pablo Romero-Sorozabal\",\"J. M. Valverde-García\",\"Annemarie F. Laudanski\",\"Álvaro Gutiérrez\",\"Eduardo Rocon\",\"Manuel Cebrian\"]","published":"2026-06-01T14:05:35Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}