{"ID":2841936,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.11899","arxiv_id":"2511.11899","title":"End to End AI System for Surgical Gesture Sequence Recognition and Clinical Outcome Prediction","abstract":"Fine-grained analysis of intraoperative behavior and its impact on patient outcomes remain a longstanding challenge. We present Frame-to-Outcome (F2O), an end-to-end system that translates tissue dissection videos into gesture sequences and uncovers patterns associated with postoperative outcomes. Leveraging transformer-based spatial and temporal modeling and frame-wise classification, F2O robustly detects consecutive short (~2 seconds) gestures in the nerve-sparing step of robot-assisted radical prostatectomy (AUC: 0.80 frame-level; 0.81 video-level). F2O-derived features (gesture frequency, duration, and transitions) predicted postoperative outcomes with accuracy comparable to human annotations (0.79 vs. 0.75; overlapping 95% CI). Across 25 shared features, effect size directions were concordant with small differences (~ 0.07), and strong correlation (r = 0.96, p \u003c 1e-14). F2O also captured key patterns linked to erectile function recovery, including prolonged tissue peeling and reduced energy use. By enabling automatic interpretable assessment, F2O establishes a foundation for data-driven surgical feedback and prospective clinical decision support.","short_abstract":"Fine-grained analysis of intraoperative behavior and its impact on patient outcomes remain a longstanding challenge. We present Frame-to-Outcome (F2O), an end-to-end system that translates tissue dissection videos into gesture sequences and uncovers patterns associated with postoperative outcomes. Leveraging transforme...","url_abs":"https://arxiv.org/abs/2511.11899","url_pdf":"https://arxiv.org/pdf/2511.11899v1","authors":"[\"Xi Li\",\"Nicholas Matsumoto\",\"Ujjwal Pasupulety\",\"Atharva Deo\",\"Cherine Yang\",\"Jay Moran\",\"Miguel E. Hernandez\",\"Peter Wager\",\"Jasmine Lin\",\"Jeanine Kim\",\"Alvin C. Goh\",\"Christian Wagner\",\"Geoffrey A. Sonn\",\"Andrew J. Hung\"]","published":"2025-11-14T22:02:46Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CV\"]","methods":"[\"Transformer\"]","has_code":false}