{"ID":2838813,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.17805","arxiv_id":"2511.17805","title":"A Stitch in Time: Learning Procedural Workflow via Self-Supervised Plackett-Luce Ranking","abstract":"Procedural activities, ranging from routine cooking to complex surgical operations, are highly structured sequences of actions performed in a specific temporal order. Despite the success of current self-supervised learning (SSL) methods on static images and short clips, these models often overlook the underlying sequential structure of such activities. We expose this lack of procedural awareness with a motivating experiment: models pretrained on forward and time-reversed sequences produce highly similar features, confirming that their representations are blind to the underlying procedural order. To address this shortcoming, we propose PL-Stitch, a self-supervised framework that harnesses the inherent temporal order of video frames as a powerful supervisory signal. Our approach integrates two novel probabilistic objectives based on the Plackett-Luce (PL) model. The primary PL objective trains the model to sort sampled frames chronologically, compelling it to learn the global workflow progression. The secondary objective, a spatio-temporal jigsaw loss, complements the learning by capturing fine-grained, cross-frame object correspondences. Our approach consistently achieves superior performance across five surgical and cooking benchmarks. Specifically, PL-Stitch yields significant gains in surgical phase recognition (e.g., +11.4 pp in k-NN accuracy on Cholec80) and cooking action segmentation (e.g., +5.7 pp in linear probing accuracy on Breakfast), demonstrating its effectiveness for procedural video representation learning. Code and models are available at https://github.com/visurg-ai/PL-Stitch.","short_abstract":"Procedural activities, ranging from routine cooking to complex surgical operations, are highly structured sequences of actions performed in a specific temporal order. Despite the success of current self-supervised learning (SSL) methods on static images and short clips, these models often overlook the underlying sequen...","url_abs":"https://arxiv.org/abs/2511.17805","url_pdf":"https://arxiv.org/pdf/2511.17805v2","authors":"[\"Chengan Che\",\"Chao Wang\",\"Xinyue Chen\",\"Sophia Tsoka\",\"Luis C. Garcia-Peraza-Herrera\"]","published":"2025-11-21T21:59:22Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[]","has_code":false,"code_links":[{"ID":606807,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2838813,"paper_url":"https://arxiv.org/abs/2511.17805","paper_title":"A Stitch in Time: Learning Procedural Workflow via Self-Supervised Plackett-Luce Ranking","repo_url":"https://github.com/visurg-ai/PL-Stitch","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}