{"ID":5438623,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-03T04:20:05.427450767Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31127","arxiv_id":"2606.31127","title":"SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos","abstract":"To enable personalized, real-time coaching using Augmented Reality glasses or fixed camera setups in domains such as sports, cooking, or music, a system must understand not just what a person does, but how well they execute an activity. In an ego-exo video setting, this requires simultaneously detecting individual skilled actions and classifying each as correct or needing improvement, which Ego-Exo4D's proficiency demonstration benchmark formalized. We first adapt seven state-of-the-art temporal action detection architectures to this task, extend the evaluation protocol to disentangle detection from grading, and show that existing methods grade near-randomly. We then introduce SkillSpotter, a pose-aware multi-view architecture that jointly detects and grades skilled actions through three task-specific modules: (1) adaptive temporal suppression to handle the varying density of skilled actions across diverse activities, (2) gated 3D body pose fusion to leverage body kinematics as a complementary signal to visual features, and (3) bidirectional cross-view attention to combine ego and exo views effectively. SkillSpotter improves class-specific mAP from 12.40 to 21.82 (+76%) and balanced accuracy from 55.99% to 60.40% over the best baseline. SkillSpotter's modules transfer to other temporal action detection models with consistent gains, and our method generalizes beyond Ego-Exo4D to HoloAssist. Code: https://github.com/eth-siplab/SkillSpotter","short_abstract":"To enable personalized, real-time coaching using Augmented Reality glasses or fixed camera setups in domains such as sports, cooking, or music, a system must understand not just what a person does, but how well they execute an activity. In an ego-exo video setting, this requires simultaneously detecting individual skil...","url_abs":"https://arxiv.org/abs/2606.31127","url_pdf":"https://arxiv.org/pdf/2606.31127v1","authors":"[\"Björn Braun\",\"Christian Holz\"]","published":"2026-06-30T04:43:03Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\",\"cs.LG\"]","methods":"[]","has_code":false,"code_links":[{"ID":613764,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-01T01:17:58.482524686Z","DeletedAt":null,"paper_id":5438623,"paper_url":"https://arxiv.org/abs/2606.31127","paper_title":"SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos","repo_url":"https://github.com/eth-siplab/SkillSpotter","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}