{"ID":2858980,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.07453","arxiv_id":"2510.07453","title":"Meaningful Pose-Based Sign Language Evaluation","abstract":"We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through automatic meta-evaluation of sign-level retrieval and a human correlation study of text-to-pose translation across different sign languages. Our findings and the open-source pose-evaluation toolkit provide a practical and reproducible way of developing and evaluating sign language translation or generation systems.","short_abstract":"We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through automatic meta-evaluation of...","url_abs":"https://arxiv.org/abs/2510.07453","url_pdf":"https://arxiv.org/pdf/2510.07453v1","authors":"[\"Zifan Jiang\",\"Colin Leong\",\"Amit Moryossef\",\"Anne Göhring\",\"Annette Rios\",\"Oliver Cory\",\"Maksym Ivashechkin\",\"Neha Tarigopula\",\"Biao Zhang\",\"Rico Sennrich\",\"Sarah Ebling\"]","published":"2025-10-08T19:00:24Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[]","has_code":false}