{"ID":6497811,"CreatedAt":"2026-07-13T01:19:40.13847098Z","UpdatedAt":"2026-07-14T01:36:59.12045529Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.09126","arxiv_id":"2607.09126","title":"VTaMo: Video-Text Alignment Model for Sign Language Translation","abstract":"Sign language translation (SLT) converts continuous sign videos into spoken language text. Gloss-free approaches leverage pre-trained visual encoders and language models but rely on implicit cross-modal alignment from translation supervision alone. We present VTaMo, a framework that introduces explicit multi-granularity alignment at three levels: (1) local alignment via entropy-regularized optimal transport with a learnable null token for fine-grained frame-to-token correspondences; (2) global alignment via a learnable orthogonal transformation that calibrates embedding space geometry through Earth Mover's Distance; and (3) position-aligned contrastive learning for discriminative token-level representations. Experiments on Phoenix-2014T, CSL-Daily, How2Sign, and OpenASL demonstrate consistent state-of-the-art performance, with ablations confirming the complementary contributions of each component. Code is available at https://github.com/junyi2005/vtamo.","short_abstract":"Sign language translation (SLT) converts continuous sign videos into spoken language text. Gloss-free approaches leverage pre-trained visual encoders and language models but rely on implicit cross-modal alignment from translation supervision alone. We present VTaMo, a framework that introduces explicit multi-granularit...","url_abs":"https://arxiv.org/abs/2607.09126","url_pdf":"https://arxiv.org/pdf/2607.09126v1","authors":"[\"Junyi Hu\",\"Zhewen He\",\"Haomian Huang\",\"Aoxiang Yang\",\"Yi Fang\"]","published":"2026-07-10T06:30:50Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.CL\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":614112,"CreatedAt":"2026-07-13T01:19:40.13847098Z","UpdatedAt":"2026-07-13T01:19:40.13847098Z","DeletedAt":null,"paper_id":6497811,"paper_url":"https://arxiv.org/abs/2607.09126","paper_title":"VTaMo: Video-Text Alignment Model for Sign Language Translation","repo_url":"https://github.com/junyi2005/vtamo","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}