{"ID":2830093,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.10275","arxiv_id":"2512.10275","title":"Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation","abstract":"Adversarial distillation in the standard min-max adversarial training framework aims to transfer adversarial robustness from a large, robust teacher network to a compact student. However, existing work often neglects to incorporate state-of-the-art robust teachers. Through extensive analysis, we find that stronger teachers do not necessarily yield more robust students-a phenomenon known as robust saturation. While typically attributed to capacity gaps, we show that such explanations are incomplete. Instead, we identify adversarial transferability-the fraction of student-crafted adversarial examples that remain effective against the teacher-as a key factor in successful robustness transfer. Based on this insight, we propose Sample-wise Adaptive Adversarial Distillation (SAAD), which reweights training examples by their measured transferability without incurring additional computational cost. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that SAAD consistently improves AutoAttack robustness over prior methods. Our code is available at https://github.com/HongsinLee/saad.","short_abstract":"Adversarial distillation in the standard min-max adversarial training framework aims to transfer adversarial robustness from a large, robust teacher network to a compact student. However, existing work often neglects to incorporate state-of-the-art robust teachers. Through extensive analysis, we find that stronger teac...","url_abs":"https://arxiv.org/abs/2512.10275","url_pdf":"https://arxiv.org/pdf/2512.10275v2","authors":"[\"Hongsin Lee\",\"Hye Won Chung\"]","published":"2025-12-11T04:31:04Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":605997,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2830093,"paper_url":"https://arxiv.org/abs/2512.10275","paper_title":"Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation","repo_url":"https://github.com/HongsinLee/saad","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}