{"ID":2838182,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.17914","arxiv_id":"2511.17914","title":"Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation","abstract":"Dataset distillation compresses large-scale datasets into compact, highly informative synthetic data, significantly reducing storage and training costs. However, existing research primarily focuses on balanced datasets and struggles to perform under real-world long-tailed distributions. In this work, we emphasize the critical role of soft labels in long-tailed dataset distillation and uncover the underlying mechanisms contributing to performance degradation. Specifically, we derive an imbalance-aware generalization bound for model trained on distilled dataset. We then identify two primary sources of soft-label bias, which originate from the distillation model and the distilled images, through systematic perturbation of the data imbalance levels. To address this, we propose ADSA, an Adaptive Soft-label Alignment module that calibrates the entangled biases. This lightweight module integrates seamlessly into existing distillation pipelines and consistently improves performance. On ImageNet-1k-LT with EDC and IPC=50, ADSA improves tail-class accuracy by up to 11.8% and raises overall accuracy to 41.4%. Extensive experiments demonstrate that ADSA provides a robust and generalizable solution under limited label budgets and across a range of distillation techniques. Code is available at: https://github.com/j-cyoung/ADSA_DD.git.","short_abstract":"Dataset distillation compresses large-scale datasets into compact, highly informative synthetic data, significantly reducing storage and training costs. However, existing research primarily focuses on balanced datasets and struggles to perform under real-world long-tailed distributions. In this work, we emphasize the c...","url_abs":"https://arxiv.org/abs/2511.17914","url_pdf":"https://arxiv.org/pdf/2511.17914v1","authors":"[\"Chenyang Jiang\",\"Hang Zhao\",\"Xinyu Zhang\",\"Zhengcen Li\",\"Qiben Shan\",\"Shaocong Wu\",\"Jingyong Su\"]","published":"2025-11-22T04:37:27Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[]","has_code":false,"code_links":[{"ID":606742,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2838182,"paper_url":"https://arxiv.org/abs/2511.17914","paper_title":"Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation","repo_url":"https://github.com/j-cyoung/ADSA_DD.git","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}