{"ID":5551674,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-04T13:20:54.626185648Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.00916","arxiv_id":"2607.00916","title":"Condensing Large-Scale Datasets Directly with Minimal Information Loss","abstract":"Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scalability to large-scale datasets, these methods suffer from prohibitive computational overhead and poor cross-architecture generalization. In this paper, we reveal the root cause of these bottlenecks: the implicit dual-compression process, from data to model and back to images, inherently induces severe information loss. Crucially, we empirically and theoretically demonstrate that this loss creates a distribution shift that fundamentally compromises the widely adopted RELABEL strategy, transforming the pre-trained model into an unreliable labeler that yields sub-optimal labels. To overcome these critical flaws, we propose CIM, a novel, metric-driven framework that abandons the flawed dual-compression paradigm. Instead, CIM explicitly quantifies and minimizes the information gap between the original and synthetic datasets. By directly aligning the data distributions, our approach ensures high-fidelity information condensation and inherently satisfies the prerequisites for effective relabeling. Extensive experiments demonstrate that CIM establishes a new state-of-the-art. Notably, it distills ImageNet-1K at an IPC=10 in merely 80 minutes on a single RTX-4090 GPU, achieving an unprecedented 48.7% Top-1 accuracy on ResNet-18 and significantly outperforming previous SOTA approaches, such as NRR-DD and DELT, by 2.6% and 2.9%, respectively. Our code is available at https://github.com/LINs-lab/CIM.","short_abstract":"Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scalability to large-scale datasets, these methods suffer from prohibitive computational overhead and poor cross-architecture generalization. In...","url_abs":"https://arxiv.org/abs/2607.00916","url_pdf":"https://arxiv.org/pdf/2607.00916v1","authors":"[\"Xinyi Shang\",\"Peng Sun\",\"Bei Shi\",\"Zixuan Wang\",\"Tao Lin\"]","published":"2026-07-01T13:21:44Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":613833,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-02T01:54:51.863792489Z","DeletedAt":null,"paper_id":5551674,"paper_url":"https://arxiv.org/abs/2607.00916","paper_title":"Condensing Large-Scale Datasets Directly with Minimal Information Loss","repo_url":"https://github.com/LINs-lab/CIM","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}