{"ID":6023398,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-10T06:38:11.380144103Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.05891","arxiv_id":"2607.05891","title":"Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation","abstract":"Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training. The problem remains challenging even in the few-shot knowledge distillation (KD) setup, where a full-scale pre-trained teacher informs the student network. Typical sample selection strategies often struggle to surpass the random selection baseline. In this paper, we showcase few-medoids, an embarrassingly simple coreset selection strategy that chooses the samples closest to the centroid (average image) of each class. We present extensive KD experiments on four datasets, covering a wide range of image classification problems, and three teacher-student model pairs, comprising both convolutional and transformer networks. Although the proposed method is embarrassingly simple, our empirical results indicate that few-medoids is able to consistently surpass the random selection baseline, as well as the other coreset selection strategies. We therefore consider that few-medoids can be used as a drop-in replacement for commonly-used baselines (e.g. herding or k-center Greedy), in future research on coreset selection. To reproduce the reported results, we publicly release our code at https://github.com/CemilAndreiDilmac/Few-Shot-KD-Coreset.","short_abstract":"Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training. The problem remains challenging even in the few-shot knowledge distillation (KD) setup, where a full-scale pre-trained teacher informs the student network. Typical sample selection strategies o...","url_abs":"https://arxiv.org/abs/2607.05891","url_pdf":"https://arxiv.org/pdf/2607.05891v1","authors":"[\"Cemil-Andrei Dilmac\",\"Florinel-Alin Croitoru\",\"Radu Tudor Ionescu\"]","published":"2026-07-07T06:40:58Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\",\"cs.LG\"]","methods":"[\"Transformer\"]","has_code":false,"code_links":[{"ID":614016,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-08T01:00:23.257252134Z","DeletedAt":null,"paper_id":6023398,"paper_url":"https://arxiv.org/abs/2607.05891","paper_title":"Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation","repo_url":"https://github.com/CemilAndreiDilmac/Few-Shot-KD-Coreset","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}