{"ID":2876099,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.01804","arxiv_id":"2509.01804","title":"Mixture of Balanced Information Bottlenecks for Long-Tailed Visual Recognition","abstract":"Deep neural networks (DNNs) have achieved significant success in various applications with large-scale and balanced data. However, data in real-world visual recognition are usually long-tailed, bringing challenges to efficient training and deployment of DNNs. Information bottleneck (IB) is an elegant approach for representation learning. In this paper, we propose a balanced information bottleneck (BIB) approach, in which loss function re-balancing and self-distillation techniques are integrated into the original IB network. BIB is thus capable of learning a sufficient representation with essential label-related information fully preserved for long-tailed visual recognition. To further enhance the representation learning capability, we also propose a novel structure of mixture of multiple balanced information bottlenecks (MBIB), where different BIBs are responsible for combining knowledge from different network layers. MBIB facilitates an end-to-end learning strategy that trains representation and classification simultaneously from an information theory perspective. We conduct experiments on commonly used long-tailed datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018. Both BIB and MBIB reach state-of-the-art performance for long-tailed visual recognition.","short_abstract":"Deep neural networks (DNNs) have achieved significant success in various applications with large-scale and balanced data. However, data in real-world visual recognition are usually long-tailed, bringing challenges to efficient training and deployment of DNNs. Information bottleneck (IB) is an elegant approach for repre...","url_abs":"https://arxiv.org/abs/2509.01804","url_pdf":"https://arxiv.org/pdf/2509.01804v1","authors":"[\"Yifan Lan\",\"Xin Cai\",\"Jun Cheng\",\"Shan Tan\"]","published":"2025-09-01T22:14:12Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.IT\"]","methods":"[\"Generative Adversarial Network\"]","has_code":false}