{"ID":2860337,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.04241","arxiv_id":"2510.04241","title":"Diffusion-Assisted Distillation for Self-Supervised Graph Representation Learning with MLPs","abstract":"For large-scale applications, there is growing interest in replacing Graph Neural Networks (GNNs) with lightweight Multi-Layer Perceptrons (MLPs) via knowledge distillation. However, distilling GNNs for self-supervised graph representation learning into MLPs is more challenging. This is because the performance of self-supervised learning is more related to the model's inductive bias than supervised learning. This motivates us to design a new distillation method to bridge a huge capacity gap between GNNs and MLPs in self-supervised graph representation learning. In this paper, we propose \\textbf{D}iffusion-\\textbf{A}ssisted \\textbf{D}istillation for \\textbf{S}elf-supervised \\textbf{G}raph representation learning with \\textbf{M}LPs (DAD-SGM). The proposed method employs a denoising diffusion model as a teacher assistant to better distill the knowledge from the teacher GNN into the student MLP. This approach enhances the generalizability and robustness of MLPs in self-supervised graph representation learning. Extensive experiments demonstrate that DAD-SGM effectively distills the knowledge of self-supervised GNNs compared to state-of-the-art GNN-to-MLP distillation methods. Our implementation is available at https://github.com/SeongJinAhn/DAD-SGM.","short_abstract":"For large-scale applications, there is growing interest in replacing Graph Neural Networks (GNNs) with lightweight Multi-Layer Perceptrons (MLPs) via knowledge distillation. However, distilling GNNs for self-supervised graph representation learning into MLPs is more challenging. This is because the performance of self-...","url_abs":"https://arxiv.org/abs/2510.04241","url_pdf":"https://arxiv.org/pdf/2510.04241v1","authors":"[\"Seong Jin Ahn\",\"Myoung-Ho Kim\"]","published":"2025-10-05T15:11:55Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Graph Neural Network\",\"Diffusion Model\"]","has_code":false,"code_links":[{"ID":608717,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2860337,"paper_url":"https://arxiv.org/abs/2510.04241","paper_title":"Diffusion-Assisted Distillation for Self-Supervised Graph Representation Learning with MLPs","repo_url":"https://github.com/SeongJinAhn/DAD-SGM","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}