{"ID":2894061,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.12419","arxiv_id":"2507.12419","title":"Mixture of Raytraced Experts","abstract":"We introduce a Mixture of Raytraced Experts, a stacked Mixture of Experts (MoE) architecture which can dynamically select sequences of experts, producing computational graphs of variable width and depth. Existing MoE architectures generally require a fixed amount of computation for a given sample. Our approach, in contrast, yields predictions with increasing accuracy as the computation cycles through the experts' sequence. We train our model by iteratively sampling from a set of candidate experts, unfolding the sequence akin to how Recurrent Neural Networks are trained. Our method does not require load-balancing mechanisms, and preliminary experiments show a reduction in training epochs of 10\\% to 40\\% with a comparable/higher accuracy. These results point to new research directions in the field of MoEs, allowing the design of potentially faster and more expressive models. The code is available at https://github.com/nutig/RayTracing","short_abstract":"We introduce a Mixture of Raytraced Experts, a stacked Mixture of Experts (MoE) architecture which can dynamically select sequences of experts, producing computational graphs of variable width and depth. Existing MoE architectures generally require a fixed amount of computation for a given sample. Our approach, in cont...","url_abs":"https://arxiv.org/abs/2507.12419","url_pdf":"https://arxiv.org/pdf/2507.12419v1","authors":"[\"Andrea Perin\",\"Giacomo Lagomarsini\",\"Claudio Gallicchio\",\"Giuseppe Nuti\"]","published":"2025-07-16T17:08:46Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Mixture of Experts\"]","has_code":false,"code_links":[{"ID":612090,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2894061,"paper_url":"https://arxiv.org/abs/2507.12419","paper_title":"Mixture of Raytraced Experts","repo_url":"https://github.com/nutig/RayTracing","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}