{"ID":2835667,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.00181","arxiv_id":"2512.00181","title":"Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning","abstract":"Tabular data drive most real-world machine learning applications, yet building general-purpose models for them remains difficult. Mixed numeric and categorical fields, weak feature structure, and limited labeled data make scaling and generalization challenging. To this end, we introduce Orion-Bix, a tabular foundation model that combines biaxial attention with meta-learned in-context reasoning for few-shot tabular learning. Its encoder alternates standard, grouped, hierarchical, and relational attention, fusing their outputs through multi-CLS summarization to capture both local and global dependencies efficiently. A label-aware ICL head adapts on the fly and scales to large label spaces via hierarchical decision routing. Meta-trained on synthetically generated, structurally diverse tables with causal priors, Orion-Bix learns transferable inductive biases across heterogeneous data. Delivered as a scikit-learn compatible foundation model, it outperforms gradient-boosting baselines and remains competitive with state-of-the-art tabular foundation models on public benchmarks, showing that biaxial attention with episodic meta-training enables robust, few-shot-ready tabular learning. The model is publicly available at https://github.com/Lexsi-Labs/Orion-BiX .","short_abstract":"Tabular data drive most real-world machine learning applications, yet building general-purpose models for them remains difficult. Mixed numeric and categorical fields, weak feature structure, and limited labeled data make scaling and generalization challenging. To this end, we introduce Orion-Bix, a tabular foundation...","url_abs":"https://arxiv.org/abs/2512.00181","url_pdf":"https://arxiv.org/pdf/2512.00181v2","authors":"[\"Mohamed Bouadi\",\"Pratinav Seth\",\"Aditya Tanna\",\"Vinay Kumar Sankarapu\"]","published":"2025-11-28T19:42:09Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"stat.ML\"]","methods":"[]","has_code":false,"code_links":[{"ID":606531,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2835667,"paper_url":"https://arxiv.org/abs/2512.00181","paper_title":"Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning","repo_url":"https://github.com/Lexsi-Labs/Orion-BiX","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}