{"ID":5937199,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-09T08:41:17.711438627Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04809","arxiv_id":"2607.04809","title":"Context-Constrained Transfer Learning for Tabular Foundation Models via Data Distillation","abstract":"Tabular Foundation Models (TFMs) have demonstrated strong empirical performance as black-box inference engines through in-context learning. However, their use in transfer learning is limited by two obstacles: strict context-size constraints and sensitivity to distribution shifts between source and target tasks. Directly pooling heterogeneous source data can therefore lead to negative transfer. To address these challenges, we propose Context-Constrained Transfer Learning via ANchoring and DIstillation (TL-ANDI), a posterior-aware distillation framework for TFMs. TL-ANDI constructs a compact source context by solving a budget-constrained optimal transport problem whose cost jointly measures target covariate coverage and posterior compatibility. The selected anchor samples are then equipped with locally distilled labels and combined with a residual calibration step using target data.","short_abstract":"Tabular Foundation Models (TFMs) have demonstrated strong empirical performance as black-box inference engines through in-context learning. However, their use in transfer learning is limited by two obstacles: strict context-size constraints and sensitivity to distribution shifts between source and target tasks. Directl...","url_abs":"https://arxiv.org/abs/2607.04809","url_pdf":"https://arxiv.org/pdf/2607.04809v1","authors":"[\"Yijun Lin\",\"Sai Li\"]","published":"2026-07-06T08:46:47Z","proceeding":"stat.ML","tasks":"[\"stat.ML\",\"cs.LG\"]","methods":"[]","has_code":false}