{"ID":2870344,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.12920","arxiv_id":"2509.12920","title":"Soft Gradient Boosting with Learnable Feature Transforms for Sequential Regression","abstract":"We propose a soft gradient boosting framework for sequential regression that embeds a learnable linear feature transform within the boosting procedure. At each boosting iteration, we train a soft decision tree and learn a linear input feature transform Q together. This approach is particularly advantageous in high-dimensional, data-scarce scenarios, as it discovers the most relevant input representations while boosting. We demonstrate, using both synthetic and real-world datasets, that our method effectively and efficiently increases the performance by an end-to-end optimization of feature selection/transform and boosting while avoiding overfitting. We also extend our algorithm to differentiable non-linear transforms if overfitting is not a problem. To support reproducibility and future work, we share our code publicly.","short_abstract":"We propose a soft gradient boosting framework for sequential regression that embeds a learnable linear feature transform within the boosting procedure. At each boosting iteration, we train a soft decision tree and learn a linear input feature transform Q together. This approach is particularly advantageous in high-dime...","url_abs":"https://arxiv.org/abs/2509.12920","url_pdf":"https://arxiv.org/pdf/2509.12920v1","authors":"[\"Huseyin Karaca\",\"Suleyman Serdar Kozat\"]","published":"2025-09-16T10:14:47Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"eess.SP\"]","methods":"[]","has_code":false}