{"ID":2836927,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.20235","arxiv_id":"2511.20235","title":"HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems","abstract":"We propose HHFT (Hierarchical Heterogeneous Feature Transformer), a Transformer-based architecture tailored for industrial CTR prediction. HHFT addresses the limitations of DNN through three key designs: (1) Semantic Feature Partitioning: Grouping heterogeneous features (e.g. user profile, item information, behaviour sequennce) into semantically coherent blocks to preserve domain-specific information; (2) Heterogeneous Transformer Encoder: Adopting block-specific QKV projections and FFNs to avoid semantic confusion between distinct feature types; (3) Hiformer Layer: Capturing high-order interactions across features. Our findings reveal that Transformers significantly outperform DNN baselines, achieving a +0.4% improvement in CTR AUC at scale. We have successfully deployed the model on Taobao's production platform, observing a significant uplift in key business metrics, including a +0.6% increase in Gross Merchandise Value (GMV).","short_abstract":"We propose HHFT (Hierarchical Heterogeneous Feature Transformer), a Transformer-based architecture tailored for industrial CTR prediction. HHFT addresses the limitations of DNN through three key designs: (1) Semantic Feature Partitioning: Grouping heterogeneous features (e.g. user profile, item information, behaviour s...","url_abs":"https://arxiv.org/abs/2511.20235","url_pdf":"https://arxiv.org/pdf/2511.20235v2","authors":"[\"Liren Yu\",\"Wenming Zhang\",\"Silu Zhou\",\"Tao Zhang\",\"Zhixuan Zhang\",\"Dan Ou\"]","published":"2025-11-25T12:07:56Z","proceeding":"cs.IR","tasks":"[\"cs.IR\"]","methods":"[\"Transformer\"]","has_code":false}