{"ID":2859075,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.07621","arxiv_id":"2510.07621","title":"Retentive Relevance: Capturing Long-Term User Value in Recommendation Systems","abstract":"Recommendation systems have traditionally relied on short-term engagement signals, such as clicks and likes, to personalize content. However, these signals are often noisy, sparse, and insufficient for capturing long-term user satisfaction and retention. We introduce Retentive Relevance, a novel content-level survey-based feedback measure that directly assesses users' intent to return to the platform for similar content. Unlike other survey measures that focus on immediate satisfaction, Retentive Relevance targets forward-looking behavioral intentions, capturing longer term user intentions and providing a stronger predictor of retention. We validate Retentive Relevance using psychometric methods, establishing its convergent, discriminant, and behavioral validity. Through large-scale offline modeling, we show that Retentive Relevance significantly outperforms both engagement signals and other survey measures in predicting next-day retention, especially for users with limited historical engagement. We develop a production-ready proxy model that integrates Retentive Relevance into the final stage of a multi-stage ranking system on a social media platform. Calibrated score adjustments based on this model yield substantial improvements in engagement, and retention, while reducing exposure to low-quality content, as demonstrated by large-scale A/B experiments. This work provides the first empirically validated framework linking content-level user perceptions to retention outcomes in production systems. We offer a scalable, user-centered solution that advances both platform growth and user experience. Our work has broad implications for responsible AI development.","short_abstract":"Recommendation systems have traditionally relied on short-term engagement signals, such as clicks and likes, to personalize content. However, these signals are often noisy, sparse, and insufficient for capturing long-term user satisfaction and retention. We introduce Retentive Relevance, a novel content-level survey-ba...","url_abs":"https://arxiv.org/abs/2510.07621","url_pdf":"https://arxiv.org/pdf/2510.07621v1","authors":"[\"Saeideh Bakhshi\",\"Phuong Mai Nguyen\",\"Robert Schiller\",\"Tiantian Xu\",\"Pawan Kodandapani\",\"Andrew Levine\",\"Cayman Simpson\",\"Qifan Wang\"]","published":"2025-10-08T23:38:57Z","proceeding":"cs.IR","tasks":"[\"cs.IR\",\"cs.AI\",\"cs.HC\",\"cs.LG\"]","methods":"[]","has_code":false}