{"ID":2852021,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.18263","arxiv_id":"2510.18263","title":"From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generation","abstract":"Subject-driven image generation models face a fundamental trade-off between identity preservation (fidelity) and prompt adherence (editability). While online reinforcement learning (RL), specifically GPRO, offers a promising solution, we find that a naive application of GRPO leads to competitive degradation, as the simple linear aggregation of rewards with static weights causes conflicting gradient signals and a misalignment with the temporal dynamics of the diffusion process. To overcome these limitations, we propose Customized-GRPO, a novel framework featuring two key innovations: (i) Synergy-Aware Reward Shaping (SARS), a non-linear mechanism that explicitly penalizes conflicted reward signals and amplifies synergistic ones, providing a sharper and more decisive gradient. (ii) Time-Aware Dynamic Weighting (TDW), which aligns the optimization pressure with the model's temporal dynamics by prioritizing prompt-following in the early, identity preservation in the later. Extensive experiments demonstrate that our method significantly outperforms naive GRPO baselines, successfully mitigating competitive degradation. Our model achieves a superior balance, generating images that both preserve key identity features and accurately adhere to complex textual prompts.","short_abstract":"Subject-driven image generation models face a fundamental trade-off between identity preservation (fidelity) and prompt adherence (editability). While online reinforcement learning (RL), specifically GPRO, offers a promising solution, we find that a naive application of GRPO leads to competitive degradation, as the sim...","url_abs":"https://arxiv.org/abs/2510.18263","url_pdf":"https://arxiv.org/pdf/2510.18263v2","authors":"[\"Ziwei Huang\",\"Ying Shu\",\"Hao Fang\",\"Quanyu Long\",\"Wenya Wang\",\"Qiushi Guo\",\"Tiezheng Ge\",\"Leilei Gan\"]","published":"2025-10-21T03:32:26Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CV\",\"cs.GR\"]","methods":"[\"Reinforcement Learning\",\"Diffusion Model\"]","has_code":false}