{"ID":2842661,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.21698","arxiv_id":"2511.21698","title":"TIP and Polish: Text-Image-Prototype Guided Multi-Modal Generation via Commonality-Discrepancy Modeling and Refinement","abstract":"Multi-modal generation struggles to ensure thematic coherence and style consistency. Semantically, existing methods suffer from cross-modal mismatch and lack explicit modeling of commonality and discrepancy. Methods that rely on fine-grained training fail to balance semantic precision with writing style consistency. These shortcomings lead to suboptimal generation quality. To tackle these issues, we propose \\textbf{\\textit{TIPPo}}, a simple yet effective framework with explicit input modeling and comprehensive optimization objectives. It extracts the input text and images via multi-modal encoder and adapters, then measures the visual prototype. \\textbf{T}extual, \\textbf{I}mage, and \\textbf{P}rototype signals are then fed to our proposed Dual Alignment Attention and Difference Operator modules before language model decoding. The proposed \\textbf{Po}lishPPO reinforces the style consistency, while the unsupervised contrastive learning during SFT mitigates inter-sample representation collapse. Experimental results demonstrate the promising performance of \\textbf{\\textit{TIPPo}} in automatic evaluation and LLM-based criteria for creativity and semantic consistency.","short_abstract":"Multi-modal generation struggles to ensure thematic coherence and style consistency. Semantically, existing methods suffer from cross-modal mismatch and lack explicit modeling of commonality and discrepancy. Methods that rely on fine-grained training fail to balance semantic precision with writing style consistency. Th...","url_abs":"https://arxiv.org/abs/2511.21698","url_pdf":"https://arxiv.org/pdf/2511.21698v1","authors":"[\"Zhiyong Ma\",\"Jiahao Chen\",\"Qingyuan Chuai\",\"Zhengping Li\"]","published":"2025-11-12T07:16:19Z","proceeding":"cs.MM","tasks":"[\"cs.MM\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}