{"ID":2877467,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.00083","arxiv_id":"2509.00083","title":"Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models","abstract":"Modern generative models risk overfitting and unintentionally memorizing rare training examples, which can be extracted by adversaries or inflate benchmark performance. We propose Generative Data Cartography (GenDataCarto), a data-centric framework that assigns each pretraining sample a difficulty score (early-epoch loss) and a memorization score (frequency of ``forget events''), then partitions examples into four quadrants to guide targeted pruning and up-/down-weighting. We prove that our memorization score lower-bounds classical influence under smoothness assumptions and that down-weighting high-memorization hotspots provably decreases the generalization gap via uniform stability bounds. Empirically, GenDataCarto reduces synthetic canary extraction success by over 40\\% at just 10\\% data pruning, while increasing validation perplexity by less than 0.5\\%. These results demonstrate that principled data interventions can dramatically mitigate leakage with minimal cost to generative performance.","short_abstract":"Modern generative models risk overfitting and unintentionally memorizing rare training examples, which can be extracted by adversaries or inflate benchmark performance. We propose Generative Data Cartography (GenDataCarto), a data-centric framework that assigns each pretraining sample a difficulty score (early-epoch lo...","url_abs":"https://arxiv.org/abs/2509.00083","url_pdf":"https://arxiv.org/pdf/2509.00083v1","authors":"[\"Laksh Patel\",\"Neel Shanbhag\"]","published":"2025-08-27T05:11:06Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"cs.CL\"]","methods":"[]","has_code":false}