{"ID":2865751,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.20734","arxiv_id":"2509.20734","title":"Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction","abstract":"Unsupervised neural grammar induction aims to learn interpretable hierarchical structures from language data. However, existing models face an expressiveness bottleneck, often resulting in unnecessarily large yet underperforming grammars. We identify a core issue, $\\textit{probability distribution collapse}$, as the underlying cause of this limitation. We analyze when and how the collapse emerges across key components of neural parameterization and introduce a targeted solution, $\\textit{collapse-relaxing neural parameterization}$, to mitigate it. Our approach substantially improves parsing performance while enabling the use of significantly more compact grammars across a wide range of languages, as demonstrated through extensive empirical analysis.","short_abstract":"Unsupervised neural grammar induction aims to learn interpretable hierarchical structures from language data. However, existing models face an expressiveness bottleneck, often resulting in unnecessarily large yet underperforming grammars. We identify a core issue, $\\textit{probability distribution collapse}$, as the un...","url_abs":"https://arxiv.org/abs/2509.20734","url_pdf":"https://arxiv.org/pdf/2509.20734v1","authors":"[\"Jinwook Park\",\"Kangil Kim\"]","published":"2025-09-25T04:31:14Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[]","has_code":false}