{"ID":2834602,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.01870","arxiv_id":"2512.01870","title":"Testing Transformer Learnability on the Arithmetic Sequence of Rooted Trees","abstract":"We study whether a Large Language Model can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $ \\mathbb{N}\\mathcal{T}$ defines an arithmetic text with measurable statistical structure. A transformer network (the GPT-2 architecture) is trained from scratch on the first $10^{11}$ elements to subsequently test its predictive ability under next-word and masked-word prediction tasks. Our results show that the model partially learns the internal grammar of $\\mathbb{N}\\mathcal{T}$, capturing non-trivial regularities and correlations. This suggests that learnability may extend beyond empirical data to the very structure of arithmetic.","short_abstract":"We study whether a Large Language Model can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $ \\mathbb{N}\\mathcal{T}$ defines an arithmetic text with measurable statistical structur...","url_abs":"https://arxiv.org/abs/2512.01870","url_pdf":"https://arxiv.org/pdf/2512.01870v1","authors":"[\"Alessandro Breccia\",\"Federica Gerace\",\"Marco Lippi\",\"Gabriele Sicuro\",\"Pierluigi Contucci\"]","published":"2025-12-01T16:51:38Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cond-mat.dis-nn\",\"math-ph\",\"math.NT\"]","methods":"[\"Transformer\",\"Language Model\"]","has_code":false}