{"ID":2861412,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.01863","arxiv_id":"2510.01863","title":"Microscaling Floating Point Formats for Large Language Models","abstract":"The increasing computational and memory demands of large language models (LLMs) necessitate innovative approaches to optimize resource usage without compromising performance. This paper leverages microscaling floating-point formats, a novel technique designed to address these challenges by reducing the storage and computational overhead associated with numerical representations in LLMs. Unlike traditional floating-point representations that allocate a dedicated scale for each value, microscaling employs a shared scale across a block of values, enabling compact one-byte floating-point representations while maintaining an extended dynamic range. We explore the application of microscaling in the context of 8-bit floating-point formats to significantly reduce memory footprint and computational costs. We tested several configurations of microscaling floats within the GPT-2 LLM architecture, demonstrating that microscaling data formats can achieve competitive accuracy during training and inference, proving its efficacy as a resource-efficient alternative for deploying LLMs at scale. The source code is publicly available at: https://github.com/unipi-dii-compressedarith/llm.c-sve","short_abstract":"The increasing computational and memory demands of large language models (LLMs) necessitate innovative approaches to optimize resource usage without compromising performance. This paper leverages microscaling floating-point formats, a novel technique designed to address these challenges by reducing the storage and comp...","url_abs":"https://arxiv.org/abs/2510.01863","url_pdf":"https://arxiv.org/pdf/2510.01863v1","authors":"[\"Marco Cococcioni\",\"Dario Pagani\",\"Federico Rossi\"]","published":"2025-10-02T10:08:59Z","proceeding":"cs.NE","tasks":"[\"cs.NE\",\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":608813,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2861412,"paper_url":"https://arxiv.org/abs/2510.01863","paper_title":"Microscaling Floating Point Formats for Large Language Models","repo_url":"https://github.com/unipi-dii-compressedarith/llm.c-sve","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}