{"ID":5675954,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-04T18:43:15.567064008Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01391","arxiv_id":"2607.01391","title":"How Should Transformers Encode Numeric Values in Electronic Health Records?","abstract":"How do we encode numeric values in transformer-based sequence processing, particularly in electronic health record (EHR) data? We systematically compare discrete, continuous, and hybrid value encoding strategies using synthetic arithmetic tasks embedded within real-world EHR data, as well as real-world clinical prediction tasks. Our study reveals trade-offs between numeric precision, optimisation stability, and architectural flexibility. We find that approaches that explicitly model value-concept interactions perform best on precision-sensitive arithmetic tasks when architectural constraints permit. Hybrid token-based approaches that retain numeric values but apply binning prior to projection provide a more robust and broadly applicable alternative, with the optimal number of bins following a simple empirically derived power-law in dataset size. Across tasks, models consistently exhibit reliable \"good enough\" numeric computation rather than exact arithmetic, while clinical gains from incorporating laboratory values are task-dependent. This suggests that robustness and deployability often outweigh maximal numeric precision in practice, motivating hybrid token-based approaches as a practical default.","short_abstract":"How do we encode numeric values in transformer-based sequence processing, particularly in electronic health record (EHR) data? We systematically compare discrete, continuous, and hybrid value encoding strategies using synthetic arithmetic tasks embedded within real-world EHR data, as well as real-world clinical predict...","url_abs":"https://arxiv.org/abs/2607.01391","url_pdf":"https://arxiv.org/pdf/2607.01391v1","authors":"[\"Maria Elkjær Montgomery\",\"Christian Igel\",\"Mikkel Odgaard\",\"Martin Sillesen\",\"Mads Nielsen\"]","published":"2026-07-01T18:49:41Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Transformer\"]","has_code":false}