# How Should Transformers Encode Numeric Values in Electronic Health Records?

DevFeed: [How Should Transformers Encode Numeric Values in Electronic Health Records?](<https://devfeed.tech/articles/how-should-transformers-encode-numeric-values-in-electronic-health-records-99431.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.01391>)

Author: Maria Elkj{\\ae}r Montgomery, Christian Igel, Mikkel Odgaard, Martin Sillesen, Mads Nielsen

Published: 2026-10-26T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Encoding](<https://devfeed.tech/topics/encoding.md>)

Tags: [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [electronic-health-records](<https://devfeed.tech/tags/electronic-health-records.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [laboratory](<https://devfeed.tech/tags/laboratory.md>), [optimisation](<https://devfeed.tech/tags/optimisation.md>), [precision](<https://devfeed.tech/tags/precision.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [study](<https://devfeed.tech/tags/study.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

## AI overview

The comparison finds that transformer numeric encodings trade precision against training stability and architectural flexibility. Explicitly modeling interactions between values and clinical concepts performs best on precision-sensitive arithmetic tasks when the architecture allows it. A hybrid approach that bins values before projection is described as more robust and broadly applicable; its best bin count follows an empirically derived power law in dataset size. Models generally achieve reliable approximate rather than exact arithmetic, and laboratory values improve clinical prediction only for some tasks. The authors therefore present hybrid token-based encoding as a practical default when robustness and deployability matter.

## Source excerpt

arXiv:2607.01391v2 Announce Type: replace Abstract: How do we encode numeric values in transformer-based sequence processing, particularly in electronic health record (EHR) data? We systematically compare discrete, continuous, and hybrid value encoding strategies using synthetic arithmetic tasks embedded within real-world EHR data, as well as real-world clinical prediction tasks. Our study reveals trade-offs between numeric precision, optimisation stability, and architectural flexibility. We find that approaches that explicitly model value-concept interactions perform best on precision-sensitive arithmetic tasks when architectural constraints permit. Hybrid token-based approaches that retain numeric values but apply binning prior to projection provide a more robust and broadly applicable alternative, with the optimal number of bins following a simple empirically derived power-law in dataset size. Across tasks, models consistently exhibit reliable "good enough" numeric computation rather than exact arithmetic, while clinical gains from incorporating laboratory values are task-dependent. This suggests that robustness and deployability often outweigh maximal numeric precision in practice, motivating hybrid token-based approaches as a practical default.