# GenoBERT: A Language Model for Accurate Genotype Imputation

DevFeed: [GenoBERT: A Language Model for Accurate Genotype Imputation](<https://devfeed.tech/articles/genobert-a-language-model-for-accurate-genotype-imputation-107883.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2604.00058>)

Author: Lei Huang, Chuan Qiu, Kuan-Jui Su, Anqi Liu, Yun Gong, Weiqiang Lin, Lindong Jiang, Chen Zhao, Meng Song, Jeffrey Deng, Qing Tian, Zhe Luo, Ping Gong, Hui Shen, Chaoyang Zhang, Hong-Wen Deng

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [bert](<https://devfeed.tech/topics/bert.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bias](<https://devfeed.tech/tags/bias.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [genomics](<https://devfeed.tech/tags/genomics.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [panel](<https://devfeed.tech/tags/panel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [q-bio-gn](<https://devfeed.tech/tags/q-bio-gn.md>), [range](<https://devfeed.tech/tags/range.md>), [resilience](<https://devfeed.tech/tags/resilience.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [sufficient](<https://devfeed.tech/tags/sufficient.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

## AI overview

GenoBERT imputes missing phased genotypes without a reference panel, using a transformer with self-attention to capture short- and long-range linkage disequilibrium. In tests on the Louisiana Osteoporosis Study and 1000 Genomes Project across ancestry groups and 5-50% missingness, it had the highest overall accuracy among the four named baselines: Beagle5.4, SCDA, BiU-Net, and STICI. The abstract reports r² around 0.98 with up to 25% missing and above 0.90 at 50% missing. It also reports consistent gains across ancestries and resilience to small samples and weak LD. LD-decay analyses supported a 128-SNP context window (about 100 Kb) for local correlations. These findings are based on the abstract; it does not provide detailed methods or per-group results.

## Source excerpt

arXiv:2604.00058v2 Announce Type: replace-cross Abstract: Genotype imputation enables dense variant coverage for genome-wide association and risk-prediction studies, yet conventional reference-panel methods remain limited by ancestry bias and reduced rare-variant accuracy. We present Genotype Bidirectional Encoder Representations from Transformers (GenoBERT), a transformer-based, reference-free framework that tokenizes phased genotypes and uses a self-attention mechanism to capture both short- and long-range linkage disequilibrium (LD) dependencies. Benchmarking on two independent datasets including the Louisiana Osteoporosis Study (LOS) and the 1000 Genomes Project (1KGP) across ancestry groups and multiple genotype missingness levels (5-50%) shows that GenoBERT achieves the highest overall accuracy compared to four baseline methods (Beagle5.4, SCDA, BiU-Net, and STICI). At practical sparsity levels (up to 25% missing), GenoBERT attains high overall imputation accuracy ($r^2 approx 0.98$) across datasets, and maintains robust performance ($r^2 > 0.90$) even at 50% missingness. Experimental results across different ancestries confirm consistent gains across datasets, with resilience to small sample sizes and weak LD. A 128-SNP (single-nucleotide polymorphism) context window (approximately 100 Kb) is validated through LD-decay analyses as sufficient to capture local correlation structures. By eliminating reference-panel dependence while preserving high accuracy, GenoBERT provides a scalable and robust solution for genotype imputation and a foundation for downstream genomic modeling.