# Same Entity, Different Script: Diagnosing Cross-Script Entity Matching Failures in Language Models

DevFeed: [Same Entity, Different Script: Diagnosing Cross-Script Entity Matching Failures in Language Models](<https://devfeed.tech/articles/same-entity-different-script-diagnosing-cross-script-entity-matching-failures-in-language-models-98212.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.31407>)

Author: Athira Gopal, Patil Prathamesh Pravin, Ankush Kadu, Arpit Jain, Ashwanth Krishnan

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Entity resolution](<https://devfeed.tech/topics/entity-resolution.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [Internationalization (i18n)](<https://devfeed.tech/topics/i18n.md>), [enhancing AI model accuracy](<https://devfeed.tech/topics/enhancing-ai-model-accuracy.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [discrimination](<https://devfeed.tech/tags/discrimination.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [matching](<https://devfeed.tech/tags/matching.md>), [script](<https://devfeed.tech/tags/script.md>), [study](<https://devfeed.tech/tags/study.md>), [tokenization](<https://devfeed.tech/tags/tokenization.md>)

## AI overview

On audited name pairs, language models often fail to judge two script variants as the same person, even when they usually identify different people correctly. For Qwen2.5-7B, same-person accuracy fell from 35.1% with Latin names to 8.5% with Devanagari under Marathi prompts; a second evaluation found 5.3% for Malayalam. Tests across five models found distinct response biases, and reducing tokenization fragmentation did not reliably eliminate the gap. Probes showed that Devanagari model states retained identity information, while scalar calibration recovered 87% of the gap between raw model and probe balanced accuracy, implicating decision criteria as a major factor. Activation patching localized causal sensitivity to the second name in Qwen's middle layers. A fact-independent intervention there raised held-out Marathi same-person accuracy from 6.5% to 33.5%, with 98.5% different-person accuracy. These findings are based on the study's audited pairs and reported model evaluations.

## Source excerpt

arXiv:2610.31407v1 Announce Type: cross Abstract: Multilingual language models can give different identity judgments when the same people's names are written in different scripts. We study this effect using 601 audited identity pairs with attested Devanagari and Latin forms, holding the question and relation fixed. Qwen2.5-7B's same-person accuracy falls from 35.1% in Latin to 8.5% in Devanagari under Marathi prompts, while different-person accuracy remains above 98%. Evaluation across five models reveals distinct response biases, and reduced tokenization fragmentation does not reliably close the gap. Replicating the design on 374 audited Malayalam pairs built from the same entities reproduces the effect in a second script and language family: same-person accuracy falls to 5.3%, about half the matched Devanagari rate, while the margin still ranks matching above non-matching names at 0.925 AUROC. Latin-trained probes recover label information from Devanagari states, exceeding a string-feature baseline under similarity-matched negatives. Scalar calibration recovers 87% of the raw model-probe balanced-accuracy gap, identifying decision criterion placement as a major contributor. Activation patching localises causal sensitivity to the second entity in Qwen's middle layers; one shared sensitivity direction transfers about half the full-state margin correction. A fact-independent intervention at this site raises held-out Marathi same-person accuracy from 6.5% to 33.5%, with 98.5% different-person accuracy. Together, these results distinguish script-dependent discrimination from criterion placement and identify a causal site for improving identity judgments.