Cross-Lingual Knowledge Consistency

Whether multilingual language models and public knowledge sources encode the same facts across languages — and what drives the gaps when they don't.

Cross-lingual disagreement is the knowledge-translation diagnostic in its purest form: only the language changes, yet the answers do too. This thread studies whether multilingual language models and public knowledge sources such as Wikipedia and Wikidata encode the same facts across languages, and what drives the gaps when they don’t.

A Wikidata-Based Framework to Measure Cross-Lingual Bias in Multilingual LLMs (KG-LLM Workshop @ LREC 2026) introduces WILA-PopQA, a popularity-matched multilingual benchmark across 9 languages that disentangles three factors multilingual probing benchmarks usually confound: the language of the question, the language of the entity, and entity popularity. Across 12 open-weight LLMs, the language of the question turns out to be the dominant factor, and matching it to the entity’s language does not reliably improve factual recall (Iferroudjene et al., 2026).

The same question applies to the knowledge sources themselves: Factual Inconsistencies in Multilingual Wikipedia Tables studies where the language editions of Wikipedia disagree on structured facts, quantifying inconsistencies that models trained on this data may reproduce (Cappa et al., 2025).

References

2026

  1. KG-LLM
    A Wikidata-Based Framework to Measure Cross-Lingual Bias in Multilingual Large Language Models
    Mouloud Iferroudjene, Lisa Poggel, Andrea Schimmenti, and 4 more authors
    In Proceedings of the Workshop on Knowledge Graphs and Large Language Models (KG-LLM @ LREC 2026), 2026
    PDF Code

2025

  1. Preprint
    Factual Inconsistencies in Multilingual Wikipedia Tables
    Silvia Cappa, Lingxiao Kong, Pille-Riin Peet, and 3 more authors
    arXiv preprint arXiv:2507.18406, 2025
    arXiv