Cross-Lingual Ability of Multilingual BERT: An Empirical Study

Dan Roth; Karthikeyan K; Stephen Mayhew; Zihan Wang

Cross-Lingual Ability of Multilingual BERT: An Empirical Study

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1912.07840 v2 pith:NNQ5R535 submitted 2019-12-17 cs.CL cs.AIcs.LG

Karthikeyan K , Zihan Wang , Stephen Mayhew , Dan Roth This is my paper

classification cs.CLcs.AIcs.LG

keywords cross-lingualdifferentlanguagesabilitybertm-bertmultilingualpage

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

Recent work has exhibited the surprising cross-lingual abilities of multilingual BERT (M-BERT) -- surprising since it is trained without any cross-lingual objective and with no aligned data. In this work, we provide a comprehensive study of the contribution of different components in M-BERT to its cross-lingual ability. We study the impact of linguistic properties of the languages, the architecture of the model, and the learning objectives. The experimental study is done in the context of three typologically different languages -- Spanish, Hindi, and Russian -- and using two conceptually different NLP tasks, textual entailment and named entity recognition. Among our key conclusions is the fact that the lexical overlap between languages plays a negligible role in the cross-lingual success, while the depth of the network is an integral part of it. All our models and implementations can be found on our project page: http://cogcomp.org/page/publication_view/900 .

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges
cs.CL 2025-08 conditional novelty 4.0 of 10

A comprehensive survey of cross-lingual aspect-based sentiment analysis that catalogs tasks, datasets, modeling paradigms, and cross-lingual transfer techniques, and identifies research gaps.

Pith tools