Pith. sign in

REVIEW 5 cited by

Incorporating BERT into Neural Machine Translation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2002.06823 v1 pith:2DZ56SGO submitted 2020-02-17 cs.CL

Incorporating BERT into Neural Machine Translation

Jinhua Zhu , Yingce Xia , Lijun Wu , Di He , Tao Qin , Wengang Zhou , Houqiang Li , Tie-Yan Liu This is my paper
classification cs.CL
keywords bertmachinetranslationbert-nmtbettercontextualembeddingexploration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

The recently proposed BERT has shown great power on a variety of natural language understanding tasks, such as text classification, reading comprehension, etc. However, how to effectively apply BERT to neural machine translation (NMT) lacks enough exploration. While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning. This motivates us to think how to better leverage BERT for NMT along this direction. We propose a new algorithm named BERT-fused model, in which we first use BERT to extract representations for an input sequence, and then the representations are fused with each layer of the encoder and decoder of the NMT model through attention mechanisms. We conduct experiments on supervised (including sentence-level and document-level translations), semi-supervised and unsupervised machine translation, and achieve state-of-the-art results on seven benchmark datasets. Our code is available at \url{https://github.com/bert-nmt/bert-nmt}.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CATCH-ME if you RAG: a dataset of Contextually Annotated multi-Turn Counterspeech against Hate and Misinformation Exchanges

    cs.CL 2026-06 unverdicted novelty 8.0

    Presents a new expert-curated dataset of multi-turn counterspeech dialogues in five languages targeting hate against seven groups, with span annotations linking to verified external knowledge for RAG applications.

  2. Assisted Counterspeech Writing at the Crossroads of Hate Speech and Misinformation

    cs.CL 2026-05 conditional novelty 6.0

    LLMs generate adequate counterspeech for co-occurring hate and misinformation in 40% of cases, with a mixed knowledge strategy from fact-checkers and NGOs proving most effective after expert revision.

  3. BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation

    cs.CV 2026-05 unverdicted novelty 6.0

    BabelDOC uses an intermediate representation to decouple layout from content for improved layout-preserving PDF translation.

  4. Neural Grammatical Error Correction for Romanian

    cs.CL 2026-04 unverdicted novelty 6.0

    A new Romanian GEC corpus of 10k pairs plus pretraining a Transformer on artificial errors generated via POS tagger yields F0.5 of 53.76, beating the 44.38 baseline from training only on the corpus.

  5. Ad Headline Generation using Self-Critical Masked Language Model

    cs.CL 2026-07 conditional novelty 4.5

    Self-critical policy-gradient training of a multi-product BERT MLM produces ad headlines that beat LSTM+RL baselines and human submissions on overlap metrics and blind quality/grammar audits.