Pith. sign in

REVIEW 5 cited by

PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1908.11828 v1 pith:DSVMEZZE submitted 2019-08-30 cs.CL

Yinfei Yang , Yuan Zhang , Chris Tar , Jason Baldridge This is my paper
classification cs.CL
keywords multilingualenglishparaphrasepawspaws-xaccuracyadversarialbest
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Most existing work on adversarial data generation focuses on English. For example, PAWS (Paraphrase Adversaries from Word Scrambling) consists of challenging English paraphrase identification pairs from Wikipedia and Quora. We remedy this gap with PAWS-X, a new dataset of 23,659 human translated PAWS evaluation pairs in six typologically distinct languages: French, Spanish, German, Chinese, Japanese, and Korean. We provide baseline numbers for three models with different capacity to capture non-local context and sentence structure, and using different multilingual training and evaluation regimes. Multilingual BERT fine-tuned on PAWS English plus machine-translated data performs the best, with a range of 83.1-90.8 accuracy across the non-English languages and an average accuracy gain of 23% over the next best model. PAWS-X shows the effectiveness of deep, multilingual pre-training while also leaving considerable headroom as a new challenge to drive multilingual research that better captures structure and contextual information.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ChiKhaPo: A Large-Scale Multilingual Benchmark for Evaluating Lexical Comprehension and Generation in Large Language Models

    cs.CL 2025-10 conditional novelty 7.0 of 10

    ChiKhaPo is an 8-subtask benchmark that measures word-level comprehension and generation in 2,700+ languages and shows state-of-the-art models perform poorly on low-resource languages.

  2. Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit

    cs.CL 2025-06 conditional novelty 6.0 of 10

    OMP sparse coding of donor token embeddings, with coefficients transferred to the base embedding space, preserves LLM performance after tokenizer replacement better than published zero-shot baselines, though simple he...

  3. Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A 7B open-weight translation model matches or outperforms far larger commercial systems across 28 languages in automatic and human evaluations.

  4. QZhou-Embedding Technical Report

    cs.CL 2025-08 conditional novelty 4.0 of 10

    QZhou-Embedding reports state-of-the-art average scores on MTEB and CMTEB as of August 27, 2025, using a two-stage multi-task pipeline with LLM-based data synthesis.

  5. Cross-lingual Few-shot Learning for Persian Sentiment Analysis with Incremental Adaptation

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Combining few-shot fine-tuning with incremental learning and regularization lets XLM-R and mDeBERTa reach about 96% accuracy on Persian sentiment analysis across five domains.

Pith tools