XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

Chenghao Zhu; Han Hu; Pengyuan Liu; Yuning Deng

REVIEW 3 major objections 4 minor 30 references

XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read XCMRC is presented as the first public cross-lingual cloze-style machine reading comprehension benchmark, with English/Chinese passage–question pairs and baselines that fall well short of a monolingual upper bound.

desk verdict New cross-lingual cloze MRC dataset with a real construct-validity problem: passage-independent baseline nearly ties passage-based models. read the letter →

arxiv 1908.05416 v1 pith:HA4LF2HQ submitted 2019-08-15 cs.CL

Pengyuan Liu , Yuning Deng , Chenghao Zhu , Han Hu This is my paper · ORCID

classification cs.CL

keywords cross-lingualmachinereadingcomprehensioncloze-stylebenchmarkdatasetpseudolow-resourcetaskmultilingualsentenceencodingtranslation-basedbaselineBiDAFEnglish-Chinese

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces XCMRC, a benchmark for cross-lingual machine reading comprehension built from parallel English–Chinese articles. Each sample asks a model to fill a masked noun in a question written in one language by reading a passage written in the other, with ten candidate nouns provided. The authors claim this is the first public cross-lingual language understanding benchmark for reading comprehension, and they show that translation-based and multilingual-encoder baselines achieve reasonable but incomplete accuracy, while pseudo low-resource approaches lag far behind the upper bound. The benchmark is meant to push research on machines that can read across languages, which matters because most real-world NLP systems encounter many languages without enough annotated data in each.

What carries the argument

The organizing mechanism is the automatic dataset generation pipeline: starting from strictly aligned Chinese–English paragraph pairs, the pipeline counts nouns, selects an answer word with corpus frequency 3–10, builds a cloze question by masking that word in a paragraph, removes the corresponding paragraph from the document, and samples nine distractor nouns from the same noun pool. This pipeline is what creates the cross-lingual test: the question and candidates are in the target language, the document is in the source language, and the answer must be recovered by connecting the two. The identical pipeline applied monolingually produces the EPEQ and CPCQ sub-datasets that act as the upper bound oracle.

What would settle it

Run a strong passage-independent model that attends only to the question and candidates on the XCMRC test sets and compare its accuracy with the translation-based and upper-bound baselines; if it matches or exceeds them, the passage is not the source of the answer signal.

Watch

Extended reading notes

Core claim

The central claim is that XCMRC is the first public benchmark for cross-lingual cloze-style machine reading comprehension, and that it is usable to measure both a general cross-lingual task and a pseudo low-resource variant. The corpus contains two dual subsets: EPCQ (English passages, Chinese questions) with 57,599 samples and CPEQ (Chinese passages, English questions) with 55,990 samples, generated automatically from an aligned bilingual corpus by choosing a noun, blanking it in a sentence, removing the corresponding paragraph from the passage, and adding nine distractor nouns. The same generator produces monolingual EPEQ and CPCQ sub-datasets that serve as an upper bound. Baselines show that the translation-based BiDAF Cloze reaches 67.28% on CPEQ and 65.99% on EPCQ, the multilingual BERT-based model reaches 63.28%, while naive and passage-independent models on the pseudo low-resource task stay near 58–62%, compared with monolingual upper bounds of 72.97% and 68.81%.

Load-bearing premise

The load-bearing premise is that the passage is necessary to answer the cloze question -- yet the paper's own passage-independent baseline achieves 59.83%, close to the passage-using baselines, which would undermine that premise.

Editorial extensions

If this is right

If XCMRC is a valid measure, cross-lingual reading comprehension between English and Chinese has substantial headroom: the best common-task baseline is about 3–6 points below the monolingual upper bound.
Pseudo low-resource cross-lingual MRC remains largely unsolved: the naive cross-lingual model learns little beyond what the question alone provides, matching the passage-independent baseline.
The upper bound of the benchmark is not a static ceiling; as monolingual cloze models improve on EPEQ and CPCQ, the estimated headroom for cross-lingual models will need to be revised upward.
Translation-based methods currently outperform multilingual sentence encoders on this benchmark, suggesting that for long-document cloze tasks, explicit translation transfers more information than shared embedding spaces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I infer that the near-parity between the passage-independent baseline (59.83%) and the passage-using naive and translation baselines (58.35–67.28%) indicates that many XCMRC questions can be answered from question and candidate statistics alone, so a revised version should filter such questions to keep the passage load-bearing.
I infer that the same generation pipeline could produce pseudo low-resource XCMRC benchmarks for other language pairs, provided an aligned bilingual corpus exists, making the resource restriction a genuinely controllable experimental variable.
I infer that newer multilingual encoders not tested in this paper could close or even reverse the gap between translation-based and encoder-based approaches, and the reported results give a concrete baseline for such comparisons.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here is my read. What is actually new: XCMRC is the first cross-lingual cloze-style MRC benchmark I know of, with two dual sub-datasets (EPCQ, CPEQ) built automatically from aligned English-Chinese news, and a pseudo low-resource task that restricts external language resources. The construction is reasonable, the scale is sufficient for training, and the authors include an explicit monolingual upper bound. They also deserve credit for reporting a passage-independent baseline at all; that is the right diagnostic instinct.

The problem is what that diagnostic shows. On EPCQ (English passage, Chinese question), the passage-independent model scores 59.83%, slightly higher than the passage-using naive BiDAF_Candidates at 58.35%. On CPEQ, the naive model does better (61.64% vs 58.20%), but the gap is small. So for a large share of items, the question plus ten candidate nouns carry enough signal to choose the answer without reading the source-language passage. The paper acknowledges the closeness but reads it as the naive model failing to learn cross-lingual information. The alternative reading is more worrying: the task itself may not require cross-lingual reading comprehension, at least in the pseudo low-resource setting. Since the stated purpose is to 'test machines on their cross-lingual reading comprehension ability,' this is a load-bearing validity issue, not a footnote. It is fixable—candidate filtering, artifact analysis, or a control showing passage-using models can clearly beat question-only—but as is, the central claim is not established.

Other soft spots are smaller. Only dev set results are reported, despite having a test split. Five runs with the same random seed are not independent; calling the result an average across runs is misleading. The BERT baseline is only partly multilingual: English passage is encoded with mBERT, Chinese question with separate pre-trained Chinese word embeddings, so it is not a clean multilingual-sentence-encoder baseline. The table layout is confusing, and the upper bound is simply a monolingual model, which is honest but loose.

Who should read this: groups working on cross-lingual QA and low-resource MRC will want to know about the dataset and the pseudo low-resource setting. I would send this to review; it deserves a serious referee, but I would expect a revision that addresses the artifact problem and reports test results. I would not cite it in its current form, but I would track a revised version.

Referee Report

3 major / 4 minor

Summary. The paper proposes XCMRC, a cross-lingual cloze-style machine reading comprehension benchmark constructed from 25,467 aligned Chinese-English Economist articles. It consists of two cross-lingual sub-datasets, EPCQ (English passages, Chinese questions) and CPEQ (Chinese passages, English questions), each with roughly 56k samples, and two monolingual sub-datasets (EPEQ, CPCQ) intended as an upper-bound setting. The task is to fill a blank in a target-language question by choosing one of ten noun candidates after reading a passage in the source language. The authors define a common XCMRC task and a pseudo low-resource task and evaluate four baseline families: translation-based BiDAF Cloze, multilingual-BERT-based BERT-Candidates, naive BiDAF-Candidates, and a passage-independent PI-Candidates baseline. Their main findings are that translation-based methods perform best (up to 67.28%) but remain far below the monolingual upper bound (72.97%/68.81%), and that pseudo low-resource methods perform close to the passage-independent baseline, indicating that they learn little cross-lingual document information.

Significance. The paper addresses a real gap: at the time of submission there was no public cross-lingual MRC benchmark, and an automatically constructed bilingual cloze dataset is a plausible and scalable route. The main strengths are the scale of the resource, the clear formalization of two task variants (common and pseudo low-resource), the release of data and code, and the inclusion of a passage-independent control baseline, which is scientifically honest. If the construct-validity concerns are resolved, the benchmark could be useful for evaluating cross-lingual representation learning. However, the near-parity of the passage-independent and passage-based baselines currently undermines the claim that XCMRC measures cross-lingual reading comprehension rather than shallow lexical statistics.

major comments (3)

[Section 4.3, Table 5, Table 1 sample] The passage-independent baseline PI_Candidates achieves 59.83% accuracy, while the passage-using BiDAF_Candidates achieves only 58.35% and 61.64% in the two directions (Table 5). This near-parity means that for a large fraction of items the question and candidates alone suffice, so the benchmark may not require reading the passage. This is not merely a curiosity: the EPCQ example in Table 1 has the answer 'discrimination' appearing verbatim in the visible text of the question itself. The paper's own discussion in Section 5.2 admits that the naive approach 'has learned very limited information about the document.' To establish construct validity, the authors should report an item-level analysis: (i) the proportion of dev items for which PI_Candidates is correct; (ii) the proportion for which a passage-based model is correct while PI_Candidates is wrong; (iii) the same statistics on a filtered subset where the answer does not appear in the question; and (iv) a human ceiling on a sample. If the fraction of items requiring the passage is small, the benchmark should be revised (e.g., by filtering such items) before it can support the paper's central claim.
[Section 5.1 and Section 5.2] Only dev set results are reported (Section 5.2: 'we only present the performance on dev set'), so there is no evidence on the held-out test set that the benchmark defines. Since the dataset's purpose is to support evaluation, at least test-set numbers (or a clear statement that the test set is reserved for a leaderboard with hidden labels) are needed. In addition, Section 5.1 states that models were 'run 5 times independently with the same random seed 1234'; identical seeds do not yield independent trials, so the reported averages cannot be used to assess variance. The authors should run with different seeds and report mean ± standard deviation.
[Table 5 and Section 5.2] Table 5 does not make clear which cells correspond to which dataset or language direction. The row labels 'English' and 'Chinese' under 'Passage' are not matched with explicit question-language columns, and the prose in Section 5.2 refers to 'EPEQ' when discussing the naive approach's performance on the cross-lingual CPEQ/EPCQ pair. Please reformat the table with explicit columns for EPCQ and CPEQ (and EPEQ/CPCQ where relevant), and correct the acronym usage so the central numerical claims can be verified.

minor comments (4)

[Section 3.2 and Table 1] Typographical errors: 'Tabel 2' in Section 3.2 should be 'Table 2', and 'addictive' in the Table 1 sample should be 'additive'.
[Table 1] In the EPCQ sample, the answer 'discrimination' appears in the question text; if this sample is representative, such leakage should be explicitly measured and discussed, or the sample replaced.
[Section 5.2] The definition of accuracy is top-1 accuracy on the ten candidates; please state this explicitly in Section 5.2, as the candidate-selection setting is not obvious from the model equations.
[Section 3.2] The paper should report whether the answer word (or its translation) appears in the document D after the aligned paragraph is removed; this statistic is crucial for interpreting whether D can contain the answer at all and for choosing a meaningful upper bound.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: benchmark construction and baseline evaluations are empirical, not predictions derived from fitted inputs.

full rationale

I find no circular dependency in this paper. XCMRC is a corpus construction and benchmarking paper. The task is defined by the generation procedure in Section 3.2 (automatic samples from aligned bilingual paragraphs), and the paper does not claim to derive a theoretical prediction from a fitted parameter. The baselines are evaluated on held-out dev data and compared with a random-choice floor (10%) and an upper-bound monolingual model; none of these quantities is defined in terms of the result it is supposed to support. The passage-independent baseline (PI_Candidates) in Section 4.3 is explicitly a diagnostic control, not a fitted input dressed as a prediction. Its near-parity with the naive BiDAF_Candidates system may threaten the construct validity of the benchmark (i.e., whether passages are necessary), but that is a correctness or validity concern, not a circularity concern, and the paper itself reports the number rather than hiding it. All methodological citations (SQuAD, BiDAF, BERT, XNLI, CNN/Daily Mail) are external prior work; there is no self-citation chain that carries a load-bearing argument. Therefore no circular step can be exhibited with a quote, and the honest finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or theoretical entities. Its main artifacts are the datasets and task definitions, which are not free parameters in the modeling sense. The free parameters and assumptions listed are the design choices that the benchmark's validity depends on.

free parameters (1)

Noun frequency interval = 3-10
In Section 3.2, the authors restrict candidate answer nouns to those with frequency between 3 and 10 in the Chinese passage. This interval is a hand-chosen design parameter that affects dataset difficulty and is not derived from any principle.

assumptions (3)

domain assumption Aligned Chinese-English paragraphs from the Economist channel are strictly aligned and semantically parallel.
The generation procedure assumes each pc_i aligns exactly with pe_i (Section 3.2). If alignment is imperfect, the cross-lingual signal in the document may be missing or misleading.
domain assumption The translation API (Baidu) produces sufficiently accurate translations for the translation-based baseline to be meaningful.
Section 5.1 relies on Baidu Translation API for translating dev set passages; translation quality is not controlled or reported.
domain assumption POS tagging via Jieba and NLTK correctly identifies nouns for candidate selection.
Section 3.2 uses these tools; errors in POS tagging could lead to incorrect answer candidates.

how reviews work

0 comments

Cite this review

Pith. "Pith review of XCMRC: Evaluating Cross-lingual Machine Reading Comprehension." pith.science (2026). https://pith.science/paper/HA4LF2HQ

@misc{pith2026190805416,
  author       = {Pith},
  title        = {Pith review of: XCMRC: Evaluating Cross-lingual Machine Reading Comprehension},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HA4LF2HQ}},
  note         = {Machine review of arXiv:1908.05416}
}

read the original abstract

We present XCMRC, the first public cross-lingual language understanding (XLU) benchmark which aims to test machines on their cross-lingual reading comprehension ability. To be specific, XCMRC is a Cross-lingual Cloze-style Machine Reading Comprehension task which requires the reader to fill in a missing word (we additionally provide ten noun candidates) in a sentence written in target language (English / Chinese) by reading a given passage written in source language (Chinese / English). Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used. In addition, we provide two baselines for common XCMRC task and two for pseudo XCMRC task respectively. We also provide an upper bound baseline for both tasks. We found that for common XCMRC task, translation-based method and multilingual sentence encoder-based method can obtain reasonable performance but still have much room for improvement. As for pseudo low-resource XCMRC task, due to strict restrictions on the use of language resources, our two approaches are far below the upper bound so there are many challenges ahead.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 28 canonical work pages

[1]

In: LREC

Agi´ c,ˇZ., Schluter, N.: Baselines and test data for cross-lingual inference. In: LREC. European Language Resource Association (2018)

work page 2018
[2]

V.D., Tiedemann, J.: Question answering with joost at clef 2007

Bouma, G., Kloosterman, G., Mur, J., Noord, G.V., Plas, L. V.D., Tiedemann, J.: Question answering with joost at clef 2007. In: AMMIR (2008)

work page 2008
[3]

In: SemEval

Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L .: Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingua l focused evaluation. In: SemEval. pp. 1–14. ACL (2017)

work page 2017
[4]

Conneau, A., Kiela, D., Schwenk, H., Barrault, L., Bordes , A.: Supervised learning of universal sentence representations from natural langua ge inference data (2017)

work page 2017
[5]

In: EMNLP

Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman , S., Schwenk, H., Stoyanov, V.: Xnli: Evaluating cross-lingual sentence rep resentations. In: EMNLP. pp. 2475–2485. ACL (2018)

work page 2018
[6]

In: COLING

Cui, Y., Liu, T., Chen, Z., Wang, S., Hu, G.: Consensus atte ntion-based neural networks for chinese reading comprehension. In: COLING. pp . 1777–1786. The COLING 2016 Organizing Committee (2016)

work page 2016
[7]

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre -training of deep bidi- rectional transformers for language understanding (2018)

work page 2018
[8]

Dunn, M., Sagun, L., Higgins, M., Guney, V.U., Cirik, V., C ho, K.: Searchqa:a new q&a dataset augmented with context from a search engine ( 2017)

work page 2017

Show all 30 references

[9]

IEEE Journal of Selected Topic s in Signal Processing PP(99), 1–1 (2017)

Espa˜ na-Bonet, C., ´Ad´ am Csaba Varga, Barr´ on-Cede˜ no, A., Genabith, J.V.: An empirical analysis of nmt-derived interlingual embedding s and their use in paral- lel sentence identiﬁcation. IEEE Journal of Selected Topic s in Signal Processing PP(99), 1–1 (2017)

2017
[10]

In: Proceedings of t he Workshop on Machine Reading for Question Answering

He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y ., Wang, Y., Wu, H., She, Q., Liu, X., Wu, T., Wang, H.: Dureader: a chinese machine rea ding comprehension dataset from real-world applications. In: Proceedings of t he Workshop on Machine Reading for Question Answ...

2018
[11]

I n: NIPS

Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt , L., Kay, W., Suleyman, M., Blunsom, P.: Teaching machines to read and comprehend. I n: NIPS. pp. 1693– 1701 (2015)

2015
[12]

Co mputer Science (2015)

Hill, F., Bordes, A., Chopra, S., Weston, J.: The goldilo cks principle: Reading children’s books with explicit memory representations. Co mputer Science (2015)

2015
[13]

I n: ACL

Joshi, M., Choi, E., Weld, D., Zettlemoyer, L.: Triviaqa : A large scale distantly supervised challenge dataset for reading comprehension. I n: ACL. pp. 1601–1611. ACL (2017)

2017
[14]

Joty, S., Nakov, P., M` arquez, L., Jaradat, I.: Cross-la nguage learning with adver- sarial neural networks (2017)

2017
[15]

Proceedings of COLING 2012 pp

Klementiev, A., Titov, I., Bhattarai, B.: Inducing cros slingual distributed repre- sentations of words. Proceedings of COLING 2012 pp. 1459–14 74 (2012)

2012
[16]

Transactions of the Association of Computational Linguistics 6, 317–328 (2018)

Koˇ cisk` y, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K.M., Melis, G., Grefen- stette, E.: The narrativeqa reading comprehension challen ge. Transactions of the Association of Computational Linguistics 6, 317–328 (2018)

2018
[17]

Kwiatkowski, T., Palomaki, J., Rhinehart, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Kelcey, M., Devlin, J., et a l.: Natural questions: a benchmark for question answering research (2019)

2019
[18]

In: EMNLP

Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: Race: Large- scale reading compre- hension dataset from examinations. In: EMNLP. pp. 785–794. ACL (2017)

2017
[19]

Proceedings of the Seventh Ntcir Works hop Meeting (2010)

Mitamura, T., Shima, H., Sakai, T., Kando, N., Mori, T., T akeda, K., Lin, C.Y., Lin, C.J., Lee, C.W.: Overview of the ntcir-8 aclia tasks: Ad vanced cross-lingual information access. Proceedings of the Seventh Ntcir Works hop Meeting (2010)

2010
[20]

arXiv preprint arXiv:1611.09268 (2016)

Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., Deng, L.: Ms marco: A human generated machine reading comprehensi on dataset. arXiv preprint arXiv:1611.09268 (2016)

2016 arXiv
[21]

In: Proceedings of TextGraphs-10: the Workshop on Gr aph-based Methods for Natural Language Processing

Pouran Ben Veyseh, A.: Cross-lingual question answerin g using common semantic space. In: Proceedings of TextGraphs-10: the Workshop on Gr aph-based Methods for Natural Language Processing. pp. 15–19. ACL, San Diego, CA, USA (June 2016)

2016
[22]

Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t kno w: Unanswerable ques- tions for squad. In: ACL. pp. 784–789. ACL (2018)

2018
[23]

In: EMNLP

Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. In: EMNLP. pp. 2383–2392. AC L (2016)

2016
[24]

In: EMNLP

Richardson, M., Burges, C.J., Renshaw, E.: Mctest: A cha llenge dataset for the open-domain machine comprehension of text. In: EMNLP. pp. 1 93–203. ACL (2013)

2013
[25]

In: LREC

Schwenk, H., Li, X.: A corpus for multilingual document c lassiﬁcation in eight languages. In: LREC. European Language Resource Associati on (2018)

2018
[26]

CoRR abs/1611.01603 (2016)

Seo, M.J., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bi directional attention ﬂow for machine comprehension. CoRR abs/1611.01603 (2016)

2016 arXiv
[27]

In: SIGIR (2016)

Soboroﬀ, I., Griﬃtt, K., Strassel, S.: The bolt ir test co llections of multilingual passage retrieval from discussion forums. In: SIGIR (2016)

2016
[28]

In: Proceedin gs of the 2nd Work- shop on Representation Learning for NLP

Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A ., Bachman, P., Suleman, K.: Newsqa: A machine comprehension dataset. In: Proceedin gs of the 2nd Work- shop on Representation Learning for NLP. pp. 191–200. ACL (2 017)
[29]

Ture, F., Boschee, E.: Learning to translate for multili ngual question answering (2016)

2016
[30]

In: ACL (2016)

Zhou, X., Wan, X., Xiao, J.: Cross-lingual sentiment cla ssiﬁcation with bilingual document representation learning. In: ACL (2016)

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

In: LREC

Agi´ c,ˇZ., Schluter, N.: Baselines and test data for cross-lingual inference. In: LREC. European Language Resource Association (2018)

work page 2018

[2] [2]

V.D., Tiedemann, J.: Question answering with joost at clef 2007

Bouma, G., Kloosterman, G., Mur, J., Noord, G.V., Plas, L. V.D., Tiedemann, J.: Question answering with joost at clef 2007. In: AMMIR (2008)

work page 2008

[3] [3]

In: SemEval

Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L .: Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingua l focused evaluation. In: SemEval. pp. 1–14. ACL (2017)

work page 2017

[4] [4]

Conneau, A., Kiela, D., Schwenk, H., Barrault, L., Bordes , A.: Supervised learning of universal sentence representations from natural langua ge inference data (2017)

work page 2017

[5] [5]

In: EMNLP

Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman , S., Schwenk, H., Stoyanov, V.: Xnli: Evaluating cross-lingual sentence rep resentations. In: EMNLP. pp. 2475–2485. ACL (2018)

work page 2018

[6] [6]

In: COLING

Cui, Y., Liu, T., Chen, Z., Wang, S., Hu, G.: Consensus atte ntion-based neural networks for chinese reading comprehension. In: COLING. pp . 1777–1786. The COLING 2016 Organizing Committee (2016)

work page 2016

[7] [7]

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre -training of deep bidi- rectional transformers for language understanding (2018)

work page 2018

[8] [8]

Dunn, M., Sagun, L., Higgins, M., Guney, V.U., Cirik, V., C ho, K.: Searchqa:a new q&a dataset augmented with context from a search engine ( 2017)

work page 2017