pith. sign in

arxiv: 1907.01686 · v1 · pith:JMW6GJEYnew · submitted 2019-06-30 · 💻 cs.CL

Machine Reading Comprehension: a Literature Review

Pith reviewed 2026-05-25 13:13 UTC · model grok-4.3

classification 💻 cs.CL
keywords machine reading comprehensionliterature reviewcorporatechniquesnatural language processingquestion answering
0
0 comments X

The pith

A review organizes machine reading comprehension work by comparing datasets and outlining techniques.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper collects and compares the specific characteristics of various machine reading comprehension corpora. It also describes the main ideas behind some typical techniques in the area. A sympathetic reader would value this because MRC is presented as a step toward machines understanding text like humans. The structure helps navigate recent advances by focusing on these two aspects. The authors aim to give an overview that highlights how different corpora and methods relate to each other.

Core claim

The paper establishes that recent advances in machine reading comprehension can be summarized by listing and comparing the characteristics of its corpora and by describing the main ideas of its typical techniques.

What carries the argument

The two-part structure that separates corpus characteristics from technique ideas, which the review uses to organize its coverage of the field.

If this is right

  • Researchers gain a structured way to select corpora based on their listed characteristics for new experiments.
  • The descriptions of typical techniques provide a baseline for understanding how models process reading comprehension tasks.
  • The comparisons across corpora can highlight differences in difficulty, size, or question types that affect model evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The review's structure could guide the design of new datasets that fill gaps in the compared characteristics.
  • Readers might extend the technique descriptions to test how well newer methods fit the outlined main ideas.

Load-bearing premise

The selected corpora and techniques are representative of the broader field and the comparisons and descriptions are comprehensive and unbiased.

What would settle it

Identification of a widely used MRC corpus or technique that was omitted or whose description differs substantially from the review's account in a way that changes the overall picture.

read the original abstract

Machine reading comprehension aims to teach machines to understand a text like a human and is a new challenging direction in Artificial Intelligence. This article summarizes recent advances in MRC, mainly focusing on two aspects (i.e., corpus and techniques). The specific characteristics of various MRC corpus are listed and compared. The main ideas of some typical MRC techniques are also described.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The manuscript is a literature review on Machine Reading Comprehension (MRC). It summarizes recent advances by focusing on two aspects: corpora and techniques. It lists and compares the specific characteristics of various MRC corpora and describes the main ideas of some typical MRC techniques.

Significance. If the selected corpora and techniques are representative and the descriptions accurate, the review could serve as a useful organizing reference for researchers entering the MRC area circa 2019, particularly by collating corpus statistics and high-level technique overviews in one place.

minor comments (2)
  1. [Abstract] The abstract states the focus on 'corpus and techniques' but does not specify the time window covered or the criteria used to select the 'various MRC corpus' and 'typical MRC techniques,' which would help readers assess scope.
  2. No explicit statement appears on how the review ensures completeness or avoids selection bias in the corpora and techniques presented.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive review and the recommendation to accept the manuscript. The comments confirm that the survey can serve as a useful reference by collating corpus statistics and technique overviews.

Circularity Check

0 steps flagged

No significant circularity; purely descriptive review

full rationale

The manuscript is a literature review whose purpose is to summarize selected MRC corpora and techniques. It contains no original equations, derivations, fitted parameters, predictions, or theorems. No load-bearing claim reduces by construction to its own inputs or to a self-citation chain. The paper simply lists and compares external work; representativeness is an external concern, not an internal circularity. This is the expected finding for a descriptive survey.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

As a literature review the paper introduces no new free parameters, axioms, or invented entities; it only summarizes existing published work.

pith-pipeline@v0.9.0 · 5570 in / 910 out tokens · 25505 ms · 2026-05-25T13:13:38.850269+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 69 canonical work pages · 38 internal anchors

  1. [1]

    Neural Machine Translation by Jointly Learning to Align and Translate

    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)

  2. [2]

    Journal of machine learning research 3(Feb), 1137–1155 (2003)

    Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. Journal of machine learning research 3(Feb), 1137–1155 (2003)

  3. [3]

    Artificial intelligence 8(2), 155–173 (1977)

    Bobrow, D.G., Kaplan, R.M., Kay, M., Norman, D.A., Thompson, H., Winograd, T.: Gus, a frame-driven dialog system. Artificial intelligence 8(2), 155–173 (1977)

  4. [4]

    A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task

    Chen, D., Bolton, J., Manning, C.D.: A thorough examination of the cnn/daily mail read- ing comprehension task. arXiv preprint arXiv:1606.02858 (2016)

  5. [5]

    Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

    Cho, K., Van Merri¨ enboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using rnn encoder-decoder for statistical ma- chine translation. arXiv preprint arXiv:1406.1078 (2014)

  6. [6]

    arXiv preprint pp

    Chollet, F.: Xception: Deep learning with depthwise separable convolutions. arXiv preprint pp. 1610–02357 (2017)

  7. [8]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., Tafjord, O.: Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018)

  8. [9]

    AI Magazine 37(1), 5–12 (2016)

    Clark, P., Etzioni, O.: My computer is an honor studentâĂ”but how intelligent is it? standardized tests as a measure of ai. AI Magazine 37(1), 5–12 (2016)

  9. [10]

    In: AAAI, pp

    Clark, P., Etzioni, O., Khot, T., Sabharwal, A., Tafjord, O., Turney, P.D., Khashabi, D.: Combining retrieval, statistics, and inference to answer elementary science questions. In: AAAI, pp. 2580–2586 (2016)

  10. [11]

    Journal of the American society for information science 41(6), 391–407 (1990)

    Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. Journal of the American society for information science 41(6), 391–407 (1990)

  11. [12]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  12. [13]

    Gated-Attention Readers for Text Comprehension

    Dhingra, B., Liu, H., Yang, Z., Cohen, W.W., Salakhutdinov, R.: Gated-attention readers for text comprehension. arXiv preprint arXiv:1606.01549 (2016)

  13. [14]

    Maxout Networks

    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. arXiv preprint arXiv:1302.4389 (2013)

  14. [15]

    In: Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference, pp

    Green Jr, B.F., Wolf, A.K., Chomsky, C., Laughery, K.: Baseball: an automatic question- answerer. In: Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference, pp. 219–224. ACM (1961)

  15. [16]

    In: Advances in Neural Information Processing Systems, pp

    Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blun- som, P.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, pp. 1693–1701 (2015)

  16. [17]

    The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations

    Hill, F., Bordes, A., Chopra, S., Weston, J.: The goldilocks principle: Reading children’s books with explicit memory representations. arXiv preprint arXiv:1511.02301 (2015)

  17. [18]

    natural language engineering 7(4), 275–300 (2001)

    Hirschman, L., Gaizauskas, R.: Natural language question answering: the view from here. natural language engineering 7(4), 275–300 (2001)

  18. [19]

    Neural computation 9(8), 1735–1780 (1997)

    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)

  19. [20]

    Adversarial Examples for Evaluating Reading Comprehension Systems

    Jia, R., Liang, P.: Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328 (2017)

  20. [21]

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

    Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: Triviaqa: A large scale distantly su- pervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 (2017)

  21. [22]

    Depthwise Separable Convolutions for Neural Machine Translation

    Kaiser, L., Gomez, A.N., Chollet, F.: Depthwise separable convolutions for neural machine translation. arXiv preprint arXiv:1706.03059 (2017)

  22. [23]

    Convolutional Neural Networks for Sentence Classification

    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)

  23. [24]

    Transactions of the Association of Computational Linguistics 6, 317–328 (2018)

    Koˇ cisk` y, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K.M., Melis, G., Grefenstette, E.: The narrativeqa reading comprehension challenge. Transactions of the Association of Computational Linguistics 6, 317–328 (2018)

  24. [25]

    RACE: Large-scale ReAding Comprehension Dataset From Examinations

    Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683 (2017)

  25. [26]

    In: Proceedings of the 5th international joint conference on Artificial intelligence-Volume 1, pp

    Lehnert, W.G.: A conceptual theory of question answering. In: Proceedings of the 5th international joint conference on Artificial intelligence-Volume 1, pp. 158–164. Morgan Kaufmann Publishers Inc. (1977)

  26. [27]

    Zero-Shot Relation Extraction via Reading Comprehension

    Levy, O., Seo, M., Choi, E., Zettlemoyer, L.: Zero-shot relation extraction via reading comprehension. arXiv preprint arXiv:1706.04115 (2017)

  27. [28]

    Generating Wikipedia by Summarizing Long Sequences

    Liu, P.J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., Shazeer, N.: Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198 (2018)

  28. [29]

    In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp

    Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp. 55–60 (2014)

  29. [30]

    Pointer Sentinel Mixture Models

    Merity, S., Xiong, C., Bradbury, J., Socher, R.: Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 (2016)

  30. [31]

    Efficient Estimation of Word Representations in Vector Space

    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  31. [32]

    MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

    Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., Deng, L.: Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016) Machine Reading Comprehension: a Literature Review 45

  32. [33]

    MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge

    Ostermann, S., Modi, A., Roth, M., Thater, S., Pinkal, M.: Mcscript: A novel dataset for assessing machine comprehension using script knowledge. arXiv preprint arXiv:1803.05223 (2018)

  33. [34]

    In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp

    Pennington, J., Socher, R., Manning, C.: Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532–1543 (2014)

  34. [35]

    Deep contextualized word representations

    Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)

  35. [36]

    Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understand- ing with unsupervised learning. Tech. rep., Technical report, OpenAI (2018)

  36. [37]

    Know What You Don't Know: Unanswerable Questions for SQuAD

    Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822 (2018)

  37. [38]

    SQuAD: 100,000+ Questions for Machine Comprehension of Text

    Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)

  38. [39]

    CoQA: A Conversational Question Answering Challenge

    Reddy, S., Chen, D., Manning, C.D.: Coqa: A conversational question answering challenge. arXiv preprint arXiv:1808.07042 (2018)

  39. [41]

    In: Proceedings of the 2013 Conference on Em- pirical Methods in Natural Language Processing, pp

    Richardson, M., Burges, C.J., Renshaw, E.: Mctest: A challenge dataset for the open- domain machine comprehension of text. In: Proceedings of the 2013 Conference on Em- pirical Methods in Natural Language Processing, pp. 193–203 (2013)

  40. [42]

    In: Herbert Robbins Selected Papers, pp

    Robbins, H., Monro, S.: A stochastic approximation method. In: Herbert Robbins Selected Papers, pp. 102–109. Springer (1985)

  41. [43]

    Reasoning about Entailment with Neural Attention

    Rockt¨ aschel, T., Grefenstette, E., Hermann, K.M., Koˇ cisk` y, T., Blunsom, P.: Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664 (2015)

  42. [44]

    Bidirectional Attention Flow for Machine Comprehension

    Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)

  43. [45]

    In: EMNLP (2018)

    Shankar, S., Garg, S., Sarawagi, S.: Surprisingly easy hard-attention for sequence to se- quence learning. In: EMNLP (2018)

  44. [46]

    Labeled Memory Networks for Online Model Adaptation

    Shankar, S., Sarawagi, S.: Label organized memory augmented neural network. CoRR abs/1707.01461 (2017)

  45. [47]

    In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp

    Shen, Y., Huang, P.S., Gao, J., Chen, W.: Reasonet: Learning to stop reading in machine comprehension. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1047–1055. ACM (2017)

  46. [48]

    Simmons, R.F.: Answering english questions by computer: a survey. Tech. rep., SYSTEM DEVELOPMENT CORP SANTA MONICA CALIF (1964)

  47. [49]

    Highway Networks

    Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway networks. arXiv preprint arXiv:1505.00387 (2015)

  48. [50]

    Journalism Bulletin 30(4), 415–433 (1953)

    Taylor, W.L.: âĂIJcloze procedureâĂİ: A new tool for measuring readability. Journalism Bulletin 30(4), 415–433 (1953)

  49. [51]

    NewsQA: A Machine Comprehension Dataset

    Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., Suleman, K.: Newsqa: A machine comprehension dataset. arXiv preprint arXiv:1611.09830 (2016)

  50. [52]

    In: Advances in Neural Information Processing Systems, pp

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, /suppress L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

  51. [53]

    Pointer Networks

    Vinyals, O., Fortunato, M., Jaitly, N.: Pointer Networks. arXiv e-prints arXiv:1506.03134 (2015)

  52. [54]

    In: Advances in Neural Informa- tion Processing Systems, pp

    Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Advances in Neural Informa- tion Processing Systems, pp. 2692–2700 (2015)

  53. [55]

    In: Proceedings of the 21st International Conference on World Wide Web, pp

    Vrandeˇ ci´ c, D.: Wikidata: A new platform for collaborative data collection. In: Proceedings of the 21st International Conference on World Wide Web, pp. 1063–1064. ACM (2012)

  54. [56]

    Towards Inference-Oriented Reading Comprehension: ParallelQA

    Wadhwa, S., Embar, V., Grabmair, M., Nyberg, E.: Towards inference-oriented reading comprehension: Parallelqa. arXiv preprint arXiv:1805.03830 (2018)

  55. [57]

    Learning Natural Language Inference with LSTM

    Wang, S., Jiang, J.: Learning natural language inference with lstm. arXiv preprint arXiv:1512.08849 (2015)

  56. [58]

    Machine Comprehension Using Match-LSTM and Answer Pointer

    Wang, S., Jiang, J.: Machine comprehension using match-lstm and answer pointer. arXiv preprint arXiv:1608.07905 (2016)

  57. [59]

    In: Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), vol

    Wang, W., Yang, N., Wei, F., Chang, B., Zhou, M.: Gated self-matching networks for reading comprehension and question answering. In: Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 189–198 (2017) 46 Xin Zhang et al

  58. [62]

    Making Neural QA as Simple as Possible but not Simpler

    Weissenborn, D., Wiese, G., Seiffe, L.: Making neural qa as simple as possible but not simpler. arXiv preprint arXiv:1703.04816 (2017)

  59. [63]

    Transactions of the Association of Computational Linguistics 6, 287–302 (2018)

    Welbl, J., Stenetorp, P., Riedel, S.: Constructing datasets for multi-hop reading compre- hension across documents. Transactions of the Association of Computational Linguistics 6, 287–302 (2018)

  60. [64]

    Memory Networks

    Weston, J., Chopra, S., Bordes, A.: Memory networks. CoRR abs/1410.3916 (2014)

  61. [65]

    Cognitive psychology 3(1), 1–191 (1972)

    Winograd, T.: Understanding natural language. Cognitive psychology 3(1), 1–191 (1972)

  62. [66]

    In: Proceedings of the June 4-8, 1973, national computer conference and exposition, pp

    Woods, W.A.: Progress in natural language understanding: an application to lunar geology. In: Proceedings of the June 4-8, 1973, national computer conference and exposition, pp. 441–450. ACM (1973)

  63. [67]

    Information Retrieval 13(3), 254–270 (2010)

    Wu, Q., Burges, C.J., Svore, K.M., Gao, J.: Adapting boosting for information retrieval measures. Information Retrieval 13(3), 254–270 (2010)

  64. [68]

    Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

    Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al.: Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

  65. [69]

    Large-scale Cloze Test Dataset Created by Teachers

    Xie, Q., Lai, G., Dai, Z., Hovy, E.: Large-scale cloze test dataset designed by teachers. arXiv preprint arXiv:1711.03225 (2017)

  66. [70]

    Dynamic Coattention Networks For Question Answering

    Xiong, C., Zhong, V., Socher, R.: Dynamic coattention networks for question answering. arXiv preprint arXiv:1611.01604 (2016)

  67. [71]

    In: International conference on machine learning, pp

    Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., Bengio, Y.: Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, pp. 2048–2057 (2015)

  68. [72]

    In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol

    Yih, W.t., Chang, M.W., Meek, C., Pastusiak, A.: Question answering using enhanced lexical semantic models. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1744–1753 (2013)

  69. [73]

    QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

    Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., Le, Q.V.: Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09541 (2018)