Machine Reading Comprehension: a Literature Review
Pith reviewed 2026-05-25 13:13 UTC · model grok-4.3
The pith
A review organizes machine reading comprehension work by comparing datasets and outlining techniques.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes that recent advances in machine reading comprehension can be summarized by listing and comparing the characteristics of its corpora and by describing the main ideas of its typical techniques.
What carries the argument
The two-part structure that separates corpus characteristics from technique ideas, which the review uses to organize its coverage of the field.
If this is right
- Researchers gain a structured way to select corpora based on their listed characteristics for new experiments.
- The descriptions of typical techniques provide a baseline for understanding how models process reading comprehension tasks.
- The comparisons across corpora can highlight differences in difficulty, size, or question types that affect model evaluation.
Where Pith is reading between the lines
- The review's structure could guide the design of new datasets that fill gaps in the compared characteristics.
- Readers might extend the technique descriptions to test how well newer methods fit the outlined main ideas.
Load-bearing premise
The selected corpora and techniques are representative of the broader field and the comparisons and descriptions are comprehensive and unbiased.
What would settle it
Identification of a widely used MRC corpus or technique that was omitted or whose description differs substantially from the review's account in a way that changes the overall picture.
read the original abstract
Machine reading comprehension aims to teach machines to understand a text like a human and is a new challenging direction in Artificial Intelligence. This article summarizes recent advances in MRC, mainly focusing on two aspects (i.e., corpus and techniques). The specific characteristics of various MRC corpus are listed and compared. The main ideas of some typical MRC techniques are also described.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript is a literature review on Machine Reading Comprehension (MRC). It summarizes recent advances by focusing on two aspects: corpora and techniques. It lists and compares the specific characteristics of various MRC corpora and describes the main ideas of some typical MRC techniques.
Significance. If the selected corpora and techniques are representative and the descriptions accurate, the review could serve as a useful organizing reference for researchers entering the MRC area circa 2019, particularly by collating corpus statistics and high-level technique overviews in one place.
minor comments (2)
- [Abstract] The abstract states the focus on 'corpus and techniques' but does not specify the time window covered or the criteria used to select the 'various MRC corpus' and 'typical MRC techniques,' which would help readers assess scope.
- No explicit statement appears on how the review ensures completeness or avoids selection bias in the corpora and techniques presented.
Simulated Author's Rebuttal
We thank the referee for the positive review and the recommendation to accept the manuscript. The comments confirm that the survey can serve as a useful reference by collating corpus statistics and technique overviews.
Circularity Check
No significant circularity; purely descriptive review
full rationale
The manuscript is a literature review whose purpose is to summarize selected MRC corpora and techniques. It contains no original equations, derivations, fitted parameters, predictions, or theorems. No load-bearing claim reduces by construction to its own inputs or to a self-citation chain. The paper simply lists and compares external work; representativeness is an external concern, not an internal circularity. This is the expected finding for a descriptive survey.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[2]
Journal of machine learning research 3(Feb), 1137–1155 (2003)
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. Journal of machine learning research 3(Feb), 1137–1155 (2003)
work page 2003
-
[3]
Artificial intelligence 8(2), 155–173 (1977)
Bobrow, D.G., Kaplan, R.M., Kay, M., Norman, D.A., Thompson, H., Winograd, T.: Gus, a frame-driven dialog system. Artificial intelligence 8(2), 155–173 (1977)
work page 1977
-
[4]
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Chen, D., Bolton, J., Manning, C.D.: A thorough examination of the cnn/daily mail read- ing comprehension task. arXiv preprint arXiv:1606.02858 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[5]
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., Van Merri¨ enboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using rnn encoder-decoder for statistical ma- chine translation. arXiv preprint arXiv:1406.1078 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[6]
Chollet, F.: Xception: Deep learning with depthwise separable convolutions. arXiv preprint pp. 1610–02357 (2017)
work page 2017
-
[8]
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., Tafjord, O.: Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[9]
AI Magazine 37(1), 5–12 (2016)
Clark, P., Etzioni, O.: My computer is an honor studentâĂ”but how intelligent is it? standardized tests as a measure of ai. AI Magazine 37(1), 5–12 (2016)
work page 2016
-
[10]
Clark, P., Etzioni, O., Khot, T., Sabharwal, A., Tafjord, O., Turney, P.D., Khashabi, D.: Combining retrieval, statistics, and inference to answer elementary science questions. In: AAAI, pp. 2580–2586 (2016)
work page 2016
-
[11]
Journal of the American society for information science 41(6), 391–407 (1990)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. Journal of the American society for information science 41(6), 391–407 (1990)
work page 1990
-
[12]
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[13]
Gated-Attention Readers for Text Comprehension
Dhingra, B., Liu, H., Yang, Z., Cohen, W.W., Salakhutdinov, R.: Gated-attention readers for text comprehension. arXiv preprint arXiv:1606.01549 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[14]
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. arXiv preprint arXiv:1302.4389 (2013)
work page internal anchor Pith review Pith/arXiv arXiv 2013
-
[15]
In: Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference, pp
Green Jr, B.F., Wolf, A.K., Chomsky, C., Laughery, K.: Baseball: an automatic question- answerer. In: Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference, pp. 219–224. ACM (1961)
work page 1961
-
[16]
In: Advances in Neural Information Processing Systems, pp
Hermann, K.M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., Blun- som, P.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, pp. 1693–1701 (2015)
work page 2015
-
[17]
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
Hill, F., Bordes, A., Chopra, S., Weston, J.: The goldilocks principle: Reading children’s books with explicit memory representations. arXiv preprint arXiv:1511.02301 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[18]
natural language engineering 7(4), 275–300 (2001)
Hirschman, L., Gaizauskas, R.: Natural language question answering: the view from here. natural language engineering 7(4), 275–300 (2001)
work page 2001
-
[19]
Neural computation 9(8), 1735–1780 (1997)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)
work page 1997
-
[20]
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., Liang, P.: Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[21]
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: Triviaqa: A large scale distantly su- pervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[22]
Depthwise Separable Convolutions for Neural Machine Translation
Kaiser, L., Gomez, A.N., Chollet, F.: Depthwise separable convolutions for neural machine translation. arXiv preprint arXiv:1706.03059 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[23]
Convolutional Neural Networks for Sentence Classification
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[24]
Transactions of the Association of Computational Linguistics 6, 317–328 (2018)
Koˇ cisk` y, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K.M., Melis, G., Grefenstette, E.: The narrativeqa reading comprehension challenge. Transactions of the Association of Computational Linguistics 6, 317–328 (2018)
work page 2018
-
[25]
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[26]
In: Proceedings of the 5th international joint conference on Artificial intelligence-Volume 1, pp
Lehnert, W.G.: A conceptual theory of question answering. In: Proceedings of the 5th international joint conference on Artificial intelligence-Volume 1, pp. 158–164. Morgan Kaufmann Publishers Inc. (1977)
work page 1977
-
[27]
Zero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., Zettlemoyer, L.: Zero-shot relation extraction via reading comprehension. arXiv preprint arXiv:1706.04115 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[28]
Generating Wikipedia by Summarizing Long Sequences
Liu, P.J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., Shazeer, N.: Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[29]
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp. 55–60 (2014)
work page 2014
-
[30]
Pointer Sentinel Mixture Models
Merity, S., Xiong, C., Bradbury, J., Socher, R.: Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[31]
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
work page internal anchor Pith review Pith/arXiv arXiv 2013
-
[32]
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., Deng, L.: Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016) Machine Reading Comprehension: a Literature Review 45
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[33]
MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge
Ostermann, S., Modi, A., Roth, M., Thater, S., Pinkal, M.: Mcscript: A novel dataset for assessing machine comprehension using script knowledge. arXiv preprint arXiv:1803.05223 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[34]
Pennington, J., Socher, R., Manning, C.: Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532–1543 (2014)
work page 2014
-
[35]
Deep contextualized word representations
Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[36]
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understand- ing with unsupervised learning. Tech. rep., Technical report, OpenAI (2018)
work page 2018
-
[37]
Know What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[38]
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[39]
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., Manning, C.D.: Coqa: A conversational question answering challenge. arXiv preprint arXiv:1808.07042 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[41]
In: Proceedings of the 2013 Conference on Em- pirical Methods in Natural Language Processing, pp
Richardson, M., Burges, C.J., Renshaw, E.: Mctest: A challenge dataset for the open- domain machine comprehension of text. In: Proceedings of the 2013 Conference on Em- pirical Methods in Natural Language Processing, pp. 193–203 (2013)
work page 2013
-
[42]
In: Herbert Robbins Selected Papers, pp
Robbins, H., Monro, S.: A stochastic approximation method. In: Herbert Robbins Selected Papers, pp. 102–109. Springer (1985)
work page 1985
-
[43]
Reasoning about Entailment with Neural Attention
Rockt¨ aschel, T., Grefenstette, E., Hermann, K.M., Koˇ cisk` y, T., Blunsom, P.: Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[44]
Bidirectional Attention Flow for Machine Comprehension
Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[45]
Shankar, S., Garg, S., Sarawagi, S.: Surprisingly easy hard-attention for sequence to se- quence learning. In: EMNLP (2018)
work page 2018
-
[46]
Labeled Memory Networks for Online Model Adaptation
Shankar, S., Sarawagi, S.: Label organized memory augmented neural network. CoRR abs/1707.01461 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[47]
Shen, Y., Huang, P.S., Gao, J., Chen, W.: Reasonet: Learning to stop reading in machine comprehension. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1047–1055. ACM (2017)
work page 2017
-
[48]
Simmons, R.F.: Answering english questions by computer: a survey. Tech. rep., SYSTEM DEVELOPMENT CORP SANTA MONICA CALIF (1964)
work page 1964
-
[49]
Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway networks. arXiv preprint arXiv:1505.00387 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[50]
Journalism Bulletin 30(4), 415–433 (1953)
Taylor, W.L.: âĂIJcloze procedureâĂİ: A new tool for measuring readability. Journalism Bulletin 30(4), 415–433 (1953)
work page 1953
-
[51]
NewsQA: A Machine Comprehension Dataset
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., Suleman, K.: Newsqa: A machine comprehension dataset. arXiv preprint arXiv:1611.09830 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[52]
In: Advances in Neural Information Processing Systems, pp
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, /suppress L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
work page 2017
-
[53]
Vinyals, O., Fortunato, M., Jaitly, N.: Pointer Networks. arXiv e-prints arXiv:1506.03134 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[54]
In: Advances in Neural Informa- tion Processing Systems, pp
Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Advances in Neural Informa- tion Processing Systems, pp. 2692–2700 (2015)
work page 2015
-
[55]
In: Proceedings of the 21st International Conference on World Wide Web, pp
Vrandeˇ ci´ c, D.: Wikidata: A new platform for collaborative data collection. In: Proceedings of the 21st International Conference on World Wide Web, pp. 1063–1064. ACM (2012)
work page 2012
-
[56]
Towards Inference-Oriented Reading Comprehension: ParallelQA
Wadhwa, S., Embar, V., Grabmair, M., Nyberg, E.: Towards inference-oriented reading comprehension: Parallelqa. arXiv preprint arXiv:1805.03830 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[57]
Learning Natural Language Inference with LSTM
Wang, S., Jiang, J.: Learning natural language inference with lstm. arXiv preprint arXiv:1512.08849 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[58]
Machine Comprehension Using Match-LSTM and Answer Pointer
Wang, S., Jiang, J.: Machine comprehension using match-lstm and answer pointer. arXiv preprint arXiv:1608.07905 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[59]
Wang, W., Yang, N., Wei, F., Chang, B., Zhou, M.: Gated self-matching networks for reading comprehension and question answering. In: Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 189–198 (2017) 46 Xin Zhang et al
work page 2017
-
[62]
Making Neural QA as Simple as Possible but not Simpler
Weissenborn, D., Wiese, G., Seiffe, L.: Making neural qa as simple as possible but not simpler. arXiv preprint arXiv:1703.04816 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[63]
Transactions of the Association of Computational Linguistics 6, 287–302 (2018)
Welbl, J., Stenetorp, P., Riedel, S.: Constructing datasets for multi-hop reading compre- hension across documents. Transactions of the Association of Computational Linguistics 6, 287–302 (2018)
work page 2018
-
[64]
Weston, J., Chopra, S., Bordes, A.: Memory networks. CoRR abs/1410.3916 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[65]
Cognitive psychology 3(1), 1–191 (1972)
Winograd, T.: Understanding natural language. Cognitive psychology 3(1), 1–191 (1972)
work page 1972
-
[66]
In: Proceedings of the June 4-8, 1973, national computer conference and exposition, pp
Woods, W.A.: Progress in natural language understanding: an application to lunar geology. In: Proceedings of the June 4-8, 1973, national computer conference and exposition, pp. 441–450. ACM (1973)
work page 1973
-
[67]
Information Retrieval 13(3), 254–270 (2010)
Wu, Q., Burges, C.J., Svore, K.M., Gao, J.: Adapting boosting for information retrieval measures. Information Retrieval 13(3), 254–270 (2010)
work page 2010
-
[68]
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al.: Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[69]
Large-scale Cloze Test Dataset Created by Teachers
Xie, Q., Lai, G., Dai, Z., Hovy, E.: Large-scale cloze test dataset designed by teachers. arXiv preprint arXiv:1711.03225 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[70]
Dynamic Coattention Networks For Question Answering
Xiong, C., Zhong, V., Socher, R.: Dynamic coattention networks for question answering. arXiv preprint arXiv:1611.01604 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[71]
In: International conference on machine learning, pp
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., Bengio, Y.: Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, pp. 2048–2057 (2015)
work page 2048
-
[72]
Yih, W.t., Chang, M.W., Meek, C., Pastusiak, A.: Question answering using enhanced lexical semantic models. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1744–1753 (2013)
work page 2013
-
[73]
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., Le, Q.V.: Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09541 (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.