Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

AmazonQA: A Review-Based Question Answering Task

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes review-based question answering and introduces AmazonQA, a dataset of 923k questions, 3.6M answers, and 14M reviews across 156k products, arguing that many product questions can be answered by synthesizing information…

desk verdict A genuinely useful real-world QA dataset, but the answerability layer and a couple of overclaims will need attention from a careful referee. read the letter →

arxiv 1908.04364 v2 pith:ATQ2GCCH submitted 2019-08-12 cs.CL cs.IR

Mansi Gupta , Nitish Kulkarni , Raghuveer Chanda , Anirudha Rayasam , Zachary C Lipton This is my paper · ORCID
classification cs.CLcs.IR
keywords questionansweringproductreviewscommunitydatasetreadingcomprehensionanswerabilityclassificatione-commercebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Customers ask thousands of questions on product pages, but answers often take days. The paper proposes that many of those questions could be answered immediately by reading the product's reviews, and formalizes this as review-based question answering: given a question and a review corpus, generate a natural-language answer. To support the task it introduces AmazonQA, with 923k questions, 3.6M answers, and 14M reviews across 156k products, and an answerability classifier that marks roughly 570k question-context pairs as answerable. The paper's claim is that this is the largest public QA dataset with descriptive (free-form) answers and the first large-scale benchmark where questions, supporting passages, and answers all come from real human interactions.

What carries the argument

The load-bearing mechanism is a three-stage pipeline. First, reviews are chunked into snippets and ranked by BM25 to give each question its top ten candidate passages. Second, a logistic regression classifier, trained on crowd labels with tf-idf and GloVe-derived features, decides whether the reviews answer the question. Third, either an LSTM sequence-to-sequence model generates an answer conditioned on question and review snippets, or a span-based reader (R-Net) extracts an answer from heuristically generated spans. The answerability classifier is what converts raw crawled data into a supervised learning resource.

What would settle it

Take a random sample of, say, 400 questions from the 570k 'answerable' pool, have independent experts label whether the provided review snippets truly answer each question, and compare against the classifier's labels; if expert precision is substantially below the reported 72% or errors correlate with question type or length, the dataset's central resource is not trustworthy.

Watch

Extended reading notes

Core claim

The central discovery is a new benchmark and task formulation. The authors extract real customer questions, community answers, and product reviews from a large crawl, filter and clean them, and add answerability labels so that a QA system can first decide whether the reviews contain an answer and then generate one. They show that standard span-based reading comprehension models underperform simple retrieval heuristics like taking the top BM25-ranked sentence, and that conditioning answer generation on both the question and reviews lowers perplexity versus question-only or no-conditioning models. This establishes the task as challenging and not solved by existing methods.

Load-bearing premise

The full dataset's answerability labels are produced by a logistic regression classifier trained on only 3,297 crowd-annotated question-context pairs and checked on 137 expert labels; if that classifier is systematically wrong about which questions can be answered from reviews, the 570k pairs that form the core training resource are biased.

Editorial extensions

If this is right

  • A deployed system could intercept a new product question, classify it as answerable from reviews, and return a provisional answer immediately instead of making the user wait days for a community response.
  • Because the questions, evidence, and answers are all real human interactions rather than artificially constructed, models trained on AmazonQA may reflect the actual difficulty of product QA better than synthetic benchmarks.
  • The product-based train/dev/test split forces models to answer unseen products from reviews alone, so performance measures genuine reading and synthesis rather than memorization of product-specific answers.
  • The result that top-sentence retrieval beats neural reading comprehension indicates that effective review-based QA may require better answer synthesis, not just better evidence selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because yes/no questions make up 15% of the data, a system that answers them correctly might score poorly on BLEU/ROUGE; an accuracy metric on that subset would give a clearer signal than token-overlap metrics alone.
  • The answerability labels are the dataset's linchpin, and they come from a classifier applied at scale; users of AmazonQA should verify label quality on their own random sample before trusting the 570k answerable pairs.
  • The 17 product categories allow a natural out-of-domain experiment: train on some categories and test on held-out ones, which could reveal whether review-based QA systems learn transferable reading behavior.
  • Observing that even random review sentences score close to human answers suggests that many reference answers share vocabulary with the reviews; evaluation may need to control for answer redundancy to measure true understanding.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces AmazonQA, a large-scale question-answering dataset built from Amazon product reviews, questions, and community answers. The authors propose the task of review-based QA: given a product's reviews and a question, generate a natural-language answer. The dataset comprises 923k questions, 3.6M answers, and 14M reviews across 156k products. To support supervised training, the authors extract top-10 BM25 review snippets per question, build an answerability classifier trained on 3,297 crowd-annotated instances (with a 137-instance expert test set), and filter to roughly 570k answerable question-answer-review instances. They split the data by product into train/development/test sets, and provide baseline experiments: three LSTM language models, heuristic sentence/review retrievers, and an R-Net span-based reading-comprehension model. The central claims are that AmazonQA is the largest public QA dataset with descriptive answers and the first large-scale benchmark where questions, support, and answers all come from real Amazon interactions.

Significance. If the dataset is sound, it is a valuable resource for the QA community. Its strengths are the realism of the data (questions, answers, and reviews are all organic user-generated content), the large scale, the product-based split that prevents answer memorization from question-product leakage, and the public release of data and baseline implementations. The paper also explicitly includes ablation-style language models (P(a), P(a|q), P(a|q,R)) to verify that reviews add information beyond the question alone. However, the paper's utility as a reading-comprehension benchmark rests on two load-bearing assumptions: that the answerability labels are reliable at scale, and that the provided review snippets actually contain the supporting evidence for answerable questions. The current manuscript provides only limited evidence for both, and some of the evidence it does provide (Table 4) points in the opposite direction. These issues are correctable with additional validation and revised claims, but they are central to the benchmark's validity rather than mere presentation concerns.

major comments (4)
  1. [Abstract and §3.2] The abstract and §3.2 state that the answerability classifier 'marks roughly 570K pairs as answerable out of total of 923K question with 72% precision,' but Table 3 reports the classifier's precision as 0.67 on the expert test set. The 72% figure is not defined as a different operating point or a different evaluation subset. Because the abstract advertises precision as a headline property of the released dataset, this inconsistency must be resolved by reporting the exact threshold and evaluation set that produce each number.
  2. [§2 and Table 4] Section 2 claims: 'We ensure that for any answerable question, the associated snippets contain all the required supporting information.' No construction step enforces this guarantee. The answerability label is produced by a logistic-regression classifier over eight features computed from the top-5 BM25 snippets (§4.2), not by a verified entailment check against the full review text or the provided 10 snippets. Table 4 directly measures the retrieval component and reports that only 0.58 of 100 expert-annotated questions are answerable from IR-selected sentences. Since the classifier's context is the same retrieval pipeline, a substantial fraction of the 570k 'answerable' instances likely lack the supporting evidence in the provided snippets. This invalidates the closed-world premise and means the BLEU/ROUGE baselines in Table 7 penalize models for information that is not in the context. The authors should either enforce the closed-world property through a verifiable filtering step, or explicitly reframe the dataset as open-world with noisy evidence and quantify the resulting label noise.
  3. [§4.2] The answerability classifier is trained on 3,297 crowd labels and then applied to the full 923k corpus with no additional validation on the target distribution. The only held-out expert evaluation consists of 137 test instances, which is too small to estimate error rates across product categories and question types. Because the utility of the 570k training instances depends on label quality, the paper should report category-wise precision/recall, the sensitivity of the labels to the decision threshold, and an additional expert-annotated sample from the deployed classifier's output to measure how label noise propagates into the training set.
  4. [§6, Table 7] The evaluation metrics are applied uniformly to yes/no and descriptive questions, although the authors acknowledge in the text that BLEU/ROUGE are 'not the best metrics' for yes/no questions. The observation that a random sentence achieves nearly the same BLEU/ROUGE as Amazon user answers (78.56 vs. 80.88 BLEU-1) suggests these metrics may not be informative for this dataset. The paper proposes a combined metric set only as future work; given that the benchmark's usefulness depends on meaningful evaluation, the authors should at least report accuracy or F1 for the yes/no subset, or provide a more detailed analysis of metric validity, before claiming that the baselines demonstrate the task's difficulty.
minor comments (6)
  1. [§3.1] The snippet construction description ('snippets of length 100, or to the end of a sentence boundary, whichever is greater') is ambiguous: it should specify whether the snippet length is measured in tokens or characters, and how the 'greater' comparison is made between a fixed length and a sentence boundary.
  2. [Table 1] The entry for RACE lists '# Qs' as '97l', which appears to be a typo for a number with a letter 'l' instead of a digit; this should be corrected.
  3. [Table 4] The caption should state explicitly that the evaluation is on a 100-question test set and that 'IR' refers to BM25 sentence retrieval, so that the reader can connect this table to the snippet-construction pipeline in §3.1.
  4. [§5.2] The R-Net citation [Group, 2017] is a blog post rather than a peer-reviewed publication; the authors should cite the original technical report or a published version if one exists.
  5. [Table 7] The last column mixes two different quantities — P(a|q) and P(a|q,R) perplexities — under a single header, which is confusing; the column should be split or the header should clearly indicate that the two numbers correspond to different models.
  6. [Abstract and §1] The abstract says 'we propose a method' but the paper primarily introduces a dataset and baseline models; the phrasing overstates the methodological novelty and should be aligned with the actual contribution.

Circularity Check

1 steps flagged · score 3.0 of 10

One evaluation baseline is a test-time oracle: 'Top-1 Using BLEU' selects a sentence with the gold answer and is then scored against the same gold answer; the dataset construction and remaining baselines are not circular.

  1. fitted input called prediction [Section 5.2 (Span Heuristics) and Section 6 (Heuristic-based answers), Table 7]
    "We then rank the spans based on heuristics such as (i) BLEU-2 [Papineni et al., 2002] or ROUGE [Lin, 2004] with the actual answers, and (ii) BM25 based IR score match with the question. ... we consider the top-ranked sentence based on IR and BLEU-2 scores as an answer to the question."

    The 'Top-1 Using BLEU' baseline in Table 7 selects the review sentence that maximizes BLEU-2 against the gold reference answer, then reports BLEU-1/2/3/4 and ROUGE against that same gold reference. Thus the gold answer is an input to the output selection, and the metric evaluates the output against the same input that generated it. By construction, this is an oracle upper bound on sentence overlap, not a prediction of an unseen answer. The paper explicitly frames these as experiments 'to predict the answer,' so the row is a fitted-input-called-prediction rather than an independent baseline. The conclusion that 'sentence-based heuristic baselines are higher than those of the span-based model' partially relies on this constructed score.

full rationale

The paper's central deliverable is a dataset, not a derived prediction, and the construction is largely self-contained: the answerability classifier is trained on 3,297 crowd labels with features from snippets and evaluated on an independently expert-annotated holdout (Table 3), the train/dev/test split is by product, and the language-model perplexities are evaluated on held-out answers. The only clear circular step is the 'Top-1 Using BLEU' heuristic, which uses the gold answer to choose the sentence and is then scored against the same gold answer; this is a minor evaluation artifact and does not affect the dataset itself. The paper's closed-world claim in Section 2 ('We ensure that for any answerable question, the associated snippets contain all the required supporting information') is asserted without an enforcement mechanism and is contradicted in spirit by Table 4's IR span answerability of 0.58, but that is a validity/quality concern, not circularity. No load-bearing self-citation or imported uniqueness theorem is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim is a dataset construction claim, not a derivation. The only numerical choice that changes the labels is the classifier decision threshold (0.6), which sets the answerable/unanswerable boundary. The paper's answerability notion rests on three domain assumptions: crowd labels approximate ground truth, the 3,297-sample classifier generalizes to 923k cases, and BM25-selected snippets preserve answerability. The evaluation rests on the additional assumption that BLEU/ROUGE against Amazon user answers is a valid measure. No invented entities are introduced: the dataset is an artifact, not a postulated entity, and there are no new forces, particles, dimensions, or conserved quantities.

free parameters (1)
  • answerability classifier decision threshold = 0.6
    Set for the logistic regression in Section 4.2 to convert the posterior into answerable/non-answerable labels; determines which 570k of 923k questions enter the QA training set.
assumptions (4)
  • domain assumption MTurk workers' Yes/Somewhat/No labels on question-context pairs correctly reflect whether the reviews answer the question.
    Used in Section 4.1 to create the training labels for the answerability classifier; the paper reports F1=0.73 for workers versus 0.86 for experts, so the main label source is less accurate than experts.
  • domain assumption The logistic regression answerability classifier trained on 3,297 retained crowd examples generalizes to the 923k-question corpus.
    Applied in Section 4.2 to classify every question across 156k products; there is no validation set from this full distribution.
  • domain assumption The top-10 BM25 review snippets contain all the information needed to answer any question that is answerable from the full set of reviews.
    Snippet extraction in Section 3.1 feeds both answerability classification and QA; if the retrieval misses the answer sentence, a genuinely answerable question can be labeled unanswerable.
  • domain assumption BLEU and ROUGE scores against Amazon user answers capture answer quality for descriptive and yes/no answers.
    Used as the automatic evaluation in Table 7; the authors themselves note the mismatch for yes/no questions in Section 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AmazonQA: A Review-Based Question Answering Task." pith.science (2026). https://pith.science/paper/ATQ2GCCH

@misc{pith2026190804364,
  author       = {Pith},
  title        = {Pith review of: AmazonQA: A Review-Based Question Answering Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ATQ2GCCH}},
  note         = {Machine review of arXiv:1908.04364}
}
read the original abstract

Every day, thousands of customers post questions on Amazon product pages. After some time, if they are fortunate, a knowledgeable customer might answer their question. Observing that many questions can be answered based upon the available product reviews, we propose the task of review-based QA. Given a corpus of reviews and a question, the QA system synthesizes an answer. To this end, we introduce a new dataset and propose a method that combines information retrieval techniques for selecting relevant reviews (given a question) and "reading comprehension" models for synthesizing an answer (given a question and review). Our dataset consists of 923k questions, 3.6M answers and 14M reviews across 156k products. Building on the well-known Amazon dataset, we collect additional annotations, marking each question as either answerable or unanswerable based on the available reviews. A deployed system could first classify a question as answerable and then attempt to generate an answer. Notably, unlike many popular QA datasets, here, the questions, passages, and answers are all extracted from real human interactions. We evaluate numerous models for answer generation and propose strong baselines, demonstrating the challenging nature of this new task.

Figures

Figures reproduced from arXiv: 1908.04364 by the authors.

Figure 1
Figure 1. A community question-answering platform that provides [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Left to Right: Question length distribution, Answer length distribution and Review length distribution for Answerable Questions [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A sample instance from the AmazonQA dataset class are shown in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Distribution of most frequent first 3 words in the questions of [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Question Type Distribution inter-annotator agreement. The responses to the decoy ques￾tions allow us to compute a lower bound of a worker’s per￾formance. With this guiding metric, we detail our most effi￾cient design template below. An example question with cor￾respond…
Figure 6
Figure 6. Figure 6: Example of an AMT page with a question, corresponding [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]
Figure 7
Figure 7. Figure 7: A schematic overview of the model P(a | q, R). The P(a | q) and P(a) models are special cases where the review repre￾sentation or both review and question are absent. All encoders and decoders are LSTM-based. 5.2 Span-based QA Model To assess the performance of span-ba…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Representational Learning of Foundation Models for Recommendation

    cs.IR 2025-06 conditional novelty 6.0 of 10

    A single recommendation model with task-aware Mixture of Low-rank Experts and convergence-based sample scheduling beats baselines on a new 13-task benchmark.

  2. QQSUM: A Novel Task and Model of Quantitative Query-Focused Summarization for Review-based Product Question Answering

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A new task and model that generates query-focused bullet-point summaries of product reviews with prevalence counts for each key point.

  3. PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models

    cs.CL 2025-04 conditional novelty 3.0 of 10

    A survey that organizes PEFT methods into additive, selective, reparameterized, hybrid, and unified families, but with no new method or verified experiments.

Reference graph

Works this paper leans on

23 extracted references · 18 canonical work pages · cited by 3 Pith papers

  1. [1]

    Approximate is better than “exact” for interval estima- tion of binomial proportions

    [Agresti and Coull, 1998] Alan Agresti and Brent A Coull. Approximate is better than “exact” for interval estima- tion of binomial proportions. The American Statistician , 52(2):119–126,

    work page 1998

  2. [6]

    Annotation artifacts in natural language inference data

    [Gururangan et al., 2018] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bow- man, and Noah A Smith. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), v...

    work page 2018

  3. [7]

    Summariza- tion of yes/no questions using a feature function model

    [He and Dai, 2011] Jing He and Decheng Dai. Summariza- tion of yes/no questions using a feature function model. In Asian Conference on Machine Learning , pages 351–366,

    work page 2011

  4. [9]

    How much reading does reading comprehension require? a critical investigation of popular benchmarks

    †https://github.com/amazonqa/amazonqa [Kaushik and Lipton, 2018] Divyansh Kaushik and Zachary C Lipton. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Empirical Methods in Natural Language Processing (EMNLP),

    work page 2018

  5. [10]

    The narrativeqa reading comprehension challenge

    [Koˇcisk`y et al., 2018] Tom´aˇs Ko ˇcisk`y, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, G ´aabor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association of Computational Linguistics, 6:317–328,

    work page 2018

  6. [11]

    Natural questions: a benchmark for question answering research

    [Kwiatkowski et al., 2019] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Asso- ciation for Computational Linguistics, 7:453–466,

    work page 2019

  7. [13]

    Rouge: A package for automatic evaluation of summaries

    [Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out,

    work page 2004

  8. [16]

    Bleu: a method for auto- matic evaluation of machine translation

    [Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for auto- matic evaluation of machine translation. InProceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics,

    work page 2002

Show all 23 references
  1. [19]

    Know what you don’t know: Unanswer- able questions for squad

    [Rajpurkar et al., 2018] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswer- able questions for squad. In Proceedings of the 56th An- nual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), volume 2, pages 784– 789,

    2018

  2. [20]

    The probabilistic relevance framework: Bm25 and beyond

    [Robertson and Zaragoza, 2009] Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389, April

    2009

  3. [22]

    Modeling ambiguity, subjectivity, and diverg- ing viewpoints in opinion question answering systems

    [Wan and McAuley, 2016] Mengting Wan and Julian McAuley. Modeling ambiguity, subjectivity, and diverg- ing viewpoints in opinion question answering systems. In Data Mining (ICDM), 2016 IEEE 16th International Conference on, pages 489–498. IEEE,

    2016

  4. [23]

    Answering opinion questions on products by ex- ploiting hierarchical organization of consumer reviews

    [Yu et al., 2012] Jianxing Yu, Zheng-Jun Zha, and Tat-Seng Chua. Answering opinion questions on products by ex- ploiting hierarchical organization of consumer reviews. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computa- ...

    2012

  5. [1998]

    Large-scale simple question answering with memory networks

    [Bordes et al., 2015] Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. Large-scale simple question answering with memory networks. CoRR, abs/1506.02075,

    2015 arXiv

  6. [2002]

    Glove: Global vectors for word representation

    [Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543,

    2014

  7. [2004]

    Addressing complex and subjective product-related queries with customer reviews

    [McAuley and Yang, 2016] Julian McAuley and Alex Yang. Addressing complex and subjective product-related queries with customer reviews. In Proceedings of the 25th International Conference on World Wide Web, pages 625–635. International World Wide Web Conferences Steering Committee,

    2016

  8. [2009]

    Newsqa: A machine comprehension dataset

    [Trischler et al., 2017] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. Newsqa: A machine comprehension dataset. ACL 2017, page 191,

    2017

  9. [2011]

    Dureader: a chinese machine reading comprehension dataset from real-world applica- tions

    [He et al., 2018] Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yuan Liu, Yizhong Wang, Hua Wu, Qiaoqiao She, et al. Dureader: a chinese machine reading comprehension dataset from real-world applica- tions. In Proceedings of the Workshop on Machine Read- ing ...

    2018

  10. [2014]

    Squad: 100,000+ ques- tions for machine comprehension of text

    [Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100,000+ ques- tions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392,

    2016

  11. [2015]

    Think you have solved question answer- ing? try arc, the ai2 reasoning challenge

    [Clark et al., 2018] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answer- ing? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457,

    2018 arXiv

  12. [2016]

    Ms marco: A human generated machine reading comprehension dataset

    [Nguyen et al., 2016] Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading comprehension dataset. choice, 2640:660,

    2016

  13. [2017]

    R-net: Machine reading comprehension with self-matching net- works

    [Group, 2017] Natural Language Computing Group. R-net: Machine reading comprehension with self-matching net- works. May

    2017

  14. [2018]

    Searchqa: A new q&a dataset augmented with context from a search engine

    [Dunn et al., 2017] Matthew Dunn, Levent Sagun, Mike Higgins, V Ugur Guney, V olkan Cirik, and Kyunghyun Cho. Searchqa: A new q&a dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179,

    2017 arXiv

  15. [2019]

    Race: Large-scale read- ing comprehension dataset from examinations

    [Lai et al., 2017] Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. Race: Large-scale read- ing comprehension dataset from examinations. In Pro- ceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794,

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.