Ensemble approach for natural language question answering problem

Anna Aniol; Marcin Pietron

REVIEW 3 major objections 5 minor 30 references

Ensemble approach for natural language question answering problem

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A class-aware weighted-voting ensemble of three attention-based reading models reaches F1 81.96 and EM 73.77 on SQuAD, beating its best single model.

desk verdict A clearly written but statistically weak ensemble study: the class-aware weighting idea is real, yet the reported 0.4 F1 gain over the best single model could easily be within run-to-run noise. read the letter →

arxiv 1908.09720 v2 pith:UWBIW7O7 submitted 2019-08-26 cs.CL cs.AIcs.LG

Anna Aniol , Marcin Pietron This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords naturallanguageprocessingmachinecomprehensiondeeplearningquestionansweringensembleweightedvotingSQuADattentionmechanism

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether combining three strong neural reading-comprehension models can beat the best single model on the SQuAD benchmark. It answers yes: an ensemble that groups questions by type and weights each model's vote by its per-class F1 measured on a 5% holdout reaches F1 81.96 and exact-match 73.77, ahead of the best member, Mnemonic Reader (F1 81.57, EM 73.25). The gain is small but appears on both metrics, and a version without question classes also beats the best single model, though by slightly less. If the result holds, it shows that question-type specialization among diverse architectures can be exploited by voting without designing a new network.

What carries the argument

The central mechanism is question-class-aware weighted voting. Fourteen question classes are defined by the interrogative phrase opening each SQuAD question: date, during, how are, how big/size, how m/m, how old, what, what time, when, where, who, whom, why, and undefined. For each class, each model's voting weight is the average F1 it scored on that class in a 5% holdout of the training set, using models trained on the remaining 95%. The voting rule then aggregates candidate answers by summing the weights of identical answers, chooses the highest total, and falls back to the globally best model for non-duplicated or undefined cases. The load-bearing observation is that the three base models disagree on most questions, leaving room for voting to correct individual errors.

What would settle it

Repeat the experiment with several different random 5% splits of the SQuAD training set, retraining all three base models from scratch for each split. If the class-aware ensemble fails to beat Mnemonic Reader in the majority of splits, or if the 0.39-point F1 advantage falls inside the split-to-split variance, the claimed improvement is not stable.

Watch

Extended reading notes

Core claim

The paper claims that a class-specific weighted-voting ensemble of BiDAF, QANet, and Mnemonic Reader outperforms every one of them on SQuAD. The voting weights come from a 5% pre-evaluation split: models are trained on the other 95% of the training set, scored separately on each of fourteen question classes derived from interrogative phrases, and each model receives the average F1 it achieves in each class as its voting weight. At test time, models trained on the full training set each propose an answer; if two or more models give the same answer, their class weights are summed, and the answer with the highest total is returned. When no answer is duplicated, or the question falls outside the defined classes, the globally best model's answer is returned. This procedure reaches F1 81.96 and EM 73.77 on the SQuAD evaluation set, exceeding the best standalone model, Mnemonic Reader, by 0.39 and 0.52 points, and exceeding a class-blind version of the same voting by 0.10 and 0.05 points.

Load-bearing premise

The class-specific F1 weights measured on a 5% holdout of the SQuAD training set, using models trained on the remaining 95%, are assumed to stay representative when the final models are trained on the full dataset and evaluated on the official evaluation set.

Editorial extensions

If this is right

An ensemble built from existing attention-based reading models improves over the strongest member on SQuAD without any architectural change, so model diversity across question types is a usable resource.
Because the class-blind weighted ensemble already beats the best single model, part of the gain comes from voting itself, and the class-aware weighting adds a further 0.10 F1 and 0.05 EM.
Per-class weights estimated from only 5% of the training set are enough, in this setup, to guide voting on the full evaluation set; the paper reports that splitting factors 0.01 and 0.1 gave worse overall results.
The ensemble is not better on every question class: for how-old questions it scores below BiDAF because adding the weights of two equal wrong answers can outweigh one correct answer, so the method's value is global rather than per-class.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported margins are small enough that a skeptic would want a significance test across several random 5% splits; the paper gives no variance or repeated-run numbers, so the stability of the 0.39-point F1 edge is open.
Since roughly 53% of SQuAD questions belong to the what class, even a modest improvement concentrated in that dominant class could explain most of the headline gain; testing on a class-balanced evaluation set would show whether the mechanism helps rare question types.
The same recipe should transfer to other reading-comprehension datasets whose questions can be typed by surface cues, but the 5% holdout may be too small on less skewed collections, where cross-validated or online weight estimation would be a natural extension.
The method is a form of mixture-of-experts gated by question type; replacing the hand-built interrogative-phrase taxonomy with a learned question embedding could automate grouping and might generalize beyond the fourteen classes used here.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a small, clearly written ensemble study. The new piece is the per-question-class weighting of three off-the-shelf QA models, with class weights computed on a 5% holdout of SQuAD training data. That idea is legitimate, and the paper is honest about its exploration: it tells you that EM-based weights, length-based classes, and voting on undefined questions all made things worse. That is useful negative information. The per-class agreement tables are the most valuable part—they show that the three models disagree in systematic ways, which is a genuinely useful observation for anyone building QA ensembles.

The problem is the load-bearing claim. The 0.39 F1 and 0.52 EM gains over Mnemonic Reader come from a single evaluation run. No error bars, no repeated seeds, no significance test. The paper also tells you it picked the final configuration by testing alternatives on the evaluation set (Section 5: F1 vs EM weights, add vs max, undefined handling, length classification). That means the headline number carries selection bias, and the paper gives you no way to correct for it. The class-aware ensemble is only 0.1 F1 and 0.05 EM better than the plain weighted ensemble; on a 10,570-question set that is a very small number of question-level changes. Given the ensemble falls back to Mnemonic Reader whenever there is no duplicate answer or the class is undefined, the improvement rests on a small number of answer swaps, which are precisely the cases most sensitive to training noise. The paper's own similarity tables (74.3% equal F1 between Mnemonic and QANet) confirm that the ensemble barely changes most predictions.

So the central empirical claim is plausible but not established. The diagnostic insight about class-level complementarity is real and worth reporting, but the 0.4-point edge should not be taken as a robust result.

Who is this for? Practitioners who want a concrete recipe for combining QA models, and anyone interested in the failure modes of small-dev-set evaluation. It could be a decent workshop paper or technical note if the authors add multi-seed runs and an honest validation split for configuration selection.

I would not desk-reject it—the method is clear and the negative results are honest—but I would send it back for a major revision requiring variance estimates and a configuration-selection protocol that does not use the evaluation set.

Referee Report

3 major / 5 minor

Summary. The paper proposes a class-aware weighted voting ensemble for SQuAD question answering. It uses three published architectures (BiDAF, QANet, Mnemonic Reader), defines 14 question classes, splits the SQuAD training set 95/5, trains the base models on the 95% subset to measure per-class average F1, and uses these as voting weights in Algorithm 2. Answers are combined by summing weights of duplicate candidate answers and otherwise falling back to the globally best model. On the SQuAD dev set, the class-aware ensemble scores F1 81.96 and EM 73.77 versus 81.57/73.25 for the best single model (Mnemonic Reader); the non-class-aware ensemble scores 81.86/73.72. The authors also experiment with variants (sum vs. max weights, undefined-class handling, EM-based weights, length-based classification).

Significance. If the reported gains are real, the paper demonstrates a simple, inexpensive way to push SQuAD accuracy beyond a strong single model, and its per-class error analysis is a useful diagnostic. The main strengths are that the voting weights are estimated on a held-out portion of the training set (a legitimate design) and that Algorithms 1–4 are specified explicitly, making the method easy to implement. However, the measured improvement is very small (0.39 F1, 0.52 EM), comes from a single run with no significance testing, and the final configuration was chosen after trying several variants on the same evaluation set. As it stands, the central 'outperforms' claim is plausible but not statistically established.

major comments (3)

[§5, Table 11] The central claim rests on a single evaluation run. No error bars, confidence intervals, or significance tests are reported, and the improvements over Mnemonic Reader (0.39 F1, 0.52 EM) and over the non-class ensemble (0.1 F1, 0.05 EM) are small relative to the size of the evaluation set. Since training is stochastic and the 5% pre-evaluation split is random, the reported gain could be within run-to-run variation. Please provide repeated runs with different seeds and split draws, and report paired significance tests (or bootstrap confidence intervals) over the 10,570 evaluation questions.
[§5, variant testing paragraphs] Several alternative design choices (adding vs. taking the max of duplicate weights, handling undefined questions with voting, using EM rather than F1 weights, and length-based classification) were evaluated directly on the SQuAD evaluation set, and the best-performing variant was then reported as the headline result. This is a form of selection on the test set; the reported advantage may overstate the true improvement of the chosen configuration. The paper should either report results for all variants with error bars, or select the configuration on a separate validation split and then apply it once to the evaluation set.
[§3, Table 3 and §4, Algorithms 1–2] The class-specific weights are estimated from a single random 5% split of the training set, where several classes have very few examples (how big/size: 4, how old: 6, what time: 11, whom: 20). These noisy estimates are then used to weight the final models trained on the full training set, but the paper does not check whether the weights are stable across different 5% splits or across the 95%-vs-100% training regimes. A k-fold or bootstrap procedure over the training set would show whether the measured substitutions that produce the 0.39 F1 gain are robust.

minor comments (5)

[Tables 5–10] QANet is evaluated on 10,482 questions rather than 10,570; please explain the discrepancy (e.g., missing predictions) and how Algorithm 2 handles questions without a QANet candidate.
[Figures 1–6] The figures lack descriptive captions and axis labels; this makes it difficult to interpret the per-class behavior that is central to the paper. Please add proper captions and label the axes.
[Throughout] The manuscript has many typos and grammatical errors, such as 'Each training contains of 30 epochs', 'Figures 5 ans 6', and 'how has the SQuAD data set been splitted'. A careful proofreading pass is needed.
[§3, merging which into what] The claim that merging which and what classes 'has no influence on the final results' is unsupported; please provide evidence (e.g., comparative results) or remove the claim.
[§5, how old example] The description of the 'how old' example is confusing: if BiDAF has the highest voting weight, it is not clear why adding the weights of Mnemonic and QANet would override it unless their summed weight exceeds BiDAF's weight. Please clarify the exact voting arithmetic in this example.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the class-weighted ensemble is derived from a holdout pre-evaluation split and measured against an external SQuAD evaluation set.

full rationale

The paper's derivation chain is empirical and externally anchored, not self-referential. The class-specific voting weights are obtained by training each base model on 95% of the SQuAD training set and evaluating it on the disjoint 5% pre-evaluation split (Section 3; Algorithm 1, steps 1-3). The final ensemble is then applied to the full SQuAD evaluation set (Algorithm 2), and the reported F1/EM numbers in Table 11 are measurements on that hold-out evaluation set, not quantities fed into the weighting procedure. The ensemble decision rule (summing weights of duplicate answers, otherwise returning the globally best model's answer) is a genuine decision rule: the paper even documents a case ('how old' questions) where the rule produces a worse result than the highest-weight base model, demonstrating that the outcome is not forced by construction. There are no load-bearing self-citations: the base architectures are attributed to external papers ([2], [23], [24]) and the implementations to external repositories ([29], [30]); no 'uniqueness' result or prior work by these authors is invoked. Concerns that the improvement over Mnemonic Reader is small, that no significance test or variance estimate is provided, and that several algorithmic variants were compared on the same evaluation set before reporting the best result are legitimate threats to the robustness and external validity of the empirical claim, but they are not circularity: the reported ensemble score is not equivalent to its inputs by definition or by construction. Accordingly, no circular step is identified and the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical objects. Its central claim rests on the transferability of class-level F1 weights from a small holdout split to the final models and the evaluation set, plus several empirically selected design choices.

free parameters (6)

class-specific voting weights = per-class average F1 per model (e.g., QANet highest for date, Mnemonic highest overall)
Computed on the 5% pre-evaluation split in Algorithm 1, Step 3. These weights determine which answer wins the vote and are fitted to a held-out subset.
splitting factor for pre-evaluation set = 0.05
Chosen because 0.01 and 0.1 produced worse general results (Section 3). This is a hyperparameter selected by experimentation.
weight metric = F1 instead of EM
Using average EM instead of F1 for class weights resulted in worse ensemble results (Section 5).
duplicate-answer combination rule = adding weights instead of taking the max
Taking the max of equal-prediction weights was tested globally and gave worse results (Section 5).
undefined-class fallback = return answer of globally best model
Applying weighted voting to undefined questions had a negative impact (Section 5).
training epochs and batch size = 30 epochs, batch size 32
Reported in Section 4; these are hand-set hyperparameters for all models.

assumptions (5)

domain assumption The SQuAD dataset is a valid and representative benchmark for reading comprehension.
Used throughout to train and evaluate the models; the paper treats leaderboard scores as the measure of QA quality.
domain assumption The public implementations of BiDAF and Mnemonic Reader are correct after the authors' bug fixes, and the unnamed QANet implementation is a faithful reproduction.
The paper states implementations are based on repositories [29] and [30] with modifications, but the QANet source is not identified (Section 4).
domain assumption A random 5% split of the training set reproduces the question-class distribution of the full SQuAD dataset.
The paper selects 0.05 as the smallest split that yields a similar class distribution (Section 3).
domain assumption Class-specific F1 measured on the pre-evaluation models transfers to the final models trained on the full training set.
The weights come from models trained on 95% of the data but are applied to models trained on 100% (Algorithms 1 and 2).
domain assumption F1 is an appropriate proxy for answer quality when setting ensemble weights.
The paper chooses F1 over EM based on empirical outcome, and treats class-average F1 as the weight (Algorithm 1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Ensemble approach for natural language question answering problem." pith.science (2026). https://pith.science/paper/UWBIW7O7

@misc{pith2026190809720,
  author       = {Pith},
  title        = {Pith review of: Ensemble approach for natural language question answering problem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UWBIW7O7}},
  note         = {Machine review of arXiv:1908.09720}
}

read the original abstract

Machine comprehension, answering a question depending on a given context paragraph is a typical task of Natural Language Understanding. It requires to model complex dependencies existing between the question and the context paragraph. There are many neural network models attempting to solve the problem of question answering. The best models have been selected, studied and compared with each other. All the selected models are based on the neural attention mechanism concept. Additionally, studies on a SQUAD dataset were performed. The subsets of queries were extracted and then each model was analyzed how it deals with specific group of queries. Based on these three model ensemble model was created and tested on SQUAD dataset. It outperforms the best Mnemonic Reader model.

Figures

Figures reproduced from arXiv: 1908.09720 by the authors.

**Figure 2.** F1 distribution over question classes (evaluation dataset) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** F1 distribution over question classes (pre [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: EM distribution over question classes (pre [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: F1 Results over question classes (evaluation data) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

**Figure 6.** Figure 6: EM Results over question classes (evaluation data) [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages

[1]

Dynamic Coattention Networks For Question Answering

Caiming Xiong, Victor Zhong and Richard Socher. Dynamic Coattention Networks For Question Answering. CoRR, https://dblp.org/rec/bib/journals/corr/XiongZS16, 2016

work page 2016
[2]

Bidirectional Attention Flow for Machine Comprehension

Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi and Hannaneh Hajishirzi. Bidirectional Attention Flow for Machine Comprehension. CoRR, https://dblp.org/rec/bib/journals/corr/SeoKFH16, 2016

work page 2016
[3]

R -NET: Machine Reading Comprehension w ith Self-matching Netw orks

Natural Language Computing Group, Microsoft Research Asia. R -NET: Machine Reading Comprehension w ith Self-matching Netw orks. https://w ww.microsoft.com/en-us/research/w p-content/uploads/2017/05/r-net.pdf, 2017

work page 2017
[4]

Pay More Attention - Neural Architectures for Question-Answering

Zia Hasan, Sebastian Fischer Pay More Attention - Neural Architectures for Question-Answering. CoRR, http://arxiv.org/abs/1803.09230, 2018

work page Pith review arXiv 2018
[5]

An Analysis of the AskMSR Qu estion -answering System

Brill, Eric and Dumais, Susan and Banko, Michele. An Analysis of the AskMSR Qu estion -answering System. Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - vol. 10, pp. 257-264, 2002, Association for Computational Linguistics, Stroudsburg, PA, USA

work page 2002
[6]

NLP Algorithm Based Question and Answering System

Sarkar, Sanglap and Madasu, Venkateshwar Rao an d SM, Baala Mithra and Rao, Subrahmanya VRK. NLP Algorithm Based Question and Answering System. Proceedings of the 2015 Seventh International Conference on Computational Intelligence, Modelling and Simulation, CIMSIM ’15, 2015, pp. 97-101, IEEE Computer Society, Washington, DC, USA

work page 2015
[7]

J. Ramos. Using TF-IDF to Determine Word Relevance in Document Queries. Technical report. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.121.1424rep=rep1type=pdf, 2003

work page 2003
[8]

Comparative Study of CNN and RNN for Natural Language Processing

Wenpeng Yin and Katharina Kann and Mo Yu and Hinrich Schütze. Comparative Study of CNN and RNN for Natural Language Processing. CoRR, vol. abs/1702.01923, http://arxiv.org/abs/1702.01923, 2017

arXiv 2017

Show all 30 references

[9]

Long Short-Term Memory

Hochreiter, Sepp and Schmidhuber, Jürgen. Long Short-Term Memory. Neural Computing, vol.9, pp.1735-1780, November 15, 1997, MIT Press, Cambridge, MA, USA

1997
[10]

Deep Learning Approaches for Question Answering System

Sharma, Yashvardhan and Gupta, Sahil. Deep Learning Approaches for Question Answering System. vol.132, pp.785-794, Procedia Computer Science, 2018

2018
[11]

Efficient Estimation of Word Representations in Vector Space

Tomas Mikolov and Kai Chen and Greg Corrado and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. CoRR, vol.abs/1301.3781, http://arxiv.org/abs/1301.3781, 2013

2013 arXiv
[12]

Tow ards AI -Complete Question Answ ering: A Set of Prerequisite Toy Tasks

Jason Weston and Antoine Bordes and Sumit Chopra and Tomas Mikolov. Tow ards AI -Complete Question Answ ering: A Set of Prerequisite Toy Tasks. vol.abs/1502.05698, http://arxiv.org/abs/1502.05698, 2015

2015 arXiv
[13]

Memory Networks

Jason Weston and Sumit Chopra and Antoine Bordes. Memory Networks. CoRR, vol.abs/1410.3916, http://arxiv.org/abs/1410.3916, 2014

2014 arXiv
[14]

Ask Me Anything: Dynamic Memory Netw orks for Natural Language Processing

Ankit Kumar and Ozan Irsoy and Jonathan Su and James Bradbury and Robert English and Brian Pierce and Peter Ondruska and Ishaan Gulrajani and Richard Socher. Ask Me Anything: Dynamic Memory Netw orks for Natural Language Processing. CoRR, vol.abs/1506.07285, http://arxiv.org/a...

2015 arXiv
[15]

Minh-Thang Luong and Hieu Pham and Christopher D. Manning. Effective Approaches to Attention-based Neural Machine Translation, CoRR, vol. abs/1508.04025, http://arxiv.org/abs/1508.04025, 2015

2015 arXiv
[16]

Reasoning about Entailment with Neural Attention

Tim Rocktäschel and Edward Grefenstette and Karl Moritz Hermann and Tomás Kociský and Phil Blunsom. Reasoning about Entailment with Neural Attention. CoRR, vol. abs/1509.06664, http://arxiv.org/abs/1509.06664, 2015

2015 arXiv
[17]

Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, Łukasz and Polosukhin, Illia

Attention is All you Need. Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, Łukasz and Polosukhin, Illia. Advances in Neural Information Processing Systems, pp.5998-6008, 2017, Curran Associates, Inc

2017
[18]

Frustrati ngly Short Attention Spans in Neural Language Modeling

Michal Daniluk and Tim Rocktäschel and Johannes Welbl and Sebastian Riedel. Frustrati ngly Short Attention Spans in Neural Language Modeling. CoRR, vol. abs/1702.04521, http://arxiv.org/abs/1702.04521, 2017

2017 arXiv
[19]

Hierarchical Attention Networks for Document Classification

Yang, Zichao and Yang, Diyi and Dyer, Chris and He, Xiaodong and Smola, Alex and Hovy, Eduard. Hierarchical Attention Networks for Document Classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human ...

2016
[20]

Multilingual Hierarchical Attention Networks for Document Classifi-cation

Nikolaos Pappas and Andrei Popescu-Belis. Multilingual Hierarchical Attention Networks for Document Classifi-cation. CoRR, vol. abs/1707.00896, http://arxiv.org/abs/1707.00896, 2017

2017 arXiv
[21]

Weakly Supervised Memory Networks

Sainbayar Sukhbaatar and Arthur Szlam and Jason Weston and Rob Fergus. Weakly Supervised Memory Networks. CoRR, vol. abs/1503.08895, http://arxiv.org/abs/1503.08895, 2015

2015 arXiv
[22]

Jeffrey Pennington and Richard Socher and Christopher D. Manning. booktitle = Empirical Methods in Nat -ural Language Processing (EMNLP), GloVe: Global Vectors for Word Representation, pp. 1532-1543, 2014, http://www.aclweb.org/anthology/D14-1162

2014
[23]

Le, QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

Adams Wei Yu and David Dohan and Minh -Thang Luong and Rui Zhao and Kai Chen and Mohammad Norouzi and Quoc V. Le, QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension. CoRR, vol. abs/1804.09541, 2018

2018 arXiv
[24]

Mnemonic Reader for Machine Comprehension

Minghao Hu and Yuxing Peng and Xipeng Qiu. Mnemonic Reader for Machine Comprehension. CoRR, vol. abs/1705.02798, http://arxiv.org/abs/1705.02798, 2017

2017 arXiv
[25]

FusionNet: Fusing via Fully-Aware At -tention with Application to Machine Comprehension

Hsin-Yuan Huang and Chenguang Zhu and Yelong Shen and Weizhu Chen. FusionNet: Fusing via Fully-Aware At -tention with Application to Machine Comprehension. CoRR, vol. abs/1711.07341, http://arxiv.org/abs/1711.07341, 2017

2017 arXiv
[26]

DCN+: Mixed Objective and Deep Residual Coattention for Question Answering

Caiming Xiong and Victor Zhong and Richard Socher. DCN+: Mixed Objective and Deep Residual Coattention for Question Answering. CoRR, vol. abs/1711.00106, http://arxiv.org/abs/1711.00106, 2017

2017 arXiv
[27]

Journal of Artificial Intelligence Research vol.11 pp.169-198, 1999

David Opitz and Richard Maclin Popular Ensemble Methods: An Empirical Study. Journal of Artificial Intelligence Research vol.11 pp.169-198, 1999

1999
[28]

Bagging Predictors

Breiman, Leo. Bagging Predictors. Machine Learning, vol. 24, pp. 123-140, 1996, Kluwer Academic Publishers, Hingham, MA, USA

1996
[29]

Re-implementation of BiDAF in PyTorch

Taeuk Kim. Re-implementation of BiDAF in PyTorch. https://github.com/galsang/BiDAF-pytorch
[30]

A PyTorch implementation of Mnemonic Reader for the Machine Comprehension task, https://github.com/HK , UST-KnowComp/MnemonicRea der

Knowledge Computation Group at HKUST, led by Yangqiu Song. A PyTorch implementation of Mnemonic Reader for the Machine Comprehension task, https://github.com/HK , UST-KnowComp/MnemonicRea der

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Dynamic Coattention Networks For Question Answering

Caiming Xiong, Victor Zhong and Richard Socher. Dynamic Coattention Networks For Question Answering. CoRR, https://dblp.org/rec/bib/journals/corr/XiongZS16, 2016

work page 2016

[2] [2]

Bidirectional Attention Flow for Machine Comprehension

Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi and Hannaneh Hajishirzi. Bidirectional Attention Flow for Machine Comprehension. CoRR, https://dblp.org/rec/bib/journals/corr/SeoKFH16, 2016

work page 2016

[3] [3]

R -NET: Machine Reading Comprehension w ith Self-matching Netw orks

Natural Language Computing Group, Microsoft Research Asia. R -NET: Machine Reading Comprehension w ith Self-matching Netw orks. https://w ww.microsoft.com/en-us/research/w p-content/uploads/2017/05/r-net.pdf, 2017

work page 2017

[4] [4]

Pay More Attention - Neural Architectures for Question-Answering

Zia Hasan, Sebastian Fischer Pay More Attention - Neural Architectures for Question-Answering. CoRR, http://arxiv.org/abs/1803.09230, 2018

work page Pith review arXiv 2018

[5] [5]

An Analysis of the AskMSR Qu estion -answering System

Brill, Eric and Dumais, Susan and Banko, Michele. An Analysis of the AskMSR Qu estion -answering System. Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - vol. 10, pp. 257-264, 2002, Association for Computational Linguistics, Stroudsburg, PA, USA

work page 2002

[6] [6]

NLP Algorithm Based Question and Answering System

Sarkar, Sanglap and Madasu, Venkateshwar Rao an d SM, Baala Mithra and Rao, Subrahmanya VRK. NLP Algorithm Based Question and Answering System. Proceedings of the 2015 Seventh International Conference on Computational Intelligence, Modelling and Simulation, CIMSIM ’15, 2015, pp. 97-101, IEEE Computer Society, Washington, DC, USA

work page 2015

[7] [7]

J. Ramos. Using TF-IDF to Determine Word Relevance in Document Queries. Technical report. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.121.1424rep=rep1type=pdf, 2003

work page 2003

[8] [8]

Comparative Study of CNN and RNN for Natural Language Processing

Wenpeng Yin and Katharina Kann and Mo Yu and Hinrich Schütze. Comparative Study of CNN and RNN for Natural Language Processing. CoRR, vol. abs/1702.01923, http://arxiv.org/abs/1702.01923, 2017

arXiv 2017