Knowledge Enhanced Attention for Robust Natural Language Inference

Abhinav Sethy; Alexander Hanbo Li

REVIEW 3 major objections 5 minor 39 references

Knowledge Enhanced Attention for Robust Natural Language Inference

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a constant attention bias for lexically related word pairs makes NLI models markedly more robust to adversarial word substitutions.

desk verdict A simple, parameter-free way to inject WordNet relations into attention heads that clearly helps on adversarial SNLI, but the benchmark is built from exactly those relations, so the robustness claim needs a transfer test before it generalizes. read the letter →

arxiv 1909.00102 v1 pith:XNFWBHZH submitted 2019-08-31 cs.CL cs.CRcs.LGstat.ML

Alexander Hanbo Li , Abhinav Sethy This is my paper · ORCID

classification cs.CLcs.CRcs.LGstat.ML

keywords knowledgeenhancedattentionnaturallanguageinferenceadversarialrobustnesslexicalrelationsWordNetmulti-headBERTbiasmatrix

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a simple way to make natural language inference models robust to lexical adversarial examples: add a constant offset $b$ to the attention logits of any word pair that has a WordNet lexical relation (synonymy, hypernymy, hyponymy, antonymy, co-hyponymy). The modification requires no new parameters and can be dropped into any attention-based model. Applied to a decomposable model, a Transformer encoder, and BERT, it substantially improves accuracy on the adversarial SNLI test set; BERT with the bias reaches 94.1%, matching the estimated human level. The paper argues this works because the bias directly injects external knowledge into the attention alignment that later layers can learn to use.

What carries the argument

The key object is the bias-augmented attention head, $$\mathrm{head}_i = \mathrm{softmax}\left(\frac{(QL_i)(KR_i)^\top}{\sqrt{d_k}} + bB_i\right)VW_i$$ where $B_i[p,q]=1$ if $(w_p,w_q)$ falls in the $i$-th lexical relation and $0$ otherwise. This is a per-head soft bias toward lexically related word pairs; it requires no extra parameters and is controlled by a scalar $b$. The paper links attention to structured embeddings and uses the bias as a cheap substitute for learning a relation-specific transformation.

What would settle it

Construct an adversarial test set from lexical relations not among the five injected (e.g., meronymy, derivational morphology, or multi-word paraphrase) and rerun the method; if accuracy falls back near baseline, the robustness is specific to the aligned relations rather than a general defense.

Watch

Extended reading notes

Core claim

The central discovery is that a constant bias term, rather than learned relation embeddings, suffices to inject lexical knowledge into multi-head attention. For each head $i$, a binary matrix $B_i$ marks word pairs in a specific lexical relation, and the attention logits are shifted by $b$ for those pairs (Equation 5). This changes the alignment so that related words attend to each other, and subsequent layers learn to exploit that signal. On adversarial SNLI, the method lifts accuracy from 50.3% to 71.1% for a small decomposable model and from 91.1% to 94.1% for BERT, the latter equaling the human estimate. Training with the bias is essential: applying it only at inference time gives no benefit.

Load-bearing premise

The adversarial SNLI test set is constructed from exactly the five WordNet lexical relations the method injects as bias matrices, so the measured robustness gains are on a test distribution aligned with the injected knowledge; other or broader perturbation types are not covered.

Editorial extensions

If this is right

Any attention-based NLI model can be made more robust to lexical word replacements by adding relation-specific bias matrices to its heads.
The bias should be present during training, not just inference, because downstream layers need to learn to use the injected alignments.
Applying the bias in early cross-attention layers works better than later ones, since it prevents noisy lexical information from propagating through the first alignment.
Combined with pretraining (BERT), the method closes the gap to human performance on adversarial SNLI while clean accuracy stays essentially unchanged.
Because the method adds no parameters and is model-agnostic, it can be transferred across architectures without redesign.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The robustness may be partly an artifact of test/train alignment: the adversarial test set is built from the same five WordNet relations the method injects, so gains could shrink or vanish under broader or different perturbation types (e.g., multi-word paraphrases or non-WordNet substitutions).
The bias can be read as a prior over token alignments; the same mechanism could inject other structured knowledge (coreference, temporal order, discourse relations) into attention, not just lexical relations.
With $b \to \infty$, the attention becomes a hard lexical matching; tuning $b$ on validation may expose whether optimal robustness requires softer, partially learned alignments.
A natural extension would be to learn the relation transformations $L_i, R_i$ jointly with the bias, potentially discovering task-specific relations rather than relying on fixed WordNet categories.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look: Li and Sethy add a fixed offset to attention logits for word pairs that stand in one of five WordNet relations (synonym, hypernym, hyponym, antonym, co-hyponym). Each of the five heads gets one relation, and the only new hyperparameter is the offset b. That's the genuinely new bit, and it's clean and easy to implement in any attention model. They show consistent gains on adversarial SNLI for a decomposable model, a Transformer encoder, and BERT, and they include a nice control: adding the bias only at inference doesn't help; the model has to be trained with it. That's honest and useful.

Where I'd push back is the evaluation. The adversarial SNLI test set from Glockner et al. is constructed by replacing words according to exactly those five relations — synonym, hypernym, hyponym, antonym, co-hyponym. The method injects exactly those five relations as attention biases. So the test distribution is aligned one-to-one with the injected knowledge. The paper calls the result "robustness" and the title generalizes, but there is no experiment separating the five relations, no held-out relation, and no test on a different perturbation family. It is entirely plausible that the model is using the bias as a benchmark-specific cue: the relation matrix marks the edited word pair, and the classifier learns to exploit that. The fact that the bias must be present during training (Table 4) is consistent with the model learning to rely on that cue, not necessarily with generalizable lexical knowledge.

A second, softer weakness: the human-level claim. 94.1% accuracy equals the human estimate in Glockner et al., but the test set is heavily imbalanced (7164 contradiction, 982 entailment, 47 neutral), and the neutral recall stays very low even with the method (23% for BERT o). So the headline number is mostly about the contradiction class. The authors do acknowledge this in Section 5.4, which is good, but the abstract and conclusion still lean on "human-level."

Also no code or data, which limits reproducibility, though the method is simple enough to reimplement.

Net: the core idea is plausible, the math is fine, and the within-benchmark gains are real. The load-bearing gap is the missing transfer test. If the authors added a held-out-relation experiment or an evaluation on a different adversarial set, the robustness claim would be much stronger. As is, I'd treat it as a solid workshop-level contribution with a good idea and an overstated headline.

I'd send it to review if I were an editor, because the technique deserves scrutiny and the gap is fixable. My own verdict would be conditional on the transfer experiment.

Referee Report

3 major / 5 minor

Summary. The paper proposes a method for making natural language inference (NLI) models more robust by injecting external lexical knowledge directly into multi-head attention. The core idea is Eq. (5): for each attention head, a constant offset b is added to the attention logits of word pairs that stand in a specified WordNet relation (synonym, hypernym, hyponym, antonym, co-hyponym), with the relation-specific mask B_i defining which pairs receive the offset. The method is applied to a decomposable NLI model (Model I), a Transformer encoder (Model II), and BERT during fine-tuning. On the adversarial SNLI dataset of Glockner et al. (2018), the authors report large accuracy gains: Model I improves from 48.2% to 81.3%, Model II from 41.4% to 58.0%, and BERT from 91.1% to 94.1%, the last figure claimed to equal estimated human performance. The paper also includes an ablation (Table 4) showing that the bias must be present during training to be effective.

Significance. If the reported robustness gains transferred to other lexical perturbations and to other NLI settings, the contribution would be practical and conceptually simple: a plug-in modification that adds no learned parameters and can be applied to any attention-based model. The connection drawn between multi-head attention and structured embeddings (Section 3.1) is suggestive, and Equation (5) is clearly the kind of 'quick fix' that could be reused. However, the central claim of general robustness is currently supported by only a single adversarial test set whose perturbation classes are exactly the five WordNet relations injected by the method. The paper is transparent about per-class results, which is a strength, but the interpretation of the 94.1% accuracy as 'human-level' is weakened by the extreme class imbalance of that test set. The significance of the paper therefore hinges on whether the evaluation can be shown not to be circular with respect to the injected knowledge.

major comments (3)

[§5.2 and §5.4 (Tables 2 and 3)] The evaluation is aligned one-to-one with the injected knowledge: the adversarial SNLI test set (Glockner et al., 2018) is constructed by replacing words with synonyms, hypernyms, hyponyms, antonyms, or co-hyponyms, and the proposed method injects exactly these five WordNet relations as bias matrices B_i in Eq. (5). The reported gains (e.g., Model I 48.2% to 81.3%, BERT 91.1% to 94.1%) may therefore reflect the bias directly marking the word pair that was modified in the test example, rather than a general mechanism for robust lexical inference. The paper needs a hold-one-out experiment—where one relation is excluded from the injected set but still present in the test set—or an evaluation on a different perturbation family (e.g., negation, numeric reasoning, or paraphrastic perturbations) to support the title and conclusion that the method makes NLI models robust. Without such a transfer test, the central claim is overstated.
[§5.4, Table 2, and Abstract] The claim that "BERT with our method achieves human-level performance" is based on overall accuracy on an adversarial test set with 7164 contradiction, 982 entailment, and only 47 neutral instances. For BERT_o, the per-class precision/recall for neutral is 4%/23%, and for Model I_o it is 1%/15%; the overall 94.1% accuracy is dominated by the contradiction class. Matching the overall accuracy of human annotators on this imbalanced set is not a meaningful measure of human-level lexical inference. The authors should either report human per-class precision/recall (if available from Glockner et al., 2018) or explicitly limit the claim to overall accuracy on this specific benchmark.
[§5.5, Table 4] The ablation shows that the bias must be present during both training and inference, and that dropping it at inference reduces adversarial accuracy from 81.3% to 57.2% while also reducing clean accuracy. This is consistent with the model learning to use the B_i masks as a benchmark-specific cue rather than acquiring generally improved lexical reasoning. The authors interpret this as an expected consequence of subsequent layers learning to use the information, but the same pattern would arise if the model were exploiting a shortcut aligned with the test distribution. The hold-out-relation experiment requested above would help distinguish these readings.

minor comments (5)

[§3.3, Eq. (5)] The magnitudes of b and the choice of modified layers are tuned on the SNLI validation set, yet the paper does not report sensitivity to b or to the layer-selection choice. Since b is called a 'hyper-parameter' and all experiments use b = 10, the authors should state whether the results are stable across a range of b values.
[§5.1] The paper states that Model II uses randomly initialized word embeddings to 'investigate the impact' of the method, but no comparison is shown for Model II with pretrained embeddings. A sentence clarifying whether random initialization is the default or a deliberate choice would improve reproducibility.
[§1 and §4.1] There are a few typographical issues: 'hypotheses' is used where 'hypothesis' is intended in several places, the notation 'll2' in the definition of H is inconsistent with 'l2', and the symbol 'L' is reused for both encoder layers and the left transformation matrix in Section 3. These do not affect the technical content but should be cleaned up.
[§5.4, Table 2] The WordNet baseline of 85.5% is reported in the table but not described in the text; the authors should explain how this baseline is computed, since it is highly competitive with the neural models and relevant to interpreting the gains.
[§5.2 and §5.5] The paper does not provide the exact code or scripts used to extract the five WordNet relations or to construct the B_i matrices. Given that the injected relations are the crux of the method, releasing this preprocessing code would materially aid reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attention-bias method is an explicit external-knowledge modification, not a fitted or self-referential prediction.

full rationale

The paper's derivation chain is self-contained. Equation (5) defines an additive bias b B_i for word pairs bearing a WordNet relation; b is a hyperparameter tuned on a validation set (Section 5.4), and the relation matrices B_i are constructed from WordNet, not learned from the adversarial test labels. The reported gains (e.g., Model I from 50.3% to 71.1%, and BERT from 91.1% to 94.1% on adversarial SNLI) are therefore measurements of a model equipped with external lexical cues, not quantities entailed by the construction. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The closest concern is evaluation alignment: Glockner et al.'s adversarial SNLI uses the same five WordNet relations (synonym, hypernym, hyponym, antonym, co-hyponym) that Section 5.2 injects, so the benchmark does not test robustness to held-out relation types. That limits external validity and makes the broad claim of robustness to lexical noise stronger than the evidence supports, but it is not circular in the derivation sense because the bias does not encode the label mapping and does not force the reported accuracies. The paper's own admission of low neutral recall (Section 5.4) is a performance caveat, not a circular step. No circular step meets the quoted-reduction standard.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the alignment between the injected WordNet relations and the construction of the adversarial test set. This is a domain assumption rather than a fitted parameter, but it is load-bearing. The only fitted hyperparameter is b, which is fixed at 10; layer and head choices are also tuned on validation data.

free parameters (3)

b (bias magnitude) = 10
Hyperparameter controlling the offset added to attention logits for word pairs with the target lexical relation. Tuned on the SNLI validation set; fixed at 10 for all reported experiments (Section 5.3 and Table 1).
Modified layer selection = varies (e.g., layers 1,2,3 for Model I_o on SNLI)
Which encoder/cross-encoder layers receive the bias matrices is selected by validation-set accuracy (Sections 5.3 and 5.4). The paper reports the best configuration.
Head-relation assignment = 5 relations assigned to first 5 heads of 12 (BERT), or 5 heads of 5 (Model I/II)
The mapping of the five WordNet relations to specific heads is chosen by the authors without a systematic search (Section 5.2). Could affect results.

assumptions (3)

domain assumption Multi-head attention similarity is a linear structured embedding (f_L = L, f_R = R) in the sense of Bordes et al. 2011
Used in Section 3.2 to motivate injecting external knowledge into heads; a conceptual bridge, not a formal equivalence.
ad hoc to paper Adding a constant b to attention logits for word pairs with relation r changes the model's output behavior in a way that improves robustness
The paper asserts this in Section 3.3. The subsequent layers are expected to learn to use the bias, which is validated only by experiments, not derived.
domain assumption The adversarial SNLI test set from Glockner et al. is a valid proxy for NLI model robustness
The paper uses it as the sole robustness evaluation and claims human-level performance on it, without testing on other adversarial benchmarks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Knowledge Enhanced Attention for Robust Natural Language Inference." pith.science (2026). https://pith.science/paper/XNFWBHZH

@misc{pith2026190900102,
  author       = {Pith},
  title        = {Pith review of: Knowledge Enhanced Attention for Robust Natural Language Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XNFWBHZH}},
  note         = {Machine review of arXiv:1909.00102}
}

read the original abstract

Neural network models have been very successful at achieving high accuracy on natural language inference (NLI) tasks. However, as demonstrated in recent literature, when tested on some simple adversarial examples, most of the models suffer a significant drop in performance. This raises the concern about the robustness of NLI models. In this paper, we propose to make NLI models robust by incorporating external knowledge to the attention mechanism using a simple transformation. We apply the new attention to two popular types of NLI models: one is Transformer encoder, and the other is a decomposable model, and show that our method can significantly improve their robustness. Moreover, when combined with BERT pretraining, our method achieves the human-level performance on the adversarial SNLI data set.

Figures

Figures reproduced from arXiv: 1909.00102 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 9 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Twenty-Fifth AAAI Conference on Artificial Intelligence

work page 2011
[4]

Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326

arXiv 2015
[5]

Samuel R Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D Manning, and Christopher Potts. 2016. A fast unified model for parsing and sentence understanding. arXiv preprint arXiv:1603.06021

arXiv 2016
[6]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2017. Neural natural language inference models enhanced with external knowledge. arXiv preprint arXiv:1711.04289

work page Pith review arXiv 2017
[7]

Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2016. Enhanced lstm for natural language inference. arXiv preprint arXiv:1609.06038

arXiv 2016
[8]

Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications. Synthesis Lectures on Human Language Technologies, 6(4):1--220

work page 2013

Show all 39 references

[9]

Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. In Advances in neural information processing systems, pages 3079--3087

2015
[10]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

2018 arXiv
[11]

Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. 2014. Retrofitting word vectors to semantic lexicons. arXiv preprint arXiv:1411.4166

2014 arXiv
[12]

Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018. Breaking nli systems with sentences that require simple lexical inferences. arXiv preprint arXiv:1805.02266

2018 arXiv
[13]

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R Bowman, and Noah A Smith. 2018. Annotation artifacts in natural language inference data. arXiv preprint arXiv:1803.02324

2018 arXiv
[14]

Aria D Haghighi, Andrew Y Ng, and Christopher D Manning. 2005. Robust textual inference via graph matching. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 387--394. Association for Computational Linguistics

2005
[15]

Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415

2016 arXiv
[16]

Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146

2018 arXiv
[17]

Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing. 2016. Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318

2016 arXiv
[18]

Dongyeop Kang, Tushar Khot, Ashish Sabharwal, and Eduard Hovy. 2018. Adventure: Adversarial training for textual entailment with knowledge-guided examples. arXiv preprint arXiv:1805.04680

2018 arXiv
[19]

Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. SciTail : A textual entailment dataset from science question answering. In AAAI

2018
[20]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

2014 arXiv
[21]

George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39--41

1995
[22]

Nikola Mrk s i \'c , Diarmuid O S \'e aghdha, Blaise Thomson, Milica Ga s i \'c , Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. arXiv preprint arXiv:1603.00892

2016 arXiv
[23]

Yixin Nie and Mohit Bansal. 2017. Shortcut-stacked sentence encoders for multi-domain inference. arXiv preprint arXiv:1708.02312

2017 arXiv
[24]

a ckstr \

Ankur P Parikh, Oscar T \"a ckstr \"o m, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933

2016 arXiv
[25]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

2014
[26]

Matthew E Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. arXiv preprint arXiv:1705.00108

2017 arXiv
[27]

Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365

2018 arXiv
[28]

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2. amazonaws. com/openai-assets/research-covers/languageunsupervised/language understanding paper. pdf

2018
[29]

Rajat Raina, Aria Haghighi, Christopher Cox, Jenny Finkel, Jeff Michels, Kristina Toutanova, Bill MacCartney, Marie-Catherine de Marneffe, Christopher D Manning, and Andrew Y Ng. 2005. Robust textual inference using diverse knowledge sources. In Proc. of the 1st. PASCAL Recogn...

2005
[30]

Tim Rockt \"a schel, Edward Grefenstette, Karl Moritz Hermann, Tom \'a s Ko c isk \`y , and Phil Blunsom. 2015. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664

2015 arXiv
[31]

Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Sen Wang, and Chengqi Zhang. 2018. Reinforced self-attention network: a hybrid of hard and soft attention for sequence modeling. arXiv preprint arXiv:1801.10296

2018 arXiv
[32]

Vivian S Silva, Siegfried Handschuh, and Andr \'e Freitas. 2018. Recognizing and justifying text entailment through distributional navigation on definition graphs. In Thirty-Second AAAI Conference on Artificial Intelligence

2018
[33]

Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2017. Compare, compress and propagate: Enhancing neural architectures with alignment factorization for natural language inference. arXiv preprint arXiv:1801.00102

2017 arXiv
[34]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008

2017
[35]

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. GLUE : A multi-task benchmark and analysis platform for natural language understanding. ArXiv preprint 1804.07461

2018 arXiv
[36]

Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. http://aclweb.org/anthology/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computationa...

2018
[37]

Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. arXiv preprint arXiv:1808.05326

2018 arXiv
[38]

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

2019 arXiv
[39]

Wanjun Zhong, Duyu Tang, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2018. Improving question answering by commonsense-based pre-training. arXiv preprint arXiv:1809.03568

2018 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Twenty-Fifth AAAI Conference on Artificial Intelligence

work page 2011

[4] [4]

Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326

arXiv 2015

[5] [5]

Samuel R Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D Manning, and Christopher Potts. 2016. A fast unified model for parsing and sentence understanding. arXiv preprint arXiv:1603.06021

arXiv 2016

[6] [6]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2017. Neural natural language inference models enhanced with external knowledge. arXiv preprint arXiv:1711.04289

work page Pith review arXiv 2017

[7] [7]

Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2016. Enhanced lstm for natural language inference. arXiv preprint arXiv:1609.06038

arXiv 2016

[8] [8]

Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications. Synthesis Lectures on Human Language Technologies, 6(4):1--220

work page 2013