Unpacking the Resilience of SNLI Contradiction Examples to Attacks

Archit Agarwal; Chetan Verma

REVIEW 4 major objections 5 minor 18 references

Unpacking the Resilience of SNLI Contradiction Examples to Attacks

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Universal adversarial triggers cause steep accuracy drops for SNLI entailment and neutral classes, while contradiction examples show only a small decline; fine-tuning on a trigger-augmented dataset restores performance to near baseline.

desk verdict Plausible class-level robustness measurement, but single-trigger design and post hoc explanation keep it from being more than a modest extension. read the letter →

arxiv 2412.11172 v1 pith:FAETSDWH submitted 2024-12-15 cs.CL

Chetan Verma , Archit Agarwal This is my paper · ORCID

classification cs.CL

keywords naturallanguageinferenceuniversaladversarialtriggersrobustnessspuriouscorrelationsSNLIELECTRAfine-tuningdatasetbias

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks why some classes of natural-language-inference examples are harder to attack than others. Using universal adversarial triggers generated on a separate model and transferred to ELECTRA-small, the authors show that prepending a single trigger word to a hypothesis collapses accuracy for entailment (90.23% to 25.78%) and neutral (86.70% to 25.76%) examples but barely dents contradiction examples (91.06% to 83.63%). They attribute this resilience to the contradiction class containing many more strongly label-correlated words, which act as giveaway signals that a trigger cannot override. Fine-tuning the model on a small augmented dataset containing both clean and trigger-modified examples restores accuracy on both the original and adversarial sets. The finding matters because it shows that adversarial triggers can expose class-specific spurious correlations in NLI benchmarks and that targeted fine-tuning is a cheap fix.

What carries the argument

The central object is the Universal Adversarial Trigger: a short sequence of tokens (here, one word) learned by HotFlip-style gradient-guided token replacement on a source model and then prepended to inputs of a target model, with the property that it is context-independent and transfers across models. The argument runs through the trigger's correlation score, p(l|w), the conditional probability of a label given a trigger word in SNLI, and through the cumulative frequency of the top five such correlated words per class. The trigger is effective on entailment and neutral because it is strongly correlated with a competing class, but the contradiction class's abundance of its own correlated words blocks the same manipulation.

What would settle it

Generate several independent universal triggers per class (different initializations, lengths, and target labels) and apply them to the full SNLI validation set. If contradiction accuracy falls by an amount comparable to entailment or neutral for any of these triggers, the claim that contradiction examples are inherently resilient to universal triggers would be refuted; alternatively, remove the top correlated words from contradiction hypotheses and show that the original trigger now flips them at the same rate as other classes.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that universal adversarial triggers—single tokens optimized to minimize loss toward a target class and prepended to hypotheses—exploit spurious correlations in SNLI unevenly across label classes. When triggers generated on ESIM with GloVe embeddings are applied to ELECTRA-small, the model's accuracy on the entailment and neutral classes drops by about 64 points, whereas the contradiction class drops by only 7.43 points. The authors propose that the contradiction class is more robust because it contains more words highly correlated with its label: the cumulative frequency of the top five correlated words is 312 for contradictions, 128 for neutral, and 57 for entailment, giving contradiction examples intrinsic giveaway cues that a prepended token cannot easily override. They then show that fine-tuning the model for one epoch on a 6,000-example dataset—half clean, half trigger-modified—brings challenge-set accuracy back to 90.13% (entailment), 87.53% (neutral), and 91.96% (contradiction), essentially matching or exceeding the pre-attack baseline. This is framed as evidence that the attack targets dataset artifacts rather than the model's core reasoning ability.

Load-bearing premise

The claim rests on the assumption that one universal trigger per class, generated on ESIM and transferred to ELECTRA, tested on a 1,000-example subset, is representative enough to conclude that the contradiction class is intrinsically more resistant to adversarial attacks than the other classes.

Editorial extensions

If this is right

A single epoch of fine-tuning on a small trigger-augmented dataset (6,000 examples, half modified) is enough to erase the accuracy gap on both the standard validation set and the adversarial challenge set.
Universal triggers are a practical diagnostic: their class-level effectiveness quantifies how much a model relies on label-correlated surface words rather than semantic inference.
The 7.43-point drop for contradictions, versus roughly 64-point drops for the other classes, implies that the model's apparent robustness on the original benchmark is partly a byproduct of how the SNLI contradiction class is written.
Because triggers generated on ESIM transfer to ELECTRA, the same pipeline can probe other NLI models and datasets without white-box access to the target.
Comparing random triggers with universal triggers isolates the effect of learned spurious correlations from mere input perturbation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the cumulative-frequency explanation is correlational, not causal: the paper does not manipulate the presence of correlated words directly, so an experiment that removes the top giveaway words from contradiction hypotheses and re-runs the attack would test whether those words are what block trigger flips.
The resilience may be a property of SNLI's annotation distribution rather than of contradiction as a semantic category; the same experiment on MultiNLI or on a balanced re-annotation of SNLI could show whether the asymmetry persists.
Since only one trigger per class was generated, aimed at one competing label, we infer that testing multiple triggers, longer triggers, and triggers targeting both alternative labels is needed to rule out trigger-specific artifacts.
We also infer that the 1,000-example validation subset used for both trigger generation and evaluation is small; re-sampling or using the full validation split would check whether the class-level drops are stable.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a small, honest empirical paper. It measures ELECTRA-small on SNLI with universal triggers prepended to hypotheses, finds entailment and neutral accuracy collapse while contradiction holds at 83.63%, and shows one epoch of fine-tuning on a trigger-augmented set restores near-baseline. The numbers in Table 3 are clear, Table 4 shows the class-wise confusion, and the authors ship code and datasets, so the result is checkable. What is actually new is the measurement, not the framework: they apply Wallace et al.'s universal triggers and Liu et al.'s inoculation recipe to a new model and report a class-level asymmetry that prior work did not highlight.

The soft spot is exactly where the stress-test note points. The central claim—contradiction resilience—rests on one universal trigger per class, generated on ESIM/GloVe and transferred to a single ELECTRA checkpoint, evaluated on one 1,000-example sample. That is thin. If the contradiction trigger 'joyously' happens to transfer poorly for reasons unrelated to class robustness, the 7.43% drop could look very different with another trigger. The paper also does not vary target labels; every trigger pushes the source class to a single competing class, so class-pair difficulty is conflated with class resilience. The correlated-word explanation in section 5.3.4 is post hoc: it counts p(label|w) but never verifies that surviving contradiction examples actually contain those words, or that flips are concentrated in examples that lack them.

I want to be fair, though: the concern is not fatal. Table 4 shows 'joyously' does flip 15.76% of contradictions to neutral, so the trigger is not inert on contradiction. The pattern is plausible, and the fine-tuning recovery is a useful replication of inoculation for ELECTRA. If I were editing, I'd send this to referees rather than desk-reject, because the measurement is real and the authors made it reproducible. The revision should add multiple triggers per class, target-label variation, error bars over seeds, and a controlled mechanism test—for instance, hold the trigger constant across classes and see whether contradiction accuracy stays high.

Who gets value: researchers working on NLI artifacts or adversarial robustness get a data point, and the released data and code save them time. For a top venue the evidence is too thin; for a workshop or a short-paper track it could be fine after those robustness fixes. I would not cite it in the next year in my own work, but I would bring it to reading group as a quick example of class-level robustness measurement.

Referee Report

4 major / 5 minor

Summary. The paper studies the robustness of ELECTRA-small on SNLI under Universal Adversarial Triggers. The authors generate single-token triggers on ESIM with GloVe embeddings, transfer them to ELECTRA-small in a black-box manner, and construct two challenge sets: Challenge Set I with universal triggers and Challenge Set II with random triggers. Table 3 reports class-level accuracy drops from the validation subset to Challenge Set I: entailment falls from 90.23% to 25.78%, neutral from 86.70% to 25.76%, while contradiction only falls from 91.06% to 83.63%. Fine-tuning on a 6,000-example trigger-augmented training set restores performance on both the validation subset and Challenge Set I. The paper attributes contradiction resilience to the class containing more correlated words, as measured by p(label|word) in Section 5.3.4.

Significance. If the class-level asymmetry is robust, the paper is a useful empirical contribution to adversarial robustness in NLI, showing that universal trigger attacks are unevenly effective across classes and that trigger-augmented fine-tuning can mitigate the observed vulnerabilities. The authors make their code and datasets publicly available, which supports reproducibility. However, the central claim currently rests on a single trigger per class, a single training run, and a 1,000-example evaluation subset; without variance estimates or additional trigger/target variation, the headline conclusion remains provisional.

major comments (4)

[Section 4.2, Table 3] The central asymmetry is based on one training run and one 1,000-example subset per challenge set, with no error bars, confidence intervals, or significance tests. The contradiction drop is 7.43 percentage points while the entailment and neutral drops exceed 60 points, so the qualitative pattern may well be real, but the paper should quantify sampling and optimization variability. Please report results over multiple seeds/fine-tuning runs, use bootstrap confidence intervals over validation samples, or evaluate on the full validation split; otherwise readers cannot tell how much of the 7.43-point drop is a class property and how much is noise.
[Section 4.1.1, Table 3] Only one universal trigger per class is generated, with one initialization on one surrogate model (ESIM with GloVe) and transferred to one ELECTRA checkpoint. The conclusion that contradiction examples are resilient is therefore vulnerable to trigger-specific effects: the particular trigger 'joyously' may transfer less effectively than 'nobody' or 'cats' for reasons unrelated to class robustness. Please generate multiple triggers per class with different random seeds and initializations and report the distribution of challenge-set accuracies, rather than a single point per class.
[Section 4.1.1, Tables 2 and 4] Target labels are confounded with source classes in the attack design. Table 2 shows that the triggers for entailment and neutral are strongly associated with the contradiction class, while the contradiction trigger is associated with the neutral class; Table 4 then shows that most neutral examples in Challenge Set I are flipped to contradiction, but most contradiction examples are flipped to neutral. This means the observed asymmetry could reflect pairwise class-pair difficulty rather than an inherent resilience of the contradiction class. Please include attacks from each source class to each target label, or at least vary the target label per source class, to isolate the class-resilience claim.
[Section 5.3.4, Eq. (4)] The correlated-word explanation is asserted after the fact and is not directly tested. The paper reports cumulative frequencies of the top five correlated words (312 for contradiction, 128 for neutral, 57 for entailment), but these frequencies are computed over the full SNLI training distribution, not over the challenge examples. The paper does not show that the contradiction examples that survive the attack actually contain those correlated words, nor that the misclassified contradiction examples are concentrated in examples lacking them. I recommend computing flip rates conditioned on the presence or absence of the identified correlated words, and on the specific trigger used, before claiming that correlated words explain the resilience.

minor comments (5)

[Section 1, Section 4.3, Table 2 captions] There are several typos and formatting issues: 'evaluate's' in the Introduction, the GitHub URL appears as 'SNLI-Attack-Analysis.g it' in Section 4.3, and Table 2's column header has a misplaced space in 'Majori ty Class'.
[Sections 3.1.2 and 4.1.1] The trigger length is never explicitly reported. The text says 'we start by selecting a trigger length' and the tables show single-word triggers, but the paper should state the chosen length and whether any length ablation was performed, since longer triggers are known to be more effective in the original universal trigger framework.
[Section 4.2, Section 5.1] The construction of the 1,000-example evaluation subset is underspecified: it is unclear whether the 1,000 examples are per class or in total, and no random seed is given. Please state the sampling scheme exactly so that the numbers in Table 3 can be reproduced.
[Section 4.3, Section 5.2] Fine-tuning hyperparameters other than batch size and number of epochs are not reported. Saying 'all other parameters were left at their default settings' depends on the library version and is not reproducible; please report learning rate, optimizer, warmup, and any relevant defaults.
[Section 5.3.4] The sentence 'This abundance of correlated words makes contradictions particularly vulnerable' appears to contradict the paper's own finding that contradictions are resilient. Please rephrase to clarify that correlated words make contradictions harder to flip away from, or explain the intended distinction between vulnerability and attack resistance.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the robustness comparison uses triggers generated on ESIM and measured on ELECTRA against external SNLI validation data, and the post-hoc explanation relies on dataset statistics independent of the fitted model.

full rationale

The paper is an empirical study, not a derivation. Its central claim is the class-level accuracy asymmetry in Table 3: universal triggers drop entailment accuracy from 90.23% to 25.78% and neutral from 86.70% to 25.76%, while contradiction drops only from 91.06% to 83.63%. The triggers are generated with ESIM and GloVe embeddings (Section 4.1.1) and then transferred to ELECTRA, so the attack inputs are not fitted to the target model whose robustness is being measured. The random-trigger baseline and the 1,000-example challenge sets are constructed from the SNLI validation split, which is external to the model's training data. The inoculation experiment fine-tunes on a Trigger-Augmented dataset built from the training split with the same universal triggers and then reports measured validation accuracy; this is a standard control from the cited external method (Liu et al., 2019), not a quantity forced by the paper's own construction. The Section 5.3.4 explanation invokes SNLI corpus statistics (cumulative frequency of top correlated words: 312 for contradiction vs 128 for neutral vs 57 for entailment) computed from p(label|word), which is independent of the ELECTRA model's predictions. That explanation is post-hoc and not directly verified by showing which examples flip, but that is an evidentiary weakness rather than circular reasoning. There are no load-bearing self-citations, no uniqueness arguments imported from the authors' own prior work, and no ansatz smuggled through citation. The main limitations are experimental design choices (one trigger per class, a single 1,000-example sample per class, and no target-label variation), which affect how strongly the class-level asymmetry can be generalized, but they do not make the paper's claims equivalent to its inputs by definition.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper's central result is an experimental measurement, so the main uncharged inputs are experimental design choices rather than fitted constants. It relies on prior-work assumptions (transferable universal triggers, SNLI dataset artifacts) and a first-order attack heuristic; no new theoretical entities are introduced. The post hoc top-five correlated words cutoff is the closest thing to an ad hoc modeling choice and directly supports the causal explanation.

free parameters (6)

trigger_length = 1
Universal triggers are a single token, chosen by hand; longer triggers might be more effective and change class-level drops (Section 4.1.2).
trigger_target_label_per_class = entailment->contradiction, neutral->contradiction, contradiction->neutral
Each universal trigger is optimized toward a specific competing label; the choice of target labels is not randomized and may affect the observed resilience of the contradiction class (Table 2, Section 5.3.2).
challenge_set_size = 1000
Each challenge set draws 1,000 examples from the SNLI validation split; this sample size is chosen without justification and affects the stability of reported accuracies (Section 4.2).
trigger_augmented_dataset_size = 6000
Fine-tuning uses 6,000 examples (3,000 modified, 3,000 unmodified), a hand-picked size that affects the restoration results (Section 4.2).
top_five_correlated_words = 5
The causal explanation of contradiction resilience uses the cumulative frequency of the top five correlated words; the choice of five is post hoc and not justified (Section 5.3.4).
fine_tuning_epochs = 1
One-epoch fine-tuning on the augmented set is chosen by the authors; more epochs could change the trade-off (Section 5.2).

assumptions (4)

domain assumption Universal adversarial triggers are transferable across models
Triggers are generated on ESIM with GloVe and evaluated on ELECTRA without white-box access, assuming transferability from Wallace et al. (2019); cited in Section 3.1.
domain assumption SNLI contains dataset artifacts that models exploit
The motivation that hypothesis-only training attains high accuracy is taken from Poliak et al. (2018) and Gururangan et al. (2018), not re-derived in this paper.
standard math HotFlip first-order Taylor approximation yields effective trigger tokens
Trigger search uses a linear approximation of the loss to choose token replacements following Ebrahimi et al. (2018); no convergence or optimality guarantee is given (Section 3.1.2).
ad hoc to paper The 1,000-example validation subset is representative
Baseline and challenge accuracies are computed on a randomly sampled subset, and its representativeness is assumed without confidence intervals or bootstrapping (Section 5.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unpacking the Resilience of SNLI Contradiction Examples to Attacks." pith.science (2026). https://pith.science/paper/FAETSDWH

@misc{pith2026241211172,
  author       = {Pith},
  title        = {Pith review of: Unpacking the Resilience of SNLI Contradiction Examples to Attacks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FAETSDWH}},
  note         = {Machine review of arXiv:2412.11172}
}

read the original abstract

Pre-trained models excel on NLI benchmarks like SNLI and MultiNLI, but their true language understanding remains uncertain. Models trained only on hypotheses and labels achieve high accuracy, indicating reliance on dataset biases and spurious correlations. To explore this issue, we applied the Universal Adversarial Attack to examine the model's vulnerabilities. Our analysis revealed substantial drops in accuracy for the entailment and neutral classes, whereas the contradiction class exhibited a smaller decline. Fine-tuning the model on an augmented dataset with adversarial examples restored its performance to near-baseline levels for both the standard and challenge sets. Our findings highlight the value of adversarial triggers in identifying spurious correlations and improving robustness while providing insights into the resilience of the contradiction class to adversarial attacks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 7 linked inside Pith

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Bowman, Gabor Angeli, Christopher Potts, and Christopher D

Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. http://arxiv.org/abs/1508.05326 A large annotated corpus for learning natural language inference . CoRR, abs/1508.05326

arXiv 2015
[4]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/p17-1152 Enhanced lstm for natural language inference . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics

doi:10.18653/v1/p17-1152 2017
[5]

Le, and Christopher D

Kevin Clark, Minh - Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. http://arxiv.org/abs/2003.10555 ELECTRA: pre-training text encoders as discriminators rather than generators . CoRR, abs/2003.10555

arXiv 2020
[6]

Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 177--190, Berlin, Heidelberg. Springer Berlin Heidelberg

2006
[7]

Ido Dagan, Dan Roth, Mark Sammons, and Fabio Zanzotto. 2013. https://doi.org/10.2200/S00509ED1V01Y201305HLT023 Recognizing textual entailment: Models and applications . Synthesis Lectures on Human Language Technologies, 6(4):1--222. Publisher Copyright: Morgan and Claypool Publishers. All rights reserved

doi:10.2200/s00509ed1v01y201305hlt023 2013
[8]

Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. https://doi.org/10.18653/v1/P18-2006 H ot F lip: White-box adversarial examples for text classification . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31--36, Melbourne, Australia. Association for Computational Linguistics

doi:10.18653/v1/p18-2006 2018

Show all 18 references

[9]

Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A

Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qian...

2020 doi
[10]

Bowman, and Noah A

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A. Smith. 2018. http://arxiv.org/abs/1803.02324 Annotation artifacts in natural language inference data . CoRR, abs/1803.02324

2018 arXiv
[11]

He He, Sheng Zha, and Haohan Wang. 2019. https://doi.org/10.18653/v1/D19-6115 Unlearn dataset bias in natural language inference by fitting the residual . In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 132--142, Hong Ko...

2019 doi
[12]

Robin Jia and Percy Liang. 2017. https://doi.org/10.18653/v1/D17-1215 Adversarial examples for evaluating reading comprehension systems . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021--2031, Copenhagen, Denmark. Associati...

2017 doi
[13]

Liu, Roy Schwartz, and Noah A

Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019. http://arxiv.org/abs/1904.02668 Inoculation by fine-tuning: A method for analyzing challenge datasets . CoRR, abs/1904.02668

2019 arXiv
[14]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. https://doi.org/10.3115/v1/D14-1162 G lo V e: Global vectors for word representation . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pages 1532--1543, Doh...

2014 doi
[15]

Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. http://arxiv.org/abs/1805.01042 Hypothesis only baselines in natural language inference . CoRR, abs/1805.01042

2018 arXiv
[16]

Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. https://doi.org/10.18653/v1/2020.acl-main.442 Beyond accuracy: Behavioral testing of NLP models with C heck L ist . In Proceedings of the 58th Annual Meeting of the Association for Computational Lingu...

2020 doi
[17]

Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. 2021. http://arxiv.org/abs/2005.00174 Universal adversarial attacks with natural triggers for text classification

2021 arXiv
[18]

Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. http://arxiv.org/abs/1908.07125 Universal adversarial triggers for NLP . CoRR, abs/1908.07125

2019 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Bowman, Gabor Angeli, Christopher Potts, and Christopher D

Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. http://arxiv.org/abs/1508.05326 A large annotated corpus for learning natural language inference . CoRR, abs/1508.05326

arXiv 2015

[4] [4]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/p17-1152 Enhanced lstm for natural language inference . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics

doi:10.18653/v1/p17-1152 2017

[5] [5]

Le, and Christopher D

Kevin Clark, Minh - Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. http://arxiv.org/abs/2003.10555 ELECTRA: pre-training text encoders as discriminators rather than generators . CoRR, abs/2003.10555

arXiv 2020

[6] [6]

Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 177--190, Berlin, Heidelberg. Springer Berlin Heidelberg

2006

[7] [7]

Ido Dagan, Dan Roth, Mark Sammons, and Fabio Zanzotto. 2013. https://doi.org/10.2200/S00509ED1V01Y201305HLT023 Recognizing textual entailment: Models and applications . Synthesis Lectures on Human Language Technologies, 6(4):1--222. Publisher Copyright: Morgan and Claypool Publishers. All rights reserved

doi:10.2200/s00509ed1v01y201305hlt023 2013

[8] [8]

Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. https://doi.org/10.18653/v1/P18-2006 H ot F lip: White-box adversarial examples for text classification . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31--36, Melbourne, Australia. Association for Computational Linguistics

doi:10.18653/v1/p18-2006 2018