Pith. sign in

REVIEW 3 major objections 5 minor 59 references

Making Language Models Robust Against Negation

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Further pre-training on two self-supervised polarity tasks makes BERT and RoBERTa substantially better at answering questions that hinge on negation.

desk verdict Solid, simple negation pre-training with real gains, but the CondaQA headline numbers need a dedup check before I'd trust them. read the letter →

arxiv 2502.07717 v1 pith:RXSHRY7H submitted 2025-02-11 cs.CL

MohammadHossein Rezaei , Eduardo Blanco This is my paper · ORCID
classification cs.CL
keywords negationlanguagemodelpre-trainingnextsentencepredictionpolarityreversalself-supervisedlearningBERTRoBERTaCondaQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that language models' well-known blindness to negation can be reduced by a cheap, fully self-supervised pre-training step that forces the model to attend to whether a sentence is negated. The authors introduce two training tasks built from unlabeled Wikipedia text: Next Sentence Polarity Prediction (NSPP), in which the model sees only the first sentence and must predict whether the next sentence contains a negation cue, and a modified Next Sentence Prediction (NSP) in which the negative example is the actual next sentence with its polarity reversed rather than a random sentence. Pre-training BERT and RoBERTa on either task consistently outperforms the off-the-shelf models on nine negation-related benchmarks, with the clearest gains on CondaQA, a question-answering corpus built around negation: accuracy rises 1.8%-9.1% and RoBERTa-base's group consistency jumps from 2.4% to 15.6%-17.6%. If this is right, robustness to negation can be instilled without new annotation, extra data beyond ordinary text, or any change at prediction time.

What carries the argument

The load-bearing mechanism is automatic polarity reversal with hand-built rules. The rules operate only on sentences whose negation cue is not, n't, or never, where the cue modifies the main verb, the sentence is not a question, and exactly one cue is present; adding negation inserts or attaches the cue to the auxiliary or main verb and swaps polarity items (some to any, already to yet), while removing negation deletes the cue, restores the auxiliary's lemma, and swaps back (any to some, yet to already, but to and). This produces the 6.4M training pairs for the two tasks. The NSP variant is the key change: negative examples are true next sentences whose polarity has been flipped, so the model must treat polarity reversal as a coherence violation rather than random unrelatedness.

What would settle it

Train the same models on polarity-reversed pairs in which the reversed sentence has been hand-edited or generated by a grammar-aware system to read naturally, and also on randomly selected negative pairs matched for length and lexical overlap; if the CondaQA gains persist only with the rule-based reversed pairs, or if the gains concentrate on CondaQA items whose cues overlap with not, n't, and never, the claim that the model has learned general negation robustness is undercut.

Watch

Extended reading notes

Core claim

The paper's central claim is that further pre-training on two polarity-focused self-supervised tasks makes BERT and RoBERTa genuinely more robust to negation, not just better on individual datasets. Given only a sentence, the model must predict whether the following sentence will contain negation (NSPP); and in a redesigned NSP, a proposed continuation is judged incoherent when its polarity has been reversed by adding or removing a negation cue, replacing the random-negative sampling of the original task. On CondaQA, all pre-trained variants show statistically significant accuracy gains and large gains in group consistency, the metric that requires answering correctly across original, paraphrased, scope-changed, and affirmed versions of the same passage. The authors interpret this as evidence that the model has learned something about how negation alters meaning and discourse coherence, and that this transfers to a corpus whose 200+ negation cues go well beyond the three cues used to create the training data.

Load-bearing premise

The whole method rests on the assumption that automatically reversing polarity in Wikipedia sentences, using rules validated on 100 examples and limited to not, n't, and never, produces training examples whose only meaningful difference is the presence or absence of negation, with no systematic grammatical or stylistic artifact for the model to latch onto.

Editorial extensions

If this is right

  • Further pre-training on NSP gives the largest and most consistent gains; it outperforms pre-training on NSPP and, except for RoBERTa-large, joint training does not add benefit.
  • The gains transfer beyond the three cues used in training: CondaQA contains over 200 negation cue types, including affixal and multi-word cues, yet accuracy improves there most.
  • Models pre-trained on polarity reversal outperform prior negation-specific approaches, including affirmative-interpretation augmentation and unlikelihood or negation-augmented pre-training, without extra inference-time computation.
  • On LAMA-Neg, error rates drop by 0.09 to 19.96 points, while on original LAMA and NLI/NLU datasets without negation, performance stays within roughly plus or minus 1.65%, indicating the pre-training does not trade away general ability.
  • Ablations show that mixing added and removed negation, that is true polarity reversal, beats training on either direction alone, and adding negation is a stronger training signal than removing it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The manual reversal rules were validated on only 100 sentences, with 96% judged correct; if the 4% errors and grammatically awkward outputs such as 'She might have not been sleeping' create surface regularities, part of the CondaQA gain could come from learning those artifacts rather than negation semantics. A control study with human-edited reversed sentences would separate these.
  • Because NSPP's signal is weaker and joint training sometimes hurts, the two tasks may be teaching overlapping but partly competing skills: detecting a cue's presence versus judging a discourse-coherence violation. This suggests the coherence-based NSP is the more promising route for further work.
  • The method's restriction to not, n't, and never leaves open whether a richer set of cues in pre-training would extend the gains to negation types absent from CondaQA, or to languages with different negation morphology; the authors note the tasks are language-agnostic but only test English.
  • Pre-training on polarity-reversed sentences might serve as a general-purpose robustness intervention for other semantic phenomena with a similar duality, such as tense or modality, where flipping a sentence's value creates a natural negative training signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces two self-supervised further pre-training tasks, Next Sentence Polarity Prediction (NSPP) and a polarity-reversed variant of Next Sentence Prediction (NSP), and applies them to BERT and RoBERTa base/large models. The authors construct approximately 6.4M (S1, S2) sentence pairs from a 2024 Wikipedia dump using rule-based polarity reversal, further pre-train on subsets of up to 1M pairs, and evaluate on CondaQA and eight other negation-related benchmarks. They report statistically significant accuracy gains on CondaQA for all model configurations, with RoBERTa-base group consistency rising from 2.4% to 15.6-17.6%, and also report improvements on NLI, NLU, and LAMA-Neg benchmarks. The paper's central claim is that this self-supervised recipe provides general robustness against negation.

Significance. If the gains are real and not an artifact of benchmark overlap, this is a practical contribution: it offers a simple, self-supervised, architecture-agnostic way to improve negation handling in smaller language models, with publicly released code and data. The use of McNemar's test for the main CondaQA accuracy comparisons is a strength relative to much prior work that reports only point estimates. However, the headline CondaQA result is vulnerable to train/eval overlap with the 2024 Wikipedia snapshot, and the LAMA degradation of the jointly pre-trained RoBERTa models conflicts with the paper's no-harm claim on non-negated inputs. The significance of the contribution therefore hinges on the outcome of a deduplication analysis.

major comments (3)
  1. [§3.3, §4.1] The pre-training corpus is built from a 2024 Wikipedia dump (Section 3.3), and CondaQA passages are also extracted from English Wikipedia (Section 4.1), but the paper never states that the pre-training data were deduplicated against CondaQA or any other downstream benchmark. Because the snapshot postdates CondaQA's construction, exact CondaQA passages and their neighboring sentences can appear in the NSPP/NSP training pairs, especially since the method extracts all sentences containing negation together with their preceding sentences. This makes it impossible to attribute the reported CondaQA gains (Table 1) to general negation robustness rather than to direct exposure to the evaluation text. Please report overlap statistics (e.g., exact sentence matches, high-n-gram overlaps) between the pre-training corpus and the CondaQA passages, and ideally rerun the main CondaQA evaluation using a deduplicated corpus.
  2. [§6.2, Table 5] The text states that "the fact that the models remain within ±1.65% of the vanilla models" demonstrates maintenance of performance on non-negated input, but this is contradicted by the RoBERTa joint models in Table 5: RoBERTa-base + NSPP+NSP drops from 14.77 to 12.06 on ConceptNet and from 11.93 to 6.76 on TREx, and RoBERTa-large + NSPP+NSP drops from 15.48 to 3.68 on TREx. The sentence acknowledges "except for RoBERTa models jointly pre-trained" but the conclusion does not incorporate the magnitude of this exception. Additionally, Appendix F states that EWC regularization is applied only for the LAMA/LAMA-Neg evaluations, so those models are pre-trained with a different objective; please clarify whether the LAMA results come from a separate pre-training run and whether the no-harm claim is intended to be limited to that configuration.
  3. [§3.3] When constructing the affirmative sentences, the authors "verify that none of the negation cues from CondaQA are present." This imports the evaluation benchmark's cue list into the pre-training data construction. The choice is not motivated in the paper, and it means the pre-training distribution is shaped by one specific downstream test. Please justify this choice or show that the results are insensitive to using a generic, benchmark-independent negation cue list.
minor comments (5)
  1. [§6.1] The text reports "1.8%-9.3% higher accuracy" while the abstract and Table 1 support a maximum of 9.1% (RoBERTa-base, 48.2 to 57.3); please correct the inconsistency.
  2. [Table 8] The header contains a typo: "BEBT-base" should be "BERT-base".
  3. [Appendix F] It would help to state explicitly that the LAMA/LAMA-Neg models are pre-trained with EWC and therefore are not the same checkpoints as the CondaQA/NLI/NLU models, so results across tables are not directly comparable.
  4. [Table 1] For BERT-base, NSPP and NSPP+NSP group consistency "All" (2.7% and 2.9%) are lower than the vanilla 3.4%; the text calls this "marginal," but it is a decrease, so please phrase the exception more precisely.
  5. [Figure 8] The QNLI example label "Not Entailment (i.e., question is not answered)" is confusing because the parenthetical gloss does not fully capture the entailment distinction; consider clarifying the label.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the pre-training objectives are self-supervised and the CondaQA results come from held-out human labels, not from the task definitions.

full rationale

The paper's derivation chain is a standard empirical pipeline: define the NSPP objective and the polarity-reversed NSP objective, generate training pairs from Wikipedia using explicit reversal rules, further pre-train BERT and RoBERTa, then fine-tune and evaluate on CondaQA and eight other benchmarks. No reported improvement is an identity or a fitted parameter renamed as a prediction. The NSP negative label ('S' is not the next sentence') is indeed defined by construction, but the downstream CondaQA accuracy and group-consistency scores are computed from crowd-authored labels on passages that are not used as training labels, so the gains are not forced by the pre-training objective. Self-citations to Rezaei and Blanco (2024) and Hossain et al. (2020; 2022) are used only as baselines, benchmark sources, or fine-tuning implementations; they are not load-bearing evidence for the central claim. Two data-hygiene caveats, while not circularity, are explicitly noted: Section 3.3 builds the pre-training corpus from the 2024 Wikipedia dump without documented deduplication against CondaQA, whose passages also come from Wikipedia, and the same section filters affirmative sentences by checking that none of CondaQA's negation cues are present. These are testable contamination concerns, not reductions of the prediction to its inputs by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests mainly on hand-built polarity reversal rules, a set of pre-training hyperparameters, and standard pretrained-model background. There is no mathematical derivation and no new theoretical entity. The largest burden is the assumption that noisy rule-based reversal of only three negation cues produces a transferable negation signal.

free parameters (5)
  • pre-training learning rate = 1e-6
    Set uniformly for all NSPP, NSP, and joint pre-training runs (Section 5.1, Appendix F). The reported results depend on this choice, though it is a standard further-pretraining value.
  • early stopping patience = 3 epochs
    Used to stop pre-training when validation loss plateaus (Section 5.1). This determines how much the model trains before downstream evaluation.
  • EWC regularization coefficient = 1e-3
    Applied only when evaluating LAMA and LAMA-Neg to reduce catastrophic forgetting (Appendix F). The LAMA and LAMA-Neg reported numbers depend on this coefficient.
  • pre-training corpus size per model = 1M pairs (base), 500K pairs (large)
    Chosen by validation loss plateau (Section 5.1, Figure 2). The gains on CondaQA are reported at these sizes, not at the full corpus size.
  • fine-tuning learning rates and batch sizes = Varies by model and task (Appendix F, Table 8)
    Each downstream task uses a separately chosen learning rate and batch size. These choices affect the reported benchmark numbers.
assumptions (5)
  • domain assumption Off-the-shelf BERT and RoBERTa are adequate base models, and further masked language modeling is redundant.
    Section 5.1 states further MLM is redundant because the models are already pre-trained on Wikipedia. This justifies restricting pre-training to the NSPP and NSP heads.
  • ad hoc to paper Rule-based polarity reversal on Wikipedia sentences restricted to not, n't, never as main-verb cues produces training pairs whose salient change is polarity.
    Section 3.2.1; the manual check on 100 samples (96% correct) leaves a 4% error rate and does not measure how often reversed sentences are grammatically awkward or introduce other surface artifacts.
  • ad hoc to paper The polarity of the next sentence is predictable enough from the current sentence in Wikipedia to make NSPP learnable and useful.
    NSPP requires a signal from S1 to S2. If no such signal existed, the task would reduce to learning the 50/50 base rate and would not transfer to negation reasoning.
  • domain assumption Knowledge gained from three English negation cues transfers to the full range of negation cues in CondaQA.
    Training data only includes not, n't, and never (Section 3.2.1, Limitations). The claim of general robustness depends on transfer to CondaQA's 200+ cues.
  • standard math McNemar's test on a single run per model is a valid significance test for the reported comparisons.
    Used to claim statistical significance (Section 5.2, Table 1). The test captures paired prediction differences but not run-to-run seed variance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Making Language Models Robust Against Negation." pith.science (2026). https://pith.science/paper/RXSHRY7H

@misc{pith2026250207717,
  author       = {Pith},
  title        = {Pith review of: Making Language Models Robust Against Negation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXSHRY7H}},
  note         = {Machine review of arXiv:2502.07717}
}
read the original abstract

Negation has been a long-standing challenge for language models. Previous studies have shown that they struggle with negation in many natural language understanding tasks. In this work, we propose a self-supervised method to make language models more robust against negation. We introduce a novel task, Next Sentence Polarity Prediction (NSPP), and a variation of the Next Sentence Prediction (NSP) task. We show that BERT and RoBERTa further pre-trained on our tasks outperform the off-the-shelf versions on nine negation-related benchmarks. Most notably, our pre-training tasks yield between 1.8% and 9.1% improvement on CondaQA, a large question-answering corpus requiring reasoning over negation.

Figures

Figures reproduced from arXiv: 2502.07717 by the authors.

Figure 1
Figure 1. An example of the training data for our self-supervised tasks. The tasks are: (a) given a sen￾tence, predict whether the next sentence will contain negation (NSPP) and (b) given two sentences, predict whether the second sentence is a coherent continuation of the first one (NSP). [MASK], BERT predicts dog as the masked token both in the original sentence and when the nega￾tion cue not is added to the sentence, A beag… view at source ↗
Figure 2
Figure 2. Trends in pre-training transformers on NSPP, NSP, and both tasks jointly from left to right. Validation loss [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of Llama-2-7B failing to remove the negation cue from a sentence. The model resists removing [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Examples of prompting ChatGPT to remove negation cues from a sentence. In the first example, the [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: An example of Llama-2-7B adding the negation cue “not” to a sentence. The model resists adding the [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Three types of edits in CondaQA are applied [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: An example from CondaQA. The original passage contains a sentence with negation. The crowdworker [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Examples from the NLU and NLI corpora used in this work. The corpora include natural language [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 24 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. https://api.semanticscholar.org/CorpusID:13385138 The second pascal recognising textual entailment challenge

    work page 2006

  4. [4]

    Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. 2009. https://tac.nist.gov/publications/2009/additional.papers/RTE5\_overview.proceedings.pdf The fifth PASCAL recognizing textual entailment challenge . In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009 . NIST

    work page 2009

  5. [5]

    Bowman, Gabor Angeli, Christopher Potts, and Christopher D

    Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. https://doi.org/10.18653/v1/D15-1075 A large annotated corpus for learning natural language inference . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632--642, Lisbon, Portugal. Association for Computational Linguistics

    doi:10.18653/v1/d15-1075 2015

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  7. [7]

    Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 177--190, Berlin, Heidelberg. Springer Berlin Heidelberg

    2006

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

Show all 59 references
  1. [9]

    Radina Dobreva and Frank Keller. 2021. https://doi.org/10.18653/v1/2021.blackboxnlp-1.27 Investigating negation in pre-trained vision-and-language models . In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 350--362, ...

    2021 doi

  2. [10]

    Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. https://aclanthology.org/L18-1544 T - RE x: A large scale alignment of natural language with knowledge base triples . In Proceedings of the Elevent...

    2018

  3. [11]

    Allyson Ettinger. 2020. https://doi.org/10.1162/tacl_a_00298 What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models . Transactions of the Association for Computational Linguistics, 8:34--48

    2020 doi

  4. [12]

    Federico Fancellu, Adam Lopez, Bonnie Webber, and Hangfeng He. 2017. https://aclanthology.org/E17-2010 Detecting negation scope is easy, except when it isn ' t . In Proceedings of the 15th Conference of the E uropean Chapter of the Association for Computational Linguistics: Vo...

    2017

  5. [13]

    Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. https://aclanthology.org/W07-1401 The third PASCAL recognizing textual entailment challenge . In Proceedings of the ACL - PASCAL Workshop on Textual Entailment and Paraphrasing , pages 1--9, Prague. Associa...

    2007

  6. [14]

    Matthew Honnibal and Mark Johnson. 2015. https://doi.org/10.18653/v1/D15-1162 An improved non-monotonic transition system for dependency parsing . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1373--1378, Lisbon, Portugal. Ass...

    2015 doi

  7. [15]

    Matthew Honnibal and Ines Montani. 2017. spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing. To appear

    2017

  8. [16]

    Md Mosharaf Hossain and Eduardo Blanco. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.393 Leveraging affirmative interpretations from negation improves natural language understanding . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processin...

    2022 doi

  9. [17]

    Md Mosharaf Hossain, Dhivya Chinnappa, and Eduardo Blanco. 2022. https://doi.org/10.18653/v1/2022.acl-short.81 An analysis of negation in natural language understanding corpora . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume...

    2022 doi

  10. [18]

    Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.732 An analysis of natural language inference benchmarks through the lens of negation . In Proceedings of the 2020 Conference...

    2020 doi

  11. [19]

    Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, and Aaron Courville. 2021. https://doi.org/10.18653/v1/2021.naacl-main.102 Understanding by understanding not: Modeling negation in language models . In Proceedings of the 2021 Conference of the N...

    2021 doi

  12. [20]

    Myeongjun Jang, Frank Mtumbuka, and Thomas Lukasiewicz. 2022. https://doi.org/10.18653/v1/2022.findings-naacl.156 Beyond distributional hypothesis: Let language models learn meaning-text correspondence . In Findings of the Association for Computational Linguistics: NAACL 2022,...

    2022 doi

  13. [21]

    Jaap Jumelet and Dieuwke Hupkes. 2018. https://doi.org/10.18653/v1/W18-5424 Do language models understand anything? on the ability of LSTM s to understand negative polarity items . In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Netw...

    2018 doi

  14. [22]

    Nora Kassner and Hinrich Sch \"u tze. 2020. https://doi.org/10.18653/v1/2020.acl-main.698 Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p...

    2020 doi

  15. [23]

    Aditya Khandelwal and Suraj Sawant. 2020. https://aclanthology.org/2020.lrec-1.704 N eg BERT : A transfer learning approach for negation detection and scope resolution . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5739--5748, Marseille, Fr...

    2020

  16. [24]

    Daniel Khashabi, Yeganeh Kordi, and Hannaneh Hajishirzi. 2022. http://arxiv.org/abs/2202.12359 Unifiedqa-v2: Stronger generalization via broader cross-format training

    2022 arXiv

  17. [25]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2017. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization

    2017 arXiv

  18. [26]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. https://doi.org/10.1073/pnas.1...

    2017 doi

  19. [27]

    Levesque, Ernest Davis, and Leora Morgenstern

    Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012. https://cs.nyu.edu/faculty/davise/papers/WSKR2012.pdf The Winograd Schema Challenge . In Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning , KR '12, page...

    2012

  20. [28]

    Hao Li and Wei Lu. 2018. https://doi.org/10.18653/v1/P18-2085 Learning with structured representations for negation scope extraction . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 533--539, Melbourne...

    2018 doi

  21. [29]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach

    2019 arXiv

  22. [30]

    Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153--157

    1947

  23. [31]

    Roser Morante and Walter Daelemans. 2009. https://aclanthology.org/W09-1105 A metalearning approach to processing the scope of negation . In Proceedings of the Thirteenth Conference on Computational Natural Language Learning ( C o NLL -2009) , pages 21--29, Boulder, Colorado. ...

    2009

  24. [32]

    Roser Morante and Walter Daelemans. 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/221_Paper.pdf C onan D oyle-neg: Annotation of negation cues and their scope in conan doyle stories . In Proceedings of the Eighth International Conference on Language Resources and Eva...

    2012

  25. [33]

    Roser Morante, Sarah Schrauwen, and Walter Daelemans. 2011. Annotation of negation cues and their scope: Guidelines v1. Technical Report CTRS-003, Computational Linguistics and Psycholinguistics Technical Report Series

    2011

  26. [34]

    OpenAI. 2023. http://arxiv.org/abs/2303.08774 Gpt-4 technical report

    2023 arXiv

  27. [35]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  28. [36]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fan...

    2019 arXiv

  29. [37]

    Fabio Petroni, Tim Rockt \"a schel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. https://doi.org/10.18653/v1/D19-1250 Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language P...

    2019 doi

  30. [38]

    Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. https://doi.org/10.18653/v1/N19-1128 W i C : the word-in-context dataset for evaluating context-sensitive meaning representations . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association ...

    2019 doi

  31. [39]

    Zhong Qian, Peifeng Li, Qiaoming Zhu, Guodong Zhou, Zhunchen Luo, and Wei Luo. 2016. https://doi.org/10.18653/v1/D16-1078 Speculation and negation scope detection via convolutional neural networks . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language...

    2016 doi

  32. [40]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-...

    2016 doi

  33. [41]

    Abhilasha Ravichander, Matt Gardner, and Ana Marasovic. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.598 CONDAQA : A contrastive reading comprehension dataset for reasoning about negation . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Pro...

    2022 doi

  34. [42]

    MohammadHossein Rezaei and Eduardo Blanco. 2024. https://doi.org/10.18653/v1/2024.acl-short.55 Paraphrasing in affirmative terms improves negation understanding . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers...

    2024 doi

  35. [43]

    Sabine Rosenberg and Sabine Bergler. 2012. https://aclanthology.org/S12-1039 UC oncordia: CL a C negation focus detection at * S em 2012 . In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the s...

    2012

  36. [44]

    Zahra Sarabi, Erin Killian, Eduardo Blanco, and Alexis Palmer. 2019. https://doi.org/10.18653/v1/S19-1017 A corpus of negations and their underlying positive interpretations . In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (* SEM 2019) , p...

    2019 doi

  37. [45]

    Longxiang Shen, Bowei Zou, Yu Hong, Guodong Zhou, Qiaoming Zhu, and AiTi Aw. 2019. https://doi.org/10.18653/v1/D19-1230 Negative focus detection via contextual attention mechanism . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and t...

    2019 doi

  38. [46]

    Rituraj Singh, Rahul Kumar, and Vivek Sridhar. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.873 NLM s: Augmenting negation in language models . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13104--13116, Singapore. Association for Com...

    2023 doi

  39. [47]

    Robyn Speer and Catherine Havasi. 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/1072_Paper.pdf Representing general relational knowledge in C oncept N et 5 . In Proceedings of the Eighth International Conference on Language Resources and Evaluation ( LREC '12) , page...

    2012

  40. [48]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  41. [49]

    Thinh Truong, Timothy Baldwin, Trevor Cohn, and Karin Verspoor. 2022. https://doi.org/10.18653/v1/2022.naacl-main.309 Improving negation detection with negation-focused pre-training . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Co...

    2022 doi

  42. [50]

    Thinh Hung Truong, Timothy Baldwin, Karin Verspoor, and Trevor Cohn. 2023. https://doi.org/10.18653/v1/2023.starsem-1.10 Language models are not naysayers: an analysis of language models on negation benchmarks . In Proceedings of the 12th Joint Conference on Lexical and Comput...

    2023 doi

  43. [51]

    Chantal van Son, Emiel van Miltenburg, and Roser Morante. 2016. https://aclanthology.org/W16-5007 Building a dictionary of affixal negations . In Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics ( E x P ro M ) , pages 49--56, O...

    2016

  44. [52]

    Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2020. http://arxiv.org/abs/1905.00537 Superglue: A stickier benchmark for general-purpose language understanding systems

    2020 arXiv

  45. [53]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/W18-5446 GLUE : A multi-task benchmark and analysis platform for natural language understanding . In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : A...

    2018 doi

  46. [54]

    Wikimedia-Foundation. 2024. https://dumps.wikimedia.org Wikimedia downloads

    2024

  47. [55]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

    2018 doi

  48. [56]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  49. [57]

    Yiyun Zhao and Steven Bethard. 2020. https://doi.org/10.18653/v1/2020.acl-main.429 How does BERT ' s attention change when you fine-tune? an analysis methodology and a case study in negation scope . In Proceedings of the 58th Annual Meeting of the Association for Computational...

    2020 doi

  50. [58]

    Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2014. https://doi.org/10.3115/v1/P14-1049 Negation focus identification with contextual discourse information . In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    2014 doi

  51. [59]

    Bowei Zou, Guodong Zhou, and Qiaoming Zhu. 2015. https://doi.org/10.18653/v1/D15-1187 Unsupervised negation focus identification with word-topic graph model . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1632--1636, Lisbon, P...

    2015 doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.