Pith. sign in

REVIEW 4 major objections 6 minor 41 references

CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A lightweight unsupervised pipeline removes conceptual shortcuts from frozen language-model embeddings, gaining up to 10 F1 points on out-of-distribution sentiment tests.

desk verdict CURE is a sensible, lightweight concept-debiasing pipeline with a genuinely useful controllability knob, but the headline OOD gains are not yet backed by enough evidence—single runs, unreported hyperparameters, and an evaluation loop that shares the same concept ontology. read the letter →

arxiv 2509.05230 v2 pith:UE7RC6IA submitted 2025-09-05 cs.CL cs.AIcs.LG

Aysenur Kocak , Shuo Yang , Bardh Prenkaj , Gjergji Kasneci This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords conceptualshortcutsspuriouscorrelationsout-of-distributionrobustnessembeddingdebiasingcontrastivelearningcontentextractionsentimentanalysisunlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CURE, an unsupervised and model-agnostic method to remove concept-level spurious correlations from pre-trained language model embeddings while preserving task-relevant content. It trains a lightweight content extractor that forces a concept classifier to become maximally uncertain, uses a reversal network to ensure content is not lost, and then adds a contrastive debiasing module that can either suppress or amplify residual concept cues. The central empirical claim is that CURE achieves about +10 F1 points on the IMDB out-of-distribution test and about +2 points on Yelp, with roughly 2% additional parameters and training time reduced to about one-tenth of LLM-driven debiasing. If correct, this offers a cheap, controllable alternative to counterfactual data augmentation and lexical debiasing for building more robust and fair text classifiers.

What carries the argument

The content extractor f_phi, a lightweight network applied to frozen PLM outputs, is trained jointly toward two objectives: a KL divergence loss that pushes the concept classifier f_omega's predictions toward a scaled uniform distribution, erasing concept information, and a reconstruction loss through a reversal network f_hat_phi that preserves non-concept content. A third component, the debiasing network f_psi, uses contrastive cosine-similarity losses with a tunable margin M to regulate how much residual concept information survives in the final embedding. This machinery translates the abstract goal of removing conceptual shortcuts into a concrete, controllable optimization problem on froz

What would settle it

Train a fresh concept classifier on the final CURE embeddings f_psi(x) and test it on held-out concept labels; if its accuracy is substantially above chance, then conceptual information survives the debiasing and the reported OOD gains come from some other regularization effect rather than from concept removal.

Watch

Extended reading notes

Core claim

CURE claims that conceptual shortcuts in PLM embeddings can be disentangled from content without human annotation or LLM-based data augmentation. The method freezes the PLM, trains a content extractor f_phi so that a separately trained concept classifier f_omega becomes maximally uncertain about concept labels, and simultaneously trains a reversal network to reconstruct the original embedding, thereby retaining content information. A subsequent contrastive debiasing network f_psi re-maps the original and content embeddings and controls their cosine similarity with a margin M, allowing the user to weaken or strengthen the influence of conceptual features. On IMDB and Yelp across DistilBERT, M

Load-bearing premise

The whole removal mechanism assumes that the trained concept classifier f_omega is a faithful and sufficient probe of all conceptual information, so that making it maximally uncertain genuinely erases concepts from the embedding rather than merely fooling that one classifier.

Editorial extensions

If this is right

  • On the paper's benchmarks, CURE improves OOD F1 by about 10 points on IMDB and about 2 points on Yelp, with only about 2% additional parameters and inference time nearly identical to the base PLM.
  • Training time drops to roughly one-tenth of LLM-driven methods because CURE does not require LLM-generated counterfactual examples during training; the only LLM use is a one-time concept annotation step.
  • The margin M gives users quantitative control over shortcut retention, allowing the same framework to either improve OOD robustness or exploit beneficial correlations for i.i.d. performance.
  • The method is model-agnostic in design and is demonstrated on three PLMs plus a preliminary LLaMA3-1B evaluation, suggesting it can be added to existing classifiers without retraining the backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because CURE operates on frozen embeddings, the same content-extractor plus debiasing stack could be retargeted to other spurious attributes such as demographic terms, genre, or product category without fine-tuning the backbone; the paper demonstrates controllability on sentiment concepts only.
  • The margin M behaves as a tunable dial between robust and i.i.d.-specialized behavior; a natural next experiment is to select M automatically on a validation OOD shift instead of treating it as a hyperparameter.
  • The reversal network ablation indicates that simple information loss is insufficient; if concept and content are genuinely separable in PLM latent spaces, CURE's gains also provide an estimate of how much of a sentiment model's signal is concept-level rather than content-level, which could inform interpretability tools.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CURE, a three-stage method to remove concept-level shortcuts from frozen PLM embeddings while retaining content information. Stage 1 (Sec. 3.2) labels documents with GPT-4o concepts and splits data based on mutual information with the task label. Stage 2 (Sec. 3.3) trains a lightweight content extractor f_phi with a KL loss (Eq. 7) that makes a frozen concept classifier f_omega uncertain, plus a reconstruction loss (Eqs. 8-10) using a reversal network to preserve content. Stage 3 (Sec. 3.4) applies a contrastive debiasing network with margin M (Eqs. 11-12) to either suppress or enhance concept information, then trains a classification head on the transformed embeddings. On IMDB and Yelp, CURE reports OOD F1 gains up to +10 (RoBERTa IMDB) and +2 (Yelp) with about 2% parameter overhead and training time roughly one-tenth of the LLM-driven RAZOR baseline.

Significance. If the claims hold, CURE would be a practically valuable, model-agnostic debiasing method for frozen embeddings, avoiding per-task LLM counterfactual generation. The paper ships code, includes an ablation showing the reversal network has a large effect (Table 5), provides a SHAP case study, a small human evaluation of concept annotations (Appendix A.3), and a LLaMA3-1B supplementary result. The reported parameter overhead (~2%) and training-time reduction are notable. However, the central claims are currently undermined by four load-bearing issues: a possible training/test leakage in the algorithm description, the circularity of the concept-removal validation, missing hyperparameters and error bars, and an OOD evaluation built from the same concept ontology that CURE is designed to remove.

major comments (4)
  1. [Sec. 4.1 vs. Appendix A.1] Algorithm 1 is written over the full dataset D and never references the Group A/Group B split defined in Sec. 4.1. Steps 4, 5, and 8 train the reversal network, content extractor, and debiasing module on D, which could include the OOD test set. If any unsupervised component is fit on Group B, Table 1's OOD results are invalid. Please state explicitly that f_omega, f_phi, the reversal network, and psi are trained only on Group A, with Group B used exclusively for evaluation.
  2. [Sec. 3.3.1, Eqs. (5)-(10)] Eq. (7) only forces the frozen probe f_omega to be uncertain on f_phi(x). Since Eq. (9) preserves enough information to reconstruct x, conceptual information may survive in directions f_omega does not read out. The paper presents no independent probe; the only evidence is that f_omega's own predictions become uniform, which holds by construction. Please add experiments with a second, independently-trained probe (different architecture or label set) and/or a linear separability test on f_phi(x) to show that the Table 1 OOD gains are due to concept removal rather than bottleneck or regularization effects.
  3. [Sec. 4.1, Table 1, Fig. 4] The main quantitative claims rest on single runs without error bars or significance tests. Moreover, the margin M in Eqs. (11)-(12), shown in Fig. 4 to strongly affect accuracy, is not specified for Table 1; lambda (Eq. 10), tau (Eq. 7), and the number k of concepts used for the biased/OOD split (Sec. 3.2) are also unreported. Without these values and variance estimates, the headline +10/+2 F1 improvements are not reproducible and the reported differences may not be stable.
  4. [Sec. 3.2, Table 1] The OOD test is constructed from the same GPT-4o meta-concept set C that CURE is then trained to remove (top-k/low mutual-information split). This makes the evaluation aligned with the method's objective and does not demonstrate robustness to concept shifts beyond C. I ask for a test with independently annotated concepts (e.g., a different concept ontology or human-defined concepts) or an evaluation on an existing OOD benchmark to validate that the gains reflect conceptual debiasing rather than overfitting to the particular GPT-4o concept taxonomy.
minor comments (6)
  1. [Sec. 4.3, Eqs. (11)-(12)] 'adjusting the value of the margin M in eq. (11) and eq. (11)' should refer to Eqs. (11) and (12); also lowercase 'm' in the sentence after Eq. (12) should be uppercase M.
  2. [Eq. (7)] The target '(1/|C|)^tau' is not a normalized distribution for tau != 1. Please define the 'scaled uniform distribution' formally and report the value of tau used.
  3. [Sec. 3.3.2] The method is described as 'back-translation' but it is a reconstruction/auto-encoding loss (Eqs. (8)-(9)). Please adjust the terminology to avoid confusion with machine-translation back-translation.
  4. [Table 1] The placement of the 'i.i.d.' and 'OOD' labels is visually ambiguous; using a separate row/column header or a panel would make the table easier to read.
  5. [Sec. 4.1] The exact number of concepts in C and the value of k for the Group A/Group B split are not stated. Include these for reproducibility.
  6. [Abstract and Limitations] The abstract calls CURE 'unsupervised,' but the pipeline uses GPT-4o concept labels (Sec. 3.2). Although the Limitations section acknowledges this, consider qualifying the abstract wording to 'unsupervised with respect to the target task' or similar.

Circularity Check

1 steps flagged · score 4.0 of 10

CURE's concept-removal validation reduces to its own training objective: f_omega's uncertainty is enforced, then cited as evidence of concept absence.

  1. self definitional [Sec. 3.3.1, Eq. (7)]
    "With the training of fϕ, we force the concept classifier ω to produce maximally uncertain predictions, indicating the absence of learnable conceptual information."

    The content extractor f_phi is trained by minimizing Eq. (7), the KL divergence between the frozen concept classifier f_omega's predicted distribution over C and a scaled uniform distribution. Therefore f_omega's uncertainty on f_phi(x) is exactly the training objective, not an independent measurement. The paper then presents this enforced uncertainty as evidence that conceptual information is absent ('absence of learnable conceptual information'). This conclusion derives from the loss that defines it; unless f_omega is proven to be a complete/sufficient probe of all concept information in x, the uncertainty only shows that this particular classifier has been fooled. The OOD benchmark (Sec. 3.2) is constructed from the same concept set C, so it is aligned with whatever f_omega can detect;

full rationale

The paper's headline OOD improvements (+10 F1 on IMDB, +2 on Yelp) are measured on held-out test sets and compared against external baselines (FL, RAZOR), so the central empirical result is not circular. However, the paper's explanation for why CURE works—that it removes conceptual shortcuts—rests on the claim that f_omega's uncertainty indicates the absence of conceptual information. This is circular in the narrow sense: the uncertainty is the training loss of Eq. (7), not an independent diagnostic. The concept set C is also used both to train f_omega and to define the biased/OOD split in Sec. 3.2, so the evaluation is aligned with the probe that CURE is trained to fool. An independent probe trained on the original embeddings could in principle still decode concepts from f_phi(x), which would mean the OOD gains come from regularization or other side effects rather than from the claimed unlearning. The paper does not provide such an independent check. Because the empirical OOD results have independent content, this is partial circularity, not full equivalence; the score reflects the self-definitional validation of the removal mechanism while acknowledging the external test-set results.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method's central claims rest on the quality of GPT-4o concept labels, on the concept classifier being a faithful probe, on the reconstruction loss preserving task content, and on the MI-based split defining a valid OOD test. Four hyperparameters (M, lambda, tau, k) are unreported and materially affect the results. No new physical or statistical entities are postulated; the content extractor, reversal network, and debiasing module are network components, not invented entities.

free parameters (4)
  • M (margin) = not reported (swept in Fig. 4)
    Controls how much conceptual information is retained (Eqs. 11-12); OOD and i.i.d. results in Table 1 depend on its value, which is not stated.
  • lambda (content retention weight) = not reported
    Balances L_concept and L_content in Eq. (10); value unspecified and materially affects the content extractor.
  • tau (reference scale in Eq. (7)) = not reported
    Scales the target distribution (1/|C|)tau in the concept-dropout loss; the strength of concept removal depends on it.
  • k (number of concepts for biased/OOD splits) = not reported
    Selects top-k and bottom-k concepts for the training set and the OOD test in Sec. 3.2; determines benchmark difficulty and test set size.
assumptions (6)
  • domain assumption GPT-4o concept annotations match the latent concepts readers would assign to reviews.
    Used to define every concept label in Sec. 3.2; validated on only 10 samples per dataset in App. A.3 (IMDB mean 3.81/5, 70% agreement above 3).
  • domain assumption A concept classifier f_omega that is maximally uncertain on f_phi(x) implies f_phi(x) does not contain concept information.
    The removal objective in Eq. (7) and the claim of 'absence of learnable conceptual information' in Sec. 3.3.1 rely on this probe being faithful; a weak or fooled classifier would not certify removal.
  • domain assumption Reconstruction of x from f_phi(x) by the frozen reversal network (Eq. 9) guarantees retention of task-relevant content.
    Sec. 3.3.2; reconstruction loss is a proxy for content preservation, not a guarantee that sentiment-discriminative features survive.
  • domain assumption The sample has balanced classification labels (Eq. 1) and biased concept labels (Eq. 2) after the MI-based split.
    Sec. 3.2; the split is constructed by the authors, and group sizes are not reported, so the assumption is not independently verifiable from the text.
  • domain assumption Samples with the lowest I(c;Y) form a valid real-world OOD test for conceptual robustness.
    Sec. 3.2; the OOD claim is evaluated entirely on this self-constructed set, which shares the GPT-4o concept ontology used to train CURE.
  • standard math Finite-sample mutual information estimates P(c,y) in Eq. (4) are adequate for concept selection.
    Applied in Sec. 3.2 to rank concepts; no confidence intervals or smoothing are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models." pith.science (2026). https://pith.science/paper/UE7RC6IA

@misc{pith2026250905230,
  author       = {Pith},
  title        = {Pith review of: CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UE7RC6IA}},
  note         = {Machine review of arXiv:2509.05230}
}
read the original abstract

Pre-trained language models have achieved remarkable success across diverse applications but remain susceptible to spurious, concept-driven correlations that impair robustness and fairness. In this work, we introduce CURE, a novel and lightweight framework that systematically disentangles and suppresses conceptual shortcuts while preserving essential content information. Our method first extracts concept-irrelevant representations via a dedicated content extractor reinforced by a reversal network, ensuring minimal loss of task-relevant information. A subsequent controllable debiasing module employs contrastive learning to finely adjust the influence of residual conceptual cues, enabling the model to either diminish harmful biases or harness beneficial correlations as appropriate for the target task. Evaluated on the IMDB and Yelp datasets using three pre-trained architectures, CURE achieves an absolute improvement of +10 points in F1 score on IMDB and +2 points on Yelp, while introducing minimal computational overhead. Our approach establishes a flexible, unsupervised blueprint for combating conceptual biases, paving the way for more reliable and fair language understanding systems.

Figures

Figures reproduced from arXiv: 2509.05230 by the authors.

Figure 1
Figure 1. Example of shortcut learning in sentiment [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The training process of our CURE involves three steps: 1) We train a content extractor to filter out [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The convergence of the content extractor. We [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The impact of the margin on classification [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Sentiment distributions in the imbalanced groups of the IMDB and Yelp datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 22 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Zeming Chen, Qiyue Gao, Antoine Bosselut, Ashish Sabharwal, and Kyle Richardson. 2023. https://doi.org/10.18653/v1/2023.acl-long.302 DISCO : Distilling counterfactuals with large language models . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5514--5528, Toronto, Canada. Associati...

    doi:10.18653/v1/2023.acl-long.302 2023

  4. [4]

    Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. https://doi.org/10.18653/v1/P19-1601 Style transformer: Unpaired text style transfer without disentangled latent representation . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997--6007, Florence, Italy. Association for Computational Linguistics

    work page doi:10.18653/v1/p19-1601 2019

  5. [5]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

  6. [6]

    Yanrui Du, Jing Yan, Yan Chen, Jing Liu, Sendong Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, and Bing Qin. 2022. Less learn shortcut: Analyzing and mitigating learning of spurious feature-label correlation. arXiv preprint arXiv:2205.12593

    work page Pith review arXiv 2022

  7. [7]

    Fabrizio Gilardi, Meysam Alizadeh, and Ma \"e l Kubli. 2023. https://api.semanticscholar.org/CorpusID:257766307 Chatgpt outperforms crowd workers for text-annotation tasks . Proceedings of the National Academy of Sciences of the United States of America, 120

    work page 2023

  8. [8]

    He He, Sheng Zha, and Haohan Wang. 2019. https://doi.org/10.18653/v1/D19-6115 Unlearn dataset bias in natural language inference by fitting the residual . In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 132--142, Hong Kong, China. Association for Computational Linguistics

    doi:10.18653/v1/d19-6115 2019

Show all 41 references
  1. [9]

    Kyunghoon Hur, Jiyoung Lee, Jungwoo Oh, Wesley Price, Younghak Kim, and Edward Choi. 2022. https://proceedings.mlr.press/v174/hur22a.html Unifying heterogeneous electronic health records systems via text-based code embedding . In Proceedings of the Conference on Health, Infere...

    2022

  2. [10]

    Amelia Jiménez-Sánchez, Dovile Juodelyte, Bethany Chamberlain, and Veronika Cheplygina. 2023. https://doi.org/10.1109/ISBI53787.2023.10230572 Detecting shortcuts in medical images - a case study in chest x-rays . In 2023 IEEE 20th International Symposium on Biomedical Imaging ...

    2023

  3. [11]

    Darsh Kaushik, Abdullah Faiz Ur Rahman Khilji, Utkarsh Sinha, and Partha Pakray. 2021. https://doi.org/10.18653/v1/2021.sdp-1.13 CNLP - NITS @ L ong S umm 2021: T ext R ank variant for generating long summaries . In Proceedings of the Second Workshop on Scholarly Document Proc...

    2021 doi

  4. [12]

    Divyansh Kaushik and Zachary C. Lipton. 2018. https://doi.org/10.18653/v1/D18-1546 How much reading does reading comprehension require? a critical investigation of popular benchmarks . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p...

    2018 doi

  5. [13]

    Nicholas A Kuiper and Rod A Martin. 1993. https://doi.org/doi:10.1515/humr.1993.6.3.251 Humor and self-concept . Humor, 6(3):251--270

    1993 doi

  6. [14]

    Sachin Kumar, Shuly Wintner, Noah A Smith, and Yulia Tsvetkov. 2019. Topics to avoid: Demoting latent confounds in text classification. arXiv preprint arXiv:1909.00453

    2019 arXiv

  7. [15]

    Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology, 140:1--55

    1932

  8. [16]

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. https://doi.org/10.1109/TPAMI.2018.2858826 Focal loss for dense object detection . IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318--327

    2020

  9. [17]

    Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

    2019

  10. [18]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 4768–4777, Red Hook, NY, USA. Curran Associates Inc

    2017

  11. [19]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. http://www.aclweb.org/anthology/P11-1015 Learning word vectors for sentiment analysis . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguist...

    2011

  12. [20]

    Meta. 2024. https://huggingface.co/meta-llama/Llama-3.2-1B Llama 3.2-1b model card . Accessed: January 23, 2025

    2024

  13. [21]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022

  14. [22]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. https://api.semanticscholar.org/CorpusID:160025533 Language models are unsupervised multitask learners . OpenAI blog, 1(8):9

    2019

  15. [23]

    Hashimoto, and Percy Liang

    Shiori Sagawa*, Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang. 2020. https://openreview.net/forum?id=ryxGuJrFvS Distributionally robust neural networks . In International Conference on Learning Representations

    2020

  16. [24]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://doi.org/10.18653/v1/P16-1009 Improving neural machine translation models with monolingual data . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ...

    2016 doi

  17. [25]

    Noam M. Shazeer. 2020. https://api.semanticscholar.org/CorpusID:211096588 Glu variants improve transformer . ArXiv, abs/2002.05202

    2020 arXiv

  18. [26]

    Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel, and Tim Rockt \"a schel. 2020. Avoiding the hypothesis-only bias in natural language inference via ensemble adversarial training. arXiv preprint arXiv:2004.07790

    2020 arXiv

  19. [27]

    Pereira, and William Bialek

    Naftali Tishby, Fernando C. Pereira, and William Bialek. 1999. https://arxiv.org/abs/physics/0004057 The information bottleneck method . In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing, pages 368--377

    1999 arXiv

  20. [28]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023 a . https://arxiv.org/abs/2302.13971 Lla...

    2023 arXiv

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  22. [30]

    Lifu Tu, Garima Lalwani, Spandana Gella, and He He. 2020. An empirical study on robustness to spurious correlations using pre-trained language models. Transactions of the Association for Computational Linguistics, 8:621--633

    2020

  23. [31]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  24. [32]

    Siyin Wang, Jie Zhou, Changzhi Sun, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. 2022. https://aclanthology.org/2022.coling-1.607 Causal intervention improves implicit sentiment analysis . In Proceedings of the 29th International Conference on Computational Linguistics, p...

    2022

  25. [33]

    Tianlu Wang, Rohit Sridhar, Diyi Yang, and Xuezhi Wang. 2021. Identifying and mitigating spurious correlations for improving robustness in nlp models. arXiv preprint arXiv:2110.07736

    2021 arXiv

  26. [34]

    Zhao Wang and Aron Culotta. 2020. Identifying spurious correlations for robust text classification. arXiv preprint arXiv:2010.02458

    2020 arXiv

  27. [35]

    Jiaxin Wen, Yeshuang Zhu, Jinchao Zhang, Jie Zhou, and Minlie Huang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.170 A uto CAD : Automatically generate counterfactuals for mitigating shortcut learning . In Findings of the Association for Computational Linguistics: EM...

    2022 doi

  28. [36]

    Weizhi Xu, Qiang Liu, Shu Wu, and Liang Wang. 2023. https://doi.org/10.18653/v1/2023.acl-long.374 Counterfactual debiasing for fact verification . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6777--67...

    2023 doi

  29. [37]

    Yadollah Yaghoobzadeh, Soroush Mehri, Remi Tachet, Timothy J Hazen, and Alessandro Sordoni. 2019. Increasing robustness to spurious correlations using forgettable examples. arXiv preprint arXiv:1911.03861

    2019 arXiv

  30. [38]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  31. [39]

    Shuo Yang, Bardh Prenkaj, and Gjergji Kasneci. 2024 b . https://arxiv.org/abs/2412.07675 Razor: Sharpening knowledge by cutting bias with unsupervised text rewriting . Preprint, arXiv:2412.07675

    2024 arXiv

  32. [40]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, NIPS'15, page 649–657, Cambridge, MA, USA. MIT Press

    2015

  33. [41]

    Yuhang Zhou, Paiheng Xu, Xiaoyu Liu, Bang An, Wei Ai, and Furong Huang. 2024. https://doi.org/10.18653/v1/2024.acl-long.28 Explore spurious correlations at the concept level in language models for text classification . In Proceedings of the 62nd Annual Meeting of the Associati...

    2024 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.