Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage procedure that first learns a shallow biased model and then fits the residual can unlearn dataset bias in natural language inference, improving scores on challenge sets while keeping most in-distribution accuracy.

desk verdict A solid, honestly-reported debiasing method with a clean derivation and real gains in matched settings, but the abstract's 'significant gains' and 'reasonable performance' are too broad and need narrowing. read the letter →

arxiv 1908.10763 v2 pith:N3TDHIXG submitted 2019-08-28 cs.CL

He He , Sheng Zha , Haohan Wang This is my paper · ORCID
classification cs.CL
keywords naturallanguageinferencedatasetbiasdebiasingresidualfittingdistributionshiftlabelannotationartifactsHANSchallengeset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish that known dataset bias in natural language inference—spurious cues such as negation words or high word overlap that correlate with labels in crowd-sourced training data—can be unlearned without deleting the cues from the input. It formalizes bias as label shift on a known insufficient feature set $I(x)$, then proposes DRiFt: first train a biased classifier on $I(x)$, freeze it, and train the full model to fit the residual, so examples the biased model already explains contribute little gradient. On SNLI and MNLI, DRiFt improves scores on the HANS and STRESS challenge sets relative to maximum-likelihood training while keeping most in-distribution accuracy, with the trade-off largest for weaker models and biased features that carry real semantics. The reason to care is that this is a general recipe: it converts prior knowledge about a specific annotation artifact into a training objective that focuses the model on examples the artifact cannot solve.

What carries the argument

The load-bearing object is the additive residual model $f^*(x)=f_s(I(x);\theta^*)+f_d(x;\phi^*)$, with the biased classifier $f_s$ fixed after being trained on insufficient features $I(x)$ (hypothesis-only text, bag-of-words embeddings, or handcrafted overlap and negation features). The residual fitting is what does the debiasing: it turns maximum-likelihood training of $f_d$ into a per-example gradient reweighting in which confidently biased examples are down-weighted toward zero gradient, forcing $f_d$ to learn from examples that cannot be solved by the bias. At test time the biased component is discarded, so the method never tries to remove or project out the biased features from the input.

What would settle it

Compute the biased classifier's accuracy on both training and a held-out set drawn from the same distribution. If $I(x)$ remains highly predictive on the held-out set, there is no label shift, and DRiFt should underperform MLE by the amount of useful signal in $I(x)$; the paper's own in-distribution results for DRiFt-CBOW and DRiFt-HAND on DA and ESIM already show drops of more than twenty points, the predicted failure mode. A sharper test: construct a challenge set where the known bias association is preserved rather than reversed—if DRiFt still improves over MLE there, residual fitting is doing something other than removing label shift.

Watch

Extended reading notes

Core claim

The central claim is that a standard NLI model trained by maximum likelihood absorbs the label association carried by shallow input features, and that this association can be removed by fitting the residual of a biased model. Given a feature map $I(x)$ believed to contain dataset bias, DRiFt first solves $\theta^* = \arg\min_\theta \mathbb{E}_P[L(f_s(I(x);\theta),y)]$, then trains a full model $f_d$ on the additive objective $f_s(I(x);\theta^*) + f_d(x;\phi)$, and at test time uses only $f_d$. Under cross-entropy, this objective is equivalent to reweighting each training example's gradient by $p_a(y\mid x)\propto p_s(y\mid I(x))p_d(y\mid x)$: when the biased classifier confidently predicts the true label, the gradient on that example is cancelled, and when the biased classifier is uninformative the gradient reduces to the MLE gradient. The paper argues this corrects the label shift $p(y\mid b(x))\neq q(y\mid b(x))$ on known biased features, and reports that DRiFt improves non-entailment F1 on HANS and entailment F1 on the negation and overlap STRESS sets while degrading in-distribution accuracy only modestly for BERT, though more sharply for the weaker DA and ESIM models.

Load-bearing premise

Everything rests on the assumption that examples the biased classifier predicts well are biased: the shallow features $I(x)$ must mark exactly the label association that changes between training and test, and they must carry no useful semantic signal that would be lost when the biased classifier is discarded at test time.

Editorial extensions

If this is right

  • On HANS, DRiFt raises non-entailment F1 across all three base models and all three biased feature sets, with the handcrafted overlap features giving the largest non-entailment gains; MLE-trained models score near zero on non-entailment.
  • On the MNLI STRESS negation and overlap subsets, DRiFt improves entailment F1 in most configurations, though weaker models such as DA and ESIM lose F1 on one or both of the other classes.
  • Under synthetic label-leaking bias, DRiFt keeps accuracy approximately flat as the cheating rate rises to 0.9, whereas MLE accuracy falls by about 20 points; with an oracle biased classifier the method is equivalent to training only on the unbiased examples.
  • In-distribution accuracy is generally lower under DRiFt than under MLE, and the drop is largest when the biased features also encode useful semantics or the base model is weak; BERT retains accuracy close to MLE.
  • The paper's analysis implies that examples with zero gradient under the biased model contribute nothing to the debiased model, so when nearly all training examples are strongly biased, DRiFt alone cannot recover and additional data or augmentation is needed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-stage recipe should transfer to any task with a known shortcut, such as reading comprehension, visual question answering, or paraphrase detection, as long as one can build a cheap classifier on the suspected shortcut; the paper notes this direction but does not run those experiments.
  • Because DRiFt's definition of bias is whatever $I(x)$ captures, a practitioner using a feature set whose label association is actually stable across train and test would train away useful signal; checking the biased classifier's accuracy on a target distribution before applying DRiFt would be a cheap safeguard.
  • The gradient-reweighting view suggests a post-hoc variant: rather than retraining a full model, one could reweight the gradient contributions of a trained model's training examples, or use the residual objective as a fine-tuning loss; the paper does not explore either.
  • The label-shift framing leaves covariate shift unaddressed; the paper's STRESS results hint at this because distractor phrases shift the input distribution as well, so a natural extension is to combine residual fitting with covariate-shift robust training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DRiFt, a two-stage training procedure for NLI under known dataset bias. A biased classifier is first trained on insufficient features I(x) (hypothesis-only, bag-of-words, or handcrafted overlap/negation features); a debiased classifier is then trained to fit the residual of the biased classifier under a joint softmax objective. At test time only the debiased classifier is used. The authors derive a gradient reweighting interpretation (Eqs. 9-13) and evaluate on SNLI and MNLI with DA, ESIM, and BERT, measuring in-distribution accuracy, synthetic bias robustness, and performance on HANS and STRESS challenge sets. The central empirical claim is that debiased models achieve significant gains on the two challenge sets while maintaining reasonable in-distribution performance.

Significance. If the stated empirical claim held across the reported configurations, DRiFt would be a practical and widely applicable debiasing method. The algorithm is simple, the derivation is self-contained, and the code is released; the best configuration (BERT with HAND) raises HANS non-entailment F1 from 7.8 to 70.9, which is a strong falsifiable demonstration. However, the results are configuration-dependent: macro-F1 on STRESS often decreases, and in-distribution drops of over 20 points for DA/ESIM with CBOW undermine the 'reasonable performance' phrasing. The contribution is therefore real but more modest than the abstract states.

major comments (4)
  1. [Section 4.6, Table 5] The abstract claims 'significant gains on two challenge test sets,' but on STRESS the gains are concentrated in the entailment class while macro-F1 often decreases. For example, DA with DRiFt-CBOW yields macro-F1 (28.4+21.4+39.5)/3 = 29.8 on STRESS-Negation versus MLE (17.4+47.3+55.3)/3 = 40.0, and a similar drop occurs on STRESS-Overlap (40.2 vs 55.0). Even BERT's macro-F1 on STRESS-Overlap drops from 53.9 to 51.6 with DRiFt-CBOW and 51.4 with DRiFt-HAND. The paper should report aggregate metrics and significance tests, and the abstract should be narrowed to note that gains are configuration- and class-dependent.
  2. [Section 4.3, Table 3] The claim in the abstract of 'maintaining reasonable performance on the original test sets' is not supported for DA and ESIM with CBOW or HAND biased classifiers. SNLI accuracy drops 22.7 points for DA with DRiFt-CBOW (85.3 to 62.6) and 25.7 for ESIM (88.0 to 62.3); MNLI drops 15.9 points for DA with DRiFt-CBOW and 9.2 with DRiFt-HAND. The paper acknowledges degradation in Section 4.3, but the abstract's 'reasonable' is too strong; the claim should be qualified to specific configurations or the method should be evaluated with selection criteria that preserve in-domain accuracy.
  3. [Section 4.5, Table 4] The text says 'DRiFt improves performance on non-entailment in all cases with little degradation on entailment,' but the DA rows do not support 'little degradation': DRiFt-HAND lowers entailment F1 from 66.6 to 60.5 on lexical, 66.6 to 61.4 on subsequence, and 66.5 to 55.9 on constituent. Also DRiFt-CBOW lowers entailment F1 on lexical from 66.6 to 65.3 and on constituent from 66.5 to 65.1. This is a load-bearing discrepancy because the conclusion that DRiFt is preferable to RM relies on preserving entailment performance while improving non-entailment; for DA, RM-HYPO preserves entailment better than DRiFt-HAND. The claim should be revised to specify for which models it holds.
  4. [Section 3.1-3.2, Figure 2] The paper assumes without proof that discarding the biased classifier at test time is sound; the derivation in Eqs. 9-13 shows that DRiFt trains the sum f_s + f_d to be a good model, not that f_d alone is calibrated for p(y|x). The synthetic experiment in Figure 2 itself shows the cost: DRiFt-HYPO is below RM-cheat, which the text attributes to HYPO capturing unbiased information. This is an inherent limitation that should be stated as an assumption and tested by comparing DRiFt with an ensemble (f_s + f_d) at test time, or by ablating the biased classifier with features that are known to be purely biased.
minor comments (4)
  1. [Section 3.2, Eq. (10)] The constant C is written as sum_{(x,y) in D} log p_s^*(k|I(x)) with a free index k; it should be p_s^*(y|I(x)) over the true labels.
  2. [Notation throughout] The baseline is typeset as 'R M' in prose (e.g., Sections 3.1 and 4.3) but as 'RM' in tables; please unify the notation.
  3. [Appendix A, Table 6] The Antonym and Length stress subsets have missing entries (shown as '-') in several rows; the caption should state that these classes are absent from those subsets.
  4. [Section 5] There is a typo in the related work paragraph on reading comprehension: 'reading comparehension' should be 'reading comprehension'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DRiFt gradient derivation is self-contained; the HAND/STRESS overlap is an evaluation-scope concern rather than a circular reduction.

full rationale

The derivation in Section 3.2 is self-contained: given the product parametrization p_a ∝ p_s p_d (Eq. 8), the DRiFt objective (Eq. 9) is expanded algebraically into an MLE term plus the regularizer R(x) = −log Σ_k p_s(k|I(x)) p_d(k|x) (Eqs. 10–12), and the gradient analysis (Eq. 13) follows by differentiating that regularizer. No fitted parameter, no target result, and no author-specific uniqueness claim is used as an input to produce the predicted debiasing behavior. The only self-citation is Wang et al. (2019a) in Related Work, which is descriptive and not load-bearing. The HAND biased model is informed by Naik et al.'s error taxonomy and is then evaluated on Naik et al.'s STRESS sets, which is a legitimate evaluation-scope concern (same taxonomy guiding feature choice and test construction), but no parameter is fitted to STRESS and the DRiFt gains are empirical outcomes of training on SNLI/MNLI, so this does not amount to a circular reduction under the stated standard. The abstract's 'reasonable performance' phrasing arguably overstates Table 3 for DRiFt-CBOW/HAND on SNLI/MNLI, but that is a correctness/scope issue, not circularity. Therefore no circular step is established.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

DRiFt introduces no new physical entities, latent variables, or fitted constants. The central claim rests on the decomposition of inputs into biased and generalizable components, on the assumption that a shallow classifier can isolate the bias, and on the choice to drop the biased classifier at test time. These assumptions are domain-level rather than mathematical axioms and are only partially validated by the experiments.

assumptions (4)
  • domain assumption Input text decomposes into biased features b(x) and generalizable features g(x) that are conditionally independent given label y, with p(g(x)|y)=q(g(x)|y) while p(y|b(x)) may differ from q(y|b(x)).
    Section 2, Eqs. 1-2. This decomposition is the formal definition of dataset bias, but real text is not guaranteed to split cleanly into biased and generalizable components.
  • domain assumption A classifier trained only on insufficient features I(x) identifies dataset bias: high predictive confidence on an example means the example is biased.
    Section 3.1: "We assume that examples predicted well by the biased classifier exhibit dataset bias." If I(x) also captures useful semantics, DRiFt reweights away useful signal.
  • ad hoc to paper The biased classifier f_s can be discarded at test time even though training optimized the sum f_s + f_d.
    Section 3.1, Eq. 4-5 and "At test time, we only use the debiased classifier fd." This is safe only if f_s contains no useful generalizable signal, which the paper does not establish.
  • domain assumption The handcrafted features in HAND, such as word overlap, negation words, and length, match the biases in the target test sets.
    Section 4.2 designs HAND from Naik et al. 2018 error analysis, and the evaluation uses HANS and STRESS; effectiveness depends on the bias family being known and named.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual." pith.science (2026). https://pith.science/paper/N3TDHIXG

@misc{pith2026190810763,
  author       = {Pith},
  title        = {Pith review of: Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N3TDHIXG}},
  note         = {Machine review of arXiv:1908.10763}
}
read the original abstract

Statistical natural language inference (NLI) models are susceptible to learning dataset bias: superficial cues that happen to associate with the label on a particular dataset, but are not useful in general, e.g., negation words indicate contradiction. As exposed by several recent challenge datasets, these models perform poorly when such association is absent, e.g., predicting that "I love dogs" contradicts "I don't love cats". Our goal is to design learning algorithms that guard against known dataset bias. We formalize the concept of dataset bias under the framework of distribution shift and present a simple debiasing algorithm based on residual fitting, which we call DRiFt. We first learn a biased model that only uses features that are known to relate to dataset bias. Then, we train a debiased model that fits to the residual of the biased model, focusing on examples that cannot be predicted well by biased features only. We use DRiFt to train three high-performing NLI models on two benchmark datasets, SNLI and MNLI. Our debiased models achieve significant gains over baseline models on two challenge test sets, while maintaining reasonable performance on the original test sets.

Figures

Figures reproduced from arXiv: 1908.10763 by the authors.

Figure 1
Figure 1. An example of dataset bias in NLI. On the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Accuracy on SNLI test set augmented with cheating features, which leak the groundtruth labels on [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 28 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Agrawal, D

    A. Agrawal, D. Batra, and D. Parikh. 2016. Analyzing the behavior of visual question answering models. In Empirical Methods in Natural Language Processing (EMNLP)

    work page 2016

  4. [4]

    J. Andreas. 2019. Good-enough compositional data augmentation. arXiv

    work page 2019

  5. [5]

    Belinkov, A

    Y. Belinkov, A. Poliak, S. M. Shieber, B. V. Durme, and A. M. Rush. 2019. Don't take the premise for granted: Mitigating artifacts in natural language inference. In Association for Computational Linguistics (ACL)

    work page 2019

  6. [6]

    Ben-David, J

    S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. 2006. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems (NeurIPS), pages 137--144

    work page 2006

  7. [7]

    Bolukbasi, K

    T. Bolukbasi, K. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems (NeurIPS), pages 4349--4357

    work page 2016

  8. [8]

    Bowman, G

    S. Bowman, G. Angeli, C. Potts, and C. D. Manning. 2015. A large annotated corpus for learning natural language inference. In Empirical Methods in Natural Language Processing (EMNLP)

    2015

Show all 46 references
  1. [9]

    Q. Chen, X. Zhu, Z. Ling, S. Wei, H. Jiang, and D. Inkpen. 2017. Enhanced LSTM for natural language inference. In Association for Computational Linguistics (ACL)

    2017

  2. [10]

    Clark, M

    C. Clark, M. Yatskar, and L. Zettlemoyer. 2019. Don't take the easy way out: Ensemble based methods for avoiding known dataset biases. In Empirical Methods in Natural Language Processing (EMNLP)

    2019

  3. [11]

    Devlin, M

    J. Devlin, M. Chang, K. Lee, and K. Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Association for Computational Linguistics (NAACL)

    2019

  4. [12]

    Duchi and H

    J. Duchi and H. Namkoong. 2018. Learning models with uniform performance via distributionally robust optimization. arXiv preprint arXiv:1810.08750

    2018 arXiv

  5. [13]

    Glockner, V

    M. Glockner, V. Shwartz, and Y. Goldberg. 2018. Breaking NLI systems with sentences that require simple lexical inferences. In Association for Computational Linguistics (ACL)

    2018

  6. [14]

    Gonen and Y

    H. Gonen and Y. Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862

    2019 arXiv

  7. [15]

    Gururangan, S

    S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. A. Smith. 2018. Annotation artifacts in natural language inference data. In North American Association for Computational Linguistics (NAACL)

    2018

  8. [16]

    W. Hu, G. Niu, I. Sato, and M. Sugiyama. 2018. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML)

    2018

  9. [17]

    Jia and P

    R. Jia and P. Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Empirical Methods in Natural Language Processing (EMNLP)

    2017

  10. [18]

    Karpukhin, O

    V. Karpukhin, O. Levy, J. Eisenstein, and M. Ghazvininejad. 2019. Training on synthetic noise improves robustness to natural noise in machine translation. arXiv

    2019

  11. [19]

    Kaushik and Z

    D. Kaushik and Z. C. Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Empirical Methods in Natural Language Processing (EMNLP)

    2018

  12. [20]

    Kingma and J

    D. Kingma and J. Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  13. [21]

    Z. C. Lipton, Y. Wang, and A. J. Smola. 2018. Detecting and correcting for label shift with black box predictors. In International Conference on Machine Learning (ICML)

    2018

  14. [22]

    N. F. Liu, R. Schwartz, and N. A. Smith. 2019. Inoculation by fine-tuning: A method for analyzing challenge datasets. In North American Association for Computational Linguistics (NAACL)

    2019

  15. [23]

    R. T. McCoy, E. Pavlick, and T. Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv:1902.01007

    2019 arXiv

  16. [24]

    L. Mou, R. Men, G. Li, Y. Xu, L. Zhang, R. Yan, and Z. Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In Association for Computational Linguistics (ACL)

    2016

  17. [25]

    A. Naik, A. Ravichander, N. Sadeh, C. Rose, and G. Neubig. 2018. Stress test evaluation for natural language inference. In International Conference on Computational Linguistics (COLING)

    2018

  18. [26]

    a ckstr\

    A. Parikh, O. T\" a ckstr\" o m, D. Das, and J. Uszkoreit. 2016. A decomposable attention model for natural language inference. In Empirical Methods in Natural Language Processing (EMNLP)

    2016

  19. [27]

    Pennington, R

    J. Pennington, R. Socher, and C. D. Manning. 2014. Glo V e: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543

    2014

  20. [28]

    Poliak, J

    A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, and B. V. Durme. 2018. Hypothesis only baselines in natural language inference. arXiv preprint arXiv:1805.01042

    2018 arXiv

  21. [29]

    M. T. Ribeiro, S. Singh, and C. Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In Association for Computational Linguistics (ACL)

    2018

  22. [30]

    Sakaguchi, R

    K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi. 2019. WINOGRANDE : An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641

    2019 arXiv

  23. [31]

    Scholkopf, D

    B. Scholkopf, D. Janzing, J. Peters, E. Sgouritsa, K. Zhang, and J. Mooij. 2012. On causal and anticausal learning. In International Conference on Machine Learning (ICML)

    2012

  24. [32]

    Schwartz, M

    R. Schwartz, M. Sap, Y. Konstas, L. Zilles, Y. Choi, and N. A. Smith. 2017. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In Computational Natural Language Learning (CoNLL)

    2017

  25. [33]

    Shimodaira

    H. Shimodaira. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90:227--244

    2000

  26. [34]

    Torralba and A

    A. Torralba and A. Efros. 2011. Unbiased look at dataset bias. In Computer Vision and Pattern Recognition (CVPR)

    2011

  27. [35]

    Tsipras, S

    D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. 2019. Robustness may be at odds with accuracy. In International Conference on Learning Representations (ICLR)

    2019

  28. [36]

    H. Wang, Z. He, Z. C. Lipton, and E. P. Xing. 2019 a . Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations (ICLR)

    2019

  29. [37]

    T. Wang, J. Zhao, M. Yatskar, K. Chang, and V. Ordonez. 2019 b . Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In International Conference on Computer Vision (ICCV)

    2019

  30. [38]

    Williams, N

    A. Williams, N. Nangia, and S. R. Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426

    2017 arXiv

  31. [39]

    Zellers, A

    R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi. 2019. HellaSwag : Can a machine really finish your sentence? In Association for Computational Linguistics (ACL)

    2019

  32. [40]

    Zhang, B

    G. Zhang, B. Bai, J. Liang, K. Bai, S. Chang, M. Yu, C. Zhu, and T. Zhao. 2019 a . Selection bias explorations and debias methods for natural language sentence matching datasets. In Association for Computational Linguistics (ACL)

    2019

  33. [41]

    Zhang, Y

    H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan. 2019 b . Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv:1901.08573

    2019 arXiv

  34. [42]

    Zhang, B

    K. Zhang, B. Schölkopf, K. Muandet, and Z. Wang. 2013. Domain adaptation under target and conditional shift. In International Conference on Machine Learning (ICML)

    2013

  35. [43]

    Zhang, J

    Y. Zhang, J. Baldridge, and L. He. 2019 c . PAWS : Paraphrase adversaries from word scrambling. In North American Association for Computational Linguistics (NAACL)

    2019

  36. [44]

    J. Zhao, T. Wang, M. Yatskar, R. Cotterell, V. Ordonez, and K. Chang. 2019. Gender bias in contextualized word embeddings. In North American Association for Computational Linguistics (NAACL)

    2019

  37. [45]

    J. Zhao, T. Wang, M. Yatskar, V. Ordonez, and K. Chang. 2018 a . Gender bias in coreference resolution:evaluation and debiasing methods. In North American Association for Computational Linguistics (NAACL)

    2018

  38. [46]

    J. Zhao, Y. Zhou, Z. Li, W. Wang, and K. Chang. 2018 b . Learning gender-neutral word embeddings. In Empirical Methods in Natural Language Processing (EMNLP)

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.