Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness

T0 review · 3 major / 4 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SSMBA generates synthetic text by corrupting a sentence and letting a masked language model reconstruct it, and the paper argues this improves out-of-domain robustness across sentiment classification, natural language inference, and…

desk verdict A useful, simple augmentation recipe with honest experiments, but the abstract's 'consistently outperforms' overstates its own tables—still worthy of serious review. read the letter →

arxiv 2009.10195 v2 pith:TKJYEMGF submitted 2020-09-21 cs.CL cs.LGstat.ML

Nathan Ng , Kyunghyun Cho , Marzyeh Ghassemi This is my paper · ORCID
classification cs.CLcs.LGstat.ML
keywords dataaugmentationout-of-domainrobustnessmaskedlanguagemodelsmanifoldassumptiondenoisingautoencodersnaturalprocessingdomainadaptationself-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SSMBA is a data-augmentation recipe that treats a pre-trained masked language model as a denoising autoencoder and uses it to walk along the data manifold of natural-language text. To augment a training example, it randomly corrupts some tokens, asks BERT or RoBERTa to reconstruct the corrupted sentence, and adds the reconstructed sentence to the training set with a label that is either preserved from the original or predicted by a supervised teacher model trained on the original data. The authors claim that, averaged over seeds and domains, this improves both in-domain and out-of-domain performance across sentiment analysis, natural language inference, and machine translation without task-specific engineering. The payoff would be a single, task-agnostic way to make NLP models more robust to domain shift.

What carries the argument

The central object is the corruption-reconstruction pair (q, r), interpreted as a pseudo-Gibbs sampler over the data manifold—a Markov chain that alternately perturbs a point off the manifold and projects it back. q is the masked-language-model corruption function that masks a fraction of tokens, and r is a pretrained masked language model (BERT or RoBERTa) acting as a denoising autoencoder that fills in the corrupted positions. Under the manifold assumption, the reconstructed sentence lies in the manifold neighborhood of the original, so training on these sentences teaches robustness to local perturbations of the input. The label-generation step is the second load-bearing piece: preserving the original label works on stable tasks, while a teacher model providing hard or soft labels becomes necessary when the manifold neighborhood crosses a decision boundary.

What would settle it

Train a fixed classifier on a low-resource domain far from standard pretraining text, such as clinical notes with heavy abbreviations, and compare OOD accuracy with and without SSMBA; if the augmented model does not beat a word-dropout baseline, the central claim that reconstruction lands on the target manifold is wrong.

Watch

Extended reading notes

Core claim

The paper's central claim is that sampling from the manifold neighborhood of a training example, rather than applying heuristic word-level edits, is the effective way to generate synthetic NLP data for out-of-domain robustness. Formally, SSMBA draws a corrupted version x' of an input x, reconstructs x_hat from x' with a pretrained masked language model, and adds (x_hat, y_hat) to the training set. The authors report that this recipe outperforms EDA, CBERT, UDA, word dropout, RAML, and SwitchOut on average, with headline gains of 0.8 percent OOD accuracy on Amazon reviews, 1.8 percent OOD accuracy on MNLI, and 1.4 BLEU on in-domain IWSLT14 German-to-English translation.

Load-bearing premise

The method assumes that reconstructing a corrupted sentence with a pre-trained masked language model maps text back onto the true data manifold of the downstream task and preserves the label; this fails when the task domain differs sharply from the masked language model's training distribution.

Editorial extensions

If this is right

  • Any supervised NLP task can be augmented with SSMBA without task-specific feature engineering, since only a corruption function and a reconstruction function are required.
  • Out-of-domain accuracy improves on sentiment analysis, natural language inference, and machine translation, including gains of 1.8 percent on MNLI and 2.4 BLEU on German-to-Romansh OOD translation.
  • Increasing the number of generated examples per input improves OOD accuracy and lowers the variance of trained models.
  • Soft labels from a sufficiently accurate teacher model outperform label preservation, while labels from a low-accuracy teacher degrade performance.
  • The method is robust to reconstruction-model capacity, so a smaller distilled model gives nearly the same OOD boost as a much larger one.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The corruption-reconstruction recipe is not tied to language: any domain with a stochastic corruption function and a reconstruction function that returns to the manifold could use the same augmentation loop, although the paper demonstrates it only on text.
  • Because gains grow with the number of augmented examples and with corruption level up to a point, SSMBA behaves like an exploration regularizer over the data manifold; a testable prediction is that OOD gains track the overlap between the target domain and the reconstruction model's training distribution.
  • SSMBA could plausibly combine with back-translation or other augmentation methods, since they explore different neighborhoods of the manifold; the paper does not test this combination.
  • The label-generation threshold implies that SSMBA's value is capped by teacher-model quality, an implicit bound that the paper does not formalize.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SSMBA, a data augmentation method that corrupts a training example and then reconstructs it with a pre-trained masked language model (BERT or RoBERTa) to generate synthetic examples that are claimed to lie on the underlying data manifold. The method is task-agnostic and is evaluated on sentiment analysis, natural language inference, and machine translation, across 9 datasets and several model families (LSTMs, CNNs, RoBERTa, transformers). The authors report improvements in out-of-domain (OOD) robustness over baselines, along with ablations of reconstruction model size, corruption amount, sampling method, augmentation amount, and label generation. They also provide code and statistical significance tests. The central claim is that SSMBA 'consistently outperforms' baselines on both in-domain and OOD data, but this claim is broader than the reported results.

Significance. If the empirical findings hold, SSMBA is a useful and simple augmentation method that can be applied to any supervised NLP task with access to a pretrained masked language model, and it appears to give substantial OOD robustness gains in several settings. The study is extensive: three tasks, nine datasets, multiple model types, multiple random seeds, significance testing, and a focused ablation analysis. The public code is a concrete strength. However, the headline claim is contradicted by the paper's own tables, and the method's task-agnosticism is weakened by per-dataset hyperparameter selection. The core empirical contribution is defensible, but the presentation needs substantial revision.

major comments (3)
  1. [Abstract; Section 1; Tables 2 and 5] The central claim that SSMBA 'consistently outperforms existing data augmentation methods and baseline models on both in-domain and OOD data' is contradicted by the paper's own results. In Table 5, OPUS de→en in-domain BLEU is 54.88 for SSMBA versus 56.99 for the unaugmented baseline, and SSMBA also trails RAML (56.76), Word Dropout (56.26), and SwitchOut (55.50). In Table 2, Movies RNN in-domain accuracy is 89.61 for SSMBA versus 90.74 for the baseline, and it also trails CBERT (91.01) and UDA (90.05). The text itself concedes these failures in Sections 6.1 and 6.3. As stated, the headline claim is universal and these are direct counterexamples, not minor caveats. The abstract and introduction should be revised to a weaker, evidence-backed claim, for example that SSMBA often improves OOD performance and usually improves or matches in-domain performance, with exceptions when the pre-trained MLM manifold is far from the target domain, and these exceptions should be disclosed up front.
  2. [Section 5.2; Appendix D, Table 8] The manuscript claims that SSMBA 'does not rely on task-specific knowledge' and 'requires no dataset-specific fine-tuning' (Section 3), but Table 8 shows that corruption percentage, sampling method, and labeling method are selected separately for each dataset and, in the Movies and ANLI cases, for each domain, based on in-domain validation performance. This is a legitimate tuning procedure, but it contradicts the strong wording of the method's task-agnosticism and means the reported gains are conditional on a per-dataset hyperparameter search. Please qualify the claim and provide a fuller discussion of why these choices are not overfitting to the validation sets.
  3. [Sections 2.3, 3, 5.2, 6.3] The theoretical motivation rests on the pseudo-Gibbs sampling result for denoising autoencoders (Bengio et al., 2013), which applies when the reconstruction function is trained on the data-generating distribution of the target domain. Here the reconstruction function is a pre-trained BERT/RoBERTa that has not been trained on the downstream domain's text. The paper acknowledges in Section 6.3 that OPUS medical text 'differs significantly ... from the learned MLM manifold,' and Table 5 shows that this is exactly where in-domain degradation occurs. The manuscript should either state the manifold assumption as a conditional empirical assumption rather than an established property of the method, or provide evidence that the MLM manifold is close enough to each target domain for the theoretical framing to apply.
minor comments (4)
  1. [Section 5.2] The sentence 'We tune tune the total percentage of tokens corrupted' contains a duplicated word.
  2. [Appendix C] The phrase 'tranfsormer models were trained with 2 GPUs' contains a typo; it should read 'transformer models.'
  3. [Section 4.1] The text 'The Y elp Review Dataset' contains an unintended space; it should read 'The Yelp Review Dataset.'
  4. [Tables 2-5] The main result tables report only averaged scores with significance stars; adding standard deviations or confidence intervals would make the multi-seed results easier to assess, as the analysis section already does in Figures 4-8.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; SSMBA's empirical claims rest on independent measurements, though the abstract overstates the results.

full rationale

No circularity was identified. SSMBA is an empirical augmentation procedure: synthetic examples are generated by corrupting original training inputs and reconstructing them with a pre-trained masked language model, with labels either preserved or produced by a teacher model trained on the original data (Algorithm 1, Section 3). Neither the corruption function nor the reconstruction function is defined in terms of the reported in-domain or OOD test metrics, and no derived constant or target quantity is fitted to those metrics: SSMBA hyperparameters such as corruption percentage, sampling method, and labelling method are explicitly selected using in-domain validation performance (Section 5.2 and Appendix D). The theoretical motivation cites Bengio et al. (2013) for pseudo-Gibbs sampling from denoising autoencoders and Lewis et al. (2019) for using masked language models as DAEs; these are external citations that do not overlap with the present authors and are not used as the source of the reported results. The one author-overlapping citation, fairseq (Ott et al., 2019, which includes author Nathan Ng), is used only as the implementation framework and is not load-bearing for any scientific claim. The paper's own data do contain exceptions to the abstract's claim that SSMBA 'consistently outperforms' baselines on in-domain data, e.g., Movies RNN in-domain accuracy 89.61 vs 90.74 for the unaugmented baseline (Table 2) and OPUS de-en in-domain BLEU 54.88 vs 56.99 (Table 5), with the text acknowledging in Sections 6.1 and 6.3 that SSMBA 'fail[s] to improve ID performance' on Movies and 'degrades ID performance' on OPUS. That is a correctness or over-claiming issue about the breadth of the empirical conclusion, not circularity: the reported numbers are independent measurements, no fitted parameter is renamed as a prediction, and no result is equivalent by construction to its own inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the manifold assumption and on treating BERT as a denoising autoencoder without verifying convergence on downstream domains. Free hyperparameters are tuned per dataset on in-domain validation.

free parameters (5)
  • corruption_percentage = 5% to 60% per dataset (Table 8)
    Tuned per dataset on in-domain validation; directly controls the degree of perturbation before reconstruction.
  • sampling_method = unrestricted sampling for all datasets (Table 8)
    Top-k variants were tested; unrestricted sampling was chosen based on validation performance.
  • labeling_method = preserve label, soft label, or hard label per dataset (Table 8)
    Chosen per dataset on in-domain validation; soft labeling requires a teacher model.
  • number_of_augmented_examples = 5 per input
    Fixed in Sec. 5.2; Fig. 7 shows more augmentation improves OOD accuracy.
  • teacher_model_f = trained on original data
    Used to generate soft or hard labels for synthetic examples; quality depends on teacher accuracy (Fig. 8).
assumptions (4)
  • domain assumption Data lies on a low-dimensional manifold (manifold assumption)
    Invoked in Sec. 3 to justify that corruption-reconstruction moves within the manifold neighborhood.
  • domain assumption Pseudo-Gibbs sampling from a denoising autoencoder converges to the data distribution
    Sec. 2.3 cites Bengio et al. 2013; this is applied to BERT as a DAE without verifying convergence on target domains.
  • domain assumption Pre-trained masked language models approximate the data manifold of every downstream domain
    Sec. 5.2 uses RoBERTa and German BERT as the reconstruction function; Sec. 6.3 acknowledges OPUS medical text differs from the learned MLM manifold.
  • domain assumption Labels (preserved or teacher-generated) remain valid for reconstructed examples
    Required for supervised training on augmented data; Fig. 8 shows validity depends on teacher accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness." pith.science (2026). https://pith.science/paper/TKJYEMGF

@misc{pith2026200910195,
  author       = {Pith},
  title        = {Pith review of: SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TKJYEMGF}},
  note         = {Machine review of arXiv:2009.10195}
}
read the original abstract

Models that perform well on a training domain often fail to generalize to out-of-domain (OOD) examples. Data augmentation is a common method used to prevent overfitting and improve OOD generalization. However, in natural language, it is difficult to generate new examples that stay on the underlying data manifold. We introduce SSMBA, a data augmentation method for generating synthetic training examples by using a pair of corruption and reconstruction functions to move randomly on a data manifold. We investigate the use of SSMBA in the natural language domain, leveraging the manifold assumption to reconstruct corrupted text with masked language models. In experiments on robustness benchmarks across 3 tasks and 9 datasets, SSMBA consistently outperforms existing data augmentation methods and baseline models on both in-domain and OOD data, achieving gains of 0.8% accuracy on OOD Amazon reviews, 1.8% accuracy on OOD MNLI, and 1.4 BLEU on in-domain IWSLT14 German-English.

Figures

Figures reproduced from arXiv: 2009.10195 by the authors.

Figure 1
Figure 1. SSMBA moves along the data manifold M by using a corruption function to perturb an example x off the data manifold, then using a reconstruction func￾tion to project it back on. (Simard et al., 1998), often by perturbing exist￾ing examples in the input space (Perez and Wang, 2017). If data concentrates on a low-dimensional manifold (Chapelle et al., 2006), then these syn￾thetic examples should lie in a manifold neigh… view at source ↗
Figure 2
Figure 2. To sample from an MLM DAE, we apply the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. SSMBA generates synthetic examples by cor [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: OOD accuracy of models trained on succes [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 7
Figure 7. Figure 7: OOD accuracy (%) of models trained with different amounts of SSMBA augmentation. 0 augmen￾tation corresponds to a baseline model. Error bars show standard deviation in OOD accuracy across models. 7.5 Amount of Augmentation How does OOD accuracy change as we generate mo…
Figure 8
Figure 8. Figure 8: Boost in OOD accuracy (%) of models trained with augmented data labelled with different su￾pervision models and label generation methods. outperforming all other models. This threshold varies depending on the difficulty of the dataset and task. In ANLI experiments, lab…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Layering Virtual Try-On

    cs.CV 2026-07 conditional novelty 7.0 of 10

    A two-stage diffusion pipeline and new benchmark let virtual try-on add, remove, or swap clothing layers while preserving inner layers, with SOTA results on the new LVTON benchmark and on VITON-HD/DressCode.

  2. Improving Out-of-Domain Robustness with Targeted Augmentation in Frequency and Pixel Spaces

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Combining amplitude-spectrum interpolation with pixel-space blending during fine-tuning improves out-of-domain accuracy by 3 to 9 points across vision, medical, audio, and astronomical benchmarks.

  3. The Synthetic Imputation Approach: Generating Optimal Synthetic Texts For Underrepresented Categories In Supervised Classification Tasks

    cs.CL 2025-04 conditional novelty 3.0 of 10

    Synthetic texts generated by GPT-4o from five random real examples can match full-sample classifier performance when at least 75 original examples exist for a category.

Reference graph

Works this paper leans on

65 extracted references · 58 canonical work pages · cited by 3 Pith papers

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020. Do not have enough data? deep learning to the rescue! In Proceedings of the 2020 AAAI

    work page 2020

  4. [4]

    Philip Bachman, Ouais Alsharif, and Doina Precup. 2014. Learning with pseudo-ensembles. In NIPS

    work page 2014

  5. [5]

    Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. 2013. Generalized denoising auto-encoders as generative models. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 1, NIPS’13, page 899–907, Red Hook, NY, USA. Curran Associates Inc

    work page 2013

  6. [6]

    John Blitzer, Mark Dredze, and Fernando Pereira. 2007. https://www.aclweb.org/anthology/P07-1056 Biographies, B ollywood, boom-boxes and blenders: Domain adaptation for sentiment classification . In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 440--447, Prague, Czech Republic. Association for Computational ...

    work page 2007

  7. [7]

    Mauro Cettolo, Jan Niehues, Sebastian St\" u ker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th iwslt evaluation campaign, iwslt 2014. In Proceedings of the 11 ^ th International Workshop on Spoken Language Translation

    work page 2014

  8. [8]

    Branden Chan, Timo Möller, Malte Pietsch, Tanay Soni, and Chin Man Yeung. 2020. https://deepset.ai/german-bert Open sourcing german bert

    work page 2020

Show all 65 references
  1. [9]

    Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien. 2006. Semi-Supervised Learning (Adaptive Computation and Machine Learning). The MIT Press

    2006

  2. [10]

    Olivier Chapelle, Jason Weston, L\' e on Bottou, and Vladimir Vapnik. 2000. Vicinal risk minimization. In NIPS

    2000

  3. [11]

    Hal Daum \'e III. 2007. https://www.aclweb.org/anthology/P07-1033 Frustratingly easy domain adaptation . In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256--263, Prague, Czech Republic. Association for Computational Linguistics

    2007

  4. [12]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. http://arxiv.org/abs/1810.04805 BERT: pre-training of deep bidirectional transformers for language understanding . CoRR, abs/1810.04805

    2018 arXiv

  5. [13]

    Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc ' Aurelio Ranzato. 2018. https://doi.org/10.18653/v1/N18-1033 Classical structured prediction losses for sequence to sequence learning . In Proceedings of the 2018 Conference of the North A merican Chapter of the ...

    2018 doi

  6. [14]

    Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. https://doi.org/10.18653/v1/P17-2090 Data augmentation for low-resource neural machine translation . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p...

    2017 doi

  7. [15]

    Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020. Pretrained transformers improve out-of-distribution robustness. In Association for Computational Linguistics

    2020

  8. [16]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735--1780

    1997

  9. [17]

    Kushal Kafle, Mohammed Yousefhussien, and Christopher Kanan. 2017. https://doi.org/10.18653/v1/W17-3529 Data augmentation for visual question answering . In Proceedings of the 10th International Conference on Natural Language Generation, pages 198--202, Santiago de Compostela,...

    2017 doi

  10. [18]

    Can Kanbak, Moosavi-Dezfooli Seyed-Mohsen, and Pascal Frossard. 2018. https://doi.org/10.1109/CVPR.2018.00467 Geometric robustness of deep networks: Analysis and improvement . pages 4441--4449

    2018

  11. [19]

    Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2014

  12. [20]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2014. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization . Cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015

    2014 arXiv

  13. [21]

    Sosuke Kobayashi. 2018. https://doi.org/10.18653/v1/N18-2072 Contextual augmentation: Data augmentation by words with paradigmatic relations . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language ...

    2018 doi

  14. [22]

    Philipp Koehn. 2004. https://www.aclweb.org/anthology/W04-3250 Statistical significance tests for machine translation evaluation . In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 388--395, Barcelona, Spain. Association for Compu...

    2004

  15. [23]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf Imagenet classification with deep convolutional neural networks . In F. Pereira, C. J. C. Burges, L. Bottou, and K...

    2012

  16. [24]

    Varun Kumar , Ashutosh Choudhary , and Eunah Cho . 2020. http://arxiv.org/abs/2003.02245 Data Augmentation using Pre-trained Transformer Models . arXiv e-prints, page arXiv:2003.02245

    2020 arXiv

  17. [25]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461

    2019 arXiv

  18. [26]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  19. [27]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. http://www.aclweb.org/anthology/P11-1015 Learning word vectors for sentiment analysis . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguist...

    2011

  20. [28]

    Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2017. http://arxiv.org/abs/1704.03976 Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

    2017 arXiv

  21. [29]

    Mathias Müller, Annette Rios Gonzales, and Rico Sennrich. 2019. Domain robustness in neural machine translation. ArXiv, abs/1911.03109

    2019 arXiv

  22. [30]

    Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fined-grained aspects. In Proceedings of EMNLP

    2019

  23. [31]

    Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2019. http://arxiv.org/abs/1910.14599 Adversarial NLI: A New Benchmark for Natural Language Understanding

    2019 arXiv

  24. [32]

    Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, and Dale Schuurmans. 2016. http://papers.nips.cc/paper/6547-reward-augmented-maximum-likelihood-for-neural-structured-prediction.pdf Reward augmented maximum likelihood for neural structure...

    2016

  25. [33]

    Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations

    2019

  26. [34]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: A method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311...

    2002

  27. [35]

    u diger G\

    Magdalini Paschali, Walter Simson, Abhijit Guha Roy, Muhammad Ferjad Naeem, R\" u diger G\" o bl, Christian Wachinger, and Nassir Navab. 2019. Data augmentation with manifold exploring geometric transformations for increased performance and robustness. arXiv

    2019

  28. [36]

    Luis Perez and Jason Wang. 2017. http://arxiv.org/abs/1712.04621 The Effectiveness of Data Augmentation in Image Classification using Deep Learning

    2017 arXiv

  29. [37]

    Matt Post. 2018. https://doi.org/10.18653/v1/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Belgium, Brussels. Association for Computational Linguistics

    2018 doi

  30. [38]

    Lawrence

    Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence. 2009. Dataset Shift in Machine Learning

    2009

  31. [39]

    Alexandra Birch Rico Sennrich, Barry Haddow. 2016. Improving neural machine translation models with monolingual data. In Proc. of ACL

    2016

  32. [40]

    Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. 2016. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In NIPS

    2016

  33. [41]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC^2 Workshop

    2019

  34. [42]

    Yves Scherrer and Bruno Cartoni. 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/685_Paper.pdf The trilingual ALLEGRA corpus: Presentation and possible use for lexicon induction . In Proceedings of the Eighth International Conference on Language Resources and Evaluatio...

    2012

  35. [43]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://doi.org/10.18653/v1/W16-2323 E dinburgh neural machine translation systems for WMT 16 . In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 371--376, Berlin, Germany....

    2016 doi

  36. [44]

    Simard, Yann A

    Patrice Y. Simard, Yann A. LeCun, John S. Denker, and Bernard Victorri. 1998. https://doi.org/10.1007/3-540-49430-8_13 Transformation Invariance in Pattern Recognition --- Tangent Distance and Tangent Propagation , pages 239--274. Springer Berlin Heidelberg, Berlin, Heidelberg

    1998 doi

  37. [45]

    Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 2015 International Conference on Learning Representations

    2015

  38. [46]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. https://www.aclweb.org/anthology/D13-1170 Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 Conferenc...

    2013

  39. [47]

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. http://arxiv.org/abs/1312.6199 Intriguing properties of neural networks . In International Conference on Learning Representations

    2014 arXiv

  40. [48]

    J\" o rg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), Istanbul, Turkey. European Language Resources Association (ELRA)

    2012

  41. [49]

    Torralba and A

    A. Torralba and A. A. Efros . 2011. Unbiased look at dataset bias. In CVPR 2011, pages 1521--1528

    2011

  42. [50]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 2017 Conference on Neural Information Processing Systems

    2017

  43. [51]

    Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning

    2008

  44. [52]

    William Yang Wang and Diyi Yang. 2015. https://doi.org/10.18653/v1/D15-1306 That ' s so annoying!!!: A lexical and frame-semantic embedding based data augmentation approach to automatic categorization of annoying behaviors using \# petpeeve tweets . In Proceedings of the 2015 ...

    2015 doi

  45. [53]

    Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. https://doi.org/10.18653/v1/D18-1100 S witch O ut: an efficient data augmentation algorithm for neural machine translation . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p...

    2018 doi

  46. [54]

    Jason Wei and Kai Zou. 2019. https://www.aclweb.org/anthology/D19-1670 EDA : Easy data augmentation techniques for boosting performance on text classification tasks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internat...

    2019

  47. [55]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. http://aclweb.org/anthology/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computationa...

    2018

  48. [56]

    Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. 2019 a . https://arxiv.org/abs/1901.10430 Pay less attention with lightweight and dynamic convolutions . In International Conference on Learning Representations

    2019 arXiv

  49. [57]

    Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019 b . Conditional bert contextual augmentation. In International Conference on Computational Science, pages 84--95. Springer

    2019

  50. [58]

    Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. 2019. Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848

    2019 arXiv

  51. [59]

    Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. 2020. https://openreview.net/forum?id=ByeL1R4FvS Unsupervised data augmentation for consistency training

    2020

  52. [60]

    Wang, Jiwei Li, Daniel Levy, Aiming Nie, Dan Jurafksy, and Andrew Y

    Ziang Xie, Sida I. Wang, Jiwei Li, Daniel Levy, Aiming Nie, Dan Jurafksy, and Andrew Y. Ng. 2017. Data noising as smoothing in neural network language models. In Proceedings of the 2017 International Conference on Learning Representations

    2017

  53. [61]

    Yiben Yang , Chaitanya Malaviya , Jared Fernandez , Swabha Swayamdipta , Ronan Le Bras , Ji-Ping Wang , Chandra Bhagavatula , Yejin Choi , and Doug Downey . 2020. http://arxiv.org/abs/2004.11546 G-DAUG: Generative Data Augmentation for Commonsense Reasoning . arXiv e-prints, p...

    2020 arXiv

  54. [62]

    Yelp open dataset

    Yelp. Yelp open dataset. https://www.yelp.com/dataset

  55. [63]

    Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. 2018. https://openreview.net/forum?id=B14TlG-RW Fast and accurate reading comprehension by combining self-attention and convolution . In International Conference on Learning Representations

    2018

  56. [64]

    Dauphin, and David Lopez-Paz

    Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. https://openreview.net/forum?id=r1Ddp1-Rb mixup: Beyond empirical risk minimization . International Conference on Learning Representations

    2018

  57. [65]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. http://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf Character-level convolutional networks for text classification . In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garn...

    2015

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.