REVIEW 3 cited by
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
read the original abstract
Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.
Forward citations
Cited by 3 Pith papers
-
DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regeneration
DIPBox is the first multi-scale testing framework for detecting adversarial dataset regeneration via four similarity metrics, backed by learning-theoretic analysis of utility-divergence trade-offs.
-
On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference
An attack aligns differently shuffled intermediate activations from secure Transformer inference queries to recover model weights with low error using roughly one dollar of queries.
-
WARBERT: A Hierarchical BERT-based Model for Web API Recommendation
WARBERT is a hierarchical BERT-based model that combines recommendation-style filtering and similarity matching to improve Web API recommendations for mashups on the ProgrammableWeb dataset.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.