Pith. sign in

REVIEW 17 cited by

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2204.02937 v2 pith:3G4PXEQF submitted 2022-04-06 cs.LG cs.CVstat.ML

Polina Kirichenko , Pavel Izmailov , Andrew Gordon Wilson This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords lastlayerspuriousfeaturesretrainingrobustnesssimpleapproaches
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Neural network classifiers can largely rely on simple spurious features, such as backgrounds, to make predictions. However, even in these cases, we show that they still often learn core features associated with the desired attributes of the data, contrary to recent findings. Inspired by this insight, we demonstrate that simple last layer retraining can match or outperform state-of-the-art approaches on spurious correlation benchmarks, but with profoundly lower complexity and computational expenses. Moreover, we show that last layer retraining on large ImageNet-trained models can also significantly reduce reliance on background and texture information, improving robustness to covariate shift, after only minutes of training on a single GPU.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 32 citations worldwide. Full citation record

  1. Subgroups Matter for Robust Bias Mitigation

    cs.LG 2025-05 accept novelty 7.0 of 10

    Subgroup choice determines whether bias mitigation helps or hurts, and the minimum KL divergence to the unbiased test distribution predicts success.

  2. Adapting Vision Foundation Models with Cascaded Semantics

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A visual prompt tuning method that injects fixed color, texture, and shape features plus cascaded self-attention maps into a frozen ViT improves accuracy across 34 datasets with 0.74% trainable parameters.

  3. Beyond Metadata: CAPRA for Hidden Subgroup Analysis under Missing Metadata in Medical Imaging

    eess.IV 2026-07 conditional novelty 6.0 of 10

    CAPRA calibrates image-derived semantic proxy axes on a small labeled split into a reusable subgroup interface for failure auditing and domain-dependent robust transfer without deployment metadata.

  4. Spatially Grounded Concept-Based Image Classification

    cs.CV 2025-10 conditional novelty 6.0 of 10

    SEG-MIL-CBM uses CLIP-guided segmentation with attention-based multiple instance learning to build a concept bottleneck model that produces spatially grounded explanations and improves worst-group accuracy on spurious...

  5. AIM: Amending Inherent Interpretability via Self-Supervised Masking

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    AIM uses multi-stage feature guidance for self-supervised masking to improve both interpretability (EPG) and accuracy on vision benchmarks.

  6. Controllable Feature Whitening for Hyperparameter-Free Bias Mitigation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Controllable Feature Whitening decorrelates target and bias features via a covariance-based whitening transform, reducing spurious-correlation reliance without adversarial training.

  7. Machine Learning from Explanations

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A two-stage optimization pipeline that alternates label loss with a KL divergence between feature maps of masked and unmasked inputs improves accuracy and robustness in small-data classification.

  8. The Pitfalls of Memorization: When Memorization Hurts Generalization

    cs.LG 2024-12 conditional novelty 6.0 of 10

    Memorization-aware training (MAT) shifts logits using calibrated held-out predictions from an XRM auxiliary model, improving worst-group accuracy under subpopulation shift.

  9. Token-Based Detection of Spurious Correlations in Vision Transformers

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A token-discarding method for vision transformers measures whether predictions rely on features outside the object's bounding box, identifying spurious correlations and problematic ImageNet classes.

  10. Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models

    cs.CL 2025-06 conditional novelty 5.0 of 10

    LLMs systematically treat modal words like 'must' as evidence of obligation even in non-obligatory contexts, more strongly than humans, and a few-shot plus reasoning prompt can lower the rate of such judgments.

  11. Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies

    cs.LG 2025-05 conditional novelty 5.0 of 10

    The paper proposes a one-to-one mapping between six causes of distribution shift and several AI safety issues, arguing for mutual method transfer through aligned definitions.

  12. Focusing Image Generation to Mitigate Spurious Correlations

    cs.CV 2024-12 conditional novelty 5.0 of 10

    SCGS generates new training images by inpainting over a classifier's misattended background regions, reducing spurious-correlation reliance without group labels.

  13. Re-evaluating Group Robustness via Adaptive Class-Specific Scaling

    cs.LG 2024-12 conditional novelty 5.0 of 10

    Class-specific score scaling at test time lets a vanilla ERM model match or surpass debiasing methods and yields a scalar robust-average accuracy trade-off metric.

  14. Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation

    cs.LG 2024-12 accept novelty 5.0 of 10

    A unifying taxonomy and formal definition that connects shortcut learning, spurious correlations, Clever Hans behavior, and confounders across detection, mitigation, and datasets.

  15. Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounders

    cs.LG 2024-11 conditional novelty 5.0 of 10

    Under weak overlap, a latent confounder can be approximately identified from a single proxy or multiple unlabeled sources, and a reweighted mixture-of-experts model adapts to confounder shift.

  16. Weight Averaging for Out-of-Distribution Generalization and Few-Shot Domain Adaptation

    cs.CV 2025-01 reject novelty 4.0 of 10

    Gradient-similarity-regularized weight averaging and WA+SAM fine-tuning are tested on OOD and few-shot domain adaptation benchmarks, with mixed results that do not support the claimed improvements.

  17. Learning Causality for Modern Machine Learning

    cs.LG 2025-06 conditional novelty 2.0 of 10

    A thesis compiling six papers that use causal invariance to improve graph neural networks' out-of-distribution generalization, interpretability, and robustness.

Pith tools