Robustness methods estimate the task covariance Sigma_task, and the matching principle requires penalty matrices to have range covering that of Sigma_task to zero deployment drift.
super hub Canonical reference
Invariant Risk Minimization
Canonical reference. 71% of citing Pith papers cite this work as background.
119
Pith papers citing it
Background
71% of classified citations
abstract
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.
hub tools
citation-role summary
background 12
method 1
other 1
citation-polarity summary
claims ledger
- abstract We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.
- background Θ ⊆ Rd are convex and compact, and letθ∗ ∈ Θ be a minimizer of the worst-group objectiveR(θ). Then there exists a distributionQ∗ ∈ Q such thatθ∗ ∈ arg minθ Ez∼Q∗[ℓ(θ;z)]. However, this equivalence breaks down when the lossℓ is non-convex: Counterexample 1. Consider a uniform data distributionP supported on two points Z = {z1,z 2}, and letℓ(θ;z) be as in Figure 4, withΘ = [0, 1]. The DRO solutionθ∗ achieves a worst-case loss of R(θ∗) = 0.6. Now consider any weights (w1,w 2) ∈ ∆2 and w.l.o.g. letw
authors
co-cited works
representative citing papers
Multi-source transfer learning incurs an intrinsic adaptation cost that can exceed one, with phase transitions separating regimes where bias-agnostic estimators match oracle performance from those where they cannot.
CouCE is a unified causal framework using Orthogonal Dictionary-Based Backdoor Adjustment and Multi-Scale Randomized Causal Intervention to debias deep metric learning against two distinct confounders.
RACL-B safely completes fast charging across all nine tested ambient-temperature and cooling-health conditions in a DFN battery model, achieving 37.9% faster charging than the safest fixed current while minimizing plated lithium.
PAPT uses adversarial prompt tuning on diffusion models to generate domain-style images while preserving category features, claiming superior single-domain generalization performance.
CREST re-anchors global readouts in dynamics models to transient events via event-versus-rest contrast on learned features, reducing OOD error on gear, impact, and bearing systems while restoring event credit.
Introduces an identification benchmark with martingale-null markets, counterfactual pairs, and trend swaps to determine if VLMs ground responses in candlestick evidence or rely on trend extrapolation.
Conditional computational barrier exists for learning k=1 invariant subspaces in samplable multi-environment instances under sparse recovery hardness; minimax risk is Theta(k(d-k)/(n|E|)) with phase transition at n* ~ k(d-k)/(|E| gamma^2).
Behavioral INR adapts INRs to behavior by mapping states to actions with FiLM-modulated episode latents for self-supervised policy inference in unlabeled data, with new policy OOD definitions.
PoisonLoRA demonstrates ~100% attack success rates for stealthy LoRA poisoning via concept hijacking and task injection on real platforms, with robustness to base model transfer and multiple remixes.
Invariant Gradient Alignment uses Logical Isomer Sets and a Continuous Gradient Conflict Mask to tighten OOD generalization bounds and boost empirical performance over ERM in reasoning distillation.
EarthShift is a new benchmark using paired datasets to measure robustness of geospatial foundation models to realistic distribution shifts, finding consistent 15-20% performance drops out-of-distribution across 8 models and 11 tasks.
CAML meta-learns a progressively refined inductive bias from active-learning queries to improve robustness to spurious correlations, reporting accuracy gains on minority groups across several benchmarks.
Establishes component-wise identifiability guarantees for partially shared causal latents in multimodal nonlinear mixing and introduces a differentiable Wasserstein-based module for recovery.
FML-Bench shows a simple greedy hill-climber nearly matches tree search on dense-opportunity tasks while an adaptive agent that broadens search on stagnation outperforms six baselines across 18 tasks.
In prediction-intervention games, stable-blanket predictors are at least as good as causal-parent predictors for two classes of follower objectives and can be worst-case optimal under additional conditions.
Introduces replay-based continual learning with sequential invariance alignment to learn domain-invariant representations, outperforming baselines on generalization to unseen domains across six datasets in vision, medicine, manufacturing, and ecology.
TILT adds a target-data penalty on an auxiliary predictor component to induce effective importance weighting for unsupervised domain adaptation under covariate shift.
A minimal model shows that the training-side switch to a shortcut rule does not uniformly produce cross-family OOD failure, as outcomes depend on the held-out family's shortcut correlation.
Spectral Gradient Surgery disentangles class-discriminative and domain-specific signals in distribution-matching distilled datasets by analyzing gradient agreement in the spectral domain, yielding better out-of-distribution performance.
A new orthogonal projection module for video anomaly detection suppresses facial attributes via weak face-presence signals and cosine alignment while preserving anomaly-relevant features like pose and motion.
Excess risk decomposes into independent alignment (trace of inverse average Hessian times gradient covariance) and curvature terms, so both flatness and gradient alignment are required; SAGE achieves this and sets new SOTA on DomainBed.
A large-scale benchmark finds that recent multimodal domain generalization methods give only marginal gains over a plain ERM baseline, with no method winning consistently and all degrading sharply under corruption or missing modalities.
eX2L improves robustness to distribution shifts by penalizing similarity between Grad-CAM maps of a label classifier and a confounder classifier, reaching new SOTA average and worst-group accuracy on the Spawrious benchmark.
citing papers explorer
-
Prediction-Intervention Games and Invariant Sets
In prediction-intervention games, stable-blanket predictors are at least as good as causal-parent predictors for two classes of follower objectives and can be worst-case optimal under additional conditions.
-
Anchor PCA
Anchor PCA recovers a maximal invariant subspace for multi-domain data via PCA on a modified target matrix that trades off explained variance with domain agreement.
-
Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning
Spurious latent factors in fine-tuning can be identified unsupervised from naive LoRA weights and removed via gradient projection of associated patterns to reduce bias and misalignment while preserving task performance.
-
Robust Representation Learning through Explicit Environment Modeling
Explicitly modeling and marginalizing environment variation via generalized random-intercept models produces representations that support robust average prediction across unseen environments and outperform invariant-learning methods in challenging distribution-shift settings.
-
Environment-Robust Representation Learning with Empirical Bayes
An empirical Bayes variational inference method learns environment-robust latent variables from multi-environment data for improved prediction in unseen environments.
-
Causality as the Statistical Conscience of Artificial Intelligence: From Pearl's Ladder to Trustworthy Machines
Causality is required for out-of-distribution generalization in AI, with a necessity theorem and unified causal estimators proposed to fix failure modes like hallucination and reward hacking.