Pith. sign in

REVIEW 2 major objections 2 minor 60 cited by

Invariant Risk Minimization finds a data representation where the same classifier is optimal for every training distribution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-12 06:25 UTC pith:DUUXOOSS

load-bearing objection IRM gives a clean objective for representations with shared optimal classifiers across environments, but the causal guarantees only hold under linear assumptions. the 2 major comments →

arxiv 1907.02893 v3 pith:DUUXOOSS submitted 2019-07-05 stat.ML cs.AIcs.LG

Invariant Risk Minimization

Martin Arjovsky , L\'eon Bottou , Ishaan Gulrajani , David Lopez-Paz This is my paper
classification stat.ML cs.AIcs.LG
keywords invariant risk minimizationout-of-distribution generalizationcausal inferencedomain generalizationrepresentation learningmachine learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a training objective that searches for features whose relationship to the target stays fixed even as the surrounding data distribution changes. It shows that this objective recovers features tied to the stable causal mechanisms in the data rather than environment-specific correlations. A reader should care because standard empirical risk minimization often latches onto spurious patterns that break when the test distribution differs from training. By enforcing invariance of the optimal predictor, the method aims to produce models that continue to work under distribution shift.

Core claim

Invariant Risk Minimization (IRM) learns a representation such that the optimal linear classifier on top of that representation is identical across all training environments. This is achieved by jointly minimizing the average risk while adding a penalty that forces the gradient of each environment's risk with respect to the classifier parameters to vanish at the shared optimum. The resulting invariant features correspond to the causal factors that govern the label in the underlying data-generating process, enabling generalization to environments not seen during training.

What carries the argument

The IRM penalty term that requires the gradient of the risk with respect to a fixed classifier to be zero in every environment, thereby enforcing that the same predictor is optimal everywhere.

Load-bearing premise

The observed environments must share the same causal mechanisms that determine the label while differing only in the distributions of non-causal variables.

What would settle it

A controlled experiment on synthetic data with known causal graph where IRM is shown to recover exactly the causal features (or fails to do so) when the environments are generated by intervening only on non-causal variables.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces Invariant Risk Minimization (IRM), a learning paradigm that estimates a data representation such that the optimal classifier on top of this representation is the same across multiple training distributions. It claims through theory and experiments that the learned invariances correspond to causal structures governing the data and enable out-of-distribution generalization.

Significance. If the central claims hold, this work offers a principled objective for learning predictors that exploit invariance across environments to achieve robust OOD performance, with a direct link to identifying causal features. This is significant for bridging empirical risk minimization with causal inference in non-i.i.d. settings, and the reproducible experimental protocols and parameter-free aspects of the formulation (where applicable) strengthen its potential impact.

major comments (2)
  1. [§4] §4: The theoretical equivalence showing that IRM recovers causal parents is derived only for linear structural causal models with additive noise and a fixed number of environments; the proof relies on linearity of the representation and identifiability of the shared optimal w. No uniqueness result is given for non-linear feature maps or general non-linear SCMs, so the broader claim that invariances learned by IRM relate to causal structures does not follow in full generality.
  2. [Eq. (3)] Eq. (3): The practical IRM objective (with the gradient penalty at w=1) enforces only a first-order stationarity condition under the linear classifier assumption. The manuscript does not show that this approximation identifies causal features or guarantees OOD generalization when the representation or SCM is non-linear, which is load-bearing for the central claim.
minor comments (2)
  1. [Abstract] The abstract and introduction could more explicitly qualify the scope of the theoretical results to linear cases to avoid overstatement of the causal connection.
  2. Experimental sections would benefit from additional details on environment construction and sensitivity to the penalty hyperparameter to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and insightful comments on our manuscript. We address each major comment below and will incorporate clarifications to better delineate the scope of our theoretical and practical results.

read point-by-point responses

  1. Referee: [§4] The theoretical equivalence showing that IRM recovers causal parents is derived only for linear structural causal models with additive noise and a fixed number of environments; the proof relies on linearity of the representation and identifiability of the shared optimal w. No uniqueness result is given for non-linear feature maps or general non-linear SCMs, so the broader claim that invariances learned by IRM relate to causal structures does not follow in full generality.

    Authors: We agree that the equivalence result in Section 4 is derived under the specific assumptions of linear structural causal models with additive noise and a fixed number of environments, relying on the linearity of the representation and the identifiability of the shared optimal classifier weights. The manuscript does not provide a uniqueness result for non-linear feature maps or general non-linear SCMs. The broader statements linking invariances to causal structures are presented as holding under these assumptions, with supporting experimental evidence in more general settings. We will revise Section 4, the abstract, and related discussion to explicitly state the assumptions and note that extensions to non-linear cases remain an open direction. revision: partial

  2. Referee: [Eq. (3)] The practical IRM objective (with the gradient penalty at w=1) enforces only a first-order stationarity condition under the linear classifier assumption. The manuscript does not show that this approximation identifies causal features or guarantees OOD generalization when the representation or SCM is non-linear, which is load-bearing for the central claim.

    Authors: The practical objective in Equation (3) uses a gradient penalty (evaluated at w=1) to enforce the invariance condition, which is exact under the linear classifier assumption but reduces to a first-order stationarity condition more generally. We do not provide a proof that this approximation identifies causal features or guarantees OOD generalization for non-linear representations or SCMs. The formulation is motivated by the linear theory, and our experiments demonstrate improved OOD performance in non-linear regimes. We will add a clarifying discussion of the approximation's nature and limitations in the revised manuscript. revision: partial

Circularity Check

0 steps flagged

No significant circularity in IRM derivation chain

full rationale

The core IRM definition (a representation Φ such that argmin_w R^e(w ∘ Φ) is identical across environments e) is stated directly from the multi-environment setup and does not reduce to any fitted target quantity or self-referential loop. Section 4 derives the link to causal parents only under explicit linear SCM + additive noise assumptions; this is a one-directional implication proved from the SCM, not a tautology or renaming of the input risks. The practical objective (Eq. 3 with gradient penalty) is an explicit relaxation of the definition, not a statistical fit called a prediction. No load-bearing self-citation or ansatz smuggling is present; the derivation remains self-contained against the stated assumptions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests primarily on the domain assumption that multiple training distributions share invariant causal structures while varying in spurious correlations.

axioms (1)
  • domain assumption Multiple training distributions share the same causal mechanisms but differ in non-causal aspects.
    This premise is required for the shared optimal classifier to isolate causal invariants.

pith-pipeline@v0.9.0 · 5361 in / 1150 out tokens · 43767 ms · 2026-05-12T06:25:21.245199+00:00 · methodology

0 comments
read the original abstract

We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top of that data representation, matches for all training distributions. Through theory and experiments, we show how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning

    cs.LG 2026-05 unverdicted novelty 8.0

    Robustness methods estimate the task covariance Sigma_task, and the matching principle requires penalty matrices to have range covering that of Sigma_task to zero deployment drift.

  2. The Statistical Cost of Adaptation in Multi-Source Transfer Learning

    math.ST 2026-05 unverdicted novelty 8.0

    Multi-source transfer learning incurs an intrinsic adaptation cost that can exceed one, with phase transitions separating regimes where bias-agnostic estimators match oracle performance from those where they cannot.

  3. CouCE: A Unified Causal Framework for Debiased Deep Metric Learning

    cs.CV 2026-06 unverdicted novelty 7.0

    CouCE is a unified causal framework using Orthogonal Dictionary-Based Backdoor Adjustment and Multi-Scale Randomized Causal Intervention to debias deep metric learning against two distinct confounders.

  4. Repair-before-veto control for safe lithium-ion fast charging under unknown ambient and cooling-fault conditions

    eess.SY 2026-06 unverdicted novelty 7.0

    RACL-B safely completes fast charging across all nine tested ambient-temperature and cooling-health conditions in a DFN battery model, achieving 37.9% faster charging than the safest fixed current while minimizing pla...

  5. Discovering Latent Groups for Robust Classification

    cs.LG 2026-06 unverdicted novelty 7.0

    NCT uses iterative correctness-based routing in a tree architecture to disentangle latent subgroups for robust classification without subgroup labels, yielding competitive performance on spurious correlation benchmarks.

  6. Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization

    cs.CV 2026-06 unverdicted novelty 7.0

    PAPT uses adversarial prompt tuning on diffusion models to generate domain-style images while preserving category features, claiming superior single-domain generalization performance.

  7. When Dynamics Models Read the Wrong Time Steps: Label-Free Event Credit Re-Anchoring for Robust Global Readouts

    cs.LG 2026-06 unverdicted novelty 7.0

    CREST re-anchors global readouts in dynamics models to transient events via event-versus-rest contrast on learned features, reducing OOD error on gear, impact, and bearing systems while restoring event credit.

  8. Martingale Doppelg\"anger-Eval: An Identification Framework for Auditing Candlestick Understanding in Vision-Language Models

    q-fin.CP 2026-06 unverdicted novelty 7.0

    Introduces an identification benchmark with martingale-null markets, counterfactual pairs, and trend swaps to determine if VLMs ground responses in candlestick evidence or rely on trend extrapolation.

  9. Is Spurious Correlation Removal Always Learnable?

    cs.LG 2026-06 unverdicted novelty 7.0

    Conditional computational barrier exists for learning k=1 invariant subspaces in samplable multi-environment instances under sparse recovery hardness; minimax risk is Theta(k(d-k)/(n|E|)) with phase transition at n* ~...

  10. Implicit Neural Representations of Individual Behavior

    cs.LG 2026-06 unverdicted novelty 7.0

    Behavioral INR adapts INRs to behavior by mapping states to actions with FiLM-modulated episode latents for self-supervised policy inference in unlabeled data, with new policy OOD definitions.

  11. Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem

    cs.CR 2026-06 unverdicted novelty 7.0

    PoisonLoRA demonstrates ~100% attack success rates for stealthy LoRA poisoning via concept hijacking and task injection on real platforms, with robustness to base model transfer and multiple remixes.

  12. Invariant Gradient Alignment for Robust Reasoning Distillation

    cs.LG 2026-06 unverdicted novelty 7.0

    Invariant Gradient Alignment uses Logical Isomer Sets and a Continuous Gradient Conflict Mask to tighten OOD generalization bounds and boost empirical performance over ERM in reasoning distillation.

  13. EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation

    cs.CV 2026-05 unverdicted novelty 7.0

    EarthShift is a new benchmark using paired datasets to measure robustness of geospatial foundation models to realistic distribution shifts, finding consistent 15-20% performance drops out-of-distribution across 8 mode...

  14. I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models

    cs.LG 2026-05 unverdicted novelty 7.0

    I-SAFE uses Wasserstein Coherence Metrics to audit distributional coherence of scientific AI models under structurally guided perturbations, revealing differences among DTI predictors that accuracy metrics miss.

  15. Cumulative Meta-Learning from Active Learning Queries for Robustness to Spurious Correlations

    cs.LG 2026-05 unverdicted novelty 7.0

    CAML meta-learns a progressively refined inductive bias from active-learning queries to improve robustness to spurious correlations, reporting accuracy gains on minority groups across several benchmarks.

  16. Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing

    cs.LG 2026-05 unverdicted novelty 7.0

    Establishes component-wise identifiability guarantees for partially shared causal latents in multimodal nonlinear mixing and introduces a differentiable Wasserstein-based module for recovery.

  17. FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics

    cs.LG 2026-05 unverdicted novelty 7.0

    FML-Bench shows a simple greedy hill-climber nearly matches tree search on dense-opportunity tasks while an adaptive agent that broadens search on stagnation outperforms six baselines across 18 tasks.

  18. Prediction-Intervention Games and Invariant Sets

    stat.ML 2026-05 unverdicted novelty 7.0

    In prediction-intervention games, stable-blanket predictors are at least as good as causal-parent predictors for two classes of follower objectives and can be worst-case optimal under additional conditions.

  19. Continual Learning of Domain-Invariant Representations

    cs.LG 2026-05 unverdicted novelty 7.0

    Introduces replay-based continual learning with sequential invariance alignment to learn domain-invariant representations, outperforming baselines on generalization to unseen domains across six datasets in vision, med...

  20. TILT: Target-induced loss tilting under covariate shift

    cs.LG 2026-05 conditional novelty 7.0

    TILT adds a target-data penalty on an auxiliary predictor component to induce effective importance weighting for unsupervised domain adaptation under covariate shift.

  21. Separating Shortcut Transition from Cross-Family OOD Failure in a Minimal Model

    cs.LG 2026-05 conditional novelty 7.0

    A minimal model analytically separates shortcut attraction during training from the switch to a shortcut rule and from cross-family out-of-distribution failure.

  22. Separating Shortcut Transition from Cross-Family OOD Failure in a Minimal Model

    cs.LG 2026-05 unverdicted novelty 7.0

    A minimal model shows that the training-side switch to a shortcut rule does not uniformly produce cross-family OOD failure, as outcomes depend on the held-out family's shortcut correlation.

  23. Spectral Gradient Surgery for Domain-Generalizable Dataset Distillation

    cs.LG 2026-05 unverdicted novelty 7.0

    Spectral Gradient Surgery disentangles class-discriminative and domain-specific signals in distribution-matching distilled datasets by analyzing gradient agreement in the spectral domain, yielding better out-of-distri...

  24. Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection

    cs.CV 2026-05 unverdicted novelty 7.0

    A new orthogonal projection module for video anomaly detection suppresses facial attributes via weak face-presence signals and cosine alignment while preserving anomaly-relevant features like pose and motion.

  25. Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    Excess risk decomposes into independent alignment (trace of inverse average Hessian times gradient covariance) and curvature terms, so both flatness and gradient alignment are required; SAGE achieves this and sets new...

  26. Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study

    cs.CV 2026-05 unverdicted novelty 7.0

    A large-scale benchmark finds that recent multimodal domain generalization methods give only marginal gains over a plain ERM baseline, with no method winning consistently and all degrading sharply under corruption or ...

  27. eXplaining to Learn (eX2L): Regularization Using Contrastive Visual Explanation Pairs for Distribution Shifts

    cs.CV 2026-05 unverdicted novelty 7.0

    eX2L improves robustness to distribution shifts by penalizing similarity between Grad-CAM maps of a label classifier and a confounder classifier, reaching new SOTA average and worst-group accuracy on the Spawrious benchmark.

  28. Domain Generalization through Spatial Relation Induction over Visual Primitives

    cs.CV 2026-05 unverdicted novelty 7.0

    PARSE improves domain generalization accuracy by factoring recognition into visual primitives and their spatial relational compositions learned end-to-end with differentiable predicates.

  29. ScriptHOI: Learning Scripted State Transitions for Open-Vocabulary Human-Object Interaction Detection

    cs.CV 2026-05 unverdicted novelty 7.0

    ScriptHOI decomposes HOI phrases into state slots and uses script coverage, conflict, interval partial-label learning, and counterfactual contrast to improve rare and unseen interaction detection while cutting afforda...

  30. Learning to Theorize the World from Observation

    cs.LG 2026-05 unverdicted novelty 7.0

    NEO is a probabilistic neural model that induces compositional programs as a learned Language of Thought from non-textual observations and executes them via a shared transition model to enable explanation-driven gener...

  31. ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interaction

    cs.LG 2026-05 unverdicted novelty 7.0

    ISAAC auditing applied to three DTI models on the Davis benchmark finds 25% relative differences in causal reasoning scores despite nearly identical AUROC values.

  32. Robust and Clinically Reliable EEG Biomarkers: A Cross Population Framework for Generalizable Parkinson's Disease Detection

    cs.LG 2026-04 conditional novelty 7.0

    A cross-population framework for EEG Parkinson's detection using exhaustive 75 directional evaluations and nested validation shows asymmetric transfer and accuracy up to 94.1% when training diversity increases, suppor...

  33. Synthetic Designed Experiments for Diagnosing Vision Model Failure

    cs.CV 2026-03 unverdicted novelty 7.0

    SDRS uses designed experiments and ANOVA decomposition on synthetic data to identify Type I coverage gaps and Type II spurious dependencies in vision models, then generates targeted data to improve performance.

  34. The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter

    cs.LG 2024-11 unverdicted novelty 7.0

    ML researchers assess spurious correlations via four pragmatic frames (relevance, generalizability, human-likeness, harmfulness) rather than a fixed statistical definition.

  35. TRACE-Seg3D: Counterfactual Context Auditing For Robust 3D Glioma Segmentation Under Institutional Shift

    cs.CV 2026-07 conditional novelty 6.0

    TRACE-Seg3D factorizes disease and imaging context representations in 3D MRI segmentation, auditing prediction stability under counterfactual context shifts to expose spurious failures missed by standard metrics.

  36. Automated Background Swapping for Robustness against Spurious Backgrounds

    cs.CV 2026-06 unverdicted novelty 6.0

    AutoBackSwap uses foreground-background disentanglement via a secondary network plus background infilling to augment training data and reduce spurious background correlations in image classifiers, outperforming priors...

  37. Counterfactual Residual Data Augmentation for Regression

    cs.LG 2026-06 unverdicted novelty 6.0

    CRDA augments regression datasets by generating counterfactual samples from invariant residuals, cutting MLP MSE by 22.9% and XGBoost MSE by 6.4% on average across benchmarks.

  38. On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity

    cs.LG 2026-06 unverdicted novelty 6.0

    On-policy self-distillation with sampled demonstrations reduces rollout diversity by amplifying existing probability gaps in the base model, unlike ideal RL which preserves ratios among correct outputs.

  39. Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts

    cs.LG 2026-06 unverdicted novelty 6.0

    Introduces subset-shared invariance using mixture-of-experts to improve domain generalization by modeling partially shared structure across domain subsets.

  40. Assessing Distribution Shift in Human Activity Recognition for Domain Generalization

    cs.AI 2026-06 unverdicted novelty 6.0

    Evaluates four distribution shifts in sensor-based HAR, finds diversity shifts dominate, and shows 28 DG methods only marginally beat ERM while releasing open benchmarks.

  41. Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

    cs.CV 2026-06 unverdicted novelty 6.0

    Proposes CBCM for diffusion-based spurious attribute mining and DCD for cross-projection debiasing, claiming SOTA worst-group accuracy on four benchmarks while tuning at most 0.22% of parameters.

  42. Measuring Behavior Portability in Large Language Models

    cs.AI 2026-06 unverdicted novelty 6.0

    A new framework measures behavioral portability of LLMs across payoff-equivalent environments and reports substantial systematic transfer losses in seven economic decision problems.

  43. Unsupervised Causal Abstractions Discovery

    cs.LG 2026-06 unverdicted novelty 6.0

    Low-rank graphs induce latents that form causal abstractions, with identifiability results and a practical objective enabling unsupervised learning of high-level SCMs from low-level measurements.

  44. Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQA

    cs.CV 2026-06 unverdicted novelty 6.0

    CREDiT applies counterfactual reasoning via structural causal models to decompose video representations into causal and non-causal parts for more reliable VideoQA on datasets like NExT-GQA and SportsQA.

  45. Bridging Expert Knowledge and Automated Feature Engineering via Self-Evolution

    cs.AI 2026-06 unverdicted novelty 6.0

    FEST uses self-evolving trees to produce expert-aligned, auditable features from unstructured data and outperforms baselines on brand, authenticity, and stress tasks while releasing the BrandGuide dataset.

  46. Anchor PCA

    stat.ML 2026-06 unverdicted novelty 6.0

    Anchor PCA recovers a maximal invariant subspace for multi-domain data via PCA on a modified target matrix that trades off explained variance with domain agreement.

  47. Mitigating Spurious Correlations with Memorization-Guided Dataset De-Biasing

    cs.LG 2026-06 unverdicted novelty 6.0

    Proposes memorization-guided two-stage scoring to select debiased training subsets, enabling ERM models to achieve better performance than SOTA debiasing techniques using only 10% of data.

  48. Shortcut to Nowhere: Demystifying Deep Spurious Regression

    cs.LG 2026-06 unverdicted novelty 6.0

    Defines DSR and introduces similarity-based calibration of label and feature distributions to mitigate continuous spurious correlations in regression.

  49. Density-Aware Translation of Spurious Correlations in Zero-Shot VLMs

    cs.CV 2026-06 unverdicted novelty 6.0

    DAT rescales CLIP image-text similarities based on local embedding density to reduce the impact of spurious correlations in zero-shot classification.

  50. Trustworthy Recommendation in the Era of Large Language Models: Opportunities and Challenges

    cs.IR 2026-05 unverdicted novelty 6.0

    A systematic review of over 200 studies concludes that LLMs in recommender systems act as a double-edged sword, creating both opportunities and new risks for trustworthiness.

  51. idSCD: Identifying Training Datasets through Semantic Correlation Descriptors

    cs.LG 2026-05 unverdicted novelty 6.0

    idSCD uses semantic correlation descriptors to perform dataset membership inference by comparing learned semantic structures, outperforming baselines in NLI, emotion, and medical text experiments.

  52. Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning

    stat.ML 2026-05 unverdicted novelty 6.0

    Spurious latent factors in fine-tuning can be identified unsupervised from naive LoRA weights and removed via gradient projection of associated patterns to reduce bias and misalignment while preserving task performance.

  53. Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty

    eess.SY 2026-05 unverdicted novelty 6.0

    The paper defines the Epistemic Trap from interacting state and dynamics uncertainties in RL and introduces an Adaptive Safety Architecture with a Compound Uncertainty Coefficient to enable active information-seeking ...

  54. The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning

    cs.LG 2026-05 unverdicted novelty 6.0

    The matching principle unifies nuisance-robust representation learning by requiring Jacobian regularization whose range covers the covariance of label-preserving deployment nuisances, with closed-form optimality proof...

  55. I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models

    cs.LG 2026-05 unverdicted novelty 6.0

    I-SAFE is a post-hoc auditing framework that applies quantile-based and Wasserstein coherence metrics to evaluate distributional response of DTI prediction models under structural perturbations from external priors li...

  56. Towards Context-Invariant Safety Alignment for Large Language Models

    cs.CL 2026-05 unverdicted novelty 6.0

    Introduces AIR, an asymmetric regularization that anchors open-ended safety prompts to verifiable ones via stop-gradient, improving invariance and accuracy when combined with group preference optimization.

  57. S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs

    cs.LG 2026-05 unverdicted novelty 6.0

    S2Aligner decouples semantic and structural components in LLM-as-Aligner pre-training for sparse TAGs and uses structure-oriented reconstruction plus domain risk balancing to improve transferability and reduce general...

  58. FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics

    cs.LG 2026-05 accept novelty 6.0

    FML-Bench shows that a simple greedy hill-climber performs nearly as well as complex tree-search agents on ML research tasks, with an adaptive strategy that switches exploration modes outperforming all tested agents.

  59. When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors

    cs.LG 2026-05 unverdicted novelty 6.0

    CliffSplit exposes at least 15% higher errors in cliff-heavy regions of QM9 while CliffLoss narrows the cliff-to-smooth error gap by up to 30% and improves overall MAE by 9.7% across several molecular tasks and backbones.

  60. Avoiding Structural Failure Modes in Tabular Fair SSL: Online Primal-Dual Allocation under Confidence Gating

    cs.LG 2026-05 unverdicted novelty 6.0

    OPDA is an online controller that uses violation, risk, and pseudo-label health signals to avoid Masking Collapse and Trivial Saturation in tabular fair SSL under confidence gating.

Reference graph

Works this paper leans on

58 extracted references · 58 canonical work pages · cited by 127 Pith papers

  1. [1]

    Autonomy

    John Aldrich. Autonomy. Oxford Economic Papers, 1989

    work page 1989

  2. [2]

    Robust supervised learning

    James Andrew Bagnell. Robust supervised learning. In AAAI, 2005

    work page 2005

  3. [3]

    Bartlett, Philip M

    Peter L. Bartlett, Philip M. Long, G´ abor Lugosi, and Alexander Tsigler. Benign Overfitting in Linear Regression. arXiv, 2019

    work page 2019

  4. [4]

    Recognition in terra incognita

    Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In ECCV, 2018

    work page 2018

  5. [5]

    Analysis of representations for domain adaptation

    Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In NIPS. 2007

    work page 2007

  6. [6]

    Robust optimiza- tion

    Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimiza- tion. Princeton University Press, 2009

    work page 2009

  7. [7]

    A meta- transfer objective for learning to disentangle causal mechanisms

    Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, S´ ebastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal. A meta- transfer objective for learning to disentangle causal mechanisms. arXiv, 2019

    work page 2019

  8. [8]

    Denker, Harris Drucker, Isabelle Guyon, Lawrence D

    L´ eon Bottou, Corinna Cortes, John S. Denker, Harris Drucker, Isabelle Guyon, Lawrence D. Jackel, Yann Le Cun, Urs A. Muller, Eduard S¨ ackinger, Patrice Simard, and Vladimir Vapnik. Comparison of classifier methods: a case study in handwritten digit recognition. In ICPR, 1994

    work page 1994

  9. [9]

    Approximating CNNs with bag-of-local- features models works surprisingly well on imagenet

    Wieland Brendel and Matthias Bethge. Approximating CNNs with bag-of-local- features models works surprisingly well on imagenet. In ICLR, 2019

    work page 2019

  10. [10]

    Invariant scattering convolution networks

    Joan Bruna and Stephane Mallat. Invariant scattering convolution networks. TPAMI, 2013

    work page 2013

  11. [11]

    In- termittent process analysis with scattering moments

    Joan Bruna, Stephane Mallat, Emmanuel Bacry, and Jean-Franois Muzy. In- termittent process analysis with scattering moments. The Annals of Statistics , 2015

    work page 2015

  12. [12]

    Two theorems on invariance and causality

    Nancy Cartwright. Two theorems on invariance and causality. Philosophy of Science, 2003. 23

    work page 2003

  13. [13]

    Cheng and Hongjing Lu

    Patricia W. Cheng and Hongjing Lu. Causal invariance as an essential constraint for creating a causal representation of the world. The Oxford handbook of causal reasoning, 2017

    work page 2017

  14. [14]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. NAACL, 2019

    work page 2019

  15. [15]

    Statistics of robust optimization: A generalized empirical likelihood approach

    John Duchi, Peter Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. arXiv, 2016

    work page 2016

  16. [16]

    Domain- adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Fran¸ cois Laviolette, Mario March, and Victor Lempitsky. Domain- adversarial training of neural networks. JMLR, 2016

    work page 2016

  17. [17]

    Wichmann, and Wieland Brendel

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. ICLR, 2019

    work page 2019

  18. [18]

    Learning causal structures using regression invariance

    AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, and Kun Zhang. Learning causal structures using regression invariance. In NIPS, 2017

    work page 2017

  19. [19]

    Patrick J. Grother. NIST Special Database 19: Handprinted forms and char- acters database. https://www.nist.gov/srd/nist-special-database-19 ,

    work page

  20. [20]

    File doc/doc.ps in the 1995 NIST CD ROM NIST Special Database 19

    work page 1995

  21. [21]

    The probability approach in econometrics

    Trygve Haavelmo. The probability approach in econometrics. Econometrica: Journal of the Econometric Society , 1944

    work page 1944

  22. [22]

    Conditional variance penalties and domain shift robustness

    Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness. arXiv, 2017

    work page 2017

  23. [23]

    Invariant causal prediction for nonlinear models

    Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant causal prediction for nonlinear models. Journal of Causal Inference , 2018

    work page 2018

  24. [24]

    Revisiting visual question answering baselines

    Allan Jabri, Armand Joulin, and Laurens Van Der Maaten. Revisiting visual question answering baselines. In ECCV, 2016

    work page 2016

  25. [25]

    Johansson, David A

    Fredrik D. Johansson, David A. Sontag, and Rajesh Ranganath. Support and invertibility in domain-invariant representations. AISTATS, 2019

    work page 2019

  26. [26]

    General- ization in anti-causal learning

    Niki Kilbertus, Giambattista Parascandolo, and Bernhard Sch¨ olkopf. General- ization in anti-causal learning. arXiv, 2018

    work page 2018

  27. [27]

    Stable prediction across unknown environments

    Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, and Bo Li. Stable prediction across unknown environments. In SIGKDD, 2018

    work page 2018

  28. [28]

    Lake, Tomer D

    Brenden M. Lake, Tomer D. Ullman, Joshua B Tenenbaum, and Samuel J. Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 2017. 24

    work page 2017

  29. [29]

    James M. Lee. Introduction to Smooth Manifolds . Springer, 2003

    work page 2003

  30. [30]

    Counterfactuals

    David Lewis. Counterfactuals. John Wiley & Sons, 2013

    work page 2013

  31. [31]

    Deep domain generalization via conditional invariant adver- sarial networks

    Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adver- sarial networks. In ECCV, 2018

    work page 2018

  32. [32]

    From dependence to causation

    David Lopez-Paz. From dependence to causation. PhD thesis, University of Cambridge, 2016

    work page 2016

  33. [33]

    Discovering causal signals in images

    David Lopez-Paz, Robert Nishihara, Soumith Chintala, Bernhard Scholkopf, and L´ eon Bottou. Discovering causal signals in images. In CVPR, 2017

    work page 2017

  34. [34]

    Learning to pivot with adversarial networks

    Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial networks. In Advances in neural information processing systems , pages 981–990, 2017

    work page 2017

  35. [35]

    Domain adaptation by using causal inference to predict invariant conditional distributions

    Sara Magliacane, Thijs van Ommen, Tom Claassen, Stephan Bongers, Philip Versteeg, and Joris M Mooij. Domain adaptation by using causal inference to predict invariant conditional distributions. In NIPS, 2018

    work page 2018

  36. [36]

    Deep learning: A critical appraisal

    Gary Marcus. Deep learning: A critical appraisal. arXiv, 2018

    work page 2018

  37. [37]

    Causality from a distributional robustness point of view

    Nicolai Meinshausen. Causality from a distributional robustness point of view. In Data Science Workshop (DSW) , 2018

    work page 2018

  38. [38]

    Maximin effects in inhomogeneous large-scale data

    Nicolai Meinshausen and Peter B¨ uhlmann. Maximin effects in inhomogeneous large-scale data. The Annals of Statistics , 2015

    work page 2015

  39. [39]

    Mitchell

    Sandra D. Mitchell. Dimensions of scientific law. Philosophy of Science , 2000

    work page 2000

  40. [40]

    Causality: Models, Reasoning, and Inference

    Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2nd edition, 2009

    work page 2009

  41. [41]

    Causal inference using invariant prediction: identification and confidence intervals

    Jonas Peters, Peter B¨ uhlmann, and Nicolai Meinshausen. Causal inference using invariant prediction: identification and confidence intervals. JRSS B , 2016

    work page 2016

  42. [42]

    MIT press, 2017

    Jonas Peters, Dominik Janzing, and Bernhard Sch¨ olkopf.Elements of causal inference: foundations and learning algorithms . MIT press, 2017

    work page 2017

  43. [43]

    Incompleteness, non locality and realism

    Michael Redhead. Incompleteness, non locality and realism. a prolegomenon to the philosophy of quantum mechanics. 1987

    work page 1987

  44. [44]

    Invariant models for causal transfer learning

    Mateo Rojas-Carulla, Bernhard Sch¨ olkopf, Richard Turner, and Jonas Peters. Invariant models for causal transfer learning. JMLR, 2018

    work page 2018

  45. [45]

    Donald B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology , 1974. 25

    work page 1974

  46. [46]

    On causal and anticausal learning

    Bernhard Sch¨ olkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. In ICML, 2012

    work page 2012

  47. [47]

    Certifying some distribu- tional robustness with principled adversarial training

    Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distribu- tional robustness with principled adversarial training. ICLR, 2018

    work page 2018

  48. [48]

    Causal necessity: a pragmatic investigation of the necessity of laws

    Brian Skyrms. Causal necessity: a pragmatic investigation of the necessity of laws. Yale University Press, 1980

    work page 1980

  49. [49]

    Bob L. Sturm. A simple method to determine if a music information retrieval system is a “horse”. IEEE Transactions on Multimedia , 2014

    work page 2014

  50. [50]

    Unbiased look at dataset bias

    Antonio Torralba and Alexei Efros. Unbiased look at dataset bias. In CVPR, 2011

    work page 2011

  51. [51]

    Principles of risk minimization for learning theory

    Vladimir Vapnik. Principles of risk minimization for learning theory. In NIPS. 1992

    work page 1992

  52. [52]

    Vladimir N. Vapnik. Statistical Learning Theory. John Wiley & Sons, 1998

    work page 1998

  53. [53]

    Do we still need models or just more data and compute?, 2019

    Max Welling. Do we still need models or just more data and compute?, 2019

    work page 2019

  54. [54]

    Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht

    Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht. The marginal value of adaptive gradient methods in machine learning. In NIPS. 2017

    work page 2017

  55. [55]

    Making things happen: A theory of causal explanation

    James Woodward. Making things happen: A theory of causal explanation . Oxford university press, 2005

    work page 2005

  56. [56]

    Correlation and causation

    Sewall Wright. Correlation and causation. Journal of agricultural research , 1921

    work page 1921

  57. [57]

    Understanding deep learning requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. ICLR, 2016. 26 A Additional theorems Theorem 10. Let Σe X,X := EX e[XeXe⊤] ∈ Sd×d + , with Sd×d + the space of symmetric positive semi-definite matrices, and Σe X,ϵ := EX e[Xeϵe] ∈ Rd. Then, for any arbitrary tuple ( ...

    work page 2016

  58. [58]

    Since domain adaptation enforcesP (Φ(Xes)) =P (Φ(Xet)), it consequently enforces P ( ˆYes) =P ( ˆYet), where ˆYe = w(Φ(Xe)), for all e ∈ {es,et}

    Using these data and the domain adaptation recipe outlined above, we build a classifierw◦Φ. Since domain adaptation enforcesP (Φ(Xes)) =P (Φ(Xet)), it consequently enforces P ( ˆYes) =P ( ˆYet), where ˆYe = w(Φ(Xe)), for all e ∈ {es,et}. Then, the classification accuracy will be at most 20%. This is worse than random guessing, in a problem where simply trai...

    work page