Pith. sign in

REVIEW 2 major objections 5 minor 79 references

Learning Causality for Modern Machine Learning

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Causal subgraph learning unlocks OOD, explainable, and attack-safe GNNs

desk verdict A thesis compiling six peer-reviewed papers into a causal-invariance narrative for graph learning; the GALA identification theorem has an unproven and likely false implication, making the 'provable' claim conditional on an unstated assumption. read the letter →

arxiv 2506.12226 v1 pith:UCHIIZCF submitted 2025-06-13 cs.LG stat.ML

Yongqiang Chen This is my paper
classification cs.LGstat.ML
keywords causalinvarianceout-of-distributiongeneralizationgraphneuralnetworksinvariantriskminimizationinterpretabilityadversarialrobustnessdistributionshiftindependentmechanisms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis argues that the causal-invariance principle—the conditional distribution of the target given its causes stays fixed under distribution shifts—should be a backbone for modern machine learning. It shows that applying this principle to graph data yields three payoffs: out-of-distribution generalization, faithful interpretability, and adversarial robustness. To make the principle practical, the thesis develops several methods: CIGA and GALA identify the invariant subgraph under provable conditions; GMT fixes interpretable GNNs; HAO makes graph injection attacks homophily-unnoticeable; PAIR and FeAT resolve the optimization conflict with empirical risk minimization. The central claim is that causality can be learned from graphs without environment labels, under minimal assumptions of variation sufficiency and variation consistency.

What carries the argument

The load-bearing object is the invariant subgraph $G_c$ generated by the latent causal part $C$ under a structural causal model, together with the information-theoretic property that $G_c$ maximizes the conditional mutual information of extracted subgraphs across environments while minimizing the spurious counterpart. The thesis operationalizes this through contrastive objectives (CIGA), an environment-assistant split of positive and negative graphs (GALA), the subgraph multilinear extension (SubMT) for interpretability (GMT), and homophily distribution regularization for adversarial unnoticeability (HAO). The optimization methods PAIR and FeAT target the ERM conflict via Pareto balancing of per-environment risks and feature-augmented training rounds.

What would settle it

Build the two-piece graph worlds $E_{tr}=\{(0.2,0.1),(0.2,0.3)\}$ and $E_{tr}'$ with the roles of $G_c$ and $G_s$ swapped so that $P(Y,G)$ is identical but the true causal subgraph differs; train CIGA and GALA on both. If either method outputs the same extracted subgraph in both worlds, the paper's no-free-graph-OOD-lunch result (Cor. 3.3.6) is falsified, while failure in one world confirms the unidentifiability claim.

Watch

Extended reading notes

Core claim

On its own terms, the thesis establishes that the independent-causal-mechanism principle can be lifted from tabular and image data to graphs: assume a graph $G$ is assembled from an invariant subgraph $G_c$ (controlled by latent $C$) and a spurious subgraph $G_s$ (affected by environment $E$); then a GNN that predicts the label $Y$ only from $G_c$ is invariant to distribution shifts. CIGA formalizes this and proves that an objective maximizing intra-class mutual information of extracted subgraphs identifies the true $G_c$ under invertibility and equal-environment assumptions, producing an invariant GNN. GALA strengthens the theory by showing that without environment labels, environment generation and inference both fail unless variation sufficiency and variation consistency hold, and gives a no-free-lunch result; under those assumptions, an ERM-trained environment assistant that separates correctly- versus incorrectly-predicted graphs enables provable identification. The interpretability chapter shows existing attention-based XGNNs fail to approximate the subgraph multilinear extension because they push a soft subgraph through a nonlinear GNN, and proposes GMT, which approximates SubMT via random subgraph sampling or a linearized variant and satisfies counterfactual fidelity. The adversarial-robustness chapter proves graph injection attacks can emulate and exceed modification attacks because of their flexibility, but that flexibility destroys homophily and makes them easily pruned; adding homophily unnoticeability restores their power. The optimization chapters, PAIR and FeAT, show the invariance objective conflicts with ERM and that Pareto balancing or feature augmentation with additional training rounds recovers causal invariance and richer feature learning.

Load-bearing premise

Graphs must be generated by a causal subgraph plus a spurious subgraph whose spurious correlations vary sufficiently across training environments while maintaining a consistent direction; the thesis itself proves that without these conditions no algorithm can learn the invariance.

Editorial extensions

If this is right

  • If the CIGA/GALA identification guarantees hold, invariant subgraph learning can serve as a principled front-end for any GNN that must survive distribution shifts, without requiring environment labels.
  • If GMT's SubMT approximation is correct, interpretable GNNs should be evaluated with counterfactual fidelity, and attention-based soft-subgraph explanations that skip SubMT are likely to mislead.
  • If HAO's homophily unnoticeability is the right threat model, graph robustness evaluations that ignore homophily shift overstate the power of injection attacks.
  • If PAIR and FeAT are correct, adding Pareto optimization or feature-augmented retraining to existing OOD algorithms can resolve the ERM conflict and improve OOD accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: use variation sufficiency and variation consistency as dataset diagnostics, and report a benchmark as unlearnable by invariant methods when they fail, which would discipline OOD benchmark construction.
  • The counterfactual-fidelity measure suggests a practical rule for explainable models: an explanation's predictive power should be continuously sensitive to small subgraph perturbations, not merely rankable by attention scores.
  • If homophily unnoticeability is adopted in standards, graph adversarial robustness comparisons should report both attack success and the induced homophily shift, since cheap pruning defenses should be neutralized before claims of attack power are made.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The thesis argues that causal invariance learning, instantiated on graph data through invariant-subgraph identification, improves out-of-distribution (OOD) generalization, interpretability, and adversarial robustness in modern machine learning. It develops six methods across three parts: CIGA (Chapter 2) and GALA (Chapter 3) for invariant graph learning under distribution shifts; GMT (Chapter 4) and HAO (Chapter 5) for interpretability and adversarial robustness as implications of causal thinking; and PAIR (Chapter 6) and FeAT (Chapter 7) for optimization and feature learning in invariant risk minimization. The central theoretical claims include Theorem 2.3.1 (CIGA induces an invariant GNN under invertible graph generation and size assumptions), Theorem 3.4.1 (GALA provably identifies the invariant subgraph under 'minimal' assumptions), and Theorem 4.5.1 (GMT approximates the subgraph multilinear extension). Each chapter is based on previously published peer-reviewed papers and includes extensive experiments on synthetic and real benchmarks.

Significance. If the theoretical claims hold, the thesis makes a valuable contribution by extending invariant causal learning from Euclidean data to graphs, providing explicit identifiability results and negative results (e.g., Corollary 3.3.6), and demonstrating consistent empirical gains across many graph benchmarks. The thesis also ships detailed proofs in the appendices and is unusually explicit about limitations and future directions in several appendices. The empirical evaluation is broad, covering synthetic motif datasets, DrugOOD, WILDS, and geometric graph benchmarks. The main significance depends on whether GALA's provable identifiability truly follows from the stated 'minimal' assumptions, because that result anchors the thesis-level claim that causal invariance learning on graphs is feasible without environment labels. The empirical comparisons and the negative results are useful regardless, but the provable-identifiability claim needs repair.

major comments (2)
  1. [§3.4.3, Theorem 3.4.1] Assumption (iii) in Theorem 3.4.1 is load-bearing but is not shown to follow from Assumption 3.3.3. The theorem requires I(G_p_s; G_n_s|Y)=0 for spurious subgraphs extracted from the assistant's positive and negative sets, and the text asserts this 'is an implication of the variation sufficiency (Assumption 3.3.3)' without proof. Variation sufficiency only guarantees that for any spurious subgraph there exist training environments with different P(Y|G_s); it does not control the conditional dependence between the spurious subgraphs in {G_p} and {G_n}. Nothing in Assumptions 3.3.3 or 3.3.5 prevents the same spurious pattern from appearing in both sets, in which case I(G_p_s; G_n_s|Y)>0 and the GALA objective can select G_s instead of G_c. Moreover, condition (iii) is stated in terms of idealized argmax subgraphs, whereas Algorithm 1 defines {G_p} and {G_n} through the ERM-trained assistant's correctness; no proof links these empirical sets to the mutual-information condition. Until this implication is proved, or assumption (iii) is explicitly added as a premise and its necessity acknowledged, the claim that GALA provably identifies the invariant subgraph under the 'minimal' assumptions is unsupported.
  2. [§2.3.3, Theorem 2.3.1 / Assumption 2.2.1] Theorem 2.3.1(i) and (ii) both rely on invertibility of f_G_gen and f_Gc_gen in Assumption 2.2.1, together with balanced environment sizes. Invertibility is a strong condition on graph generation functions; it rules out many-to-one assemblies such as random attachment of G_s to G_c and aggregate node or edge features. The thesis does not test or discuss whether the SCMs behind its synthetic and real benchmarks satisfy this condition. Since Theorem 2.3.1 is the basis for the abstract's claim that CIGA can provably identify the underlying G_c, the scope of this guarantee should be stated precisely, and evidence or argument should be provided that at least the evaluated settings meet the invertibility condition.
minor comments (5)
  1. [Abstract] The text reads 'An Al-aided drug discovery system'; this should be 'An AI-aided drug discovery system'.
  2. [Table 2.2 caption] The phrase 'in the blankets' should be 'in the parentheses' (averaged rank is reported in the parentheses).
  3. [§3.5.1 and Table 3.2] The dataset is referred to as both 'Graph-SST' and 'Graph-SST2' in different places; the notation should be unified.
  4. [§3.4.1, Eq. (3.3)] The notation pG_c^s = g(G^s) with 'G^s ~ P(G|Y)' is confusing: the superscript 's' is not defined as a sample index and can be mistaken for the spurious subgraph. Consider using an index like G' or G_j.
  5. [Chapter 3, Table 3.1] The checkmark/cross table for DisC and CIGA is informative, but the table should define whether GALA is claimed to handle both H(S|Y)<H(C|Y) and H(S|Y)>H(C|Y) on the same data-generation process or under the additional condition (iii); this would clarify the scope of the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the thesis compiles the author's prior peer-reviewed results, but the central derivations are conditional on explicit SCM assumptions and are not equivalent to their inputs by construction.

full rationale

I walked the main derivation chains: CIGA (Ch. 2) defines graph generation via SCMs and derives contrastive objectives whose identifiability theorems are proved in Appendices A.4.2/A.4.3; GALA (Ch. 3) establishes no-free-lunch results and proves its identifiability theorem in Appendix B.5.5 under stated assumptions; GMT (Ch. 4), HAO (Ch. 5), PAIR (Ch. 6), and FeAT (Ch. 7) similarly give explicit architectures/objectives and appendix proofs. None of the central claims is obtained by fitting a parameter to a target and then renaming the fit as a prediction. The heavy self-citation reflects the thesis's nature as a compilation of the author's own papers; the cited prior work is peer-reviewed and the assumptions, not the conclusions, are the load-bearing premises. The potential concern that Theorem 3.4.1's condition iii) is asserted to follow from variation sufficiency without proof is a rigor question rather than a circularity: the main text explicitly refers to Appendix B.5.5 for the proof, and even if that proof were incomplete the failure mode would be an unsupported implication, not an equation that reduces to its own input. I therefore find no constructional circularity and no fitted-input-called-prediction pattern.

Assumptions & free parameters 4 free parameters · 7 assumptions · 3 invented entities

The central claim relies on a structural causal model with explicit latent variables (C, S) and subgraphs (Gc, Gs), plus strong assumptions on environment variation (3.3.3, 3.3.5) and invertibility. The methods introduce several hyperparameters that must be tuned per dataset. No independent evidence exists for the latent constructs beyond the assumed framework.

free parameters (4)
  • sc (size of invariant subgraph) = unknown; set by hyperparameter in CIGA
    CIGA requires knowing the size of the invariant subgraph Gc or uses a selection ratio. The performance is sensitive to this parameter, and it is tuned per dataset.
  • alpha and beta (contrastive and hinge loss weights) = tuned per dataset (figures 2.4, 2.5)
    In CIGA, alpha controls the contrastive loss and beta controls the hinge loss. The thesis reports sensitivity but does not provide a principled way to set them.
  • lambda (information bottleneck weight) = tuned per dataset in GMT
    The IB objective in interpretable GNNs requires a trade-off weight lambda, chosen by hand.
  • penalty weight and upsampling times in GALA = tuned per dataset
    GALA introduces a penalty weight for the contrastive term and upsampling times for handling imbalanced positive/negative sets; both are hyperparameters.
assumptions (7)
  • domain assumption Independent Causal Mechanism (ICM) principle
    The thesis assumes that causal mechanisms are independent and that P(Y|PA(Y)) is invariant under shifts. This is the foundation of all methods.
  • domain assumption Assumption 2.2.1: Graph generation SCM
    The graph is generated from latent invariant C and spurious S via functions f_Gc_gen, f_Gs_gen, and f_G_gen.
  • domain assumption Assumption 2.2.2 and 2.2.3: FIIF and PIIF SCMs
    The interaction between C and S is either fully informative (Y independent of S given C) or partially informative (S depends on Y and E).
  • domain assumption Assumption 2.2.4: Better Clustered Invariant Features
    The invariant features are better clustered given the label than spurious features, i.e., H(C|Y) <= H(S|Y). This is a separation assumption.
  • domain assumption Assumption 3.3.3: Variation sufficiency
    Training environments must reveal all variations of spurious subgraphs, otherwise Gc is unidentifiable.
  • domain assumption Assumption 3.3.5: Variation consistency
    The correlation strengths between invariant/spurious subgraphs and labels must be consistent, e.g., H(C|Y) != H(S|Y), to avoid the unidentifiable mixing case.
  • domain assumption Invertibility of generation functions in Theorem 2.3.1
    Theorem 2.3.1 assumes f_G_gen and f_Gc_gen are invertible, which is a strong condition on the data generation process.
invented entities (3)
  • Latent invariant variable C
    purpose: Captures the underlying causes of the label Y; assumed to be invariant across environments.
    C is a latent construct with no direct falsifiable handle; it is inferred from data under the assumed SCM.
  • Latent spurious variable S
    purpose: Captures environment-dependent variation that spuriously correlates with Y.
    S is a latent variable introduced to model distribution shifts; it is not observable and has no independent evidence.
  • Invariant subgraph Gc and spurious subgraph Gs
    purpose: The graph-level realizations of C and S; Gc is the target of extraction in all methods.
    These subgraphs are postulated to exist in every input graph and are the objects of estimation. Their existence and separability are assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Causality for Modern Machine Learning." pith.science (2026). https://pith.science/paper/UCHIIZCF

@misc{pith2026250612226,
  author       = {Pith},
  title        = {Pith review of: Learning Causality for Modern Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UCHIIZCF}},
  note         = {Machine review of arXiv:2506.12226}
}
read the original abstract

In the past decades, machine learning with Empirical Risk Minimization (ERM) has demonstrated great capability in learning and exploiting the statistical patterns from data, or even surpassing humans. Despite the success, ERM avoids the modeling of causality the way of understanding and handling changes, which is fundamental to human intelligence. When deploying models beyond the training environment, distribution shifts are everywhere. For example, an autopilot system often needs to deal with new weather conditions that have not been seen during training, An Al-aided drug discovery system needs to predict the biochemical properties of molecules with respect to new viruses such as COVID-19. It renders the problem of Out-of-Distribution (OOD) generalization challenging to conventional machine learning. In this thesis, we investigate how to incorporate and realize the causality for broader tasks in modern machine learning. In particular, we exploit the invariance implied by the principle of independent causal mechanisms (ICM), that is, the causal mechanisms generating the effects from causes do not inform or influence each other. Therefore, the conditional distribution between the target variable given its causes is invariant under distribution shifts. With the causal invariance principle, we first instantiate it to graphs -- a general data structure ubiquitous in many real-world industry and scientific applications, such as financial networks and molecules. Then, we shall see how learning the causality benefits many of the desirable properties of modern machine learning, in terms of (i) OOD generalization capability; (ii) interpretability; and (iii) robustness to adversarial attacks. Realizing the causality in machine learning, on the other hand, raises a dilemma for optimization in conventional machine learning, as it often contradicts the objective of ERM...

Figures

Figures reproduced from arXiv: 2506.12226 by the authors.

Figure 2.1
Figure 2.1. Illustration of Causality Inspired Invariant Graph LeArning (CIGA): GNNs need to classify graphs based on the specific motif (“House” or “Cycle”). The featurizer g will extract an (orange-colored) subgraph Gp c from each input for the classifier fc to predict the label. The training objective of g is implemented in a contrastive strategy where the distribution of Gp c at the latent sphere will be optimized to maximi… view at source ↗
Figure 2.2
Figure 2.2. SCMs on graph distribution shifts. on Gc . To extract the desired subgraph Gc , we derive an information-theoretic objective for the featurizer to identify subgraphs that maximally preserve the invariant intra-class information across a set of different (unknown) environments. We theoretically show that this approach can provably identify the underlying Gc under mild assumptions (Sec. 2.3). Experiments on 16 synthet… view at source ↗
Figure 2.3
Figure 2.3. Failures of OOD generalization on graphs: (a) GNNs are required to classify whether the graph contains a “house” or “cycle” motif, where the colors represent node features. However, distribution shifts in the training data exist at both structure-level (from left to right: “house” mostly co-occur with a hexagon), attribute-level (from upper to lower: nodes are mostly colored green if the graph contains a “house”, or… view at source ↗
Figures from the paper (21 more)
Figure 2.4
Figure 2.4. Figure 2.4: Hyperparameter sensitivity analysis on the coefficient of contrastive loss (α). 1 2 3 4 5 6 7 8 beta 35 40 45 50 55 60 test auc erm CIGAv2 (a) SPMotif-Mixed (bias=0.9, α=4) 1 2 3 4 5 6 7 8 beta 68.0 68.5 69.0 69.5 70.0 70.5 test auc erm CIGAv2 (b) DrugOOD-Scaffold (α…
Figure 2.5
Figure 2.5. Figure 2.5: Hyperparameter sensitivity analysis on the coefficient of hinge loss (β). Hyperparameter sensitivity analysis. To examine how sensitive CIGA is to the hyper￾paramters α and β for contrastive loss and hinge loss, respectively. We conduct experiments based on the harde…
Figure 3.1
Figure 3.1. Figure 3.1: An illustration of GALA with the task of classifying graphs according to whether there exists a “House” or “Cycle” motif. Given the training data where the “House” subgraph often co-occurs with a “Grid” and the “Cycle” subgraph often co-occurs with a “Hexagon”. An ER…
Figure 3.2
Figure 3.2. Figure 3.2: SCMs on graph distribution shifts. OOD generalization on graphs. Basi￾cally, we follow the same data genera￾tion assumptions as Sec. 2.2.1, which is inspired by real-world drug discovery task (Murray & Rees, 2009a) and cov￾ers a broad case of graph distribution shift…
Figure 3.3
Figure 3.3. Figure 3.3: Ablation studies. Correlation strengths of tG p u and tG n u. We conduct experiments with the two-piece graph datasets evaluated in [PITH_FULL_IMAGE:figures/full_fig_p073_3_3.png]
Figure 4.1
Figure 4.1. Figure 4.1: Illustration of Subgraph Multilinear Extension (SubMT). The task is to classify whether a graph contains a specific “house” or “cycle” motif. An XGNN f = fc ˝ g predicts the label with the classifier fc based on the extracted soft subgraph Gp c = g(G), denoted as the…
Figure 4.2
Figure 4.2. Figure 4.2: Illustration of counterfactual faithfulness. When given more complicated GNNs, the approximation error to SubMT can be even higher, as verified in Appendix C.6.6. For example, when k = 2 and |Y| = 1, Eq. 4.9 is convex, and we have fc(E[A]) ď E[ fc(A)] due to Jensen’s…
Figure 4.3
Figure 4.3. Figure 4.3: Ablation studies. optimization, as discussed in Appendix C.5.2. 4.5.3 Learning neural subgraph multilinear extension Although GMT trained with GMT-sam improve interpretability, GMT-sam still requires multiple random subgraph sampling to approximate SubMT and costs mu…
Figure 5.1
Figure 5.1. Figure 5.1: The lower test robustness indicates better attack performance. (a) Without defenses: GIA performs consistently better than GMA; (b) With defenses: GIA without HAO performs consistently worse than GMA, while GIA with HAO performs the best; (c) Homophily indicates the …
Figure 5.2
Figure 5.2. Figure 5.2: Power and pitfalls of Graph Injection Attack As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p100_5_2.png]
Figure 5.3
Figure 5.3. Figure 5.3: Effects of HAO with different weights. Performance of non-targeted attacks. In [PITH_FULL_IMAGE:figures/full_fig_p109_5_3.png]
Figure 5.4
Figure 5.4. Figure 5.4: (a) Homophily changes after attacked by GIA without HAO (orange) and GIA with HAO (canny); (b), (c) Attack performance against GCN and EGuard with different node and edge budgets. ‚ indicates attack with defenses and ▲ indicates attack without defenses; Analysis of t…
Figure 5.5
Figure 5.5. Figure 5.5: SCMs of the graph adversarial attacks. Chen et al. (2022b), the generative processes of the observed graph G and the target label Y are controlled by latent variables C and S. Graph Models. For the generation of the k-hop ego-graph of node u, G, more specifically, C …
Figure 6.1
Figure 6.1. Figure 6.1: Optimization issues in OOD algorithms. (a) OOD objectives such as IRM usually require several relaxations for the ease of optimization, which however introduces huge gaps. The ellipsoids denote solutions that satisfy the invariance constraints of practical IRM varian…
Figure 6.2
Figure 6.2. Figure 6.2: Pareto front of ERM losses w.r.t. environ￾ments. Understanding the IRM failures through the MOO per￾spective. To begin with, it is natural to reformulate the prac￾tical IRM problem (Eq. 6.3) as a MOO problem: min φ (LERM,LIRM) T , (6.5) where LERM = 1 |Etr| ř ePEtr L…
Figure 6.3
Figure 6.3. Figure 6.3: Variance distribution. A solution that is simultaneously optimal to all training environments (i.e., satisfying the original IRM constraints) is also a stationary point of ERM loss w.r.t. some OOD distribution: BLt/B fIRM = 0, Lt P tÿ ePEtr λeLe | ÿ ePEtr λe = 1u, (6…
Figure 6.4
Figure 6.4. Figure 6.4: Recovery of causal invariance. The causal invariance (Definition. 6.3.2) requires the model predictions to be independent of the spurious features within the overlapped invariant features. In this example, intuitively it requires the colored belts to be perpendicular…
Figure 6.5
Figure 6.5. Figure 6.5: (a) Each point is the best performed IRMX among corresponding pretraining epoch (x-axis), the IRMv1 penalty weights (y-axis) and all possible VREx penalty weights. Despite the substantial tunning efforts, IRMX performs no better than PAIR. That is because (b) PAIR ca…
Figure 7.1
Figure 7.1. Figure 7.1: (a) An illustration of FeAT (top row) compared to ERM (bottom row). Different colors in samples denote the respective dominant features. As the original data is dominated by spurious features (blue), ERM tends to learn more spurious features but limited invariant fea…
Figure 7.2
Figure 7.2. Figure 7.2: The convergences of CIRMv1 and feature learning coefficients (FL) with or with￾out ERM pre-training (PT). The invariant and spurious feature learning terms are the mean of xwj,r , jv1y and xwj,r , jv2y for j P t˘1u,r P [m], respectively. The training environments are…
Figure 7.3
Figure 7.3. Figure 7.3: GradCAM visualization on ColoredMNIST-025, where the shortcuts are now concen￾trated to a colored path at the up left. Three visualizations are drawn for each sample: the original figure, the gray-colored gradcam, and the gradcam. It can be found that ERM can not pro…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

79 extracted references · 40 canonical work pages

  1. [4]

    Linear unit-tests for invariance discovery.arXiv preprint arXiv:2102.10867,

    Aubin, B., Słowik, A., Arjovsky, M., Bottou, L., and Lopez-Paz, D. Linear unit-tests for invariance discovery.arXiv preprint arXiv:2102.10867,

    arXiv

  2. [5]

    J., Kiros, J

    Ba, L. J., Kiros, J. R., and Hinton, G. E. Layer normalization.arXiv preprint, arXiv:1607.06450,

    arXiv

  3. [8]

    Accounting for Unobserved Confounding in Domain Generalization

    Bellot, A. and van der Schaar, M. Generalization and invariances in the presence of unobserved confounding.arXiv preprint, arXiv:2007.10653,

    work page Pith review arXiv 2007

  4. [10]

    T., Li, Y., Lundberg, S

    Bubeck, S., Chandrasekaran, V ., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P ., Lee, Y. T., Li, Y., Lundberg, S. M., Nori, H., Palangi, H., Ribeiro, M. T., and Zhang, Y. Sparks of artificial general intelligence: Early experiments with GPT-4.arXiv preprint, arXiv:2303.12712,

    arXiv

  5. [14]

    Estimating generalization under distribution shifts via domain-invariant representations

    Chuang, C., Torralba, A., and Jegelka, S. Estimating generalization under distribution shifts via domain-invariant representations. InInternational Conference on Machine Learning, volume 119, pp. 1984–1994. PMLR,

    work page 1984

  6. [17]

    Robust learning with progressive data expansion against spurious correlation.arXiv preprint, arXiv:2306.04949,

    Deng, Y., Yang, Y., Mirzasoleiman, B., and Gu, Q. Robust learning with progressive data expansion against spurious correlation.arXiv preprint, arXiv:2306.04949,

    arXiv

  7. [19]

    Distributionally Robust Losses for Latent Covariate Mixtures

    BIBLIOGRAPHY432 Duchin, J. C., Hashimoto, T., and Namkoong, H. Distributionally robust losses for latent covariate mixtures.arXiv preprint arXiv:2007.13982,

    work page Pith review arXiv 2007

  8. [20]

    P ., Joshi, C

    Dwivedi, V . P ., Joshi, C. K., Laurent, T., Bengio, Y., and Bresson, X. Benchmarking graph neural networks.arXiv preprint, arXiv:2003.00982,

    arXiv 2003

Show all 79 references
  1. [21]

    Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    arXiv

  2. [22]

    Double equivariance for inductive link prediction for both new nodes and new relation types.arXiv preprint, arXiv:2302.01313,

    Gao, J., Zhou, Y., Zhou, J., and Ribeiro, B. Double equivariance for inductive link prediction for both new nodes and new relation types.arXiv preprint, arXiv:2302.01313,

    arXiv

  3. [23]

    F., Peters, M

    Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P ., Liu, N. F., Peters, M. E., Schmitz, M., and Zettlemoyer, L. Allennlp: A deep semantic natural language processing platform. arXiv preprint, arXiv:1803.07640,

    arXiv

  4. [24]

    Joint learning of label and environment causal independence for graph out-of-distribution generalization.arXiv preprint, arXiv:2306.01103, 2023a

    Gui, S., Liu, M., Li, X., Luo, Y., and Ji, S. Joint learning of label and environment causal independence for graph out-of-distribution generalization.arXiv preprint, arXiv:2306.01103, 2023a. Gui, S., Liu, M., Li, X., Luo, Y., and Ji, S. Joint learning of label and environment...

    arXiv

  5. [25]

    Counterfactual learning on graphs: A survey.arXiv preprint, arXiv:2304.01391,

    Guo, Z., Xiao, T., Aggarwal, C., Liu, H., and Wang, S. Counterfactual learning on graphs: A survey.arXiv preprint, arXiv:2304.01391,

    arXiv

  6. [26]

    SIGUA: Forgetting may make learning with noisy labels more robust

    BIBLIOGRAPHY435 Han, B., Niu, G., Yu, X., Yao, Q., Xu, M., Tsang, I., and Sugiyama, M. SIGUA: Forgetting may make learning with noisy labels more robust. InProceedings of the 37th International Conference on Machine Learning, pp. 4006–4016, 2020a. Han, B., Yao, Q., Liu, T., Ni...

    2011 arXiv

  7. [27]

    Does distributionally robust supervised learning give robust classifiers? InInternational Conference on Machine Learning, pp

    BIBLIOGRAPHY436 Hu, W., Niu, G., Sato, I., and Sugiyama, M. Does distributionally robust supervised learning give robust classifiers? InInternational Conference on Machine Learning, pp. 2034–2042,

  8. [28]

    Graph neural networks prov- ably benefit from structural information: A feature learning perspective.arXiv preprint arXiv:2306.13926,

    Huang, W., Cao, Y., Wang, H., Cao, X., and Suzuki, T. Graph neural networks prov- ably benefit from structural information: A feature learning perspective.arXiv preprint arXiv:2306.13926,

    arXiv

  9. [29]

    and Wallace, B

    BIBLIOGRAPHY437 Jain, S. and Wallace, B. C. Attention is not explanation. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3543–3556,

    2019

  10. [30]

    Ji, Y., Zhang, L., Wu, J., Wu, B., Huang, L.-K., Xu, T., Rong, Y., Li, L., Ren, J., Xue, D., Lai, H., Xu, S., Feng, J., Liu, W., Luo, P ., Zhou, S., Huang, J., Zhao, P ., and Bian, Y. DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery ...

    arXiv

  11. [31]

    Empowering graph representation learning with test-time graph transformation.arXiv preprint, arXiv:2210.03561,

    Jin, W., Zhao, T., Ding, J., Liu, Y., Tang, J., and Shah, N. Empowering graph representation learning with test-time graph transformation.arXiv preprint, arXiv:2210.03561,

    arXiv

  12. [32]

    E., Pillai, N

    Johndrow, J. E., Pillai, N. S., and Smith, A. No free lunch for approximate mcmc.arXiv preprint, arXiv:2010.12514,

    2010 arXiv

  13. [33]

    Kipf, T. N. and Welling, M. Variational graph auto-encoders.arXiv preprint, arXiv:1611.07308,

    arXiv

  14. [34]

    Kirichenko, P ., Izmailov, P ., and Wilson, A. G. Last layer re-training is sufficient for robustness to spurious correlations.arXiv preprint arXiv:2204.02937,

    arXiv

  15. [35]

    W., and Clark, B

    Kochkov, D., Pfaff, T., Sanchez-Gonzalez, A., Battaglia, P . W., and Clark, B. K. Learn- ing ground states of quantum hamiltonians with graph networks.arXiv preprint, arXiv:2110.06390,

    arXiv

  16. [36]

    FLAG: adversarial data augmentation for graph neural networks.arXiv preprint, arXiv:2010.09891,

    BIBLIOGRAPHY440 Kong, K., Li, G., Ding, M., Wu, Z., Zhu, C., Ghanem, B., Taylor, G., and Goldstein, T. FLAG: adversarial data augmentation for graph neural networks.arXiv preprint, arXiv:2010.09891,

    2010 arXiv

  17. [37]

    and Yamaguchi, S

    Koyama, M. and Yamaguchi, S. Out-of-distribution generalization with maximal invariant predictor.arXiv preprint arXiv:2008.01883, 2020a. Koyama, M. and Yamaguchi, S. Out-of-distribution generalization with maximal invariant predictor.arXiv preprint, arXiv:2008.01883, 2020b. Kr...

    2008 arXiv

  18. [39]

    Deeper insights into graph convolutional networks for semi- supervised learning

    Li, Q., Han, Z., and Wu, X. Deeper insights into graph convolutional networks for semi- supervised learning. InAAAI Conference on Artificial Intelligence, pp. 3538–3545, 2018a. Li, X., Gui, S., Luo, Y., and Ji, S. Graph structure and feature extrapolation for out-of- distribut...

    arXiv

  19. [40]

    Bayesian invariant risk minimization

    Lin, Y., Dong, H., Wang, H., and Zhang, T. Bayesian invariant risk minimization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009, 2022b. Lin, Y., Zhu, S., Tan, L., and Cui, P . ZIN: When and how to learn invariance without environment partiti...

    arXiv

  20. [41]

    Z., Haghgoo, B., Chen, A

    Liu, E. Z., Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P . W., Sagawa, S., Liang, P ., and Finn, C. Just train twice: Improving group robustness without training group information. InInternational Conference on Machine Learning, pp. 6781–6792, 2021a. Liu, G., Zhao, T., Xu...

    arXiv

  21. [42]

    Clear: Generative counterfactual explanations on graphs

    Ma, J., Guo, R., Mishra, S., Zhang, A., and Li, J. Clear: Generative counterfactual explanations on graphs. InAdvances in Neural Information Processing Systems, pp. 25895–25907, 2022a. Ma, K., Yang, H., Yang, H., Jin, T., Chen, P ., Chen, Y., Kamhoua, B. F., and Cheng, J. Impr...

    arXiv

  22. [43]

    Towards better out-of-distribution generalization of neural algorithmic reasoning tasks.arXiv preprint arXiv:2211.00692,

    BIBLIOGRAPHY445 Mahdavi, S., Swersky, K., Kipf, T., Hashemi, M., Thrampoulidis, C., and Liao, R. Towards better out-of-distribution generalization of neural algorithmic reasoning tasks.arXiv preprint arXiv:2211.00692,

    arXiv

  23. [44]

    Fisher discriminant analysis with kernels

    Mika, S., Ratsch, G., Weston, J., Scholkopf, B., and Mullers, K.-R. Fisher discriminant analysis with kernels. InNeural networks for signal processing IX: Proceedings of the 1999 IEEE signal processing society workshop (cat. no. 98th8468), pp. 41–48,

    1999

  24. [45]

    Towards stable backdoor purification through feature shift tuning.arXiv preprint arXiv:2310.01875,

    Min, R., Qin, Z., Shen, L., and Cheng, M. Towards stable backdoor purification through feature shift tuning.arXiv preprint arXiv:2310.01875,

    arXiv

  25. [46]

    M., Bause, F., Kersting, K., Mutzel, P ., and Neumann, M

    Morris, C., Kriege, N. M., Bause, F., Kersting, K., Mutzel, P ., and Neumann, M. Tudataset: A collection of benchmark datasets for learning with graphs.arXiv preprint, arXiv:2007.08663,

    2007 arXiv

  26. [47]

    M., Grohe, M., Fey, M., and Borgwardt, K

    Morris, C., Lipman, Y., Maron, H., Rieck, B., Kriege, N. M., Grohe, M., Fey, M., and Borgwardt, K. M. Weisfeiler and leman go machine learning: The story so far.arXiv preprint, arXiv:2112.09992,

    arXiv

  27. [48]

    and Rees, D

    Murray, C. and Rees, D. The rise of fragment-based drug discovery.Nature chemistry, 1: 187–92, 06 2009a. Murray, C. W. and Rees, D. C. The rise of fragment-based drug discovery.Nature chemistry, 1 3:187–92, 2009b. Naganuma, H., Ahuja, K., Mitliagkas, I., Takagi, S., Motokawa, ...

    arXiv

  28. [49]

    M., Nicolicioiu, A

    Nicolicioiu, A. M., Nicolicioiu, A. L., Alexe, B., and Teney, D. Learning diverse features in vision transformers for improved generalization.ICML 2023 Workshop on Spurious Correlations, Invariance, and Stability,

    2023

  29. [50]

    Discovering environments with XRM.arXiv preprint, arXiv:2309.16748,

    Pezeshki, M., Bouchacourt, D., Ibrahim, M., Ballas, N., Vincent, P ., and Lopez-Paz, D. Discovering environments with XRM.arXiv preprint, arXiv:2309.16748,

    arXiv

  30. [51]

    Fishr: Invariant gradient variances for out-of- distribution generalization.arXiv preprint arXiv:2109.02934,

    BIBLIOGRAPHY449 Rame, A., Dancette, C., and Cord, M. Fishr: Invariant gradient variances for out-of- distribution generalization.arXiv preprint arXiv:2109.02934,

    arXiv

  31. [52]

    Model rata- touille: Recycling diverse models for out-of-distribution generalization.arXiv preprint arXiv:2212.10445,

    Ramé, A., Ahuja, K., Zhang, J., Cord, M., Bottou, L., and Lopez-Paz, D. Model rata- touille: Recycling diverse models for out-of-distribution generalization.arXiv preprint arXiv:2212.10445,

    arXiv

  32. [53]

    Domain-adjusted regression or: Erm may already learn features sufficient for out-of-distribution generalization.arXiv preprint arXiv:2202.06856,

    Rosenfeld, E., Ravikumar, P ., and Risteski, A. Domain-adjusted regression or: Erm may already learn features sufficient for out-of-distribution generalization.arXiv preprint arXiv:2202.06856,

    arXiv

  33. [54]

    Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108,

    Sanh, V ., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108,

    1910 arXiv

  34. [55]

    Causality for machine learning.arXiv preprint, arXiv:1911.10500,

    Schölkopf, B. Causality for machine learning.arXiv preprint, arXiv:1911.10500,

    1911 arXiv

  35. [57]

    S., and Li, B

    Sun, L., Dou, Y., Yang, C., Wang, J., Yu, P . S., and Li, B. Adversarial attack and defense on graph data: A survey.arXiv preprint, arXiv:1812.10528,

    arXiv

  36. [58]

    Sun, Y., Wang, S., Tang, X., Hsieh, T., and Honavar, V . G. Adversarial attacks on graph neural networks via node injections: A hierarchical reinforcement learning approach. In The Web Conference 2020, pp. 673–683,

    2020

  37. [61]

    Evading the simplicity bias: Training a diverse set of models discovers solutions with superior OOD generalization

    Teney, D., Abbasnejad, E., Lucey, S., and van den Hengel, A. Evading the simplicity bias: Training a diverse set of models discovers solutions with superior OOD generalization. arXiv preprint arXiv:2105.05612,

    arXiv

  38. [62]

    Evading the simplicity bias: Training a diverse set of models discovers solutions with superior OOD generalization

    Teney, D., Abbasnejad, E., Lucey, S., and van den Hengel, A. Evading the simplicity bias: Training a diverse set of models discovers solutions with superior OOD generalization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16740–16751, 2022a. Teney, D....

    arXiv

  39. [64]

    Representation learning with contrastive predictive coding.arXiv preprint, arXiv:1807.03748,

    van den Oord, A., Li, Y., and Vinyals, O. Representation learning with contrastive predictive coding.arXiv preprint, arXiv:1807.03748,

    arXiv

  40. [65]

    Wang, X., Cheng, M., Eaton, J., Hsieh, C., and Wu, S. F. Attack graph convolutional networks by adding fake nodes.arXiv preprint, arXiv:1810.10751,

    arXiv

  41. [66]

    Towards out-of- distribution generalizable predictions of chemical kinetics properties

    Wang, Z., Chen, Y., Duan, Y., Li, W., Han, B., Cheng, J., and Tong, H. Towards out-of- distribution generalizable predictions of chemical kinetics properties. InNeurIPS 2023 AI for Science Workshop, 2023b. Wen, Z. and Li, Y. Toward understanding the feature learning process of...

    2023

  42. [67]

    Recent advances in reliable deep graph learning: Adversarial attack, inherent noise, and distribution shift.arXiv preprint arXiv:2202.07114, 2022a

    Wu, B., Li, J., Hou, C., Fu, G., Bian, Y., Chen, L., and Huang, J. Recent advances in reliable deep graph learning: Adversarial attack, inherent noise, and distribution shift.arXiv preprint arXiv:2202.07114, 2022a. Wu, F., Jr., A. H. S., Zhang, T., Fifty, C., Yu, T., and Weinb...

    arXiv

  43. [68]

    Optimization of graph neural networks: BIBLIOGRAPHY459 Implicit acceleration by skip connections and more depth

    Xu, K., Zhang, M., Jegelka, S., and Kawaguchi, K. Optimization of graph neural networks: BIBLIOGRAPHY459 Implicit acceleration by skip connections and more depth. InInternational Conference on Machine Learning, pp. 11592–11602, 2021a. Xu, K., Zhang, M., Li, J., Du, S. S., Kawa...

    arXiv

  44. [69]

    Freeze then train: Towards provable representation learning under spurious correlations and feature noise.arXiv preprint arXiv:2210.11075,

    Ye, H., Zou, J., and Zhang, L. Freeze then train: Towards provable representation learning under spurious correlations and feature noise.arXiv preprint arXiv:2210.11075,

  45. [70]

    XGNN: towards model-level explanations of graph neural networks

    Yuan, H., Tang, J., Hu, X., and Ji, S. XGNN: towards model-level explanations of graph neural networks. InThe 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 430–438, 2020a. Yuan, H., Yu, H., Gui, S., and Ji, S. Explainability in graph neural networks: A...

    2012 arXiv

  46. [71]

    Z., and Ravikumar, P

    Zhai, R., Dan, C., Kolter, J. Z., and Ravikumar, P . Understanding why generalized reweight- ing does not improve over ERM.arXiv preprint arXiv:2201.12293,

    arXiv

  47. [72]

    and Bottou, L

    Zhang, J. and Bottou, L. Learning useful representations for shifting tasks and distributions. arXiv preprint arXiv:2212.07346,

    arXiv

  48. [73]

    Rich feature construction for the optimization- generalization dilemma.arXiv preprint arXiv:2203.15516, 2022a

    BIBLIOGRAPHY462 Zhang, J., Lopez-Paz, D., and Bottou, L. Rich feature construction for the optimization- generalization dilemma.arXiv preprint arXiv:2203.15516, 2022a. Zhang, M., Sohoni, N. S., Zhang, H. R., Finn, C., and Ré, C. Correct-n-contrast: A contrastive approach for i...

    arXiv

  49. [74]

    F., Tehrani, A

    Zhang, X., Wang, L., Helwig, J., Luo, Y., Fu, C., Xie, Y., Liu, M., Lin, Y., Xu, Z., Yan, K., Adams, K., Weiler, M., Li, X., Fu, T., Wang, Y., Yu, H., Xie, Y., Fu, X., Strasser, A., Xu, S., Liu, Y., Du, Y., Saxton, A., Ling, H., Lawrence, H., Stärk, H., Gui, S., Edwards, C., G...

    arXiv

  50. [75]

    S., Gordon, G

    Zhao, H., Dan, C., Aragam, B., Jaakkola, T. S., Gordon, G. J., and Ravikumar, P . Fundamental limits and tradeoffs in invariant representation learning.arXiv preprint arXiv:2012.10713,

    2012 arXiv

  51. [76]

    Graph neural networks: A review of methods and applications.AI Open, 1:57–81, 2020a

    Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., and Sun, M. Graph neural networks: A review of methods and applications.AI Open, 1:57–81, 2020a. Zhou, J., Bevilacqua, B., and Ribeiro, B. An ood multi-task perspective for link prediction with new rela...

    arXiv

  52. [77]

    Explaining and adapting graph conditional shift.arXiv preprint, arXiv:2306.03256,

    Zhu, Q., Jiao, Y., Ponomareva, N., Han, J., and Perozzi, B. Explaining and adapting graph conditional shift.arXiv preprint, arXiv:2306.03256,

    arXiv

  53. [78]

    Understanding the generalization of adam in learning neural networks with proper regularization.arXiv preprint arXiv:2108.11371, 2021a

    Zou, D., Cao, Y., Li, Y., and Gu, Q. Understanding the generalization of adam in learning neural networks with proper regularization.arXiv preprint arXiv:2108.11371, 2021a. Zou, D., Liu, S., Miao, S., Fung, V ., Chang, S., and Li, P . GDL-DS: A benchmark for geometric deep lea...

    arXiv

  54. [79]

    Adversarial attacks on neural networks for graph data

    Zügner, D., Akbarnejad, A., and Günnemann, S. Adversarial attacks on neural networks for graph data. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2847–2856, 2018

    2018

  55. [1996]

    Nuanced metrics for measuring unintended bias with real data for text classification

    Borkan, D., Dixon, L., Sorensen, J., Thain, N., and Vasserman, L. Nuanced metrics for measuring unintended bias with real data for text classification. InCompanion of The 2019 World Wide Web Conference, pp. 491–500,

    2019

  56. [1998]

    Towards better generalization with flexible representation of multi-module graph neural networks.arXiv preprint, arXiv:2209.06589,

    Lee, H., Park, H., and Yoon, K. Towards better generalization with flexible representation of multi-module graph neural networks.arXiv preprint, arXiv:2209.06589,

    arXiv

  57. [1999]

    Ulyanov, D., Vedaldi, A., and Lempitsky, V . S. Instance normalization: The missing ingredient for fast stylization.arXiv preprint, arXiv:1607.08022,

    arXiv

  58. [2005]

    Understanding and improving graph injection attack by promoting unnoticeability

    Chen, Y., Yang, H., Zhang, Y., Ma, K., Liu, T., Han, B., and Cheng, J. Understanding and improving graph injection attack by promoting unnoticeability. InInternational Conference on Learning Representations, 2022a. Chen, Y., Zhang, Y., Bian, Y., Yang, H., Ma, K., Xie, B., Liu,...

    2024

  59. [2006]

    Crabbé, J

    ISBN 0471241954. Crabbé, J. and van der Schaar, M. Evaluating the robustness of interpretability methods through explanation invariance and equivariance.arXiv preprint, arXiv:2304.06715,

    arXiv

  60. [2008]

    Idea: Invariant causal defense for graph adversarial robustness.arXiv preprint, arXiv:2305.15792,

    Tao, S., Cao, Q., Shen, H., Wu, Y., Xu, B., and Cheng, X. Idea: Invariant causal defense for graph adversarial robustness.arXiv preprint, arXiv:2305.15792,

    arXiv

  61. [2013]

    E., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R

    BIBLIOGRAPHY453 Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(1):1929–1958,

    1929

  62. [2014]

    A., Hassani, K., Jayaraman, P

    BIBLIOGRAPHY454 Taghanaki, S. A., Hassani, K., Jayaraman, P . K., Ahmadi, A. H. K., and Custis, T. Point- mask: Towards interpretable and bias-resilient point cloud processing.arXiv preprint, arXiv:2007.04525,

    2007 arXiv

  63. [2015]

    Causality and independence enhancement for biased node classification

    Chen, G., Wang, Y., Guo, F., Guo, Q., Shao, J., Shen, H., and Cheng, X. Causality and independence enhancement for biased node classification. InACM International Conference on Information and Knowledge Management, pp. 203–212, 2023a. Chen, H. S., Lee, Y., Setlur, A., Levine, ...

    arXiv

  64. [2016]

    W., Hamrick, J

    Battaglia, P . W., Hamrick, J. B., Bapst, V ., Sanchez-Gonzalez, A., Zambaldi, V . F., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gülçehre, Ç., Song, H. F., Ballard, A. J., Gilmer, J., Dahl, G. E., Vaswani, A., Allen, K. R., Nash, C., Langston, V ., ...

    arXiv

  65. [2017]

    and Li, Y

    Allen-Zhu, Z. and Li, Y. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning.arXiv preprint arXiv:2012.09816,

    2012 arXiv

  66. [2018]

    The iwildcam 2020 competition dataset.arXiv preprint arXiv:2004.10340,

    Beery, S., Cole, E., and Gjoka, A. The iwildcam 2020 competition dataset.arXiv preprint arXiv:2004.10340,

    2020 arXiv

  67. [2019]

    A closer look at distribution shifts and out-of-distribution generalization on graphs

    Ding, M., Kong, K., Chen, J., Kirchenbauer, J., Goldblum, M., Wipf, D., Huang, F., and Goldstein, T. A closer look at distribution shifts and out-of-distribution generalization on graphs. InNeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications,

    2021

  68. [2020]

    Learning a similarity metric discriminatively, with application to face verification

    Chopra, S., Hadsell, R., and LeCun, Y. Learning a similarity metric discriminatively, with application to face verification. In2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 20-26 June 2005, San Diego, CA, USA, pp. 539–546,

    2005

  69. [2021]

    Invariant risk minimization.arXiv preprint arXiv:1907.02893,

    Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez-Paz, D. Invariant risk minimization.arXiv preprint arXiv:1907.02893,

    1907 arXiv

  70. [2022]

    The evolution of out-of-distribution robustness throughout fine-tuning.arXiv preprint arXiv:2106.15831,

    Andreassen, A., Bahri, Y., Neyshabur, B., and Roelofs, R. The evolution of out-of-distribution robustness throughout fine-tuning.arXiv preprint arXiv:2106.15831,

    arXiv

  71. [2023]

    W., Xu, R., Cranmer, K., Spergel, D

    Cranmer, M., Sanchez-Gonzalez, A., Battaglia, P . W., Xu, R., Cranmer, K., Spergel, D. N., and Ho, S. Discovering symbolic models from deep learning with inductive biases.arXiv preprint, arXiv:2006.11287,

    2006 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.