{"id":"e4ada8f3-0399-4abe-b5a1-ba16424920f8","arxiv_id":"2501.04102","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DLG unifies augmented and invariant graph generation via edge masks and adds distribution and label consistency losses, reporting improved OOD accuracy on graph benchmarks.","lead":"This paper introduces DLG, a graph neural network training method that creates augmented and invariant graphs from the same edge-mask modifier, aiming to improve out-of-distribution generalization. It reports accuracy gains over existing graph OOD methods on several graph classification benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ld in Eq. (9) is computed from the original graph's representation ha, not from the augmented graph Ga, so the claimed distribution-consistency loss cannot actually constrain Ga; the MI derivation in Eqs. (3)-(9) is therefore not a valid support for the central mechanism.","rationale":"The reader's weakest assumption identified exactly the load-bearing spot: Eq. (9)'s contrastive loss is built on the original graph representation ha rather than the augmented graph Ga, and the substitution of p(Ga|Gi) by a label indicator is not justified. My stress-test confirms this and adds a sharper consequence: because Ld does not involve the sampled augmented graph, its gradient with respect to the sampled adjacency is zero, so the loss cannot be said to enhance the distribution consistency of augmented graphs. The claimed mutual-information maximization is thus not realized by the stated objective. This does not refute the empirical results or the possibility that DLG works for other reasons, such as the label-consistency loss Lc acting on Ga and Gv. It does mean the paper's central mechanism is unsupported as written, which reinforces the reader's conditional verdict: the empirical claims need the theoretical/implementational gap resolved before the paper's explanation of its own success can be accepted. I therefore recommend keeping the CONDITIONAL verdict rather than upgrading to acceptance or moving to rejection based on this concern alone.","tokens_in":11073,"tokens_out":5156,"duration_ms":54359,"concrete_test":"Inspect the released code and compute the gradient of Ld (Eq. 9) with respect to the Gumbel-softmax logits or the sampled adjacency matrix A_a of the augmented graph. If this gradient is identically zero, Ld does not depend on Ga, confirming that the loss cannot enforce distribution consistency of augmented graphs. As a positive control, rerun the Table I experiments with Ld computed from a true augmented-graph representation readout(GNNa(Ga, X⊕Y)) instead of ha; if results change materially, the current formulation acts differently than the paper claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-B claims that Ld maximizes I(Ga; S) to keep augmented graphs distribution-consistent. The derivation has three unjustified steps. First, Eq. (3) writes I(Ga; S) as a sum over individual support graphs, which is not the mutual information between Ga and the set S. Second, Eq. (6) replaces p(Ga|Gi) with the label indicator 1(Y=yi), conflating the data-generation process with label equality. Most importantly, Eq. (9) uses ha = readout(GNNa(A, X⊕Y)), the representation of the original graph G, not a representation of the sampled augmented graph Ga, and it inserts hv into the denominator so the softmax is no longer a normalized distribution over support graphs. Consequently Ld's gradient is zero with respect to the sampled augmented adjacency A_a: it only regularizes G's own representation. The claimed distribution consistency of Ga is therefore not enforced by Ld. The ablation 'w/o Ld' removes a representation regularizer, not a distribution-consistency constraint on augmented graphs. If this holds, the paper's central mechanism is unsupported, although the empirical ranking in Table I may still be correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DLG, a framework for graph out-of-distribution (OOD) generalization. DLG uses a shared edge-mask modifier to generate both an invariant subgraph Gv and an augmented graph Ga from an input graph G. Two consistency losses are introduced: Ld, intended to maximize the mutual information between the augmented graph and a support set of training graphs (distribution consistency), and Lc, which combines a classification loss on Gv with a self-distillation term between the predictions of Gv and Ga (label consistency). The authors evaluate DLG on TPG, DrugOOD, SP-Motif, MNIST-75sp, Graph-SST2, and Molhiv, reporting improvements over ERM, IRM, V-REx, DIR, GIL, CIGA, and GALA on most datasets.","tokens_in":11331,"tokens_out":6244,"duration_ms":56390,"significance":"The overarching idea of unifying augmentation and invariant subgraph extraction with explicit consistency regularization is timely and could be useful for graph OOD generalization. The label consistency loss Lc in Eq. (10) is a reasonable classification-plus-distillation objective that directly supervises the augmented graph through qa = f(Ga). The code is publicly available, which is a strength. However, the paper's central theoretical justification for Ld—that it maximizes I(Ga; S)—is not supported by the derivation in Eqs. (3)–(9). As written, Ld is a supervised contrastive loss on the representation of the original graph G, not on the augmented graph Ga. This significantly weakens the claimed mechanism, though the empirical gains may still hold if Ld acts as a useful representation regularizer.","major_comments":[{"comment":"The derivation of Ld as a mutual information objective is invalid for three reasons. First, Eq. (3) expresses I(Ga; S) as a sum of per-graph terms p(Ga, Gi), which is not the mutual information between Ga and the set S; the correct object would involve the joint distribution of the entire support set. Second, Eq. (6) replaces p(Ga|Gi) with the label indicator 1(Y = yi), which conflates label equality with the conditional data-generation distribution and is not an approximation with a clear error bound. Third, and most importantly, Eq. (9) uses ha, the graph-level representation of the original graph G (as stated in the text: 'as Ga is generated from G, here we use the graph representation of G as ha'), rather than a representation of the sampled augmented graph Ga. Consequently, the gradient of Ld with respect to the sampled adjacency matrix Aa is zero; the loss only regularizes the encoder output Ha, which indirectly affects the mask generator. The ablation 'w/o Ld' therefore removes a representation-level contrastive regularizer, not a distribution-consistency constraint on Ga. The authors should either reframe Ld as an auxiliary representation regularizer or provide a derivation that actually involves the augmented graph.","section":"IV-B, Eqs. (3)-(9)"},{"comment":"The abstract and conclusion claim experiments on 'both graph-level and node-level' OOD generalization datasets, but Section V and Table I contain only graph classification results. No node-level datasets, protocols, or results are presented anywhere in the manuscript. This discrepancy should be corrected either by adding node-level experiments or by removing the claim.","section":"V and Conclusion"},{"comment":"The dataset aggregation is underspecified. The text states that TPG has four variants and DrugOOD includes six datasets, yet Table I reports only one TPG column and two DrugOOD columns (EC50 and Ki). It is unclear whether the reported numbers are averages over variants/splits, which splits of DrugOOD are used (Assay, Scaffold, Size), and how the standard deviations are computed. Without this information, the reader cannot assess the statistical significance of the reported improvements.","section":"Table I"}],"minor_comments":[{"comment":"The denominator uses an undefined symbol 'hd' in the softmax expression; it should be 'ha'.","section":"Eq. (8)"},{"comment":"Reference [20] lists the journal as 'IEEE TPMI'; the correct abbreviation is 'IEEE TPAMI'.","section":"References"},{"comment":"The description of Molhiv does not specify the scaffold split or the evaluation protocol used; please provide the exact split and metric.","section":"Section V-A"},{"comment":"The ablation study in Fig. 2 is described only briefly; please include the exact α values tested and the datasets shown in the figure.","section":"Section V-C"}],"recommendation":"major_revision","confidential_remarks":"The empirical results may justify publication after a revision that corrects the derivation of Ld and either includes node-level experiments or revises the claims. The MI derivation is a load-bearing issue because the paper's title and contribution hinge on distribution consistency of augmented graphs. However, the issue is fixable by reframing Ld as an encoder-level contrastive regularizer, so I do not recommend rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DLG is a workmanlike integration of established graph-OOD machinery. The genuinely new part is a single edge-mask modifier shared by the invariant-subgraph extractor and the augmented-graph generator, plus two consistency losses: a supervised contrastive term Ld and a classification-plus-self-distillation term Lc. That combination does not appear in DIR, CIGA, or GALA. The paper runs six graph-classification benchmarks, includes ablations, and promises code. Where comparisons are available, DLG is usually at the top; the largest margins are on SP-Motif (73.34 vs. CIGA's 69.86) and MNIST-75sp (31.22 vs. 24.81). On Graph-SST2 and Molhiv the gains are small and within standard deviation.\n\nThe soft spots are real but not fatal. The derivation of Ld in Eqs. (3)-(9) does not do what it says. The sum over support graphs is not the mutual information I(Ga;S); replacing p(Ga|Gi) with the label indicator is a strong assumption; and ha in Eq. (9) is the representation of the original graph G, not of the sampled augmented graph Ga. So Ld is effectively a supervised contrastive loss on original-graph representations, not a direct constraint on Ga. The stress-test claim that the gradient is zero with respect to A_a is literally true. Even so, Ld is not inert: it updates GNNa, and since the edge masks for Ga are derived from Ha, it indirectly shapes the distribution of Ga. The paper even says it wants the representation used for generation to be similar to same-label graphs. That is a defensible objective, but it is not maximizing I(Ga;S). The MI framing should be replaced or heavily qualified.\n\nTwo smaller issues. The abstract and conclusion claim node-level experiments; the paper contains none. And the baseline comparisons are incomplete: GALA is not evaluated on the DIR datasets and DIR is not evaluated on TPG/DrugOOD, so 'best across most datasets' is a patchwork comparison rather than a uniform one.\n\nWho is this for? People working on graph OOD will want it as a strong baseline and a useful reminder that a shared modifier plus consistency losses helps. The empirical table is worth taking seriously; the theory section is not.\n\nMy recommendation: send to peer review, but flag the Ld derivation for major revision. The authors should either fix the derivation or reframe Ld honestly as a representation regularizer, and they should either add node-level experiments or delete the claim. With those changes it would be a solid paper; as it stands, it is a useful empirical contribution with an overstated theory.","headline":"DLG is a solid empirical integration of known graph-OOD ideas with a new shared edge-mask modifier; the Ld derivation overclaims and the node-level experiments are missing, but the benchmark results deserve a serious referee.","tokens_in":11848,"tokens_out":3574,"would_cite":true,"duration_ms":35577,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DLG claims that unifying augmented and invariant graph generation under two consistency losses improves graph out-of-distribution generalization beyond existing invariant-learning baselines.","keywords":["graph out-of-distribution generalization","distribution shift","graph neural networks","invariant subgraph","data augmentation","distribution consistency","label consistency","edge mask"],"falsifier":"Test DLG on a synthetic OOD benchmark with a known invariant subgraph, comparing it to a version whose Ld is replaced by ordinary supervised contrastive learning on original graphs; if accuracies match, the distribution-consistency mechanism is not the source of the gain, and one could also check whether larger Ld values correspond to larger estimated mutual information between generated augmented graphs and held-out same-label graphs.","tokens_in":10879,"feed_emoji":"🧠","tokens_out":7923,"duration_ms":67499,"temperature":0.7,"pith_summary":"Graph neural networks lose accuracy when test graphs come from a different distribution than training graphs. The paper claims the standard two-step remedy—first augment training graphs into new environments, then extract invariant subgraphs—is suboptimal because each step can break a consistency: augmented graphs can drift away from plausible data, and extracted subgraphs can lose their link to the label. DLG instead learns edge masks with GNN encoders and differentiable gumbel-softmax sampling to produce both augmented and invariant graphs in one unified pass, and optimizes a distribution-consistency loss Ld and a label-consistency loss Lc. On TPG, DrugOOD, SP-Motif, MNIST-75sp, Graph-SST2, and Molhiv, the paper reports DLG outperforming IRM, V-REx, DIR, GIL, CIGA, and GALA on most datasets.","feed_headline":"Two consistency losses beat graph OOD baselines on seven benchmarks","feed_subtitle":"A shared edge-mask modifier keeps augmented graphs on-distribution and invariant subgraphs label-true.","key_machinery":"The central mechanism is a shared edge-mask modifier: two GNN encoders (GNNv for invariant graphs, GNNa for augmented graphs) produce node representations, an MLP turns pairs of node representations into per-edge sampling probabilities via a sigmoid dot product, and gumbel-softmax makes the edge sampling differentiable. The same modifier yields Gv and Ga, so augmentation and invariant-subgraph extraction are unified as two kinds of edge modification. Ld maximizes a contrastive surrogate for I(Ga; S), pulling the augmented representation toward same-label support graphs and away from different-label graphs and from the original graph; Lc combines the true-label cross-entropy on Gv with a term qv log qa that keeps the class distributions of Gv and Ga close.","core_discovery":"The paper's central claim is that the usual two-step graph OOD recipe fails on consistency, and that explicitly optimizing distribution consistency and label consistency fixes it. Concretely, DLG uses two GNN encoders plus an MLP to compute per-edge sampling probabilities, then applies gumbel-softmax sampling to generate an invariant graph Gv and an augmented graph Ga from the same input graph. The augmented graph is trained with Ld to maximize a contrastive approximation of mutual information with existing same-label training graphs while staying distinct from the original graph, and both generated graphs are trained with Lc to preserve the original label and align their predicted class distributions. The reported experiments show DLG achieving the best accuracy on most of the synthetic and real-world graph classification datasets considered.","pith_inferences":["In the paper's derivation of Ld, p(Ga|Gi) is replaced by the label indicator 1(Y=yi), so the objective is effectively a supervised contrastive loss; the paper does not demonstrate that this surrogate bounds or tracks the true mutual information, so the distribution-consistency claim is only as strong as that leap.","Because Ld is computed from the original graph's representation ha rather than from Ga itself, the encoder's stability during training is an unstated requirement: if the encoder changes, the alignment target shifts, and the loss may not reflect augmented-graph consistency.","A natural extension the paper only gestures at is applying the same two-consistency modifier to node-level OOD settings; the conclusion claims node-level experiments, but the body reports only graph-level tables, so that transfer is not yet evidenced.","The diversity term inside Ld, which pushes augmented representations away from Gv, could be tested directly: measure whether augmented graphs actually explore new label-consistent regions or merely act as a regularizer that improves the encoder."],"forward_implications":["If DLG is right, graph OOD generalization can be improved without explicitly partitioning training environments; one modifier plus two consistency losses suffices.","Augmented graphs that stay close to existing same-label graphs should reduce unrealistic molecular edits, which matters for drug-affinity prediction on DrugOOD and Molhiv.","Label consistency gives a concrete way to keep supervision information in invariant subgraphs, and the ablations suggest this matters most under large distribution shifts such as SP-Motif and MNIST-75sp.","The framework's gains on low-dimensional node-feature datasets indicate that consistency-preserving augmentation can retain informativeness where richer features are unavailable."],"supporting_citations":[{"why":"Baseline that learns causally invariant subgraphs; supplies the invariant-subgraph approach DLG compares against.","marker":"[48]"},{"why":"Baseline using causal rationales and interventional augmentations; also provides the SP-Motif, MNIST-75sp, Graph-SST2, and Molhiv datasets and protocols.","marker":"[34]"},{"why":"Baseline and source of the TPG and DrugOOD datasets; motivates the need to handle unknown causal/spurious variance.","marker":"[50]"},{"why":"Early graph OOD method that generates domains adversarially; motivates environment augmentation in graphs.","marker":"[18]"},{"why":"Contrastive predictive coding is the basis for the mutual-information estimation used in the Ld derivation.","marker":"[53]"},{"why":"Gumbel-softmax makes the edge-mask sampling differentiable, allowing end-to-end optimization.","marker":"[51]"},{"why":"DrugOOD benchmark provides six real-world drug-affinity OOD datasets used in evaluation.","marker":"[55]"}],"fun_headline_variants":["Dual consistency losses outdo graph OOD baselines","Unified modifier with dual consistency beats graph OOD","Label and distribution consistency key to graph OOD success","One modifier ensures distribution and label consistency for OOD","Graph OOD: two consistency objectives outperform baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The distribution-consistency loss rests on treating 'the probability that an augmented graph was generated from a support graph' as simply 'the two graphs have the same label,' so the contrastive loss is assumed to be a faithful stand-in for the mutual information between augmented and existing graphs.","fun_headline_variants_meta":{"raw":{"variants":["Dual consistency losses outdo graph OOD baselines","Unified modifier with dual consistency beats graph OOD","Label and distribution consistency key to graph OOD success","One modifier ensures distribution and label consistency for OOD","Graph OOD: two consistency objectives outperform baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000297,"raw_usage":{"total_tokens":1698,"prompt_tokens":901,"completion_tokens":797,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":517,"tokens_out":797,"duration_ms":7382,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:40:38.474263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Test DLG on a synthetic OOD benchmark with a known invariant subgraph, comparing it to a version whose Ld is replaced by ordinary supervised contrastive learning on original graphs; if accuracies match, the distribution-consistency mechanism is not the source of the gain, and one could also check whether larger Ld values correspond to larger estimated mutual information between generated augmented graphs and held-out same-label graphs.","supporting_citations":[{"cited_title":"Learning causally invariant representations for out-of- distribution generalization on graphs,","cited_arxiv_id":null,"evidence_quote":"Baseline that learns causally invariant subgraphs; supplies the invariant-subgraph approach DLG compares against."},{"cited_title":"Discovering invariant rationales for graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Baseline using causal rationales and interventional augmentations; also provides the SP-Motif, MNIST-75sp, Graph-SST2, and Molhiv datasets and protocols."},{"cited_title":"Does in- variant graph learning via environment augmentation learn invariance?","cited_arxiv_id":null,"evidence_quote":"Baseline and source of the TPG and DrugOOD datasets; motivates the need to handle unknown causal/spurious variance."},{"cited_title":"Handling distribution shifts on graphs: An invariance perspective,","cited_arxiv_id":null,"evidence_quote":"Early graph OOD method that generates domains adversarially; motivates environment augmentation in graphs."},{"cited_title":"Categorical reparameterization with gumbel-softmax,","cited_arxiv_id":null,"evidence_quote":"Gumbel-softmax makes the edge-mask sampling differentiable, allowing end-to-end optimization."}],"review_version":1}