{"id":"d6d449b7-99f5-4053-8aee-d1b88df6879c","arxiv_id":"2505.04956","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Graffe uses a diffusion decoder conditioned on graph encoder outputs to learn node and graph representations, and it argues (with a flawed proof step) that the denoising objective is a lower bound on conditional mutual information.","lead":"Graffe is a self-supervised method that trains a graph encoder by using its embeddings to condition a diffusion decoder that reconstructs node features. It reports state-of-the-art accuracy on 9 of 11 graph classification benchmarks and adds a theoretical claim that denoising loss is a lower bound on conditional mutual information.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3's proof drops the 2/d factor when exponentiating Eq (26); as stated the mutual-information lower bound does not follow. The bound may be repairable with a factor d/2 on the loss term, but the theorem must be restated and reproven.","rationale":"The reader's weakest assumption correctly identifies the exponentiation error between Eqs. (26) and (27), and this is the most load-bearing flaw in the paper: the theorem advertised as the Diff-InfoMax principle is not established as stated. My re-derivation confirms that the correct exponent on the mutual information term is -2I/d, not -I, which changes the final bound's coefficients and its scaling with dimension d. This is not a cosmetic slip; the stated inequality I >= -log L + log(d/(2*pi*e)) + (2/d)h cannot be recovered from the preceding steps, and the claimed bound would be implausible for large d. The same proof steps do suggest a corrected bound of the form I >= h(x0|xt) - (d/2) log L + (d/2) log(d/(2*pi*e)), so the qualitative message that minimizing the conditional DSM loss raises a lower bound on conditional mutual information may survive after revision. I therefore agree with the reader's CONDITIONAL assessment rather than moving to REJECT: the empirical contribution is strong, with state-of-the-art or near-state-of-the-art linear probe results on 9 of 11 datasets and careful ablations supporting the architectural choices. The lack of released code is a secondary reproducibility concern. The theoretical section, however, must be corrected before the paper's central claims can be accepted as proven.","tokens_in":23655,"tokens_out":7902,"duration_ms":76197,"concrete_test":"Independently re-derive Eq. (27) from Eq. (26) by exponentiating both sides: write B = (2*pi*e/d)E[Tr], so B^{d/2} >= exp(h(x0|xt) - I); raising both sides to 2/d yields B >= exp(2h/d) exp(-2I/d), i.e. E[Tr] >= (d/(2*pi*e)) exp(2h/d) exp(-2I/d). Then substitute this corrected inequality into Eq. (29), take logs, and verify whether the result matches Theorem 3 (Eq. 30) or instead reads I >= h(x0|xt) - (d/2) log L + (d/2) log(d/(2*pi*e)). A numerical sanity check with d=10 and diagonal conditional covariances can confirm which inequality is actually satisfied.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is Theorem 3 (Eq. 20/30): I(x0;Ephi(x0)|xt) >= -log L_{x0,DSM,phi,t} + log(d/(2*pi*e)) + (2/d)h(x0|xt). The decisive step is between Eq. (26), (d/2) log((2*pi*e/d) E[Tr]) >= h(x0|xt) - I, and Eq. (27), E[Tr] >= (d/(2*pi*e)) exp(2h/d) exp(-I). Exponentiating Eq. (26) correctly gives B = (2*pi*e/d)E[Tr] satisfying B^{d/2} >= exp(h - I), hence B >= exp(2h/d) exp(-2I/d), i.e. E[Tr] >= (d/(2*pi*e)) exp(2h/d) exp(-2I/d). The mutual information appears with coefficient -2I/d, not -I, so Eqs. (29) and (30) inherit this error and the theorem as stated is not proven. Repeating the final logarithm with the corrected exponent yields I >= h(x0|xt) - (d/2) log L + (d/2) log(d/(2*pi*e)), a different bound whose dependence on d and on the loss is not the claimed one. The stated bound is also dimensionally implausible for large d: both I and the natural scale of -log L grow with d, whereas the claimed right-hand side has -log L with coefficient 1 and (2/d)h. Separately, the claimed 'tractable' constant C includes the unknown differential entropy h(x0|xt), so the bound cannot be evaluated without additional estimation; it is constant in phi but not tractable as stated. These issues affect the theoretical section; the empirical results and ablations remain independent evidence for the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Graffe, a self-supervised graph representation learning method based on conditional diffusion models. A GNN encoder maps a masked graph into node or graph representations, and these representations condition a GraphU-Net or MLP denoising decoder that reconstructs node features. The paper's central theoretical claim is Theorem 3: the negative logarithm of a per-time denoising score matching loss lower-bounds the conditional mutual information I(x0; E_phi(x0) | xt) up to a constant, which the authors call the Diff-InfoMax principle. The paper also reports extensive node and graph classification experiments on 11 datasets, several of which achieve state-of-the-art accuracy, together with ablations on masking, reconstruction target, encoder, and decoder design.","tokens_in":24086,"tokens_out":10810,"duration_ms":109369,"significance":"If the theoretical claim were correct, the paper would offer a principled information-theoretic justification for diffusion-based representation learning and a strong new graph self-supervised method. The empirical evaluation is a clear strength: 11 datasets, reported error bars, many baselines, and thorough ablations. However, the proof of the central theorem contains an algebraic error at Eqs. (26)--(27) that invalidates the stated bound, and the claimed 'tractable' constant is not actually computable without estimating an unknown entropy term. The practical method may still be sound, but the paper's main theoretical contribution is not established in its current form.","major_comments":[{"comment":"The step from Eq. (26) to Eq. (27) is algebraically incorrect. With A = (2*pi*e/d) E[Tr(Cov[x0|xt,E_phi(x0)])], Eq. (26) states (d/2) log A >= h(x0|xt) - I, where I = I(x0; E_phi(x0)|xt). Exponentiating gives A >= exp(2h/d) * exp(-2I/d), not A >= exp(2h/d) * exp(-I). This missing factor of 2/d propagates through Eqs. (29)--(30), so Theorem 3 as stated is not proven. A corrected derivation would yield a different bound, e.g. I >= h(x0|xt) - (d/2) log((2*pi*e/d) E[Tr]), and the coefficients on the loss and the entropy terms would differ from those in Eq. (30). The theorem, the Diff-InfoMax principle, and the discussion in Section IV-B must be reworked accordingly.","section":"Section IV-B, Eqs. (26)--(30)"},{"comment":"The paper repeatedly describes the mutual-information lower bound as 'tractable,' but the constant C in Eq. (20) contains the differential entropy h(x0|xt), which is not known in closed form for graph feature distributions and is not estimated anywhere in the paper. This quantity is independent of phi, so it is constant with respect to the encoder parameters, but it is not a computable constant. The claim that the negative log of the denoising score matching loss is a 'tractable lower bound' is therefore not supported without an additional estimation procedure.","section":"Abstract and Section IV-B, Eq. (20)"},{"comment":"The empirical evidence in Figure 3 plots the negative logarithm of the training loss against linear probing accuracy on the same dataset, which shows only that lower training loss is associated with higher downstream accuracy. This is in-sample correlation and does not test the mutual-information inequality in Theorem 3; in particular, it cannot distinguish the proposed bound from a generic effect of improved reconstruction. The paper should present this as indirect motivation rather than as validation of the theoretical claim.","section":"Section IV-B, Figure 3"}],"minor_comments":[{"comment":"The subsection title contains a typo: 'Backgound' should be 'Background.'","section":"Section III-A"},{"comment":"There are misspellings in the related work section, including 'constractive' and 'constrastive'; these should be corrected to 'contrastive.'","section":"Section II-A"},{"comment":"In the proof of Theorem 1, the phrase 'objective objective' appears twice; one occurrence should be removed.","section":"Appendix A"},{"comment":"The section title 'HYPER-PARAMTER CONFIGURATIONS' should be 'HYPER-PARAMETER CONFIGURATIONS.'","section":"Appendix B"},{"comment":"Theorem 3 is stated for a per-time loss L_{x0,DSM,phi,t}, while the main training objective in Eq. (6) integrates over time. The paper should clarify explicitly how the per-time theorem relates to the integrated training objective.","section":"Section IV-B, Eq. (20)"},{"comment":"No source code is released. Given the number of implementation choices (noise schedules, mask ratios, decoder details), a public implementation would substantially improve reproducibility.","section":"Overall"}],"recommendation":"major_revision","confidential_remarks":"The paper's central theoretical claim is load-bearing for its framing as a principled InfoMax-style method, and the current proof of Theorem 3 is not correct. I am not recommending rejection because the empirical method appears strong and the theoretical result may be repairable with a corrected statement and proof, but the authors will need to significantly rework Section IV-B and the Diff-InfoMax principle. I would also encourage the editor to request code release as part of the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The empirical side of this paper is genuinely strong. Graffe is a clever and well-executed instantiation of diffusion-based graph SSL: an explicit GNN encoder conditions a tailored Graph-UNet decoder that denoises corrupted node features. The design choices are motivated, the ablations are informative, and the results are impressive—state-of-the-art on 9 of 11 datasets, with sensible gains over DDM and other generative baselines. The frequency-domain discussion, while qualitative, is a nice addition. If the paper were only this, I would be fairly enthusiastic.\n\nThe theory, however, has a load-bearing flaw. The step from Eq. (26) to Eq. (27) in the proof of Theorem 3 is wrong: exponentiating (d/2) log((2πe/d) E[Tr]) ≥ h(x0|xt) − I gives E[Tr] ≥ (d/(2πe)) exp(2h/d) exp(−2I/d), not exp(−I). So the stated lower bound I(x0; Eφ(x0)|xt) ≥ −log L + C does not follow. The stress-test note is correct, and the error propagates to Eqs. (29) and (30). I also agree that calling C \"tractable\" is misleading, since it hides the unknown differential entropy h(x0|xt); it is constant in φ, but not computable without extra estimation. This is not a nitpick—it is the central theoretical claim of the paper.\n\nThat said, the theory is not the only thing here. Theorem 1 and Theorem 2 are standard conditional-expectation facts and are proved correctly. The flawed step looks repairable in spirit (the bound might be restated with a d/2 factor on the loss term), but as written the theorem is unproven. The empirical results and ablations do not depend on Theorem 3, so the method can stand on its own evidence. I would not call the paper incoherent; the flaw is a genuine error, not a hidden agenda.\n\nMinor concerns: no code is released, which limits verification; and Figure 3 correlates the training loss itself with downstream accuracy, which is suggestive but somewhat circular as theoretical validation. The authors should also be more careful about calling the DSM loss \"equivalent\" to the data-prediction loss without spelling out the weighting change.\n\nWho is this for? Anyone working on graph self-supervised learning or diffusion-based representation learning will want to know about Graffe's design and results. It deserves a serious referee—the empirical contribution is meaningful and the theory can be corrected in revision, but the current version should not be accepted without fixing Theorem 3.","headline":"Solid empirical graph-SSL paper with a real exponentiation error in its main mutual-information theorem; worth peer review for the method, not for the theory as stated.","tokens_in":24627,"tokens_out":1168,"would_cite":true,"duration_ms":13592,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68R10","94A17"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the negative log of the conditional denoising score matching loss is a tractable lower bound on conditional mutual information between data and learned representation, and it uses this Diff-InfoMax principle to…","keywords":["diffusion probabilistic models","graph representation learning","self-supervised learning","denoising score matching","conditional mutual information","InfoMax principle","graph neural networks"],"falsifier":"Re-derive Eq. (27) from Eq. (26) symbolically: exponentiating $(d/2)\\log(2\\pi e/d \\cdot \\mathbb{E}[\\mathrm{Tr}]) \\geq h(x_0 \\mid x_t) - I$ gives $\\mathbb{E}[\\mathrm{Tr}] \\geq (d/2\\pi e)\\exp(2h/d)\\exp(-2I/d)$. If the paper's $\\exp(-I)$ is used instead, plug in any concrete joint distribution with known conditional mutual information, such as a Gaussian linear model, and check numerically whether the claimed inequality $I(x_0; E_\\phi(x_0) \\mid x_t) \\geq -\\log L + C$ actually holds; if it fails, Theorem 3 as stated is false.","tokens_in":23434,"feed_emoji":"🕸️","tokens_out":8842,"duration_ms":77405,"temperature":0.7,"pith_summary":"The paper argues that diffusion models can be repurposed from generators into representation learners for graphs. Its central theoretical claim is that the negative logarithm of the conditional denoising score matching loss is a lower bound on the conditional mutual information between the data and a learned representation given the noisy observation. The paper calls this the Diff-InfoMax principle and uses it to justify training a graph encoder jointly with a conditional diffusion decoder. Empirically, Graffe achieves the best reported linear-probing accuracy on 9 of the 11 node and graph classification datasets tested.","feed_headline":"Denoising loss provably bounds representation information","feed_subtitle":"The paper also builds Graffe, a graph diffusion model that uses this bound and sets a new best on 9 of 11 datasets.","key_machinery":"The central object is the conditional denoising score matching loss $L_{x_0,\\mathrm{DSM},\\phi,t} = \\mathbb{E}[\\|x_\\theta(x_t, t, E_\\phi(x_0)) - x_0\\|^2]$, the reconstruction error of a diffusion decoder that receives the noisy data $x_t$, the time step $t$, and the representation $z = E_\\phi(x_0)$ as a condition. Theorem 1 identifies the minimum of this loss with the trace of the conditional covariance $\\mathrm{Tr}(\\mathrm{Cov}[x_0 \\mid x_t, E_\\phi(x_0)])$; Lemma 3 bounds that trace from below by entropy; Theorem 3 assembles these into the Diff-InfoMax inequality. The Diff-InfoMax principle states that a representation $f(x)$ should maximize $\\int \\lambda(t)\\, I(x; f(x) \\mid x_t)\\, dt$, the conditional mutual information between data and representation given the noisy observation, and the paper argues that the denoising objective is a tractable surrogate for it. In Graffe, random feature masking before the encoder prevents the trivial identity shortcut, and the decoder is a Graph-UNet that is GNN-based for graph-level tasks and MLP-based for node-level tasks, fusing $x_t$, $t$, and $z$ to reconstruct the original features.","core_discovery":"At the center of the paper is Theorem 3: for $x_0 \\in \\mathbb{R}^d$, the conditional denoising score matching loss at time $t$, $L_{x_0,\\mathrm{DSM},\\phi,t} = \\mathbb{E}[\\|x_\\theta(x_t, t, E_\\phi(x_0))-x_0\\|^2]$, satisfies $I(x_0; E_\\phi(x_0) \\mid x_t) \\geq -\\log L_{x_0,\\mathrm{DSM},\\phi,t} + C$, with $C = \\log(d/(2\\pi e)) + \\frac{2}{d}h(x_0 \\mid x_t)$. Interpreted plainly, pushing down the denoising loss is pushing up a lower bound on the conditional mutual information, so the diffusion reconstruction objective is a principled surrogate for representation quality. The paper also proves that conditioning on an extra representation strictly decreases the minimal achievable denoising loss (Theorem 2), and that the standard InfoMax objective arises as the special case where the noisy observation is pure noise (Remark 1). The Graffe model converts these statements into an algorithm: a graph encoder produces the conditioning representation, and a Graph-UNet decoder reconstructs masked, noised node features.","pith_inferences":["If the exponent step in the proof of Theorem 3 is repaired, the same Diff-InfoMax logic would likely extend to any differentiable encoder, suggesting the theoretical result is not graph-specific even though Graffe's architecture is.","A natural testable extension is to condition the decoder on a set of representations extracted at multiple noise levels, potentially forcing the encoder to preserve both low- and high-frequency information simultaneously.","The masking ratio could be made adaptive and learned rather than tuned per dataset, since the paper's own ablation shows the optimal mask ratio varies widely across tasks, which is an empirical knob the theory does not currently explain.","A direct falsification of the proof's algebra would be to compute both sides of Theorem 3 on a simple synthetic distribution with known conditional mutual information; if the inequality fails, the theorem as stated cannot be salvaged by the given proof."],"forward_implications":["If Theorem 3 holds, the standard diffusion reconstruction loss is not just a generation objective: minimizing it is equivalent to maximizing a lower bound on conditional mutual information, giving a principled justification for using diffusion probabilistic models as representation learners.","Because conditioning on a more informative representation lowers the denoising loss (Theorem 2), the same training signal pushes the encoder toward richer representations, which explains why the representation condition outperforms label-conditioned and unconditional denoising in the paper's experiments.","The Diff-InfoMax principle subsumes the classical InfoMax objective as the special case where the noisy observation is pure Gaussian noise, so diffusion-based self-supervised learning can be seen as a multi-scale generalization of InfoMax rather than a competing paradigm.","Graffe's linear-probing results on 9 of 11 datasets imply that generative self-supervision can be competitive with or better than contrastive and masked-autoencoder methods on graphs, at least under the linear evaluation protocol.","For node-level tasks, using an MLP rather than a GNN in the decoder works better because GNN message passing mixes nodes with different noise levels; this suggests that decoder design should isolate noise per node."],"supporting_citations":[{"why":"introduces the denoising score matching loss that Theorems 1 and 3 are built on.","marker":"[8]"},{"why":"supplies the DDPM parameterization and diffusion process used in the decoder design.","marker":"[7]"},{"why":"establishes the reverse-time SDE that defines the conditional denoising problem.","marker":"[40]"},{"why":"defines the InfoMax principle that Diff-InfoMax generalizes.","marker":"[20]"},{"why":"provides tractable lower bounds for mutual information, motivating the bound in Theorem 3.","marker":"[44]"},{"why":"InfoDiffusion is the prior principled probabilistic diffusion representation learning model that Graffe builds on.","marker":"[11]"},{"why":"SODA demonstrates conditional diffusion for representation learning and supports the encoder-decoder design choice.","marker":"[12]"},{"why":"Diffusion autoencoders introduce the semantic latent from diffusion models that motivates conditioning the decoder on a learned representation.","marker":"[37]"},{"why":"GraphMAE is the leading masked graph autoencoder baseline that Graffe compares against and whose feature-reconstruction strategy it adopts.","marker":"[33]"}],"fun_headline_variants":["Denoising loss provably bounds representation info; Graffe tops 9/11 datasets","Provable MI bound: denoising loss is a graph representation surrogate","Diffusion denoising bound links info to graphs; Graffe sets 9/11 SOTA","Denoising loss lower-bounds MI; graph diffusion Graffe achieves SOTA","Graffe: diffusion-based graph embeddings with a provable info bound"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is a single algebra step in the proof of Theorem 3: exponentiating inequality (26) gives a factor $\\exp(-2I/d)$, not $\\exp(-I)$, so the stated lower bound depends on that step being repaired.","fun_headline_variants_meta":{"raw":{"variants":["Denoising loss provably bounds representation info; Graffe tops 9/11 datasets","Provable MI bound: denoising loss is a graph representation surrogate","Diffusion denoising bound links info to graphs; Graffe sets 9/11 SOTA","Denoising loss lower-bounds MI; graph diffusion Graffe achieves SOTA","Graffe: diffusion-based graph embeddings with a provable info bound"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0015,"raw_usage":{"total_tokens":6064,"prompt_tokens":1040,"completion_tokens":5024,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":4917}},"tokens_in":656,"tokens_out":5024,"duration_ms":34664,"temperature":1.0,"reasoning_tokens":4917,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:17:35.667206+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-derive Eq. (27) from Eq. (26) symbolically: exponentiating $(d/2)\\log(2\\pi e/d \\cdot \\mathbb{E}[\\mathrm{Tr}]) \\geq h(x_0 \\mid x_t) - I$ gives $\\mathbb{E}[\\mathrm{Tr}] \\geq (d/2\\pi e)\\exp(2h/d)\\exp(-2I/d)$. If the paper's $\\exp(-I)$ is used instead, plug in any concrete joint distribution with known conditional mutual information, such as a Gaussian linear model, and check numerically whether the claimed inequality $I(x_0; E_\\phi(x_0) \\mid x_t) \\geq -\\log L + C$ actually holds; if it fails, Theorem 3 as stated is false.","supporting_citations":[{"cited_title":"Infodiffusion: Representation learning using information maximizing diffusion models,","cited_arxiv_id":null,"evidence_quote":"InfoDiffusion is the prior principled probabilistic diffusion representation learning model that Graffe builds on."},{"cited_title":"Soda: Bottleneck diffusion models for representation learning,","cited_arxiv_id":null,"evidence_quote":"SODA demonstrates conditional diffusion for representation learning and supports the encoder-decoder design choice."},{"cited_title":"Diffusion autoencoders: Toward a meaningful and decodable representa- tion,","cited_arxiv_id":null,"evidence_quote":"Diffusion autoencoders introduce the semantic latent from diffusion models that motivates conditioning the decoder on a learned representation."},{"cited_title":"Graphmae: Self-supervised masked graph autoencoders,","cited_arxiv_id":null,"evidence_quote":"GraphMAE is the leading masked graph autoencoder baseline that Graffe compares against and whose feature-reconstruction strategy it adopts."}],"review_version":1}