{"id":"951041eb-de94-4229-995d-9934ab9ad6ed","arxiv_id":"2501.08850","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Counterfactual graph explanations are generated by gradient descent in the latent space of a permutation-equivariant graph VAE, steering the graph's encoding to the opposite class.","lead":"This paper presents a method for explaining graph-classifier predictions by nudging a graph's latent encoding across the classifier's decision boundary, then decoding it back into a graph. The authors test it on three molecular datasets and report that their method reliably changes the predicted class while keeping the graph relatively similar.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The traversal in Eq. 5 optimizes the classifier on Gumbel-softmax relaxed graphs, while the classifier was trained on discrete graphs; if the soft-input decision boundary misaligns with hard decoding, the reported flip-ratios may not reflect real counterfactual flips.","rationale":"The reader's weakest assumption focused on whether VAE latent-space traversal lands in-distribution, citing the paper's own limitation about boundary points far from training encodings. That is a real concern and is acknowledged in the paper. My concern is more targeted and, I think, more fundamental to the algorithm as stated: the gradient in Eq. 5 is obtained by evaluating a classifier trained on discrete graphs at Gumbel-softmax continuous relaxations of graphs. This is a train/serve distribution mismatch that can break the correspondence between the loss being minimized and the property being measured, even when the VAE latent space is perfectly smooth and semantically meaningful. The paper does not report any check of classifier behavior on soft inputs, nor does it compare soft-decoded and hard-decoded predictions. Because the headline empirical claim is 'consistently high-performing and more robust than the baselines,' and flip-ratio is the primary validity metric, this untested soft/hard alignment is the single most load-bearing gap. The proposed concrete test is easy to run with the released code and would settle the question: if the soft and hard predictions largely agree, my concern is resolved and the method's empirical claims stand on much firmer ground. If they disagree, the reported flip-ratios may be inflated by the relaxation, and the paper would need to either use a straight-through estimator with validation, or re-optimize in the hard domain. This does not change the overall conditional verdict: the idea is plausible and the paper is explicit about its preliminary nature, but the current evidence is insufficient to accept the robustness claim without this check.","tokens_in":12056,"tokens_out":3679,"duration_ms":47291,"concrete_test":"Recompute the Table 1 flip-ratio and SIC using hard-decoded graphs (argmax or sampled discrete graphs) while logging the classifier prediction on the paired Gumbel-softmax relaxation used during optimization. For a random batch of latent codes, measure the agreement rate C(soft_decode(z)) == C(hard_decode(z)). If this agreement is substantially below 100%, or if the hard-decoded flip-ratio drops materially relative to the soft-decoded flip-ratio, then Eq. 5 is chasing a relaxation boundary that is not the true discrete classifier boundary, and the empirical validity claims need to be re-evaluated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central mechanism in Section 2.5 is Eq. 5, where the gradient is computed through yi = (C ∘ D)(zi). Because the decoder outputs a distribution pθ(G | z), the paper substitutes a Gumbel-Softmax relaxation so that backpropagation is possible. However, the classifier C was trained on discrete, one-hot node and edge attributes (Appendix A.1.1), not on the continuous outputs of a Gumbel-Softmax at temperature τ = 1. The relaxed graph is out-of-distribution for C, so its predictions on such inputs are not validated by the reported test-set AUROC (Table A.2). The optimization therefore solves a surrogate problem: minimize the classifier's cross-entropy on soft, relaxed graphs. The final counterfactual, by contrast, must be a discrete graph decoded from pθ(G | z). Nothing in the paper establishes that a point z that flips the classifier under the soft relaxation also flips it when a hard sample or argmax decode is used. A high flip-ratio measured on the final discrete graphs could still arise if the soft-to-hard gap is small, but the paper never measures this gap. This is more specific and more load-bearing than the general VAE-manifold concern: even a perfectly semantically organized latent space would fail if the gradient signal from C is computed on inputs the classifier was never trained on. The paper's own Limitations paragraph acknowledges that decoded counterfactuals can be unreasonable away from latent training points, but it does not address the soft-versus-hard classifier mismatch, which is internal to the proposed algorithm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CGCF, a method for generating counterfactual explanations of graph classifiers by traversing the latent space of a permutation-equivariant variational autoencoder. Given a factual graph, the method encodes it, then iteratively updates the latent code with gradient descent on a cross-entropy loss (with L2 regularization) that pushes the decoded graph toward a desired class, using a Gumbel-Softmax relaxation to differentiate through the discrete decoder. The method is evaluated on three molecular graph datasets (AIDS, Mutagenicity, NCI1) against three latent-space baselines (random prior samples, nearest training graph, decoded mean of k-NN), and the paper claims that CGCF is consistently high-performing and more robust than the baselines, while removing the need to define an explicit graph distance.","tokens_in":12318,"tokens_out":3284,"duration_ms":38199,"significance":"If the result holds, the paper offers a principled and elegant way to generate graph counterfactuals without a hand-defined graph metric, leveraging equivariance to preserve node alignment and a pre-trained generative model to stay near the data manifold. Strengths of the paper include its explicit treatment of permutation equivariance, the algorithmic clarity of Section 2.5, the acknowledgment of limitations, and the promise of public source code. However, the central evaluation is weakened by a circularity between the optimized objective and the headline metric, and the main traversal mechanism relies on an unvalidated assumption that classifier gradients computed on Gumbel-Softmax relaxed graphs align with hard-decoded counterfactual flips. These issues must be addressed before the contribution can be considered established.","major_comments":[{"comment":"The traversal minimizes the classifier's cross-entropy on Gumbel-Softmax relaxed graphs at temperature τ=1, but the classifier was trained on discrete one-hot graphs. The paper never verifies that a latent point that flips the classifier under the soft relaxation also flips it when the decoded graph is obtained by hard sampling or argmax decoding, so the reported flip-ratios may reflect a surrogate problem rather than true counterfactual flips. I request an explicit measurement of the soft-versus-hard flip agreement, and, if the gap is nontrivial, a modification such as temperature annealing or straight-through estimation with a re-evaluation of all reported metrics.","section":"Section 2.5, Eq. (5)-(6); Appendix A.1.1 and A.2"},{"comment":"The primary validity metric, Flip-Ratio, measures exactly the objective being optimized in Eq. (6): the traversal minimizes cross-entropy toward the desired class, and Flip-Ratio records whether that class is achieved. This makes the headline claim partly circular. I recommend adding independent validity measures, such as chemical validity of decoded molecules (e.g., via RDKit), distance of the counterfactual latent code to latent encodings of training graphs, and agreement with a held-out or differently initialized classifier, and reporting Flip-Ratio with confidence intervals.","section":"Section 3.2, Table 1, and Eq. (6)"},{"comment":"The claim that CGCF is \"consistently high-performing and more robust than the baselines\" is not yet supported because all baselines are weak and share the same VAE latent space; no comparison is made with established graph counterfactual methods such as CLEAR, CF-GNNExplainer, or D4Explainer. Additionally, the paper's own Limitations paragraph concedes that decoded counterfactuals can be unreasonable when the latent code is far from training points, but no experiment checks the distance of the obtained counterfactual latent codes to the latent training distribution or otherwise validates the in-distribution assumption. Adding such an analysis and stronger baselines is necessary to substantiate the robustness claim.","section":"Section 3.3, Section 3.4, and Limitations (Section 4)"}],"minor_comments":[{"comment":"The title contains a typo: \"T raversal\" should be \"Traversal\".","section":"Title"},{"comment":"The regularization term is written as λ∥z_i∥; please clarify whether this is an L2 norm or a squared L2 norm, and define the indicator notation 1{D=k}(y_D) more explicitly.","section":"Section 2.5, Eq. (6)"},{"comment":"The stopping condition compares the classifier output y_i to the desired label y_D, but since the classifier outputs probabilities, the threshold used to declare a flip should be specified; also, once the desired label is achieved, the algorithm freezes further updates but still returns the graph from the final iteration, which may differ from the first successful counterfactual.","section":"Algorithm 1"},{"comment":"Flip-Ratio is reported as a single scalar without variance or sample size, unlike the identity-preservation metrics; please provide confidence intervals or standard errors.","section":"Table 1"},{"comment":"The hyperparameters τ, λ, and the number of iterations N are fixed a priori without sensitivity analysis; a brief study of their influence on flip-ratio and identity preservation would strengthen the practical claims.","section":"Appendix A.2"},{"comment":"The claim that the method can produce an unlimited number of counterfactuals is only true if one samples from the decoder likelihood after the traversal; the traversal itself is deterministic given the factual graph and hyperparameters, so please clarify the intended stochasticity.","section":"Discussion, 'arbitrary number of explanations'"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the equivariance framework is well motivated, but the current evaluation does not convincingly demonstrate that the method produces valid counterfactuals: the soft-versus-hard gap is unmeasured and the main metric is the optimized objective itself. I believe the paper can be made acceptable with a focused revision that adds the soft-hard agreement analysis, non-circular validity checks, and a comparison against at least one non-latent graph counterfactual baseline. I do not see a fundamental flaw that would force rejection, but the missing evidence is substantial."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid preliminary paper with a real idea, but the evaluation is not yet strong enough to back the \"robust\" claim. The method (CGCF) is simple: encode a graph with a permutation-equivariant VAE, then take gradient steps in latent space guided by a differentiable classifier, decoding via Gumbel-Softmax. The novelty is modest but real — the specific combination of an equivariant VAE and traversal without a counterfactual training objective appears new; CLEAR trains a counterfactual loss, here they don't.\n\nWhat's good: the writing is clear, the equivariance reasoning is correct, and the authors are upfront about limitations and about the baselines being self-constructed rather than SOTA. They also ship code. The trade-off plots (flip-ratio vs identity preservation) are a reasonable way to look at the results.\n\nNow the soft spots, in order of severity. First, the stress-test concern is legitimate: the classifier was trained on discrete one-hot graphs, but the traversal computes gradients through Gumbel-Softmax relaxed graphs at τ=1. No experiment checks whether a latent vector that flips the classifier under the soft relaxation also flips it when decoded to a hard graph. The flip-ratio numbers are reported on the final decoded graphs, so the gap might be small, but it's unmeasured. That's the load-bearing assumption and it needs a direct test (e.g., hard vs soft accuracy, chemical validity of decoded graphs).\n\nSecond, the flip-ratio metric is close to the optimized objective. Achieving a high flip-ratio is expected; the interesting part is identity preservation and whether the counterfactuals are chemically plausible. The identity-preservation metrics are there, but the baselines are all from the same VAE, so they don't tell you whether CGCF beats an existing graph-counterfactual method like CLEAR. The authors acknowledge this, but the \"robust\" claim in the abstract overreaches.\n\nThird, no error bars across training runs; the ± values are across test samples only. That's a minor point for a preliminary paper.\n\nThe central idea is defensible; none of these issues invalidate it. But the evidence as presented supports \"promising direction,\" not \"consistently high-performing and more robust.\" I'd send it to peer review with a request for stronger baselines and a soft/hard validation experiment. It deserves a serious referee, but the referee should be told to focus on the soft-hard gap.","headline":"A clearly written but explicitly preliminary paper on latent-space traversal for graph counterfactuals; the core idea is plausible, the evaluation is too weak to support the robustness claim.","tokens_in":12954,"tokens_out":2821,"would_cite":false,"duration_ms":28535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Latent-space traversal of a permutation-equivariant graph VAE generates counterfactual explanations for any differentiable graph classifier without defining a graph distance.","keywords":["counterfactual explanations","graph neural networks","variational autoencoder","permutation equivariance","latent space traversal","graph explainability","molecular graphs"],"falsifier":"Compute the chemical validity of the decoded counterfactual graphs and the latent distance from each counterfactual code to the nearest training-set encoding; if a substantial fraction of flips are chemically invalid, or if the walked codes cluster far outside the region occupied by training encodings, the premise that traversal stays in-distribution is false.","tokens_in":11773,"feed_emoji":"🧪","tokens_out":8189,"duration_ms":73165,"temperature":0.7,"pith_summary":"The paper proposes a way to explain a graph classifier's decision by producing counterfactual graphs: alternative inputs that look as close as possible to the original while flipping the predicted class. The method, CGCF, encodes a factual graph with a permutation-equivariant variational autoencoder, then walks the latent code in the direction that most increases the classifier's probability of the desired class, decoding at each step until the label flips. Because the walk happens in a learned latent space, the paper never has to choose a graph distance, which is normally the arbitrary part of counterfactual explanation. On NCI1, Mutagenicity, and AIDS, the classifier-guided walks reached the highest flip-ratio among the compared methods while staying competitive on identity preservation. The authors present the procedure as a general recipe for any differentiable graph classifier, contingent on the latent space actually representing realistic in-distribution graphs.","feed_headline":"Latent-space walks explain graph AI without a graph distance","feed_subtitle":"A classifier-steered walk in a molecule autoencoder's latent space flips predictions consistently on NCI1, Mutagenicity, and AIDS.","key_machinery":"The central object is the permutation-equivariant graph variational autoencoder (PEGVAE), whose latent space is traversed to reach the counterfactual. Equivariant linear layers, built from a fixed set of Bell-number basis elements, make the encoder and decoder commute with node permutations, so a relabeled input produces a relabeled counterfactual and the factual/counterfactual alignment is preserved. The traversal itself is the update $z_{i+1} = z_i - \\epsilon \\nabla \\mathcal{L}(z_i, y_D)$, where $\\mathcal{L}$ is cross-entropy against the desired class $y_D$ plus an $\\ell^2$ penalty on the latent code, and a Gumbel-Softmax approximation to $p_\\theta(G \\mid z)$ makes the decoder differentiable so the classifier's gradient can steer the walk.","core_discovery":"The paper claims that classifier-guided traversal of a permutation-equivariant VAE latent space produces valid, aligned counterfactual graphs for any differentiable graph classifier, without an explicit graph metric and without training the generative model with a counterfactual-specific loss. The walk is steered by gradient descent on the cross-entropy of the decoded graph with respect to the desired class, regularized by an L2 penalty on the latent code, and a Gumbel-Softmax relaxation lets gradients flow through the discrete decoder. In the reported experiments, CGCF is consistently high-performing and more robust than the baselines: it achieves the highest flip-ratio on all three datasets, and in the trade-off between identity preservation and validity it outperforms on NCI1 and Mutagenicity while remaining competitive on AIDS. The authors also note that because the method is probabilistic, an arbitrary number of counterfactual explanations can be sampled for a given factual graph.","pith_inferences":["A testable extension would measure the latent distance from each counterfactual code to the nearest training encoding; if many walked codes land far from the data manifold, the flips may reflect out-of-distribution artifacts rather than meaningful molecular changes.","The same traversal recipe could apply to other structured data with a group action, such as point clouds or meshes, by substituting an appropriately equivariant autoencoder and differentiable decoder.","Because the walk is continuous, the method could be run in reverse from the counterfactual side to trace the decision boundary, yielding per-class minimal perturbations instead of per-instance explanations.","The reliance on a pretrained VAE suggests a practical shortcut: when a new classifier is deployed on the same graph domain, the expensive generative model can be reused and only the short traversal needs to be repeated."],"forward_implications":["A practitioner can generate counterfactual graphs for any differentiable black-box graph classifier without choosing a graph distance; the latent metric replaces that choice.","Because the decoder defines a distribution $p_\\theta(G \\mid z)$, the method can produce arbitrarily many explanations per factual graph by sampling at the found latent code.","The full pipeline is permutation equivariant, so explanations respect node ordering: permuting the input nodes permutes the counterfactual in the same way.","If the classifier is replaced, new counterfactuals can be generated for the new decision boundary without retraining the generative model.","The framework is stated to extend beyond the categorical molecule attributes used in the experiments to continuous node and edge attributes."],"supporting_citations":[{"why":"Supplies the permutation-equivariant graph variational autoencoder architecture whose latent space is traversed.","marker":"[8]"},{"why":"Provides the equivariant linear layer construction, based on basis elements, that makes the encoder and decoder permutation equivariant.","marker":"[39]"},{"why":"Introduces the Gumbel-Softmax reparameterization that lets gradients flow through the discrete decoder during the traversal.","marker":"[44]"},{"why":"Gives the beta-weighted KL regularization used to train the VAE latent space.","marker":"[43]"},{"why":"Supplies the dataset preprocessing and filtering pipeline applied to all three molecular datasets.","marker":"[50]"},{"why":"Supplies the NCI1 molecular dataset used in the experiments.","marker":"[45]"},{"why":"Supplies the Mutagenicity molecular dataset used in the experiments.","marker":"[46]"},{"why":"Supplies the AIDS molecular dataset used in the experiments.","marker":"[47]"}],"fun_headline_variants":["Latent walk flips graph predictions without a distance metric","Traverse VAE latent space to explain any graph classifier","Equivariant VAE latent traversal yields robust graph counterfactuals","Gradient-steered latent walks craft valid graph counterfactuals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a VAE trained only with reconstruction and KL objectives produces a latent space in which gradient steps toward the classifier's decision boundary decode to realistic in-distribution graphs, and the paper's own limitations section acknowledges that a latent counterfactual obtained far from any training point may not be reasonable.","fun_headline_variants_meta":{"raw":{"variants":["Latent walk flips graph predictions without a distance metric","Traverse VAE latent space to explain any graph classifier","Equivariant VAE latent traversal yields robust graph counterfactuals","Gradient-steered latent walks craft valid graph counterfactuals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000337,"raw_usage":{"total_tokens":1866,"prompt_tokens":948,"completion_tokens":918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":848}},"tokens_in":564,"tokens_out":918,"duration_ms":8347,"temperature":1.0,"reasoning_tokens":848,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:16:03.254760+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the chemical validity of the decoded counterfactual graphs and the latent distance from each counterfactual code to the nearest training-set encoding; if a substantial fraction of flips are chemically invalid, or if the walked codes cluster far outside the region occupied by training encodings, the premise that traversal stays in-distribution is false.","supporting_citations":[{"cited_title":"Categorical reparameterization with gumbel- softmax","cited_arxiv_id":null,"evidence_quote":"Introduces the Gumbel-Softmax reparameterization that lets gradients flow through the discrete decoder during the traversal."},{"cited_title":"beta-vae: Learning basic visual concepts with a constrained variational framework","cited_arxiv_id":null,"evidence_quote":"Gives the beta-weighted KL regularization used to train the VAE latent space."},{"cited_title":"Global counterfactual explainer for graph neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset preprocessing and filtering pipeline applied to all three molecular datasets."},{"cited_title":"Comparison of descriptor spaces for chemical compound retrieval and classification","cited_arxiv_id":null,"evidence_quote":"Supplies the NCI1 molecular dataset used in the experiments."},{"cited_title":"Derivation and validation of toxicophores for mutagenicity prediction","cited_arxiv_id":null,"evidence_quote":"Supplies the Mutagenicity molecular dataset used in the experiments."},{"cited_title":"Iam graph database repository for graph based pattern recognition and machine learning","cited_arxiv_id":null,"evidence_quote":"Supplies the AIDS molecular dataset used in the experiments."}],"review_version":1}