{"id":"9ed57a1e-837f-46b5-8bf7-cff46a27d869","arxiv_id":"2506.12427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Edge-case graphs show that permutation-invariant and cyclic-invariant quantum neural networks do not learn a simple edge-counting surrogate for graph connectedness.","lead":"This paper tests whether quantum neural networks trained to classify graphs as connected or disconnected are secretly just counting edges. By hand-picking unusual graphs, the authors show that the symmetric quantum networks do not rely on this simple surrogate, though the evidence is based on a small set of examples.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The cyclic-invariant ansatz's sole counterexample to edge-counting rests on a single near-boundary output (-4e-4) that the paper's own 0.01 cutoff would call undecided; without per-seed uncertainty the central claim is not established for that circuit.","rationale":"The reader's weakest_assumption is that the edge-case graphs might have appeared during training, making the results memorization rather than generalization. That is a legitimate concern, but it is not the most load-bearing one. Even if an edge-case graph appeared in training, a network that classifies it correctly would still refute a pure edge-counting surrogate at that input; the memorization issue mainly weakens the broader generalization story. The more direct blocker is that the cyclic-invariant ansatz's only decisive counterexample is a single near-boundary reading of -4e-4. The paper itself introduces a 0.01 cutoff for meaningful predictions in Section IV(b), and under that cutoff the value is not a meaningful negative prediction. Table II nevertheless reports it as correct. Since Figure 2 shows that the authors are able to report 3-sigma deviations over 10 runs, the absence of such statistics for the edge-case table is an omission, not a fundamental impossibility. A simple retraining-and-resampling check would settle whether the cyclic circuit reliably produces a negative sign on graph 1. If it does not, the central claim should be explicitly restricted to the permutation-invariant ansatz. The paper remains a useful short empirical study, and the permutation-invariant evidence is reasonably strong, so conditional acceptance is appropriate; no verdict change is needed, but the condition should be sharpened to include repeated edge-case measurements.","tokens_in":5529,"tokens_out":8753,"duration_ms":118802,"concrete_test":"Retrain each ansatz with, say, 10 fresh seeds using the paper's protocol; for all seven edge-case graphs, record the mean and standard deviation of the raw measurement. Define a robust correct classification as the mean having the correct sign and |mean| > 0.01, or >2 standard errors, whichever is stricter. Also save every graph generated during the 50 training epochs and check whether any of the seven edge-case graphs, especially graph 1, ever occurs, to quantify the memorization risk. Report the resulting table. If the cyclic-invariant graph-1 mean is not robustly negative, amend the conclusion to apply only to the permutation-invariant ansatz, or add a confidence threshold to the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim covers both symmetric ansatze. The decisive refutation of edge-counting is graph 1 (a complete subgraph of 7 nodes plus one isolated node: high edge count, disconnected). The permutation-invariant circuit gives -0.4, which is far from the decision boundary. The cyclic-invariant circuit gives -4e-4, i.e., essentially at zero. Section IV(b) explicitly introduces an arbitrary cutoff of 0.01 for a meaningful prediction; under that criterion the cyclic output is undecided, yet Table II marks it as a correct disconnected classification. This is an internal inconsistency in the evidence for the cyclic ansatz. Moreover, Table II reports a single scalar per graph and circuit, with no indication of whether it comes from one trained model, the average over the 10 runs shown in Figure 2, or something else, and no variance. Because variational training is stochastic, the sign of a -4e-4 expectation is plausibly seed-dependent. If the sign flips across runs, the cyclic-invariant circuit no longer provides the counterexample needed for the no-edge-counting claim. The reader's memorization concern is valid but secondary: even a training-set counterexample would refute a pure edge-counting function, though it would weaken the generalization narrative. The robustness issue is what currently blocks the claim for one of the two architectures.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies quantum neural networks (QNNs) trained to classify 8-node graphs as connected or disconnected, comparing a permutation-invariant ansatz, a cyclic-invariant ansatz, and a standard strongly entangling ansatz. After training, the authors test the trained circuits on a small set of hand-designed 'edge case' graphs, such as a complete graph on seven nodes plus one isolated node, and graphs with identical edge counts but different connectivity. The central claim is that the symmetric QNNs do not simply learn an edge-counting surrogate for connectedness, because their predictions on these edge cases are inconsistent with any edge-counting rule. The paper reports numeric outcomes for each graph and circuit in Table II, but no per-run statistics or error bars, and it introduces an arbitrary 0.01 cutoff in Section IV(b) for what counts as a decisive prediction.","tokens_in":5793,"tokens_out":2533,"duration_ms":33397,"significance":"The approach of using structurally extreme graphs to falsify a hypothesized surrogate model is a valuable and relatively inexpensive diagnostic for quantum machine learning models. If the edge-case results are robust, the permutation-invariant circuit indeed provides a clean counterexample to edge-counting: graph 1 has high edge count yet is disconnected, and the circuit outputs -0.4, far from the decision boundary. This is a concrete, falsifiable statement that goes beyond aggregate validation accuracy. However, the paper's central claim is made for both symmetric ansatze, and the cyclic-invariant circuit's sole counterexample rests on a single near-zero output (-4e-4) that the paper's own cutoff would call undecided. The lack of any per-seed variance or description of how the tabulated numbers were obtained makes that part of the claim currently unsubstantiated. Methodologically, the paper also asserts, but does not verify, that the edge-case graphs are unlikely in the training data; this matters for the generalization interpretation.","major_comments":[{"comment":"The cyclic-invariant circuit's output for graph 1 is -4e-4, which by the paper's own criterion in §IV(b) (an arbitrary cutoff of 0.01 for a meaningful prediction) is 'undecided', not a correct classification. Yet Table II marks this entry as correct, and the text in §IV(a) states that the first graph refutes the edge-counting hypothesis 'from the single graph'. For the cyclic ansatz the evidence is therefore internally inconsistent: the only counterexample is a value that the paper itself would treat as statistically indistinguishable from zero. Either the cutoff must be abandoned or the cyclic claim must be restricted; as written, the paper's conclusion that both symmetric circuits refute edge-counting is not supported.","section":"§IV(b), Table II"},{"comment":"The table reports a single scalar per graph and circuit with no indication of whether each value comes from one trained model, the average over the ten simulation runs shown in Figure 2, or something else, and no variance or confidence interval. Quantum variational training is stochastic, so the sign of a prediction with magnitude 1e-4 or 3e-3 is plausibly seed-dependent. If the cyclic-invariant circuit's graph-1 output flips sign across runs, the only edge-counting counterexample for that ansatz disappears. The paper must report per-seed results or at least the distribution over the ten runs (e.g., mean and standard deviation) for each entry in Table II, and state whether the success criterion is the sign of the mean or per-run majority.","section":"Table II and §III(d)"},{"comment":"The abstract asserts that the selected edge-case graphs are 'unlikely to occur in the training data', but the paper never verifies whether any of the seven test graphs (or their isomorphic copies) appeared in the 100-example training epochs across the ten runs. This is load-bearing for the generalization narrative: if a trained network encountered the exact graph during training, its correct prediction could be memorization rather than a refutation of the surrogate model. The authors should check their training data (or the random generation process) and report whether any of the test graphs are provably absent from all training epochs, or else soften the claim to 'edge-counting is refuted as a functional rule' rather than 'the network understands connectedness'.","section":"Abstract and §III(b)"},{"comment":"The conclusion states that 'the method allows to refute some of the hypotheses', which is too strong given the cyclic-ansatz issue. Even for the permutation-invariant ansatz, the refutation currently hinges on a handful of single-run outputs with no statistical backing. The discussion should be rephrased to acknowledge that the edge-case test is a necessary sanity check, not a proof of a learned semantic property, and that the evidence for the cyclic-invariant circuit is inconclusive pending the robustness analysis requested above.","section":"§V, Discussion"}],"minor_comments":[{"comment":"Typo: 'lesser extend' should be 'lesser extent'.","section":"§III(d)"},{"comment":"The abbreviations 'perm-inv', 'cyc-inv', and 'strongly-e' are used without being defined in the caption or the text; please expand the first occurrence or add a note to the caption.","section":"Table II"},{"comment":"The phrase 'some form of confidence, on an arbitrary non-linear scale' is vague; it would help to state explicitly how the measurement expectation value is supposed to map to confidence and why the cutoff is chosen as 0.01.","section":"§IV(b)"},{"comment":"The paper references [9], [10], [11] for circuit constructions, but does not give the parameter count or the exact single-layer structure in text; a short equation or a sentence summarizing the layer structure (beyond Table I) would make the paper more self-contained.","section":"§III(c)"},{"comment":"Reference [4] lacks page numbers and journal issue details, making it harder to locate; the other references appear complete.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short contribution that would be suitable for a workshop or a letters venue if the statistical robustness of the edge-case table is established. The main concern for the editor is that the cyclic-invariant ansatz's key counterexample is a near-zero output that the authors themselves would classify as undecided under their own cutoff; this must be resolved before the claimed refutation of edge-counting can be credibly extended to that architecture. The paper also leans heavily on the authors' previous work for circuit definitions, which is acceptable in a series but limits self-containedness. I did not find evidence of deliberate circularity; the edge-case measurements appear to be new and independent of fitted parameters."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a short empirical paper that does one thing well. It asks whether quantum neural networks trained on 8-node graph connectivity are secretly just counting edges. For the permutation-invariant circuit, the answer is clearly no: on a graph with a complete K7 plus an isolated node, it outputs -0.4 (strongly \"disconnected\"), which no edge-counting model would do. The two same-edge-count trees in graphs 6 and 7 also get opposite predictions, which is another clean counterexample for both symmetric circuits. That refutation is the core contribution, and it holds up logically.\n\nThe edge-case testing method itself is simple and transferable. The paper is also honest about its own limits: it labels the 0.01 cutoff as arbitrary, and it explicitly says the strongly entangling ansatz results are effectively random and shouldn't be read into. That's good practice.\n\nThe soft spots are real but fixable. The biggest issue is that Table II gives one number per graph and circuit, with no error bars and no statement about whether it's a single run, an average, or a median of the 10 runs in Figure 2. The cyclic-invariant result for graph 1 is -4e-4, which is inside the paper's own 0.01 \"undecided\" zone, yet it's marked as a correct classification. The paper's overall claim for the cyclic circuit doesn't actually depend on that one number—graphs 6 and 7 provide a better refutation—but without per-seed variance we can't tell whether those signs are stable either. The abstract says the test graphs are \"unlikely to occur in the training data\" but no check is shown; that's a quick thing to verify and should be added.\n\nWho is this for? People working on interpretability of quantum machine learning, especially anyone concerned about surrogate learning in graph tasks. It's not a theoretical breakthrough, but it's a legitimate empirical diagnostic that deserves a serious referee.\n\nRecommendation: send it to peer review as a short contribution. The central refutation is likely correct; the paper just needs to add per-run statistics, fix the cutoff inconsistency for graph 1, and check the training data.","headline":"A slim but useful empirical probe that refutes edge-counting for a permutation-invariant QNN; the cyclic-invariant case is under-supported because the key table lacks error bars and one near-zero result contradicts the paper's own cutoff.","tokens_in":6312,"tokens_out":4418,"would_cite":false,"duration_ms":50086,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Edge-case graphs show quantum nets don't just count edges","keywords":["quantum machine learning","quantum neural networks","graph connectedness","permutation-invariant circuits","surrogate models","edge-case testing","random graphs","symmetry in quantum circuits"],"falsifier":"Re-run the reported training for the permutation-invariant and cyclic-invariant circuits while recording every graph (up to isomorphism) that appears in each of the 100-example epochs; if any edge-case graph from the results table occurs in training, the correct classifications of that graph can be explained by memorization, and the claim that edge-counting is refuted would lose its support.","tokens_in":5359,"feed_emoji":"🧩","tokens_out":5608,"duration_ms":65081,"temperature":0.7,"pith_summary":"The paper tests whether quantum neural networks trained to classify graph connectedness merely approximate a simpler surrogate: counting the number of edges. It constructs eight-node graphs that are unlikely to appear in the random training data and whose connectedness contradicts edge counts, then inspects how three circuits with different internal symmetries classify them. The permutation-invariant and cyclic-invariant circuits classify the first counterexample as disconnected, refuting the edge-counting hypothesis, while a deeper tree is classified opposite to a star with the same edge count. The paper's wider point is that accuracy on random graphs can hide surrogate learning, and that hand-picked edge cases make such hidden behavior visible in quantum machine learning.","feed_headline":"Edge-case graphs show quantum nets don't just count edges","feed_subtitle":"Symmetry-adapted quantum circuits classify connectedness on rare graphs that would fool a simple edge-counting model.","key_machinery":"The analysis rests on a graph encoding in which each of eight nodes is a qubit, edges are drawn as CZ gates (abelian and self-inverse, like unweighted edges), and Hadamard gates sandwich the edge layer; the resulting state is measured by a permutation-invariant observable whose sign gives the label. Against this encoding, three circuits are compared: a permutation-invariant layer, a cyclic-invariant layer, and a generic strongly entangling layer, each tuned to roughly 120 parameters. The load-bearing test objects are the seven edge-case graphs, chosen to be very unlikely under the random graph model while being extreme for candidate surrogate rules.","core_discovery":"The central claim is that a permutation-invariant quantum neural network, and to a weaker degree a cyclic-invariant one, does not learn a simple edge-counting surrogate for graph connectedness. The evidence is a table of seven edge-case graphs: a complete graph on seven nodes plus an isolated node has many edges yet is classified as disconnected, and a depth-two tree is classified opposite to a star graph with the same number of edges. The failures on other edge cases show the decision boundary required by the continuous Hilbert-space embedding is not uniformly sharp, so what the networks have learned is more structured than edge counting but still imperfect.","pith_inferences":["One extension would be to train the same circuits with edge-counting counterexamples deliberately included in the training set, which would help separate memorization from genuine rule acquisition.","The same edge-case methodology could be pointed at other candidate surrogates, such as the number of connected components, maximum degree, or the size of the largest clique, by constructing graphs where those counts disagree with the true label.","Because the strongly entangling circuit fails to converge, the comparison is really between symmetry-inductive biases; an untrained or randomly initialized version of that circuit would be the right baseline to quantify how much of the difference comes from convergence rather than symmetry."],"forward_implications":["If the claimed refutation holds, high validation accuracy on random graphs cannot be taken as evidence that a quantum network has learned the true graph property.","Symmetry-aligned circuits (permutation-invariant, and to a lesser extent cyclic-invariant) are the ones whose edge-case behavior matches a non-edge-counting rule; the standard strongly entangling circuit's classifications are effectively random because it fails to converge.","Edge-case testing becomes a general diagnostic: before trusting a quantum or classical model on structured inputs, one should probe it with low-probability graphs that separate candidate surrogate rules.","The observed wrong classifications of near-threshold graphs indicate that the continuous embedding places similar graphs close together, so deployment would require either sharper decision boundaries or a different encoding."],"supporting_citations":[{"why":"Established the probabilistic threshold between edge count and connectedness in random graphs, which motivates the surrogate hypothesis that the paper refutes.","marker":"[4]"},{"why":"Defines the notion of surrogate models and surrogate learning that the paper uses to frame the edge-counting hypothesis.","marker":"[2]"},{"why":"Reports the earlier permutation-invariant quantum machine learning results on graph classification that this paper analyzes further.","marker":"[11]"},{"why":"Supplies the construction of permutation-invariant quantum circuits used as one of the three tested networks.","marker":"[9]"},{"why":"Supplies the scaling and symmetry-restriction analysis behind the cyclic-invariant circuit construction.","marker":"[10]"},{"why":"Provides the standard strongly entangling circuit used as the symmetry-free baseline in the comparison.","marker":"[1]"}],"fun_headline_variants":["Quantum nets see beyond edge counts in graph tests","Rare graphs disprove edge-counting in quantum nets","Quantum nets outsmart edge-counting on special graphs","Quantum nets don't just count edges on rare graphs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's core refutation assumes that the seven edge-case graphs were never seen during training, but it does not verify their absence from the 100-example epochs, so memorization would weaken the conclusion that the networks generalize rather than count edges.","fun_headline_variants_meta":{"raw":{"variants":["Quantum nets see beyond edge counts in graph tests","Rare graphs disprove edge-counting in quantum nets","Quantum nets outsmart edge-counting on special graphs","Quantum nets don't just count edges on rare graphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00083,"raw_usage":{"total_tokens":3518,"prompt_tokens":729,"completion_tokens":2789,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":345,"completion_tokens_details":{"reasoning_tokens":2726}},"tokens_in":345,"tokens_out":2789,"duration_ms":27759,"temperature":1.0,"reasoning_tokens":2726,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:50:46.609353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the reported training for the permutation-invariant and cyclic-invariant circuits while recording every graph (up to isomorphism) that appears in each of the 100-example epochs; if any edge-case graph from the results table occurs in training, the correct classifications of that graph can be explained by memorization, and the claim that edge-counting is refuted would lose its support.","supporting_citations":[{"cited_title":"On random graphs. I","cited_arxiv_id":null,"evidence_quote":"Established the probabilistic threshold between edge count and connectedness in random graphs, which motivates the surrogate hypothesis that the paper refutes."},{"cited_title":"Solving graph problems using permutation-invariant quantum machine learning","cited_arxiv_id":"2505.12764","evidence_quote":"Reports the earlier permutation-invariant quantum machine learning results on graph classification that this paper analyzes further."}],"review_version":1}