REVIEW 3 major objections 5 minor 23 references
Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies
T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper claims that learned random walks on a graph hierarchy can beat the theoretical accuracy bound that limits walks on the original topology.
desk verdict A neat, honest idea—combining METIS hierarchy with learned RW policies—but the evidence is a single tiny synthetic dataset and the 'bound violation' is partly due to feature aggregation in virtual nodes. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the hierarchical graph G_H = (V_H, E_H) built by METIS coarsening with refinement factor 1/2, where every node has exactly one parent and virtual node features are initialized as the mean of descendants. The carrying mechanism is the adaptive transition function q(u,v,e_uv)—an MLP whose softmax over neighbors determines the next step—combined with non-backtracking masking and Gumbel-Softmax sampling, so the walk can learn to prefer upward jumps when long-range transfer is needed and horizontal edges when local detail matters. A bottom-up DeepSet pooling (Eq. 1) pre-trains hierarchical embeddings, and a Mamba sequence model processes each sampled walk.
What would settle it
Run the PrefixSum task with a hierarchy whose virtual-node features are set to random noise instead of the mean of descendants; if the walker still exceeds 1/2 + L/(2n), the topology alone is driving the gain. Conversely, if it drops to the bound, the gain depends on information surviving the coarsening.
Extended reading notes
Core claim
The central discovery is that a random walk equipped with a learned transition function and access to a METIS-based hierarchy of coarsened graphs can beat the theoretical limit that applies to any method moving only on the original topology. For the PrefixSum benchmark on n=16 line graphs, the expected accuracy of any walk of length L≤n on the original graph is at most 1/2 + L/(2n), because the first bit can only reach L+1 of the n nodes. Both a random walker and a learned walker on the hierarchy exceed this bound, and the learned walker does so with fewer sampled walks and shorter lengths. This shows the coarsening is not just a computational shortcut: it changes the information-theoretical
Load-bearing premise
The load-bearing premise is that the METIS hierarchy, with one parent per node and mean-pooled virtual features, preserves the task-relevant signal (notably the first bit x0) that the walker needs to recover; if coarsening destroys it, the hierarchy only transports noise, and the method would fall back to the bound.
Editorial extensions
If this is right
- On graphs with a task-aligned hierarchy, walk length can be sublinear in graph diameter while matching the accuracy of linear-length walks on the original topology.
- The theoretical bound E[A] ≤ 1/2 + L/(2n) is not a fundamental limit of walk-based learning; it is a limit of the original topology alone.
- Fewer sampled walks per node are needed when transitions are learned, which reduces the inference cost of walk-based graph models.
- The advantage of the hierarchy appears even for random transitions, but learning the transitions multiplies the gain when the sampling budget is tight.
Reading between the lines
- The paper does not test it, but if the result transfers to larger graphs, the logarithmic number of METIS levels suggests the same mechanism could yield walks of logarithmic length for diameter-scale dependencies.
- An ablation that destroys the mean-pooled virtual-node information (e.g., zeroing virtual features) would isolate whether the gain comes from the transport layer alone or from the information carried through it; the paper reports no such ablation.
- The adaptive-jump idea could be applied to other coarsening schemes or to auxiliary graphs without a clean hierarchy, potentially extending the bound-breaking behavior to settings where METIS coarsening does not align with task structure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method that combines a METIS-based hierarchical coarsening of an input graph with learnable random walks. A transition policy chooses among original and hierarchical edges, walk sequences are processed by a Mamba model, and the resulting embeddings are aggregated for node-level prediction. The method is evaluated on a synthetic PrefixSum task on line graphs with n=16. The authors report that random walks on the hierarchy exceed the theoretical accuracy bound E[A] <= 1/2 + L/(2n) that holds for models restricted to the original topology, and that learned transitions achieve this with fewer sampled walks than random transitions.
Significance. If the claims are correct, the paper contributes a useful idea: combining coarsened graph hierarchies with learned walk policies to reduce walk length for long-range tasks. The synthetic PrefixSum setup is a good testbed because it admits a quantitative bound for original-topology methods. The paper also makes a clear distinction between learned and random walkers on the same hierarchy, and the figures suggest the learned policy is more sample-efficient. However, the central claim—that hierarchical walks 'exceed the theoretical bound'—is currently overstated because the bound does not apply to the proposed model, which uses additional virtual nodes and aggregated descendant features. The paper also does not provide code, data, error bars, or sufficient hyperparameter details, limiting reproducibility. Credit is due for the clean problem formulation and for explicitly separating learned vs. random transitions in the experiments; these are strengths that make the central idea testable.
major comments (3)
- [Section III, Eq. (1)] The virtual-node embeddings are precomputed before any walk as DeepSet summaries of entire descendant subtrees. Thus a walk of length 2 that hops to a coarse node can read a learned summary of all nodes in that subtree. The comparison with Eq. (5), which assumes a walk on the original topology sees only O(L) local nodes, therefore does not isolate the benefit of hierarchical shortcuts. Please add an ablation that removes or degrades the descendant-summary features—for example random virtual-node features, or omitting Eq. (1)—while keeping the same hierarchy and transition policy, and report whether the walker still exceeds the original-topology bound. Without this, the headline claim is confounded by non-local feature aggregation.
- [Section IV, Eq. (5)] The bound E[A] <= 1/2 + L/(2n) is derived only for models operating on the original graph topology. The proposed model operates on GH, which contains additional virtual nodes and edges and, crucially, receives aggregated features from all descendants. Exceeding this original-topology bound is therefore not evidence that the hierarchical walk topology itself shortens information propagation; it is largely expected once features are aggregated. The manuscript should either derive a corresponding bound or receptive-field analysis for the hierarchical graph, or explicitly state that the comparison is against original-topology methods and that no bound is claimed for the proposed method. As written, the abstract and Section V imply the proposed method violates a constraint that does not actually apply to it.
- [Section V, Figures 2 and 3] The central quantitative claims are based on accuracies averaged over 5 runs, but the figures report no error bars or per-seed variability. Without variance information it is difficult to assess whether the reported differences between learned and random walkers, and the crossing of the bound, are reliable. The experimental section also omits several implementation details needed to reproduce the results: MLP and Mamba dimensions, number of hierarchy levels for n=16, Gumbel-Softmax temperature schedule, and the exact construction of the non-backtracking mask. Please provide code/data or, at minimum, complete hyperparameters and run-level statistics.
minor comments (5)
- [Section IV, Eq. (5)] The displayed formula appears garbled: 'E[A] ≤ 1 − (n−L)/n^2' is not algebraically equal to '1/2 + L/(2n)'. The surrounding text also says x0 can be part of the receptive field of (n−L) nodes, while the bound 1/2 + L/(2n) corresponds to L nodes being able to see x0. Please clarify the counting convention (number of visited nodes vs. steps) and fix the formula.
- [Section III] The symbol L is used both for the number of hierarchy levels and for the walk length. These are different quantities; consider using K or H for the hierarchy depth.
- [Section III, Eq. (2)] The text says Gumbel-Softmax is used for differentiable discrete sampling, but Eq. (2) defines a softmax distribution. Please clarify whether walks are sampled as discrete sequences with a Gumbel-Softmax relaxation or whether the softmax probabilities are used directly as transition probabilities in training.
- [Section III, Eq. (4)] The aggregation formula omits the index of the position i within a walk; the sum over s_i should presumably be over all positions of each walk. Please define the summation range explicitly.
- [General] The paper is missing a limitations paragraph. Given that the results are on one synthetic line-graph task with n=16, the claim that the approach will be useful for 'very large graphs' is speculative and should be accompanied by a discussion of when METIS coarsening may destroy task-relevant information.
Circularity Check
No circular derivation; the bound exceeded is explicitly for original-topology models, and the empirical comparison is self-contained.
full rationale
The paper's claimed result—adaptive hierarchical walks exceed the theoretical bound E[A] ≤ 1/2 + L/(2n)—does not reduce to its own inputs by construction. Eq. (5) is stated as a bound 'for any model that operates on the original topology'; the proposed method deliberately operates on an augmented graph G_H with extra hierarchical edges and virtual nodes. Exceeding the bound is therefore a scope change, not a circular derivation. The empirical comparison between learned and random walkers on the same G_H is a genuine experiment. The virtual-node features are precomputed summaries (Eq. 1), which means a single jump can expose aggregate subtree information; this is a potential confound for interpreting the walk-length comparison, but it is a correctness/validity concern rather than a circularity: the accuracy values are not derived from the features by construction, they are learned and tested on held-out data. The only self-citation is [23] (by the first author) for the PrefixSum task and its provable bound. The paper sketches the bound's reasoning and it is independently derivable, so the citation is not load-bearing circularity. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from authors' prior work. Therefore no circular step is present; score 2 reflects the minor self-citation only.
Assumptions & free parameters
free parameters (2)
- hierarchical refinement factor =
1/2 (chosen, not fitted)
- number of sampled walks k =
10 for training; varied at inference (1, 2, 5, 10)
assumptions (3)
- standard math The PrefixSum bound E[A] <= 1/2 + L/(2n) applies to every model operating on the original graph topology.
- domain assumption The METIS coarsening plus mean-pooled virtual features preserves enough information for the learned walker to navigate the hierarchy and solve the task.
- domain assumption The synthetic line-graph PrefixSum task at n=16 is a meaningful proxy for long-range dependencies in practical large graphs.
invented entities (1)
-
Hierarchical virtual nodes and parent edges (the graph G_H)
Cite this review
Pith. "Pith review of Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies." pith.science (2026). https://pith.science/paper/PMDLUX65
@misc{pith2026250901381,
author = {Pith},
title = {Pith review of: Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies},
year = {2026},
howpublished = {\url{https://pith.science/paper/PMDLUX65}},
note = {Machine review of arXiv:2509.01381}
}
read the original abstract
Message-passing architectures struggle to sufficiently model long-range dependencies in node and graph prediction tasks. We propose a novel approach exploiting hierarchical graph structures and adaptive random walks to address this challenge. Our method introduces learnable transition probabilities that decide whether the walk should prefer the original graph or travel across hierarchical shortcuts. On a synthetic long-range task, we demonstrate that our approach can exceed the theoretical bound that constrains traditional approaches operating solely on the original topology. Specifically, walks that prefer the hierarchy achieve the same performance as longer walks on the original graph. These preliminary findings open a promising direction for efficiently processing large graphs while effectively capturing long-range dependencies.
Figures
Reference graph
Works this paper leans on
-
[1]
A gentle introduction to deep learning for graphs,
D. Bacciu, F. Errica, A. Micheli, and M. Podda, “A gentle introduction to deep learning for graphs,” Neural Networks, vol. 129, pp. 203–221, 9 2020
work page 2020
-
[2]
The graph neural network model,
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009
2009
-
[3]
Neural network for graphs: A contextual constructive approach,
A. Micheli, “Neural network for graphs: A contextual constructive approach,” IEEE Transactions on Neural Networks , vol. 20, no. 3, pp. 498–511, 2009
work page 2009
-
[4]
On the bottleneck of graph neural networks and its practical implications,
U. Alon and E. Yahav, “On the bottleneck of graph neural networks and its practical implications,” in 9th International Conference on Learning Representations (ICLR), 2021
work page 2021
-
[5]
Hierarchical graph neural nets can capture long-range interactions,
L. Ramp ´aˇsek and G. Wolf, “Hierarchical graph neural nets can capture long-range interactions,” in 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP) , 2021, pp. 1–6
work page 2021
-
[6]
W. Zhu, Y . Zhang, D. Zhao, J. Xu, and L. Wang, “Hignn: Hierarchical informative graph neural networks for molecular property prediction equipped with feature-wise attention,” arXiv, 2022
work page 2022
-
[7]
H. Dong, J. Xu, Y . Yang, R. Zhao, S. Wu, C. Yuan, X. Li, C. J. Maddison, and L. Han, “Megraph: capturing long-range interactions by alternating local and hierarchical aggregation on multi-scaled graph hierarchy,” in Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), 2023
work page 2023
-
[8]
Graph neural networks with learnable structural and positional representations,
V . P. Dwivedi, A. T. Luu, T. Laurent, Y . Bengio, and X. Bresson, “Graph neural networks with learnable structural and positional representations,” in 10th International Conference on Learning Representations , 2022
work page 2022
Show all 23 references
-
[9]
Deepwalk: online learning of social representations,
B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: online learning of social representations,” in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’14. New York, NY , USA: Association for Computing Machinery, 2014, p....
2014
-
[10]
node2vec: Scalable feature learning for networks,
A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’16. New York, NY , USA: Association for Computing Machinery, 2016, p. 855–864. [Onli...
2016
-
[11]
Agent-based graph neural networks,
K. Martinkus, P. A. Papp, B. Schesch, and R. Wattenhofer, “Agent-based graph neural networks,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=8WTAh0tj2jC
2023
-
[12]
Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing,
J. T ¨onshoff, M. Ritzert, H. Wolf, and M. Grohe, “Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing,” Transactions on Machine Learning Research, 2023. [Online]. Available: https://openreview.net/forum?id=vgXnEyeWVY
2023
-
[13]
Graph mamba: Towards learning on graphs with state space models,
A. Behrouz and F. Hashemi, “Graph mamba: Towards learning on graphs with state space models,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ser. KDD ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 119–130. [Onl...
2024
-
[14]
Learning long range dependencies on graphs via random walks,
D. Chen, T. H. Schulz, and K. Borgwardt, “Learning long range dependencies on graphs via random walks,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=kJ5H7oGT2M
2025
-
[15]
Revisiting random walks for learning on graphs,
J. Kim, O. Zaghen, A. Suleymanzade, Y . Ryou, and S. Hong, “Revisiting random walks for learning on graphs,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=SG1R2H3fa1
2025
-
[16]
Non-convolutional graph neural networks
Y . Wang and K. Cho, “Non-convolutional graph neural networks.” in Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc., 2024, pp. 32 705–32 730
2024
-
[17]
A fast and high quality multilevel scheme for partitioning irregular graphs,
G. Karypis and V . Kumar, “A fast and high quality multilevel scheme for partitioning irregular graphs,” SIAM Journal on Scientific Computing, vol. 20, no. 1, pp. 359–392, 1998. [Online]. Available: https://doi.org/10.1137/S1064827595287997
1998 doi
-
[18]
Next level message- passing with hierarchical support graphs,
C. V onessen, F. Gr ¨otschla, and R. Wattenhofer, “Next level message- passing with hierarchical support graphs,” 2024. [Online]. Available: https://arxiv.org/abs/2406.15852
2024 arXiv
-
[19]
Graph pooling for graph neural networks: progress, challenges, and opportunities,
C. Liu, Y . Zhan, J. Wu, C. Li, B. Du, W. Hu, T. Liu, and D. Tao, “Graph pooling for graph neural networks: progress, challenges, and opportunities,” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence , ser. IJCAI ’23, 2023. [Online]....
2023 doi
-
[20]
Deep sets,
M. Zaheer, S. Kottur, S. Ravanbhakhsh, B. P ´oczos, R. Salakhutdinov, and A. J. Smola, “Deep sets,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS’17. Red Hook, NY , USA: Curran Associates Inc., 2017, p. 3394–3404
2017
-
[21]
Categorical reparame- terization with gumbel-softmax,
E. Jang, S. Gu, and B. Poole, “Categorical reparame- terization with gumbel-softmax,” in International Conference on Learning Representations , 2017. [Online]. Available: https://openreview.net/forum?id=rkE3y85ee
2017
-
[22]
Mamba: Linear-time sequence modeling with selective state spaces,
A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” inFirst Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?id=tEYskw1VY2
2024
-
[23]
Flood and echo net: Algorithmically aligned gnns that generalize,
J. Mathys, F. Gr ¨otschla, K. V . Nadimpalli, and R. Wattenhofer, “Flood and echo net: Algorithmically aligned gnns that generalize,” 2024. [Online]. Available: https://arxiv.org/abs/2310.06970
2024 arXiv
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.