Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read The paper claims that learned random walks on a graph hierarchy can beat the theoretical accuracy bound that limits walks on the original topology.

desk verdict A neat, honest idea—combining METIS hierarchy with learned RW policies—but the evidence is a single tiny synthetic dataset and the 'bound violation' is partly due to feature aggregation in virtual nodes. read the letter →

arxiv 2509.01381 v1 pith:PMDLUX65 submitted 2025-09-01 cs.LG

classification cs.LG
keywords graphneuralnetworkslong-rangedependenciesrandomwalkshierarchiesadaptivetransitionsoversquashingPrefixSumtask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that adding a coarsened hierarchy above a graph, and letting a random walk learn when to travel up, down, or across the original edges, changes what can be achieved with short walks. On a synthetic prefix-sum task where any model restricted to the original topology provably cannot exceed accuracy 1/2 + L/(2n) for walk length L, the proposed hierarchical adaptive walker goes beyond that bound. The same accuracy that needs a long walk on the original graph is reached with shorter walks that use the hierarchy as a shortcut. Two ingredients carry the result: the coarsened structure creates virtual nodes that compress distant regions, and the learnable transition policy decides when to jump rather than walk locally. The paper positions this as a first step toward processing very large graphs with sublinear-length walks.

What carries the argument

The central object is the hierarchical graph G_H = (V_H, E_H) built by METIS coarsening with refinement factor 1/2, where every node has exactly one parent and virtual node features are initialized as the mean of descendants. The carrying mechanism is the adaptive transition function q(u,v,e_uv)—an MLP whose softmax over neighbors determines the next step—combined with non-backtracking masking and Gumbel-Softmax sampling, so the walk can learn to prefer upward jumps when long-range transfer is needed and horizontal edges when local detail matters. A bottom-up DeepSet pooling (Eq. 1) pre-trains hierarchical embeddings, and a Mamba sequence model processes each sampled walk.

What would settle it

Run the PrefixSum task with a hierarchy whose virtual-node features are set to random noise instead of the mean of descendants; if the walker still exceeds 1/2 + L/(2n), the topology alone is driving the gain. Conversely, if it drops to the bound, the gain depends on information surviving the coarsening.

Watch

Extended reading notes

Core claim

The central discovery is that a random walk equipped with a learned transition function and access to a METIS-based hierarchy of coarsened graphs can beat the theoretical limit that applies to any method moving only on the original topology. For the PrefixSum benchmark on n=16 line graphs, the expected accuracy of any walk of length L≤n on the original graph is at most 1/2 + L/(2n), because the first bit can only reach L+1 of the n nodes. Both a random walker and a learned walker on the hierarchy exceed this bound, and the learned walker does so with fewer sampled walks and shorter lengths. This shows the coarsening is not just a computational shortcut: it changes the information-theoretical

Load-bearing premise

The load-bearing premise is that the METIS hierarchy, with one parent per node and mean-pooled virtual features, preserves the task-relevant signal (notably the first bit x0) that the walker needs to recover; if coarsening destroys it, the hierarchy only transports noise, and the method would fall back to the bound.

Editorial extensions

If this is right

  • On graphs with a task-aligned hierarchy, walk length can be sublinear in graph diameter while matching the accuracy of linear-length walks on the original topology.
  • The theoretical bound E[A] ≤ 1/2 + L/(2n) is not a fundamental limit of walk-based learning; it is a limit of the original topology alone.
  • Fewer sampled walks per node are needed when transitions are learned, which reduces the inference cost of walk-based graph models.
  • The advantage of the hierarchy appears even for random transitions, but learning the transitions multiplies the gain when the sampling budget is tight.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test it, but if the result transfers to larger graphs, the logarithmic number of METIS levels suggests the same mechanism could yield walks of logarithmic length for diameter-scale dependencies.
  • An ablation that destroys the mean-pooled virtual-node information (e.g., zeroing virtual features) would isolate whether the gain comes from the transport layer alone or from the information carried through it; the paper reports no such ablation.
  • The adaptive-jump idea could be applied to other coarsening schemes or to auxiliary graphs without a clean hierarchy, potentially extending the bound-breaking behavior to settings where METIS coarsening does not align with task structure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a method that combines a METIS-based hierarchical coarsening of an input graph with learnable random walks. A transition policy chooses among original and hierarchical edges, walk sequences are processed by a Mamba model, and the resulting embeddings are aggregated for node-level prediction. The method is evaluated on a synthetic PrefixSum task on line graphs with n=16. The authors report that random walks on the hierarchy exceed the theoretical accuracy bound E[A] <= 1/2 + L/(2n) that holds for models restricted to the original topology, and that learned transitions achieve this with fewer sampled walks than random transitions.

Significance. If the claims are correct, the paper contributes a useful idea: combining coarsened graph hierarchies with learned walk policies to reduce walk length for long-range tasks. The synthetic PrefixSum setup is a good testbed because it admits a quantitative bound for original-topology methods. The paper also makes a clear distinction between learned and random walkers on the same hierarchy, and the figures suggest the learned policy is more sample-efficient. However, the central claim—that hierarchical walks 'exceed the theoretical bound'—is currently overstated because the bound does not apply to the proposed model, which uses additional virtual nodes and aggregated descendant features. The paper also does not provide code, data, error bars, or sufficient hyperparameter details, limiting reproducibility. Credit is due for the clean problem formulation and for explicitly separating learned vs. random transitions in the experiments; these are strengths that make the central idea testable.

major comments (3)
  1. [Section III, Eq. (1)] The virtual-node embeddings are precomputed before any walk as DeepSet summaries of entire descendant subtrees. Thus a walk of length 2 that hops to a coarse node can read a learned summary of all nodes in that subtree. The comparison with Eq. (5), which assumes a walk on the original topology sees only O(L) local nodes, therefore does not isolate the benefit of hierarchical shortcuts. Please add an ablation that removes or degrades the descendant-summary features—for example random virtual-node features, or omitting Eq. (1)—while keeping the same hierarchy and transition policy, and report whether the walker still exceeds the original-topology bound. Without this, the headline claim is confounded by non-local feature aggregation.
  2. [Section IV, Eq. (5)] The bound E[A] <= 1/2 + L/(2n) is derived only for models operating on the original graph topology. The proposed model operates on GH, which contains additional virtual nodes and edges and, crucially, receives aggregated features from all descendants. Exceeding this original-topology bound is therefore not evidence that the hierarchical walk topology itself shortens information propagation; it is largely expected once features are aggregated. The manuscript should either derive a corresponding bound or receptive-field analysis for the hierarchical graph, or explicitly state that the comparison is against original-topology methods and that no bound is claimed for the proposed method. As written, the abstract and Section V imply the proposed method violates a constraint that does not actually apply to it.
  3. [Section V, Figures 2 and 3] The central quantitative claims are based on accuracies averaged over 5 runs, but the figures report no error bars or per-seed variability. Without variance information it is difficult to assess whether the reported differences between learned and random walkers, and the crossing of the bound, are reliable. The experimental section also omits several implementation details needed to reproduce the results: MLP and Mamba dimensions, number of hierarchy levels for n=16, Gumbel-Softmax temperature schedule, and the exact construction of the non-backtracking mask. Please provide code/data or, at minimum, complete hyperparameters and run-level statistics.
minor comments (5)
  1. [Section IV, Eq. (5)] The displayed formula appears garbled: 'E[A] ≤ 1 − (n−L)/n^2' is not algebraically equal to '1/2 + L/(2n)'. The surrounding text also says x0 can be part of the receptive field of (n−L) nodes, while the bound 1/2 + L/(2n) corresponds to L nodes being able to see x0. Please clarify the counting convention (number of visited nodes vs. steps) and fix the formula.
  2. [Section III] The symbol L is used both for the number of hierarchy levels and for the walk length. These are different quantities; consider using K or H for the hierarchy depth.
  3. [Section III, Eq. (2)] The text says Gumbel-Softmax is used for differentiable discrete sampling, but Eq. (2) defines a softmax distribution. Please clarify whether walks are sampled as discrete sequences with a Gumbel-Softmax relaxation or whether the softmax probabilities are used directly as transition probabilities in training.
  4. [Section III, Eq. (4)] The aggregation formula omits the index of the position i within a walk; the sum over s_i should presumably be over all positions of each walk. Please define the summation range explicitly.
  5. [General] The paper is missing a limitations paragraph. Given that the results are on one synthetic line-graph task with n=16, the claim that the approach will be useful for 'very large graphs' is speculative and should be accompanied by a discussion of when METIS coarsening may destroy task-relevant information.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the bound exceeded is explicitly for original-topology models, and the empirical comparison is self-contained.

full rationale

The paper's claimed result—adaptive hierarchical walks exceed the theoretical bound E[A] ≤ 1/2 + L/(2n)—does not reduce to its own inputs by construction. Eq. (5) is stated as a bound 'for any model that operates on the original topology'; the proposed method deliberately operates on an augmented graph G_H with extra hierarchical edges and virtual nodes. Exceeding the bound is therefore a scope change, not a circular derivation. The empirical comparison between learned and random walkers on the same G_H is a genuine experiment. The virtual-node features are precomputed summaries (Eq. 1), which means a single jump can expose aggregate subtree information; this is a potential confound for interpreting the walk-length comparison, but it is a correctness/validity concern rather than a circularity: the accuracy values are not derived from the features by construction, they are learned and tested on held-out data. The only self-citation is [23] (by the first author) for the PrefixSum task and its provable bound. The paper sketches the bound's reasoning and it is independently derivable, so the citation is not load-bearing circularity. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from authors' prior work. Therefore no circular step is present; score 2 reflects the minor self-citation only.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claim is not forced by hand-fitted constants: the model is trained end-to-end. The load-bearing assumptions are external (the PrefixSum bound), the hierarchy construction, and the synthetic-task proxy. The invented entity is the hierarchical graph itself, which is a model component with no independent evidence outside this paper.

free parameters (2)
  • hierarchical refinement factor = 1/2 (chosen, not fitted)
    Set in Section III to halve the number of nodes at each level. It controls the number of hierarchy levels and thus the available shortcuts; the experimental results depend on this hand-chosen value.
  • number of sampled walks k = 10 for training; varied at inference (1, 2, 5, 10)
    Set in Section IV. Smaller k changes the learned-vs-random comparison in Figure 3, so the method's efficiency claim depends on this hyperparameter.
assumptions (3)
  • standard math The PrefixSum bound E[A] <= 1/2 + L/(2n) applies to every model operating on the original graph topology.
    Invoked in Section IV, Eq. 5, attributed to [23]. It is not proved in this paper, and its displayed form has a likely off-by-one error, so the exact baseline is fragile.
  • domain assumption The METIS coarsening plus mean-pooled virtual features preserves enough information for the learned walker to navigate the hierarchy and solve the task.
    Introduced in Section III, 'Hierarchical Graph'. The entire empirical gain depends on this construction; no ablation tests information-destroying coarsenings.
  • domain assumption The synthetic line-graph PrefixSum task at n=16 is a meaningful proxy for long-range dependencies in practical large graphs.
    The only evaluation is Section IV on n=16 line graphs. Section VI generalizes to very large graphs without evidence, making this proxy assumption load-bearing for the paper's broader significance.
invented entities (1)
  • Hierarchical virtual nodes and parent edges (the graph G_H)
    purpose: Provide O(log |V|) shortcuts so short random walks can jump between distant regions of the original graph.
    Introduced in Section III as a constructed model component. There is no external evidence that such a hierarchy exists in the data; it is a modeling invention whose only support is the paper's own experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies." pith.science (2026). https://pith.science/paper/PMDLUX65

@misc{pith2026250901381,
  author       = {Pith},
  title        = {Pith review of: Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PMDLUX65}},
  note         = {Machine review of arXiv:2509.01381}
}
read the original abstract

Message-passing architectures struggle to sufficiently model long-range dependencies in node and graph prediction tasks. We propose a novel approach exploiting hierarchical graph structures and adaptive random walks to address this challenge. Our method introduces learnable transition probabilities that decide whether the walk should prefer the original graph or travel across hierarchical shortcuts. On a synthetic long-range task, we demonstrate that our approach can exceed the theoretical bound that constrains traditional approaches operating solely on the original topology. Specifically, walks that prefer the hierarchy achieve the same performance as longer walks on the original graph. These preliminary findings open a promising direction for efficiently processing large graphs while effectively capturing long-range dependencies.

Figures

Figures reproduced from arXiv: 2509.01381 by the authors.

Figure 1
Figure 1. Starting from the original graph topology, we create a hierarchical [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Restricted to the graph, models cannot go beyond the expected [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. When we decrease the number of sampled walks per node [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages

  1. [1]

    A gentle introduction to deep learning for graphs,

    D. Bacciu, F. Errica, A. Micheli, and M. Podda, “A gentle introduction to deep learning for graphs,” Neural Networks, vol. 129, pp. 203–221, 9 2020

  2. [2]

    The graph neural network model,

    F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009

  3. [3]

    Neural network for graphs: A contextual constructive approach,

    A. Micheli, “Neural network for graphs: A contextual constructive approach,” IEEE Transactions on Neural Networks , vol. 20, no. 3, pp. 498–511, 2009

  4. [4]

    On the bottleneck of graph neural networks and its practical implications,

    U. Alon and E. Yahav, “On the bottleneck of graph neural networks and its practical implications,” in 9th International Conference on Learning Representations (ICLR), 2021

  5. [5]

    Hierarchical graph neural nets can capture long-range interactions,

    L. Ramp ´aˇsek and G. Wolf, “Hierarchical graph neural nets can capture long-range interactions,” in 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP) , 2021, pp. 1–6

  6. [6]

    Hignn: Hierarchical informative graph neural networks for molecular property prediction equipped with feature-wise attention,

    W. Zhu, Y . Zhang, D. Zhao, J. Xu, and L. Wang, “Hignn: Hierarchical informative graph neural networks for molecular property prediction equipped with feature-wise attention,” arXiv, 2022

  7. [7]

    Megraph: capturing long-range interactions by alternating local and hierarchical aggregation on multi-scaled graph hierarchy,

    H. Dong, J. Xu, Y . Yang, R. Zhao, S. Wu, C. Yuan, X. Li, C. J. Maddison, and L. Han, “Megraph: capturing long-range interactions by alternating local and hierarchical aggregation on multi-scaled graph hierarchy,” in Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), 2023

  8. [8]

    Graph neural networks with learnable structural and positional representations,

    V . P. Dwivedi, A. T. Luu, T. Laurent, Y . Bengio, and X. Bresson, “Graph neural networks with learnable structural and positional representations,” in 10th International Conference on Learning Representations , 2022

Show all 23 references
  1. [9]

    Deepwalk: online learning of social representations,

    B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: online learning of social representations,” in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’14. New York, NY , USA: Association for Computing Machinery, 2014, p....

  2. [10]

    node2vec: Scalable feature learning for networks,

    A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’16. New York, NY , USA: Association for Computing Machinery, 2016, p. 855–864. [Onli...

  3. [11]

    Agent-based graph neural networks,

    K. Martinkus, P. A. Papp, B. Schesch, and R. Wattenhofer, “Agent-based graph neural networks,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=8WTAh0tj2jC

  4. [12]

    Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing,

    J. T ¨onshoff, M. Ritzert, H. Wolf, and M. Grohe, “Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing,” Transactions on Machine Learning Research, 2023. [Online]. Available: https://openreview.net/forum?id=vgXnEyeWVY

  5. [13]

    Graph mamba: Towards learning on graphs with state space models,

    A. Behrouz and F. Hashemi, “Graph mamba: Towards learning on graphs with state space models,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ser. KDD ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 119–130. [Onl...

  6. [14]

    Learning long range dependencies on graphs via random walks,

    D. Chen, T. H. Schulz, and K. Borgwardt, “Learning long range dependencies on graphs via random walks,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=kJ5H7oGT2M

  7. [15]

    Revisiting random walks for learning on graphs,

    J. Kim, O. Zaghen, A. Suleymanzade, Y . Ryou, and S. Hong, “Revisiting random walks for learning on graphs,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/forum?id=SG1R2H3fa1

  8. [16]

    Non-convolutional graph neural networks

    Y . Wang and K. Cho, “Non-convolutional graph neural networks.” in Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc., 2024, pp. 32 705–32 730

  9. [17]

    A fast and high quality multilevel scheme for partitioning irregular graphs,

    G. Karypis and V . Kumar, “A fast and high quality multilevel scheme for partitioning irregular graphs,” SIAM Journal on Scientific Computing, vol. 20, no. 1, pp. 359–392, 1998. [Online]. Available: https://doi.org/10.1137/S1064827595287997

  10. [18]

    Next level message- passing with hierarchical support graphs,

    C. V onessen, F. Gr ¨otschla, and R. Wattenhofer, “Next level message- passing with hierarchical support graphs,” 2024. [Online]. Available: https://arxiv.org/abs/2406.15852

  11. [19]

    Graph pooling for graph neural networks: progress, challenges, and opportunities,

    C. Liu, Y . Zhan, J. Wu, C. Li, B. Du, W. Hu, T. Liu, and D. Tao, “Graph pooling for graph neural networks: progress, challenges, and opportunities,” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence , ser. IJCAI ’23, 2023. [Online]....

  12. [20]

    Deep sets,

    M. Zaheer, S. Kottur, S. Ravanbhakhsh, B. P ´oczos, R. Salakhutdinov, and A. J. Smola, “Deep sets,” in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS’17. Red Hook, NY , USA: Curran Associates Inc., 2017, p. 3394–3404

  13. [21]

    Categorical reparame- terization with gumbel-softmax,

    E. Jang, S. Gu, and B. Poole, “Categorical reparame- terization with gumbel-softmax,” in International Conference on Learning Representations , 2017. [Online]. Available: https://openreview.net/forum?id=rkE3y85ee

  14. [22]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” inFirst Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?id=tEYskw1VY2

  15. [23]

    Flood and echo net: Algorithmically aligned gnns that generalize,

    J. Mathys, F. Gr ¨otschla, K. V . Nadimpalli, and R. Wattenhofer, “Flood and echo net: Algorithmically aligned gnns that generalize,” 2024. [Online]. Available: https://arxiv.org/abs/2310.06970

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.