REVIEW 3 major objections 5 minor 38 references
WILTing Trees: Interpreting the Distance Between MPNN Embeddings
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Trained MPNNs position graphs by task-relevant functional distance, not structural similarity, and a distilled Weisfeiler-Leman tree reveals the few subgraphs that set this geometry.
desk verdict Solid theoretical core and a genuinely new distance, but the interpretability claims rest on an in-sample fit that is never tested on held-out graphs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Weisfeiler-Leman Labeling Tree (WILT): a rooted tree whose non-root vertices are the WL colors that appear during the WL test on the dataset, with the root connected to the initial colors and edges linking a color to the color of the same node in the previous iteration; injectivity of WL hashing makes this a tree. With non-negative edge weights $w$, the ground metric is shortest-path length, and the WILTing distance is the optimal transport distance between the color multisets of two graphs. Proposition 5.3 reduces it to the weighted Manhattan distance $\sum_{c \in V(T_D)\setminus\{r\}} w(e_{\{c,p(c)\}}) |\nu^G_c - \nu^H_c|$ on WILT embeddings $\nu^G$, which is what makes it linear-time computable and gives each edge weight a direct reading: a color with a large weight contributes proportionally to the distance whenever its abundance differs between two graphs. Two normalizations adapt the distance to unequal graph sizes, namely size normalization ($\dot{\nu}^G = \nu^G/|V_G|$), which generalizes $d_{\mathrm{WWL}}$, and dummy-node normalization ($\bar{\nu}$), which generalizes $d_{\mathrm{WLOA}}$, and the expressiveness theorem (Theorem 5.4) links the former to mean-pooled and the latter to sum-pooled MPNNs. The learning procedure (Algorithm 1) fits the edge weights by minimizing the squared error between $d_{\mathrm{WILT}}$ and $d_{\mathrm{MPNN}}$ over graph pairs, after which the largest weights identify the WL colors whose presence or absence most affects the MPNN distance.
What would settle it
Fit WILT edge weights on a training split of a dataset, then compute the RMSE between $d_{\mathrm{WILT}}$ and $d_{\mathrm{MPNN}}$ on held-out graphs and compare the highest-weight WL colors to those fitted on the full dataset: if the held-out fit degrades sharply relative to the in-sample fit, or the top-weight colors change substantially between splits, the claim that the learned weights identify the causal drivers of the MPNN distance fails its central test.
Extended reading notes
Core claim
The central claim is that the distance an MPNN places between graphs is not generic structural similarity but a task-specific pseudometric encoding the functional distance between prediction targets. Training consistently increases $\mathrm{ALI}_k(d_{\mathrm{MPNN}}, d_{\mathrm{func}})$, and the Spearman correlation between this alignment and performance is positive and consistent across datasets (accuracy for classification, RMSE for regression), whereas the same check against structural distances shows neither improvement nor correlation. Because MPNN embeddings are functions of multisets of WL colors, the paper defines $d_{\mathrm{WILT}}$, the optimal transport distance on a weighted Weisfeiler-Leman labeling tree, and proves it equals a weighted $\ell^1$ difference of WILT embeddings, giving linear-time computation; size and dummy-node normalizations recover the Wasserstein WL distance ($d_{\mathrm{WWL}}$) and the WL optimal assignment distance ($d_{\mathrm{WLOA}}$) as special cases, and expressiveness theorems match the size-normalized variant to mean-pooled MPNNs and the dummy-node variant to sum-pooled MPNNs. Distilled onto trained GCNs and GINs, the fitted $d_{\mathrm{WILT}}$ approximates $d_{\mathrm{MPNN}}$ much better than the fixed-weight baselines, and most edge weights collapse toward zero: with $\ell^1$ regularization, about 95% of edges can be zeroed while keeping a better fit than $d_{\mathrm{WLOA}}$. On Mutagenicity, the WL colors with the largest weights correspond to subgraphs with known mutagenic function, supporting the claim that MPNNs build their embedding geometry from a small set of task-relevant substructures.
Load-bearing premise
The load-bearing premise is that the distilled WILT distance faithfully reproduces the MPNN's embedding distance, so that the largest learned edge weights mark the subgraphs that actually steer where graphs sit in embedding space, and the paper only checks this fit on the same graphs used to fit the weights.
Editorial extensions
If this is right
- The property that tracks MPNN predictive performance is alignment of embedding distance with the task's functional distance, not alignment with task-agnostic structural distances; the paper's data show that training increases this alignment and that higher alignment correlates with higher accuracy or lower RMSE.
- The WILTing distance with learned edge weights approximates $d_{\mathrm{MPNN}}$ substantially better than the fixed-weight WL-based distances $d_{\mathrm{WWL}}$ and $d_{\mathrm{WLOA}}$, which are its special cases.
- Size-normalized WILT is the right surrogate for mean-pooled MPNNs and dummy-node-normalized WILT for sum-pooled MPNNs, matching the expressiveness analysis; sum-pooled MPNNs are in fact not bounded by the size-normalized distance.
- MPNN embedding geometry is sparse: only a small fraction of WL colors carry it, since distilling with $\ell^1$ regularization keeps a good fit with roughly 95% of the edge weights zeroed.
- The subgraphs behind the largest weights align with domain-known functional substructures (toxicophores in mutagenicity prediction), giving a global interpretation of where graphs sit relative to each other in MPNN space.
Reading between the lines
- The ALI-to-performance correlation is measured across a grid of model architectures and hyperparameters, so it is a between-model correlation; the paper does not test whether alignment rises within a single training run before accuracy does, which would be needed to argue that the alignment causes the performance rather than merely co-occurs with it.
- The WILT fit is evaluated on the same graphs used to fit the edge weights, so an out-of-sample check, fitting on a training split and measuring RMSE and top-weight colors on a held-out split, would show whether the identified subgraphs are stable or an artifact of in-sample fitting.
- The authors note the tree construction extends to higher-order WL hierarchies and to node-embedding distances; a natural next experiment is training WILT edge weights from scratch as a standalone interpretable graph classifier, a use they mention as a possibility.
- In domains without established toxicophore lists, the distillation could serve as an automated hypothesis generator: the highest-weight WL colors propose candidate functional substructures that domain experts could then validate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the Euclidean distance between MPNN graph embeddings (d_MPNN) and asks whether this learned distance is task-aligned and interpretable. It first measures alignment between d_MPNN and a task-defined functional pseudometric d_func, reporting that training increases this alignment and that alignment correlates with predictive performance. The main methodological contribution is the Weighted Weisfeiler-Leman Labeling Tree (WILT): a weighted tree of WL colors whose shortest-path/optimal-transport distance d_WILT is shown to generalize the Wasserstein WL distance and the WL optimal assignment distance, to be computable in linear time via an explicit embedding, and to sit in a proved expressiveness hierarchy relative to d_MPNN. Edge weights are fitted by minimizing squared error between d_WILT and d_MPNN on a graph dataset, after which large learned weights are interpreted as identifying WL subgraphs that strongly influence the MPNN embedding distance. Experiments on five datasets with GCN and GIN report good in-sample RMSE for the distilled distance, a skewed weight distribution, and top-weight subgraphs that match known toxicophores on Mutagenicity.
Significance. If its empirical claims hold, the paper offers a concrete, computationally affordable surrogate for MPNN embedding geometry and a new way to produce global, subgraph-level interpretations of trained MPNNs. The theoretical parts are a genuine contribution: Proposition 5.3 gives an exact tree-Wasserstein/Manhattan equivalence, Theorems A.7 and A.8 identify WWL and WLOA as special cases, and Theorems A.9--A.11 give a clean expressiveness ordering that correctly predicts the mean-pooling versus sum-pooling experimental trends. The paper ships code and the derivations are self-contained. The main weakness is that the central interpretability claim rests on an in-sample fit of d_WILT to d_MPNN; without held-out validation, the chain from learned edge weights to statements about which subgraphs 'determine' d_MPNN is not fully established.
major comments (3)
- [Section 5.4] The approximation quality of d_WILT is evaluated only in-sample. Appendix E states that "We used the entire data set for D in L" when minimizing the distillation loss, and Section 6 reports RMSE computed from 1000 pairs drawn from the same D^2. Because d_WILT is linear in the edge weights (Proposition 5.3) and the number of weights can be large, a low in-sample RMSE does not establish that d_WILT approximates d_MPNN as a function on unseen graphs. This is load-bearing: the Section 5.4 inference "large w => large d_WILT => large d_MPNN" and the sparsity and top-subgraph analyses in Figures 4--6 and Tables 7--8 all use the same full-data fit. I ask the authors to split the dataset (or at least the graph pairs) into distillation and evaluation parts, report RMSE on held-out pairs, and recompute the top-weight subgraphs from a model fitted only on the training split.
- [Section 6] The qualitative claim that the largest edge weights identify subgraphs known to be functionally important is not compared with any baseline. Since the method can only output WL colors that survive a frequency filter, it is unclear whether the 4/10 or 3/10 toxicophore matches are significantly better than what a random or frequency-matched set of WL colors would achieve. A simple enrichment analysis, such as precision@k against the Kazius toxicophore list with a random-color control, would make the central interpretability claim much stronger.
- [Section 4] The claim that alignment between d_MPNN and d_func is "key to the high predictive performance of MPNNs" is supported by Spearman correlations computed across models that simultaneously vary in architecture, depth, width, and pooling. This design confounds alignment with model capacity and other hyperparameters. The evidence is consistent with correlation but does not establish that alignment is the causal driver; an ablation that varies training signal or regularizes d_MPNN toward/away from d_func while holding architecture fixed would be needed for the stronger causal wording used in the abstract and Section 1.
minor comments (5)
- [Section 7] The word "psudometrics" should be "pseudometrics".
- [Appendix E] In the paragraph discussing partial matches, "limiation" should be "limitation".
- [Tables 7 and 8] The label "alphatic halide" should be "aliphatic halide".
- [Figure 3] In the text as provided, the colors in the figure are rendered as blank boxes; if this is not a text-extraction artifact, the example in Figure 3 is very hard to follow and should be fixed in the camera-ready version.
- [Algorithm 1] The projection step "w <- max(w, 0)" is applied after every Adam update; since Adam state depends on past gradients, this is a valid projected-gradient approach but the paper should state that the projection is applied per iteration rather than only at the end.
Circularity Check
No circularity: fitted edge weights are an interpretative summary; core distance equivalence and expressiveness proofs are self-contained.
full rationale
The paper's derivation chain is self-contained. Proposition 5.3 (dWILT as weighted L1 distance over WILT embeddings) is a direct tree-Wasserstein/Manhattan equivalence and is proved from the definition of the WILT embedding, not from dMPNN. Theorems A.7 and A.8 show the fixed-weight special cases dWWL and dWLOA by explicit calculation. The expressiveness statements (Theorem 5.4 / A.9-A.11) follow from WL color multiset identities and the definitions of mean/sum pooling; they do not presuppose the fitting objective. The only fitted component is the edge weight learning in Section 5.4: L(w) = sum (dWILT(G,H;w) - dMPNN(G,H))^2, with Algorithm 1. The later claim that large weights identify colors whose presence/absence affects dMPNN is explicitly presented as an interpretation of the distilled distance (Section 5.4's implication chain uses 'dWILT approximates dMPNN'), not as a held-out prediction. This is a post-hoc summary of the fitted parameters, so it is not a fitted input renamed as an independent prediction. The in-sample evaluation (Table 2 uses 1000 pairs from D^2, and Appendix E states 'We used the entire data set for D in L') is a real generalization/validity limitation: if dWILT overfits, the interpretive chain could fail on unseen graphs. The conclusion's limitation paragraph (Section 7) likewise narrows the evidence to two architectures. These concerns belong to correctness risk, not circularity; no equation defines the claimed interpretation as the fitting objective without the additional, empirically checked approximation step. Self-citations (e.g., Pluska et al. 2024, Graziani et al. 2024) occur only in related-work and extension remarks and are not load-bearing for any theorem or main result; no uniqueness theorem is imported from the authors. The acknowledgements joke about the title carries no technical claim. Overall: no definitional, self-citation, or ansatz-by-citation circularity.
Assumptions & free parameters
free parameters (1)
- WILT edge weights w(e) =
learned via Adam; not enumerated
assumptions (4)
- domain assumption The Weisfeiler-Leman algorithm with injective aggregation yields a tree structure on the set of colors (WILT).
- domain assumption For any MPNN, the node embedding h_v^(l) is a function of the WL color c_v^(l).
- standard math Optimal transport on a tree equals a weighted Manhattan distance on counts of subtree masses (Le et al., 2019).
- standard math The functional pseudometric dfunc is a pseudometric (triangle inequality holds for the binary and normalized absolute difference).
invented entities (1)
-
Weighted Weisfeiler Leman Labeling Tree (WILT)
independent evidence
Cite this review
Pith. "Pith review of WILTing Trees: Interpreting the Distance Between MPNN Embeddings." pith.science (2026). https://pith.science/paper/BM5Z2DXZ
@misc{pith2026250524642,
author = {Pith},
title = {Pith review of: WILTing Trees: Interpreting the Distance Between MPNN Embeddings},
year = {2026},
howpublished = {\url{https://pith.science/paper/BM5Z2DXZ}},
note = {Machine review of arXiv:2505.24642}
}
read the original abstract
We investigate the distance function learned by message passing neural networks (MPNNs) in specific tasks, aiming to capture the functional distance between prediction targets that MPNNs implicitly learn. This contrasts with previous work, which links MPNN distances on arbitrary tasks to structural distances on graphs that ignore task-specific information. To address this gap, we distill the distance between MPNN embeddings into an interpretable graph distance. Our method uses optimal transport on the Weisfeiler Leman Labeling Tree (WILT), where the edge weights reveal subgraphs that strongly influence the distance between embeddings. This approach generalizes two well-known graph kernels and can be computed in linear time. Through extensive experiments, we demonstrate that MPNNs define the relative position of embeddings by focusing on a small set of subgraphs that are known to be functionally important in the domain.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Global explainability of GNNs via logic combination of learned concepts
Azzolin, S., Longa, A., Barbiero, P., Li \` o , P., and Passerini, A. Global explainability of GNNs via logic combination of learned concepts. In International Conference on Learning Representations, 2023
work page 2023
-
[2]
Fine-grained expressivity of graph neural networks
B \"o ker, J., Levie, R., Huang, N., Villar, S., and Morris, C. Fine-grained expressivity of graph neural networks. Advances in Neural Information Processing Systems, 36, 2024
work page 2024
-
[3]
An optimal lower bound on the number of variables for graph identification
Cai, J., F \" u rer, M., and Immerman, N. An optimal lower bound on the number of variables for graph identification. Combinatorica, 12 0 (4): 0 389--410, 1992
work page 1992
-
[4]
Chuang, C.-Y. and Jegelka, S. Tree mover's distance: Bridging graph metrics and stability of graph neural networks. Advances in Neural Information Processing Systems, 2022
work page 2022
-
[5]
J., Morris, C., Velingker, A., and Geerts, F
Franks, B. J., Morris, C., Velingker, A., and Geerts, F. Weisfeiler-leman at the margin: When more expressivity matters. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 , 2024
work page 2024
-
[6]
Frasca, F., Bevilacqua, B., Bronstein, M. M., and Maron, H. Understanding and extending subgraph GNNs by rethinking their symmetries. In Advances in Neural Information Processing Systems, 2022
work page 2022
-
[7]
Geerts, F. and Reutter, J. L. Expressiveness and approximation properties of graph neural networks. In International Conference on Learning Representations, 2022
work page 2022
-
[8]
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chemistry. In International Conference on Machine Learning, 2017
work page 2017
Show all 38 references
-
[9]
Expressiveness and approximation properties of graph neural network
Graziani, C., Drucks, T., Jogl, F., Bianchini, M., Scarselli, F., and Gärtner, T. Expressiveness and approximation properties of graph neural network. In International Conference on Machine Learning, 2024
2024
-
[10]
Derivation and validation of toxicophores for mutagenicity prediction
Kazius, J., McGuire, R., and Bursi, R. Derivation and validation of toxicophores for mutagenicity prediction. Journal of Medicinal Chemistry, 48 0 (1): 0 312--320, 2005
2005
-
[11]
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017
2017
-
[12]
A neural collapse perspective on feature evolution in graph neural networks
Kothapalli, V., Tirer, T., and Bruna, J. A neural collapse perspective on feature evolution in graph neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023
2023
-
[13]
M., Giscard, P., and Wilson, R
Kriege, N. M., Giscard, P., and Wilson, R. C. On valid optimal assignment kernels and applications to graph classification. In Advances in Neural Information Processing Systems, 2016
2016
-
[14]
M., Johansson, F
Kriege, N. M., Johansson, F. D., and Morris, C. A survey on graph kernels. Applied Network Science, 5: 0 1--42, 2020
2020
-
[15]
and Heindorf, S
Köhler, D. and Heindorf, S. Utilizing description logics for global explanations of heterogeneous graph neural networks. arXiv preprint:2405.12654, 2024
2024 arXiv
-
[16]
Tree-sliced variants of Wasserstein distances
Le, T., Yamada, M., Fukumizu, K., and Cuturi, M. Tree-sliced variants of Wasserstein distances. Advances in Neural Information Processing Systems, 2019
2019
-
[17]
and Leskovec, J
Li, P. and Leskovec, J. The expressive power of graph neural networks. In Graph Neural Networks: Foundations, Frontiers, and Applications. 2022
2022
-
[18]
Interpretability in graph neural networks
Liu, N., Feng, Q., and Hu, X. Interpretability in graph neural networks. In Graph Neural Networks: Foundations, Frontiers, and Applications. 2022 a
2022
-
[19]
Exploring consistency in graph representations: from graph kernels to graph neural networks
Liu, X., Cai, Y., Yang, Q., and Yan, Y. Exploring consistency in graph representations: from graph kernels to graph neural networks. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024
2024
-
[20]
Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F., and Yu, P. S. Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering, 35 0 (6): 0 5879--5900, 2022 b
2022
-
[21]
Provably powerful graph networks
Maron, H., Ben-Hamu, H., Serviansky, H., and Lipman, Y. Provably powerful graph networks. Advances in Neural Information Processing Systems, 2019
2019
-
[22]
L., Lenssen, J
Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., and Grohe, M. Weisfeiler and Leman go neural: Higher-order graph neural networks. In AAAI Conference on Artificial Intelligence, 2019
2019
-
[23]
M., Bause, F., Kersting, K., Mutzel, P., and Neumann, M
Morris, C., Kriege, N. M., Bause, F., Kersting, K., Mutzel, P., and Neumann, M. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML Workshop on Graph Representation Learning and Beyond, 2020
2020
-
[24]
M., Grohe, M., Fey, M., and Borgwardt, K
Morris, C., Lipman, Y., Maron, H., Rieck, B., Kriege, N. M., Grohe, M., Fey, M., and Borgwardt, K. M. Weisfeiler and leman go machine learning: The story so far. Journal of Machine Learning Research, 24: 0 333:1--333:59, 2023
2023
-
[25]
\.I ., Levie, R., Lim, D., Bronstein, M
Morris, C., Frasca, F., Dym, N., Maron, H., Ceylan, \.I . \.I ., Levie, R., Lim, D., Bronstein, M. M., Grohe, M., and Jegelka, S. Position: Future directions in the theory of graph machine learning. In International Conference on Machine Learning, 2024
2024
-
[26]
GraphChef : Decision-tree recipes to explain graph neural networks
Müller, P., Faber, L., Martinkus, K., and Wattenhofer, R. GraphChef : Decision-tree recipes to explain graph neural networks. In International Conference on Learning Representations, 2024
2024
-
[27]
Logical distillation of graph neural networks
Pluska, A., Welke, P., Gärtner, T., and Malhotra, S. Logical distillation of graph neural networks. In International Conference on Knowledge Representation and Reasoning, 2024
2024
-
[28]
and Fu, K
Sanfeliu, A. and Fu, K. A distance measure between attributed relational graphs for pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics: Systems , 13 0 (3): 0 353--362, 1983
1983
-
[29]
Wasserstein Weisfeiler-Lehman graph kernels
Togninalli, M., Ghisu, E., Llinares-L \'o pez, F., Rieck, B., and Borgwardt, K. Wasserstein Weisfeiler-Lehman graph kernels. Advances in Neural Information Processing Systems, 2019
2019
-
[30]
Optimal transport: old and new, volume 338 of Grundlehren der mathematischen Wissenschaften
Villani, C. Optimal transport: old and new, volume 338 of Grundlehren der mathematischen Wissenschaften. Springer, 2009
2009
-
[31]
N., Gomes, J., Geniesse, C., Pappu, A
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learning. Chemical Science, 9 0 (2): 0 513--530, 2018
2018
-
[32]
How powerful are graph neural networks? In International Conference on Learning Representations, 2019
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In International Conference on Learning Representations, 2019
2019
-
[33]
Explainability in graph neural networks: A taxonomic survey
Yuan, H., Yu, H., Gui, S., and Ji, S. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (5): 0 5782--5799, 2022
2022
-
[34]
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., and Sun, M. Graph neural networks: A review of methods and applications. AI open, 1: 0 57--81, 2020
2020
-
[35]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[36]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
-
[37]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
-
[38]
oq̟μ. 5` ]ndX_ oxzn (no1 a px zr HL/^A u cW| IbV8 A ,7eD
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.