REVIEW 2 major objections 6 minor 25 references
PhyloVAE: Unsupervised Learning of Phylogenetic Trees via Variational Autoencoders
T0 review · 2 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read PhyloVAE learns a latent representation of phylogenetic tree topologies and generates new trees from it, using a bijective linear-time encoding that makes the discrete tree space tractable for a variational autoencoder.
desk verdict Useful new VAE for tree topologies with a real algorithmic claim, but the O(N) proof has a gap that needs fixing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the linear-time bijective encoding of tree topologies (Theorem 1), carried out by Algorithm 1: decompose $\tau$ by removing pendant leaves in reverse leaf order while recording each edge decision, then rebuild the tree from the three-leaf starting topology while assigning integer indices to every node, so the recorded decisions become the vector $s(\tau)=(s_3,\ldots,s_{N-1})$. This encoding turns discrete tree space into vectors, which enables a factorized non-autoregressive generative model over the coordinates and an inference model $q_\phi(z|\tau)$ whose representation comes from topological node embeddings followed by gated message passing and sum-pooling. Each coordinate's conditional distribution is masked so only indices that correspond to real edges of an ordinal tree are allowed, guaranteeing that decoded vectors map back to valid topologies.
What would settle it
Implement Algorithm 1 exactly and time it on random trees with $N=50,100,200,400$ leaves; if total runtime grows superlinearly in $N$, the linear-time theorem fails and the speed advantage over Phylo2Vec loses its basis. A second check is to re-estimate the DS1-8 KL divergences with an independent importance-sampling estimator and compare PhyloVAE with latent dimension 10 against the ARTree values reported in the paper.
Extended reading notes
Core claim
The paper's central claim is that the discrete space of tree topologies admits a complete representation-learning pipeline built on an encoding that is bijective and computable in $O(N)$ time. For a fixed leaf order, a tree topology $\tau$ is decomposed by removing pendant leaves in reverse order while recording the edge on which each leaf was attached; replaying those decisions while indexing nodes yields the integer vector $s(\tau)=(s_3,\ldots,s_{N-1})$. The generative model factors the distribution of $s(\tau)$ given a latent variable $z$ as a product over coordinates of masked discrete distributions, so every coordinate is generated in parallel; the inference model $q_\phi(z|\tau)$ is built from learnable topological node embeddings computed by minimizing a Dirichlet energy and then passing messages through a gated recurrent network. On eight benchmark data sets (DS1-8) the paper reports that PhyloVAE with latent dimension 10 matches or improves on the ARTree baseline on several data sets while training and generating several times faster, and that its two-dimensional latent representations separate distinct groups of trees more clearly than MDS.
Load-bearing premise
The $O(N)$ encoding claim rests on the assumption that each edge decision recorded while peeling a tree can be located again in constant time while rebuilding the tree; the proof points to a dictionary built from the original tree, but the reconstruction loop builds a fresh tree whose nodes are not the recorded edge objects, and the constant-time mapping between the two is not specified.
Editorial extensions
If this is right
- Any point in the learned latent space decodes to a valid tree topology, so PhyloVAE can interpolate between observed trees and visualize neighborhoods of tree space in a way MDS cannot.
- Because generation is non-autoregressive, training and sampling parallelize; the paper's runtime measurements show PhyloVAE is several times faster than ARTree at the same particle count ($K=32$).
- With latent dimension 10, PhyloVAE's KL divergence to the reference posterior is comparable to ARTree across DS1-8 and better on DS3, DS5, and DS6, so the speed gain does not come at a clear cost in density-estimation accuracy.
- The two-dimensional latent representations separate distinct tree distributions (five gene posteriors, three simulated mixture peaks, five independent posterior runs) more cleanly than MDS, and each run's consensus tree falls inside its correct high-density region.
- The bijective encoding itself is an exact description of the topology, so no information is lost before the VAE compresses the vector into the latent representation.
Reading between the lines
- Because $s(\tau)$ is a bijective, size-$N$ description of a topology, it could be used directly as a fixed feature map for clustering, regression, or classification of tree collections, without training any neural network.
- The latent manifold suggests conditional generation: conditioning the prior or encoder on covariates such as gene, sequence length, or geographic region could produce tree distributions for unobserved conditions; the paper does not explore this.
- The clean separation of independent posterior runs hints at a convergence diagnostic based on comparing latent densities across runs; the paper shows the phenomenon but does not formalize a statistical test.
- One open question the paper leaves implicit is whether straight-line interpolation between two latent means yields biologically plausible intermediate trees; the continuity example suggests it might, but no biological validity check is reported.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces PhyloVAE, an unsupervised variational autoencoder for collections of unrooted phylogenetic tree topologies. The method encodes a tree topology into an integer vector by sequentially removing and then re-adding pendant leaves, uses this encoding to define a non-autoregressive generative model with conditionally independent entries given a latent variable, and builds an inference model on learnable topological node features from a GNN. The authors claim a bijective, linear-time encoding mechanism, fast parallelized training and generation relative to the autoregressive ARTree baseline, and useful low-dimensional latent representations. Experiments cover representation learning on simulated and real phylogenies, density estimation on the DS1-8 benchmark sets, and runtime comparisons with ARTree.
Significance. If the central algorithmic and modeling claims hold, PhyloVAE is a useful contribution: it is one of the first deep generative frameworks that simultaneously provides low-dimensional representations of tree topologies and a way to map latent points back to trees, it releases code, and it reports competitive density estimates on DS1-8 at substantially reduced training and generation times. The representation results, especially the separation of gene-tree and BEAST-run distributions, are visually compelling. However, the proof of the O(N) encoding theorem has a genuine gap that directly affects the linear-time and speed claims, so the paper needs a corrected or clarified algorithmic argument before the central claims can be fully verified.
major comments (2)
- [Appendix C.1, Theorem 1] The proof of Theorem 1 does not establish the O(1) lookup step in the reconstruction loop. The dictionary D is constructed from the original tree before decomposition and its keys are original node objects, but during the reconstruction loop in Algorithm 1 the internal nodes t_{n+1} are new nodes that do not occur in D. For example, in Figure 2 the encoding entry s_4 = 7 refers to the internal node created when the fourth leaf is re-attached, not to a node in the original tree's dictionary. The sentence 'This can be done in O(1) time by indexing k1_n, k2_n in D' therefore does not cover the endpoints of the edges that must be located in the reconstructed tree. The proof must specify a data structure, such as an array of current nodes indexed by Index(v) that is updated as nodes are created, that returns the current node for the required Index value in O(1) per reconstruction step. Without this, the O(N) claim in Theorem 1 and the runtime comparisons in Figure 5 are not verifiable from the manuscript as written.
- [Section 3.2] The bijectivity of the proposed encoding is asserted but not proved or precisely reduced to a previous result. The generative model in Eq. (5) defines p_theta(tau|z) through the encoding s(tau), so the inverse direction from encoding vectors to tree topologies is load-bearing for the probability model. The paper cites the ARTree decision-sequence bijection, but the Index-based encoding and reconstruction loop introduce an additional indexing layer that is not covered verbatim by the ARTree statement. Please add an explicit bijectivity theorem or a precise reduction to Xie & Zhang (2023) that accounts for the indexing rules in Algorithm 1, especially given the reconstruction-lookup gap identified above.
minor comments (6)
- [Eq. (4)] Equation (4) uses the symbol n in the range and in the dimension, writing [s3, ..., s_{n-1}] and N^{n-3}; the intended index is the fixed total number of leaves N, and the dimension should be N-3. Please correct this notational inconsistency.
- [Algorithm 3] Algorithm 3 sets Index(t_{n+1}) <- N + n - 1, while Algorithm 1 and the main text in Section 3.2 use N + n - 2. These cannot both be correct for the same indexing scheme, and the values in Figure 2 are consistent with N + n - 2.
- [Table 1 and Section 5.3] The text says PhyloVAE with d=10 achieves 'comparable (if not better) results to ARTree', but on DS1, DS7, and DS8 the reported KL divergences are 0.0189 vs 0.0045, 0.0370 vs 0.0191, and 0.1061 vs 0.0741, respectively. The comparison should be stated more precisely: PhyloVAE is better on DS3, DS5, and DS6, comparable on DS2 and DS4, and noticeably worse on DS1, DS7, and DS8.
- [Appendix E.3, Table 3] The claim that increasing the number of particles K generally improves approximation accuracy is not supported by DS2 and by the DS3 row with d=10, where KL divergences increase from 0.0453 at K=16 to 0.0515 at K=64. Please temper the statement or discuss this non-monotonicity.
- [Figure 5] The runtime comparison reports a single measurement for each setting without variance or repeated runs; given that the speed advantage is a headline contribution, reporting means and standard deviations over several runs would be more convincing.
- [Figure 3] The caption uses the phrase 'partition presentation'; this appears to be a typo for 'partition representation'.
Circularity Check
No significant circularity: the VAE objective, encoding, and benchmarks are self-contained; the O(N) proof gap is a completeness issue, not a circular reduction.
full rationale
PhyloVAE's derivation chain is: (1) a bijective encoding of tree topologies into integer vectors adapted from ARTree; (2) a VAE whose generative model factorizes over encoding coordinates and whose inference model uses learnable graph features; (3) training by maximizing the multi-sample ELBO on observed tree topologies; and (4) evaluation against independent MrBayes ground-truth distributions. None of these steps is defined in terms of the quantity it claims to predict. The encoding is not fitted: it is an exact combinatorial map, and the generative model's mask only enforces validity of the encoding space. The inference model's parameters are learned, but the reported latent separations and KL divergences are evaluated on held-out or independently generated ground-truth posteriors, not on the training targets. The citations to ARTree (Xie & Zhang, 2023) and Zhang (2023) are used as building blocks (an autoregressive decomposition and a graph-feature architecture); ARTree's bijection is a parameter-free combinatorial fact with stated assumptions and is not invoked to forbid alternatives, so these self-citations are not load-bearing in a circular sense. The only notable weakness is in Appendix C.1's proof of Theorem 1: the proof says recorded edges can be located in O(1) by indexing k1_n, k2_n in dictionary D, but D is built on the original tree while the reconstruction loop builds a fresh tree, so the node-to-edge mapping in the rebuilt tree is unspecified. This is a real presentation gap and a correctness risk for the linear-time claim, but it is not a circularity: the theorem does not reduce to its inputs by construction. Overall, the central claims are independently testable against external benchmarks, and no prediction is forced by a fit or by a self-citation chain.
Assumptions & free parameters
free parameters (3)
- latent dimension d =
2 or 10
- IWAE particles K =
32
- message passing rounds L =
2
assumptions (3)
- domain assumption ARTree's edge decision sequence is bijective with tree topologies (Xie & Zhang 2023).
- domain assumption The topological node embeddings from Zhang (2023) provide useful, information-preserving features.
- ad hoc to paper The factorized generative model in Eq. (5) (conditional independence of encoding elements given z) is expressive enough for tree topology distributions.
invented entities (1)
-
edge decision encoding vector s(τ)
independent evidence
Cite this review
Pith. "Pith review of PhyloVAE: Unsupervised Learning of Phylogenetic Trees via Variational Autoencoders." pith.science (2026). https://pith.science/paper/YZZXOSU2
@misc{pith2026250204730,
author = {Pith},
title = {Pith review of: PhyloVAE: Unsupervised Learning of Phylogenetic Trees via Variational Autoencoders},
year = {2026},
howpublished = {\url{https://pith.science/paper/YZZXOSU2}},
note = {Machine review of arXiv:2502.04730}
}
read the original abstract
Learning informative representations of phylogenetic tree structures is essential for analyzing evolutionary relationships. Classical distance-based methods have been widely used to project phylogenetic trees into Euclidean space, but they are often sensitive to the choice of distance metric and may lack sufficient resolution. In this paper, we introduce phylogenetic variational autoencoders (PhyloVAEs), an unsupervised learning framework designed for representation learning and generative modeling of tree topologies. Leveraging an efficient encoding mechanism inspired by autoregressive tree topology generation, we develop a deep latent-variable generative model that facilitates fast, parallelized topology generation. PhyloVAE combines this generative model with a collaborative inference model based on learnable topological features, allowing for high-resolution representations of phylogenetic tree samples. Extensive experiments demonstrate PhyloVAE's robust representation learning capabilities and fast generation of phylogenetic tree topologies.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
Given a tree topologyτ with N leaf nodes, the time complexity of computing its encoding vector s(τ ) is O(N ). Proof of Theorem 1 Assume the tree topology τ = (V, E) is stored in the binary tree data structure, where each node other than the root node also has a parent node pointer (the root node is arbitrarily selected from the internal nodes for the sto...
work page 2025
-
[2]
doi: 10.1109/INFVIS.2002. 1173150. Yuri Burda, Roger B. Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In International Conference on Learning Representations,
-
[3]
Let τn = (Vn, En) be a tree topology with n (n ≤ N ) leaf nodes in X
leaf nodes. Let τn = (Vn, En) be a tree topology with n (n ≤ N ) leaf nodes in X . We say τn is an ordinal tree topology of rank n, if its leaf nodes are the first n elements of X , i.e., Vn ∩ X= {x1, . . . , xn}. The tree topology generation process begins with τ3 (the unique ordinal tree topology of rank 3). At the n-th step, assume we have an ordinal t...
work page 2023
-
[8]
Here are definitions of the Robinson-Foulds (RF) distance and the path difference (PD) distance between two tree topologies (the edge lengths are not considered). 16 Published as a conference paper at ICLR 2025 ABF HGE DC0.714286 (a) The pre-selected tree topology for the first peak. DFH CBA GE0.714286 (b) The pre-selected tree topology for the second pea...
work page 1981
-
[9]
We also give the Newick representation of this ground truth tree (from Hillis et al. (2005)) for reproducing the results: 17 Published as a conference paper at ICLR 2025 OpossumDiprotodontian SlothAnteaterArmadillo HedgehogShrewMolePhyllostomidFree tailed batFalse vampire batFlying FoxRousette FruitbatWhaleDolphinHippoRuminantPigLlamaHorseRhinoTapirCatCan...
work page 2005
-
[14]
PhyloGFN: Phylogenetic inference with generative flow networks
URL http://jmlr.org/ papers/v25/22-0348.html. Mingyang Zhou, Zichao Yan, Elliot Layne, Nikolay Malkin, Dinghuai Zhang, Moksh Jain, Mathieu Blanchette, and Yoshua Bengio. PhyloGFN: Phylogenetic inference with generative flow networks. ArXiv, abs/2310.08774,
-
[15]
13 Published as a conference paper at ICLR 2025 A P HYLOGENETIC TREES AND PHYLOGENETIC INFERENCE The common structure for describing evolutionary history is a phylogenetic tree, which consists of a bifurcating tree topology τ and the associated non-negative edge lengths q (see Figure 9 for a real example). The tree topology τ is a bifurcating tree graph (...
work page 2025
-
[16]
generates a tree topology in an autoregressive way. The sequential generating process in ARTree facilitates a probabilistic model over tree topologies which archives leading results in phylogenetic inference. We introduce the tree topology generating process of ARTree as below and most of the statements are adapted from Xie & Zhang (2023). Let τn = (Vn, E...
work page 2023
Show all 25 references
-
[19]
D E XPERIMENTAL DETAILS For all experiments, PhyloV AE is implemented in PyTorch (Paszke et al., 2019)
Algorithm 3 is similar to the reconstruction loop in Algorithm 1, and has time complexity O(N ). D E XPERIMENTAL DETAILS For all experiments, PhyloV AE is implemented in PyTorch (Paszke et al., 2019). The optimizer is Adam (Kingma & Ba,
2019
-
[24]
20 Published as a conference paper at ICLR 2025 −4 −2 0 2 4 µ1 −4 −2 0 2 4 µ2 Mammal gene trees (seqlen =
of DS1-8 produced by PhyloV AE. 20 Published as a conference paper at ICLR 2025 −4 −2 0 2 4 µ1 −4 −2 0 2 4 µ2 Mammal gene trees (seqlen =
2025
-
[25]
ground truth on DS1
635.67 637.72 655.69 785.75 689.15 701.30 784.41 818.72 21 Published as a conference paper at ICLR 2025 10□4 10□3 10□2 10□1 100 ground truth 10□4 10□3 10□2 10□1 100 estimated probability d = 2, K= 32 peak 1 peak 2 10□4 10□3 10□2 10□1 100 ground truth 10□4 10□3 10□2 10□1 100 es...
2025
-
[32]
The experiments are run on a single 2.3 GHz CPU
The learning rate is set to 0.00001 at the beginning and anneals according to a cosine schedule. The experiments are run on a single 2.3 GHz CPU. D.5 G ENERATIVE MODELING CAPACITY ON BENCHMARK DATA SETS Following Zhang & Matsen IV (2018); Xie & Zhang (2023), we construct the t...
2018
-
[1999]
Gated graph sequence neural networks
11 Published as a conference paper at ICLR 2025 Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493,
2025 arXiv
-
[2001]
doi: 10.1126/science.1067179. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilam...
-
[2002]
INFOVIS 2002., pp. 71–74,
2002
-
[2012]
These 6000 tree topologies with uniform weights constitute the training set of PhyloV AE
runs for 10 million iterations, and gather the tree samples per 1000 iterations from the last 3 million iterations. These 6000 tree topologies with uniform weights constitute the training set of PhyloV AE. We find that 6000 tree topologies in the training set are distinct, whi...
2025
-
[2013]
doi: 10.1093/ sysbio/syt014
ISSN 1063-5157. doi: 10.1093/ sysbio/syt014. URL http://dx.doi.org/10.1093/sysbio/syt014. Bret R. Larget and D. L. Simon. Markov chain Monte Carlo algorithms for the Bayesian analysis of phylogenetic trees. Molecular Biology and Evolution, 16:750–750,
-
[2014]
Variational graph auto-encoders.arXiv preprint arXiv:1611.07308,
Thomas N Kipf and Max Welling. Variational graph auto-encoders.arXiv preprint arXiv:1611.07308,
-
[2015]
doi: 10.1093/sysbio/syv006
ISSN 1063-5157, 1076-836X. doi: 10.1093/sysbio/syv006. URL http://dx.doi.org/10.1093/sysbio/syv006. Robin Winter, Frank Noé, and Djork-Arné Clevert. Permutation-invariant variational autoencoder for graph-level representation learning. Advances in Neural Information Processing...
-
[2016]
Learning phrase representations using RNN encoder-decoder for statistical machine translation
10 Published as a conference paper at ICLR 2025 Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv prepr...
2025 arXiv
-
[2017]
doi: 10.1111/1755-0998.12676
ISSN 1755-098X, 1755-0998. doi: 10.1111/1755-0998.12676. URL http://dx.doi.org/10.1111/1755-0998.12676. Thomas H Jukes, Charles R Cantor, et al. Evolution of protein molecules. Mammalian protein metabolism, 3:21–132,
-
[2018]
doi: 10.1093/ve/vey016
ISSN 2057-1577. doi: 10.1093/ve/vey016. URL http: //dx.doi.org/10.1093/ve/vey016. 12 Published as a conference paper at ICLR 2025 Mafalda Viana, Julio A Benavides, Alice Broos, Darcy Ibañez Loayza, Ruby Niño, Jordan Bone, Ana da Silva Filipe, Richard Orton, William Valderrama ...
2025 doi
-
[2021]
Flowseq: Non-autoregressive conditional sequence generation with generative flow
Xuezhe Ma, Chunting Zhou, Xian Li, Graham Neubig, and Eduard Hovy. Flowseq: Non-autoregressive conditional sequence generation with generative flow. arXiv preprint arXiv:1909.02480,
1909 arXiv
-
[2023]
Variational bayesian phylo- genetic inference with semi-implicit branch length distributions
Tianyu Xie, Frederick A Matsen IV , Marc A Suchard, and Cheng Zhang. Variational bayesian phylo- genetic inference with semi-implicit branch length distributions. arXiv preprint arXiv:2408.05058, 2024a. Tianyu Xie, Musu Yuan, Minghua Deng, and Cheng Zhang. Improving tree proba...
-
[2024]
doi: 10.1093/sysbio/syae030
ISSN 1063-5157. doi: 10.1093/sysbio/syae030. URL https://doi.org/10. 1093/sysbio/syae030. David F Robinson and Leslie R Foulds. Comparison of phylogenetic trees.Mathematical Biosciences, 53(1-2):131–147,
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.