{"id":"a1db7969-f1ae-486d-becf-d7d48f78ac85","arxiv_id":"2412.18827","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A language-model-based method claims to infer phylogenies from unaligned DNA and to beat MCMC baselines on all eight benchmark datasets, but the reported scores rely on an unspecified likelihood.","lead":"PhyloGen generates evolutionary trees directly from raw DNA sequences using a language model and variational inference, skipping alignment and explicit evolutionary models. The authors report that it beats standard phylogenetic methods on eight benchmark datasets, but the comparison depends on a likelihood that the paper never defines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported MLL/ELBO superiority is uninterpretable because the likelihood p(Y | τ(z), Bτ) is never defined in §3.2; every comparison to MrBayes therefore rests on an undefined target.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing gap: the paper never defines the likelihood p(Y | τ(z), Bτ), so the reported MLL and ELBO values cannot be interpreted as marginal likelihoods of the observed sequences. I read the paper in good faith and looked for an implicit definition: the Feature Extraction module uses DNABERT2 to produce embeddings E, and those embeddings are used to construct a latent variable z* and a distance matrix for Neighbor-Joining. But none of these steps defines a probability distribution over the original sequence data Y given a tree and branch lengths. The variational objective in Eqs. 6-8 repeatedly invokes p(Y | τ(z), Bτ) without specifying its functional form, its normalization, or how it would be evaluated during training. The same issue affects the MLL estimate in §E.1, which requires a target posterior distribution for importance sampling. Without that likelihood, the comparison to MrBayes is not a comparison of equivalent quantities. I did not base my verdict on the secondary mechanical problems, such as the undefined XOR operation on continuous embeddings or the absence of a VaiPhy baseline, because even if those were corrected, the undefined likelihood would remain fatal to the central claim. The method is novel in spirit, and a clearly specified latent-variable model for tree-structured phylogenetic inference could be valuable, but as written the experimental section does not establish the claimed state-of-the-art performance.","tokens_in":19083,"tokens_out":3581,"duration_ms":36646,"concrete_test":"The decisive check is to request the explicit likelihood formula p(Y | τ(z), Bτ) and the released code, then independently recompute the DS1 MLL using Eq. 8 with 1000 importance samples from the stated variational distribution. If no well-defined, normalized density over DNA sequences can be written down, or if the code does not evaluate such a density, the reported MLL and ELBO values are not marginal likelihoods and the headline claim fails.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing step is Eq. 7 (and its multi-sample form Eq. 8), where the objective contains p(Y, Bτ | τ(z)) = p(Y | τ(z), Bτ) p(Bτ | τ(z)). The paper states that p(Y | τ(z), Bτ) is 'the conditional probability of the observed data Y' but never defines a probability model over DNA sequences conditional on a tree. The Feature Extraction module maps Y to embeddings E through DNABERT2; no inverse generative model, sequence-level decoder, or likelihood over Y is specified anywhere in the text. Consequently Eq. 8 is not an ELBO for the marginal likelihood of Y, and the 'MLL estimate' described in §E.1, obtained by sampling 1000 importance samples, has no well-defined target distribution. The comparison to MrBayes, whose likelihood is an explicit substitution model, is therefore not a comparison of the same quantity. The size of the reported gaps (roughly 200 nats on DS1 and 500 nats on DS8) is not itself a flaw, but it is consistent with the model optimizing a different, undefined objective. This is an internal gap rather than a disagreement with consensus: the paper's own equations require a likelihood that the text never supplies, and without it the central claim of state-of-the-art MLL and ELBO is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PhyloGen, a method that infers phylogenetic trees directly from raw DNA sequences by combining a pretrained genomic language model (DNABERT2) with a graph-structure-generation framework. The pipeline extracts embeddings, constructs an initial tree via Neighbor-Joining on a latent distance matrix, and then jointly models topology and branch lengths through variational inference, a tree encoder/decoder, a DGCNN branch-length module, and an auxiliary scoring function. The main empirical claim, stated in the abstract and Section 4.2, is that PhyloGen achieves the highest Marginal Log Likelihood (MLL) and Evidence Lower Bound (ELBO) on all eight benchmark datasets compared with MCMC-based, tree-representation, and tree-generation baselines, without relying on evolutionary models or equal-length sequence alignment.","tokens_in":19415,"tokens_out":8435,"duration_ms":76076,"significance":"If the claims were correct, PhyloGen would be a substantial advance: a single differentiable pipeline from unaligned raw sequences to a joint posterior over topology and branch lengths, avoiding explicit substitution models and alignment. The manuscript reports a broad set of experiments across eight standard benchmarks, including topological diversity, robustness to node addition/deletion, ablations, and bipartition-frequency comparisons, which is a reasonable breadth of evidence. The core quantitative claim, however, rests on an undefined likelihood: p(Y | tau(z), B_tau) is never specified as a probability density over the observed sequences, so the reported MLL and ELBO values have no well-defined target. The bipartition-frequency comparison to MrBayes is the only external validation, but it is visual, limited to three datasets, and not quantified. The paper does not provide code or data, and the baseline numbers in Section 4.2 are accompanied by a contradictory provenance statement. For these reasons, the central claim of state-of-the-art MLL/ELBO is not currently supported.","major_comments":[{"comment":"The objective is not a well-defined probabilistic model because p(Y | tau(z), B_tau) is never defined. The Feature Extraction module maps sequences Y to embeddings E via DNABERT2 (Section 3.A), and no inverse model, sequence-level decoder, or density over Y is specified anywhere. Section 3.2 refers to 'the conditional probability of the observed data Y' without giving a density, so Eq. 8 is not an ELBO for a marginal likelihood of Y, and the MLL estimate in Appendix E.1, obtained by sampling 1000 importance samples, has no well-defined target distribution. Since the abstract and Section 4.2 base the main claim on highest MLL/ELBO values, the reported hundreds-of-nats advantage over MrBayes is uninterpretable as a likelihood comparison.","section":"Section 3.2, Eqs. (6)-(8); Appendix D.2"},{"comment":"The distance matrix definition D(i,j) = sum_{i,j=1}^N z_i^* xor z_j^* is not computable as written: the double summation over i and j makes the right-hand side independent of the chosen pair (i,j), and the symbol xor is said to represent an 'XOR operation reflecting nucleotide mismatches' even though z_i^* are continuous real-valued latent vectors. No definition of XOR on continuous vectors, or a mapping from embeddings to nucleotides, is supplied. Because this D is fed into the Neighbor-Joining step that produces tau(z*), the initial tree construction is ill-defined and not reproducible.","section":"Section 3.B"},{"comment":"The gradient derivation is algebraically inconsistent. Differentiating an expectation with respect to Q_theta gives grad_theta E_Q[f] = E_Q[grad_theta log Q f + grad_theta f], but Eq. (27) keeps only the score-function term. Eq. (30) then reintroduces grad_theta H[Q_theta(z)] with the incorrect identity grad_theta H[Q] = -E_Q[grad_theta log Q], and the sign conventions in Eq. (10) do not follow from Eq. (7). As a result, the paper does not show that the training procedure optimizes the stated ELBO, even under the assumption that the likelihood is defined.","section":"Appendix D.2, Eqs. (25)-(30); Section 3.3, Eqs. (10)-(12)"},{"comment":"The provenance of the baseline MLL values is unclear. Appendix E.2 states that 'the results of all baseline methods are not included in the MLL tables, as some of the baseline methods are not provided with source code, and the results of the MLL metrics are not shown in the original paper,' yet Table 1 lists MLL values for MrBayes, SBN, VBPI, VBPI-GNN, ARTree, GeoPhy, and PhyloGFN. The authors must state which numbers are taken from prior papers, which are recomputed, and with which settings; otherwise the central 'outperforms all baselines' claim cannot be checked. Table 1 also contains an obvious numerical typo: the GeoPhy DS4 entry is -133342.71, three orders of magnitude lower than the neighboring DS4 values around -13330.","section":"Section 4.2, Table 1; Appendix E.2"}],"minor_comments":[{"comment":"The symbol xor is used for two different operations: in the distance matrix D it denotes the alleged XOR on latent vectors, and in Eq. (4) it denotes MAX aggregation. Reusing one symbol for two unrelated operations makes the method statement confusing.","section":"Section 3.B and Section 3.C.2, Eq. (4)"},{"comment":"The claimed similarity of the bipartition-frequency curves to MrBayes is only assessed visually and only for DS1-DS3; report a quantitative divergence measure, such as Jensen-Shannon divergence, and include the remaining datasets.","section":"Section 4.4, Fig. 5"},{"comment":"The statement that 'the closer the S curve is to the ELBO curve, the more it proves S can effectively evaluate the model performance' is not a valid evaluation because S is an auxiliary network trained jointly with the same ELBO objective; closeness of the two curves would be expected even if S carries no useful information.","section":"Section 3.1, Fig. 3"},{"comment":"The training description is ambiguous: Section E.1 says K = 2 Monte Carlo samples and a total of one million Monte Carlo samples, while Algorithm 1 updates parameters per iteration; please clarify the relationship between training steps, Monte Carlo samples, and the 1000-sample importance estimate used for MLL.","section":"Appendix E.1, Algorithm 1"},{"comment":"The Delta columns in Table 4 are not defined relative to a stated reference value, and the claim that positive Delta after node deletion represents 'improved performance' is counterintuitive for likelihood-based metrics; please specify the reference and explain the sign convention.","section":"Section 4.5, Table 4"}],"recommendation":"reject","confidential_remarks":"The central issue is the undefined likelihood p(Y | tau(z), B_tau), which is an internal mathematical gap rather than a simple disagreement with the field's consensus. In addition to the likelihood gap, the manuscript does not state code/data availability, and the baseline provenance in Table 1 is contradictory. These problems are load-bearing for the paper's main claim and would require substantial redefinition of the model and rerunning of all experiments to fix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's headline result — state-of-the-art MLL and ELBO on all eight benchmarks — is not established: the likelihood p(Y | tau(z), B_tau) that appears in the ELBO (Eq. 7) and in the reported MLL is never defined. Without a probability model that maps a tree and branch lengths to raw sequences, those numbers don't have a well-defined target.\n\nThe idea is genuinely new in this area: using DNABERT2 embeddings as features, generating a start tree via NJ in a learned latent space, and co-optimizing topology and branch lengths with a graph-based variational model. That is a reasonable combination of existing pieces. The authors also do a few things well: the bipartition frequency comparison against MrBayes is a legitimate external sanity check, the robustness experiments (node deletion/addition) are a sensible stress test, and the ablation study includes a useful one-hot baseline that shows the pretrained embeddings matter. The writing is clear enough that a reader can follow the pipeline and reproduce the general setup.\n\nThe undefined likelihood is the big one. The stress test got it right: Eq. 7 treats p(Y | tau(z), B_tau) as if it were a standard sequence likelihood, but no such likelihood is specified anywhere. The model maps Y to embeddings via DNABERT2 and never defines an inverse generative model or a sequence-level decoder. So the ELBO is not a bound on the log marginal likelihood of Y, and the MLL estimates in Table 1 are not comparable to MrBayes's marginal likelihoods. The gap of 200-500 nats is exactly what you'd expect when optimizing a different objective. Also, the XOR distance in Section 3.B is undefined for continuous vectors; they later say it reflects nucleotide mismatches, but the embeddings are continuous floats. The gradient derivations in Appendix D.2 have some algebraic inconsistencies (the entropy term appears and disappears, and Eq. 33 seems to drop the expectation over epsilon properly). No code or data is released, and VaiPhy is listed in related work but missing from the baseline tables. Hyperparameters are tuned on DS1 and then applied to the rest, which at least inflates significance for DS1.\n\nThe central argument doesn't hold up as written, but this is not a case of a fundamentally wrong idea. The approach is plausible; it just needs a well-defined objective, which likely means either defining a proper likelihood in embedding space or framing the method as a heuristic tree estimator rather than a Bayesian inference engine. That is a substantial revision, not a patch.\n\nThis paper is for people working on scalable phylogenetic inference, especially those already using VI or graph-based methods. A serious referee should engage with it, because the idea has legs and the empirical claims, if they survive reformulation, would matter. Send it to review, but ask reviewers to check whether the likelihood is ever defined and whether the reported MLL numbers are meaningful. My own verdict: reject in present form, but the core idea deserves a chance.","headline":"The core claim of state-of-the-art MLL/ELBO is unsupported because the likelihood is never defined, but the idea of embedding raw sequences with a genome LM for joint tree generation is novel and worth a serious look.","tokens_in":19911,"tokens_out":1879,"would_cite":false,"duration_ms":17171,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PhyloGen claims that a pretrained genomic language model can generate phylogenetic trees from raw DNA and jointly optimize topology and branch lengths, beating MCMC-based methods on all eight benchmark datasets.","keywords":["phylogenetic inference","variational inference","genomic language model","tree topology generation","branch length estimation","Neighbor-Joining","marginal log-likelihood","DNA embeddings"],"falsifier":"Run the paper's stated loss (Eq. 8) with an explicitly written and implemented $p(Y \\mid \\tau(z), B_\\tau)$ on DS1 and check whether the MLL reproduces the reported -6910.02; either the likelihood specification fails to close or the number does not reproduce, and the central outperformance claim would be settled.","tokens_in":18871,"feed_emoji":"🧬","tokens_out":7389,"duration_ms":61334,"temperature":0.7,"pith_summary":"PhyloGen claims that phylogenetic inference can be recast as a graph-structure generation problem solved end-to-end from raw DNA sequences. Instead of requiring aligned sequences and an explicit evolutionary substitution model, the method embeds each genome with a pretrained language model, builds an initial tree with Neighbor-Joining on learned distances, and then jointly refines topology and branch lengths through variational inference. On eight standard benchmark datasets, the paper reports the highest marginal log-likelihood and ELBO values of all methods compared, including MCMC baselines such as MrBayes. If this holds, a single differentiable pipeline could replace slow MCMC sampling for routine tree inference.","feed_headline":"Genome language model outperforms MCMC on all 8 tree benchmarks","feed_subtitle":"No alignment or evolutionary model needed; topology and branch lengths are optimized together from raw DNA.","key_machinery":"The carrying mechanism is a three-module pipeline: DNABERT2 feature extraction turns raw sequences into embeddings; the PhyloTree Construction module maps those embeddings through an MLP to a latent variable $z^*$, computes a distance matrix, and feeds it to the Neighbor-Joining algorithm to get an initial tree; the PhyloTree Structure Modeling module then co-optimizes topology and branch lengths through a TreeEncoder/TreeDecoder pair, a dual-pass traversal enhanced by DGCNN, and reparameterized branch-length sampling. A scoring function $S$ adds extra gradient information that the paper shows tracks the ELBO curve, while a multi-sample ELBO with an annealed prior forms the training objective.","core_discovery":"The paper's central finding is that phylogenetic inference can be solved as a conditional tree-structure generation problem: a genomic language model embeds raw DNA, a distance matrix built from the embedding space seeds an initial tree, and variational refinement jointly updates topology and branch lengths. On eight real-world datasets with 27 to 64 species, PhyloGen reports the highest MLL and ELBO of every method compared, including MrBayes; for example, on DS1 the MLL goes from -7108.42 (MrBayes) to -6910.02. The paper also reports broad topological diversity and bipartition frequencies that track the MrBayes posterior, which it offers as evidence that the learned trees are biologically faithful.","pith_inferences":["If the reported MLL values are reproducible, the result suggests that DNA language-model embeddings compress enough evolutionary signal to substitute for an explicit substitution model, at least on the small-to-medium datasets tested.","Because the initial tree comes from Neighbor-Joining, the method inherits NJ's known sensitivity to long-branch attraction; replacing NJ with a learned construction step would test whether that bottleneck matters.","The unspecified conditional likelihood $p(Y \\mid \\tau(z), B_\\tau)$ means the numerical MLL values should be checked against an explicit implementation; a public release with the likelihood written out would settle whether the comparison to MrBayes is on equal footing."],"forward_implications":["Raw, unaligned DNA can be used as direct input to phylogenetic inference, removing a preprocessing step that constrains many current methods.","Tree topology and branch lengths are optimized jointly in one differentiable objective instead of being estimated in separate stages.","The reported runtimes on DS1 (about 6.5 hours) are far below those of PhyloGFN and GeoPhy, suggesting a practical speed advantage for datasets of this size.","The learned latent distances carry phylogenetic signal: replacing them with Euclidean or cosine distances degrades MLL, tying the method's accuracy to its embedding-based distance matrix."],"supporting_citations":[{"why":"Supplies the pretrained genomic language model whose embeddings E are the sole input representation of raw DNA, so the whole pipeline depends on its sequence features.","marker":"[49]"},{"why":"The Neighbor-Joining algorithm turns the learned distance matrix into the initial tree topology τ(z*) on which all later refinement operates.","marker":"[31]"},{"why":"Establishes the variational Bayesian phylogenetic inference framework and ELBO objective that PhyloGen's learning objective extends and compares against.","marker":"[46]"},{"why":"MrBayes is the MCMC gold standard whose stepping-stone marginal likelihoods are the main baseline values PhyloGen claims to outperform.","marker":"[29]"},{"why":"The dynamic graph convolution layer provides the edge convolution used to enhance node features before branch lengths are reparameterized.","marker":"[35]"},{"why":"The variational autoencoder reparameterization trick lets the model backpropagate through sampled latent variables z and branch lengths.","marker":"[15]"},{"why":"The multi-sample Monte Carlo objective from Mnih and Rezende underlies the multi-sample ELBO used in Eq. 8 to lower gradient variance.","marker":"[23]"},{"why":"Subsplit Bayesian networks provide the topological prior and annealing schedule the method adopts, and serve as a structure-representation baseline.","marker":"[45]"}],"fun_headline_variants":["Genomic LM beats MrBayes on all 8, no alignment needed","Language model infers trees without alignments or evolutionary models","Joint topology-branch length optimization lifts likelihood past MrBayes","No evolutionary model needed: genomic LM outperforms MCMC","PhyloGen: language model-based trees beat MrBayes on 8 sets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a well-defined probability of the observed sequences given the tree, $p(Y \\mid \\tau(z), B_\\tau)$, exists and is actually evaluated by the model, since the paper never writes this likelihood down; if it is left undefined, the reported MLL and ELBO values cannot be compared with MCMC results.","fun_headline_variants_meta":{"raw":{"variants":["Genomic LM beats MrBayes on all 8, no alignment needed","Language model infers trees without alignments or evolutionary models","Joint topology-branch length optimization lifts likelihood past MrBayes","No evolutionary model needed: genomic LM outperforms MCMC","PhyloGen: language model-based trees beat MrBayes on 8 sets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001142,"raw_usage":{"total_tokens":4701,"prompt_tokens":870,"completion_tokens":3831,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":3740}},"tokens_in":486,"tokens_out":3831,"duration_ms":26742,"temperature":1.0,"reasoning_tokens":3740,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:25:49.115966+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's stated loss (Eq. 8) with an explicitly written and implemented $p(Y \\mid \\tau(z), B_\\tau)$ on DS1 and check whether the MLL reproduces the reported -6910.02; either the likelihood specification fails to close or the number does not reproduce, and the central outperformance claim would be settled.","supporting_citations":[{"cited_title":"Saitou and M","cited_arxiv_id":null,"evidence_quote":"The Neighbor-Joining algorithm turns the learned distance matrix into the initial tree topology τ(z*) on which all later refinement operates."},{"cited_title":"Zhang and F","cited_arxiv_id":null,"evidence_quote":"Establishes the variational Bayesian phylogenetic inference framework and ELBO objective that PhyloGen's learning objective extends and compares against."},{"cited_title":"Ronquist, M","cited_arxiv_id":null,"evidence_quote":"MrBayes is the MCMC gold standard whose stepping-stone marginal likelihoods are the main baseline values PhyloGen claims to outperform."},{"cited_title":"Mnih and D","cited_arxiv_id":null,"evidence_quote":"The multi-sample Monte Carlo objective from Mnih and Rezende underlies the multi-sample ELBO used in Eq. 8 to lower gradient variance."},{"cited_title":"Zhang and F","cited_arxiv_id":null,"evidence_quote":"Subsplit Bayesian networks provide the topological prior and annealing schedule the method adopts, and serve as a structure-representation baseline."}],"review_version":1}