{"id":"8ac3b367-8da2-4ab5-aed4-0cbd84929b70","arxiv_id":"2411.17154","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A sequence-only model, Emergenet, uses learned residue dependencies to forecast seasonal flu strains and rank animal strains for pandemic risk, with strong backtests but a non-rigorous central derivation.","lead":"Researchers built a model called Emergenet that reads influenza virus sequences and tries to predict which animal flu strains are most likely to jump to humans, and which strains will dominate next season. In backtests it matched or beat WHO vaccine picks and its risk scores correlated with CDC expert scores, but the mathematical core has gaps and the highest-risk strain count in the abstract is not backed by the tables.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The emergence-risk score depends on an unproved pseudo-likelihood factorization (Eq. 20); without it, Theorem 1 does not establish that E-distance bounds jump probability, and Eq. (4) lacks the claimed probabilistic basis.","rationale":"The reader's weakest_assumption (Eq. 20) identifies the same load-bearing point: the novelty of Emergenet as a risk-assessment tool, rather than a sequence-distance heuristic, is the proof that E-distance scales with log jump probability. I credit the seasonal vaccine results and the in-silico corroboration as useful empirical support that the E-distance captures relevant structure. However, the emergence-risk claim specifically relies on Eq. (5), which follows from Theorem 1; if the pseudo-likelihood factorization in Eq. (20) is not a theorem, the bounds in Eq. (10) do not apply to the actual biological transition probability, and Eq. (4) is not a probability-based emergence score. The IRAT correlation (r=0.721, n=23) is suggestive and could support a reframed, purely empirical claim, but the paper presents it as validation of a theory-backed score and uses the same 23 strains for the regression line. Separately, the abstract's '35 strains with high emergence scores (>7.7)' is not supported by Extended Data Table 9, whose per-subtype counts sum to 4 above 7.7; this is a reporting inconsistency but secondary to the theorem gap. Because the central claim as stated is not established, the reader's REJECT verdict stands; a rigorous derivation of Eq. (20) or an explicit reframing with held-out IRAT validation would be needed to change that.","tokens_in":76559,"tokens_out":11276,"duration_ms":106526,"concrete_test":"Independently re-derive Theorem 1 from Definitions 1–3 and Eq. (8) without invoking Eq. (20). A decisive numerical check: build a two-site random field with a known joint distribution and a strong interaction, compute the true conditional probability Pr(Y=y|X=x) and the Eq. (20) product ∏_i P_i(x_{-i})(y_i) from the same model; if they differ, Eq. (20) is an additional assumption rather than a theorem. If no derivation of the theorem goes through without this assumption, the central probabilistic interpretation of the E-risk score is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that E-distance (Eq. 8) can estimate emergence risk for animal strains because Theorem 1 bounds log Pr(x→y) by ±C·ρ(x,y). The proof of Theorem 1 ('Proof of Probability Bounds') hinges on Eq. (20), which asserts Pr(x→y) = ∏_i P_i(x_{-i})(y_i). This is a pseudo-likelihood factorization, not a consequence of the random-field definition or of Sanov's theorem; Sanov's theorem governs large deviations of empirical measures, not single-sequence transition probabilities. The identity assumes that, conditional on the source sequence x, target positions mutate independently and simultaneously, and it is not the conditional distribution P(Y=y|X=x) of the model. The subsequent Pinsker/Jensen-Shannon manipulation therefore bounds only a pseudo-likelihood ratio, not the actual jump probability. Eq. (27) then replaces a0, defined as the smallest nonzero per-site probability, with (1−α)/N, a significance-threshold substitution that is not derived and that creates the N² factor in Eq. (10). Without Eq. (20) and Eq. (27), the bounds used to justify Eq. (5), and hence the interpretation of Eq. (4) as an emergence probability rather than a sequence dissimilarity, are not established. The seasonal vaccine backtest and the 23-strain IRAT correlation are useful empirical results, but they do not repair the missing theoretical bridge because the paper explicitly claims a theory-backed scalable risk score and uses Theorem 1 to justify that claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Emergenet, a sequence-based model of influenza A evolution built from conditional inference trees over HA and NA residues. The authors define an E-distance between strains and, via a claimed theorem (Theorem 1), assert that this distance bounds the log probability of one strain evolving into another. They use this to (i) recommend seasonal vaccine strains retrospectively and (ii) compute an emergence-risk score for animal strains, which they validate by correlation with CDC IRAT scores for 23 strains and then apply to 6,354 animal strains collected after 2020. The vaccine-strain backtests are extensive and the IRAT correlation is suggestive, but the central theoretical derivation of the probability bounds is not sound as written.","tokens_in":76960,"tokens_out":7245,"duration_ms":72472,"significance":"If the probabilistic bridge between E-distance and jump probability were valid, the paper would offer a notable new capability: scalable, sequence-only risk assessment for animal influenza strains before human spillover, with open-source software and inferred models made available. The 20-year WHO comparison is a substantial empirical exercise, and the SHAP-based residue analysis is a useful addition. However, the main emergence-risk claim depends on Theorem 1, and the proof of that theorem contains a load-bearing unjustified factorization, while the IRAT validation is computed on the calibration set rather than out of sample. These issues currently prevent the paper from supporting its headline claim.","major_comments":[{"comment":"The proof of Theorem 1 begins by asserting Pr(x→y) = ∏_i P_i(x_{-i})(y_i), attributed to Sanov's theorem. Sanov's theorem is a large-deviation statement about empirical measures and does not imply this exact product form for single-sequence transition probabilities. Moreover, the Emergenet is defined in Definition 1 as a collection of conditional predictors, and Fig. 1b explicitly exhibits a cyclic dependency (14 depends on 63, 63 on 155, 155 on 223, 223 on 14); full conditional distributions from such a recursive structure need not be consistent with any joint distribution, so the product is not evidently a probability. Unless Eq. (20) is introduced as an explicit modeling assumption (a pseudo-likelihood), the Pinsker/JS manipulations that follow bound only a pseudo-likelihood ratio, not the jump probability Pr(x→y). Since Eq. (10) and Eq. (5) are then used to justify the risk score in Eq. (4), the probabilistic interpretation of the E-risk score is not established.","section":"Methods & Materials, Proof of Probability Bounds, Eq. (20)"},{"comment":"The replacement of a0, defined as the smallest nonzero per-site probability, by (1−α)/N is not derived. If a0 is the per-site probability of a 'failure' and α is the significance level, the equation 1−(1−a0)^N = α gives a0 ≈ α/N for large N, not (1−α)/N. Because a0 controls the constant C1 = sqrt(8)N^2/(1−α) in Eq. (10), the N^2 factor and the numerical constants in the claimed bounds are not justified.","section":"Methods & Materials, Proof of Probability Bounds, Eq. (27)"},{"comment":"The reported total least squares correlation r = 0.721 is computed after fitting the linear regression y = 0.47x + 2.77 to the same 23 IRAT strains. This is an in-sample measure of linear association, not an out-of-sample validation. The text states 'we find strong out-of-sample support,' but the presented analysis does not support that claim. A leave-one-out or independent validation set is needed before the predicted IRAT scores for the 6,354 animal strains can be interpreted as validated predictions.","section":"Fig. 3 and Extended Data Table 8"},{"comment":"Theorem 1 gives |log Pr(x→y) − C0| ≤ C1 ρ(x,y), i.e., log probability is controlled by ρ, not by −log ρ. The negative-log-geometric-mean form of the risk score in Eq. (4) is therefore not a consequence of the theorem; it is an additional monotone transform chosen for calibration to IRAT scores. Even if Theorem 1 were repaired, Eq. (4) would need a separate argument linking its score to the probability bound.","section":"Measure of Emergence Potential, Eqs. (4) and (18)"}],"minor_comments":[{"comment":"There are several typographical errors, including 'Hemagglutinnin' in the abstract, 'are are based' in the Emergenet section, and 'clutser' in the Methods discussion around Eq. (14).","section":"Abstract and main text"},{"comment":"The inequality following Eq. (23) should state the absolute value explicitly; as written, the sign of 1 − Q_i(y_{-i})(y_i)/P_i(x_{-i})(y_i) is not handled, and the division requires that a0 be a valid lower bound on both the P and Q probabilities.","section":"Proof of Probability Bounds, Eq. (23)"},{"comment":"The theorem is stated 'at significance level α,' but α only enters through the heuristic replacement in Eq. (27). The statistical meaning of α in the bound should be clarified or the statement revised.","section":"Theorem 1 statement"},{"comment":"The proof refers to 'the q-distance between x,y' in the text after Eq. (19), but the term should be 'E-distance' for consistency with Definition 2 and the rest of the paper.","section":"Theorem 1 proof"}],"recommendation":"reject","confidential_remarks":"The vaccine-forecasting results may be separable and publishable on their own, but the emergence-risk claim rests on Theorem 1, whose proof is invalid as written, and on an in-sample IRAT calibration. A resubmission would need either a rigorous derivation of Eq. (20) from a clearly defined generative model, or a reframing of the risk score as an empirically calibrated dissimilarity with proper out-of-sample validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this paper is two different things. The seasonal vaccine strain prediction is a solid empirical contribution; the emergence-risk score for animal strains has a real theoretical gap in the proof that makes its central claim unsupported as written.\n\nWhat's genuinely new: they learn per-position conditional inference trees from large sequence databases, define an E-distance as average Jensen-Shannon divergence between those conditionals, and use it to select vaccine strains and score animal strains. The vaccine backtest is substantial: 20 years of WHO comparisons, 220k sequences, and consistent gains in Hamming distance to the future population, with code and models publicly available. That part deserves attention.\n\nThe soft spot is Theorem 1. The proof's Eq. (20) simply asserts that the jump probability factorizes as the product of per-position conditional distributions from the source population. That is a pseudo-likelihood assumption, not a consequence of the random-field definition or of Sanov's theorem. Sanov governs empirical measures, not single-sequence transitions. And Eq. (27) replaces a0 with (1-alpha)/N without derivation. Without those two steps, the bound connecting E-distance to jump probability doesn't follow, and the interpretation of Eq. (4) as an emergence probability collapses to 'a sequence dissimilarity'. The seasonal vaccine selection criterion is also derived from that bound, but the empirical results don't depend on the proof—they stand as a heuristic that works in backtest.\n\nThe IRAT correlation is suggestive but weaker than it looks: the regression is fit on the same 23 strains used to report the correlation, so no out-of-sample validation. The abstract's '35 strains >7.7' is not clearly tied to the tables. These are fixable or at least reframe-able.\n\nMy take: the authors have a real empirical result in the vaccine prediction part, and a reproducible pipeline. The theory needs either a rigorous derivation or an explicit retreat to 'heuristic distance'. A serious referee should see this, because the vaccine result is worth publishing and the gap is precisely the kind of thing peer review is for. I'd send it out, but I'd expect major revision or a reframing.","headline":"Solid vaccine-strain backtest, but the emergence-risk scoring rests on an unproved factorization in Theorem 1, so the paper is only half supported.","tokens_in":77431,"tokens_out":2932,"would_cite":false,"duration_ms":29740,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sequence-only score ranks animal flu strains by pandemic potential","keywords":["influenza A","emergence risk assessment","zoonotic spillover","sequence evolution","conditional inference trees","E-distance","vaccine strain prediction","pandemic preparedness"],"falsifier":"Take animal strains that later caused documented human infections or were selected as pandemic candidate vaccine viruses, and contemporaneous animal strains that never did; compute their emergence scores using models trained only on data available before each event. If the emerged strains do not cluster in the top percentiles of the score distribution, or if many never-emerged strains score higher, the central claim is refuted.","tokens_in":76361,"feed_emoji":"🦠","tokens_out":7925,"duration_ms":73643,"temperature":0.7,"pith_summary":"Emergenet is a generative model of influenza A sequence evolution, trained only on viral protein sequences, that the paper uses to answer two questions: which strains will dominate next season, and which animal strains could seed the next human outbreak. Its emergence-risk claim is that an animal strain's potential to jump into humans can be scored as the negative logarithm of the geometric mean of the smallest E-distances from its HA and NA sequences to recently circulating human sequences, where E-distance is a divergence between conditional residue distributions learned from sequence data. The paper reports that these scores track expert-assessed IRAT emergence scores with a correlation of 0.721 (p = $10^{{-4}}$) across 23 animal strains, and that the scores are cheap enough to screen 6,354 post-2020 animal strains in seconds, flagging 35 above 7.7. The same E-distance machinery, used to nominate seasonal vaccine strains, outperforms official recommendations over two decades, an average improvement of 3.73 amino acids for H1N1 in northern-hemisphere seasons. If the framework holds, it would make quantitative, sequence-only risk ranking available at pandemic-relevant speed.","feed_headline":"A sequence-only score ranks animal flu strains by pandemic potential","feed_subtitle":"Influenza A strains get risk scores in seconds from sequence alone, matching expert assessments and flagging 35 high-risk strains.","key_machinery":"The load-bearing object is the Emergenet: a forest of conditional inference trees, one per residue position, in which each tree predicts the amino acid at its position from the residues at other positions, keeping only statistically significant splits. From this forest the paper defines the E-distance $\\rho(x,y)$ between two sequences as the average over positions of the square-root of the Jensen-Shannon divergence between the conditional distributions the two sequences' contexts induce. It also defines the persistence probability $\\omega_y^Q = \\prod_j \\hat{P}^Q_j(y_{-j} \\mid y_j)$, the model's estimate that strain $y$ reproduces itself. The proof of Theorem 1 combines Sanov's theorem with a refinement of Pinsker's inequality to show $\\ln \\Pr(x \\to y)$ is within a constant of the persistence-adjusted E-distance, which is what licenses using $\\rho$ as a proxy for jump probability and hence for emergence risk.","core_discovery":"The central discovery is that the evolutionary reachability of a strain from the current human-adapted population can be quantified from sequences alone, and that this quantity behaves like an emergence risk. Reachability is measured by the E-distance: for each residue position, the Emergenet's tree predicts a conditional distribution over amino acids given the rest of the sequence, and the distance between two strains is the average over positions of the square root of the Jensen-Shannon divergence between the conditional distribution assigned to one strain's context and that assigned to the other's. Theorem 1 states that the probability of a spontaneous jump $\\Pr(x \\to y)$ is bounded above and below by the persistence probability of the target times exponentials of $\\pm c\\,\\rho(x,y)$, so a small E-distance to a well-adapted human strain implies high jump probability. The emergence score $\\mathcal{E}_t(x) = -\\log \\min_{y,z \\in H_t} \\sqrt{\\rho_r(x_H,y_H)\\,\\rho_s(x_N,z_N)}$ combines HA and NA. The authors treat the out-of-sample correlation of 0.721 with expert IRAT scores as evidence that this sequence-only ranking captures the same information as multi-assay expert assessment.","pith_inferences":["If the E-distance-to-jump-probability link is right, the recipe is portable: learn position-wise conditional dependencies from a large sequence database, then measure a new strain's distance to the currently adapted human distribution; that same recipe could be applied to other rapidly mutating zoonotic viruses with sufficient sequence diversity.","The IRAT correlation validates relative ordering, not absolute probability: a high score says a strain is evolutionarily close to currently circulating human strains, not that a pandemic is inevitable, and the paper says so explicitly but the caveat is easy to lose in the headline number.","A decisive prospective test would freeze the model at each historical assessment date, score all then-available animal strains, and ask whether later human-isolated strains fall in the top percentiles; the paper's examples suggest this but do not fully demonstrate it."],"forward_implications":["Newly sequenced animal influenza strains can be risk-ranked in about 30 seconds per strain, which makes continuous, genome-scale surveillance for spillover candidates practical.","Seasonal vaccine-strain nomination improves over official recommendations when two cluster-specific strains are reported, suggesting the framework could supply sequence-only vaccine-strain candidates each season.","The method can score strains whose subtype combination has never been seen in humans, such as H14N12, by comparing each segment against existing human HA and NA models.","The predicted high-risk set is dominated by swine and avian H5N1 strains, pointing toward animal hosts, especially pigs and poultry, as the natural targets for preemptive vaccination.","Because scores are stable under random 75% resampling of the human strain pool, sparse wild-reservoir sampling does not by itself destabilize the rankings."],"supporting_citations":[{"why":"Supplies the conditional inference tree algorithm used for each per-position predictor in the Emergenet.","marker":"[11]"},{"why":"Provides the expert-assessed IRAT emergence scores used as the validation target for the emergence score.","marker":"[16]"},{"why":"Provides the genotype-phenotype seasonal forecast baseline and the distance-to-future evaluation metric the paper compares against.","marker":"[6]"},{"why":"Supplies the Jensen-Shannon divergence and Sanov's theorem that underpin the E-distance and the probability bounds.","marker":"[28]"},{"why":"Provides the refinement of Pinsker's inequality used in the proof of Theorem 1.","marker":"[54]"},{"why":"Supplies one of the public influenza sequence databases used for training and screening.","marker":"[55]"},{"why":"Supplies the other influenza sequence database used for training and screening.","marker":"[56]"}],"fun_headline_variants":["Sequence-only model ranks flu strains for pandemic risk in seconds","Fast sequence-based scoring flags 35 high-risk animal flu strains","Digital twin of flu evolution predicts pandemic risk from sequence alone","Emergenet ranks animal flu strains by pandemic risk from sequence alone","Sequence-only risk scores for animal flu match expert assessments in seconds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that small E-distance implies high jump probability assumes that the probability of one strain turning into another equals the product, over sequence positions, of the per-site conditional probabilities the model learns; the paper asserts this factorization rather than deriving it from a mechanistic model of influenza evolution.","fun_headline_variants_meta":{"raw":{"variants":["Sequence-only model ranks flu strains for pandemic risk in seconds","Fast sequence-based scoring flags 35 high-risk animal flu strains","Digital twin of flu evolution predicts pandemic risk from sequence alone","Emergenet ranks animal flu strains by pandemic risk from sequence alone","Sequence-only risk scores for animal flu match expert assessments in seconds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000881,"raw_usage":{"total_tokens":3857,"prompt_tokens":1046,"completion_tokens":2811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":2726}},"tokens_in":662,"tokens_out":2811,"duration_ms":17564,"temperature":1.0,"reasoning_tokens":2726,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:27:17.398844+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take animal strains that later caused documented human infections or were selected as pandemic candidate vaccine viruses, and contemporaneous animal strains that never did; compute their emergence scores using models trained only on data available before each event. If the emerged strains do not cluster in the top percentiles of the score distribution, or if many never-emerged strains score higher, the central claim is refuted.","supporting_citations":[{"cited_title":"& Zeileis, A","cited_arxiv_id":null,"evidence_quote":"Supplies the conditional inference tree algorithm used for each per-position predictor in the Emergenet."},{"cited_title":"Influenza risk assessment tool (irat)","cited_arxiv_id":null,"evidence_quote":"Provides the expert-assessed IRAT emergence scores used as the validation target for the emergence score."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the genotype-phenotype seasonal forecast baseline and the distance-to-future evaluation metric the paper compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Jensen-Shannon divergence and Sanov's theorem that underpin the E-distance and the probability bounds."},{"cited_title":"A., Harremo ¨es, P","cited_arxiv_id":null,"evidence_quote":"Provides the refinement of Pinsker's inequality used in the proof of Theorem 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies one of the public influenza sequence databases used for training and screening."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the other influenza sequence database used for training and screening."}],"review_version":1}