{"id":"983d7766-b3fa-4c5e-b7e3-5db912b362e0","arxiv_id":"2608.06486","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A source-derived baseline for Integrated Gradients gives signed, species-specific attributions for disease-versus-health decisions in a feature-tokenized microbiome transformer, something unsigned attention weights cannot provide.","lead":"This paper shows how to get signed, direction-aware explanations from a gut-microbiome AI model: each species is labeled as either supporting an inflammatory bowel disease (IBD) prediction or supporting a healthy prediction, which attention weights alone cannot do. The proposed baseline is simple and should carry over to other transformer models that mix a fixed identity with a measured value.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The straight-line IG path in abundance-embedding space may not correspond to biologically meaningful abundance changes, so the signed directional claims are not yet established.","rationale":"The mathematical contribution is sound: under additive fusion, the source-derived baseline T' = S + A0 makes fused-token IG equivalent to abundance-source IG, and species attribution is zero by construction because the species embedding is held fixed. The internal chain-rule argument in Section 3.4 is correct, and completeness holds for the IBD-healthy margin. The reader's conditional verdict is appropriate, and my concern does not move the verdict: the method is acceptable in principle but needs stronger empirical support. My partial disagreement with the reader concerns the locus of the risk. The reader emphasizes that A0 = MLP_AB(0) could be out-of-distribution, but zero abundance is by far the most common microbial observation, so the endpoint is likely in-distribution. The more fragile assumption is the interpolation path: linear interpolation in abundance-embedding space may traverse embeddings that correspond to no valid abundance value, and gradients at those points need not reflect how the model responds to actual abundance increases. This is not an internal inconsistency; it is a correctness risk for the biological interpretation of the attributions. The paper acknowledges the issue by citing UDIG and leaving a tokenizer-aware path to future work, but the headline empirical claims are stated without this caveat. A single concrete check, comparing straight-line-path IG with tokenizer-aware-path IG, would settle whether the signed rankings are robust to path choice. If they diverge, the biological claims must be reframed as conditional on the path. The lack of confidence intervals and external validation is also real, but it is a secondary concern; the path semantics are more load-bearing for the central claim.","tokens_in":10894,"tokens_out":4740,"duration_ms":45733,"concrete_test":"Recompute attributions on the same 7,382 samples using a tokenizer-aware path, A(alpha) = MLP_AB(alpha * a_i) for alpha in [0,1], integrating through the learned abundance encoder instead of linearly between endpoint embeddings. Compare signed species rankings and per-sample margin attribution with the straight-line embedding-path IG. Report Kendall tau and sign agreement among the top 50 species. If the rankings are not highly correlated, the straight-line path is not representative, and the signed biological claims in Figures 4 and 6 should be treated as path-dependent artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The derivation that fused-token IG with T' = S + A0 equals abundance-source IG is mathematically correct for additive fusion. The load-bearing step is semantic: the attribution is interpreted as 'moving abundance from zero to observed value,' with positive signs supporting IBD and negative signs supporting healthy. That interpretation requires the straight-line path A0 + alpha*(A - A0) in embedding space, and the gradients along it, to correspond to actual abundance variation. Nothing in the paper verifies this. A0 is only one endpoint; intermediate path points need not decode to any abundance input and may lie off the manifold learned by MLP_AB. The paper itself cites UDIG (Ref. 13) for the concern that straight-line interpolation is poorly matched to tokenized models, and Section 6 defers a tokenizer-aware path to future work. If gradients along off-manifold points are unrepresentative, a species' IGA sign can be an artifact of the chosen path rather than evidence about abundance. That would undermine the central claim that IG separates pathogenic from protective microbial signals and the response profiles in Figure 6. This concern is distinct from the absence of confidence intervals: it targets the meaning of the attribution itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a source-derived Integrated Gradients baseline, T' = S + A0, for feature-tokenized transformers with additive token fusion, where S is a fixed species embedding and A is a sample-dependent abundance embedding. It proves that, under additive fusion, fused-token IG with this baseline is equivalent to abundance-source IG, with zero species attribution by construction. The method is then applied to a reimplemented BiomeGPT-style microbiome transformer for IBD versus healthy classification, and the paper reports signed attribution differences from [CLS] attention, path-averaged sensitivity diagnostics, and abundance-attribution response profiles. The central argument is that signed IG separates disease-supporting from health-supporting evidence where unsigned attention weights cannot, and the paper proposes second-order Integrated Hessians as a future extension for species-abundance interactions.","tokens_in":11057,"tokens_out":6520,"duration_ms":60594,"significance":"If the empirical and path-interpretation issues are resolved, the paper makes a clean and useful contribution: the derivation in Section 3.4 is correct and self-contained, and the source-derived baseline generalizes naturally to other additive feature-tokenized architectures such as FT-Transformer. The availability of code and the detailed reimplementation description are strengths. However, the empirical demonstration is currently preliminary: aggregate statistics are reported without uncertainty, the tested model is a reimplementation rather than the released BiomeGPT, and the biological interpretation of the attribution signs depends on an unvalidated straight-line path in embedding space. These issues limit the strength of the claims that the method 'uncovers species-abundance directional relationships' and 'separates pathogenic from protective microbial signals.'","major_comments":[{"comment":"The central interpretative claim, that IGA measures the effect of moving species abundance from zero to its observed value, requires the linear path A0 + alpha*(A - A0) to traverse abundance-embedding points that correspond to valid abundance variation. The paper itself cites UDIG (Ref. 13) for the concern that straight-line interpolation is poorly matched to tokenized models and explicitly defers a tokenizer-aware path to future work in Section 6, yet no experiment checks whether intermediate path points decode to valid abundance inputs, whether A0 is in-distribution, or whether the attribution signs are stable under an alternative path. Without such diagnostics, the signed rankings in Figures 4 and 6 may be artifacts of off-manifold interpolation rather than evidence about actual abundance effects, so the biological interpretation is not yet established.","section":"Section 3.4 and Section 6"},{"comment":"The quantitative evidence for the central empirical claim is thin. The only aggregate attribution numbers are the overall mean (-0.0011), the true-positive mean (0.0027), and the true-negative mean (-0.0045), all reported without standard deviations, confidence intervals, or significance tests, and the baseline margin F(S, A0) is never reported. Completeness only relates the total attribution sum to F(x) - F(x'), so these aggregate means carry little per-species information. The claimed 'clear contrast' between attention and IG, and the directional species rankings in Figure 4, require per-species distributions, effect sizes, and preferably a statistical comparison. Please report the baseline margin, per-species and per-sample attribution distributions, and appropriate uncertainty estimates.","section":"Section 5"},{"comment":"The experiments use 'a BiomeGPT-style microbiome transformer encoder' rather than the actual released BiomeGPT model, but the abstract and conclusion attribute the empirical findings to 'BiomeGPT' without qualification. The manuscript does not state whether the reimplementation matches the released BiomeGPT tokenizer, architecture, pretraining corpus, or pretrained weights, nor whether any external validation was performed. Because the empirical results are the only support for the biological directional claims, please specify exactly which components are shared with BiomeGPT and which are new, and adjust the abstract and conclusion claims so they refer to the reimplementation unless the released model is actually used.","section":"Section 4 and Abstract/Conclusion"}],"minor_comments":[{"comment":"The sentence 'preserves the fixed identity feature coordinate component and interpolate only the component encoding the observed value' has a grammatical subject-verb disagreement; it should read '...and interpolates only the component encoding the observed value.'","section":"Section 3.4"},{"comment":"The all-zero baseline is written as 'T' = 0 in R^512', but the fused input is a sequence of 1663 tokens each of dimension 512; please clarify whether the baseline is a per-token vector or a full sequence tensor.","section":"Section 4"},{"comment":"Please define 'True Positive' and 'True Negative' explicitly as correctly classified IBD samples and correctly classified healthy samples, respectively, since these terms could otherwise be confused with clinical ground truth.","section":"Section 5"},{"comment":"Please clarify whether abundance bin 0 is an actual discrete input bin in the tokenizer vocabulary or a special out-of-vocabulary input; the definition of A0 = MLP_AB(0) depends on this distinction.","section":"Section 3.1"},{"comment":"The claim that prior applications of IG to feature-tokenized architectures 'have not been explicitly derived or documented' is strong; please soften it to 'to the best of our knowledge' after a more thorough literature search, or cite the specific prior work that was checked.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The theoretical derivation is sound and I see no circularity. The main risk is over-interpretation of attribution scores as biological directional evidence. The path-validity diagnostics and stronger statistics requested in the major comments are, in my view, within the scope of a revision using the existing codebase, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead Nelson's BiomeGPT attribution paper. The short version: the central derivation is right and useful for interpretability in feature-tokenized transformers. The empirical part is not strong enough to support the stronger claims, and the straight-line path assumption is the real soft spot.\n\nWhat's actually new: the source-derived baseline T' = S + A0, where A0 is the learned zero-abundance embedding, and the proof that fused-token IG with that baseline equals abundance-source IG under additive fusion. That's clean, self-contained, and I don't see it in the cited literature. It gives practitioners a principled way to run signed attribution on this class of models, and the connection to FT-Transformer's tokenizer is sensible. The paper also correctly points out that [CLS] attention cannot separate support for one class from another; that's not a new observation, but it's applied appropriately here.\n\nWhere it gets weaker: the empirical validation. The aggregate attribution means are tiny (overall -0.0011, TP 0.0027, TN -0.0045) and reported without confidence intervals or significance tests. Those numbers wouldn't convince me the model's evidence is cleanly signed. The baseline margin F(S, A0) is never reported, so completeness isn't checked against the actual baseline output. There's one post hoc bottom-5% abundance cutoff, no external validation, and no comparison against another signed attribution method. The paper's own limitations section is honest about not being a clinical benchmark, so that's fair, but the results language still overreaches.\n\nThe bigger issue, which the stress-test correctly flags, is the path semantics. The derivation is mathematically fine, but interpreting the result as 'moving abundance from zero to observed' requires that the straight-line path in embedding space passes through points that behave like abundance variation. Nothing in the paper checks that. A0 is one endpoint; the intermediate points are arbitrary linear combinations of embeddings, and gradients there could be unrepresentative of actual abundance changes. The paper cites UDIG for exactly this concern and defers a tokenizer-aware path to future work, but that means the directional species relationships and the response profiles in Figure 6 are not yet established. I don't think this is fatal to the method—you can add path diagnostics, perturbation tests, or integrate through the abundance encoder—but as it stands the biological interpretation is an assumption.\n\nOverall: the math is solid, the writing is clear, and the citation pattern is normal. This deserves serious peer review, but it needs major revision on the empirical side and a careful statement about what the attributions mean under straight-line interpolation. I'd bring it to a reading group for the derivation, but I wouldn't cite the empirical claims yet.","headline":"The baseline derivation is correct and worth stealing; the empirical case for the biological claims is thin and the straight-line path assumption needs work.","tokens_in":11615,"tokens_out":2471,"would_cite":true,"duration_ms":20904,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Signed attribution of the IBD–healthy margin replaces unsigned attention in feature-tokenized transformer explanations.","keywords":["Integrated Gradients","feature-tokenized transformer","microbiome","attention explanation","IBD classification","signed attribution","baseline selection","BiomeGPT"],"falsifier":"A concrete test: on the same trained classifier, compute abundance attributions with the source-derived baseline, with an all-zero fused-token baseline, and with an integration path that runs through the learned abundance encoder rather than straight-line in embedding space; if the sign assignments for the top-ranked species or the aggregate true-positive/true-negative mean signs are not stable across baselines and paths, the claimed directional evidence is an artifact of the chosen baseline rather than a property of the model.","tokens_in":10635,"feed_emoji":"🦠","tokens_out":7019,"duration_ms":53162,"temperature":0.7,"pith_summary":"This paper argues that the widely used [CLS]-attention weights cannot explain decisions of feature-tokenized transformers such as the microbiome model BiomeGPT: attention is nonnegative and operates on fused species-abundance tokens, so a token can receive high attention while pushing the prediction away from the class it seems to support. The author proposes to replace attention with Integrated Gradients applied to the disease-versus-health logit margin $F = z_{\\mathrm{IBD}} - z_{\\mathrm{Healthy}}$, using a token-fusion baseline $T' = S + A_0$ that keeps the species identity fixed and interpolates only the abundance embedding from its learned zero state. Under additive token fusion this makes fused-token attribution equivalent to abundance-source attribution and yields signed values: positive supports IBD, negative supports healthy. If correct, the approach gives feature-tokenized transformers a principled, signed explanation that distinguishes disease-driving from protective evidence, and it motivates second-order Integrated Hessians for species-abundance interaction analysis.","feed_headline":"Signed gradients split microbiome evidence into disease vs health","feed_subtitle":"A species-preserving baseline lets Integrated Gradients sign each abundance's effect on the disease-health margin.","key_machinery":"The load-bearing object is the source-derived baseline $T' = S + A_0$, with $A_0 = \\mathrm{MLP}_{AB}(0)$ the learned zero-abundance embedding. Integrated Gradients of the margin $F(S,A) = z_{\\mathrm{IBD}} - z_{\\mathrm{Healthy}}$ are computed along the interpolation $S_\\alpha = S$, $A_\\alpha = A_0 + \\alpha(A - A_0)$; the identity-map fusion makes $\\partial F/\\partial S = \\partial F/\\partial A = \\partial F/\\partial T$ along the path, so the displacement factor $(A - A_0)$ alone distinguishes abundance-source attribution, and fused-token IG collapses to abundance IG. The species embedding is never removed; it stays in the forward pass and conditions the gradients, while the reported $G_A = \\int_0^1 (\\partial F/\\partial A)(S, A_\\alpha)\\,d\\alpha$ offers a non-complete sensitivity diagnostic that is not weighted by abundance displacement.","core_discovery":"The central claim is that signed attribution of the IBD–healthy margin recovers directional species-abundance evidence that unsigned attention cannot express. In the author's construction, the baseline is derived from the token sources rather than set to zero: with $T = S + A$ and $A_0 = \\mathrm{MLP}_{AB}(0)$, the baseline is $T' = S + A_0$, so the path moves abundance from its zero state to its observed value while the species embedding remains fixed. Because both sources enter the fusion additively with identity Jacobians, the path-averaged gradients of $F$ with respect to $S$ and $A$ coincide, and the fused-token IG equals the abundance-source IG; species attribution is zero by construction yet species identity conditions every gradient along the path. Empirically the author reports that across 7,382 samples true-positive samples carry positive mean attribution and true-negative samples negative mean attribution, and that several of the highest-[CLS]-attention species are among the strongest health-supporting signals rather than IBD drivers. The paper also introduces path-averaged abundance sensitivity $G_A$ and abundance-attribution response profiles as diagnostics.","pith_inferences":["If the additive-fusion equivalence is generic, then every additively fused feature-tokenized model inherits the same baseline rule: keep the feature's learned zero-value embedding fixed while preserving the feature identity, making the method portable beyond microbiome data.","The observation that high-attention species are often health-supporting suggests that attention-based biomarker extraction in other disease classifiers may systematically invert rankings; this is testable by rerunning the same signed-margin analysis on other fine-tuned feature-tokenized models.","The use of embedding-space interpolation is the delicate step; a tokenizer-aware path through the abundance encoder would give a direct robustness check and may change sign assignments for rare species.","The proposed second-order Integrated Hessian score $\\Gamma^A_{ij}$ could be computed per species pair as a community-interaction map, offering a model-based alternative to correlation networks for candidate microbial interactions."],"forward_implications":["Attention rankings should not be read as disease-driver rankings in feature-tokenized models, because the paper's results show top-attention species can be health-supporting.","The signed margin attribution gives each species-abundance observation a direction relative to the IBD–healthy decision, enabling lists of IBD-supporting versus health-supporting taxa from a single trained model.","Path-averaged abundance sensitivity $G_A$ can surface low-abundance species with high per-unit influence that full IG ranks low and attention ignores entirely.","Abundance-attribution response profiles show non-monotonic and context-dependent species effects, motivating segmented or path-resolved IG as the next diagnostic.","The baseline construction transfers to any additively fused feature-tokenized transformer, where the fixed identity embedding is preserved and only the value embedding is interpolated."],"supporting_citations":[{"why":"Supplies the Integrated Gradients definition, the completeness property, and the 'absence of signal' baseline criterion that the paper builds on.","marker":"[14]"},{"why":"Introduces feature tokenization with additive identity-plus-value embeddings, giving the pattern the source-derived baseline generalizes.","marker":"[5]"},{"why":"Defines the BiomeGPT microbiome transformer and its [CLS]-attention interpretation, the object and limitation this paper addresses.","marker":"[10]"},{"why":"Documents faithfulness violations of attention-based explanations, grounding the claim that attention cannot express polarity.","marker":"[9]"},{"why":"Provides the multi-input Integrated Gradients implementation used to compute the reported attributions and convergence deltas.","marker":"[8]"},{"why":"Extends Integrated Gradients to pairwise interaction attribution, motivating the suggested second-order Hessian analysis.","marker":"[7]"},{"why":"Notes that straight-line interpolation in embedding space may be poorly matched to tokenized models, the limitation the paper flags for future tokenizer-aware paths.","marker":"[13]"},{"why":"Distinguishes local gradient sensitivity from full path-integrated attribution, used to frame $G_A$ as a diagnostic rather than an attribution.","marker":"[1]"}],"fun_headline_variants":["Signed gradients reveal disease vs health signals unseen by attention","Species-preserving baseline lets gradients sign microbiome evidence","Integrated gradients split microbial evidence into health vs disease drivers","Signed IG separates protective from pathogenic microbiome signals","Attribution polarity in BiomeGPT: gradients mark disease vs health drivers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on treating the learned zero-abundance embedding $A_0$ as a meaningful 'absence of signal' baseline and on assuming that straight-line interpolation in abundance-embedding space passes through regions where the model's gradients remain semantically interpretable, so a failure of either assumption would strip the signed attributions of biological meaning.","fun_headline_variants_meta":{"raw":{"variants":["Signed gradients reveal disease vs health signals unseen by attention","Species-preserving baseline lets gradients sign microbiome evidence","Integrated gradients split microbial evidence into health vs disease drivers","Signed IG separates protective from pathogenic microbiome signals","Attribution polarity in BiomeGPT: gradients mark disease vs health drivers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001102,"raw_usage":{"total_tokens":4696,"prompt_tokens":1143,"completion_tokens":3553,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":759,"completion_tokens_details":{"reasoning_tokens":3477}},"tokens_in":759,"tokens_out":3553,"duration_ms":23025,"temperature":1.0,"reasoning_tokens":3477,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:33:08.274392+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: on the same trained classifier, compute abundance attributions with the source-derived baseline, with an all-zero fused-token baseline, and with an integration path that runs through the learned abundance encoder rather than straight-line in embedding space; if the sign assignments for the top-ranked species or the aggregate true-positive/true-negative mean signs are not stable across baselines and paths, the claimed directional evidence is an artifact of the chosen baseline rather than a property of the model.","supporting_citations":[{"cited_title":"Revisiting deep learning models for tabular data","cited_arxiv_id":null,"evidence_quote":"Introduces feature tokenization with additive identity-plus-value embeddings, giving the pattern the source-derived baseline generalizes."},{"cited_title":"Medearis, Siyao Zhu, and Ali R","cited_arxiv_id":null,"evidence_quote":"Defines the BiomeGPT microbiome transformer and its [CLS]-attention interpretation, the object and limitation this paper addresses."},{"cited_title":"Rethinking attention- model explainability through faithfulness violation test","cited_arxiv_id":null,"evidence_quote":"Documents faithfulness violations of attention-based explanations, grounding the claim that attention cannot express polarity."},{"cited_title":"Explaining Explanations: Axiomatic Feature Interactions for Deep Networks","cited_arxiv_id":"2002.04138","evidence_quote":"Extends Integrated Gradients to pairwise interaction attribution, motivating the suggested second-order Hessian analysis."},{"cited_title":"Uniform Discretized Integrated Gradients: An effective attribution based method for explaining large language models","cited_arxiv_id":"2412.03886","evidence_quote":"Notes that straight-line interpolation in embedding space may be poorly matched to tokenized models, the limitation the paper flags for future tokenizer-aware paths."}],"review_version":2}