REVIEW 3 major objections 6 minor 6 references
Assessing microbiome engraftment extent following fecal microbiota transplant with q2-fmt
T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read q2-fmt is an open-source plugin that measures how much of a donor gut microbiome persists in a transplant recipient.
desk verdict Useful QIIME 2 plugin for FMT engraftment metrics, but the new permutation test needs validation and the rarefaction ambiguity fixed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the ratio of donor features observed in a recipient sample to the total number of donor features, generalized from an earlier strain-level metric. Around this ratio, the plugin builds a permutation test that resamples mismatched donor-recipient pairs with replacement, rarifies each pair to an even depth, recomputes the ratio, and forms a p-value from the fraction of mismatched pairs whose ratio reaches or exceeds the true pair's value. A second ratio tracks the baseline recipient features still present after transplant, and the newly defined recipient-level metric counts, for each donor feature, the fraction of recipients whose donor carried it and in whom it later appeared. Supporting machinery includes a differential-abundance step for flagging donor indicator features and a plotting pipeline for following these quantities over time.
What would settle it
Run the permutation test on synthetic data in which no true donor-recipient transfer occurs; if the resulting p-values are not roughly uniform, or cluster near zero, the test overstates engraftment.
Extended reading notes
Core claim
On its own terms, the paper claims that a single plugin can cover all three published criteria for FMT engraftment: chimeric asymmetric community coalescence, donated microbiome indicator features, and temporal stability. To that end it contributes six actions and two pipelines: community-coalescence visualizations, calculation of the proportional engraftment of donor features with a mismatched-donor permutation test, persistent-recipient-feature calculation, a new recipient-level donor-feature metric, donor-indicator detection through differential abundance, and heatmap and raincloud visualization. The load-bearing new piece is the permutation test: actual donor-recipient feature-sharing values are compared against values computed from donor-recipient pairs that are not actually matched, yielding a p-value for whether observed sharing exceeds chance. The paper also defines a metric equal to the fraction of recipients in which a donor feature appears when the donor carries it, as a way to rank which microbes are most transplantable.
Load-bearing premise
The weakest load-bearing premise is that the new permutation test's null distribution, built from mismatched donor-recipient pairs, truly represents chance-level feature sharing; the preprint reports no simulation or gold-standard check of this assumption.
Editorial extensions
If this is right
- Engraftment results from different FMT studies can be placed on the same quantitative footing, because the suite fixes the definitions of donor-feature sharing, persistence, and temporal stability.
- A significant p-value from the mismatched-donor permutation test gives a study-specific check that an observed donor-feature proportion is higher than would be expected from coincidental sharing alone.
- The new recipient-level metric yields a per-feature engraftment rate across recipients, so researchers can identify which donor microbes are most likely to take hold and track whether that changes over time.
- Longitudinal visualizations make it possible to watch engraftment, detect whether an individualized microbiome re-emerges years later, and separate treatment failure from failure to engraft.
Reading between the lines
- The same ratio-and-permutation machinery could be adapted to quantify engraftment in other microbial transfer settings, such as vaginal or skin transplants, where a donor community is introduced into a recipient.
- If the recipient-level feature metric proves stable across studies, it could be used to screen donor candidates by preferentially selecting donors whose features have high historical engraftment rates.
- The permutation test's validity could be checked retrospectively on existing FMT datasets by comparing its p-values against strain-level transmission calls from a gold-standard method; that comparison is not made in the paper.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents q2-fmt, a QIIME 2 plugin for quantifying microbiome engraftment after fecal microbiota transplant (FMT). It implements three conceptual criteria from a companion review: Chimeric Asymmetric Community Coalescence (the cc pipeline), Donated Microbiome Indicator Features (PEDF, PRDF, and detect-donor-indicators), and Temporal Stability (through longitudinal analysis). The plugin introduces a new permutation test (pedf-permutation-test) intended to distinguish true donor-to-recipient feature transfer from features shared by chance. The paper describes the design, actions, unit tests, and availability of the plugin, and illustrates the cc pipeline on an autologous FMT dataset from Taur et al. (2018).
Significance. If the plugin works as described, it would provide the microbiome field with a standardized, reproducible toolkit for FMT engraftment assessment, which is currently lacking. The paper's strengths include the use of the QIIME 2 framework with provenance tracking, unit tests run on every commit, open-source BSD-3 licensing, and worked documentation on real data. The arithmetic for PEDF, PRDF, and PPRF is straightforward and internally consistent, and these metrics are clearly tied to prior work by Aggarwala et al. However, the central novel contribution—the permutation test—is presented without any statistical validation, and the manuscript text contains an asymmetry in how actual versus mismatched donor-recipient pairs are treated with respect to rarefaction. These issues need to be addressed before the claimed significance check can be considered reliable.
major comments (3)
- [Proportional Engraftment of Donor Features (PEDF), pedf-permutation-test] The permutation test as described rarefies mismatched donor-recipient pairs to a user-specified even sampling depth, but the sentence 'PEDF is computed for each actual donor-recipient pair' does not state that the actual pairs are rarefied to the same depth. Since PEDF is the proportion of donor features observed in a recipient sample, an unrarefied recipient sample with higher sequencing depth can spuriously elevate PEDF, biasing the permutation p-value toward significance. This is a load-bearing procedural detail for the central statistical claim. Please clarify whether actual pairs are also rarefied to the same depth; if they are, state this explicitly, and if they are not, rarefy them or otherwise correct for depth, and validate the procedure with simulations.
- [Proportional Engraftment of Donor Features (PEDF), pedf-permutation-test] The novel permutation test is presented as a significance check for genuine donor feature transfer, but the manuscript provides no statistical validation: no simulations to verify Type I error control, no power analysis, no comparison against a gold-standard or biological expectation, and no real-data example showing the null distribution. The unit tests confirm that the code runs, but they do not establish that the test answers the scientific question correctly. Given that this is the paper's main new algorithmic contribution, the authors should either add a benchmark demonstrating the test's validity or revise the claims to avoid asserting statistical validity without support.
- [Proportional Engraftment of Donor Features (PEDF), pedf-permutation-test] The p-value calculation is described ambiguously: 'the number of mismatched donor pairs that achieved a PEDS score greater than or equal to the actual pair plus one is divided by num_resamples plus one.' It is unclear whether '+1' applies to the score threshold or to both numerator and denominator as a Monte Carlo correction, and 'PEDS' is an undefined acronym at this point in the text. For a statistical procedure, the exact definition of the p-value must be stated unambiguously and without typographical variants; otherwise the procedure is not reproducible from the manuscript.
minor comments (6)
- [Proportion of Recipients with Donor Feature (PRDF)] The displayed PRDF equation is garbled in the manuscript text, appearing as a fraction whose denominator is itself a fraction; the prose description is clear, but the equation should be properly typeset as number of recipients in which the feature engrafted divided by number of recipients whose donor contained the feature.
- [Figure 1] The caption says 'Jaccard Distance to Donor following FMT intervention' while the main text describes raincloud plots and associated statistics; please clarify whether the figure is a raincloud plot or another visualization, and add uncertainty intervals or point to the interactive version in the documentation.
- [Design and Implementation, PEDF] In the pedf-permutation-test description, 'PEDS' is used instead of 'PEDF' in the sentence about the permutation p-value; this is confusing because PEDS is a strain-level metric from Aggarwala et al. and the plugin's action is called PEDF.
- [Availability and Future Directions] The sentence 'qiimefmtpedfis similarly able to be visualized' in the PPRF paragraph is missing a space and should read 'qiime fmt pedf'; this appears to be a typesetting artifact but should be corrected.
- [Results] The paper states that q2-fmt is a 'comprehensive plugin providing 6 actions and 2 pipelines' but does not provide a complete list of these actions and pipelines in one place; a summary table of actions, inputs, outputs, and corresponding manuscript sections would greatly improve usability and verifiability.
- [Temporal Stability] The claim that 'all actions' in q2-fmt investigate temporal stability through longitudinal data analysis is a stretch, because PEDF, PRDF, and PPRF can be run on a single post-FMT time point; the text should clarify which actions require longitudinal data and which merely support it.
Circularity Check
No significant circularity: the plugin implements externally defined metrics, and the only self-citation is a minor organizational framework rather than a load-bearing derivation.
full rationale
The derivation chain in this manuscript is mostly definitional and implementational rather than inferential. q2-fmt implements metrics defined elsewhere (PEDS/PPRS from Aggarwala et al. 2021, ANCOMBC from external methods) and defines PEDF, PRDF, and PPRF as simple presence-based proportions. The only load-bearing internal definition is the three-criteria framework, credited to Herman et al. 2024, a same-author literature review; however, the plugin does not derive any numeric result from that review, and the review itself synthesizes external literature. The permutation test compares actual donor-recipient PEDF values to mismatched pairs, which is a falsifiable Monte Carlo procedure rather than a fitted prediction. No parameters are fitted to data used for evaluation, and no output is defined in terms of an input quantity in a way that guarantees the conclusion. The self-citation to Herman et al. 2024 is present but minor; it is not used to forbid alternatives or to supply a uniqueness theorem. A separate correctness concern (whether actual pairs are rarefied to the same depth as mismatched pairs) could affect the validity of the permutation test, but that is a confound, not a circularity, and no reduction by construction is exhibited. Therefore the circularity score is low.
Assumptions & free parameters
free parameters (1)
- num_resamples =
999
assumptions (4)
- domain assumption The three engraftment criteria (Chimeric Asymmetric Community Coalescence, Donated Microbiome Indicator Features, and Temporal Stability) as defined in Herman et al. 2024 are appropriate measures of FMT engraftment.
- domain assumption Rarefying to an even sampling depth is a valid preprocessing step for comparing donor and recipient feature tables.
- standard math The Wilcoxon signed-rank test is appropriate for paired distance comparisons in the cc pipeline.
- domain assumption ANCOMBC correctly identifies differentially abundant features between donors and baseline recipient samples.
Cite this review
Pith. "Pith review of Assessing microbiome engraftment extent following fecal microbiota transplant with q2-fmt." pith.science (2026). https://pith.science/paper/HYQ2X5AM
@misc{pith2026241117901,
author = {Pith},
title = {Pith review of: Assessing microbiome engraftment extent following fecal microbiota transplant with q2-fmt},
year = {2026},
howpublished = {\url{https://pith.science/paper/HYQ2X5AM}},
note = {Machine review of arXiv:2411.17901}
}
read the original abstract
We present q2-fmt, a QIIME 2 plugin that provides diverse methods for assessing the extent of microbiome engraftment following fecal microbiota transplant. The methods implemented here were informed by a recent literature review on approaches for assessing FMT engraftment, and cover aspects of engraftment including Chimeric Asymmetric Community Coalescence, Donated Microbiome Indicator Features, and Temporal Stability. q2-fmt is free for all use, and detailed documentation illustrating worked examples on a real-world data set are provided in the project's documentation.
Reference graph
Works this paper leans on
-
[1]
PathogenandMicrobiomeInstitute,NorthernArizonaUniversity, Flagstaff,AZ,USA.2. SchoolofInformatics,ComputingandCyberSystems,NorthernArizonaUniversity, Flagstaff,AZ,USA Abstract Wepresentq2-fmt,aQIIME2pluginthatprovidesdiversemethodsforassessingtheextentofmicrobiomeengraftmentfollowingfecalmicrobiotatransplant.Themethodsimplementedherewereinformedbyarecentl...
work page 2024
-
[2]
Herman, C. et al. AssessingEngraftment FollowingFecal MicrobiotaTransplant. arXiv[q-bio.QM](2024)
work page 2024
-
[3]
Taur, Y. et al. Reconstitutionof thegut microbiotaof antibiotic-treatedpatientsbyautologousfecal microbiota transplant. Sci. Transl. Med. 10, eaap9489(2018)
work page 2018
-
[4]
Aggarwala, V. et al. Precisequantificationof bacterial strainsafter fecal microbiotatransplantationdelineates long-termengraftment andexplainsoutcomes. NatureMicrobiology6, 1309–1318(2021)
work page 2021
-
[5]
Kang, D.-W. et al. Long-termbenefit of MicrobiotaTransfer Therapyonautismsymptomsandgut microbiota. Sci. Rep. 9, 5821(2019)
work page 2019
-
[6]
Keefe, C. R. et al. FacilitatingbioinformaticsreproducibilitywithQIIME2ProvenanceReplay. PLoSComput. Biol. 19, e1011676(2023). Figure1.JaccardDistancetoDonorfollowingFMTintervention,asassessedusingqiimefmt cc.ThisVisualizationallowsforuserstointeractivelyinvestigatewhetherrecipientmicrobiomesbecomemoresimilartodonormicrobiomeswithtreatment.Inthisfigure,FM...
work page 2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.