pith. sign in

arxiv: 2402.05231 · v2 · submitted 2024-02-07 · 📊 stat.ME · stat.AP

Estimating Fold Changes from Partially Observed Outcomes with Applications in Microbial Metagenomics

David S Clausen , Sarah Teichman , Amy D Willis This is my paper

Pith reviewed 2026-05-24 03:40 UTC · model grok-4.3

classification 📊 stat.ME stat.AP
keywords fold change estimationmetagenomicspartial identifiabilitymicrobial abundancescore testpenalized estimationcolorectal cancercoordinate descent
0
0 comments X

The pith

Imposing interpretable parameter constraints renders fold-change parameters fully identifiable despite unknown sample- and category-specific perturbations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops methods to estimate fold changes in the expected values of multivariate outcomes observed under unknown multiplicative perturbations that vary by sample and by category. This setting commonly appears in metagenomic sequencing where taxon detection is biased relative to true abundances. The base model leaves the target fold changes only partially identifiable. Adding constraints on the parameters achieves full identifiability, while an asymptotically negligible penalty on the estimating function ensures estimators exist even with sparse counts. Algorithms for point estimation and testing under the null are supplied, together with a model-robust score test whose validity holds for small samples and under distributional misspecification, as shown in a colorectal cancer meta-analysis.

Core claim

We consider the problem of estimating fold-changes in the expected value of a multivariate outcome observed with unknown sample-specific and category-specific perturbations. This challenge arises in high-throughput sequencing studies of the abundance of microbial taxa because microbes are systematically over- and under-detected relative to their true abundances. Our model admits a partially identifiable estimand, and we establish full identifiability by imposing interpretable parameter constraints. To reduce bias and guarantee the existence of estimators in the presence of sparse observations, we apply an asymptotically negligible and constraint-invariant penalty to our estimating function.

What carries the argument

The model with a partially identifiable estimand for fold changes under multiplicative perturbations, rendered fully identifiable by the addition of interpretable parameter constraints, together with a penalized estimating function.

If this is right

  • Estimators exist and are consistent even when many observations are sparse or zero.
  • The model-robust score test maintains correct size and power for small sample sizes and under violated distributional assumptions.
  • The penalized estimating function is invariant to the particular choice of constraints.
  • The method recovers microbial associations with colorectal cancer in a meta-analysis setting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same structure of sample- and category-specific multiplicative biases appears in other sequencing modalities such as RNA-seq or single-cell data.
  • External measurements of detection efficiencies could be used to choose or validate the constraints in practice.
  • The approach could be extended to longitudinal or spatial sampling designs that inherit analogous perturbation patterns.

Load-bearing premise

The chosen parameter constraints suffice to separate the fold-change parameters from the unknown sample- and category-specific multiplicative perturbations.

What would settle it

A simulation in which data are generated from a process that violates the imposed constraints, after which the estimator recovers fold changes that deviate from the known truth.

Figures

Figures reproduced from arXiv: 2402.05231 by Amy D Willis, David S Clausen, Sarah Teichman.

Figure 1
Figure 1. Figure 1: Motivated by differential abundance in microbiome studies, we consider the gen [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Q-Q plots comparing empirical quantiles (y-axis) of the robust score (dark blue) [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Empirical power to reject H0 at the 5% level (y-axis) across a range of effect sizes (x-axis), estimated from 500 simulations. β1 = 1 represents an ≈ 3-fold difference in the ratio of true abundances across groups while β1 = 5 represents an ≈ 150-fold ratio. Only valid tests (those that control Type 1 error rates for the given sample size, number of categories and data distributions) are shown. 13 [PITH_F… view at source ↗
Figure 4
Figure 4. Figure 4: 95% Wald confidence intervals for taxon-specific CRC status effects estimated [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Taxa associated with CRC status in adjusted model at FDR level 0.1. Species [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Negative log10 p-values from distinct approaches to differential abundance analy￾ses based on the Wirbel et al. (2019) dataset. The proposed method’s robust score test p￾values are shown on the x-axis, and comparator methods on the y-axis. Wirbel et al. (2019) report some p-values equal to zero; to allow graphing on the log scale, we replace these val￾ues with 10−25. In addition to the missing/uncomputable… view at source ↗
Figure 7
Figure 7. Figure 7: β is only partially identifiable and cannot be estimated on the “absolute” scale without further assumptions (e.g., a reference category that is known to be equal in mean abundance across groups). Full identifiability is established via a constraint function, al￾lowing us to estimate the log-fold differences in true abundances across groups relative to typical differences. The above illustrates the favorab… view at source ↗
read the original abstract

We consider the problem of estimating fold-changes in the expected value of a multivariate outcome observed with unknown sample-specific and category-specific perturbations. This challenge arises in high-throughput sequencing studies of the abundance of microbial taxa because microbes are systematically over- and under-detected relative to their true abundances. Our model admits a partially identifiable estimand, and we establish full identifiability by imposing interpretable parameter constraints. To reduce bias and guarantee the existence of estimators in the presence of sparse observations, we apply an asymptotically negligible and constraint-invariant penalty to our estimating function. We develop a fast coordinate descent algorithm for estimation, and an augmented Lagrangian algorithm for estimation under null hypotheses. We construct a model-robust score test and demonstrate valid inference even for small sample sizes and violated distributional assumptions. The flexibility of the approach and comparisons to related methods are illustrated through a meta-analysis of microbial associations with colorectal cancer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The manuscript develops a statistical framework for estimating fold changes in the expected value of a multivariate outcome observed subject to unknown sample-specific and category-specific multiplicative perturbations. Motivated by microbial metagenomics, the model is shown to be only partially identifiable for the target fold-change parameters; full identifiability is recovered by imposing interpretable constraints on the perturbation terms. An asymptotically negligible, constraint-invariant penalty is added to the estimating function to reduce bias and guarantee existence under sparse data. Coordinate-descent and augmented-Lagrangian algorithms are derived for point estimation and constrained (null) estimation, respectively, together with a model-robust score test whose validity is demonstrated for small samples and under distributional misspecification. The method is illustrated on a meta-analysis of microbial associations with colorectal cancer.

Significance. If the identifiability result, the limiting behavior of the penalized estimator, and the robustness of the score test hold, the work supplies a practical and theoretically grounded approach to fold-change estimation in the presence of systematic detection biases that are ubiquitous in sequencing data. The constraint-based resolution of partial identifiability, the penalty construction that preserves the target parameters, and the fast algorithms constitute concrete methodological contributions that could be adopted in compositional data analysis more broadly.

major comments (2)
  1. [§3] §3, Theorem 1 (identifiability): the proof establishes full identifiability once the stated constraints on the multiplicative perturbations are imposed, but the manuscript does not examine the sensitivity of the resulting fold-change estimates when those constraints are only approximately satisfied; a brief simulation or analytic bound quantifying the resulting bias would directly address the central modeling assumption.
  2. [§5.2] §5.2, score-test simulations: the reported empirical type-I error is close to nominal for n=20 under the chosen misspecifications, yet the power curves are presented for only a single effect size; additional results across a range of effect sizes and sample sizes are needed to substantiate the claim of reliable inference for the small-sample regimes typical in metagenomics.
minor comments (3)
  1. [§4] Eq. (7) and surrounding text: the indexing of the category-specific perturbation parameters is not fully consistent with the earlier definition in §2; a single clarifying sentence would remove ambiguity.
  2. [§6] Figure 3: the color scale for the estimated fold changes is not labeled with numerical values; adding tick marks or a legend would improve readability.
  3. The manuscript does not indicate whether code or simulation scripts are made available; adding a reproducibility statement would strengthen the practical contribution.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive review and recommendation of minor revision. We address each major comment below and will incorporate the suggested additions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [§3] §3, Theorem 1 (identifiability): the proof establishes full identifiability once the stated constraints on the multiplicative perturbations are imposed, but the manuscript does not examine the sensitivity of the resulting fold-change estimates when those constraints are only approximately satisfied; a brief simulation or analytic bound quantifying the resulting bias would directly address the central modeling assumption.

    Authors: We agree that sensitivity to approximate satisfaction of the constraints is a central modeling assumption worth quantifying. In the revision we will add a short simulation study that perturbs the constraints by small amounts (e.g., additive noise on the log-scale perturbations) and reports the resulting bias and MSE in the target fold-change estimates across a range of perturbation magnitudes and sample sizes. revision: yes

  2. Referee: [§5.2] §5.2, score-test simulations: the reported empirical type-I error is close to nominal for n=20 under the chosen misspecifications, yet the power curves are presented for only a single effect size; additional results across a range of effect sizes and sample sizes are needed to substantiate the claim of reliable inference for the small-sample regimes typical in metagenomics.

    Authors: We agree that a broader set of power results will better support the small-sample claims. In the revision we will expand §5.2 to include power curves for multiple effect sizes (e.g., log-fold changes of 0.5, 1.0, 1.5) and additional sample sizes (n=10, 30, 50) under the same misspecification scenarios, thereby providing a more complete picture of test performance in the regimes relevant to metagenomics. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The derivation begins with a model that is explicitly partially identifiable and then imposes separate, interpretable parameter constraints to achieve full identifiability. This is a standard modeling step that does not redefine the target estimand in terms of fitted values or reduce any prediction to a self-citation chain. The abstract and reader's summary locate the technical step in explicit constraints rather than in re-expression of fitted parameters, and no load-bearing equation or uniqueness claim is shown to collapse to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review prevents exhaustive enumeration; the ledger is therefore limited to the elements explicitly named. The central claim rests on (1) the existence of interpretable parameter constraints that restore identifiability and (2) the asymptotic negligibility of the added penalty. No free parameters, invented entities, or additional axioms are visible from the abstract.

axioms (1)
  • domain assumption Interpretable parameter constraints suffice to achieve full identifiability of the fold-change estimand
    Stated in the abstract as the step that converts partial to full identifiability

pith-pipeline@v0.9.0 · 5683 in / 1417 out tokens · 21732 ms · 2026-05-24T03:40:44.235111+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages

  1. [1]

    and Anderson, J.A

    Albert, A. and Anderson, J.A. (1984). On the existence of maximum likelihood estimates in logistic regression models. Biometrika\/ 71\/ (1), 1--10

    work page 1984

  2. [2]

    Barron, J.T. (2019). A general and adaptive robust loss function. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp.\ 4331--4339

    work page 2019

  3. [3]

    and Hochberg, Y

    Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B\/ 57\/ (1), 289--300

    work page 1995

  4. [4]

    Birch, M. (1963). Maximum likelihood in three-way contingency tables. Journal of the Royal Statistical Society Series B: Statistical Methodology\/ 25\/ (1), 220--233

    work page 1963

  5. [5]

    and Greenwood, C.M

    Bull, S.B., Mak, C. and Greenwood, C.M. (2002). A modified score function estimator for multinomial logistic regression in small samples. Computational Statistics & Data Analysis\/ 39\/ (1), 57--74

    work page 2002

  6. [6]

    and Willis, A.D

    Clausen, D.S. and Willis, A.D. (2022). Modeling complex measurement error in microbiome experiments. arXiv preprint arXiv:2204.12733\/

    work page arXiv 2022

  7. [7]

    et al (2018)

    Davis, N.M. et al (2018). Simple statistical identification and removal of contaminant sequences in marker-gene and metagenomics data. Microbiome\/ 6\/ (1), 1--14

    work page 2018

  8. [8]

    et al (2014)

    Fernandes, A.D. et al (2014). Unifying the analysis of high-throughput sequencing datasets: characterizing rna-seq, 16s rrna gene sequencing and selective growth experiments by compositional data analysis. Microbiome\/ 2\/ (1), 1--13

    work page 2014

  9. [9]

    Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika\/ 80\/ (1), 27--38

    work page 1993

  10. [10]

    et al (2005)

    Guo, X. et al (2005). Small-sample performance of the robust score test and its modifications in generalized estimating equations. Statistics in Medicine\/ 24\/ (22), 3479--3495

    work page 2005

  11. [11]

    Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences\/ 186\/ (1007), 453--461

    work page 1946

  12. [12]

    et al (2017)

    Kaul, A. et al (2017). Analysis of microbiome data in the presence of excess zeros. Frontiers in Microbiology\/ 8 , 2114

    work page 2017

  13. [13]

    and Firth, D

    Kosmidis, I. and Firth, D. (2011). Multinomial logit bias reduction via the poisson log-linear model. Biometrika\/ 98\/ (3), 755--759

    work page 2011

  14. [14]

    et al (2021)

    Li, Z. et al (2021). Ifaa: robust association identification and inference for absolute abundance in microbiome analyses. Journal of the American Statistical Association\/ 116\/ (536), 1595--1608

    work page 2021

  15. [15]

    and Peddada, S.D

    Lin, H. and Peddada, S.D. (2020). Analysis of compositions of microbiomes with bias correction. Nature Communications\/ 11\/ (1), 3514

    work page 2020

  16. [16]

    and Anders, S

    Love, M.I., Huber, W. and Anders, S. (2014). Moderated estimation of fold change and dispersion for rna-seq data with deseq2. Genome Biology\/ 15\/ (12), 1--21

    work page 2014

  17. [17]

    et al (2015)

    Mandal, S. et al (2015). Analysis of composition of microbiomes: a novel method for studying microbial composition. Microbial ecology in health and disease\/ 26\/ (1), 27663

    work page 2015

  18. [18]

    and Willis, A.D

    Martin, B.D., Witten, D. and Willis, A.D. (2020). Modeling microbial abundances and dysbiosis with beta-binomial regression. The Annals of Applied Statistics\/ 14\/ (1), 94

    work page 2020

  19. [19]

    and Callahan, B.J

    McLaren, M.R., Willis, A.D. and Callahan, B.J. (2019). Consistent and correctable bias in metagenomic sequencing experiments. eLife\/ 8

    work page 2019

  20. [20]

    et al (2019)

    Milanese, A. et al (2019). Microbial abundance, activity and population genomic profiling with motus2. Nature Communications\/ 10\/ (1), 1014

    work page 2019

  21. [21]

    and Cookson, W.O

    Moffatt, M.F. and Cookson, W.O. (2017). The lung microbiome in health and disease. Clinical Medicine\/ 17\/ (6), 525

    work page 2017

  22. [22]

    et al (2022)

    Nearing, J.T. et al (2022). Microbiome differential abundance methods produce different results across 38 datasets. Nature Communications\/ 13\/ (1), 1--16

    work page 2022

  23. [23]

    and Scott, E.L

    Neyman, J. and Scott, E.L. (1948). Consistent estimates based on partially consistent observations. Econometrica: Journal of the Econometric Society\/ , 1--32

    work page 1948

  24. [24]

    and Wright, S.J

    Nocedal, J. and Wright, S.J. (1999). Numerical optimization . Spinger

    work page 1999

  25. [25]

    and Wright, S.J

    Nocedal, J. and Wright, S.J. (2006). Numerical optimization . Spinger

    work page 2006

  26. [26]

    and Bravo, H.C

    Paulson, J.N., Pop, M. and Bravo, H.C. (2013). metagenomeseq: Statistical analysis for sparse high-throughput sequencing. Bioconductor package\/ 1\/ (0), 191

    work page 2013

  27. [27]

    and Smyth, G.K

    Robinson, M.D., McCarthy, D.J. and Smyth, G.K. (2010). edge R : a bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics\/ 26\/ (1), 139--140

    work page 2010

  28. [28]

    et al (2011)

    Segata, N. et al (2011). Metagenomic biomarker discovery and explanation. Genome Biology\/ 12\/ (6), 1--18

    work page 2011

  29. [29]

    and Morrison, W.J

    Sherman, J. and Morrison, W.J. (1950). Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. The Annals of Mathematical Statistics\/ 21\/ (1), 124--127

    work page 1950

  30. [30]

    and Young, V.B

    Shreiner, A.B., Kao, J.Y. and Young, V.B. (2015). The gut microbiome in health and in disease. Current Opinion in Gastroenterology\/ 31\/ (1), 69

    work page 2015

  31. [31]

    White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica\/ , 1--25

    work page 1982

  32. [32]

    and Willis, A.D

    Williamson, B.D., Hughes, J.P. and Willis, A.D. (2022). A multiview model for relative and absolute microbial abundances. Biometrics\/ 78\/ (3), 1181--1194

    work page 2022

  33. [33]

    et al (2019)

    Wirbel, J. et al (2019). Meta-analysis of fecal metagenomes reveals global microbial signatures that are specific for colorectal cancer. Nature Medicine\/ 25\/ (4), 679--689

    work page 2019

  34. [34]

    and Takeshita, T

    Yamashita, Y. and Takeshita, T. (2017). The oral microbiome and human health. Journal of Oral Science\/ 59\/ (2), 201--206

    work page 2017

  35. [35]

    Young, V.B. (2017). The role of the microbiome in human health and disease: an introduction for clinicians. BMJ\/ 356

    work page 2017