Pith. sign in

REVIEW 3 major objections 5 minor 9 references

YOTO jointly learns a discrete top-k gene subset and a multi-task predictor in one differentiable model, claiming consistent gains in F1 over two-stage baselines and the non-sparse gradient baseline once panels reach 32 genes and above.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 15:10 UTC pith:LBSECSKA

load-bearing objection A plausible method that overclaims: the end-to-end selection idea is sound, but the paper's own AUROC tables contradict the 'consistently outperforms' headline. the 3 major comments →

arxiv 2512.17678 v2 pith:LBSECSKA submitted 2025-12-19 cs.LG cs.AI

You Only Train Once: Differentiable Subset Selection for Omics Data

Daphn\'e Chopard , Jorge da Silva Gon\c{c}alves , Irene Cannistraci , Thomas M. Sutter , Julia E. Vogt This is my paper
classification cs.LG cs.AI
keywords end-to-end feature selectiongene subset selectionsingle-cell RNA-seqdifferentiable rankingmulti-task learningstraight-through estimatorPlackett-Lucesparse selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that sparse, end-to-end gene subset selection is not only feasible but better than two-stage pipelines: a single differentiable model that ranks genes, keeps only the top k in its forward pass, and runs multiple prediction tasks on the same selected genes attains higher F1-scores on single-cell RNA-seq datasets than classical selectors followed by a Random Forest, and than the non-sparse gradient-based baseline. The authors evaluate this on mouse visual-cortex cells and COVID-19 patient PBMCs, using panels from 16 to 256 genes. Across panel sizes of 32 genes and above, YOTO consistently matches or beats all baselines; at 64 genes it also outperforms them on the harder tasks. A controlled ablation attributes the gains to the sparse discrete selection mechanism itself, not to multi-task learning or model capacity. A sympathetic reader would care because biomarker panels that can be measured cheaply and used immediately, without retraining a downstream classifier, would lower the cost of spatial transcriptomics and targeted clinical profiling.

Core claim

YOTO is a single neural architecture that performs feature selection and prediction simultaneously. Each gene gets a learnable score; a differentiable permutation matrix derived from the Plackett–Luce ranking model turns those scores into a soft top-k selection, and a straight-through estimator makes the forward pass use a hard binary mask so that only the k selected genes reach the encoder and task heads. The prediction loss therefore backpropagates through the relaxed ranking, closing a feedback loop where the task guides selection and selection shapes representation. The central discovery claimed is that this closed loop, with exact sparsity during training, yields gene panels whose downs

What carries the argument

YOTO's engine is a differentiable permutation matrix derived from the Plackett–Luce ranking model, which converts learnable per-gene scores into a relaxed top-k selection; a straight-through estimator then forces the forward pass to use a hard binary mask so only k genes enter the shared encoder, while gradients flow back through the relaxed matrix. A temperature schedule sharpens the ranking over training, and k itself is annealed from all genes down to the target panel size. This construction is what lets selection and prediction be optimized together with exact sparsity present at inference.

Load-bearing premise

The load-bearing premise is that gradients computed through the relaxed top-k permutation accurately improve the hard discrete choice; the paper assumes the straight-through estimate of the gradient is a reliable optimizer of the true discrete objective, without measuring the discrepancy between the relaxed and hard selections.

What would settle it

Train YOTO on a synthetic dataset where a known set of 32 ground-truth informative genes is planted among thousands of noise features, keeping the rest of the pipeline unchanged. If the learned subset, evaluated by the same F1 metric, is not statistically distinguishable from a random or uniform top-k subset, or if the gradient signal for a gene outside the top-k does not align with whether that gene could plausibly enter the top-k, then the straight-through feedback loop is not doing the work claimed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • With panels of 32 genes or more on the visual-cortex benchmark, and at 64 genes on the COVID-PBMC benchmark, YOTO's F1 is at or above every baseline tested, and the gap widens as panel size grows.
  • A single training run serves all tasks simultaneously (three per dataset), whereas baselines require three to six separate models for feature selection and prediction per task.
  • No downstream classifier is needed: because the forward pass is truly sparse, the model's own prediction heads directly evaluate the selected panel, eliminating retraining.
  • The ablation with identical architecture, single-task objective, and temperature schedule shows the sparse end-to-end selection module, not model size or multi-task supervision, drives the gains.
  • The learned gene sets are claimed to capture interactions and complementarity among genes, not just individually informative markers, which is why performance improves more steeply as more genes are allowed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct stress test would be to swap the Plackett–Luce permutation for independent Gumbel-Softmax top-k sampling; if performance holds, the exact ranking structure is not the active ingredient, pointing to the sparse mask itself.
  • The straight-through mismatch could be diagnosed by measuring the gradient alignment between the hard-mask selection and the relaxed permutation at identical weights; low alignment would imply the training signal is a noisy proxy for the deployed discrete choice.
  • Since the method's advantage appears only from 32 genes upward, the practical deployment window is spatial transcriptomics or targeted gene panels with at least that many measurements; the paper acknowledges the k=16 regime is not where YOTO wins.
  • Turning the same closed loop toward an unsupervised or semi-supervised objective, such as reconstructing the full transcriptome from the selected genes, could produce interpretable biomarker panels without requiring any labels, an extension the paper does not test.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes YOTO, an end-to-end differentiable framework for joint gene subset selection and multi-task prediction in single-cell transcriptomics. Selection is performed by learning gene importance scores and applying a hard top-k mask in the forward pass while using a relaxed permutation matrix (based on the Plackett-Luce model) in the backward pass, enabling gradient-based training of the discrete selection. The authors evaluate on two datasets (mouse VISp spatial transcriptomics and COVID-PBMC scRNA-seq), comparing against Seurat, mRMR variants, Block HSIC Lasso, and PERSIST, and report F1, accuracy, AUROC, and AUPRC across subset sizes k = 16–256. The central claim is that YOTO "consistently outperforms state-of-the-art baselines" while providing a single-model multi-task solution.

Significance. If the claims were fully supported, the contribution would be practically valuable: a single-stage, sparse, end-to-end feature selector with a built-in classifier eliminates the two-stage pipeline overhead, and the multi-task formulation allows partially labeled datasets to share information. The paper includes a well-motivated ablation isolating the selection module, reports results over three seeds, and examines multiple imbalanced-classification metrics. However, the paper's own tables do not support the "consistently outperforms" claim, and at several evaluated subset sizes and metrics YOTO is clearly worse than baselines. The overstated central claim, together with the lack of any diagnostic for the straight-through gradient approximation, requires revisions before the paper can be accepted.

major comments (3)
  1. [Abstract and §4.4.1, Tables 5 and 6] The abstract's claim that YOTO "consistently outperforms state-of-the-art baselines" is contradicted by the paper's own results. On VISp at k=16 (Table 5) YOTO achieves F1 64.0±4.0 versus PERSIST 74.3±0.8 and mRMR-rf 67.2±1.6; on COVID-PBMC (Table 6) YOTO is second at k=16 (71.5 vs. mRMR-f 78.0) and at k=32 (79.5 vs. 81.7). The text acknowledges only "one notable exception" at small panel sizes, but these are multiple settings where YOTO trails. The claim needs to be weakened to "competitive and often superior at moderate/larger panel sizes," or the experiments and analysis need to be expanded to justify the original wording.
  2. [§4.4.3, Tables 7 and 8] The robustness claim across metrics is undermined by the AUROC results. At k=64 on VISp (Table 7), YOTO's AUROC is 91.9±0.1 while mRMR-rf and Block HSIC Lasso reach 99.9±0.0. On COVID-PBMC (Table 8), YOTO's AUROC is 80.9±0.2 versus 98.3±0.2 for mRMR-f and 95.8±0.1 for PERSIST — a 17-point gap. The text describes this as "slightly lower," which is a mischaracterization. Since AUROC is a standard metric for imbalanced classification and the paper claims robustness across metrics (§4.4.3), the paper must either report these results honestly, provide a concrete reason why AUROC is not informative here beyond a general expectation, or revise the claim.
  3. [§3.1, Eq. (3) and straight-through estimator] The entire method rests on the unpublished assumption that the straight-through estimator (hard forward mask, relaxed permutation matrix in the backward pass) provides a faithful gradient signal for the discrete top-k selection. The paper provides no analysis, diagnostic, or comparison of the learned score rankings against the performance of the hard mask. For example, one could test whether the selected genes at convergence are stable across seeds or whether the relaxed gradients mis-credit non-selected genes; the k=16 underperformance may reflect this mismatch. Since the method is not derivable as a proper estimator for the discrete objective, a correctness-risk concern remains. The authors should add at least one diagnostic or a formal argument that the relaxed gradient correlates with the true discrete selection objective.
minor comments (5)
  1. [Tables 5–8 and Figure 2] The tables and figure use inconsistent formatting: 'celltype5' in the text and 'celltype5' in Table 8 caption, but 'celltype5' in Table 6; also 'cell_types_25' appears as a task name. The method row is labeled 'Ours' in tables but the paper name is YOTO; unify terminology.
  2. [§4.4.1 and Figure 2] Figure 2 omits Seurat and mRMR-f 'due to consistently low performance,' but these baselines appear in Tables 5 and 6 and are discussed. Omitting them makes the figure less complete; it would be clearer to include all baselines or state explicitly that only a subset is shown for visual clarity.
  3. [§3.1, Eq. (2)] The notation π(s) is used both for the differentiable permutation matrix and as an argument in P(π(s)), which is confusing because π(s) is a matrix, not a permutation. Use different symbols, e.g., M(s) for the matrix.
  4. [§4.4.4, Table 4] The ablation says 'temperature parameters' are identical, but no temperature schedule is specified. Provide the annealing schedule or a reference to where it is given, so the equivalence is verifiable.
  5. [Appendix A.2] Tables 7 and 8 are presented as extended results but are crucial for the AUROC discussion in §4.4.3; consider moving them to the main text or explicitly cross-referencing them earlier.

Circularity Check

0 steps flagged

No circularity: YOTO is an empirical learning method whose selection scores are trained and evaluated on held-out data; same-group citations are descriptive, not load-bearing.

full rationale

YOTO's derivation chain is an empirical training procedure, not a set of analytical results that could reduce to their inputs. The learnable gene scores s are parameters; the forward pass applies the hard top-k mask Γ_k(s) (Eq. 3), and the task losses L_t (Eqs. 4-5) backpropagate through the relaxed permutation matrix π(s) (Eq. 1) to update both the encoder and the scores. The selected subset is therefore optimized against prediction loss on training data and evaluated on a held-out 80-20 split, which is standard supervised feature selection rather than circular reasoning. The paper's references to Sutter et al. 2023a/b, which share authors with the present work, are used only to describe the inspiration for the differentiable ranking module ('We employ a differentiable feature selection mechanism, inspired by the Differentiable Random Partition Model (DRPM, Sutter et al., 2023a)'); the implemented permutation matrix is attributed to Grover et al. 2019 and the straight-through estimator to Bengio et al. 2013, both external. No uniqueness theorem, fitted quantity renamed as a prediction, or definitional equivalence is used. The apparent inconsistency between the prose claim of 'consistently outperforming state-of-the-art baselines' and the reported AUROC values (e.g., Table 8 shows YOTO at 80.9±0.2 vs mRMR-f at 98.3±0.2, described as 'slightly lower') is a correctness/overclaiming concern, not a circularity concern, because the reported numbers are empirical results rather than quantities forced by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper introduces no new particles, mediators, or conserved quantities. The free parameters are the learned gene scores and the hand-chosen subset size and temperature schedule. The main implicit axioms are the validity of the differentiable ranking relaxation and the straight-through estimator, plus the preprocessing decisions. These are standard for deep-learning feature selection, but they are load-bearing and largely unverified in the paper.

free parameters (3)
  • gene importance scores s = learned per gene, not reported
    The differentiable ranking selects genes based on a learnable score vector s (Section 3.1). These scores are the core learned parameters that determine the selected subset.
  • subset size k = 16, 32, 64, 128, 256
    The number of selected genes is chosen by hand per experiment and is annealed during training (Section 3.1). Performance depends strongly on k, and the paper's claims are only supported for larger k.
  • temperature schedule tau = not reported
    The ranking relaxation uses a temperature parameter with 'an exponential annealing schedule suggested in Jang et al. (2016)' (Section 3.1), but the exact schedule and hyperparameters are omitted.
axioms (3)
  • standard math The differentiable permutation matrix pi(s) from Grover et al. (2019) provides a valid relaxation of the top-k ranking operator.
    The subset selection builds pi(s) via Eq. 1 and uses it for the mask; this is a known relaxation but its approximation error is not analyzed in the paper.
  • ad hoc to paper The straight-through estimator gives a faithful gradient signal for the discrete selection.
    Section 3.1 uses a binary mask in the forward pass and the relaxed permutation in the backward pass. The paper assumes this mismatch does not derail learning of the correct subset, but provides no bias/variance analysis.
  • domain assumption The HVG preprocessing and binarization choices are appropriate for all methods compared.
    All models use top 5,000 or 10,000 highly variable genes (Section 4.1); VISp data is binarized. This preprocessing is not part of the method and could favor one approach over another, but no sensitivity analysis is given.

pith-pipeline@v1.3.0-alltime-deepseek · 13100 in / 9925 out tokens · 103938 ms · 2026-08-03T15:10:26.792253+00:00 · methodology

0 comments
read the original abstract

Selecting compact and informative gene subsets from single-cell transcriptomic data is essential for biomarker discovery, improving interpretability, and cost-effective profiling. However, most existing feature selection approaches either operate as multi-stage pipelines or rely on post hoc feature attribution, making selection and prediction weakly coupled. In this work, we present YOTO (you only train once), an end-to-end framework that jointly identifies discrete gene subsets and performs prediction within a single differentiable architecture. In our model, the prediction task directly guides which genes are selected, while the learned subsets, in turn, shape the predictive representation. This closed feedback loop enables the model to iteratively refine both what it selects and how it predicts during training. Unlike existing approaches, YOTO enforces sparsity so that only the selected genes contribute to inference, eliminating the need to train additional downstream classifiers. Through a multi-task learning design, the model learns shared representations across related objectives, allowing partially labeled datasets to inform one another, and discovering gene subsets that generalize across tasks without additional training steps. We evaluate YOTO on two representative single-cell RNA-seq datasets, showing that it consistently outperforms state-of-the-art baselines. These results demonstrate that sparse, end-to-end, multi-task gene subset selection improves predictive performance and yields compact and meaningful gene subsets, advancing biomarker discovery and single-cell analysis.

Figures

Figures reproduced from arXiv: 2512.17678 by Daphn\'e Chopard, Irene Cannistraci, Jorge da Silva Gon\c{c}alves, Julia E. Vogt, Thomas M. Sutter.

Figure 1
Figure 1. Figure 1: Overview of YOTO. Our method, YOTO, consists of three main building blocks: (1) a subset selection block (section 3.1), (2) a shared encoder for all tasks and (3) a multi-task learning block (section 3.2). The subset selection blocks a binary mask such that we can select the k most informative genes for a given set of tasks. The shared encoder encodes the k gene values into a latent representation, which i… view at source ↗
Figure 2
Figure 2. Figure 2: Downstream F1-score across different number of selected genes. F1-scores for the cell_types_25 task on the VISp dataset using gene subsets of sizes k ∈ {16, 32, 64, 128, 256}. The Seurat and mRMR-f baselines are omitted due to consistently low performance. size (k = 16), where performance is highly sensitive to individual gene choice, a known challenge when the number of input features is extremely constra… view at source ↗
Figure 3
Figure 3. Figure 3: Comprehensive evaluation using multiple classification metrics. F1-score, Accuracy, AUROC, AUPRC) for the celltype5 task on the COVID-PBMC dataset using a gene subset of size k = 64. Results [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Label distributions for all tasks in the VISp and COVID-PBMC datasets. In the VISp panels, the [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

9 extracted references · 5 linked inside Pith

  1. [1]

    Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

    Pith/arXiv arXiv

  2. [5]

    N-act: An interpretable deep learning model for automatic cell type and salient gene identification

    A Ali Heydari and A Oscar. N-act: An interpretable deep learning model for automatic cell type and salient gene identification. InThe Proceedings of the 2022 International Conference on Machine Learning Workshop on Computational Biology,

    2022

  3. [7]

    The concrete distribution: A continuous relaxation of discrete random variables.arXiv preprint arXiv:1611.00712,

    Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables.arXiv preprint arXiv:1611.00712,

    Pith/arXiv arXiv

  4. [2003]

    URL https://doi.org/10.1109/CSB.2003.1227396

    doi: 10.1109/CSB.2003.1227396. URL https://doi.org/10.1109/CSB.2003.1227396. Aditya Grover, Eric Wang, Aaron Zweig, and Stefano Ermon. Stochastic optimization of sorting networks via continuous relaxations. InInternational Conference on Learning Representations,

    Pith/arXiv arXiv 2003

  5. [2012]

    Model-agnostic interpretability of machine learn- ing.arXiv preprint arXiv:1606.05386,

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Model-agnostic interpretability of machine learn- ing.arXiv preprint arXiv:1606.05386,

    Pith/arXiv arXiv

  6. [2015]

    Comprehensive integration of single-cell data.cell, 177(7):1888–1902,

    Tim Stuart, Andrew Butler, Paul Hoffman, Christoph Hafemeister, Efthymia Papalexi, William M Mauck, Yuhan Hao, Marlon Stoeckius, Peter Smibert, and Rahul Satija. Comprehensive integration of single-cell data.cell, 177(7):1888–1902,

    1902

  7. [2018]

    Categorical reparameterization with gumbel-softmax.arXiv preprint arXiv:1611.01144,

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax.arXiv preprint arXiv:1611.01144,

    Pith/arXiv arXiv

  8. [2019]

    Predictive and robust gene selection for spatial transcriptomics.Nature Communications, 14(1):2091,

    Ian Covert, Rohan Gala, Tim Wang, Karel Svoboda, Uygar Sümbül, and Su-In Lee. Predictive and robust gene selection for spatial transcriptomics.Nature Communications, 14(1):2091,

    2091

  9. [2023]

    Chris H. Q. Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expres- sion data. In2nd IEEE Computer Society Bioinformatics Conference, CSB 2003, Stanford, CA, USA, August 11-14, 2003, pp. 523–529. IEEE Computer Society,

    2003