{"id":"78d53020-b41b-4da3-95dd-a841ec6cc603","arxiv_id":"2411.15076","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RankByGene aligns histology image features with gene expression using a ranking-based loss and EMA knowledge distillation, improving gene expression prediction and downstream clinical tasks.","lead":"Spatial transcriptomics measures gene activity at many spots inside tissue, and the new RankByGene method uses this data to teach AI systems to read cancer-related signals directly from standard pathology images. The method consistently improves gene expression prediction and modestly improves slide-level classification and survival analysis across seven public datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ranking loss's target is a co-trained gene encoder; without a stop-gradient, improved rank consistency may reflect gene-feature adaptation rather than image-to-gene alignment. A frozen-gene-encoder retest would settle it.","rationale":"Reader's verdict is CONDITIONAL and mostly on reproducibility and statistical rigor. My stress-test focused on the causal interpretation of the headline PCC gains. The paper's loss in Section 3.2 explicitly uses gene features emitted by a trainable encoder; the paper never states that G is detached in Lrank. Because the same Lrank is minimized by adjusting either modality, the reported rank-consistency improvement is not direct evidence that gene-expression structure was transferred into image features. This is a mechanism-level concern, not an attack: the InfoNCE term prevents trivial collapse, and the end-to-end gene-prediction results are consistent and mostly non-overlapping with baselines. The proposed frozen-gene retest would cleanly separate 'image features learned to reproduce a fixed gene-derived ranking' from 'gene encoder and image encoder co-adapted to a common ranking.' The reader's identified assumption (spreading pressure may distort geometry) is adjacent but less specific; my test targets the non-stationary target directly. If the test passes, the central claim is strengthened and the paper can move toward acceptance; if it fails, the gene-guided interpretation needs qualification.","tokens_in":21078,"tokens_out":12466,"duration_ms":129424,"concrete_test":"Retrain the Breast-ST1 model with Lrank gradients blocked from the gene encoder (equivalently, detach G in Eq. (3)), keeping everything else identical, and compare PCC to Table 1. A second, confirmatory run can replace gene-encoder similarities in Lrank with similarities computed directly on the preprocessed raw gene-expression vectors. If frozen-gene PCC falls by more than the Lrank on/off gap in Table 3c (~0.007 PCC), the ranking benefit depends on co-adaptation of the gene encoder. If PCC is unchanged or improves, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Eq. (4) makes image-feature rankings consistent with gene-expression rankings, so the learned image features are gene-guided. But the 'gene rankings' in Eq. (3) are computed from the output of a 3-layer MLP gene encoder that is trained jointly (Section 3 and Section 8), and no stop-gradient is mentioned for Lrank with respect to that encoder. Thus the constraint SI_p,q - SI_p,r >= SG_p,q - SG_p,r (when SG_p,q > SG_p,r) can be satisfied by moving the gene features instead of, or in addition to, moving the image features. The gene encoder is anchored by InfoNCE, so it cannot collapse, but its similarity geometry is free to drift; rank accuracy in Section 4.3 is even monitored on gene embeddings rather than raw expression values. If a large part of the benefit of Lrank comes from reshaping the gene encoder to be easier to mimic, the method is not purely 'gene-guided' and the comparison to baselines (which use fixed or differently trained gene encoders) is not apples-to-apples. This is a correctness risk in the mechanism, not a disagreement with consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RankByGene, a representation learning framework for aligning spatial transcriptomics (ST) gene expression with histopathology image patches. The method combines three losses: a gene-image InfoNCE contrastive loss for spot-level alignment, a cross-modal ranking consistency loss that encourages the image-feature similarity ranking of spot triplets to match the gene-feature ranking (with the additional requirement that image similarity gaps be at least as large as gene similarity gaps), and an intra-modal teacher-student distillation loss for robustness to augmentation and gene-expression noise. The image encoder is trained on ST data and then evaluated on gene expression prediction (three ST test sets), slide-level classification (BCNB, TCGA-LUAD mutation), and survival analysis (TCGA-BRCA, TCGA-LUAD). The central claim is that the ranking-based alignment produces image features that better reflect gene expression, yielding 9--34% relative PCC improvements over the best baseline in Table 1 and improved downstream performance on most endpoints.","tokens_in":21292,"tokens_out":4485,"duration_ms":43462,"significance":"If the mechanism holds, the ranking-consistency idea is a plausible and potentially useful alternative to direct distance alignment for cross-modal ST-histology learning, and the paper's grounding in cancer-specific gene lists from the Human Protein Atlas is a strength. The experimental work is systematic: consistent improvements on gene expression prediction across three ST test sets, ablations of each loss component and hyperparameter, and evaluation on multiple downstream tasks. The manuscript is generally clearly written, and the computational acceleration of the O(N^3) ranking loss to O(N^2) sampled triplets is practical. However, the central mechanism is not yet fully established: the ranking loss is computed using the output of a jointly trained gene encoder without a stop-gradient, so the improvement could be partly attributable to reshaping the gene-feature geometry rather than to genuinely gene-guided image features. This is a correctness risk that requires an additional experiment, not just a wording change. Reproducibility is also incomplete because no code or gene lists are released.","major_comments":[{"comment":"The ranking loss Lrank is computed using SG values from the same 3-layer MLP gene encoder that is trained jointly with the image encoder, and no stop-gradient is applied to the gene encoder for Lrank. The constraint SI_p,q - SI_p,r >= SG_p,q - SG_p,r can therefore be satisfied by deforming gene-feature similarities instead of improving image-to-gene alignment. The gene encoder is anchored by InfoNCE so it cannot collapse, but its similarity geometry is free to drift, and the rank-accuracy criterion in Section 8 is measured on those gene embeddings. This undermines the central claim that the learned image features are gene-guided. Please add an ablation with a frozen gene encoder (or with gradients detached from the gene encoder for Lrank) and confirm that the improvement in Table 1 persists, or show that the benefit of Lrank is not primarily due to gene-feature adaptation.","section":"Section 3.2, Eq. (4); Section 8"},{"comment":"Rank Accuracy is computed using gene embeddings rather than raw gene expression values and is used as the criterion for model convergence. Because those embeddings are produced by the co-trained gene encoder, this metric is partly self-referential: an increase in Rank Accuracy can reflect changes in gene-feature geometry induced by Lrank rather than a genuine correspondence with the raw transcriptomic signal. Please recompute Rank Accuracy using raw expression profiles or a held-out gene encoder and report it in Figure 6, so that Figure 6 supports the claim that rank relationships in the gene features are captured by the image features.","section":"Section 4.3 and Supplementary Section 8"},{"comment":"The survival and several classification improvements over the strongest baseline are within one standard deviation when the reported error bars are considered. For example, TCGA-BRCA C-Index is 0.681±0.051 for RankByGene versus 0.672±0.039 for BLEEP, and EGFR AUC is 0.855±0.037 for RankByGene versus 0.862±0.028 for BLEEP, with overlapping intervals. The text in Section 4.1 states that RankByGene 'achieves strong AUC performance on most mutation statuses' and 'achieves the highest C-Index'; this overstates the evidence. Please add a statistical significance test (e.g., paired bootstrap over cross-validation folds) or soften the downstream claims and present the gene expression prediction results in Table 1 as the primary evidence for the alignment improvement.","section":"Table 2 and Section 4.1"}],"minor_comments":[{"comment":"The sentence 'we need a loss to enforce ℓ(·) to be nonnega-tive' contradicts the intended goal; it should read 'non-positive', since the hinge loss max{0, ℓ(p,q,r)} penalizes positive values of ℓ.","section":"Section 3.2, text after Eq. (4)"},{"comment":"The caption states 'the gap between gene feature similarities SI_p,q and SI_p,r'; the superscript should be G, not I, to denote gene-feature similarities.","section":"Figure 3 caption"},{"comment":"The cancer-specific gene lists are stated to be available 'upon acceptance' and no code is released; please provide the complete gene lists and code to support reproducibility.","section":"Supplementary Section 6 and Implementation Details"},{"comment":"The abstract says 'seven public datasets,' but the main text reports three ST test sets, two classification sets, and two survival cohorts. Please clarify whether these are seven evaluation datasets (some from the same TCGA/BCNB collections) to avoid ambiguity.","section":"Abstract and Section 4"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the co-trained gene encoder without a stop-gradient: if the frozen-gene retest does not reproduce the reported gains, the mechanism claim would need substantial revision. I recommend requesting that experiment explicitly. The paper is otherwise competently executed and the gene expression prediction results are promising; the downstream claims should be matched to the statistical strength of the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: RankByGene delivers consistently better gene expression prediction numbers than the baselines on the breast and lung ST datasets tested, and the ablations show both the ranking loss and distillation contribute. What it does not deliver is proof of the mechanism it claims.\n\nWhat is actually new: the specific combination of a cross-modal ranking loss with a gene-gap margin and EMA teacher-student distillation for ST image-gene alignment. That combination is not in BLEEP, HEST-FT, or mclSTExp. The experiments are well executed: seven public datasets, standardized preprocessing, externally sourced cancer-specific gene lists from HPA, and ablations over λ1, λ2, and encoder backbones. The paper is clearly written and honestly states its limitations.\n\nThe main soft spot is exactly what the stress-test flags: the gene encoder is trained jointly, and Lrank is optimized over both encoders. No stop-gradient is mentioned, so the ranking consistency can be satisfied in part by reshaping the gene encoder's similarity geometry instead of pulling the image features toward a fixed gene structure. Rank accuracy is even computed on gene embeddings, not raw expression values, which makes the metric sensitive to co-adaptation. This doesn't undo the empirical gains, but it weakens the claim that the image features are gene-guided in the sense argued. A frozen-gene-encoder retest or a stop-gradient variant would settle it.\n\nSecond soft spot: the clinical endpoints are oversold. In Table 2b the survival C-index improvements are within one standard deviation of the baselines (e.g., 0.681±0.051 vs 0.672±0.039 for TCGA-BRCA), and no significance tests are reported. The same is true for several of the mutation AUCs. Keep the strong claim to gene expression prediction; the slide-level transfer results are suggestive, not demonstrated.\n\nMinor but worth fixing: Section 3.2 says 'nonnega-tive' where it should say 'non-positive'. The math is coherent once that typo is ignored. The complete gene lists and code are not released yet; for a subfield built on reproducible foundations, that should be a condition of acceptance.\n\nRecommendation: send it to peer review. This is a solid, well-executed contribution with one mechanistically unresolved question that a good referee can turn into a decisive ablation. With the frozen-gene-encoder experiment and a more careful hedge on the clinical endpoints, the paper would be substantially stronger.","headline":"Genuine improvements on gene expression prediction, but the co-trained gene encoder leaves the 'gene-guided' mechanism underdetermined until a frozen-encoder ablation is shown.","tokens_in":21890,"tokens_out":3737,"would_cite":true,"duration_ms":33244,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RankByGene claims that transferring similarity rankings from gene expression space to histology image space, rather than matching exact distances, yields image features that align with gene activity and improve slide-level clinical…","keywords":["spatial transcriptomics","histopathology image analysis","cross-modal representation learning","ranking loss","knowledge distillation","gene expression prediction","whole slide images","multi-modal alignment"],"falsifier":"Shuffle the pairing between tissue-spot images and their gene profiles during training; if RankByGene's Pearson correlation gains over contrastive-only training persist under shuffled pairing, the ranking loss is not actually transferring gene information and the alignment claim fails.","tokens_in":1678,"feed_emoji":"🧬","tokens_out":3607,"duration_ms":92846,"temperature":0.7,"pith_summary":"The paper tries to establish that the right way to align spatial transcriptomics with histology images is to match the ranking of similarities between tissue spots across modalities, not the distances themselves. It introduces RankByGene, a training objective that combines a gene-image contrastive loss for local spot matching with a cross-modal ranking consistency loss that enforces, for sampled triples of spots, that image-feature similarity gaps are at least as large as gene-feature similarity gaps. A teacher-student distillation loss stabilizes this alignment against the sparsity, noise, and high dimensionality of gene expression. If the claim holds, image features learned from paired gene-image data can capture functional, gene-related visual cues, improving gene expression prediction and slide-level clinical tasks on image-only cohorts.","feed_headline":"Ranking loss lifts gene–image correlation up to 35 percent","feed_subtitle":"A ranking-based loss plus teacher–student distillation makes histology models predict gene activity and survival better.","key_machinery":"The load-bearing mechanism is the cross-modal ranking consistency loss. For each anchor tissue spot $p$ and pairs $(q,r)$, it defines $\\ell(p,q,r) = \\operatorname{sign}(S^G_{p,q}-S^G_{p,r})\\cdot((S^G_{p,q}-S^G_{p,r})-(S^I_{p,q}-S^I_{p,r}))$ and penalizes positive values with a hinge loss over all sampled triplets. This is a modified ordinal ranking loss: beyond requiring the image similarities to preserve the gene-similarity order, it pushes the image-feature gap to meet or exceed the gene-feature gap, which the authors argue accelerates alignment and counteracts the tendency of image features to be tightly packed. The ranking loss is combined with an InfoNCE gene-image contrastive loss and an intra-modal distillation loss in which a teacher encoder processes weakly augmented patches and a student processes strongly augmented ones, with teacher weights updated by exponential moving average.","core_discovery":"The paper's central discovery is that cross-modal alignment improves when similarity rankings are transferred from gene space to image space at both local and global scales, instead of relying on point-wise or pair-wise distance matching. Concretely, RankByGene outperforms the best prior baseline on gene expression prediction with Pearson correlation improvements of 9% to 34% on the top 250 highly expressed genes and 7% to 35% on the top 250 cancer-specific genes across breast and lung datasets, and it achieves the highest AUC on most mutation-status classification endpoints and the highest C-Index in both survival cohorts tested. The learned teacher image encoder is the only component used at inference, so the benefit transfers to image-only histopathology data.","pith_inferences":["If the ranking loss works by transferring ordinal structure rather than exact distances, the same objective could be applied to other paired modalities where one side is sparse or noisy, such as aligning histology with proteomics or methylation arrays, as long as a meaningful similarity order exists.","The reported gains might partly come from the gap-enlargement term acting as a feature-space regularizer rather than from alignment per se; a controlled comparison against a ranking loss that only enforces order, without the gene-gap target, would separate the two effects.","The method's transfer promise suggests a testable pathway: fine-tune on one organ's spatial transcriptomics pairs and evaluate whether the learned image encoder improves gene-guided tasks on other organs; the authors note only breast and lung have been tested, so the natural next experiment is brain, kidney, or skin data.","Because the teacher encoder starts from a foundation model and is fine-tuned with gene ranking signals, the resulting representation could serve as a drop-in initializer for weakly supervised whole slide models, potentially improving tasks where molecular subtype is the label but only slides are available."],"forward_implications":["Gene expression prediction from histology improves: PCC rises by 9% to 34% for the top 250 highly expressed genes and 7% to 35% for the top 250 cancer-specific genes over the best baseline on breast and lung test slides.","Slide-level classification improves on most endpoints: RankByGene achieves the highest AUC on BCNB receptor-status classification and on most TCGA-LUAD mutation-status tasks in the paper's comparisons.","Survival prediction improves: RankByGene reaches a C-Index of 0.681 on TCGA-BRCA and 0.595 on TCGA-LUAD, the highest among the compared image encoders.","The benefit survives transfer to image-only cohorts: at inference only the teacher image encoder is used, so the alignment learned from spatial transcriptomics pairs can be applied to whole slide images without gene data.","The ranking loss adds value beyond local contrastive alignment: the paper's ablation on Breast-ST1 shows the best PCC (0.1874) requires both the ranking loss and the distillation loss, versus 0.1419 with neither and intermediate values with only one."],"supporting_citations":[{"why":"Supplies the HEST-1k breast and lung spatial transcriptomics datasets and the HEST-FT baseline and preprocessing protocol the method builds on.","marker":"[32]"},{"why":"BLEEP is the strongest gene-image pretraining baseline; its MLP gene encoder, cosine similarity, and evaluation setup are adopted.","marker":"[77]"},{"why":"UNI initializes the image encoder backbone for both teacher and student networks.","marker":"[15]"},{"why":"InfoNCE is the contrastive objective used for local gene-image spot alignment and within the distillation loss.","marker":"[54]"},{"why":"Provides the classic ordinal ranking loss that the cross-modal ranking consistency loss modifies.","marker":"[10]"},{"why":"DINO-style self-supervised distillation motivates the teacher-student architecture with weak and strong augmentations and EMA updates.","marker":"[11]"},{"why":"Establishes the gene expression prediction task and the top-250 highly expressed gene evaluation protocol used for comparison.","marker":"[17]"},{"why":"ABMIL is used to aggregate patch features for the slide-level classification and survival evaluations.","marker":"[30]"}],"fun_headline_variants":["RankByGene: ranking loss lifts gene–image alignment","Gene–image ranking beats point-wise matching in histology","Teacher-student distillation sharpens spatial transcriptomics","Ranking consistency improves gene prediction and survival","RankByGene: up to 35% better gene expression prediction"],"cache_read_input_tokens":23936,"weakest_assumption_plain":"The method's gains rest on the assumption that making image-feature similarity gaps at least as large as gene-feature similarity gaps, for sampled groups of three tissue spots, improves rather than distorts the learned image representation; the paper supports this only empirically on breast and lung data.","fun_headline_variants_meta":{"raw":{"variants":["RankByGene: ranking loss lifts gene–image alignment","Gene–image ranking beats point-wise matching in histology","Teacher-student distillation sharpens spatial transcriptomics","Ranking consistency improves gene prediction and survival","RankByGene: up to 35% better gene expression prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000138,"raw_usage":{"total_tokens":1101,"prompt_tokens":841,"completion_tokens":260,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":182}},"tokens_in":457,"tokens_out":260,"duration_ms":3482,"temperature":1.0,"reasoning_tokens":182,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:33:37.465884+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle the pairing between tissue-spot images and their gene profiles during training; if RankByGene's Pearson correlation gains over contrastive-only training persist under shuffled pairing, the ranking loss is not actually transferring gene information and the alignment claim fails.","supporting_citations":[{"cited_title":"Hest-1k: A dataset for spatial transcriptomics and his- tology image analysis","cited_arxiv_id":null,"evidence_quote":"Supplies the HEST-1k breast and lung spatial transcriptomics datasets and the HEST-FT baseline and preprocessing protocol the method builds on."},{"cited_title":"Spatially resolved gene expression prediction from histology images via bi- modal contrastive learning","cited_arxiv_id":null,"evidence_quote":"BLEEP is the strongest gene-image pretraining baseline; its MLP gene encoder, cosine similarity, and evaluation setup are adopted."},{"cited_title":"Accurate spatial gene expression prediction by integrating multi-resolution features","cited_arxiv_id":null,"evidence_quote":"Establishes the gene expression prediction task and the top-250 highly expressed gene evaluation protocol used for comparison."},{"cited_title":"Attention-based deep multiple instance learning","cited_arxiv_id":null,"evidence_quote":"ABMIL is used to aggregate patch features for the slide-level classification and survival evaluations."}],"review_version":1}