REVIEW 7 cited by
Multimodal Prototyping for cancer survival prediction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multimodal survival methods combining gigapixel histology whole-slide images (WSIs) and transcriptomic profiles are particularly promising for patient prognostication and stratification. Current approaches involve tokenizing the WSIs into smaller patches (>10,000 patches) and transcriptomics into gene groups, which are then integrated using a Transformer for predicting outcomes. However, this process generates many tokens, which leads to high memory requirements for computing attention and complicates post-hoc interpretability analyses. Instead, we hypothesize that we can: (1) effectively summarize the morphological content of a WSI by condensing its constituting tokens using morphological prototypes, achieving more than 300x compression; and (2) accurately characterize cellular functions by encoding the transcriptomic profile with biological pathway prototypes, all in an unsupervised fashion. The resulting multimodal tokens are then processed by a fusion network, either with a Transformer or an optimal transport cross-alignment, which now operates with a small and fixed number of tokens without approximations. Extensive evaluation on six cancer types shows that our framework outperforms state-of-the-art methods with much less computation while unlocking new interpretability analyses.
Forward citations
Cited by 7 Pith papers
-
AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis
An adaptive Mamba architecture with dynamic cross-modal weighting and semantic reordering of WSI patches reports ~2% C-Index gains over baselines on five TCGA datasets.
-
Pathway-Structured Privileged Distillation for Deployable Computational Pathology
MoPE is a privileged distillation framework that transfers RNA-derived pathway supervision to histology experts via memory-usage alignment, improving whole-slide image only inference on cancer benchmarks.
-
Bridging the Modality Bottleneck in Pathology MIL through Virtual Molecular Staining
MIST augments MIL projection layers with cross-modal gene-expression prototypes derived from spatial transcriptomics, yielding consistent gains on survival, subtyping, and biomarker tasks across 23 endpoints and 8 agg...
-
Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysis
SlotSPE is a slot-attention framework that decomposes multimodal cancer data into structural prognostic event slots to improve survival prediction and interpretability.
-
Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis
The paper introduces a shuffling-equivalence regularizer for MIL-based WSI analysis, claiming consistent gains over state-of-the-art methods.
-
HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning
HeLo fuses physiological and behavioral features with optimal transport and label-correlation-driven attention, reporting the best average rank on DMER and WESAD emotion distribution benchmarks.
-
CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment
CIGTSurv uses clinical text embeddings to guide cross-attention and distribution alignment between pathology and genomics, reaching an average C-index of 0.788 across five TCGA cohorts.
Discussion (0). Sign in to comment.