REVIEW 6 cited by
Multimodal Prototyping for cancer survival prediction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Multimodal survival methods combining gigapixel histology whole-slide images (WSIs) and transcriptomic profiles are particularly promising for patient prognostication and stratification. Current approaches involve tokenizing the WSIs into smaller patches (>10,000 patches) and transcriptomics into gene groups, which are then integrated using a Transformer for predicting outcomes. However, this process generates many tokens, which leads to high memory requirements for computing attention and complicates post-hoc interpretability analyses. Instead, we hypothesize that we can: (1) effectively summarize the morphological content of a WSI by condensing its constituting tokens using morphological prototypes, achieving more than 300x compression; and (2) accurately characterize cellular functions by encoding the transcriptomic profile with biological pathway prototypes, all in an unsupervised fashion. The resulting multimodal tokens are then processed by a fusion network, either with a Transformer or an optimal transport cross-alignment, which now operates with a small and fixed number of tokens without approximations. Extensive evaluation on six cancer types shows that our framework outperforms state-of-the-art methods with much less computation while unlocking new interpretability analyses.
Forward citations
Cited by 6 Pith papers
-
AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis
An adaptive Mamba architecture with dynamic cross-modal weighting and semantic reordering of WSI patches reports ~2% C-Index gains over baselines on five TCGA datasets.
-
How Effective Can Dropout Be in Multiple Instance Learning ?
Dropping the top-k most important instances in a bag regularizes MIL training and improves classification; the proposed MIL-Dropout applies this idea to existing MIL aggregators.
-
Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis
The paper introduces a shuffling-equivalence regularizer for MIL-based WSI analysis, claiming consistent gains over state-of-the-art methods.
-
HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning
HeLo fuses physiological and behavioral features with optimal transport and label-correlation-driven attention, reporting the best average rank on DMER and WESAD emotion distribution benchmarks.
-
Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learning
A transformer-based temporal attention network that integrates longitudinal blood, imaging, and medication data predicts mortality in immunotherapy patients with AUCs around 0.81 to 0.84, slightly beating blood-only models.
-
CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment
CIGTSurv uses clinical text embeddings to guide cross-attention and distribution alignment between pathology and genomics, reaching an average C-index of 0.788 across five TCGA cohorts.
Discussion (0). Continue with ORCID to comment.