REVIEW 3 major objections 5 minor 27 references
Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction
T0 review · 3 major / 5 minor · reviewed 2026-07-11 · grok-4.5
Pith's one-line read A graph model of DCE-MRI trajectories predicts breast cancer complete response better than vision and self-supervised baselines.
desk verdict Solid empirical package on ISPY-2: temporal DAG + GraphSAGE plus three asymmetric trajectory losses that beat re-implemented baselines; absolute numbers stay modest and the asymmetry is intentional but under-tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Asymmetric Treatment Response Loss: three complementary self-supervised terms (population-level alignment that attracts responders while repelling non-responders, patient-level decorrelation of successive embeddings, and temporal consistency that requires the final embedding to match the sum of intermediate latent differences) applied only to responders, together with a temporal DAG whose edges run strictly forward in time.
What would settle it
Retrain and re-evaluate the identical GNN-pCR pipeline on an external multi-center DCE-MRI cohort whose inter-scan intervals vary substantially; if balanced accuracy and AUC fall to the level of the CNN or LSTM baselines, the claim that the trajectory losses capture generalizable longitudinal dynamics is falsified.
Extended reading notes
Core claim
A time-aware GraphSAGE network operating on a directed acyclic graph of ResNet embeddings, trained with three asymmetric trajectory losses (responder alignment, adjacent-timepoint decorrelation, and additive temporal consistency), yields higher 5-fold pCR prediction performance (bACC 0.6844, AUC 0.7203, MCC 0.3561) than vision and self-supervised baselines on 585 ISPY-2 patients, establishing that structured longitudinal graph modeling improves individualized treatment-response prediction.
Load-bearing premise
That the three asymmetric losses learn clinically meaningful disease trajectories rather than artifacts of the homogeneous ISPY-2 scan schedule and the fixed ResNet18 features.
Editorial extensions
If this is right
- Early-response models that use only the first two time points can still outperform concatenation and LSTM baselines, supporting earlier therapy adaptation.
- The same graph-plus-asymmetric-loss recipe can be reused for other longitudinal imaging prediction tasks beyond breast-cancer pCR.
- Public release of the curation library and code enables standardized multi-split benchmarks for future longitudinal pCR methods.
- Including raw inter-scan time gaps yields only modest gains under ISPY-2's near-uniform schedule, suggesting the graph topology already encodes the dominant temporal structure.
Reading between the lines
- Because the alignment loss deliberately omits an attract term for non-responders, the method implicitly treats non-response as a heterogeneous residual class; this design choice may limit calibration when partial responders form a clinically important intermediate group.
- The same DAG-plus-asymmetric-loss construction could be tested on longitudinal CT or PET trajectories in other solid tumors where complete-response surrogates exist.
- If the additive temporal-consistency term is the main driver of early-prediction gains, replacing it with a learned ODE or continuous-time GNN layer would be a direct next experiment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes GNN-pCR, a 3D spatio-temporal framework for predicting pathological complete response (pCR) to neoadjuvant chemotherapy from longitudinal DCE-MRI. Image volumes at each timepoint are encoded by a shared ResNet18 into embeddings that form a directed acyclic graph (edges from every earlier to every later timepoint); GraphSAGE aggregates the graph into a patient-level embedding. Training is driven by an Asymmetric Treatment Response Loss whose three novel terms (population-level responder alignment, adjacent-timepoint decorrelation, and additive temporal consistency of latent differences) are applied only to responders, while non-responders receive only a repel term. On 585 ISPY-2 patients with four timepoints the method reports 5-fold means of bACC 0.6844, AUC 0.7203 and MCC 0.3561, outperforming CNN, CNN+LSTM, DINOv3 and two SSL baselines; ablations, early-timepoint experiments and inter-scan time-difference tests are also provided. Code and a PyPI curation library are promised.
Significance. If the comparative gains hold under external validation, the work supplies a concrete, reproducible imaging-only benchmark for longitudinal pCR prediction and a usable graph-plus-asymmetric-SSL recipe that improves on both supervised sequence models and recent SSL baselines on the same public cohort. The multi-split protocol, shared encoder, Wilcoxon tests, component ablations and planned open-source release are genuine strengths that raise the bar relative to many prior single-split ISPY-2 studies. The clinical claim that the learned representations capture 'clinically meaningful longitudinal' dynamics remains provisional until the asymmetry of the losses and the lack of multi-center data are addressed, but the engineering contribution is already useful to the community.
major comments (3)
- §2.2, Eqs. (1)–(3) and the final L_total formula: L_align, L_decorrelate and L_temporal are all gated by 1{i∈R}; non-responders (381/585) receive only the repel term. The manuscript never reports a symmetric counterpart that applies the same three trajectory objectives to both classes (or to non-responders alone). Without that control it is impossible to know whether the gains in Tables 1–2 arise from genuine trajectory modeling or from a one-sided regularizer that simply densifies the minority class. This is load-bearing for the claim of 'clinically meaningful longitudinal' representations for the full population.
- All results (Tables 1–2) are obtained on a single public multi-center trial (ISPY-2) with a near-homogeneous acquisition schedule. The Discussion itself notes that inter-scan time differences add little value for this reason. External validation on an independent cohort (or at least a leave-one-site-out analysis) is required before the method can be presented as a generalizable pCR benchmark; otherwise the reported margins may be dataset-specific.
- Table 1 ablation 'w/o GNN' replaces GraphSAGE by a linear head and still yields bACC 0.6610 (close to the full model). Combined with the fact that the three losses already operate on the same embeddings, it remains unclear how much of the lift is attributable to relational message passing versus the asymmetric SSL objectives alone. A clearer isolation (e.g., GNN trained only with supervised loss, or losses applied after mean-pooling) would strengthen the architectural claim.
minor comments (5)
- Abstract and p. 8: 'medical imagaging' is a typographical error; correct to 'imaging'.
- §3 Implementation Details: DINOv3 is applied to a single 2D axial slice while all other methods use full 3D volumes; the comparison is therefore not modality-matched and should be flagged more explicitly.
- Eq. (1): the random permutation π is described only in text; a short formal definition would improve reproducibility.
- Table 2 caption and body: the inter-scan time-difference experiment is interesting but the near-null result for most methods is under-discussed relative to the claim that temporal modeling is clinically meaningful.
- References [23] and [24] are arXiv preprints; if they remain unpublished at camera-ready, note the version and access date.
Circularity Check
No derivation circularity: GNN-pCR losses and topology are design choices evaluated by held-out 5-fold metrics and ablations, not algebraic restatements of the pCR label.
full rationale
The paper proposes a GraphSAGE head on a temporal DAG of ResNet18 embeddings plus three asymmetric self-supervised losses (population alignment among responders only, adjacent-timepoint decorrelation, additive temporal consistency of latent differences). These are applied only to the responder indicator in L_total and are never claimed to be derived from first principles; they are design choices whose utility is measured by independent 5-fold classification metrics (bACC/AUC/MCC) against vision and SSL baselines and by systematic ablations that remove each term or the GNN. The pCR labels enter only as the supervised evaluation target and as the R/N partition that gates the losses; the losses themselves do not algebraically restate or fit the binary label. Mild author-adjacent re-implementation of 3D-LART and ordinary self-citation of related longitudinal work exist but are not load-bearing for the central performance claim. The asymmetry noted by the skeptic (objectives applied only to responders) is a methodological limitation that may affect generalization claims, but it is not circularity under the definition used here: nothing reduces by construction to its own inputs. Score 1 reflects only the minor self-reference, not any forced prediction.
Assumptions & free parameters
free parameters (3)
- SGD learning rate and schedule =
1e-2, 100 epochs
- ResNet18 embedding dimension D and GraphSAGE aggregation
- Number and selection of DCE-MRI time points (T=4) =
T=4
assumptions (5)
- domain assumption pCR is a clinically meaningful binary surrogate for long-term outcome after NACT
- ad hoc to paper Directed acyclic graph with edges from every earlier to every later time point correctly encodes temporal treatment trajectories
- ad hoc to paper Non-responder embeddings should not be attracted to one another because their clinical outcomes are heterogeneous
- domain assumption GraphSAGE message passing and ResNet18 global-average-pooled features are adequate backbones for 3D DCE-MRI volumes of shape (3,64,64,64)
- domain assumption Stratified 5-fold CV on the 585-patient ISPY-2 subset with complete four time points yields generalizable performance estimates
invented entities (2)
-
Asymmetric Treatment Response Loss (L_align + L_decorrelate + L_temporal applied only to responders; L_repel to non-responders)
-
Temporal DAG patient trajectory graph for DCE-MRI embeddings
Cite this review
Pith. "Pith review of Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction." pith.science (2026). https://pith.science/paper/3WCFM75X
@misc{pith2026260704912,
author = {Pith},
title = {Pith review of: Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/3WCFM75X}},
note = {Machine review of arXiv:2607.04912}
}
read the original abstract
In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2 dataset demonstrate that our method substantially outperforms both vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for dataset curation upon publication, effectively promoting reproducible open-source research.
Figures
Reference graph
Works this paper leans on
-
[1]
Breast cancer: Global patterns of incidence, mortality, and survival
World Health Organization. Breast cancer: Global patterns of incidence, mortality, and survival. https://www.who.int/news-room/fact-sheets/detail/breast-cancer,
-
[2]
Accessed: 2026-02-11
2026
-
[3]
Predicting breast cancer re- sponse to neoadjuvant treatment using multi-feature MRI: results from the I-SPY 2 trail.NPJ Breast Cancer, 6(1):63, 2020
Wen Li, David C Newitt, Jessica Gibbs, Lisa J Wilmes, Ella F Jones, Vignesh A Arasu, Fredrik Strand, Natsuko Onishi, Alex Anh-Tu Nguyen, John Kornak, Laura J Esserman, Don Berry, and Nola M Hylton. Predicting breast cancer re- sponse to neoadjuvant treatment using multi-feature MRI: results from the I-SPY 2 trail.NPJ Breast Cancer, 6(1):63, 2020
2020
-
[4]
Laura M Spring, Geoffrey Fell, Andrea Arfe, Chandni Sharma, Rachel Greenup, Kerry L Reynolds, Barbara L Smith, Brian Alexander, Beverly Moy, Steven J Isakoff, Giovanni Parmigiani, Lorenzo Trippa, and Aditya Bardia. Pathologic complete response after neoadjuvant chemotherapy and impact on breast cancer recurrence and survival: a comprehensive meta-analysis...
2020
-
[5]
Hylton and et al
N.M. Hylton and et al. Locally advanced breast cancer: MR imaging for prediction of response to neoadjuvant chemotherapy-results from ACRIN 6657/ISPY trial. Radiology, 279(1):44–55, 2016
2016
-
[6]
Yunsong Peng et al. Pretreatment DCE-MRI-based deep learning outperforms radiomics analysis in predicting pathologic complete response to neoadjuvant chemotherapy in breast cancer.Frontiers in Oncology, 12:846775, 2022
2022
-
[7]
Deep learning prediction of pathologic complete response in breast cancer using MRI and other clinical data: a systematic review.Tomography, 8(6):2784– 2795, 2022
Nabeeha Khan, Richard Adam, Pauline Huang, Takouhie Maldjian, and Tim Q Duong. Deep learning prediction of pathologic complete response in breast cancer using MRI and other clinical data: a systematic review.Tomography, 8(6):2784– 2795, 2022
2022
-
[8]
Aaquib Syed, Richard Adam, Thomas Ren, Jinyu Lu, Takouhie Maldjian, and Tim Q Duong. Machine learning with textural analysis of longitudinal multi- parametric MRI and molecular subtypes accurately predicts pathologic complete response in patients with invasive breast cancer.PLOS One, 18(1):e0280320, 2023
2023
Show all 27 references
-
[9]
Prediction of pathological complete response to chemotherapy for breast cancer using deep neural network with uncertainty quantification.Medical Physics, 51(12):9385–9393, 2024
Bowen Jing, Kai Wang, Erich Schmitz, Shanshan Tang, Yunxiang Li, You Zhang, and Jing Wang. Prediction of pathological complete response to chemotherapy for breast cancer using deep neural network with uncertainty quantification.Medical Physics, 51(12):9385–9393, 2024
2024
-
[10]
Breast multiparametric MRI for predic- tion of neoadjuvant chemotherapy response in breast cancer: the BMMR2 chal- lenge.Radiology: Imaging Cancer, 6(1):e230033, 2024
Wen Li, Savannah C Partridge, David C Newitt, Jon Steingrimsson, Helga S Marques, Patrick J Bolan, Michael Hirano, Benjamin Aaron Bearce, Jayashree Kalpathy-Cramer, Michael A Boss, et al. Breast multiparametric MRI for predic- tion of neoadjuvant chemotherapy response in breas...
2024
-
[11]
Multimodal deep learning for predicting neoadjuvant treat- ment outcomes in breast cancer: a systematic review.Biology Direct, 20(1):72, 2025
Eriseld Krasniqi, Lorena Filomeno, Teresa Arcuri, Gianluigi Ferretti, Simona Gas- parro, Alberto Fulvi, Arianna Roselli, Loretta D’Onofrio, Laura Pizzuti, Mad- dalena Barba, et al. Multimodal deep learning for predicting neoadjuvant treat- ment outcomes in breast cancer: a sys...
2025
-
[12]
Temporal represen- tation learning of phenotype trajectories for pCR prediction in breast cancer
Ivana Janíčková, Yen Y Tan, Thomas H Helbich, Konstantin Miloserdov, Zsuzsanna Bago-Horvath, Ulrike Heber, and Georg Langs. Temporal represen- tation learning of phenotype trajectories for pCR prediction in breast cancer. In International Conference on Medical Image Computing ...
2025
-
[13]
I-SPY 2 breast dynamic contrast enhanced MRI trial (ISPY2)(version 1).The Cancer Imaging Archive (TCIA), 2022
Wen Li, DC Newitt, J Gibbs, LJ Wilmes, EF Jones, VA Arasu, F Strand, N Onishi, AAT Nguyen, J Kornak, et al. I-SPY 2 breast dynamic contrast enhanced MRI trial (ISPY2)(version 1).The Cancer Imaging Archive (TCIA), 2022
2022
-
[14]
ACRIN 6698/I-SPY2 breast DWI dataset (2021).DOI: https://doi
David C Newitt, Savannah C Partridge, Zheng Zhang, Jessica E Gibbs, Thomas L Chenevert, Mark A Rosen, Patrick J Bolan, Helga Marques, J Romanoff, L Cimino, and Nola M Hylton. ACRIN 6698/I-SPY2 breast DWI dataset (2021).DOI: https://doi. org/10.7937/tcia. kk02-6d95, 2025
2021 doi
-
[15]
Early prediction of neoadjuvant chemotherapy responsebyexploitingatransferlearningapproachonbreastDCE-MRIs.Scientific Reports, 11(1):14123, 2021
Maria Colomba Comes, Annarita Fanizzi, Samantha Bove, Vittorio Didonna, Ser- gioDiotaiuti,DanieleLaForgia,AgneseLatorre,EugenioMartinelli,AriannaMen- cattini, Annalisa Nardone, et al. Early prediction of neoadjuvant chemotherapy responsebyexploitingatransferlearningapproachonb...
2021
-
[16]
M2fusion: Multi-time multimodal fusion for prediction of pathological complete response in breast cancer
Song Zhang, Siyao Du, Caixia Sun, Bao Li, Lizhi Shao, Lina Zhang, Kun Wang, Zhenyu Liu, and Jie Tian. M2fusion: Multi-time multimodal fusion for prediction of pathological complete response in breast cancer. InInternational Conference on Medical Image Computing and Computer-As...
2024
-
[17]
Longitudinal detec- tion of radiological abnormalities with time-modulated LSTM
Ruggiero Santeramo, Samuel Withey, and Giovanni Montana. Longitudinal detec- tion of radiological abnormalities with time-modulated LSTM. InInternational Workshop on Deep Learning in Medical Image Analysis, pages 326–333. Springer, 2018
2018
-
[18]
On tackling domain shift in breast MRI using only publicly-available data: Repro- ducible breast cancer segmentation and pCR prediction
Jessica Kächele, Dimitrios Bounias, Alexandra Ertl, and Klaus Maier-Hein. On tackling domain shift in breast MRI using only publicly-available data: Repro- ducible breast cancer segmentation and pCR prediction. InDeep Breast Workshop on AI and Imaging for Diagnostic and Treatm...
2025
-
[19]
DeepResidualLearning for Image Recognition
KaimingHe,XiangyuZhang,ShaoqingRen,andJianSun. DeepResidualLearning for Image Recognition. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016
2016
-
[20]
Inductive Representation Learning on Large Graphs.Advances in Neural Information Processing Systems, 2017
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive Representation Learning on Large Graphs.Advances in Neural Information Processing Systems, 2017
2017
-
[21]
Neural Message Passing for Quantum Chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural Message Passing for Quantum Chemistry. InInternational Confer- ence on Machine Learning, pages 1263–1272. PMLR, 2017
2017
-
[22]
David C Newitt, Sheye O Aliu, Neil Witcomb, Gal Sela, John Kornak, Laura Esserman, and Nola M Hylton. Real-time measurement of functional tumor volume by MRI to assess treatment response in breast cancer neoadjuvant clinical trials: validation of the aegis SER software platfor...
2014
-
[23]
MONAI: An open-source framework for deep learning in healthcare.arXiv preprint arXiv:2211.02701, 2022
M Jorge Cardoso, Wenqi Li, and Richard Nic Brown. MONAI: An open-source framework for deep learning in healthcare.arXiv preprint arXiv:2211.02701, 2022
2022 arXiv
-
[24]
DINOv3.arXiv preprint arXiv:2508.10104, 2025
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Rama- monjisoa, et al. DINOv3.arXiv preprint arXiv:2508.10104, 2025. Temporal Graph Representation Learning of Patient Trajectories 11
2025 arXiv
-
[25]
SSL-AD: Spatiotemporal self-supervised learning for generalizability and adaptability across alzheimer’s prediction tasks and datasets.arXiv preprint arXiv:2509.10453, 2025
Emily Kaczmarek, Justin Szeto, Brennan Nichyporuk, and Tal Arbel. SSL-AD: Spatiotemporal self-supervised learning for generalizability and adaptability across alzheimer’s prediction tasks and datasets.arXiv preprint arXiv:2509.10453, 2025
2025 arXiv
-
[26]
Temporal representation learning of phenotype trajectories for pCR prediction in breast cancer
cirmuw. Temporal representation learning of phenotype trajectories for pCR prediction in breast cancer. https://github.com/cirmuw/temporal-representation- learning/blob/a1c721872bb6c2caa4dbe7c352cd878e283e3dc8/README.md,
-
[27]
GitHub repository, commit a1c7218, accessed 25 Feb 2026
2026
Reviewed July 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.