Pith. sign in

REVIEW 3 major objections 2 minor 56 references

DiffSight-Former predicts future glaucoma onset from sequential fundus images by quantifying structural differences in the optic disc, cup, and retinal vasculature across visits.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-27 17:13 UTC pith:OUMAXCG3

load-bearing objection Solid incremental architecture for longitudinal fundus prediction but label quality and experimental details are the real gatekeepers. the 3 major comments →

arxiv 2606.09140 v1 pith:OUMAXCG3 submitted 2026-06-08 cs.CV

DiffSight-Former: Modeling Structural Differences and Temporal Dynamics for Glaucoma Progression Prediction

Yi Huang , Lei Bi , Jinman Kim This is my paper
classification cs.CV
keywords glaucoma progression predictionsequential fundus imagesstructural difference modelingtime-aware Transformeroptic disc and vasculature changeslongitudinal disease monitoringearly risk prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a new model called DiffSight-Former that processes a series of eye photographs taken at different times to forecast whether glaucoma will progress. Current single-image methods miss the gradual changes that signal early worsening, while many sequential approaches require fixed numbers of visits or already-diseased images. DiffSight-Former extracts stable anatomical features, calculates how the optic disc region and blood vessels have changed between visits, and uses a Transformer that respects the actual time gaps to output a progression probability. If the approach holds, clinicians could intervene before irreversible vision loss occurs rather than waiting for clear damage on one snapshot. Results on two patient cohorts show AUC of 91.54 percent on SIGF and average accuracy of 87.48 percent on GRAPE across standard visual-field criteria.

Core claim

DiffSight-Former incorporates a time-variant feature extraction module based on a fundus-specific foundation model to obtain robust anatomical representations, introduces a multi-structure difference modeling module to quantify progression-related changes in the optic disc/cup region and retinal vasculature, integrates these representations with temporal interval embeddings, and processes them by a time-aware Transformer to model disease progression and estimate the probability of future glaucoma onset.

What carries the argument

The multi-structure difference modeling module that quantifies changes in optic disc/cup and retinal vasculature, combined with temporal interval embeddings fed into a time-aware Transformer.

Load-bearing premise

The longitudinal datasets provide accurate progression labels that reflect true clinical outcomes and the model works on new patient groups without biases from image quality or demographics.

What would settle it

Evaluating the trained model on an independent longitudinal fundus sequence dataset with progression labels verified by a separate clinical panel; performance dropping below 80 percent AUC or 80 percent sensitivity would falsify the reported gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The model handles variable-length sequences and different time intervals between visits without requiring fixed inputs.
  • It detects subtle early progression signals that single time-point methods miss by explicitly modeling structural differences.
  • Performance remains consistent across the two tested cohorts and multiple clinical progression criteria.
  • The framework supports repeated monitoring during routine follow-up rather than one-time diagnosis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same difference-modeling approach could be tested on other progressive retinal conditions that produce visible vessel or disc changes over time.
  • Deployment in screening programs would require checking whether performance holds when image quality varies more widely than in the study cohorts.
  • Replacing the foundation model backbone with alternatives would test how much the reported numbers depend on that specific pre-training.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes DiffSight-Former for glaucoma progression prediction from sequential fundus images. The model uses a time-variant feature extraction based on a fundus foundation model, a multi-structure difference modeling module for optic disc/cup and vasculature, and a time-aware Transformer with temporal interval embeddings to predict future glaucoma onset. It reports an AUC of 91.54% and sensitivity of 92.16% on the SIGF dataset (405 sequences) and an average accuracy of 87.48% on the GRAPE dataset (263 sequences) across three visual-field progression criteria.

Significance. Should the reported performance be confirmed with transparent validation protocols, the framework could contribute to the field by addressing limitations in existing sequential models for detecting subtle early progression signals in glaucoma, offering potential for improved longitudinal monitoring in clinical settings.

major comments (3)
  1. [Methods] The procedure for assigning progression labels from the three clinical visual-field criteria is not described with respect to the temporal window relative to the input image sequences; this is critical because without explicit temporal separation, the high sensitivity may reflect label leakage rather than genuine future prediction.
  2. [Experiments] No details are provided on the train-test split strategy, handling of variable sequence lengths, or statistical significance testing for the performance metrics, which are load-bearing for verifying the robustness claim across different temporal settings.
  3. [Results] Baseline comparisons are mentioned but not quantified with specific metrics or tables showing improvements, making it difficult to assess the contribution of the difference modeling and temporal components.
minor comments (2)
  1. [Abstract] The acronyms SIGF and GRAPE are not expanded upon first use.
  2. [Figure 1] The model architecture diagram would benefit from clearer labeling of the multi-structure difference modeling module.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important areas for clarification. We will revise the manuscript to address each point by adding the requested methodological and experimental details.

read point-by-point responses

  1. Referee: [Methods] The procedure for assigning progression labels from the three clinical visual-field criteria is not described with respect to the temporal window relative to the input image sequences; this is critical because without explicit temporal separation, the high sensitivity may reflect label leakage rather than genuine future prediction.

    Authors: We agree that explicit specification of the temporal window is essential to demonstrate absence of label leakage. The revised Methods section will describe how progression labels derived from the three visual-field criteria are assigned exclusively from examinations performed after the final image in each input sequence, thereby ensuring the task is future prediction. revision: yes

  2. Referee: [Experiments] No details are provided on the train-test split strategy, handling of variable sequence lengths, or statistical significance testing for the performance metrics, which are load-bearing for verifying the robustness claim across different temporal settings.

    Authors: We will expand the Experiments section to include the patient-level train-test split protocol, the mechanisms used to accommodate variable sequence lengths within the time-aware Transformer, and the statistical tests applied to the reported metrics. These additions will support verification of robustness across temporal settings. revision: yes

  3. Referee: [Results] Baseline comparisons are mentioned but not quantified with specific metrics or tables showing improvements, making it difficult to assess the contribution of the difference modeling and temporal components.

    Authors: A quantitative comparison table will be added to the Results section, reporting AUC, accuracy, sensitivity, and specificity for DiffSight-Former against the relevant baseline models. This will allow direct assessment of the contributions from the multi-structure difference modeling and temporal interval embeddings. revision: yes

Circularity Check

0 steps flagged

No circularity: standard empirical ML evaluation on external datasets

full rationale

The paper proposes a Transformer-based architecture (time-variant feature extraction via foundation model, multi-structure difference module, temporal embeddings) and evaluates it via standard supervised training and held-out test metrics (AUC, sensitivity, accuracy) on the SIGF and GRAPE longitudinal datasets. No equations, fitted parameters renamed as predictions, self-definitional loops, or load-bearing self-citations appear in the abstract or described pipeline. Performance numbers are ordinary empirical results, not reductions to inputs by construction; label provenance and generalization are separate correctness questions outside the circularity analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so the ledger is necessarily incomplete. The central claim rests on the unstated assumptions that the two named datasets are correctly labeled and representative, that the foundation model provides robust features, and that the Transformer can learn progression from the extracted differences.

pith-pipeline@v0.9.1-grok · 5818 in / 1146 out tokens · 23353 ms · 2026-06-27T17:13:41.598173+00:00 · methodology

0 comments
read the original abstract

Glaucoma is a leading cause of irreversible blindness worldwide, and early detection from fundus images is critical for effective disease management. While deep learning has achieved promising performance in fundus image analysis, most existing methods rely on single time-point images and fail to capture longitudinal structural and vascular changes associated with disease progression. Sequential fundus images acquired during clinical follow-up provide valuable temporal information; however, current sequential models often struggle to detect subtle early progression signals and commonly depend on fixed-length inputs or diagnostic cues from already glaucomatous images, limiting their clinical utility for early prediction. To address these limitations, we propose DiffSight-Former, a framework for glaucoma progression prediction from sequential fundus images. It incorporates a time-variant feature extraction module based on a fundus-specific foundation model to obtain robust anatomical representations. A multi-structure difference modeling module is introduced to quantify progression-related changes in the optic disc/cup region and retinal vasculature. These representations are integrated with temporal interval embeddings and processed by a time-aware Transformer to model disease progression and estimate the probability of future glaucoma onset. Experiments were conducted on two longitudinal datasets, SIGF (405 sequences) and GRAPE (263 sequences). On SIGF, DiffSight-Former achieved an AUC of 91.54% and a sensitivity of 92.16% for progression prediction. On GRAPE, it achieved an average accuracy of 87.48% across three clinical visual-field progression criteria. Compared with existing approaches, DiffSight-Former demonstrates strong performance and robustness across different temporal settings, highlighting its potential for longitudinal glaucoma monitoring and early risk prediction.

Figures

Figures reproduced from arXiv: 2606.09140 by Jinman Kim, Lei Bi, Yi Huang.

Figure 1
Figure 1. Figure 1: Comparison between existing approaches and the proposed method for glaucoma progression prediction from sequential fundus images. Existing methods primarily rely on global image representations derived from appearance-driven features, which are subsequently aggregated with temporal embeddings for prediction. The proposed framework explicitly models inter-visit structural differences by leveraging anatomica… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed DiffSight-Former, which consists of three modules: Time-Variant Feature Extraction (TFE), Multi-structure Difference Modulation (MDM) and Time-Aware Transformer (TAT). In the TFE module, sequential fundus images are first polar-transformed and then encoded using a fundus-specific foundation model (RETFound) to obtain time-variant image features. In MDM module (a), the vessel and op… view at source ↗
Figure 3
Figure 3. Figure 3: Different input sequences. (a) Input sequences in previous studies: the negative and positive glaucoma images are all used to predict with fixed input length. (b) Input sequences in our method: only negative glaucoma images are used to predict, with flexible input length. D. Loss function In our task, the progression outcome is binary, so we adopt the cross-entropy loss to optimize the glaucoma progression… view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study results on the Multi-structure Difference Modu￾lation. Different structural maps is used to compute difference features. (unit:%) is insufficient for progression prediction, as it does not ex￾plicitly encode inter-visit change. Incorporating TAT improves performance to 83.87% ACC and 79.81% AUC, suggesting that modeling irregular temporal intervals enables the network to better weight observ… view at source ↗
Figure 5
Figure 5. Figure 5: The results comparison on different sequence length (T = 4, 3, 2) between our method and the MST-Former. (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The t-SNE distribution analysis of the difference features on test set under different temporal configurations produced by our method and MST-Former. In (a),(b),(c), the sequences used are T = 2,3,4 respectively. As expected, both methods show slightly improved class separability when more time points are available, reflecting the additional temporal context. The observations suggest that the improved sepa… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references

  1. [1]

    Glaucoma as a neurodegenerative disease,

    H. Danesh-Meyer and L. Levin, “Glaucoma as a neurodegenerative disease,”J. Neuro-Ophthalmol, vol. 35, pp. 22–28, 2015

    2015

  2. [2]

    Global causes of blindness and distance vision impairment 1990–2020: a systematic review and meta-analysise,

    S. R. Flaxman, R. R. A. Bourne, S. Resnikoff, P. Ackland, and et al., “Global causes of blindness and distance vision impairment 1990–2020: a systematic review and meta-analysise,”Lancet Glob Health., vol. 5, pp. 1221–1234, 2017

    1990

  3. [3]

    Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis,

    Y .-C. Tham, X. Li, T. Y . Wong, H. A. Quigley, T. Aung, and C.-Y . Cheng, “Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis,”Ophthal- mology, vol. 121, pp. 2081–2090, 2014

    2040

  4. [4]

    A compre- hensive review of deep learning strategies in retinal disease diagnosis using fundus images,

    B. Goutam, M. F. Hashmi, Z. W. Geem, and N. D. Bokde, “A compre- hensive review of deep learning strategies in retinal disease diagnosis using fundus images,”IEEE Access, vol. 10, pp. 57 796–57 823, 2022

    2022

  5. [5]

    Recent trends and advances in fundus image analysis: A review,

    S. Iqbal, T. M. Khan, K. Naveed, S. S. Naqvi, and S. J. Nawaz, “Recent trends and advances in fundus image analysis: A review,”Comput. Biol. Med., vol. 151, p. 106277, 2022

    2022

  6. [6]

    The ocular hypertension treatment study: Design and baseline description of the participants,

    M. O. Gordon and M. A. Kass, “The ocular hypertension treatment study: Design and baseline description of the participants,”Arch. Oph- thalmol., vol. 117, pp. 573–583, 1999

    1999

  7. [7]

    The age-related eye disease study (AREDS): design implications areds report no. 1,

    A.-R. E. D. S. R. Groupet al., “The age-related eye disease study (AREDS): design implications areds report no. 1,”Controlled clinical trials, vol. 20, no. 6, p. 573, 1999

    1999

  8. [8]

    DeepGF: Glaucoma forecast using the sequential fundus images,

    L. Li, X. Wang, M. Xu, H. Liu, and X. Chen, “DeepGF: Glaucoma forecast using the sequential fundus images,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2020, pp. 626–635

    2020

  9. [9]

    GRAPE: A multi-modal dataset of longitudinal follow-up visual field and fundus images for glaucoma management,

    X. Huang, X. Kong, Z. Shen, and et al., “GRAPE: A multi-modal dataset of longitudinal follow-up visual field and fundus images for glaucoma management,”Sci. Data, vol. 10, p. 520, 2023

    2023

  10. [10]

    A deep learning system for myopia onset prediction and intervention effectiveness evaluation in children,

    Z. Qi, T. Li, J. Chen, J. C. Yam, and et al., “A deep learning system for myopia onset prediction and intervention effectiveness evaluation in children,”npj Digit. Med., vol. 7, no. 1, p. 206, 2024

    2024

  11. [11]

    CANet: Cross- disease attention network for joint diabetic retinopathy and diabetic macular edema grading,

    X. Li, X. Hu, L. Yu, L. Zhu, C.-W. Fu, and P.-A. Heng, “CANet: Cross- disease attention network for joint diabetic retinopathy and diabetic macular edema grading,”IEEE Trans. Med. Imag., vol. 39, no. 5, pp. 1483–1493, 2020

    2020

  12. [12]

    CABNet: Category attention block for imbalanced diabetic retinopathy grading,

    A. He, T. Li, N. Li, K. Wang, and H. Fu, “CABNet: Category attention block for imbalanced diabetic retinopathy grading,”IEEE Trans. Med. Imag., vol. 40, no. 1, pp. 143–153, 2021

    2021

  13. [13]

    Cataract detection and visualization based on multi-scale deep features by rinet tuned with cyclic learning rate hyperparameter,

    K. Pammi and S. Priyank, “Cataract detection and visualization based on multi-scale deep features by rinet tuned with cyclic learning rate hyperparameter,”Biomed. Signal Process. Control, vol. 87, p. 105452, 2024

    2024

  14. [14]

    Accelerating cardiac radial-mri: Fully polar based technique using compressed sensing and deep learning,

    V . Ghodrati, J. Duan, F. Ali, A. Bedayat, A. Prosper, and M. Bydder, “Accelerating cardiac radial-mri: Fully polar based technique using compressed sensing and deep learning,”Med. Image Anal., vol. 105, p. 103732, 2025

    2025

  15. [15]

    Joint optic disc and cup segmentation based on multi-label deep network and polar transformation,

    H. Fu, J. Cheng, Y . Xu, D. W. K. Wong, J. Liu, and X. Cao, “Joint optic disc and cup segmentation based on multi-label deep network and polar transformation,”IEEE Trans. Med. Imaging, vol. 37, no. 7, pp. 1597–1605, 2018

    2018

  16. [16]

    A novel glau- coma detection model using unet++-based segmentation and resnet with gru-based optimized deep learning,

    V . V . N. S. Kumar, G. H. Reddy, and M. GiriPrasad, “A novel glau- coma detection model using unet++-based segmentation and resnet with gru-based optimized deep learning,”Biomed. Signal Process. Control, vol. 86, p. 105069, 2023

    2023

  17. [17]

    CDAM- Net: Channel shuffle dual attention based multi-scale cnn for efficient glaucoma detection using fundus images,

    D. Das, D. R. Nayak, S. V . Bhandary, and U. R. Acharya, “CDAM- Net: Channel shuffle dual attention based multi-scale cnn for efficient glaucoma detection using fundus images,”Eng. Appl. Artif. Intel., vol. 133, p. 108454, 2024

    2024

  18. [18]

    A generalizable deep learning regression model for automated glaucoma screening from fundus images,

    R. Hemelings, B. Elen, A. Schuster, and et al., “A generalizable deep learning regression model for automated glaucoma screening from fundus images,”npj Digit. Med., vol. 6, p. 112, 2023

    2023

  19. [19]

    RDD-Net: Randomized joint data- feature augmentation and deep-shallow feature fusion networks for automated diagnosis of glaucoma,

    Y . Tang, M. Zhang, and J. Feng, “RDD-Net: Randomized joint data- feature augmentation and deep-shallow feature fusion networks for automated diagnosis of glaucoma,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2024, pp. 575–584

    2024

  20. [20]

    Convolutional neural network transfer for automated glaucoma iden- tification,

    J. I. Orlando, E. Prokofyeva, M. del Fresno, and M. B. Blaschko, “Convolutional neural network transfer for automated glaucoma iden- tification,” inInt. Symp. Med. Info. Process. Anal., vol. 10160, 2017, pp. 241–250

    2017

  21. [21]

    A fast and fully automated system for glaucoma detection using color fundus photographs,

    S. Saha, J. Vignarajan, and S. Frost, “A fast and fully automated system for glaucoma detection using color fundus photographs,”Sci. Rep., vol. 13, p. 18408, 2023

    2023

  22. [22]

    Deep relation transformer for diagnosing glaucoma with optical coherence tomography and visual field function,

    D. Song, B. Fu, F. Li, J. Xiong, J. He, X. Zhang, and Y . Qiao, “Deep relation transformer for diagnosing glaucoma with optical coherence tomography and visual field function,”IEEE Trans. Med. Imag., vol. 40, no. 9, pp. 2392–2402, 2021

    2021

  23. [23]

    Parallel multi-path network for ocular disease detection inspired by visual cognition mechanism,

    T. Deng, Y . Huang, and C. Yang, “Parallel multi-path network for ocular disease detection inspired by visual cognition mechanism,”IEEE J. Biomed. Health. Inf., vol. 29, no. 1, pp. 345–357, 2025

    2025

  24. [24]

    Long short-term memory model based microaneurysm sequence classification in fundus images,

    R. Acharya and N. B. Puhan, “Long short-term memory model based microaneurysm sequence classification in fundus images,” inIEEE Int. Conf. on Signal Processing and Communications (SPCOM), 2022, pp. 1–5

    2022

  25. [25]

    GLIM-Net: Chronic glaucoma forecast transformer for irregularly sampled sequential fundus images,

    X. Hu, L.-X. Zhang, L. Gao, W. Dai, X. Han, Y .-K. Lai, and Y . Chen, “GLIM-Net: Chronic glaucoma forecast transformer for irregularly sampled sequential fundus images,”IEEE Trans. Med. Imaging, vol. 42, no. 6, pp. 1875–1884, 2023

    2023

  26. [26]

    Multi-scale spatio-temporal transformer-based imbalanced longitudinal learning for glaucoma forecasting from irregular time series images,

    X. Yang, J. Wu, X. Wang, Y . Yuan, J. Li, G. Chen, N. L. Wang, and P.-A. Heng, “Multi-scale spatio-temporal transformer-based imbalanced longitudinal learning for glaucoma forecasting from irregular time series images,”IEEE J. Biomed. Health. Inf., vol. 29, no. 4, pp. 2859–2870, 2025

    2025

  27. [27]

    A foun- dation model for generalizable disease detection from retinal images,

    Y . Zhou, M. A. Chia, S. K. Wagner, M. S. Ayhan, and et al., “A foun- dation model for generalizable disease detection from retinal images,” Nature, vol. 622, pp. 156–163, 2023

    2023

  28. [28]

    Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence,

    J. Qiu, J. Wu, H. Wei, and et al., “Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence,”NEJM AI, vol. 1, no. 12, p. AIoa2300221, 2024

    2024

  29. [29]

    An eyecare foundation model for clinical assistance: a randomized controlled trial,

    Y . Wu, B. Qian, T. Li, and et al., “An eyecare foundation model for clinical assistance: a randomized controlled trial,”Nat. Med., vol. 31, p. 3404–3413, 2025. 12 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS

    2025

  30. [30]

    A multimodal visual–language foundation model for computational ophthalmology,

    D. Shi, W. Zhang, J. Yang, and et al., “A multimodal visual–language foundation model for computational ophthalmology,”npj Digit. Med., vol. 8, p. 381, 2025

    2025

  31. [31]

    Multi-knowledge informed deep learning model for multi-point prediction of alzheimer’s disease progression,

    K. Wu, H. Wang, F. Feng, T. Liu, and Y . Sun, “Multi-knowledge informed deep learning model for multi-point prediction of alzheimer’s disease progression,”Neural Networks, vol. 185, p. 107203, 2025

    2025

  32. [32]

    Prediction of disease- related femur shape changes using geometric encoding and clinical context on a hip disease ct database,

    G. Li, Y . Otake, M. Soufi, M. Masuda, and et al., “Prediction of disease- related femur shape changes using geometric encoding and clinical context on a hip disease ct database,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2024, pp. 368–378

    2024

  33. [33]

    Risk estimation of knee os- teoarthritis progression via predictive multi-task modelling from efficient diffusion model using X-Ray images,

    A. H. David Butler and G. Carneiro, “Risk estimation of knee os- teoarthritis progression via predictive multi-task modelling from efficient diffusion model using X-Ray images,” 2025, pp. 539 – 549

    2025

  34. [34]

    Morph- ssl: Self-supervision with longitudinal morphing for forecasting amd progression from oct volumes,

    A. Chakravarty, T. Emre, O. Leingang, S. Riedl, and et al., “Morph- ssl: Self-supervision with longitudinal morphing for forecasting amd progression from oct volumes,”IEEE Trans. Med. Imaging, vol. 43, no. 9, pp. 3224–3239, 2024

    2024

  35. [35]

    Learning spatio-temporal model of disease progression with neuralodes from longitudinal volumetric data,

    D. Lachinov, A. Chakravarty, C. Grechenig, U. Schmidt-Erfurth, and H. Bogunovi ´c, “Learning spatio-temporal model of disease progression with neuralodes from longitudinal volumetric data,”IEEE Trans. Med. Imaging, vol. 43, no. 3, pp. 1165–1179, 2024

    2024

  36. [36]

    LaTiM: Longitudinal representation learning in continuous-time models to predict disease progression,

    R. Zeghlache, P.-H. Conze, M. E. H. Daho, Y . Li, H. L. Boit ´e, and et al., “LaTiM: Longitudinal representation learning in continuous-time models to predict disease progression,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2024, pp. 404–414

    2024

  37. [37]

    Convolutional LSTM network: A machine learning approach for precipitation nowcasting,

    X. Shi, Z. Chen, H. Wang, D.-Y . Yeung, W. kin Wong, and W. chun woo, “Convolutional LSTM network: A machine learning approach for precipitation nowcasting,” inProc. Int. Conf. Neural Inf. Proces. Syst. (NeurIPS), 2015, pp. 802 – 810

    2015

  38. [38]

    RetainVis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records,

    B. C. Kwon, M.-J. Choi, J. T. Kim, E. Choi, Y . B. Kim, S. Kwon, J. Sun, and J. Choo, “RetainVis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records,” IEEE Trans. Vis. Comput. Graph., vol. 25, no. 1, pp. 299–309, 2019

    2019

  39. [39]

    DATA-GRU: Dual-attention time-aware gated recurrent unit for irregular multivariate time series,

    Q. Tan, M. Ye, B. Yang, S. Liu, and et al., “DATA-GRU: Dual-attention time-aware gated recurrent unit for irregular multivariate time series,” inProc. AAAI Conf. Artif. Intell. (AAAI), 2020, pp. 930–937

    2020

  40. [40]

    A spatiotemporal convolution recurrent neural network for pixel-level peripapillary atro- phy prediction using sequential fundus images,

    M. Li, W. Zhang, H. Zhao, Y . Xu, J. Xu, and H. Li, “A spatiotemporal convolution recurrent neural network for pixel-level peripapillary atro- phy prediction using sequential fundus images,”Appl. Soft Comput., vol. 155, p. 111431, 2024

    2024

  41. [41]

    Transformers in medical imaging: A survey,

    F. Shamshad, S. Khan, S. W. Zamir, M. H. Khan, M. Hayat, F. S. Khan, and H. Fu, “Transformers in medical imaging: A survey,”Med. Image Anal., vol. 88, p. 102802, 2023

    2023

  42. [42]

    Transformers in medical imaging: A survey,

    Q. Pu, Z. Xi, S. Yin, Z. Zhao, and L. Zhao, “Transformers in medical imaging: A survey,”BioMed. Eng. OnLine, vol. 23, p. 14, 2024

    2024

  43. [43]

    Med-Former: A transformer based archi- tecture for medical image classification,

    G. J. Chowdary and Z. Yin, “Med-Former: A transformer based archi- tecture for medical image classification,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2024, pp. 448–457

    2024

  44. [44]

    ReconFormer: Accelerated MRI reconstruction using recurrent transformer,

    P. Guo, Y . Mei, J. Zhou, S. Jiang, and V . M. Patel, “ReconFormer: Accelerated MRI reconstruction using recurrent transformer,”IEEE Trans. Med. Imaging, vol. 43, no. 1, pp. 582–593, 2024

    2024

  45. [45]

    METransformer: Radiology report generation by transformer with multiple learnable expert tokens,

    Z. Wang, L. Liu, L. Wang, and L. Zhou, “METransformer: Radiology report generation by transformer with multiple learnable expert tokens,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 11 558–11 567

    2023

  46. [46]

    Longitudinal multimodal transformer integrating imaging and latent clinical signatures from routine ehrs for pulmonary nodule classification,

    T. Z. Li, J. M. Still, K. Xu, H. H. Lee, and et al., “Longitudinal multimodal transformer integrating imaging and latent clinical signatures from routine ehrs for pulmonary nodule classification,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2023, pp. 649–659

    2023

  47. [47]

    Pattern-aware transformer: Hierarchical pattern propagation in sequential medical images,

    L. Wu, X. Gao, Z. Hu, and S. Zhang, “Pattern-aware transformer: Hierarchical pattern propagation in sequential medical images,”IEEE Trans. Med. Imaging, vol. 43, no. 1, pp. 405–415, 2024

    2024

  48. [48]

    Spatiotemporal representation learning for short and long medical image time series,

    C. Shen, M. J. Menten, H. Bogunovi ´c, U. Schmidt-Erfurth, and et al., “Spatiotemporal representation learning for short and long medical image time series,” inProc. Int. Conf. Med. Image Comput. Comput. Assi. Inte. (MICCAI), 2024, pp. 656–666

    2024

  49. [49]

    LongFormer: Longitudinal transformer for alzheimer’s disease classification with structural mris,

    Q. Chen, Q. Fu, H. Bai, and Y . Hong, “LongFormer: Longitudinal transformer for alzheimer’s disease classification with structural mris,” inProc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), 2024, pp. 3563–3572

    2024

  50. [50]

    Multimodal deep learning for predicting in-hospital mortality in heart failure patients using longitudinal chest x-rays and electronic health records,

    D. Li, W. Xing, J. Zhao, C. Shi, and F. Wang, “Multimodal deep learning for predicting in-hospital mortality in heart failure patients using longitudinal chest x-rays and electronic health records,”Int. J. Cardiovas. Imag., vol. 41, pp. 427–440, 2025

    2025

  51. [51]

    Time series as images: Vision transformer for irregularly sampled time series,

    Z. Li, S. Li, and X. Yan, “Time series as images: Vision transformer for irregularly sampled time series,” inProc. Int. Conf. Neural Inf. Proces. Syst. (NeurIPS), 2023, pp. 49 187 – 49 204

    2023

  52. [52]

    XTSFormer: Cross- temporal-scale transformer for irregular-time event prediction in clinical applications,

    T. Xiao, Z. Xu, W. He, Z. Xiao, and et al., “XTSFormer: Cross- temporal-scale transformer for irregular-time event prediction in clinical applications,” inProc. AAAI Conf. Artif. Intell. (AAAI), 2025, pp. 28 502– 28 510

    2025

  53. [53]

    Uniformer: Unifying convolution and self-attention for visual recognition,

    K. Li, Y . Wang, J. Zhang, P. Gao, G. Song, Y . Liu, H. Li, and Y . Qiao, “Uniformer: Unifying convolution and self-attention for visual recognition,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 10, pp. 12 581–12 600, 2023

    2023

  54. [54]

    Harnessing the power of longitu- dinal medical imaging for eye disease prognosis using transformer-based sequence modeling,

    G. Holste, M. Lin, R. Zhou, and et al., “Harnessing the power of longitu- dinal medical imaging for eye disease prognosis using transformer-based sequence modeling,”npj Digit. Med., vol. 7, p. 216, 2024

    2024

  55. [55]

    AutoMorph: Automated retinal vascular morphology quantifica- tion via a deep learning pipeline,

    Y . Zhou, S. K. Wagner, M. A. Chia, A. Zhao, P. Woodward-Court, and et al., “AutoMorph: Automated retinal vascular morphology quantifica- tion via a deep learning pipeline,”Transl. Vis. Sci. Technol., vol. 11, p. 12, 2022

    2022

  56. [56]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, and et al., “Attention is all you need,” inProc. Int. Conf. Neural Inf. Proces. Syst. (NeurIPS), 2015, pp. 6000 – 6010

    2015