Pith. sign in

REVIEW 5 major objections 6 minor 16 references

Patch Progression Masked Autoencoder with Fusion CNN Network for Classifying Evolution Between Two Pairs of 2D OCT Slices

T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read This paper claims that a masked autoencoder can generate a future OCT scan from a current one accurately enough to classify AMD progression from a single exam.

desk verdict PPMAE is a sensible new twist on MAE for OCT forecasting, but the paper's own baseline description is contradictory and the reported gains need more evidence. read the letter →

arxiv 2508.20064 v1 pith:PD35BOGR submitted 2025-08-27 cs.CV cs.AI

classification cs.CVcs.AI
keywords age-relatedmaculardegenerationopticalcoherencetomographyprogressionpredictionmaskedautoencoderfutureimagegenerationclassificationmodelensemble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that age-related macular degeneration (AMD) progression can be classified from a single optical coherence tomography (OCT) exam by first generating what the next exam would look like. Its patch progression masked autoencoder (PPMAE) masks most of a baseline scan and learns to predict patches of the follow-up scan rather than reconstructing the same scan. This synthetic follow-up is accurate enough that a two-scan classifier, originally trained on real consecutive exams, can be applied to the current scan plus the generated one. On the challenge validation set, the approach beats a standard masked autoencoder across several metrics and achieves the best overall Task 2 result when the classifier is finetuned on Target 2 data. If correct, this would enable monitoring of neovascular AMD activity without waiting for the next physical follow-up visit.

What carries the argument

The central object is the Patch Progression Masked Autoencoder (PPMAE), a Vision-Transformer autoencoder built on the Masked Autoencoder idea but with the reconstruction target changed from the input image to the follow-up image. Masking 75% of the t0 scan removes spatial locations the model must later fill in from t1; the transformer encoder sees only the visible t0 patches, and the decoder outputs t1 patches for the masked positions. Because the target is a different time point, the model cannot copy the input; it must learn how retinal structure at t0 changes into the follow-up visit. The generated t1 image is then paired with t0 and classified by the Task 1 two-scan fusion network (late

What would settle it

Measure how well the generated 'worsened' scans reproduce the actual fluid-volume change seen in the true follow-up scan; if a retina specialist can reliably spot the generated images among real follow-ups, or if the generated images fail to show new fluid where real scans show it, then using PPMAE as a surrogate for the next visit is not clinically valid.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a masked autoencoder can be trained to predict the next examination instead of reconstructing the same image. Patch Progression MAE masks 75% of a baseline OCT scan at time t0 and trains the decoder to predict the corresponding image patches of the follow-up scan at time t1. With retina-flattening preprocessing, the generated follow-up scan reaches an MSE of 0.1724 on the challenge validation set, compared with 0.5572 for a standard MAE trained to reconstruct the same target. The current scan and the generated future scan are then fed into a two-scan fusion classifier trained for Task 1; after finetuning that classifier on Task 2 labels, the pipeline rea

Load-bearing premise

The load-bearing premise is that a classifier trained on real pairs of consecutive OCT scans remains trustworthy when one member of the pair is produced by a generative model.

Editorial extensions

If this is right

  • A single current OCT can be turned into a plausible follow-up image, so monitoring neovascular AMD activity no longer strictly requires waiting for the next physical exam.
  • Reconstruction quality tracks task performance: the PPMAE's lower MSE (0.1724 with preprocessing) coincides with higher Task 2 F1, specificity, rank correlation, and weighted quadratic kappa than the standard MAE baseline.
  • The classifier finetuned on Task 2 labels reaches the best mean metric (0.3748) and F1 (0.6933), indicating that even a good synthetic pair benefits from a short adaptation step to the target distribution.
  • Retina-flattening preprocessing improves both classification and future-image reconstruction, making anatomical normalization a building block for this type of temporal OCT analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not separate finetuning from image quality: the best Task 2 result uses both PPMAE and finetuning, so some of the gain may come from the finetuning step rather than from the generated future scan being more realistic than the standard MAE.
  • Because the model predicts one deterministic future image, it likely averages over possible futures; for patients who suddenly worsen, the averaged scan may look more stable than the actual outcome, biasing the classifier toward the stable class. This can be tested by measuring reconstruction error separately for worsened cases.
  • The cross-time masking objective should transfer to other registered longitudinal imaging settings; conditioning the decoder on the interval between visits, which the current model ignores, is a natural testable extension since the follow-up interval is not uniform across the dataset.
  • Low overall MSE can be dominated by unchanged background; a stronger evaluation would weight error in regions of retinal fluid or neovascular change, where the clinical classification actually lives.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript reports the authors' solution to the MARIO challenge. For Task 1, it trains and ensembles four ResNet50-based early/late fusion CNNs on OCT image pairs, using the OCTIP preprocessing tool, and reports validation F1 scores up to 0.8223. For Task 2, it proposes Patch Progression Masked Autoencoder (PPMAE), which masks 75% of the baseline OCT slice at time t0 and predicts patches of the follow-up slice at t1. The reconstructed t1 is then fed together with t0 into the Task 1 classifier to predict three-class progression. The paper reports PPMAE reconstruction MSE of 0.1724 versus 0.5572 for a standard MAE baseline, and Task 2 validation F1 of 0.6933 after finetuning on Task 2 data, claiming Top-10 placement in both challenge tasks.

Significance. If the PPMAE results are substantiated, the method is a reasonable and practical solution for single-scan AMD progression prediction: it combines a future-slice generator with an existing pairwise classifier, and the authors release code, use a meaningful preprocessing step, and disclose a potential conflict of interest. The separate training of the reconstruction and classification components avoids obvious circularity. However, the central quantitative claims currently rest on an ambiguous and possibly unfair MAE baseline, missing control baselines, and validation-only metrics without uncertainty estimates. The paper is therefore a useful challenge report, but its main contribution is not yet established as written.

major comments (5)
  1. [§2.4, Table 3 footnote] The description of the MAE baseline is internally contradictory. Section 2.4 states that traditional MAE reconstructs the same image from the unmasked part, whereas the footnote to Tables 3 and 4 says 'In MAE... The goal remains to reconstruct the image at time t1.' If the baseline MAE reconstructs t0, then its MSE against t1 (0.5572) is an expected artifact, and the observation that MAE outputs resemble t0 and give 'stable' predictions is not evidence for PPMAE. If the baseline MAE was trained to reconstruct t1, then its training details are missing. Either way, the comparison in Tables 3 and 4 does not currently support the claim that PPMAE improves progression prediction over a properly matched MAE baseline.
  2. [Table 4] Task 2 results compare only MAE+classifier against PPMAE+classifier. There is no no-reconstruction baseline, e.g., a classifier trained directly on t0 alone, or the Task 1 classifier applied to (t0, t0). Since Task 2 is single-scan progression classification, such a baseline is essential to show that generate-then-classify adds value over ignoring the reconstructed image. Without it, the reported gains could reflect task mismatch rather than the quality of the predicted t1.
  3. [§2.1, Table 4] Task 2 labels exclude the 'Other' category, but the Task 1 classifiers used in Table 4 are four-class models. The manuscript does not state how 'Other' predictions are handled when these classifiers are applied to Task 2 data, nor whether the 'finetuned on data Task 2' model has a three-class output head or simply continued four-class training. This affects F1, WQK, and the mean metric, and must be clarified.
  4. [Tables 2 and 4] All quantitative results are single-run validation metrics without error bars, confidence intervals, or significance tests. Several differences are small (e.g., Task 2 F1 0.6465 vs 0.6491, Rk 0.0790 vs 0.0938) and may be within noise. The adjectives 'significantly' and 'best overall' are not supported by the reported evidence. The 'Top 10' claim is also not backed by official test scores in the manuscript.
  5. [§2.4, Table 4] The proposed pipeline applies a classifier trained on real OCT pairs to pairs containing a synthetically generated t1. The paper itself notes that for MAE, reconstruction failures lead to systematically 'stable' predictions, showing that synthetic images can bias the downstream classifier. No analogous check is provided for PPMAE: MSE and visual examples do not guarantee that generated t1 images follow the same conditional distribution as real follow-up scans. A direct test, e.g., comparing classifier predictions on real vs generated pairs with matched labels, is needed before claiming that PPMAE-based classification is clinically reliable.
minor comments (6)
  1. [Table 1] The table formatting is unclear: the columns 'Task1', 'Task2 Finetuning', and 'PPMAE' are not cleanly separated, and the 'Train/Val split 1:0' for Task2 finetuning should be explained as using the challenge-provided training fold only, not the validation set.
  2. [Tables 1, 2, 4] Abbreviations RkC, WQK, and Rk correlation are used without definition. Please define them in the text.
  3. [§2.4, Table 3] Specify whether the MSE in Table 3 is computed only on masked patches or on the full reconstructed image, and report the intensity normalization used. Without this, the absolute MSE values are not interpretable.
  4. [Figure 5] The figure would benefit from captions indicating which category is shown, whether OCTIP preprocessing is applied, and a representative failure case.
  5. [§2.4] The phrase 'mask 75%' should state the patch size and number of patches. It should also be acknowledged that the unmasked regions of t0 are retained, so the 'reconstructed future image' is not entirely predicted from t1.
  6. [Abstract / Discussion] The claim of 'Top 10 for both tasks' should be supported by official test scores or a reference to the leaderboard; otherwise it is unverifiable.

Circularity Check

1 steps flagged · score 6.0 of 10

MAE baseline's t1 reconstruction reduces to t0 by construction, undercutting the main superiority comparison.

  1. fitted input called prediction [Section 2.4 (Task 2); Table 3 footnote; Section 3 (Results)]
    ""However, instead of reconstructing the same image from the unmasked part, PPMAE predicts the patches of the follow-up image at time t1." ... "(In MAE, we follow the traditional training approach by masking parts of the image and attempting to reconstruct the masked patches. The goal remains to reconstruct the image at time t1.)" ... "in MAE, the reconstruction of t1 often fails, with the reconstructed image resembling t0 more than t1.""

    Section 2.4 defines standard MAE as reconstructing the same t0 image from its unmasked patches. A MAE trained this way outputs a reconstruction of t0, so its 'reconstruction of t1' is t0 by construction. The reported MSE (0.5572) is then just the t0–t1 image distance, and the observation that MAE outputs resemble t0 is an identity, not evidence. The Table 3 footnote asserting the goal was t1 contradicts Section 2.4 and leaves the actual MAE training unspecified; with the Section 2.4 setup the comparison is forced by construction, and with the footnote setup the baseline is undefined. Either way, the claim that PPMAE improves t1 reconstruction over a matched MAE is not established by Tables 3–4.

full rationale

The proposed PPMAE itself is trained on (t0,t1) pairs with an MSE target at t1; that is an honest supervised objective, and the Task 1 classifier is trained on separate splits, so the core pipeline is not circular. OCTIP preprocessing is an external tool; citations [12], [15], and [16] are background or routine methodological citations and are not load-bearing. The single significant circularity is the MAE baseline. Section 2.4 defines standard MAE as reconstructing the same (t0) image, while the Table 3 footnote asserts the MAE target was t1 and Section 3 reports MAE outputs resembling t0. If MAE was trained as a standard t0 autoencoder, its output on t0 is t0 by construction, so the reported MSE against t1 (0.5572) and the 'stable' class predictions are predetermined by the t0–t1 distance, not by any predictive failure. This makes the headline comparison in Tables 3 and 4 an artifact unless a t1-targeted MAE training is specified, which the paper does not do consistently. That is partial circularity in the evidence for PPMAE's advantage, though PPMAE's own reconstruction numbers are not circular.

Assumptions & free parameters 4 free parameters · 2 assumptions · 0 invented entities

This is an empirical deep learning paper; the 'free parameters' are standard hyperparameters chosen by hand rather than quantities fitted to derive a law. The main hidden dependencies are the annotation quality and the correctness of the OCTIP preprocessing step. No new physical or conceptual entities are introduced.

free parameters (4)
  • mask_ratio = 0.75
    The fraction of baseline image patches masked in PPMAE is chosen by hand, following the standard MAE setting; it directly controls the reconstruction task.
  • learning_rate = 1e-4 (Task 1), 1e-5 (Task 2 and PPMAE)
    Learning rates are hand-chosen for each stage and influence convergence and final performance.
  • train_val_split = 0.75:0.25 for Task 1 and PPMAE; 1:0 for Task 2 finetuning
    The split ratio is chosen by hand and defines how validation metrics are computed.
  • input_size = 512x200 (Task 1), 224x224 (Task 2/PPMAE)
    Input resolution is a hand-chosen hyperparameter affecting model capacity and data preprocessing.
assumptions (2)
  • domain assumption The ground-truth labels for training and validation are correct as annotated by a single retina specialist.
    All reported results depend on label accuracy; if training labels are noisy or biased, the measured performance is unreliable. This is stated in Section 2.1.
  • domain assumption The OCTIP preprocessing correctly aligns and flattens the retina across time points.
    OCTIP segmentation and depth alignment are used in both tasks to improve comparability. If the alignment fails on some images, the downstream classification and reconstruction degrade. See Section 2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Patch Progression Masked Autoencoder with Fusion CNN Network for Classifying Evolution Between Two Pairs of 2D OCT Slices." pith.science (2026). https://pith.science/paper/PD35BOGR

@misc{pith2026250820064,
  author       = {Pith},
  title        = {Pith review of: Patch Progression Masked Autoencoder with Fusion CNN Network for Classifying Evolution Between Two Pairs of 2D OCT Slices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PD35BOGR}},
  note         = {Machine review of arXiv:2508.20064}
}
read the original abstract

Age-related Macular Degeneration (AMD) is a prevalent eye condition affecting visual acuity. Anti-vascular endothelial growth factor (anti-VEGF) treatments have been effective in slowing the progression of neovascular AMD, with better outcomes achieved through timely diagnosis and consistent monitoring. Tracking the progression of neovascular activity in OCT scans of patients with exudative AMD allows for the development of more personalized and effective treatment plans. This was the focus of the Monitoring Age-related Macular Degeneration Progression in Optical Coherence Tomography (MARIO) challenge, in which we participated. In Task 1, which involved classifying the evolution between two pairs of 2D slices from consecutive OCT acquisitions, we employed a fusion CNN network with model ensembling to further enhance the model's performance. For Task 2, which focused on predicting progression over the next three months based on current exam data, we proposed the Patch Progression Masked Autoencoder that generates an OCT for the next exam and then classifies the evolution between the current OCT and the one generated using our solution from Task 1. The results we achieved allowed us to place in the Top 10 for both tasks. Some team members are part of the same organization as the challenge organizers; therefore, we are not eligible to compete for the prize.

Figures

Figures reproduced from arXiv: 2508.20064 by the authors.

Figure 1
Figure 1. Optical Coherence Tomography Image Preprocessing [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Early Fusion CNN Network In Task 1, we explored two fusion methods [12]: Early Fusion and Late Fusion networks. During the experimentation phase, we tested various backbones, and ResNet50 showed the best performance. Early Fusion Network Illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Late Fusion CNN Network Additionally, we adopted a 4-fold cross-validation strategy, training four mod￾els on different folds (Model 1 to Model 4) and combining their results through ensembling to further improve the model’s robustness and generalization[16]. 2.4 Task 2 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Vision Transformer Patch Progression Masked AutoEncoder [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Visual Comparison of OCT Reconstruction Results by PPMAE [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 14 canonical work pages

  1. [1]

    Ashir, A.M., Ibrahim, S., Abdulghani, M., Ibrahim, A.A., Anwar, M.S.: Diabetic retinopathy detection using local extrema quantized haralick features with long short-termmemorynetwork.InternationalJournalofBiomedicalImaging 2021(1), 6618666 (2021)

  2. [2]

    Translational vision science & technology 9(2), 25–25 (2020)

    Bhuiyan, A., Wong, T.Y., Ting, D.S.W., Govindaiah, A., Souied, E.H., Smith, R.T.: Artificial intelligence to stratify severity of age-related macular degeneration (amd) and predict risk of progression to late amd. Translational vision science & technology 9(2), 25–25 (2020)

  3. [3]

    Nature communications12(1), 3242 (2021)

    Dai, L., Wu, L., Li, H., Cai, C., Wu, Q., Kong, H., Liu, R., Wang, X., Hou, X., Liu, Y., et al.: A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nature communications12(1), 3242 (2021)

  4. [4]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A.: An image is worth 16x16 words: Transformers for image recogni- tion at scale. arXiv preprint arXiv:2010.11929 (2020)

  5. [5]

    Retina 35(8), 1489–1506 (2015)

    Freund, K.B., Korobelnik, J.F., Devenyi, R., Framme, C., Galic, J., Herbert, E., Hoerauf, H., Lanzetta, P., Michels, S., Mitchell, P., et al.: Treat-and-extend reg- imens with anti-vegf agents in retinal diseases: a literature review and consensus recommendations. Retina 35(8), 1489–1506 (2015)

  6. [6]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

  7. [7]

    Neural Computation MIT-Press (1997)

    Hochreiter, S.: Long short-term memory. Neural Computation MIT-Press (1997)

  8. [8]

    npj Digital Medicine 7(1), 216 (2024)

    Holste, G., Lin, M., Zhou, R., Wang, F., Liu, L., Yan, Q., Van Tassel, S.H., Kovacs, K.,Chew,E.Y.,Lu,Z.,etal.:Harnessingthepoweroflongitudinalmedicalimaging for eye disease prognosis using transformer-based sequence modeling. npj Digital Medicine 7(1), 216 (2024)

Show all 16 references
  1. [9]

    Asia-Pacific Journal of Ophthalmology 6(6), 493–497 (2017)

    Jonas, J.B., Cheung, C.M.G., Panda-Jonas, S.: Updates on the epidemiology of age-related macular degeneration. Asia-Pacific Journal of Ophthalmology 6(6), 493–497 (2017)

  2. [10]

    Progress in retinal and eye research84, 100951 (2021)

    Lains, I., Wang, J.C., Cui, Y., Katz, R., Vingopoulos, F., Staurenghi, G., Vavvas, D.G., Miller, J.W., Miller, J.B.: Retinal applications of swept source optical co- herence tomography (oct) and optical coherence tomography angiography (octa). Progress in retinal and eye resea...

  3. [11]

    Cochrane Database of Systematic Reviews (5) (2020)

    Li, E., Donati, S., Lindsley, K.B., Krzystolik, M.G., Virgili, G.: Treatment regimens for administration of anti-vascular endothelial growth factor agents for neovascular age-related macular degeneration. Cochrane Database of Systematic Reviews (5) (2020)

  4. [12]

    Computers in Biology and Medicine p

    Li, Y., Daho, M.E.H., Conze, P.H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., Quellec, G.: A review of deep learning-based informa- tion fusion techniques for multimodal medical image classification. Computers in Biology and Medicine p. 108635 (2024)

  5. [13]

    Current Opinion in Ophthalmology 25(3), 158–163 (2014)

    Rasmussen, A., Sander, B.: Long-term longitudinal study of patients treated with ranibizumab for neovascular age-related macular degeneration. Current Opinion in Ophthalmology 25(3), 158–163 (2014)

  6. [14]

    New England Journal of Medicine355(14), 1419–1431 (2006)

    Rosenfeld, P.J., Brown, D.M., Heier, J.S., Boyer, D.S., Kaiser, P.K., Chung, C.Y., Kim, R.Y.: Ranibizumab for neovascular age-related macular degeneration. New England Journal of Medicine355(14), 1419–1431 (2006)

  7. [15]

    Thomas, M.: Reconnaissance par apprentissage profond de l’évolution de l’activité néovasculaire des DMLA en tomographie par cohérence optique. Ph.D. thesis, Uni- versité de Bretagne Occidentale, Brest (2021) 10 P. Zhang et al

  8. [16]

    In: 2024 IEEE International Symposium on Biomedical Imaging (ISBI)

    Zhang, P., Li, Y., Zhang, J., Jiang, W., Conze, P.H., Lamard, M., Quellec, G., Daho, M.E.H.: Detection and classification of glaucoma in the justraigs challenge: Achievements in binary and multilabel classification. In: 2024 IEEE International Symposium on Biomedical Imaging (...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.