REVIEW 4 major objections 5 minor 41 references
Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A masked autoencoder pretrained on over 5,000 unlabeled chest CT scans can outperform fully supervised baselines at classifying interstitial lung diseases when labelled data is scarce.
desk verdict Likely data leakage between MAE pretraining and the fine-tuning set undercuts the only strong result; the idea is worth a revision, not a rejection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the masked autoencoder: a ViT-B encoder that sees only 25% of the patchified volume (128×128×128, patch size 16) and must reconstruct the 75% masked patches; the decoder is shallow and the loss is mean absolute error on the masked voxels. The forced reconstruction of invisible structure is what pushes the encoder to learn spatial and textural regularities of lung parenchyma rather than surface cues. At classification time a [CLS] token is appended to the patch sequence, and either the whole encoder is fine-tuned or only the final LayerNorm-plus-linear head is trained on class-weighted cross-entropy; the paper's key comparison is between these two readouts and supervised baselin
What would settle it
Take the patient identifiers or file hashes of the 105 certified ILD scans and check them against the pretraining corpus. If any overlap exists, retrain the linear probe on the non-overlapping subset and see whether the 49.1% multiclass balanced accuracy falls toward the 42.5% radiomic baseline.
Extended reading notes
Core claim
On its own terms, the paper establishes that a masked autoencoder pretrained on CT volumes from multiple related conditions can serve as a feature extractor for interstitial lung disease classification when expert labels are scarce. The authors pretrain a ViT-B encoder on 128-cubed volumes from more than 5,000 chest CTs, using 75% random masking and voxel-level reconstruction, then evaluate two readouts on 105 certified ILD scans: full fine-tuning and linear probing on the [CLS] token. They report that linear probing gives 72.3% balanced accuracy on the binary UIP-versus-other task—tying the best supervised baseline—and 49.1% on the four-way task, beating the Inception-ResNet baseline (40.8%
Load-bearing premise
The load-bearing assumption is that the 105 certified ILD scans used to fine-tune and evaluate the model were not also among the unlabelled scans the MAE was pretrained on; if any of them were, the reported accuracy gains could reflect memorization of the evaluation patients rather than learned representations.
Editorial extensions
If this is right
- Large unlabelled CT collections from related conditions such as COVID-19 and pneumonia can be substituted for expensive ILD annotations in pretraining, reducing the labelled-data barrier.
- Linear probing is the recommended readout: it beat full fine-tuning in both the binary (72.3% vs 69.1%) and multiclass (49.1% vs 43.1%) tasks.
- The multiclass margin over a supervised baseline (49.1% vs 40.8%) is the paper's evidence that MAE features separate overlapping UIP/non-IPF patterns better than transfer-learning baselines.
- The released encoder and lung-segmentation model let other groups fine-tune for additional diffuse lung disease tasks without repeating the 5,000-scan pretraining.
Reading between the lines
- If these results survive an external cohort, the same recipe—MAE pretraining on related high-volume diseases, linear probe on a small certified set—could transfer to other rare diffuse lung diseases whose CT patterns overlap with more common ones.
- The consistent advantage of linear probing over full fine-tuning in both tasks suggests the pretrained features are informative but fragile; a middle path that updates only a few later layers might do even better.
- Because the multiclass task requires separating overlapping UIP patterns, the MAE's larger margin there suggests reconstruction pretraining encodes textural boundaries that supervised transfer from natural-image-style weights does not capture.
- A practical test: reconstruct masked lung patches and see whether UIP-specific reticulation and honeycombing are visually regenerated; if they are, reconstruction quality could double as a confidence signal for diagnosis.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using a 3D masked autoencoder (MAE) pretrained on a large, largely unlabelled chest CT collection (over 5,000 scans) to learn representations for interstitial lung disease (ILD) classification. The pretrained encoder is evaluated on two downstream tasks: a binary task (UIP/probable UIP vs. indeterminate UIP/non-IPF) and a four-class multiclass task, using both full fine-tuning and linear probing. The main evidence is that the MAE linear probe achieves a multiclass balanced accuracy of 49.1% compared to 40.8% for Inception-ResNet and 42.5% for radiomics. The paper also presents ablations on patch size and a lung-aware reconstruction loss. Code and models are released.
Significance. If the reported gains are real, the work would be a useful demonstration that self-supervised reconstruction pretraining on heterogeneous CT data can benefit a rare-disease classification task with only 105 labelled scans. The study is honest about the small evaluation set, and it compares against several baselines, including a clinical radiomics pipeline, which is a strength. However, the central claim hinges on an unstated and load-bearing data-separation assumption: that the fine-tuning scans were not part of pretraining. This issue, together with the absence of statistical significance testing, means the headline multiclass result is not yet established. The paper is reproducible in principle (code and models are promised), but the current text does not rule out a leakage-based explanation.
major comments (4)
- [Section 2.2 (Datasets), Tables 1 and 5] The manuscript never states that the 105 certified ILD scans used for fine-tuning were excluded from MAE pretraining. The pretraining ILD data are described as coming from '360 individuals in our study,' and Table 1 reports 105 certified ILD scans from the same institution. If these 105 volumes were among the pretraining data, the encoder could memorize patient-specific intensity patterns, directly inflating the linear-probe results in Tables 4 and 5. In particular, the multiclass advantage of MAE LP (49.1%) over Inception-ResNet (40.8%) is the only strong evidence for the paper's central claim, so this ambiguity is load-bearing. Please state explicitly whether the 105 certified scans are a subset of the 360 in-house pretraining volumes, and if they are, retrain the MAE with these volumes excluded and re-report all downstream numbers.
- [Section 3 (Tables 4 and 5)] All results are reported as means and standard deviations over five random 70:30 splits, but no statistical significance tests, confidence intervals, or per-split results are provided. The key multiclass difference (MAE LP 49.1 ± 7.0 vs. Inception-ResNet 40.8 ± 4.7) is within one standard deviation, and with only five splits the difference could plausibly be noise. Please report paired significance tests (e.g., paired t-test or Wilcoxon across the five splits) or bootstrap confidence intervals, and include per-split balanced accuracies. Without this, the claim that the MAE 'significantly outperforms' the baseline (Discussion) is unsupported.
- [Section 2.2 and Table 1] There is a numeric inconsistency in the ILD counts. The text says the ILD-positive pretraining data consist of '360 individuals in our study' plus '176 from open-source platforms', which totals 536. Table 1, however, lists 105 certified ILD and 472 uncertified ILD, which totals 577. Please reconcile these numbers and clarify how 'certified' and 'uncertified' ILD cases relate to the fine-tuning set and to the pretraining set. This ambiguity makes it impossible to verify the data separation assumed in the evaluation.
- [Section 2.3 (Evaluation metrics) and Section 3.1] The evaluation uses scan-level 70:30 splits, but it is not stated whether the 105 certified ILD scans come from distinct patients or whether some patients contribute multiple scans. If a patient appears in both the training and validation split, identity-based shortcuts could again inflate performance. Please state whether the splits are at the patient level, and if not, rerun the evaluation with patient-level splitting.
minor comments (5)
- [Section 5 title] 'Acknoledgements' is a typo; should be 'Acknowledgements'.
- [Table 4 caption] The caption contains a duplicated phrase: 'show our show our comparison'. Please fix.
- [Section 3.2 (Lung-cavity-aware Reconstruction Loss)] The loss introduces two hyperparameters (the 25% overlap threshold and α). Only one value of α is tested (α = 0.01), and no sensitivity analysis is reported. A sentence on the choice of these values would help reproducibility.
- [Section 2.2 (Preprocessing)] The preprocessing pipeline resamples all scans to a fixed 128×128×128 volume and applies a lung mask. It would be helpful to state the resampling target spacing explicitly, since this affects patch semantics.
- [Section 3.2 (Patch Size ablation)] The patch-size-8 experiment trains for 2000 epochs at batch size 8, whereas the main MAE training uses 198,000 iterations at batch size 64. The comparison is therefore confounded by compute budget. Please note this explicitly in the text.
Circularity Check
MAE pretraining dataset includes the 105 fine-tuning scans, so the reported downstream gains are not independent evidence.
-
other
[Section 2.2, Table 1 and 'Fine-tuning datasets' paragraph]
"Table 1: Number of CT scans per disease category. Certified refers to a diagnosis assigned through board consensus. Uncertified lacks board consensus. Disease Category Certified ILD Uncertified ILD COVID-19 Normal Pneumonia Number of Scans 105 472 4,296 330 60 ... As seen in Table 1, we have 105 CT scans which have been labelled according to the diagnostic criteria of the Fleischner Society [22]."
The pretraining dataset described in Table 1 includes exactly 105 Certified ILD scans, and the fine-tuning dataset is exactly those same 105 certified scans. No sentence anywhere states that these 105 scans were excluded from the MAE reconstruction pretraining. Therefore, the MAE encoder was trained to reconstruct the very volumes later used to evaluate the downstream classifier. The reported multiclass improvement of MAE linear probing (49.1% balanced accuracy) over Inception-ResNet (40.8%) and radiomic features (42.5%) is not clean evidence of transferable representation learning; it can be explained by the model having memorized patient-specific intensity patterns during pretraining. The supervised baselines never saw these unlabeled volumes, making the comparison unfair. The central cl
full rationale
The only substantive circularity is the dataset overlap: Table 1, which defines the pretraining collection, lists 105 Certified ILD scans, and the fine-tuning set is described as the same 105 certified scans, with no stated exclusion from pretraining. This is a load-bearing leak because the paper's strongest evidence (multiclass linear-probe superiority) depends on the MAE having learned features from unlabeled data that are then transferred to a labeled task; if those exact volumes were seen during reconstruction pretraining, the gains are confounded by memorization. No other circular patterns were found. Self-citations to prior work by the authors (e.g., [7], [9], [14]) are used for experimental setup or baselines and are not load-bearing for the central claim. The paper's own limitation statement acknowledges the small evaluation dataset and the need for an independent cohort, but it does not address the pretraining/fine-tuning overlap. Because the dataset definitions in the paper make the overlap apparent, the circularity is severe enough to undermine the main conclusion.
Assumptions & free parameters
free parameters (2)
- alpha (non-lung loss weight) =
0.01
- lung-overlap threshold =
25%
assumptions (3)
- domain assumption COVID-19 and ILD share sufficient CT features that pretraining on COVID and pneumonia scans transfers to ILD
- domain assumption Fleischner Society board-consensus labels are reliable ground truth
- domain assumption Lung cropping via LungMask and Hounsfield clipping preserve diagnostically relevant information
Cite this review
Pith. "Pith review of Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis." pith.science (2026). https://pith.science/paper/UX3QA6GZ
@misc{pith2026250804429,
author = {Pith},
title = {Pith review of: Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis},
year = {2026},
howpublished = {\url{https://pith.science/paper/UX3QA6GZ}},
note = {Machine review of arXiv:2508.04429}
}
read the original abstract
Masked autoencoders (MAEs) have emerged as a powerful approach for pre-training on unlabelled data, capable of learning robust and informative feature representations. This is particularly advantageous in diffused lung disease research, where annotated imaging datasets are scarce. To leverage this, we train an MAE on a curated collection of over 5,000 chest computed tomography (CT) scans, combining in-house data with publicly available scans from related conditions that exhibit similar radiological patterns, such as COVID-19 and bacterial pneumonia. The pretrained MAE is then fine-tuned on a downstream classification task for diffused lung disease diagnosis. Our findings demonstrate that MAEs can effectively extract clinically meaningful features and improve diagnostic performance, even in the absence of large-scale labelled datasets. The code and the models are available here: https://github.com/eedack01/lung_masked_autoencoder.
Figures
Reference graph
Works this paper leans on
-
[1]
An, P., Xu, S., Harmon, S.A., et al.: Ct images in covid-19 (2020) 4
work page 2020
-
[2]
Balestriero, R., LeCun, Y.: Learning by reconstruction produces uninformative features for perception (2024) 2
work page 2024
-
[3]
Cardoso, M.J., Li, et al.: Monai: An open-source framework for deep learning in healthcare (2022) 2
work page 2022
-
[4]
Chen, C., Zhang, J., Xu, Y., et al.: Why do we need large batchsizes in contrastive learning? a gradient-bias perspective (2022) 2
work page 2022
-
[5]
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations (2020) 2
work page 2020
-
[6]
Chen, Z., Agarwal, D., Aggarwal, K., et al.: Masked image modeling advances 3d medical image analysis (2022) 2, 5
work page 2022
-
[7]
Christe, A., Peters, A., Drakopoulos, e.a.: Computer-aided diagnosis of pulmonary fibrosis using deep learning and ct images. Invest Radiol (2019) 3, 4
work page 2019
-
[8]
Dack, E., Christe, A., Fontanellaz, M., et al.: Artificial intelligence and interstitial lung disease: Diagnosis and prognosis. Invest Radiol (2023) 1, 7
work page 2023
Show all 41 references
-
[9]
In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision (2023) 2
Dack, E., Brigato, L., McMurray, M., et al.: An empirical analysis for zero-shot multi-label classification on covid-19 ct scans and uncurated reports. In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision (2023) 2
2023
-
[10]
Health Technol
Das, S., Ayus, I., Gupta, D.: A comprehensive review of COVID-19 detection with machine learning and deep learning techniques. Health Technol. (2023) 1
2023
-
[11]
Computerized Medical Imaging and Graphics (2012) 4
Depeursinge, A., Vargas, A., Platon, e.a.: Building a reference multimedia database for interstitial lung diseases. Computerized Medical Imaging and Graphics (2012) 4
2012
-
[12]
Desai, S., Baghal, A., Wongsurawat, T., et al.: Data from chest imaging with clinical and genomic correlates representing a rural covid-19 positive population (2020) 4 10 Dack, Brigato, et al
2020
-
[13]
Clinical Imaging (2023) 4
Durhan, G.e.a.: Two in one: Overlapping ct findings of covid-19 and underlying lung diseases. Clinical Imaging (2023) 4
2023
-
[14]
IEEE Access (2024) 3, 4, 5, 6, 8
Fontanellaz, M., Christe, A., Christodoulidis, e.a.: Computer-aided diagnosis sys- temforlungfibrosis:fromtheeffectofradiomicfeaturesandmulti-layer-perceptron mixers to pre-clinical evaluation. IEEE Access (2024) 3, 4, 5, 6, 8
2024
-
[15]
Gani, H., Naseer, M., Yaqub, M.: How to train vision transformer on small-scale datasets? (2022) 8
2022
-
[16]
He, K., Chen, X., Xie, S., et al.: Masked autoencoders are scalable vision learners (2021) 2
2021
-
[17]
Eur Radiol Exp (2020) 4
Hofmanninger, J., Prayer, F., Pan, J., et al.: Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur Radiol Exp (2020) 4
2020
-
[18]
npj Digital Medicine (2023) 2
Huang, S., Pareek, A., Jensen, M., et al.: Self-supervised learning for medical im- age classification: a systematic review and implementation guidelines. npj Digital Medicine (2023) 2
2023
-
[19]
Nature Methods (2020) 5
Isensee, F., Jaeger, P.F., Kohl, e.a.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods (2020) 5
2020
-
[20]
Kiser, K.J.e.a.: Data from the thoracic volume and pleural effusion segmentations in diseased lungs for benchmarking chest ct processing pipelines (plethora) (2020) 4
2020
-
[21]
In: MICCAI (2024) 2
Kunanbayev, K., Shen, V., Kim, D.S.: Training ViT with Limited Data for Alzheimer’s Disease Classification: an Empirical Study . In: MICCAI (2024) 2
2024
-
[22]
Lancet Respiratory Medicine (2018) 2, 4
Lynch, D.A., Sverzellati, N., Travis, W.D., et al.: Diagnostic criteria for idiopathic pulmonary fibrosis: A fleischner society white paper. Lancet Respiratory Medicine (2018) 2, 4
2018
-
[23]
Medical Physics (2021) 4
Ma, J., Wang, Y., An, X., et al.: Towards data-efficient learning: A benchmark for covid-19 ct lung and infection segmentation. Medical Physics (2021) 4
2021
-
[24]
Radiology: Artificial Intelligence (2022) 5
Mei, X., Liu, Z., Robson, P.M., et al.: Radimagenet: An open radiologic deep learn- ing research dataset for effective transfer learning. Radiology: Artificial Intelligence (2022) 5
2022
-
[25]
Morozov, S.P., Andreychenko, A.E., Pavlov, N.A., et al.: MosMedData: Chest CT scans with COVID-19 related findings dataset (2020) 4
2020
-
[26]
medRxiv (2022) 1
Ng, Y.A., et al.: A novel workflow for the safe and effective integration of ai as supporting reader in double reading breast cancer screening: A large-scale retro- spective evaluation. medRxiv (2022) 1
2022
-
[27]
Oikonomou, A., et al., K.P.: COVID-CT-MD: COVID-19 Computed Tomography Scan Dataset Applicable in Machine Learning and Deep Learning (2021) 4
2021
-
[28]
International Journal of Molecular Sciences (2021) 1
Pratella, D., Ait-El-Mkadem Saadi, S., Bannwarth, S., et al.: A survey of autoen- coder algorithms to pave the diagnosis of rare diseases. International Journal of Molecular Sciences (2021) 1
2021
-
[29]
ex- panding horizons in radiology
Rea, G., Sverzellati, N., Bocchino, M., et al.: Beyond visual interpretation: Quan- titative analysis and artificial intelligence in interstitial lung disease diagnosis "ex- panding horizons in radiology". Diagnostics (Basel) (2023) 1
2023
-
[30]
Ra- diology (2021) 4
Revel, Marie-Pierre, e.a.: Study of thoracic ct in covid-19: The stoic project. Ra- diology (2021) 4
2021
-
[31]
Procedia Computer Science (2024) 2
Saha, S., Kumar, A., Nandi, D.: Vit-ild: A vision transformer-based neural network for detection of interstitial lung disease from ct images. Procedia Computer Science (2024) 2
2024
-
[32]
Kaggle (2020) 4 Unmasking Interstitial Lung Diseases 11
Shahin, A., Wegworth, C., David, Estes, E., et al.: Osic pulmonary fibrosis pro- gression. Kaggle (2020) 4 Unmasking Interstitial Lung Diseases 11
2020
-
[33]
Steiner, A., Kolesnikov, A., Zhai, X., et al.: How to train your vit? data, augmen- tation, and regularization in vision transformers (2022) 8
2022
-
[34]
Nature Biomedical Engineering (2022) 5
Tiu, E., Talius, E., Patel, P., et al.: Expert-level detection of pathologies from unannotated chest x-ray images via self-supervised learning. Nature Biomedical Engineering (2022) 5
2022
-
[35]
Eur Radiol (2020) 1, 7
Trusculescu, A., Manolescu, D., Tudorache, E., et al.: Deep learning in interstitial lung disease-how long until daily practice. Eur Radiol (2020) 1, 7
2020
-
[36]
Lancet Respir Med (2018) 5, 6, 8
Walsh, S., Calandriello, L., Silva, M., et al.: Deep learning for classifying fibrotic lung disease on high-resolution computed tomography: a case-cohort study. Lancet Respir Med (2018) 5, 6, 8
2018
-
[37]
Frontiers in Medicine (2023) 2
WaseemSabir,M.,Farhan,M.,Almalki,N.S.,etal.:FibroVit—Visiontransformer- based framework for detection and classification of pulmonary fibrosis from chest CT images. Frontiers in Medicine (2023) 2
2023
-
[38]
In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2023) 7
Xiao, J., Bai, Y., Yuille, e.a.: Delving into masked autoencoders for multi-label thorax disease classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2023) 7
2023
-
[39]
arXiv:2107.01682 (2021) 2
Xiaohong Gao, Yu Qian, A.G.: Covid-vit: Classification of covid-19 from ct chest images based on vision transformer models. arXiv:2107.01682 (2021) 2
2021 arXiv
-
[40]
Bioengineering (2021) 4
Zaffino, P., Marzullo, A., Moccia, S., et al.: An open-source covid-19 ct dataset with automatic lung tissue classification for radiomics. Bioengineering (2021) 4
2021
-
[41]
Zhou, L., Liu, H., Bae, J., et al.: Self pre-training with masked autoencoders for medical image classification and segmentation (2023) 2
2023
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.