{"id":"77dd7d6f-d61a-41de-8bfa-5fc9675109e0","arxiv_id":"2504.19357","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MERA, a self-supervised Vision Transformer pipeline with sparse-seeded active learning, reports 86.2% malignancy accuracy on LIDC with roughly 1% of training labels, close to its 87.6% fully supervised accuracy, while producing global, case-based, visual, and concept explanations.","lead":"A lung nodule diagnosis model built mostly from unlabeled CT images plus a tiny number of labeled examples (about 1%) reaches accuracy close to models trained on fully labeled data on the LIDC dataset, while producing several types of human-readable explanations. A generalist reader should pay attention because it is an open-sourced demonstration that self-supervised pretraining and active learning can cut annotation costs in medical imaging.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unequal baselines, not the disclosed ImageNet pretraining, are the main threat: the 'exceeding SOTA' claim rests on cross-protocol numbers never re-run on MERA's own split.","rationale":"The reader correctly identified the ImageNet-pretrained DINO initialization as a factor, but I see it as secondary: the paper discloses the initialization in Sec. 4.1.2, and the claim at issue is only about the number of annotated target-domain samples, not about training from scratch. The more directly load-bearing problem is the baseline comparison. The abstract promises 'comparable to or exceeding state-of-the-art methods that require full annotation.' That comparative claim is supported only by citing empirical numbers from papers that use different data selection, preprocessing, input dimensionality, and auxiliary supervision. The manuscript's own statement in Sec. 4.1.1 admits these differences make direct comparison difficult, but it never re-runs any competitor on its own split. Moreover, the single within-protocol full-annotation baseline it does report (ResNet-50, 88.08%) beats MERA at both 1% and 100% labels, which means the 'exceeding' wording is already undercut by the paper's own controlled result. The proposed concrete test—re-running at least one strong competitor on the same 730-nodule protocol—would settle whether the central claim is just an artifact of protocol mismatch. If the re-run baselines exceed MERA's 1% accuracy, the claim should be toned down to 'competitive with full-annotation baselines on this protocol' or the comparison should be made equal. If MERA still matches/exceeds them, then the 1%-annotation result is genuinely surprising. In either case, the current evidence is insufficient to support 'exceeding SOTA' as stated, so the conditional verdict remains appropriate.","tokens_in":23271,"tokens_out":7160,"duration_ms":69449,"concrete_test":"Re-run the compared SOTA methods (or at least X-Caps as the most similar, plus a strong 2D/3D full-annotation baseline such as MTMR) on MERA's exact data protocol: LIDC nodules with ≥3 radiologist annotations, slice thickness ≤2.5 mm, 2D 32×32 central axial patches, 70/30 nodule-level split (518 train/212 test), median-aggregated labels, malignancy binarized at threshold 3. Compare their full-annotation malignancy accuracy with MERA's 1%-annotation result (86.22±2.51%). If a re-run baseline exceeds 86.22%, the 'exceeding SOTA' claim is falsified; if MERA matches or exceeds it, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim 'comparable to or exceeding state-of-the-art methods requiring full annotation' is not testable from the reported experiments. All comparisons to HSCNN, X-Caps, MSN-JCN, MTMR, and WeakSup are against published numbers from different pipelines: 1149–4252 nodules, often 3D volumes, and extra supervision (segmentation masks, diameters, all slices). The paper itself concedes in Sec. 4.1.1 that these discrepancies 'increase the difficulty in making direct comparisons,' yet no baseline is re-run on MERA's protocol (730 nodules, 70/30 nodule-level split, 2D 32×32 patches). Within that protocol, the only full-annotation baseline reported—a conventional ResNet-50 end-to-end CNN—achieves 88.08% (Tab. 3), which exceeds both MERA's full-annotation 87.56% and its 1%-annotation 86.22±2.51%. Thus the 'exceeding' portion of the claim rests entirely on incomparable literature numbers. The ImageNet-pretrained DINO initialization (Sec. 4.1.2) is disclosed and affects absolute accuracy, but it does not invalidate the annotation-efficiency claim; the comparison mismatch does.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MERA, a two-stage pipeline for lung nodule malignancy diagnosis that combines self-supervised contrastive pretraining of a Vision Transformer with a weakly supervised hierarchical predictor over radiologist-defined nodule attributes. The authors claim that with only 1% of annotated samples MERA reaches diagnostic accuracy comparable to or exceeding that of state-of-the-art methods that use full annotation, while also providing model-level, instance-level, local visual, and concept-level explanations. Experiments on the LIDC dataset report 86.22±2.51% malignancy accuracy with 1% annotations versus 87.56±0.61% with full annotations under the same protocol, together with per-attribute accuracies above 90% and a series of qualitative explanation case studies.","tokens_in":23388,"tokens_out":8092,"duration_ms":77021,"significance":"The core same-protocol result, 1% versus 100% annotation (86.22±2.51% versus 87.56±0.61%, Table 2), is internally consistent and represents a genuinely valuable empirical contribution: it shows that sparse seeding followed by dynamic pseudo-labelling with quenching can stabilize training in a very-low-label regime. The paper also ships its code, reports standard deviations, and includes ablations of the seeding, acquisition, pseudo-labelling, and quenching components. If the central comparison to prior work were placed on equal footing, the proposed pipeline would be a meaningful step toward transparent, low-annotation medical image diagnosis. The positive assessment is conditional, however, because the headline 'exceeding state-of-the-art' claim currently rests on published numbers from substantially different data protocols that are never re-run on the authors' split.","major_comments":[{"comment":"The central claim that MERA with 1% annotations 'exceeds state-of-the-art methods requiring full annotation' is not supported by the reported experiments. All five cited baselines (HSCNN, X-Caps, MSN-JCN, MTMR, WeakSup) are taken from published numbers obtained on different protocols (1149-4252 nodules, 3D volumes or multiple 2D slices, and in some cases additional supervision such as segmentation masks or diameter information), a limitation the manuscript itself acknowledges in Sec. 4.1.1. The only full-annotation method re-run on MERA's own 730-nodule, 70/30 nodule-level split is an end-to-end ResNet-50, which reaches 88.08% (Table 3), above both MERA's full-annotation 87.56% and its 1%-annotation 86.22±2.51%. The 'exceeding' part of the headline is therefore not testable from the reported data; the authors should either re-run the baselines on their split or explicitly downgrade the claim to 'comparable' with a clear protocol caveat.","section":"Abstract, Sec. 4.1.1, Tables 1-2"},{"comment":"The 1%-annotation result is conditional on initializing the ViT with ImageNet-pretrained self-supervised DINO weights. Table 3 shows that removing this initialization drops ViT two-stage malignancy accuracy from 87.56% to 79.19%, so the reported absolute performance largely inherits ImageNet-learned representations rather than demonstrating that unlabelled lung CT alone plus 1% labels suffices. This does not invalidate the annotation-efficiency comparison between 1% and 100% labels, because both use the same initialization, but the abstract and Sec. 5 should state this dependency explicitly when describing the method as 'primarily unsupervised.'","section":"Sec. 4.1.2, Table 3"},{"comment":"The manuscript presents uncertainty-sampling active learning as a core component of the annotation exploitation mechanism, but the footnote to Table 4 states that the 1% column 'Does not contain requested annotations.' This means the headline 1%-annotation result is obtained without active learning, using only sparse seeding, pseudo-labelling, and quenching. Please state this explicitly in the method description and abstract, or clarify how the annotation budget is counted if actively requested labels are in fact used at the 1% setting.","section":"Sec. 3.1.2, Table 4 footnote"}],"minor_comments":[{"comment":"In the 1% column, the rows 'sparse integrated entropy' and 'sparse malignancy confidence' report exactly the same mean and standard deviation (86.22±2.51). Please clarify whether this is a typo or a genuine coincidence under the no-requested-annotations setting.","section":"Table 4"},{"comment":"For the k-NN case-based explanation results under partial annotation, the paper should specify how unlabeled training samples are treated when selecting the k nearest neighbors and when assigning the majority label, since with only 10% of training labels the nearest neighbors may often lack ground-truth annotations.","section":"Sec. 4.3, Table 1"},{"comment":"The annotations 'Annotaion 10x' and 'Accuracy 0.43%' are ambiguous, and the figure would benefit from stating explicitly which baseline is used and how its annotation reduction is performed.","section":"Fig. 11"},{"comment":"There are numerous rendering artifacts and typos, such as 'di fferent' and 'o ffer' instead of 'different' and 'offer', and 'inference phrase' in Sec. 3 instead of 'inference phase'; these should be corrected during copy-editing.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core same-protocol experiment is sound and publishable in principle, but the superiority claim over full-annotation state-of-the-art methods must be repaired by re-running baselines under the same protocol or by substantially qualifying the claim. Please also check the duplicated entries in Table 4 and clarify the active-learning accounting at the 1% setting before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's core within-protocol result—1% labels give 86.22±2.51% vs 87.56±0.61% with full labels on the same 730-nodule LIDC split—is internally consistent, ablated, and backed by open-source code. That's a real contribution to low-annotation medical imaging. The headline, however, overreaches: 'exceeding state-of-the-art' rests on literature numbers from different protocols (1149–4252 nodules, 3D volumes, extra supervision) that are never re-run on MERA's split. The paper even concedes the mismatch in Sec. 4.1.1. And the one full-annotation baseline they do run on their own protocol, a ResNet-50 CNN, gets 88.08%, above both MERA's 1% and 100% numbers. So the 'exceeding' part is not supported; 'comparable within protocol' would be.\n\nWhat's genuinely new is the specific integration: DINO/ViT features, k-means seeded sparse annotation, uncertainty sampling plus pseudo-labelling with quenching, and attribute-to-malignancy hierarchical prediction, all in one system at 1% annotation. The ablations in Tab. 4 are useful—sparse seeding is what stabilizes the 1% regime. Attribute accuracies of 91–96% at 1% are strong in context. Open-source code is a real plus.\n\nSoft spots, in order. First, the SOTA comparison is the load-bearing issue; baselines must be re-run on the same split or the claim toned down. Second, the ImageNet-pretrained DINO initialization is disclosed and honestly quantified (Tab. 3 drops ViT two-stage from 87.56 to 79.19 without it), but it means 'unsupervised' mostly inherits ImageNet representations; that doesn't invalidate the annotation-efficiency claim, it just qualifies the 'reduced annotation' framing. Third, explainability is evaluated qualitatively plus a k-NN proxy; attention maps are not scored against any localization benchmark, so 'superior' explainability is not established. Fourth, the delta over the authors' own cRedAnno/cRedAnno+ conference papers is never stated, which matters since the code repo is shared.\n\nWho should read this: people working on low-annotation medical image classification or concept-bottleneck-style models. It deserves a serious referee. My recommendation: send to review, but require same-protocol baseline comparisons (or a scaled-back headline) and an explicit comparison with the prior cRedAnno papers. The central within-protocol result looks solid; the framing is what needs fixing.","headline":"Solid within-protocol annotation-efficiency result; the exceeding-SOTA headline rests on cross-protocol baselines that were never re-run on MERA's own split.","tokens_in":24090,"tokens_out":2957,"would_cite":false,"duration_ms":28041,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lung nodule AI needs only 1% of labels to match fully supervised accuracy.","keywords":["explainable artificial intelligence","lung nodule diagnosis","self-supervised learning","Vision Transformer","semi-supervised active learning","LIDC dataset","reduced annotation"],"falsifier":"Train the Stage 1 encoder from scratch on unlabelled LIDC CT patches with no ImageNet weights and measure malignancy accuracy at 1% annotation; if accuracy falls toward the 79.19% level reported in Table 3 rather than staying near 86%, the 1%-annotation result is an artefact of pretrained representation transfer, not of self-supervised learning on the target domain. Re-running the literature baselines on MERA's exact 730-nodule split would similarly test whether the performance advantage survives equal footing.","tokens_in":1634,"feed_emoji":"🫁","tokens_out":1796,"duration_ms":56570,"temperature":0.7,"pith_summary":"This paper tries to show that a lung nodule malignancy classifier can be built almost without labels: using only 1% of annotated training samples on the LIDC protocol, MERA reports malignancy accuracy of 86.22 ± 2.51%, close to the 87.56 ± 0.61% it reaches with full annotation and competitive with fully supervised methods. The same model predicts clinically defined nodule attributes at 91–96% accuracy and emits four kinds of explanations—attention maps, nearest-case retrieval, latent-space clustering, and attribute-based concept explanations—most of which need no labels at all. The motivation is practical: expert annotation of CT nodules is scarce and costly, and trust in diagnostic AI depends on explanations that align with radiological reasoning.","feed_headline":"Lung nodule AI matches full-annotation accuracy with 1% of labels","feed_subtitle":"A self-supervised encoder plus sparse active learning yields 86% accuracy and multi-level explanations from a handful of labels.","key_machinery":"The load-bearing component is the two-stage training schedule. Stage 1 uses DINO self-supervised contrastive learning on a ViT-Small encoder (starting from ImageNet-pretrained DINO weights) to map 32 × 32 axial CT patches into a semantically organised latent space. Stage 2 trains one linear predictor per nodule attribute plus a malignancy predictor on the concatenation of image features and predicted attributes; the annotation exploitation mechanism selects seeds by k-means clustering, requests labels for low-confidence samples while pseudo-labelling high-confidence ones, and periodically reinitialises the predictors ('quenching') to curb confirmation bias. The argument is that Stage 1 supplies most of the representation, so Stage 2 needs only a handful of labels.","core_discovery":"MERA's central claim is that a self-supervised Vision Transformer, trained with DINO-style contrastive learning on unlabelled nodule patches, creates a latent space with enough semantic separability that a linear predictor trained on a tiny labelled seed can classify malignancy and nodule attributes nearly as well as a fully supervised model. The predictor is trained with sparse seeding via clustering, then refined by semi-supervised active learning with dynamic pseudo-labels and periodic reinitialisation ('quenching'). On the paper's 730-nodule LIDC split, 1% annotation yields 86.22 ± 2.51% malignancy accuracy and 91–96% per-attribute accuracy, versus 87.56 ± 0.61% with full labels. Explanations are generated intrinsically: t-SNE clustering shows malignancy-correlated groupings, k-nearest neighbours supply case-based rationale, averaged self-attention maps localise diagnostically relevant features, and predicted nodule attributes feed the malignancy decision.","pith_inferences":["Editorial inference: because removing the ImageNet DINO initialisation drops two-stage ViT malignancy accuracy from 87.56% to 79.19% (Table 3), the paper supports 'a good latent representation plus a few labels', not yet 'unlabelled lung CT alone suffices'. A definitive test would pretrain on unlabelled thoracic CT without ImageNet and re-measure the 1% accuracy.","Editorial inference: the 'exceeding state-of-the-art' comparisons use literature numbers from different data protocols (1,149–4,252 nodules, 3D volumes, extra supervision). Re-running those baselines on MERA's exact split would settle whether the advantage is real or protocol-driven.","Editorial inference: the same two-stage recipe—self-supervised encoder, clustered seed selection, and dynamic pseudo-labelling with quenching—could transfer to other lesion-classification tasks with scarce labels, provided a similar semantic attribute set exists; that transfer is not tested in the paper."],"forward_implications":["If the finding holds, a diagnostic AI for lung nodules can be trained with about five labelled nodules on a 518-nodule training set, cutting annotation cost by roughly two orders of magnitude.","The four explanation channels (attention maps, nearest cases, cluster structure, attribute concepts) are generated without additional supervision, except for attribute prediction, so explainability does not require a large labelled dataset.","Simultaneous high accuracy on all nodule attributes means the concept explanations are not just rhetorical; they can serve as a checkable intermediate output before the malignancy verdict.","The reported robustness under annotation reduction—comparable accuracy at 1%, 10%, and 100% labels—suggests the model keeps working as expert labels become scarce.","The method's reliance on hundreds of unlabelled CT patches indicates that unlabelled data curation, not annotation volume, becomes the main practical requirement for deploying it."],"supporting_citations":[{"why":"Supplies the DINO self-supervised contrastive learning method that trains the Stage 1 encoder.","marker":"[14]"},{"why":"Supplies the Vision Transformer backbone that the method relies on for unsupervised feature extraction.","marker":"[15]"},{"why":"Provides the LIDC dataset that all experiments and baseline comparisons are built on.","marker":"[19]"},{"why":"X-Caps is the most similar fully supervised explainable baseline that MERA must beat.","marker":"[5]"},{"why":"WeakSup is the weakly supervised partial-annotation baseline that MERA compares against.","marker":"[8]"},{"why":"Defines the sample-selection and preprocessing protocol that determines the 730-nodule split.","marker":"[51]"},{"why":"Basis for the semi-supervised active learning strategy used to request and pseudo-label annotations in Stage 2.","marker":"[16]"},{"why":"Supplies the clustering approach used to select sparse seed samples in the learned latent space.","marker":"[46]"}],"fun_headline_variants":["Self-explaining AI reads lung scans with 1% annotation","Lung nodule AI matches full data with just 1% of labels","1% labels: lung nodule AI achieves full-annotation accuracy","MERA: explainable lung nodule AI from 1% of labels","AI explains lung nodule calls using only 1% of labels"],"cache_read_input_tokens":25984,"weakest_assumption_plain":"The central claim that 1% annotations suffice rests on initializing the encoder with ImageNet self-supervised weights, since removing that initialization drops two-stage malignancy accuracy from 87.56% to 79.19% in Table 3.","fun_headline_variants_meta":{"raw":{"variants":["Self-explaining AI reads lung scans with 1% annotation","Lung nodule AI matches full data with just 1% of labels","1% labels: lung nodule AI achieves full-annotation accuracy","MERA: explainable lung nodule AI from 1% of labels","AI explains lung nodule calls using only 1% of labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000588,"raw_usage":{"total_tokens":2806,"prompt_tokens":1034,"completion_tokens":1772,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":1678}},"tokens_in":650,"tokens_out":1772,"duration_ms":11825,"temperature":1.0,"reasoning_tokens":1678,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:56:56.395110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the Stage 1 encoder from scratch on unlabelled LIDC CT patches with no ImageNet weights and measure malignancy accuracy at 1% annotation; if accuracy falls toward the 79.19% level reported in Table 3 rather than staying near 86%, the 1%-annotation result is an artefact of pretrained representation transfer, not of self-supervised learning on the target domain. Re-running the literature baselines on MERA's exact 730-nodule split would similarly test whether the performance advantage survives equal footing.","supporting_citations":[{"cited_title":"Caron, H","cited_arxiv_id":null,"evidence_quote":"Supplies the DINO self-supervised contrastive learning method that trains the Stage 1 encoder."},{"cited_title":"Dosovitskiy, L","cited_arxiv_id":null,"evidence_quote":"Supplies the Vision Transformer backbone that the method relies on for unsupervised feature extraction."},{"cited_title":"LaLonde, D","cited_arxiv_id":null,"evidence_quote":"X-Caps is the most similar fully supervised explainable baseline that MERA must beat."},{"cited_title":"Joshi, J","cited_arxiv_id":null,"evidence_quote":"WeakSup is the weakly supervised partial-annotation baseline that MERA compares against."},{"cited_title":"Baltatzis, K.-M","cited_arxiv_id":null,"evidence_quote":"Defines the sample-selection and preprocessing protocol that determines the 730-nodule split."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the clustering approach used to select sparse seed samples in the learned latent space."}],"review_version":1}