{"id":"0b14050e-115f-405d-b053-4c79d485abc0","arxiv_id":"2501.05488","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"EndoDINO, a DINOv2-style model pre-trained on up to 10 million curated endoscopy images, reports strong or state-of-the-art results on three GI endoscopy benchmarks with frozen features and simple heads.","lead":"The authors trained a family of AI models called EndoDINO on millions of endoscopy images from over 130,000 videos, then showed frozen versions can handle landmark classification, polyp segmentation, and ulcerative colitis scoring with small prediction heads. If the results hold, one pre-trained backbone could power several real-time endoscopy AI tools instead of separate models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Checkpoint selection on the LIMUC target task makes the MES state-of-the-art claim circular; without a validation-only selection protocol the headline result is unverifiable.","rationale":"The reader's verdict is CONDITIONAL, and the reader's rationale already identifies checkpoint selection on the LIMUC target task as 'the most serious methodological issue.' My stress test converges on the same mechanism but makes it the primary load-bearing concern, rather than the unverifiable data provenance cited as the weakest assumption. The checkpoint-selection issue is more concrete and more directly testable: if Figure 1 used the LIMUC test set to pick the model, the central 'state-of-the-art' claim for Mayo endoscopic scoring is circular, and Table 5 cannot be compared to prior results. The landmark split issue is also real but secondary, since the paper itself does not claim a comparable 80/10/10 SOTA. Because the concern is not yet confirmed—the paper could have used only validation splits—the appropriate outcome is to keep the CONDITIONAL verdict, with the condition being a clarification and re-analysis of checkpoint selection on LIMUC. I therefore recommend UNCHANGED rather than REJECT, while noting that a confirmed test-set selection would push the verdict toward rejection of the MES claim.","tokens_in":8088,"tokens_out":4631,"duration_ms":44833,"concrete_test":"Inspect the code or checkpoint logs to determine which LIMUC images were used to produce the red curve in Figure 1. If any test-set image contributed to checkpoint selection, re-run Table 5 by selecting checkpoints using only the 85% training partitions (e.g., via the 10-fold cross-validation within the training data) and report the best AUROC and macro F1 on the untouched 15% test set. If the selected checkpoint changes and the new numbers drop below prior work, the SOTA claim for MES is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 states that the authors 'created an evaluation pipeline on an array of downstream tasks to select our best-performing checkpoints,' and Figure 1 plots 'performance on our LIMUC 4 class MES task' as a function of training step. Section 3.4 then reports Table 5 on the LIMUC 15% holdout test set using the same folds as Polat et al. If the checkpoint-selection curve in Figure 1 was computed on that same test set, or on any split that includes test images, the reported AUROC of 0.942 and macro F1 of 0.715 are the result of test-set selection. This would invalidate the comparison to prior work, which did not select checkpoints on the target test set. The paper never states that checkpoint selection was restricted to training or validation partitions; the phrase 'evaluation pipeline' and the direct overlay with a target task in Figure 1 suggest otherwise. A secondary issue reinforces the concern: the abstract's landmark-classification SOTA claim rests on a 1/10/10 split with no prior baseline reported on that split, while the comparable 80/10/10 result is only equal to LVD-142M DINOv2 (0.833 macro F1), not SOTA.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EndoDINO, a family of DINOv2-style vision transformers (ViT-B/14, ViT-L/14, ViT-g/14) pre-trained on a private dataset of up to 10M curated frames sampled from 130,037 VirgoCloud GI endoscopy videos. The authors evaluate EndoDINO as a frozen feature encoder with simple heads on four public benchmarks: HyperKvasir anatomical landmark classification (80/10/10 and 1/10/10 splits), KvasirSEG polyp segmentation, HyperKvasir 3-class Mayo endoscopic scoring, and LIMUC 4-class Mayo endoscopic scoring. The central claim, stated in the abstract, is that EndoDINO achieves state-of-the-art performance on these tasks.","tokens_in":8322,"tokens_out":2964,"duration_ms":30050,"significance":"If the stated results are valid, the paper would provide useful evidence that a large, domain-specific self-supervised pre-training corpus can yield a frozen backbone whose features match or exceed ImageNet/LVD-pretrained features on downstream GI endoscopy tasks with minimal task-specific adaptation. The scale of the pre-training video pool (130K videos) is substantially larger than prior public efforts, and the use of a curation pipeline based on deduplication and hierarchical k-means is methodologically interesting. However, the paper does not release the pre-training dataset, the pre-trained checkpoints, or evaluation code, and the central generalization claim currently rests on an unverifiable assumption that the private pre-training data are unrelated to the public evaluation sets. The checkpoint-selection procedure and the overstatement of the landmark-classification result are additional load-bearing concerns that need to be addressed before the main claims can be accepted.","major_comments":[{"comment":"The abstract claims state-of-the-art performance in anatomical landmark classification, but Table 2 does not support this claim on the standard 80/10/10 split: EndoDINO ViT-B/14 achieves macro F1 0.833, which is equal to both the LVD-142M DINOv2 ViT-B/14 and ViT-g/14 baselines, not greater. The 1/10/10 split results (macro F1 up to 0.997) have no prior baseline reported on the same split, so they cannot be called state-of-the-art either. Please either revise the claim or add a same-split prior comparison.","section":"§3.1, Table 2 and Abstract"},{"comment":"The checkpoint-selection procedure creates a circularity risk for the headline LIMUC 4-class MES result. Section 2.2 states that an evaluation pipeline on downstream tasks was used to select the best checkpoints, and Figure 1 plots performance on the LIMUC 4-class MES task as a function of training step. Section 3.4 then reports Table 5 on the LIMUC 15% holdout test set using the folds from Polat et al. If the selection curve in Figure 1 was computed on that same test set, or on any partition containing test images, then the reported AUROC of 0.942 and macro F1 of 0.715 are the result of test-set selection, and the comparison to prior work is invalid. The manuscript never states that checkpoint selection was restricted to training or validation partitions. Please specify the exact splits used for checkpoint selection and, if selection used test data, re-run the evaluation with a validation-only selection protocol.","section":"§2.2, Figure 1, §3.4, Table 5"},{"comment":"The central generalization claim in Section 4 is that evaluation datasets are 'completely unrelated data capture efforts' to the pre-training data, so the reported performance reflects generalization rather than memorization. This is unverifiable as written because the pre-training videos are private and no data provenance or leakage analysis is provided. If any HyperKvasir or LIMUC images originated from the same source, patient population, or recording session as the VirgoCloud pre-training videos, the generalization claim would not follow. Please provide a concrete overlap/deduplication analysis between pre-training and evaluation data, or explicitly weaken the claim to state that no such analysis was performed.","section":"§4 (Discussion) and §2 (Data)"}],"minor_comments":[{"comment":"The phrase 'evaluation pipeline on an array of downstream tasks' is vague; the manuscript should list which downstream tasks and which splits were used for checkpoint selection, separately from the final evaluation protocol.","section":"§2.2"},{"comment":"The comparison across EndoDINO-1M, EndoDINO-5M, and EndoDINO-10M is confounded: model size and pre-training dataset size change simultaneously, so the table cannot support any scaling conclusion about dataset size alone.","section":"Table 2"},{"comment":"The paper reports point estimates without standard deviations or confidence intervals, despite using 2-fold cross-validation (Table 4) and 10-fold cross-validation (Table 5). Reporting variability would help assess whether the observed gaps are meaningful.","section":"Tables 4 and 5"},{"comment":"The text says the ViT-g/14 linear head (1k to 6k parameters) is 'almost able to achieve similar performance' to the best end-to-end ViT-B/14 model; the gap in mDice is 0.021 (0.896 vs 0.875), which may be worth stating explicitly to avoid over-interpreting the qualitative phrase.","section":"§3.2, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is operator-dependent in a way that is difficult to verify: the pre-training data are proprietary, the checkpoints are not released, and the checkpoint-selection protocol is not fully specified. The circularity concern on LIMUC is the most serious issue; if the authors cannot demonstrate validation-only checkpoint selection, the MES result should be downgraded. The landmark-classification claim in the abstract should be corrected regardless. The paper is potentially valuable, but the revision needs to address the selection protocol and the overclaim before it can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"EndoDINO is a genuinely useful empirical study of in-domain pre-training for GI endoscopy, but the headline claims are overstated and the MES result has a circular-checkpoint-selection problem.\n\nWhat is new: the authors assembled the largest GI endoscopy video pool we know of (130K videos, 3.5B frames), curated it with near-duplicate removal and hierarchical k-means, and pre-trained ViT-B/L/g models on 1M-10M curated images. That resource and the scaling study (data size vs. model size) are real contributions. The evaluation design is also cleaner than much prior work: frozen backbones with simple heads on public benchmarks (HyperKvasir, KvasirSEG, LIMUC), with direct DINOv2 comparisons. On polyp segmentation and 3-class MES, EndoDINO gives modest but consistent gains over LVD-142M DINOv2.\n\nSoft spots:\n\n1. The abstract's landmark-classification SOTA claim does not survive Table 2. On the standard 80/10/10 split, EndoDINO matches DINOv2 (0.833 macro F1) rather than beating it. The few-shot 1/10/10 results are impressive but have no prior baseline, so 'state-of-the-art' is unsupported.\n\n2. The LIMUC 4-class MES result is compromised by checkpoint selection. Section 2.2 states they selected checkpoints by evaluating on 'an array of downstream tasks,' and Figure 1 plots LIMUC 4-class MES performance versus training step. Table 5 then reports on the 15% holdout from LIMUC. Unless selection was restricted to a validation split (the paper never says so), the reported AUROC of 0.942 is partly a product of test-set selection. Prior work did not get to peek at the test set for checkpoint selection. This undermines the paper's strongest headline.\n\n3. The generalization claim depends on the pre-training data being unrelated to the evaluation data. That is plausible but unverifiable: the videos are private and the paper gives no provenance or leakage analysis.\n\n4. No weights, code, or data are released. That limits practical value and makes independent verification impossible.\n\nBottom line: the scaling story holds up as a general trend, and the paper is worth reading for anyone building endoscopy AI. But the SOTA claims need qualification, and the MES result needs to be rerun with a validation-only selection protocol. A serious referee should engage with this; I would want to see the revision before trusting the numbers.","headline":"A useful large-scale pre-training study for GI endoscopy, but the SOTA claims are overstated and the LIMUC MES result is compromised by circular checkpoint selection.","tokens_in":8866,"tokens_out":3553,"would_cite":false,"duration_ms":30074,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EndoDINO shows that a self-supervised vision transformer trained on a large private corpus of endoscopy videos, then frozen, can beat task-specific models on anatomical landmark classification, polyp segmentation, and Mayo endoscopic…","keywords":["Artificial Intelligence","Endoscopy","Gastroenterology","Foundation Model","Precision Medicine","Self-Supervised Learning","DINOv2"],"falsifier":"One decisive check would be to embed the HyperKvasir and LIMUC test images with the same DINOv2 encoder used for curation and search for near-duplicates in the 112-million-frame pre-training index; if a nontrivial fraction of test images have near-duplicate matches, or if the pre-training videos can be traced to the same clinical sites or patients as the evaluation data, the paper's generalization claim would be undermined. Publishing the pre-training video identifiers or an overlap report would settle the question.","tokens_in":7861,"feed_emoji":"🩺","tokens_out":6910,"duration_ms":64477,"temperature":0.7,"pith_summary":"EndoDINO aims to show that a vision transformer pre-trained without labels on a very large, private collection of GI endoscopy videos becomes a general-purpose feature extractor for endoscopy AI. The authors freeze the encoder and attach simple classification or segmentation heads, and report state-of-the-art results on anatomical landmark classification, polyp segmentation, and 3- and 4-class Mayo endoscopic scoring for ulcerative colitis. The central claim is that this transfer works across evaluation datasets that were collected independently of the pre-training videos, so the features generalize rather than memorize. If the claim holds, developers could train accurate endoscopy tools with far less labeled data and compute by reusing one shared backbone.","feed_headline":"Self-trained endoscopy model beats fine-tuned baselines on three tasks","feed_subtitle":"Pre-trained on 3.5B endoscopy frames, its frozen features top landmark, polyp, and Mayo-scoring benchmarks.","key_machinery":"The engine of the paper is DINOv2-style self-supervised pre-training applied to endoscopy-specific data. DINOv2 trains a model to make features agree across differently cropped views of the same image without any labels; here it is applied to curated frames sampled at 5 frames per second from 130,037 videos, reduced by near-duplicate removal to an index of 112 million frames, and then balanced-sampled from a four-level hierarchical k-means clustering (5M, 250K, 25K, and 5K clusters) to build pre-training sets of 100K to 10M images. The resulting frozen ViT encoders (ViT-B/14, ViT-L/14, ViT-g/14) are evaluated by fitting only simple heads: linear probing, a boosted linear head that concatenates patch tokens from the last four layers, and the DPT dense-prediction head. A notable procedural choice is that the DINOv2 training loss did not track downstream performance, so the authors selected checkpoints by evaluating them directly on the Mayo scoring task.","core_discovery":"On the paper's own terms, the discovery is that scaling self-supervised DINOv2-style pre-training to an endoscopy-specific corpus of up to 10 million curated images yields frozen features that outperform prior supervised and self-supervised models on multiple standard GI endoscopy benchmarks. The best EndoDINO ViT-g/14 model reaches a macro F1 of 0.748 on 3-class HyperKvasir Mayo scoring and 0.715 with an AUROC of 0.942 on 4-class LIMUC Mayo scoring, surpasses the prior best polyp segmentation mIoU of 0.834 with an mIoU of 0.864 using a ViT-L/14 frozen backbone, and with only 1% of the landmark training data achieves macro F1 of 0.995–0.997. The authors emphasize that these gains come with simple decoder heads and no end-to-end fine-tuning, which they attribute to the size and diversity of the pre-training data and to automated curation via deduplication and hierarchical clustering.","pith_inferences":["Beyond the paper: the striking few-shot landmark numbers come from a 1% split that is more class-balanced than the full set, so part of the gain may reflect label balance rather than pure feature quality; the paper does not isolate these factors.","Beyond the paper: if the private pre-training videos and the public evaluation datasets truly share no sites, equipment, or patients, EndoDINO would be a strong existence proof for domain-specific self-supervised learning in endoscopy; releasing the pre-training index or a leakage analysis would make that check possible.","Beyond the paper: because the authors selected checkpoints by downstream performance rather than SSL loss, practitioners working in similar medical domains should probably also monitor target-task metrics and not assume SSL loss is a reliable guide.","Beyond the paper: the same frozen features could plausibly support tasks the paper did not test, such as depth estimation, quality metrics, and localization, since those also rely on dense visual structure; that is an extension, not something the paper demonstrates."],"forward_implications":["A single frozen EndoDINO pass can feed several downstream heads, so multiple real-time AI tasks such as polyp segmentation, classification, and severity scoring could run in parallel on one modest GPU during an endoscopy.","Because the 1% landmark-training experiment still reaches macro F1 above 0.99, few-shot development of new endoscopy tasks becomes plausible with only a handful of labeled examples per class.","Increasing curated pre-training data from 1M to 10M progressively improves the 4-class Mayo scoring task, suggesting that further scaling the video pool may keep helping.","The simple linear and boosted-linear heads match or beat end-to-end trained segmentation models, indicating that a large part of the task knowledge lives in the frozen features, not in the decoder.","Pre-training on a broad mix of equipment, indications, and upper/lower GI procedures, rather than on one disease's trial videos, appears to be what lets the model transfer to unrelated public datasets."],"supporting_citations":[{"why":"Supplies the DINOv2 pre-training recipe, the checkpoint-selection approach, and the boosted-linear head design used throughout.","marker":"[6]"},{"why":"Provides the HyperKvasir benchmark data for landmark classification, polyp segmentation, and 3-class Mayo scoring.","marker":"[12]"},{"why":"Supplies the 3-class Mayo scoring protocol and the prior Etrolizumab SSL results that EndoDINO is compared against.","marker":"[14]"},{"why":"Represents prior large-scale self-supervised endoscopy pretraining that the paper positions against in dataset scale and generalization.","marker":"[16]"},{"why":"Supplies the hierarchical k-means balanced-sampling curation method that the authors adapt to build EndoDINO's pre-training set.","marker":"[20]"},{"why":"Provides the LIMUC dataset and the 15% holdout / 10-fold cross-validation protocol for 4-class Mayo scoring.","marker":"[21]"},{"why":"Provides the DPT dense-prediction head used for polyp segmentation.","marker":"[22]"},{"why":"Supports the claim that the 130,037-video pool is the largest reported in the literature.","marker":"[23]"}],"fun_headline_variants":["EndoDINO: 10M-image pretraining tops 3 endoscopy tasks","Frozen EndoDINO features beat fine-tuned rivals on 3 benchmarks","Scaling self-supervised GI endoscopy to 10M images","Endoscopy foundation model: SOTA with frozen features","Multi-task win: EndoDINO's self-supervised features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim that EndoDINO generalizes rather than memorizes rests on the assertion that the public evaluation datasets are entirely unrelated to the private pre-training videos, and because the pre-training corpus is closed and no leakage or provenance analysis is given, that premise is not currently checkable.","fun_headline_variants_meta":{"raw":{"variants":["EndoDINO: 10M-image pretraining tops 3 endoscopy tasks","Frozen EndoDINO features beat fine-tuned rivals on 3 benchmarks","Scaling self-supervised GI endoscopy to 10M images","Endoscopy foundation model: SOTA with frozen features","Multi-task win: EndoDINO's self-supervised features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000993,"raw_usage":{"total_tokens":4163,"prompt_tokens":859,"completion_tokens":3304,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":3209}},"tokens_in":475,"tokens_out":3304,"duration_ms":24056,"temperature":1.0,"reasoning_tokens":3209,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:29:58.240332+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One decisive check would be to embed the HyperKvasir and LIMUC test images with the same DINOv2 encoder used for curation and search for near-duplicates in the 112-million-frame pre-training index; if a nontrivial fraction of test images have near-duplicate matches, or if the pre-training videos can be traced to the same clinical sites or patients as the evaluation data, the paper's generalization claim would be undermined. Publishing the pre-training video identifiers or an overlap report would settle the question.","supporting_citations":[{"cited_title":"HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy","cited_arxiv_id":null,"evidence_quote":"Provides the HyperKvasir benchmark data for landmark classification, polyp segmentation, and 3-class Mayo scoring."},{"cited_title":"Unsupervised segmentation of colonoscopy images, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the 3-class Mayo scoring protocol and the prior Etrolizumab SSL results that EndoDINO is compared against."},{"cited_title":"Foundation model for endoscopy video analysis via large-scale self-supervised pre-train, 2024","cited_arxiv_id":null,"evidence_quote":"Represents prior large-scale self-supervised endoscopy pretraining that the paper positions against in dataset scale and generalization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the hierarchical k-means balanced-sampling curation method that the authors adapt to build EndoDINO's pre-training set."},{"cited_title":"Improving the computer-aided estimation of ulcerative colitis severity according to mayo endoscopic score by using regression-based deep learning","cited_arxiv_id":null,"evidence_quote":"Provides the LIMUC dataset and the 15% holdout / 10-fold cross-validation protocol for 4-class Mayo scoring."},{"cited_title":"Public imaging datasets of gastrointestinal endoscopy for artificial intelligence: a review","cited_arxiv_id":null,"evidence_quote":"Supports the claim that the 130,037-video pool is the largest reported in the literature."}],"review_version":1}