{"id":"3af971c8-e37f-4f79-a285-443e0188447e","arxiv_id":"2412.00702","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A workflow combining DINO self-supervised retraining and active domain adaptation with ten annotated samples improves AUPRC on most of ten skin lesion target domains.","lead":"This paper tests whether self-supervised pretraining on unlabeled skin images plus active domain adaptation with ten labeled examples improves skin lesion classification across different patient groups and imaging settings. It reports gains on most of ten target datasets, offering a low-annotation recipe for adapting models to new clinics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II contradicts the paper's central 9-of-10 superior claim: Dino retrained is worse than BSP on HLP and BA, not only BLH.","rationale":"The reader's weakest assumption focused on the comparability of numbers borrowed from [10]. That is a valid external-validity concern, but the more load-bearing issue is internal: the paper's own Table II does not support the 9-of-10 claim in Section V. On the reported numbers, the proposed 'Dino retrained' model outperforms the BSP UDA method on only 7 of 10 target domains, with clear exceptions on HLP and BA in addition to BLH. The sentence claiming a single exception (BLH) is factually inconsistent with the table, meaning the central quantitative result is misreported. This does not necessarily destroy the overall idea that SSL retraining can help, but it means the paper's headline evidence, as presented, is unreliable. The reader's rationale already noted a conflict for three datasets, so there is partial agreement; however, the reader's formal weakest_assumption did not elevate this internal contradiction to the primary concern. The condition of the conditional verdict was partly addressable (e.g., releasing code, re-running baselines), but a central claim that contradicts the paper's own results is a more serious correctness problem that should not be accepted until corrected and re-verified.","tokens_in":10768,"tokens_out":6240,"duration_ms":49639,"concrete_test":"Recompute from Table II, for each of the ten target domains, whether the Dino retrained AUPRC exceeds both the baseline and the BSP values cited from [10]. Count the exceptions. A correct count of 7 (HLP, BA, BLH) would confirm the 9-of-10 statement is erroneous; a count of 9 would indicate the intended reference row is not the Dino retrained row, and the paper should state exactly which row was used. Also verify whether any other row (e.g., Dino pre-trained or SL pre-trained) actually yields 9-of-10 against both baselines.","verdict_should_be":"REJECT","load_bearing_attack":"The central comparative claim in Section V ('our approach achieved superior performance on 9 of the 10 target domains, with slightly worse results observed only for BLH dataset') is not supported by the paper's own Table II. Treating the 'Dino retrained' row as 'our approach' and comparing with the baseline and BSP rows cited from [10], the Dino retrained AUPRC is lower than BSP on HLP (0.50 vs 0.65), BA (0.31 vs 0.34), and BLH (0.84 vs 0.86): three exceptions, not one. Against the no-adaptation baseline, Dino retrained is higher on all ten domains, so the '9 of 10' wording is not explained by a baseline-only comparison either. Since this 9-of-10 result is the primary quantitative support for the claim that SSL retraining is an effective UDA method, the stated conclusion is internally inconsistent with the evidence. This issue is independent of, and compounds, the separate concern that the [10] numbers were not re-run under the paper's own training protocol: even taking the external numbers at face value, the headline comparison does not hold as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a three-stage workflow for skin lesion classification under domain shift: continue DINO self-supervised training on unlabeled skin-lesion images from all available datasets, fine-tune a linear classifier on a labeled source domain (dataset H), and then apply active domain adaptation with ten annotated target samples. The authors evaluate AUPRC on ten target domains derived from HAM10000/BCN20000/MSK and compare DINO variants with baseline and BSP numbers from Chamarthi et al. [10]. They report that the in-domain retrained DINO model outperforms the earlier UDA baseline on 9 of 10 target domains, that SSL pretraining beats supervised pretraining, and that AADA-DANN is the best ADA method.","tokens_in":11025,"tokens_out":4953,"duration_ms":40796,"significance":"If the headline comparisons held, the workflow would be a useful addition: it combines a standard SSL retraining step with a small annotation budget, which is clinically plausible, and it evaluates across ten domain shifts rather than a single pair. The paper also has concrete strengths: it uses a common backbone (ResNet-50) to facilitate comparison, reports mean and standard deviation over five seeds in Table II, and its claim that in-domain SSL retraining improves over no retraining on 6 of 10 datasets is actually supported by the numbers. However, the central quantitative claims and the evidence for the ADA comparisons need correction and additional reporting before the conclusions can be accepted.","major_comments":[{"comment":"The statement that 'our approach achieved superior performance on 9 of the 10 target domains, with slightly worse results observed only for BLH' is contradicted by the table in which the Dino retrained row is compared with the BSP row: AUPRC is lower on HLP (0.50 vs 0.65), BA (0.31 vs 0.34), and BLH (0.84 vs 0.86). This is three exceptions, not one. The same paragraph's claim that the SL pre-trained model performed worse than the SSL pre-trained model on 9 of 10 datasets is also not supported: comparing the Dino pre-trained and SL pre-trained rows gives five datasets where Dino is higher, two ties (BLH, M), and three where it is lower (HA, HLP, MA). Both claims need to be corrected or the comparison must be redefined clearly.","section":"Section V, first paragraph; Table II"},{"comment":"The headline comparison to Chamarthi et al. [10] uses the baseline and BSP numbers from that paper without re-running either method under this paper's training protocol. Because the protocol differences (source dataset H, fixed linear classifier, five-seed averaging, DINO backbone, augmentation, training epochs) could change the reported numbers, the external numbers are not directly comparable as evidence for 'our approach outperforms.' Either re-run baseline and BSP under identical conditions, or present the comparison as cross-paper and explicitly discuss the protocol differences; as written, the main UDA-effectiveness claim rests on an unverified comparability assumption.","section":"Section V; Table II"},{"comment":"The ADA results, including the claim that AADA-DANN is the best method and that it is better than or similar to baseline on 9 of 10 datasets, are reported only in figures with no numeric AUPRC values, standard deviations, or significance tests. The reader cannot verify the ordering of the five ADA methods or the stochastic stability of the comparison with only ten annotated samples. Please provide a table with mean +/- std for each ADA method on each target domain and, ideally, a paired test or confidence interval.","section":"Section V, Figures 4-5"},{"comment":"The SSL retraining step is performed on all datasets listed in Table I, which includes the target domains that are later evaluated. Since the model sees unlabeled images from each target domain before evaluation, the reported gains of 'SSL retraining' relative to the baseline may partly reflect transductive exposure to the target distribution rather than the acquisition of domain-general features. This is disclosed and is standard in UDA, but the paper's claim that 'SSL is an effective UDA method' is ambiguous. Please add an ablation that retrains only on source-domain data (or on held-out domains) and/or explicitly reframes the claim as a transductive UDA result.","section":"Section III; Section V"}],"minor_comments":[{"comment":"There is a typo: 'handels' should be 'handles', and the first sentence of Section V says 'skine lesion' instead of 'skin lesion'.","section":"Section IV-B"},{"comment":"The DINO loss equation is incomplete as typeset: the cross-entropy between teacher and student distributions should specify the temperature softmax and the stop-gradient on the teacher; otherwise the equation is not reproducible.","section":"Section III"},{"comment":"The AADA sampling criterion is formatted ambiguously as a fraction with Gd(Gf(x)) in both numerator and denominator; please rewrite with explicit non-italic function names and clear parentheses.","section":"Section III"},{"comment":"Reference [9] contains a doubled comma: 'Jaworek-Korjakowska, , and'.","section":"Reference list"},{"comment":"The contribution list claims 'first attempt to combine SSL and DA for skin lesion classification', but the related work discusses SSL plus active learning for skin lesion segmentation in [79]; please scope the novelty claim to classification explicitly.","section":"Section I"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claims are internally inconsistent with its own Table II, and the ADA evidence is only visual. I do not see this as a reject because the workflow is simple, reproducible in principle, and some of the component claims (e.g., the 6-of-10 improvement from in-domain retraining) are supported. A corrected revision with re-run or clearly labeled cross-paper baselines, numeric ADA tables, and the transductive-ablation analysis would be within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is reasonable: take a DINO-pretrained ResNet50, retrain it with SSL on all available skin lesion data, fine-tune on a labeled source domain, then apply active domain adaptation with a tiny annotation budget. That is a practical workflow, and the evaluation span is real: ten target domains with five ADA methods, using AUPRC because the data are imbalanced. If the results held up, this would be a useful clinical-deployment baseline.\n\nThe problem is that the results do not hold up as written. The headline claim in Section V—superior on 9 of 10 target domains versus Chamarthi et al.'s baseline and BSP, only BLH worse—is contradicted by Table II. The Dino retrained row is worse than BSP on HLP (0.50 vs 0.65), BA (0.31 vs 0.34), and BLH (0.84 vs 0.86). Three exceptions, not one. I checked the stress-test note against the table and it is correct. The secondary claim that SL pretraining is worse than SSL pretraining on 9 of 10 datasets also does not match the table; it is closer to 5 or 6, with ties on BLH and M. So the paper's central quantitative support is internally inconsistent.\n\nBeyond that, the comparison against [10] is under-controlled. The baseline and BSP numbers are borrowed from another paper without rerunning those methods under this protocol. That would already be a weakness; combined with the table mismatch, the \"9 of 10\" claim is simply not trustworthy.\n\nThe ADA results are another soft spot. Figures 4 and 5 show deltas, but there are no numeric values, error bars, or significance tests. The claim that AADA-DANN is best rests on eyeballing plots. That is a fixable reporting issue, but it matters because the ADA component is half the paper's contribution.\n\nTo be fair, the SSL retraining step is disclosed as using unlabeled target-domain data before evaluation, so the \"SSL as UDA\" framing is partially transductive exposure rather than domain alignment. That is a common practice, but it should be framed as such. Also, the \"first\" claim is weakened by the cited ALFREDO work, which already does active learning plus domain adaptation for medical image classification.\n\nWho gets value from this? Researchers working on cross-domain medical image classification will find the multi-domain benchmark and the practical recipe useful, even if the numbers need re-derivation. But as submitted, the paper overclaims and under-reports. I would send it to peer review because the experimental scope and the question are worth referee time, but I would expect major revision: rerun the baselines, fix the text to match the tables, put the ADA results in numeric form, and soften the novelty claims.","headline":"A sensible SSL+ADA workflow for skin lesion classification, but the paper's central '9 of 10' claim collapses against its own Table II and the comparison is under-controlled.","tokens_in":11473,"tokens_out":4264,"would_cite":false,"duration_ms":36279,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Combining DINO self-supervised retraining with active domain adaptation improves skin lesion classification under domain shift, reporting AUPRC gains over reference baselines on nine of ten target domains.","keywords":["skin lesion classification","self-supervised learning","active domain adaptation","DINO","domain shift","transfer learning","medical imaging","AUPRC"],"falsifier":"Re-run the baseline and BSP methods under this paper's exact training protocol (same ImageNet-pretrained ResNet50, same DINO augmentations, same frozen-backbone linear fine-tuning, same five seeds, same train and validation splits) and check whether DINO retraining plus ADA still beats them on 9 of 10 target domains. If the gap disappears or shrinks substantially, the reported advantage is a protocol artifact rather than a property of the method.","tokens_in":10574,"feed_emoji":"🩺","tokens_out":6962,"duration_ms":60115,"temperature":0.7,"pith_summary":"The paper argues that skin lesion classifiers generalize across dataset domains when self-supervised pretraining and active domain adaptation are combined. Its workflow starts from a DINO model trained on ImageNet, continues DINO training on unlabeled skin lesion images, fine-tunes only a linear classifier on labeled source data, then adapts to each target domain using active learning with just ten expert-labeled target samples. On ten skin-lesion target domains defined by biological traits such as age and lesion location, the approach reports area under the precision-recall curve (AUPRC) gains over the compared baseline and BSP unsupervised adaptation on nine of ten domains. The paper also claims the adversarial active method AADA-DANN performs best among the five ADA methods tested. The clinical motivation is that this workflow fits into a feedback loop where clinicians annotate a few informative cases and the model improves iteratively.","feed_headline":"Two-stage domain adaptation beats baseline on 9 of 10 skin sets","feed_subtitle":"DINO retraining plus 10 expert-labeled samples lifts AUPRC across ten target domains.","key_machinery":"The load-bearing object is the DINO self-supervised teacher-student distillation objective, used first on ImageNet and then continued on all unlabeled skin lesion datasets. DINO minimizes the cross-entropy between teacher and student probability distributions over augmented views, which learns local-to-global correspondences without labels. On top of that, the paper applies active domain adaptation: an optional domain-adversarial loss (DANN-style) aligns source and target features, while an active-learning criterion selects target samples that are both uncertain and domain-discriminative for expert annotation. The paper freezes the ResNet50 backbone and trains only a linear classifier, isolating the contribution of feature quality.","core_discovery":"The central claim is that SSL retraining on unlabeled in-domain data is itself an effective unsupervised domain adaptation step for skin lesion classification, and that adding active domain adaptation with a small labeling budget gives further, mostly consistent gains. Concretely, the paper reports that its DINO-retrained ResNet50, fine-tuned on the source domain H, outperforms the baseline and BSP results reported in [10] on 9 of 10 target domains, with BLH the only exception; that the SL-pretrained model is worse than the SSL-pretrained model on 9 of 10 datasets; and that AADA-DANN outperforms or matches the no-ADA baseline on 9 of 10 target domains. The experiments use AUPRC as the metric because the datasets are imbalanced, with melanoma ratios as low as 4 percent.","pith_inferences":["Inference: because all ten target datasets are included in the unlabeled SSL retraining pool, part of DINO's advantage may come from broad exposure to the target domains; an ablation that SSL-retrains only on source-domain data would separate in-domain SSL from unseen-domain exposure.","Inference: the 9-of-10 superiority claim is a cross-paper comparison with [10], so a same-protocol rerun of baseline and BSP under this paper's augmentations, seeds, and training schedule would be the decisive test of that specific ranking.","Inference: the same SSL plus active domain adaptation recipe may transfer to other label-scarce medical imaging tasks, since DINO is domain-agnostic and ADA needs only a handful of expert annotations per target site.","Inference: testing with vision transformers and larger iterative labeling budgets would show whether the 10-sample regime is the sweet spot for ADA in this setting."],"forward_implications":["If the reported numbers hold, SSL retraining on unlabeled in-domain data alone can serve as an effective unsupervised domain adaptation step, matching or beating a dedicated unsupervised adaptation method (BSP) on most target domains.","Adding active domain adaptation with only 10 annotated samples from the target domain provides further improvement over the no-ADA baseline on most domains, so clinical annotation burden can stay very small.","The AADA-DANN variant is the best of the five tested ADA methods, suggesting that combining a diversity/uncertainty sampling criterion with adversarial domain alignment is the most useful configuration in this workflow.","The workflow is compatible with an iterative clinical loop: as experts annotate selected cases, the model can be updated and re-adapted.","Because the backbone is frozen during fine-tuning, the reported gains come from feature quality rather than a more powerful classifier, making the DINO SSL component the primary driver."],"supporting_citations":[{"why":"Supplies the baseline and BSP AUPRC numbers that the paper claims to beat on 9 of 10 target domains.","marker":"[10]"},{"why":"Defines DINO, the self-supervised method used for the initial pretrained model and for retraining on skin lesion data.","marker":"[8]"},{"why":"Shows that SSL training on in-domain medical data before fine-tuning improves out-of-domain generalization, the direct precedent for the paper's second workflow step.","marker":"[3]"},{"why":"Demonstrates that SSL retraining on skin lesion data improves a model pretrained on ImageNet, providing evidence for the SSL retraining step.","marker":"[30]"},{"why":"Provides the grouped HAM10000, BCN20000, and MSK skin lesion datasets and the domain-shift quantification used in the experiments.","marker":"[40]"},{"why":"Introduces AADA, the active adversarial domain adaptation method that the paper finds to perform best.","marker":"[65]"},{"why":"Introduces CLUE, one of the active-learning and domain adaptation methods compared in the experiments.","marker":"[56]"},{"why":"Introduces BADGE, another compared active-learning method.","marker":"[2]"},{"why":"Introduces DANN, the adversarial domain adaptation component used with AADA and as an optional UDA step.","marker":"[25]"}],"fun_headline_variants":["Skin lesion model improves on 9/10 domains via SSL retraining","Active DA beats baseline in 9 of 10 skin lesion targets","Self-supervised retraining improves skin lesion transfer","Combining SSL and active DA boosts skin lesion AUPRC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison against the baseline and BSP methods takes published AUPRC numbers from [10] as directly comparable, without re-running those methods under this paper's training protocol.","fun_headline_variants_meta":{"raw":{"variants":["Skin lesion model improves on 9/10 domains via SSL retraining","Active DA beats baseline in 9 of 10 skin lesion targets","Self-supervised retraining improves skin lesion transfer","Combining SSL and active DA boosts skin lesion AUPRC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000838,"raw_usage":{"total_tokens":3580,"prompt_tokens":799,"completion_tokens":2781,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":415,"completion_tokens_details":{"reasoning_tokens":2710}},"tokens_in":415,"tokens_out":2781,"duration_ms":18954,"temperature":1.0,"reasoning_tokens":2710,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:05:35.373199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the baseline and BSP methods under this paper's exact training protocol (same ImageNet-pretrained ResNet50, same DINO augmentations, same frozen-backbone linear fine-tuning, same five seeds, same train and validation splits) and check whether DINO retraining plus ADA still beats them on 9 of 10 target domains. If the gap disappears or shrinks substantially, the reported advantage is a protocol artifact rather than a property of the method.","supporting_citations":[{"cited_title":"Brinker, and Julia Niebling","cited_arxiv_id":null,"evidence_quote":"Supplies the baseline and BSP AUPRC numbers that the paper claims to beat on 9 of 10 target domains."},{"cited_title":"Caron et al","cited_arxiv_id":null,"evidence_quote":"Defines DINO, the self-supervised method used for the initial pretrained model and for retraining on skin lesion data."},{"cited_title":"Azizi et al","cited_arxiv_id":null,"evidence_quote":"Shows that SSL training on in-domain medical data before fine-tuning improves out-of-domain generalization, the direct precedent for the paper's second workflow step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates that SSL retraining on skin lesion data improves a model pretrained on ImageNet, providing evidence for the SSL retraining step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the grouped HAM10000, BCN20000, and MSK skin lesion datasets and the domain-shift quantification used in the experiments."},{"cited_title":"Active adversarial domain adaptation","cited_arxiv_id":null,"evidence_quote":"Introduces AADA, the active adversarial domain adaptation method that the paper finds to perform best."},{"cited_title":"Active domain adaptation via clustering uncertainty- weighted embeddings","cited_arxiv_id":null,"evidence_quote":"Introduces CLUE, one of the active-learning and domain adaptation methods compared in the experiments."},{"cited_title":"Deep batch active learning by diverse, uncertain gradient lower bounds","cited_arxiv_id":null,"evidence_quote":"Introduces BADGE, another compared active-learning method."}],"review_version":1}