{"id":"c3b02e5e-4b0b-4933-bd37-7322210546d6","arxiv_id":"2602.19857","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Contrastive pre-training followed by meta-domain guided tuning improves skin lesion classification accuracy on PAD-UFES-20 and DDI when transferring from HAM10000 dermoscopic training.","lead":"The paper proposes a two-stage training pipeline—contrastive pre-training plus target-guided meta-domain fine-tuning—to make skin-lesion classifiers transfer from dermoscopic images to noisier smartphone clinical images. On two clinical datasets it reports higher accuracy than fine-tuning baselines, but the evaluation lacks error bars, code, and a credible naive baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Label-space compatibility across datasets is never defined; without a shared class mapping, §4.2's Table 1 and Fig. 5 comparisons are invalid.","rationale":"The reader's weakest assumption already identifies the absence of explicit label remapping across HAM10000, PAD-UFES-20, and DDI as a core risk. I agree and single this out as the most load-bearing concern because it directly undermines the comparability of every number in Table 1 and the forgetting evaluation in Fig. 5. The paper does not provide any evidence that the label spaces are compatible, and the published taxonomies are not identical, so a non-trivial mapping is required but is never described. The DDI naive baseline collapse to 0.12 ACC / 0.05 F1 is consistent with a label-index mismatch, which would make all relative improvements spurious. This is an internal consistency issue, not a mere reproducibility gap: the loss in Eq. 4 assumes a shared output space. Since the concern is addressable by specifying and validating a label mapping and re-running the experiments, I keep the reader's CONDITIONAL verdict rather than moving to REJECT. No change to the verdict is needed, but the condition should be sharpened to require the label mapping and re-evaluation.","tokens_in":5396,"tokens_out":6726,"duration_ms":59908,"concrete_test":"Obtain the official label sets of HAM10000, PAD-UFES-20, and DDI, construct the exact class mapping implied by §3.3, and re-run the full pipeline (same backbone, splits, CT/GT configuration) under a single common taxonomy (e.g., binary malignant/benign or a curated subset of the seven HAM classes). Then recompute Table 1 and the forgetting curves in Fig. 5. If the relative gains of CT-pretrain+GT over Naive+FT shrink below the reported margins, or if the Naive DDI baseline ceases to be near-chance, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim: GT and CT+GT improve accuracy/F1 on clinical datasets while preserving prior dermoscopic knowledge. For this to hold, the same classifier must be trainable on HAM10000, fine-tunable on PAD-UFES-20 and DDI, and evaluable on all three. The paper never states the label mapping between HAM10000's seven classes and the taxonomies of PAD-UFES-20 and DDI. If the sets differ, L_o(S_dt; θ_t) in Eq. (4) is undefined unless a common label space is imposed, and the Fig. 5 'previously learned domain' curves are uninterpretable. The DDI Naive baselines (ACC 0.12, F1 0.05) are below chance for any usual number of classes and suggest a label-alignment bug, not a genuine lower bound. Since all Table 1 gains are relative to these baselines, the headline results could be artifacts of mismatched label indices. This is an internal inconsistency of the experimental design, not a matter of missing error bars.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a two-stage domain-adaptation pipeline for skin-lesion classification: a multi-transform contrastive pre-training step on dermoscopic images (HAM10000) followed by a 'guided-tuning' (GT) meta-domain adaptation stage that uses target-domain color and blur statistics to simulate K=2 meta-domains and fine-tune the model on clinical datasets (PAD-UFES-20 and DDI). The authors report that CT-pretrain + GT achieves the best accuracy and F1 scores on both clinical datasets, while reducing catastrophic forgetting of previously learned dermoscopic domains. The central claim is that the proposed guided-tuning strategy enables stable adaptation to clinical domains while preserving knowledge from dermoscopic training.","tokens_in":5721,"tokens_out":4662,"duration_ms":47031,"significance":"If substantiated, the proposed pipeline would be a practical contribution to clinical skin-lesion classification, where domain shift is a recognized barrier to deployment. The idea of using target-domain color and blur statistics to synthesize 'meta-domains' is interesting and potentially useful. However, the experimental validation as presented is not reliable: the paper never defines a shared label space across datasets, Table 1 reports no variance estimates or seeds, and the DDI Naive baseline (ACC 0.12, F1 0.05) is implausibly low, suggesting a label-alignment bug. Because all headline gains are relative to these baselines, the central claim is currently unsupported. The paper is also difficult to reproduce due to missing implementation details (optimizer, epochs, batch size, augmentation magnitudes). These issues are load-bearing rather than cosmetic.","major_comments":[{"comment":"The paper never defines the label mapping between HAM10000's seven classes and the taxonomies of PAD-UFES-20 and DDI. Eq. (4) uses the cross-entropy loss L_o on both source and target sets; this loss is undefined if the label spaces differ. If the output head is replaced or reinitialized during adaptation, the claim of 'preserving previously learned domains' (Fig. 5) is not meaningful without specifying how the source labels are mapped. The DDI Naive baseline (ACC 0.12, F1 0.05) is far below chance and far below the cited Daneshjou et al. baseline of 0.77, strongly suggesting a label-index mismatch. Since every improvement in Table 1 is measured against this Naive baseline, the central experimental claim is not currently supported.","section":"§3.3, Table 1, Eq. (4)"},{"comment":"No variance estimates, number of seeds, or confidence intervals are reported anywhere. All results are single runs. Differences such as GT vs. CT+GT on DDI (ACC 0.79 vs 0.79, F1 0.79 vs 0.81) are within plausible run-to-run noise. Without at least five seeds with mean±std and statistical testing, the reported improvements cannot be interpreted. This is especially important for a domain-adaptation method, where training is stochastic and the calibration subsets are randomly partitioned.","section":"Table 1, Figs. 3–5"},{"comment":"The calibration subset size is not specified, yet the method's domain simulation relies on estimating LAB color statistics and blur features from this subset. The manuscript does not characterize how sensitive the estimates are to subset size or composition. In addition, the hyperparameters K=2, β1=β2=0.5, and augmentation probability 0.5 are hand-set with no sensitivity analysis. The claim that GT enables 'stable adaptation with limited target-domain data' (Fig. 4) is based on a single curve without error bars, so it does not establish stability.","section":"§3.2.2, Eq. (4)"},{"comment":"The degraded test set used to evaluate robustness is not described reproducibly: the types and magnitudes of blur, sensor noise, illumination shifts, motion blur, and overexposure are not specified. Moreover, the number of augmented views N and the temperature τ in Eq. (2) are never reported. These details are essential for reproducing the contrastive pre-training experiments and for assessing the claim that CT-pretrain improves robustness to clinical artifacts.","section":"§4.1, Fig. 3, Eq. (2)"},{"comment":"The manuscript omits basic training details: the specific EfficientNet variant, image resolution, optimizer, learning rate schedule, batch size, number of epochs, and hardware. It also does not state whether the HAM10000-trained model is fine-tuned as a whole or only the head. These omissions make the experiments non-reproducible and prevent an independent check of the reported numbers.","section":"§3.3, experimental setup"}],"minor_comments":[{"comment":"Numerous typographical errors, including 'ADAPTA TION' and 'Clasifica tion' in the title, 'Adressing' in §3.2, 'Constrative' in §4.1, and 'CT-pretain' in Table 1. The manuscript should be proofread.","section":"Throughout"},{"comment":"The relation between the single-positive InfoNCE loss in Eq. (1) and the multi-positive loss in Eq. (2) is not clearly explained. The notation α=1/(N+1) is confusing because the sum runs over k=0..N. Please clarify the indexing and the exact composition of both losses.","section":"Eqs. (1)–(2)"},{"comment":"The set S_adapt = S_cal ∪ S_ds′ is not precisely defined. In particular, it is unclear whether S_ds′ is the entire transformed source set or a sampled mini-batch, and how the union is formed when the calibration subset and the source set have different sizes and label spaces.","section":"§3.2.2"},{"comment":"The x-axis is labeled 'Training set size' with values 0%–100%, but the text says 'PAD validation increasing training set size'. It would be clearer to state explicitly that the percentages refer to the fraction of the PAD-UFES-20 training set used and how the remaining fraction is treated.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising idea but the experimental evaluation is currently invalid as presented. The label-mapping issue is the most serious: without a stated common label space, the numbers in Table 1 cannot be trusted, and the DDI Naive baseline suggests a bug. I recommend asking the authors to (i) explicitly define/remap label taxonomies, (ii) rerun all experiments with that mapping and with multiple seeds, (iii) explain why Naive backprop on DDI collapses to 0.12 ACC, and (iv) provide full implementation details and code. If the DDI baseline cannot be fixed, the central claim loses support. The manuscript also needs a thorough proofread. Given the scope of the required changes, major revision is appropriate rather than outright rejection, because the methodological idea itself may still be viable after a corrected evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of arXiv:2602.19857. The paper proposes a two-stage recipe: multi-transform InfoNCE contrastive pre-training on HAM10000, then a “guided-tuning” meta-domain adaptation that simulates K=2 meta-domains by transferring LAB color and blur statistics from a small calibration subset of the target. That is a reasonable engineering idea, and the combination on HAM→PAD/DDI appears to be new. If the numbers hold, it is a useful recipe for deploying classifiers on smartphone dermatology images. The motivation is sound and the method description is mostly clear; credit where due for reusing Volpi-style meta-learning with a fixed loss objective and target-guided augmentations.\n\nThe problem is the empirical support. The central table has no seeds, no error bars, no confidence intervals. More seriously, the Naive backprop baseline on DDI collapses to ACC 0.12, F1 0.05, far below chance for any usual number of classes and far below the cited Daneshjou baseline of 0.77. That is not a plausible lower bound; it smells like a label-alignment bug between HAM and DDI classes. The paper never states the label mapping across HAM10000, PAD-UFES-20, and DDI. If the taxonomies differ, Eq. (4) and Table 1 are not well-defined, and the Fig. 5 “previously learned domain” curves don’t mean what the authors claim. Because every reported gain is relative to that collapsed baseline, the headline result could be an artifact. The stress-test note lands.\n\nThere are also smaller issues: hyperparameters (β1=β2=0.5, K=2, augmentation probability, temperature τ) are hand-set without sensitivity analysis, and no code or trained models are provided. These are fixable, but they matter for reproducibility.\n\nWho this is for: people working on medical domain adaptation and deployment of dermatology classifiers. The paper could be a solid engineering contribution after careful revision, but not as is. I would not desk-reject it: the method is coherent and worth a serious referee’s time, but only with a strict request to provide code, variance estimates, the exact label mapping, and an explanation—or fix—of the DDI Naive baseline. If those don’t hold, the paper should not be published.","headline":"Plausible domain-adaptation recipe undermined by an unexplained DDI baseline collapse and an unspecified label mapping across datasets; the evidence needs a re-run before the headline results are credible.","tokens_in":6199,"tokens_out":2353,"would_cite":false,"duration_ms":25637,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a two-stage training pipeline can adapt dermoscopy-trained skin-lesion classifiers to clinical smartphone photos while preserving prior-domain knowledge.","keywords":["skin lesion classification","domain adaptation","contrastive learning","continual learning","meta-domain","dermoscopy","clinical imaging","catastrophic forgetting"],"falsifier":"Take a clinical test set acquired at a site that contributed no images to the calibration subset and compare per-class accuracy of guided-tuning against naive training; if the accuracy reverts to naive levels, the calibration statistics do not capture the full domain shift and the central claim fails.","tokens_in":5298,"feed_emoji":"🩺","tokens_out":5944,"duration_ms":51772,"temperature":0.7,"pith_summary":"The paper is trying to establish that the poor performance of skin-lesion classifiers on clinical images is caused by acquisition artifacts and domain shift, and that both can be mitigated with a domain-aware training recipe. The recipe has two parts: a contrastive pre-training step that makes the model's representations invariant to image degradations and separates different lesions, and a guided-tuning step that adapts the model to the clinical domain using a small calibration subset while keeping what it learned on dermoscopy. On two clinical smartphone datasets, the combined pipeline raises accuracy/F1 from very low naive baselines (0.35/0.38 and 0.12/0.05) to 0.88/0.84 and 0.79/0.81, and it preserves performance on the previously learned dermoscopic domain. A sympathetic reader would care because this addresses the main barrier to deploying dermatology AI in real clinics: models trained on clean dermoscopic datasets fail when faced with variable clinical photos.","feed_headline":"Guided tuning adapts skin-lesion AI to clinical photos","feed_subtitle":"A contrastive pre-training step plus target-guided meta-domains lifts accuracy to 0.88 while preserving dermoscopic knowledge.","key_machinery":"The central object is the visual meta-domain: a synthetic variant of the source dataset whose appearance is transformed to match the target clinical domain using color statistics (LAB mean and standard deviation) and blur statistics (Laplacian variance and gradient measures) estimated from a small calibration subset of target images. K such meta-domains are generated stochastically and used in a guided-tuning loss that combines the target-domain objective with meta-domain losses, preventing catastrophic forgetting while adapting. The companion machinery is multi-transform contrastive pre-training, which uses a multi-positive InfoNCE loss on stochastically augmented views of dermoscopic image","core_discovery":"The central claim is that adaptation to clinical skin-lesion images can be made stable—accurate on the new domain and non-forgetting of the old one—by combining contrastive pre-training with guided tuning. Guided tuning works by extracting a small calibration subset from the target clinical data, estimating its global color statistics (LAB mean/std) and blur characteristics (Laplacian variance, gradient measures), and using those estimates to transform source dermoscopic images into K synthetic meta-domains. The model is then optimized with a loss that mixes the target-domain fine-tuning objective with evaluations on these meta-domains, which acts as a continual-learning regularizer. In the","pith_inferences":["If the mechanism is primarily matching low-level color and blur statistics, the same guided-tuning recipe may transfer to other medical imaging domains where acquisition shifts dominate over content differences, but that transfer is an extension the paper does not test.","The protocol's reliance on shared label spaces is untested; on a clinical dataset with a different lesion taxonomy, the adaptation objective would need explicit label mapping or an open-set formulation before those accuracy numbers are meaningful.","A direct robustness test would vary the size and acquisition-site composition of the calibration subset; stable performance under such variation would support the claim that the estimated statistics capture the domain shift rather than overfit the calibration images.","The multi-transform contrastive objective could be applied to unlabeled clinical images to reduce domain shift further, a semi-supervised extension implied by the method's design but not explored here."],"forward_implications":["Contrastive pre-training on dermoscopic images yields models that degrade less under blur, noise, illumination shifts, and overexposure than models trained with standard supervised backpropagation.","Guided tuning improves clinical dataset accuracy/F1 over naive training, standard fine-tuning, and random augmentation, and it does so even when the target-domain training set is small.","The guided-tuning loss preserves performance on previously learned domains, so a model can be continually adapted to new clinical sites without catastrophic forgetting.","Combining contrastive pre-training with guided tuning gives the best reported results on both clinical datasets evaluated, suggesting the two components are complementary."],"fun_headline_variants":["Meta-domains help skin-lesion AI adapt to clinical photos","Contrastive adaptation keeps skin-lesion AI accurate across domains","New method stabilizes skin-lesion AI on clinical images","Skin-lesion AI tuned with meta-domains resists clinical variability"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The pipeline assumes the dermoscopic source and clinical target share the same lesion classes, and that color and blur statistics measured on a small calibration subset of the target training set represent the entire clinical domain.","fun_headline_variants_meta":{"raw":{"variants":["Meta-domains help skin-lesion AI adapt to clinical photos","Contrastive adaptation keeps skin-lesion AI accurate across domains","New method stabilizes skin-lesion AI on clinical images","Skin-lesion AI tuned with meta-domains resists clinical variability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1274,"prompt_tokens":614,"completion_tokens":660,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":358,"completion_tokens_details":{"reasoning_tokens":589}},"tokens_in":358,"tokens_out":660,"duration_ms":6412,"temperature":1.0,"reasoning_tokens":589,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T21:28:58.470733+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a clinical test set acquired at a site that contributed no images to the calibration subset and compare per-class accuracy of guided-tuning against naive training; if the accuracy reverts to naive levels, the calibration statistics do not capture the full domain shift and the central claim fails.","supporting_citations":[],"review_version":1}