{"id":"61463a40-89ad-4aea-80ef-54767603c5a4","arxiv_id":"2507.00474","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"ADAptation selects breast ultrasound images for expert annotation by combining diffusion-based style transfer, hypersphere contrastive learning, and a dual uncertainty-representativeness score, and claims improved fine-tuning accuracy at low annotation budgets.","lead":"This paper proposes ADAptation, a pipeline that reconstructs breast ultrasound images from a target hospital in the style of a labeled source dataset, then scores each image for uncertainty and representativeness to decide which ones an expert should label first. The authors report accuracy gains over several active learning baselines at a 20 percent annotation budget, but their own table shows the method below random selection at 30 and 50 percent budgets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 contradicts the central claim: at 30% and 50% annotation budgets, Random sampling outperforms ADAptation, so the reported superiority is internally inconsistent.","rationale":"The reader's weakest_assumption focused on reconstruction fidelity, but the reader's rationale identified the same decisive problem I see: Table 2 itself contradicts the paper's headline claim. I agree with the reader's overall verdict (REJECT) and partially with the reasoning. My stress test narrows the load-bearing issue to the internal inconsistency in Table 2, because this is the most direct falsification available: the method's central empirical claim is undercut by its own reported numbers, independent of any assumptions about diffusion fidelity, contrastive learning, or scoring mechanisms. The reconstruction-quality concern (Table 4, PSNR 10.40-11.89, MS-SSIM 0.23-0.37) is real but secondary; it concerns whether the method's mechanism works as intended, whereas the table issue concerns whether the method works at all in the paper's own evaluation. The missing omega in Eq. (4) and the absence of the closest baseline ALFREDO further weaken the paper, but they do not need to be resolved to reject the current version. My concrete test is designed to settle the table discrepancy: if the numbers reproduce, the verdict is REJECT with high confidence; if they were a formatting error, the correct experimental comparison must still be supplied before the claims can be evaluated. I therefore recommend keeping the reader's REJECT verdict, though the confidence may rise if the table is confirmed as accurate.","tokens_in":9716,"tokens_out":2492,"duration_ms":35654,"concrete_test":"Run the 30% and 50% annotation-ratio experiments from the anonymized repository using the same data splits, seeds, and five downstream classifiers, and report per-target-domain and per-classifier accuracies for ADAptation and Random. If ADAptation reproduces Table 2's values (0.8213 at 30%, 0.8814 at 50%), the central claim is falsified. If the table rows were misaligned, provide corrected full tables including the missing baseline rows under the 80% heading, ideally with raw result logs.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim is empirical: ADAptation 'surpasses existing strong AL-based competitors.' But Table 2, the paper's own headline experiment, shows the reverse at two of four budgets. At 30% annotation, ADAptation averages 0.8213 versus Random's 0.8585; at 50%, ADAptation averages 0.8814 versus Random's 0.9232. These gaps (3.7 and 4.2 percentage points) are large and systematic across the five classifiers. The text claims 'average improvements of 3.95% and 2.87%' at 30% and 50% over second-best methods, but the table shows ADAptation ranked last at 30% and near-last at 50%. This is not a subtle statistical question; the numbers directly contradict the prose. Additionally, the 80% block contains only the ADAptation row, with no baselines shown, so the claim that ADAptation 'approaches the upper bound' at 80% cannot be checked. If the table is a typographical artifact, the experiment must be rerun and reported correctly; if not, the central claim is falsified by the paper's own data. Reconstruction fidelity is a secondary concern: even if the diffusion reconstructions preserved all diagnostic content, the selection method still fails to outperform random sampling at the budgets where it is tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ADAptation, an unsupervised active learning (AL) method for cross-domain breast ultrasound classification. It first fine-tunes a ControlNet-guided diffusion model on the source domain, reconstructs target images in source style, learns hypersphere-constrained contrastive embeddings of target images and their reconstructions, and then ranks unlabeled target samples by a dual uncertainty-representativeness score. The selected samples are used to fine-tune five classifiers, evaluated on three target datasets at 20%, 30%, 50%, and 80% annotation budgets. The paper reports the best performance at 20% (0.8081 average accuracy) and claims superiority over AL baselines across all budgets.","tokens_in":10031,"tokens_out":5962,"duration_ms":62830,"significance":"Adapting breast ultrasound classifiers to new scanners or hospitals with minimal labeling is clinically relevant, and the combination of diffusion-based style transfer with AL is an interesting idea. The evaluation is broad: three public and one internal dataset, five classifiers, and four annotation budgets, with code released. However, the central claim of superiority is not supported by the paper's own Table 2 at 30% and 50% budgets, and the 80% comparison lacks baseline rows. The reconstruction fidelity results in Table 4 raise additional doubts about whether the representativeness score is measuring clinical content. The method also leaves the dual-score weight in Eq. (4) unspecified, and the cluster-number selection appears to use the test sets. These issues are load-bearing, so the contribution is not established as presented.","major_comments":[{"comment":"At 30% annotation, ADAptation averages 0.8213, while Random reaches 0.8585, Max-Entropy 0.8689, and LfOSA 0.8744; at 50%, ADAptation averages 0.8814, while Random reaches 0.9232, VAAL 0.9090, and LfOSA 0.9188. The text in Section 3 claims 'average improvements of 3.95% and 2.87%' at these budgets, which directly contradicts the table. Since the abstract and conclusion claim that ADAptation 'surpasses existing strong AL-based competitors,' this internal contradiction undermines the central claim. Please verify the numbers, rerun the comparison if necessary, and reconcile the text with the table.","section":"Table 2"},{"comment":"The dual-scoring equation is underspecified. The weight omega appears with no value, range, or selection procedure, although the dual-score balance is a key contribution. In addition, the uncertainty term is written as argmin over p,q of |theta_p - theta_q|, i.e., the minimum pairwise angular difference, whereas the text describes the absolute difference between the smallest and largest angular differences (max - min). These are different quantities, and the equation should match the described algorithm.","section":"Eq. (4)"},{"comment":"The 80% annotation block reports only the ADAptation row and no baseline or random-sampling rows. Consequently, the claim that ADAptation 'approaches the upper bound (0.9435)' and the discussion contrasting ADAptation with other methods at high budgets cannot be checked. Please include all baseline rows at every budget.","section":"Table 2, 80% block"},{"comment":"The reconstruction stage, which supplies the representativeness prior, has low fidelity on target domains: PSNR 10.40 to 11.89, MS-SSIM 0.23 to 0.37, and RMSE increases of 14.80 to 24.38 relative to source. If reconstructions distort lesion-relevant content, the SphericalDist term in Eq. (4) may measure reconstruction error rather than source-domain divergence. The assertion that reconstructions preserve 'critical medical characteristics' should be validated with diagnostic or classification-level metrics, not only pixel-level metrics.","section":"Table 4 and Section 2.1"},{"comment":"The ablation table is evaluated 'test on the target domain sets,' and the cluster number (4) that gives the best average accuracy on those test sets is then used in the main comparison in Table 2 on the same test sets. This introduces selection bias into the headline numbers; hyperparameters should be chosen on a validation split, or the test-set reuse should be justified and its effect quantified.","section":"Table 3"}],"minor_comments":[{"comment":"The text introduces teacher and student networks as \\hat{f} and g', but Eq. (3) uses f and g without the hat or prime; also, the projection is described as a '255-dimensional hypersphere' while the embedding is z in R^256. Please align the notation.","section":"Section 2.2"},{"comment":"'KNN clustering with k centroids' is not standard terminology; if the method uses k-means-like centroids, please state this explicitly.","section":"Section 2.3"},{"comment":"The claim of statistical significance (p < 0.01) is made only for the 20% budget, without stating the test used, the number of runs, or how multiple comparisons were handled.","section":"Section 3"},{"comment":"The 'Amount' column appears to contain two numbers per row (e.g., 210 and 437) without column labels, and the 'Domain Bias' values after homogenization are not explained in the caption.","section":"Table 1"},{"comment":"The sampling procedure for reconstruction is not defined (e.g., DDPM or DDIM steps, guidance scale), which prevents reproducibility of the reconstruction stage.","section":"Eq. (1)"},{"comment":"There are several presentation issues, including 'Domian' in Table 4 and 'we proposed ADAptation' in the introduction, which should be corrected.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The central empirical claim is contradicted by the manuscript's own headline table at two of four annotation budgets. Unless the experiments are rerun and the table and text are reconciled, the paper cannot support its main contribution. This is beyond a presentational fix, so I recommend rejection rather than major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version. The assembly is genuinely new: source-guided diffusion reconstruction with ControlNet+BiomedCLIP, hypersphere-constrained contrastive teacher-student learning, and a dual uncertainty-representativeness ranking. That combination, applied to single-iteration multi-classifier selection for breast ultrasound, I haven't seen before. The paper also does a broad evaluation across four datasets and five classifiers, and ships code (anonymized). Credit where due: the method section is mostly clear, and the idea of using reconstruction distance as a representativeness proxy is interesting even if the fidelity numbers are low.\n\nBut the central empirical claim doesn't survive contact with the paper's own Table 2. At 30% annotation, ADAptation averages 0.8213, while Random gets 0.8585; at 50%, 0.8814 vs 0.9232. The prose claims 3.95% and 2.87% improvements over second-best at those budgets, which directly contradicts the table. Five classifiers show the same pattern, so this is not noise. The 80% block shows only the ADAptation row, so the 'approaches the upper bound' claim is unverifiable. If the table is a typographical artifact, the experiment has to be rerun and reported honestly; if it isn't, the method fails against a trivial baseline at two of four budgets. Either way, the paper cannot be trusted as-is.\n\nOther soft spots are minor in comparison: Eq. (4) omits the omega weight it references, and the text's description of the uncertainty term (smallest vs largest angular difference) doesn't match the argmin in the equation. The cluster number k=4 was chosen via ablation on the target test sets, so the headline result is a selected-hyperparameter estimate. And ALFREDO, the closest published baseline, is absent from the comparison even though it is cited.\n\nMy take: this is a potentially useful framework that is now unsupported by its own reporting. I'd want the authors to correct the table, rerun the experiments, add ALFREDO, and report the selection bias of the ablation before spending referee time. For now, reject. If a revised version with internally consistent numbers appears, it deserves another look.","headline":"A genuinely novel assembly of diffusion reconstruction, hypersphere contrastive learning, and dual-scoring for active domain adaptation, but the paper's own Table 2 contradicts its central claim at 30% and 50% annotation budgets.","tokens_in":10565,"tokens_out":3316,"would_cite":false,"duration_ms":36705,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ADAptation selects breast ultrasound samples for annotation after reconstructing them in source style, reaching 0.8081 average accuracy at a 20% annotation budget across five classifiers.","keywords":["active learning","domain adaptation","diffusion model","contrastive learning","breast ultrasound","medical image classification","sample selection"],"falsifier":"A direct test would be to check, for each target image, whether the lesion is preserved in its reconstruction (for example by comparing lesion masks or edge maps before and after reconstruction) and then see whether the samples ADAptation ranks as most representative are the ones with preserved lesions. If images whose reconstructions erase the lesion are selected as representative, or if filtering out low-fidelity reconstructions (for instance PSNR below 10 or multi-scale structural similarity below 0.2) improves downstream accuracy, then the reconstruction prior is carrying the selection signal in a different way than claimed.","tokens_in":1800,"feed_emoji":"🩺","tokens_out":3746,"duration_ms":124308,"temperature":0.7,"pith_summary":"This paper is trying to establish that active learning can be made to work across hospital-to-hospital shifts in breast ultrasound images if the domain gap is removed before sample selection. It proposes ADAptation, which translates unlabeled target images into source-domain style with a diffusion model, learns compact hypersphere-constrained embeddings of originals and reconstructions, and ranks images by a combined uncertainty-representativeness score. On four breast ultrasound datasets and five classifiers, the authors report that selecting the top 20% of samples this way reaches an average accuracy of 0.8081, beating six active-learning baselines by a margin they report as significant ($p < 0.01$). If correct, the framework would let a clinical site annotate only a small, well-chosen fraction of new-scanner data and then fine-tune several diagnostic models at once.","feed_headline":"Diffusion-guided picks beat active-learning baselines at 20% labels","feed_subtitle":"Reconstructing target ultrasound scans in source style lets one labeling round fine-tune five classifiers across four datasets.","key_machinery":"The machinery is a three-stage pipeline. Stage I fine-tunes a diffusion model on the labeled source dataset, conditioned on a text prompt and on each image's canny edge map, so the model learns the source appearance while keeping the target's edge structure. Stage II feeds each unlabeled target image through the frozen diffusion model to produce a source-style reconstruction. Stage III uses a teacher-student contrastive network with a convolutional backbone to project each original image and its reconstruction onto the unit sphere in $\\mathbb{R}^{256}$ via $L^2$ normalization, and pulls each pair together with the angular contrastive loss $L = \\frac{1}{N}\\sum_i \\left(m \\cdot \\arccos\\left(\\hat{f}(x_i^u) \\cdot g'(x_i^r)\\right)\\right)^2$ with scaling factor $m=4$. The selection score is $I_i = \\min_{p,q} |\\theta_p - \\theta_q| + \\omega \\cdot \\mathrm{SphericalDist}(x_i^u, x_i^r)$, where the first term is KNN-based uncertainty (small when the sample sits between cluster centroids) and the second is representativeness (the spherical distance from an image to its reconstruction); the samples with the smallest scores are sent for annotation.","core_discovery":"On the paper's own terms, the central discovery is that reconstruction is a usable prior for unsupervised active learning under domain shift. Instead of decorating features or relying on a downstream model's confidence, ADAptation asks a source-trained diffusion model to redraw each target image in source style, then aligns original and reconstruction in a normalized hyperspherical embedding space. Informativeness is a single score made of two terms: an uncertainty term that measures how ambiguous a sample is relative to the cluster centroids, and a representativeness term that measures how far the sample has moved from its reconstruction. The headline quantitative claim is that this procedure, applied once at a 20% annotation budget, yields an average accuracy of 0.8081 across five classifiers on target breast ultrasound test sets, improving on the second-best active-learning method by 4.83% as the paper reports it, and it remains the most accurate method at 30% and 50% budgets.","pith_inferences":["Editorial inference: The paper's own reconstruction metrics show low pixel-level fidelity on target domains, so the mechanism that helps selection is likely structural rather than photometric; a targeted ablation varying reconstruction fidelity while holding the selection rule fixed would make this explicit.","Editorial inference: Because the representativeness term relies only on the spherical distance between an image and its source-style reconstruction, the method may transfer to other ultrasound tasks or even other imaging modalities where edge structure carries diagnostic information, though the paper does not test this.","Editorial inference: The uncertainty score is computed in embedding space rather than from any classifier's confidence, which may explain why one selection set works across five architectures; this property could also make the framework useful for model-agnostic data curation beyond the four datasets studied."],"forward_implications":["A single annotation round selected by ADAptation improves all five tested classifiers over the six compared active-learning baselines at a 20% budget, suggesting the selection transfers across model architectures.","Because selection happens once and is not tied to a particular downstream model, the framework matches the clinical workflow of rapidly updating several diagnostic models with new data.","At an 80% annotation ratio ADAptation reaches an average accuracy of 0.9351, approaching the full-data upper bound of 0.9435, while several baselines perform worse than random sampling at that budget.","The ablation results indicate that contrastive learning, the hypersphere constraint, and the reconstruction prior each contribute to the gain, and that four cluster centroids give the best uncertainty estimate."],"supporting_citations":[{"why":"Supplies the labeled source dataset used to fine-tune the diffusion model and pre-train the feature extractor.","marker":"[1]"},{"why":"One of the target breast ultrasound datasets whose unlabeled pool is sampled for annotation.","marker":"[4]"},{"why":"Another target dataset (UDIAT) used to evaluate cross-domain sample selection.","marker":"[26]"},{"why":"Provides the denoising diffusion generation process that the source-guided reconstruction is built on.","marker":"[8]"},{"why":"Supplies the teacher-student self-supervised learning objective that the angular contrastive loss and hypersphere constraint adapt.","marker":"[5]"},{"why":"Provides the conditional control mechanism (canny-edge conditioning) used to regenerate target images in source style.","marker":"[27]"},{"why":"Provides the biomedical text encoder that anchors the reconstruction prompt to medical semantics.","marker":"[28]"},{"why":"A representative-sampling active-learning baseline that ADAptation must beat at every annotation ratio.","marker":"[22]"},{"why":"The strongest prior active-learning baseline at the 20% budget, used as the main comparison for the claimed improvement.","marker":"[21]"}],"fun_headline_variants":["Reconstruction-based active learning beats baselines on breast ultrasound","Diffusion-guided sample selection improves domain adaptation in ultrasound","Reconstruct then pick: active learning for breast ultrasound under shift","Style-transfer sampling outranks active learning across five classifiers","Adaptation via reconstruction: efficient active learning for ultrasound AI"],"cache_read_input_tokens":12672,"weakest_assumption_plain":"The method's load-bearing premise is that source-guided diffusion reconstruction keeps the diagnostically relevant content of each target ultrasound image intact, so that the contrastive alignment and the reconstruction-distance representativeness score are built on clinically meaningful structure rather than on style-transfer artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Reconstruction-based active learning beats baselines on breast ultrasound","Diffusion-guided sample selection improves domain adaptation in ultrasound","Reconstruct then pick: active learning for breast ultrasound under shift","Style-transfer sampling outranks active learning across five classifiers","Adaptation via reconstruction: efficient active learning for ultrasound AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000248,"raw_usage":{"total_tokens":1562,"prompt_tokens":974,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":507}},"tokens_in":590,"tokens_out":588,"duration_ms":6916,"temperature":1.0,"reasoning_tokens":507,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:14:54.229301+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to check, for each target image, whether the lesion is preserved in its reconstruction (for example by comparing lesion masks or edge maps before and after reconstruction) and then see whether the samples ADAptation ranks as most representative are the ones with preserved lesions. If images whose reconstructions erase the lesion are selected as representative, or if filtering out low-fidelity reconstructions (for instance PSNR below 10 or multi-scale structural similarity below 0.2) improves downstream accuracy, then the reconstruction prior is carrying the selection signal in a different way than claimed.","supporting_citations":[{"cited_title":"Medical Physics51(4), 3110–3123 (2024)","cited_arxiv_id":null,"evidence_quote":"One of the target breast ultrasound datasets whose unlabeled pool is sampled for annotation."},{"cited_title":"IEEE journal of biomedical and health informatics22(4), 1218– 1226 (2017)","cited_arxiv_id":null,"evidence_quote":"Another target dataset (UDIAT) used to evaluate cross-domain sample selection."},{"cited_title":"Advances in neural information processing systems33, 6840–6851 (2020) 10 Y","cited_arxiv_id":null,"evidence_quote":"Provides the denoising diffusion generation process that the source-guided reconstruction is built on."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"The strongest prior active-learning baseline at the 20% budget, used as the main comparison for the claimed improvement."}],"review_version":1}