{"id":"598ca694-a032-4b7b-bd3f-caf16a613deb","arxiv_id":"2506.19464","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Hard-label model stealing succeeds on medical image classifiers at a 5,000-query budget, and the proposed QueryWise method boosts clone accuracy for gallbladder cancer but not for COVID-19.","lead":"This paper tests whether medical imaging AI models sold as a black-box service can be cloned with just 5,000 queries, even when the attacker only sees the model's top label. It introduces QueryWise, which uses public unlabeled ultrasound images to improve the clone, and reports a cloned model that beats radiologists on gallbladder cancer classification.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GBC headline may be an artifact of proxy leakage: the 'public' proxy videos [6] come from the same group as the victim and its GBSU test set, so the 77% thief accuracy could reflect data overlap rather than a transferable hard-label attack.","rationale":"The reader's weakest assumption is that the victim's hard labels on out-of-distribution proxy data are informative enough, and that the GBC proxy is unusually close because it was created by the same research group. I agree with that general concern, but the sharper and more load-bearing version is potential data leakage: the public proxy videos may share patients, acquisitions, or even near-duplicate frames with the victim's GBSU test set. If so, the attack is not stealing a proprietary model from an external public dataset; it is using a dataset that overlaps the victim's own private distribution. This directly threatens the 'no access to victim's training data' condition in the threat model and the headline claim that a thief can outperform radiologists. The single-run 77% result without error bars amplifies the risk: a small overlap could easily explain the margin over the 70% radiologist baseline. The reader's conditional verdict already asks for the overlap to be addressed, so my concern does not change the verdict direction; it strengthens the condition. I credit the paper for a clear threat model, a low query budget relative to natural-image attacks, and for publicly releasing code; these are real positives. But the central empirical demonstration needs the overlap check before the risk claim can be accepted.","tokens_in":10109,"tokens_out":7157,"duration_ms":78411,"concrete_test":"Compute perceptual hashes (or feature-space nearest neighbors) for every proxy video frame and every GBSU test image, and identify near-duplicate pairs at a lenient threshold. Rerun the DeiT Random+QW attack (Table 1) with all near-duplicate frames removed from the proxy pool. If thief accuracy drops below the 70% radiologist baseline, the headline claim is attributable to proxy leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 (Model stealing setup) describes the GBC victim as RadFormer [5], trained on the GBSU static-image dataset [4], and the proxy as the GBC US videos dataset [6]. Both [4] and [6] have the same research group (Basu et al., with Arora as co-author), and the paper does not establish that the videos are from different patients or acquisitions than the GBSU images. If frames from [6] overlap with, or are near-duplicates of, the GBSU test images used to report Table 1, then the 'outperform radiologist' result (DeiT Random+QW, 77.05%) is not an instance of stealing a proprietary model from a public proxy; it is distillation on data that already belongs to the victim's distribution. This would violate the threat model's claim that the attacker lacks access to the victim's training data (Section 2). Because the headline result rests on this single configuration without error bars or an external-domain control, the central risk claim is not yet substantiated. The COVID-19 experiment, where the proxy comes from a different group (COVIDx-US), shows QueryWise underperforms the Random baseline (63.87 vs 65.97), which is consistent with the concern that the GBC success is driven by proxy closeness rather than by the proposed method.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates model stealing attacks on black-box medical image classifiers under a hard-label setting with a limited query budget of 5,000. The authors propose QueryWise, a two-stage method that first trains an anchor model on queried labels and then trains a final student model on both labeled and unlabeled proxy data, using pseudo-labels from the anchor and an EMA-updated teacher. They evaluate QueryWise on two medical tasks — Gallbladder Cancer (GBC) and COVID-19 classification — and report that a DeiT-based thief using QueryWise achieves 77.05% accuracy on GBC, surpassing an expert radiologist (70%). They also evaluate existing model stealing defenses and find them ineffective. Supplementary experiments cover natural image datasets.","tokens_in":10302,"tokens_out":7651,"duration_ms":74506,"significance":"If substantiated, the paper addresses an important and under-explored vulnerability in medical MLaaS: that proprietary diagnostic models can be stolen with only hard-label queries and a public proxy dataset, and that the cloned model can surpass human expert accuracy. The threat model is realistic (no access to training data, hard labels only, 5,000 queries), and the release of source code supports reproducibility. The defense evaluation is a useful contribution. However, the headline GBC result rests on a proxy dataset from the same research group as the victim model, and the proposed method's benefit is inconsistent across tasks — on COVID-19 (Table S1), QueryWise reduces total and class-wise accuracy relative to the Random baseline. These issues must be resolved before the central claims can be fully credited.","major_comments":[{"comment":"The GBC victim RadFormer [5] is trained on the GBSU static-image dataset [4], and the proxy is the GBC US videos dataset [6], which originates from the same research group (Basu et al., with Arora as co-author). The authors do not report whether the video frames in [6] come from the same patients or imaging sessions as the GBSU images, nor do they measure the distributional distance between the two datasets. If frames from [6] overlap with, or are near-duplicates of, the GBSU test images, the reported 77.05% DeiT Random+QW accuracy would reflect in-distribution distillation rather than OOD proxy stealing, which would violate the threat model described in Section 2. Please provide patient-level overlap analysis, acquisition-site information, and a control experiment using a proxy from a different institution or public source.","section":"§4, Table 1"},{"comment":"On the COVID-19 task, QueryWise (Random+QW) achieves lower total accuracy (63.87) and lower COVID-19 class accuracy (33.12) than the Random anchor (65.97 and 40.13, respectively). This directly weakens the paper's central claim that QueryWise 'effectively utilizes unlabeled data' to enhance thief performance under low query budgets. The authors should either explain these negative results (e.g., class imbalance, larger OOD gap) or revise the claim to state that the benefit is task-dependent. As presented, the effectiveness claim is only supported for GBC.","section":"§4, Table S1"},{"comment":"The headline comparison to the radiologist (77.05% vs 70.00%) is based on a single run without error bars, confidence intervals, or repeated seeds. The GBC test set size is not reported, and the difference between DeiT Random+QW (77.05%) and DeiT Random (71.31%) is just 5.74 points. Without statistical significance information, the claim that the thief 'outperforms Radiologists' accuracy' is not yet substantiated. Please report mean ± std over at least three runs and state the test-set size.","section":"Table 1, §4"}],"minor_comments":[{"comment":"Typo: 'siginificant' should be 'significant' in the paragraph following Table 1.","section":"§4"},{"comment":"Typo: 'the models's prediction' should be 'the model's prediction'.","section":"§2"},{"comment":"The phrase 'anadditionalteacher model' should be 'an additional teacher model'.","section":"§1"},{"comment":"The indicator function uses the anchor's maximum softmax probability (`σ(q_i^a) > ρ`) in both the teacher and anchor KD loss terms; the text should clarify whether the threshold is intentionally applied to the anchor's confidence only, and how this interacts with the teacher's confidence for samples near the threshold.","section":"§3.2, Eqs. (5)-(6)"},{"comment":"The data-free Dual Students baseline [7] is compared at a different query budget (500K vs 5K), so the comparison is not apples-to-apples; the authors should note this limitation in the caption or text.","section":"Supplementary Table S2"}],"recommendation":"major_revision","confidential_remarks":"The overlap between the GBC victim model, its training data (GBSU [4]), and the proxy videos ([6]) is a genuine risk to the paper's main claim. I recommend that the editor require a patient-level overlap and distributional-distance analysis before acceptance; if the authors cannot demonstrate non-overlap, the headline result should be removed or re-framed. The paper also overclaims in its conclusion relative to the mixed COVID-19 results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper gives the community something real: a hard-label, 5,000-query model-stealing evaluation on two medical imaging tasks, with code, and a method (QueryWise) that reuses unlabeled proxy data via anchor-plus-teacher pseudo-labeling. The COVID-19 and natural-image experiments are useful baselines. If the GBC result held up, the claim that a thief can exceed radiologist accuracy would be genuinely alarming for MLaaS in medicine.\n\nThat said, the central GBC result is softer than it looks. The victim (RadFormer), the test set (GBSU), and the proxy (GBC US videos) all trace to the same Basu–Arora group; the paper never shows the videos come from different patients or acquisitions. If frames from the videos overlap with GBSU test images, the \"77% vs 70%\" headline is not stealing a proprietary model from a public proxy—it is distillation on data already close to the victim's training distribution. The stress-test note is right to flag this, and the paper does not address it. This is the paper's load-bearing weakness, and it is not minor.\n\nThere are smaller problems too. QueryWise does not consistently beat Random: on COVID-19 it is worse (63.87 vs 65.97), and class-wise COVID sensitivity drops. The defense experiments omit QueryWise entirely, despite the paper claiming to evaluate the proposed attack. The headline number is a single run with no error bars or seeds. And the supplementary natural-image tables show no clear edge for QueryWise over k-Center on several datasets.\n\nWhat the paper does well is scope the threat model honestly: hard labels, small budget, proxy data from the same modality but not necessarily the same distribution. The authors admit hard labels on OOD data can be uninformative. They just don't carry that caution into the GBC interpretation.\n\nVerdict: worth a serious referee, but a responsible reviewer should ask for (1) proof that the GBC proxy videos are from distinct patients/acquisitions, or re-run with an external proxy; (2) error bars across seeds; (3) QueryWise in the defense table; and (4) an honest discussion of where the method fails. If the leakage concern is resolved, the paper is a solid empirical contribution. As-is, it overstates both the attack's effectiveness and QueryWise's generality.","headline":"Plausible hard-label model-stealing demo for medical imaging, but the headline GBC result is likely inflated by proxy data from the victim's own group, and QueryWise itself is inconsistent.","tokens_in":10948,"tokens_out":1004,"would_cite":false,"duration_ms":11793,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 5,000-query attack can functionally clone a black-box medical imaging model.","keywords":["model stealing","model extraction","medical imaging","black-box attack","hard-label query","knowledge distillation","pseudo-labeling","gallbladder cancer classification"],"falsifier":"Run QueryWise with a 5,000-query budget against a black-box medical classifier using a public proxy dataset from a different institution, scanner, or patient population than the victim's training data, and check whether thief accuracy still exceeds the radiologist baseline or even the random-query baseline; if accuracy collapses toward chance once the proxy distribution is truly external, the paper's central generalization claim fails.","tokens_in":1579,"feed_emoji":"🩺","tokens_out":1499,"duration_ms":94773,"temperature":0.7,"pith_summary":"The paper sets out to show that proprietary black-box medical imaging models can be functionally cloned by an adversary who has no access to the victim model's weights, training data, or soft labels, only its top-1 prediction. It proposes a two-step attack, QueryWise, that first trains an anchor model on 5,000 queried hard labels and then trains a student thief on both the labeled queries and the remaining unlabeled proxy images, using soft pseudo-labels from the anchor and an exponential-moving-average teacher. On a gallbladder-cancer ultrasound classification task the resulting thief reaches 77 percent accuracy, above the 70 percent of a published radiologist baseline, and it improves agreement with the victim across architectures and tasks. The paper takes this as evidence that the common belief in the safety of hidden-weight diagnostic services is unfounded, and that current perturbation-based defenses do not reliably stop the attack.","feed_headline":"5,000 queries clone a black-box medical AI model","feed_subtitle":"Public ultrasound images plus a two-step thief beat radiologist accuracy on gallbladder cancer detection.","key_machinery":"The load-bearing mechanism is QueryWise's two-step distillation loop. Step one trains an anchor model purely on the queried hard labels. Step two trains a student thief on a combined loss: on labeled images the student is supervised by the victim's hard labels plus the anchor's softened logits via knowledge distillation; on unlabeled images it is supervised by softened outputs of the anchor and of a teacher whose weights are an exponential moving average of the student's own weights, with only samples whose maximum softmax probability exceeds a threshold contributing. Logit adjustment counters the class imbalance induced by querying an out-of-distribution proxy set. The student becomes the final thief model.","core_discovery":"The paper claims that model stealing is a realistic threat for medical imaging classifiers under a strict threat model: hard-label-only access, a 5,000-query budget, and no access to the victim's training distribution. The central empirical discovery is that a two-stage training procedure, QueryWise, which reuses the unlabeled proxy data that ordinary stealing methods discard, produces thieves that outperform standard stealing baselines and, on the gallbladder-cancer task, surpass published radiologist accuracy (77.05 percent vs 70.00 percent). The authors also report that the attack transfers across convolutional and transformer architectures and to a COVID-19 lung-ultrasound classifier, and that three state-of-the-art perturbation-based defenses fail to reduce thief accuracy consistently without also damaging the victim. In the authors' reading, this invalidates the assumption that keeping model weights hidden is sufficient protection for monetized diagnostic models.","pith_inferences":["In the gallbladder-cancer experiment the proxy video frames were produced by the same research group that built the victim model, so the proxy-to-victim domain gap is probably smaller than in a real attack on an unrelated proprietary service; whether 5,000 queries still suffice across a genuinely large domain shift is untested by the paper.","On the COVID-19 task QueryWise raised agreement with the victim without raising accuracy, suggesting the method can copy decision boundaries rather than the true clinical label; a thief that mimics a biased victim would inherit that bias.","A sharper test of the attack would use a proxy dataset that provably shares no acquisition pipeline or patient population with the victim's training data, and would report radiologist-level labels on the same test set instead of published baseline numbers.","The confidence threshold on pseudo-labels is likely a key lever; varying it should reveal whether the gain comes from high-quality pseudo-labels or simply from more training data."],"forward_implications":["An adversary with 5,000 top-1 labels and a public same-modality image collection can reproduce enough of a proprietary medical classifier's behavior to surpass expert-level accuracy on at least one clinically meaningful task.","Three state-of-the-art perturbation-based defenses fail to reduce thief accuracy consistently without also reducing the victim's own accuracy, so hiding weights is not a reliable protection strategy.","The attack transfers across thief architectures and tasks, including transformer backbones and COVID-19 lung ultrasound, so the risk is not specific to one model family.","QueryWise's reliance on a proxy distribution means the practical danger is greatest where public unlabeled images of the same modality are easy to obtain.","For model owners, the observable query response surface itself leaks enough information to clone functionality, even when the model returns only its top prediction."],"supporting_citations":[{"why":"Supplies the transformer-based victim model for the main gallbladder-cancer experiment.","marker":"[5]"},{"why":"Supplies the labeled ultrasound dataset the victim was trained on and on which the thief is evaluated.","marker":"[4]"},{"why":"Supplies the public ultrasound video frames used as the proxy dataset for querying the victim.","marker":"[6]"},{"why":"Supplies the convolutional victim model for the COVID-19 classification experiment.","marker":"[8]"},{"why":"Supplies the public lung-ultrasound proxy dataset used in the COVID-19 experiment.","marker":"[10]"},{"why":"Supplies the random-selection baseline and one of the two anchor-model training procedures QueryWise builds on.","marker":"[21]"},{"why":"Supplies the k-center active-learning baseline and the other anchor-model training procedure.","marker":"[23]"},{"why":"Provides the semi-supervised consistency baseline against which QueryWise is compared.","marker":"[26]"},{"why":"Provides the knowledge-distillation loss that carries soft supervision from the anchor and teacher to the student.","marker":"[14]"},{"why":"Supplies the exponential-moving-average teacher update that stabilizes pseudo-label training.","marker":"[28]"}],"fun_headline_variants":["QueryWise: 5,000 queries clone medical AI","Stealing medical AI: 5k queries beat radiologists","Black-box medical AI cloned in 5,000 queries","Two-step attack clones medical imaging models"],"cache_read_input_tokens":12928,"weakest_assumption_plain":"The attack's success assumes the victim's top-1 labels on the attacker's out-of-distribution proxy images are informative enough to train a thief that generalizes to the victim's real test distribution; in the main experiment that proxy data comes from the same research group that built the victim, so the assumption is easier to satisfy than in a genuine external attack.","fun_headline_variants_meta":{"raw":{"variants":["QueryWise: 5,000 queries clone medical AI","Stealing medical AI: 5k queries beat radiologists","Black-box medical AI cloned in 5,000 queries","Two-step attack clones medical imaging models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2645,"prompt_tokens":928,"completion_tokens":1717,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":1652}},"tokens_in":544,"tokens_out":1717,"duration_ms":11164,"temperature":1.0,"reasoning_tokens":1652,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:07:10.578058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run QueryWise with a 5,000-query budget against a black-box medical classifier using a public proxy dataset from a different institution, scanner, or patient population than the victim's training data, and check whether thief accuracy still exceeds the radiologist baseline or even the random-query baseline; if accuracy collapses toward chance once the proxy distribution is truly external, the paper's central generalization claim fails.","supporting_citations":[{"cited_title":"Medical Image Analysis83, 102676 (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer-based victim model for the main gallbladder-cancer experiment."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the labeled ultrasound dataset the victim was trained on and on which the thief is evaluated."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"Supplies the public ultrasound video frames used as the proxy dataset for querying the victim."},{"cited_title":"Applied Sciences11(2), 672 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional victim model for the COVID-19 classification experiment."},{"cited_title":"COVIDx-US -- An open-access benchmark dataset of ultrasound imaging data for AI-driven COVID-19 analytics","cited_arxiv_id":"2103.10003","evidence_quote":"Supplies the public lung-ultrasound proxy dataset used in the COVID-19 experiment."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the random-selection baseline and one of the two anchor-model training procedures QueryWise builds on."},{"cited_title":"In: Proceed- ings of the AAAI Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"Supplies the k-center active-learning baseline and the other anchor-model training procedure."}],"review_version":1}