{"id":"34788f88-be75-4f89-9f9b-16b1b6554313","arxiv_id":"2604.23875","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"Standard noise-robust methods do not guarantee low clinical risk from false negatives in noisy medical labels, but combining them with cost-sensitive optimization reduces risk without harming utility.","lead":"The paper tests several noise-robust training methods on medical image datasets with simulated label errors and finds that these methods still produce high clinical risk from missed diagnoses. Adding explicit cost-sensitive training that penalizes false negatives more heavily lowers that risk while keeping overall performance similar.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"Simulated symmetric label noise on binarized small datasets may not match real clinical error patterns or asymmetric consequences.","rationale":"The reader’s weakest assumption is precisely the load-bearing point. Because the full text is now accessible yet the core issue is ecological validity of the simulation (not an internal contradiction or missing proof), the appropriate adjustment is CONDITIONAL rather than UNVERDICTED or REJECT. No other technical flaw in the abstract-level argument rises to the same level.","tokens_in":1731,"tokens_out":386,"duration_ms":15377,"concrete_test":"Re-run the full pipeline on a real multi-rater medical dataset (e.g., a dermatology or pathology cohort with recorded inter-observer labels); derive the noise transition matrix directly from the observed disagreements rather than uniform rates, keep the same Global Risk cost weights, and compare risk reduction of the cost-sensitive variants versus baselines. If the risk gap shrinks below statistical significance or reverses, the headline claim does not generalize.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—that SOTA noise-robust methods (Co-teaching, DivideMix, UNICON, GMM) fail to ensure clinical safety while cost-sensitive integration reduces Global Risk—rests on the experimental setup. Noise is added at fixed 20%/40% rates to binarized DermaMNIST/PathMNIST; Global Risk explicitly weights FN higher than FP. Real medical label noise is typically asymmetric (e.g., higher FN rates on ambiguous borderline cases due to inter-observer variability) and correlated with image features, not uniform random flips. If the chosen noise model and cost matrix do not reproduce the actual error distribution or patient-outcome costs, the observed risk reduction and the “robustness does not guarantee safety” conclusion do not transfer. The abstract provides no evidence that the simulation was calibrated against real annotation studies.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that state-of-the-art noise-robust methods (Co-teaching, DivideMix, UNICON, GMM filtering) do not guarantee clinical safety under label noise in medical image classification, as their accuracy robustness fails to control a cost-sensitive Global Risk metric that penalizes false negatives more heavily; it further claims that integrating cost-sensitive optimization into these methods significantly reduces Global Risk while preserving model utility, based on systematic experiments with symmetric noise injection on binarized DermaMNIST and PathMNIST at 0/20/40% noise rates.","tokens_in":1917,"tokens_out":642,"duration_ms":22604,"significance":"If the results hold, the work highlights a gap between accuracy-based robustness and clinical safety in noisy-label medical imaging, providing evidence that cost-sensitive integration can mitigate asymmetric error costs without sacrificing utility; this could influence evaluation practices in safety-critical ML applications.","major_comments":[{"comment":"The central claim that 'robustness does not guarantee clinical safety' rests on symmetric uniform label flips at fixed rates; this noise model does not capture real clinical annotation errors, which are typically asymmetric (higher FN on ambiguous cases), feature-correlated, and arising from inter-observer variability, as noted in the abstract's own motivation.","section":"§4 (Experimental Setup)"},{"comment":"Binarization of DermaMNIST (originally 7 classes) and PathMNIST (originally 9 classes) into binary tasks is used without justification or ablation; this alters the clinical diagnostic scenario and may not preserve the asymmetric consequences the Global Risk metric is intended to model.","section":"§4.1 (Datasets)"},{"comment":"The Global Risk formulation includes a free parameter (false_negative_cost_weight) whose value is not subjected to sensitivity analysis; different choices could change which methods appear safer and whether the reported risk reductions remain significant.","section":"§3.2 (Risk Metric)"},{"comment":"No details are provided on how cost-sensitive optimization is integrated into each baseline (e.g., loss modifications in DivideMix or UNICON), making it impossible to determine whether the observed Global Risk improvements are due to the proposed integration or to other implementation choices.","section":"§3.3 (Proposed Integration)"}],"minor_comments":[{"comment":"Abstract contains typo 'mantaining' (should be 'maintaining') and inconsistent method naming ('Coteaching' vs. standard 'Co-teaching').","section":"Abstract"},{"comment":"The manuscript should report the number of random seeds, exact train/val/test splits, and any statistical significance tests for the risk comparisons to support the 'systematic' evaluation claim.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The experimental design relies entirely on synthetic symmetric noise; the paper would be stronger with at least one real-world noisy medical label dataset for validation, which may affect fit for a clinical-applications-focused venue."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments, which help clarify the scope and presentation of our work. We respond to each major comment below and indicate planned revisions to address valid concerns while preserving the core contributions.","responses":[{"response":"We acknowledge that symmetric uniform noise is a controlled simplification and does not fully replicate the asymmetric, feature-dependent, and inter-observer nature of real clinical label errors. This model was selected, following common practice in the noise-robust learning literature, to isolate the impact of noise rate on the gap between accuracy robustness and clinical risk. The observed failure of standard methods to control Global Risk even under this setting still supports the central claim. In revision we will add an expanded limitations paragraph discussing the noise model and outlining future directions for more realistic clinical noise simulation.","revision_made":"partial","referee_comment":"[§4 (Experimental Setup)] The central claim that 'robustness does not guarantee clinical safety' rests on symmetric uniform label flips at fixed rates; this noise model does not capture real clinical annotation errors, which are typically asymmetric (higher FN on ambiguous cases), feature-correlated, and arising from inter-observer variability, as noted in the abstract's own motivation."},{"response":"Binarization creates well-defined positive/negative classes for which the asymmetric false-negative costs in the Global Risk metric are clinically interpretable. We will insert explicit justification in Section 4.1 explaining this alignment with binary diagnostic tasks and, space permitting, include a short ablation on alternative class groupings to demonstrate robustness of the reported trends.","revision_made":"yes","referee_comment":"[§4.1 (Datasets)] Binarization of DermaMNIST (originally 7 classes) and PathMNIST (originally 9 classes) into binary tasks is used without justification or ablation; this alters the clinical diagnostic scenario and may not preserve the asymmetric consequences the Global Risk metric is intended to model."},{"response":"The default weight of 5 was chosen to reflect typical clinical priorities, yet we agree that sensitivity analysis strengthens the claims. We will add experiments in the revision varying the weight over {2, 5, 10} and show that the risk-reduction benefits of cost-sensitive integration remain consistent across this range.","revision_made":"yes","referee_comment":"[§3.2 (Risk Metric)] The Global Risk formulation includes a free parameter (false_negative_cost_weight) whose value is not subjected to sensitivity analysis; different choices could change which methods appear safer and whether the reported risk reductions remain significant."},{"response":"Section 3.3 describes the high-level replacement of the standard loss with a cost-sensitive weighted loss. To improve clarity and reproducibility we will append concrete implementation details together with pseudocode showing the exact loss modifications for Co-teaching, DivideMix, UNICON, and the GMM filter.","revision_made":"yes","referee_comment":"[§3.3 (Proposed Integration)] No details are provided on how cost-sensitive optimization is integrated into each baseline (e.g., loss modifications in DivideMix or UNICON), making it impossible to determine whether the observed Global Risk improvements are due to the proposed integration or to other implementation choices."}],"tokens_in":1488,"tokens_out":696,"duration_ms":32925,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core observation is that methods like Co-teaching, DivideMix, UNICON, and GMM filtering hold up under label noise when measured by balanced accuracy, yet they still produce too many missed diagnoses when false negatives carry higher clinical cost. Adding cost-sensitive optimization during training cuts that Global Risk without much loss in utility. This is the useful empirical point the work delivers on DermaMNIST and PathMNIST at 20% and 40% noise levels.","headline":"The paper shows that standard noise-robust methods leave high false-negative risk on noisy medical images while a cost-sensitive addition lowers it, but the symmetric simulated noise on binarized small datasets weakens how much the result generalizes.","tokens_in":2375,"tokens_out":182,"would_cite":false,"duration_ms":20248,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Noise-robust methods for medical image classification do not ensure clinical safety under label noise, but integrating cost-sensitive optimization reduces risk while preserving utility.","keywords":["noisy labels","medical imaging","clinical risk","cost-sensitive learning","noise-robust training","false negatives","image classification"],"falsifier":"Collecting real multi-annotator labels on medical images with known patient outcomes and comparing the clinical risk of standard robust models versus cost-sensitive variants on that data.","tokens_in":2644,"feed_emoji":"🏥","tokens_out":677,"duration_ms":22659,"temperature":0.7,"pith_summary":"Noisy labels from annotation errors are common in medical imaging, and standard noise-robust training methods focus on accuracy but overlook that missing a disease (false negative) is far more dangerous than a false alarm. The paper evaluates several state-of-the-art methods on skin and pathology image datasets with added noise, using a risk metric that penalizes missed diagnoses more heavily. It finds that these methods can still lead to high clinical risk despite robustness. Adding cost-sensitive training that accounts for this asymmetry lowers the risk without harming overall performance. This shows that medical AI needs evaluation and training that reflect real patient consequences rather than balanced error rates.","feed_headline":"Cost-sensitive training reduces clinical risk under label noise","feed_subtitle":"Robust methods keep accuracy but miss the higher cost of false negatives in diagnosis; adding asymmetric penalties cuts risk without losing ","key_machinery":"A Global Risk cost formulation that assigns higher penalties to false negatives, integrated into noise-robust training pipelines to optimize for clinical safety in binary medical image classification.","core_discovery":"The paper establishes that the robustness of methods such as Coteaching, DivideMix, UNICON, and GMM-based filtering to label noise does not guarantee low clinical risk when evaluated with a Global Risk formulation that explicitly penalizes false negatives. On binarized versions of DermaMNIST and PathMNIST under 20% and 40% label noise, these methods maintain balanced accuracy but can exhibit elevated risk. Integrating cost-sensitive optimization into the noise-robust training process significantly reduces the clinical risk metric while maintaining model utility.","pith_inferences":["Extending this to multi-class settings with varying risk profiles per class could further improve real-world applicability.","Using actual inter-observer variability data from clinics instead of simulated noise would provide stronger validation.","This risk-aware approach might generalize to other noisy label problems in high-stakes domains like radiology or autonomous systems."],"forward_implications":["Robustness to label noise alone is insufficient to ensure low clinical risk in medical diagnosis tasks.","Cost-sensitive optimization can be combined with existing noise-robust methods to achieve both noise tolerance and risk reduction.","Clinical risk must be explicitly modeled with asymmetric costs to evaluate and improve medical classifiers under noisy labels.","The approach applies to datasets like skin lesion and histopathology images at moderate to high noise levels."],"fun_headline_variants":["Robust methods do not guarantee reduced clinical risk under noise","Standard robust training overlooks clinical risk from false negatives","Cost-sensitive optimization lowers risk in noisy medical classification","Label noise robustness insufficient for clinical safety in diagnosis"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That the simulated label noise rates and the specific Global Risk cost formulation based on binarized datasets sufficiently represent real clinical annotation errors and the asymmetric consequences for patients.","fun_headline_variants_meta":{"raw":{"variants":["Robust methods do not guarantee reduced clinical risk under noise","Standard robust training overlooks clinical risk from false negatives","Cost-sensitive optimization lowers risk in noisy medical classification","Label noise robustness insufficient for clinical safety in diagnosis"]},"model":"grok-4.3","cost_usd":0.009258,"raw_usage":{"total_tokens":4088,"prompt_tokens":716,"num_sources_used":0,"completion_tokens":58,"cost_in_usd_ticks":92578000,"prompt_tokens_details":{"text_tokens":716,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3314,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":716,"tokens_out":58,"duration_ms":27327,"temperature":1.0,"reasoning_tokens":3314,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-08T06:27:13.395743+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Collecting real multi-annotator labels on medical images with known patient outcomes and comparing the clinical risk of standard robust models versus cost-sensitive variants on that data.","supporting_citations":[],"review_version":1}