{"id":"40f3204c-7771-4110-a314-dc9ca82e7b62","arxiv_id":"2508.00235","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A weakly supervised multi-task U-Net with vesselness priors segments intracranial aneurysms in TOF-MRA with Dice 0.614 and detects them with 92.9% sensitivity and 1.47 false positives per scan.","lead":"The authors propose a deep learning system that detects and segments brain aneurysms in MRI scans using only coarse labels for training. It combines a standard U-Net architecture with vessel-shape priors and reports strong accuracy on an internal test set and an external benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline metrics rely on refined internal test labels from the same Lausanne database as training; without proof that refinement never used model outputs and that patients are fully disjoint, the reported Dice and sensitivity may be inflated.","rationale":"The reader's weakest assumption correctly identifies the refined internal labels as the primary risk. My stress test reaches the same conclusion: the abstract does not establish that the internal test labels are independent of the training process, nor that patient-level separation was maintained. The external ADAM validation is real supporting evidence for generalizability, but it does not directly validate the internal segmentation and detection numbers. Because only the abstract is available, the correct verdict remains unverified; therefore no change to the reader's UNVERDICTED verdict is needed. The proposed test would settle the concern by requiring an audit of the refinement protocol and a re-evaluation on truly independent expert annotations.","tokens_in":815,"tokens_out":2639,"duration_ms":30038,"concrete_test":"Request the patient-level train/test split and the full label-refinement protocol, specifically whether any model outputs, predictions, or saliency maps were consulted during refinement. Then re-evaluate the trained model on an independent held-out set of TOF-MRA volumes with expert annotations that were never touched by the refinement process, ideally with two independent raters, and report Dice, 95% HD, and per-aneurysm sensitivity. If the refined labels were influenced by model outputs, or if the metrics on the independent set fall materially below 0.614 Dice and 92.9% sensitivity, the claimed superiority over SOTA is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of superior segmentation (Dice = 0.614, 95%HD = 1.38 mm) and detection (FPR = 1.47, sensitivity = 92.9%) depends on evaluation against a refined-label test set from the same Lausanne database used to produce the coarse training labels. Two conditions must hold for these numbers to be credible. First, the refinement process must not have been influenced by the proposed model or its predictions; otherwise the test labels are not an independent gold standard and both Dice and sensitivity are biased upward. Second, there must be no patient overlap between the coarse-labeled training set and the refined-labeled test set; the abstract is silent on patient-level separation. The external ADAM validation is a useful generalizability check, but it does not repair a possible internal label-leakage problem because the headline numbers come from the internal set. In addition, the comparison to unspecified SOTA techniques cannot be audited from the abstract alone. These are verification gaps, not evidence of misconduct, but they mean the central claim is currently unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This abstract-only manuscript proposes a weakly supervised 3D multi-task UNet that incorporates Frangi vesselness priors to jointly detect and segment intracranial aneurysms in TOF-MRA. The model is trained on the Lausanne dataset with coarse ground-truth labels and evaluated on refined labels from the same database, with additional external validation on the ADAM dataset. The abstract reports a segmentation Dice of 0.614, a 95th-percentile Hausdorff distance of 1.38 mm, a detection false positive rate of 1.47, and a sensitivity of 92.9%, claiming superior performance over state-of-the-art techniques.","tokens_in":1019,"tokens_out":2888,"duration_ms":29224,"significance":"If the reported results hold, the paper makes a clinically relevant contribution by reducing the annotation burden through weak supervision while retaining detection and segmentation performance. The inclusion of an external ADAM validation is a positive sign for generalizability, and the use of vesselness priors in a multi-task architecture is a plausible technical direction. However, because the review is based on the abstract alone, the central claim of superiority over state-of-the-art methods cannot currently be assessed; the missing evaluation details and statistical support leave the reported numbers unverified.","major_comments":[{"comment":"The evaluation protocol is underspecified in a way that is load-bearing for the headline claim. The abstract states that the model is trained on coarse labels from the Lausanne dataset and evaluated on refined labels from the same database, but it does not state whether the label-refinement process ever used the proposed model's outputs, nor whether patients are disjoint between the coarse-labeled training set and the refined-labeled test set. If either condition fails, the reported Dice and sensitivity could be inflated, and the external ADAM validation does not remove this internal risk because the headline numbers are taken from the internal set.","section":"Abstract"},{"comment":"The comparison to 'SOTA techniques' is not auditable: no names, configurations, or statistical significance tests are reported for the point estimates (Dice = 0.614, 95%HD = 1.38 mm, FPR = 1.47, sensitivity = 92.9%). Without confidence intervals, standard deviations, paired comparisons, or at least an enumeration of the compared methods, the claim of superior performance cannot be verified from the abstract.","section":"Abstract"},{"comment":"The abstract provides no measure of uncertainty or sample size for the reported metrics. For a detection task on small lesions, a sensitivity of 92.9% and a false positive rate of 1.47 may be driven by a small test set; the manuscript should report the number of patients and aneurysms in each test set, together with confidence intervals or per-case results, to establish that the internal and external findings are stable.","section":"Abstract"}],"minor_comments":[{"comment":"The false positive rate is not defined; the abstract should state whether it is per patient, per scan, or per volume so that the detection result is interpretable.","section":"Abstract"},{"comment":"The metric '95%HD' should be spelled out and defined, even though the unit (mm) is given; clarification of whether it is the 95th percentile of Hausdorff distances and how outliers are handled would improve precision.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"The abstract-only submission prevents independent verification of the key evaluation-protocol details. I recommend obtaining the full manuscript and checking specifically whether the refined internal test labels were generated independently of the model and whether patient-level splits between training and test are explicitly enforced. The external ADAM validation is useful but does not mitigate a potential internal label-leakage issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is an abstract-only review, but the work seems genuinely useful if the full paper holds up. The contribution is a specific combination—a 3D multi-task UNet for joint IA detection and segmentation, with Frangi vesselness priors used both as input and as an attention signal. That is a reasonable extension of known components, and the external ADAM validation is a real plus: too many medical imaging papers never leave their home dataset.\n\nThe reported numbers (Dice 0.614, 95%HD 1.38 mm, sensitivity 92.9%, FPR 1.47) are in the right ballpark for this difficult task, but I can't yet call them superior to SOTA. The abstract doesn't name the SOTA methods, give confidence intervals, or report ablations. More importantly, the internal test set uses refined labels from the same Lausanne database that produced the coarse training labels. The abstract is silent on how the refinement was done and on patient-level separation between train and test. If the refinement used model outputs, or if patients overlap, the internal numbers are inflated. The external ADAM validation helps generalizability, but it does not repair a possible internal label-contamination problem because the headline claims come from the internal set.\n\nI want to be clear: I have no evidence of misconduct. These are verification gaps. A serious referee should ask for the label-refinement protocol, patient-level splits, and a comparison table with named methods, ideally with code or an appendix.\n\nWould I bring it to reading group? Maybe, if someone is working in medical imaging. I would not cite it before seeing the full paper. But I would send it to peer review—the method is plausible, the external validation is a mark of effort, and the questions are answerable with a revision.\n\nBottom line: give it a referee, but insist on the missing methodological details before trusting the headline numbers.","headline":"A plausible weakly supervised aneurysm detection/segmentation method with an external validation set, but the headline numbers rest on internal refined labels whose protocol is not described in the abstract.","tokens_in":1497,"tokens_out":1155,"would_cite":false,"duration_ms":14183,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a weakly supervised 3D multi-task UNet with vesselness priors jointly detects and segments intracranial aneurysms in TOF-MRA, and that on the Lausanne test set with refined labels it reaches a Dice of 0.614, a 95%…","keywords":["weak supervision","intracranial aneurysm","TOF-MRA","vesselness filter","multi-task UNet","aneurysm detection","aneurysm segmentation","deep learning"],"falsifier":"If an audit of the Lausanne data shows patient-level overlap between the coarse-labeled training set and the refined test set, or if label refinement used the model's predictions, then the reported Dice and sensitivity are inflated. A clean check is to re-run the evaluation on a fully independent, expert-annotated dataset with patient-level separation and compare Dice, 95% Hausdorff distance, false positive rate, and sensitivity.","tokens_in":655,"feed_emoji":"🧠","tokens_out":3898,"duration_ms":33174,"temperature":0.7,"pith_summary":"This paper tries to show that accurate aneurysm detection and segmentation can be done without voxel-wise expert annotations, using a weakly supervised 3D multi-task UNet trained on coarse labels. The network jointly performs detection and segmentation in time-of-flight MR angiography by feeding vesselness-derived soft priors into both the input and an attention block. On the Lausanne test set with refined labels, the method reports a Dice of 0.614, a 95% Hausdorff distance of 1.38 mm, a false positive rate of 1.47, and a sensitivity of 92.9%, outperforming state-of-the-art techniques. A validation on the external ADAM dataset is used to support generalizability. If these numbers hold, the approach would ease the annotation bottleneck in cerebrovascular imaging while improving clinical aneurysm screening.","feed_headline":"Aneurysm screening hits 92.9% sensitivity with weakly supervised UNet","feed_subtitle":"Vesselness priors plus coarse labels beat prior methods for aneurysm detection and segmentation in MR angiography.","key_machinery":"The core mechanism is the vesselness prior computed by a Hessian-based vessel enhancement filter. This soft cerebrovascular prior is used twice: once as additional input channels to the 3D UNet, and once inside an attention block that reweighs decoder features, so segmentation and the auxiliary detection branch focus on vessel-like structures. The multi-task design lets detection and segmentation share representations, and weak supervision from coarse labels supplies training signal without voxel-wise expert annotations.","core_discovery":"The central claim is that integrating vesselness priors into a 3D multi-task UNet improves both aneurysm detection and segmentation under weak supervision. The vesselness filter supplies soft cerebrovascular priors that are concatenated into the network input and used in an attention block, guiding the decoder for segmentation and an auxiliary branch for detection. Trained on coarse ground truth from the Lausanne database and tested on refined labels from the same database, the model achieves a Dice of 0.614 and a 95% Hausdorff distance of 1.38 mm for segmentation, with a false positive rate of 1.47 and a sensitivity of 92.9% for detection, which the authors state outperforms current state-of-the-art methods. External validation on the ADAM dataset is reported as further evidence of generalizability.","pith_inferences":["The same vesselness-gated multi-task design could plausibly transfer to other vascular abnormalities, such as stenoses or arteriovenous malformations, where lesion shape and vessel context matter.","Because the internal test labels were refined from the same database used for training, the reported Dice and sensitivity are credible only if no patient overlaps the coarse training set and the refinement was done without looking at model outputs; the paper does not state this.","A direct comparison against the same architecture trained without vesselness priors would isolate how much of the gain comes from the prior versus the multi-task setup.","The method's performance on the external ADAM dataset could be reported separately to quantify the distribution shift between databases."],"forward_implications":["If the reported results are reproducible, weakly supervised training on coarse labels can replace expensive voxel-wise annotation for aneurysm segmentation.","Joint detection and segmentation in one network would let a single pass through a TOF-MRA volume flag suspected aneurysms and measure their size and shape for rupture-risk assessment.","The 92.9% sensitivity at 1.47 false positives per volume suggests the method could serve as a screening tool that reduces missed aneurysms.","Vesselness priors appear to help detect small, low-contrast lesions, which is the hardest case in clinical practice.","External validation on ADAM points toward deployment across scanners and protocols, though the internal test set remains the source of the headline numbers."],"supporting_citations":[],"fun_headline_variants":["Weakly supervised UNet spots aneurysms with 92.9% sensitivity","Vesselness priors boost aneurysm detection in MR angiography","Coarse labels, fine results: weakly supervised aneurysm tools","Multi-task UNet with vesselness prior tops aneurysm segmentation","External validation backs weakly supervised aneurysm UNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the refined test labels, created from the same Lausanne database used for training, are an unbiased gold standard that was not influenced by the model's own outputs and that no patient overlaps the training set.","fun_headline_variants_meta":{"raw":{"variants":["Weakly supervised UNet spots aneurysms with 92.9% sensitivity","Vesselness priors boost aneurysm detection in MR angiography","Coarse labels, fine results: weakly supervised aneurysm tools","Multi-task UNet with vesselness prior tops aneurysm segmentation","External validation backs weakly supervised aneurysm UNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1434,"prompt_tokens":981,"completion_tokens":453,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":370}},"tokens_in":597,"tokens_out":453,"duration_ms":4773,"temperature":1.0,"reasoning_tokens":370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:16:15.341767+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If an audit of the Lausanne data shows patient-level overlap between the coarse-labeled training set and the refined test set, or if label refinement used the model's predictions, then the reported Dice and sensitivity are inflated. A clean check is to re-run the evaluation on a fully independent, expert-annotated dataset with patient-level separation and compare Dice, 95% Hausdorff distance, false positive rate, and sensitivity.","supporting_citations":[],"review_version":1}