{"id":"2f9ee096-d569-4c15-97d2-116edb597015","arxiv_id":"2507.00832","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Applying brain, artery, and vein segmentation masks after deep learning detection cut false positives by 51-71% in two aneurysm models with no loss of true positives on the test set.","lead":"Two deep learning aneurysm detectors were cleaned up with a simple post-processing step: masks of the brain, arteries, and veins were used to delete detections that sat in veins or outside the brain. The step removed most false alarms without dropping any true detections, though the rules were chosen after looking at the same test cases.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central 'no-TP-loss' claim is based on post-processing rules selected on the same 143-case test set; independent validation is required before accepting it.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence. I agree with that verdict. The most load-bearing concern is that the post-processing method was developed and evaluated on the same dataset, making the 'no-TP-loss' result an in-sample statistic. The paper is transparent about this: the Discussion explicitly says the methods were devised 'based on this FP analysis' of the test outputs. The five methods and the specific parameters (brain dilation, CVS expansion, overlap rule) were selected after looking at the TP/FP labels. Thus, the reported 0 TP loss is the result of selecting a rule that achieved 0, not an independent test of a pre-specified rule. The test set is small (143 CTAs, 218 aneurysms) and single-institution; additionally, the same data were used in the prior 3D-CNN-TR paper (ref 2), further reducing independence. The reader's weakest_assumption (venous segmentation accuracy) is a plausible mechanism by which the rule could fail on new data, but it is secondary to the selection problem: even with perfect segmentations, a rule chosen on this cohort may not transfer. For example, the 'vein > artery' criterion could by chance be well-calibrated to the particular mix of aneurysm locations in this cohort. A concrete external validation, or at least a bootstrap/leave-one-out analysis, would substantially de-risk the claim. I do not recommend changing the verdict: the work is a useful proof-of-concept with interpretable methods and a detailed FP taxonomy, and the conditional verdict already requires independent validation. My concern is essentially the same as the reader's overall rationale, though I would place the primary weight on in-sample selection rather than segmentation accuracy, hence 'partial' agreement.","tokens_in":8409,"tokens_out":7486,"duration_ms":81055,"concrete_test":"Apply the pre-specified method 5 with all parameters frozen (brain mask dilation 3.6 mm, CVS region expansion 3.2 mm, vein-mask overlap > artery-mask overlap, CVS subtracted from vein mask) to an independent, multi-institutional CTA cohort, using the same 0.8 detection threshold. Record TP and FP counts before and after post-processing, with bootstrap confidence intervals over cases. If TP count decreases by even one lesion, or FP reduction falls below roughly 30% (i.e., from the reported 70.6%/51.6%), the central claim needs to be revised to a proof-of-concept with dataset-specific tuning. If no external cohort is available, a leave-one-out or bootstrap analysis over the 143 cases should be reported, though it cannot fully eliminate the selection bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest claim is that method 5 (brain mask + vein-overlap > artery-overlap) removes 70.6% and 51.6% of FPs without losing any TP. The load-bearing concern is not primarily segmentation accuracy but in-sample method selection. In the Discussion, the authors state: 'Based on this FP analysis, we devised post-processing methods.' The five methods (and their parameters, e.g., 3.6 mm brain dilation, 3.2 mm CVS expansion, the 'vein more than artery' rule, and the CVS subtraction) were chosen after inspecting the FP/TP outcomes on the same 143 CTAs used for the reported evaluation. Therefore, the zero-TP-loss result is an in-sample optimum and may not reflect performance on new data. The test set is also the same cohort used in the prior 3D-CNN-TR paper (ref. 2), so it is not a fresh independent evaluation for one of the two detectors. With 218 aneurysms, a zero TP loss in this sample does not guarantee a TP loss rate below a clinically meaningful threshold in a different distribution, especially at locations where artery and vein masks are anatomically close. The central claim 'without reducing TP' is plausible but not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an automated anatomy-based post-processing method to reduce false positives in deep learning-based intracranial aneurysm detection on CTA. The method applies brain, artery, vein, and cavernous venous sinus (CVS) masks to the bounding-box outputs of two detectors (CPM-Net and a 3D-CNN-transformer hybrid, 3D-CNN-TR) and evaluates five removal rules on 143 held-out CTAs with 218 aneurysms. The authors report that the best rule (brain mask plus 'vein overlap greater than artery overlap', method 5) reduces false positives from 126 to 37 for CPM-Net and from 182 to 88 for 3D-CNN-TR, without reducing true positives, lowering FP/case from 0.88 to 0.26 and from 1.27 to 0.62 respectively.","tokens_in":8669,"tokens_out":8705,"duration_ms":89214,"significance":"If these results hold, the proposed post-processing is a simple, computationally cheap, and model-agnostic step that could meaningfully improve the clinical usability of DL-based aneurysm detectors. The systematic categorization of false positives by anatomic structure is a useful contribution to understanding detector failure modes. However, the central 'no true positive loss' claim is currently supported only by an in-sample estimate: the post-processing rules and mask parameters were selected after inspecting the FP/TP outputs on the same test set, and one of the two models was previously evaluated on this same cohort. The result is therefore promising but not yet established.","major_comments":[{"comment":"The post-processing rules and parameters were chosen after inspecting FP/TP outputs on the same 143-CT test set used for the reported evaluation: the Discussion states 'Based on this FP analysis, we devised post-processing methods.' The 3.6 mm brain-mask dilation, the 3.2 mm CVS expansion, the vein-over-artery criterion, and the CVS subtraction are all effectively tuned on the test set. Consequently, the FP reductions in Table 2 are in-sample fitted estimates, and the claim that method 5 removes FPs 'without reducing TP' is not an out-of-sample result. For 3D-CNN-TR, this test set is also the same cohort used in its prior development paper (ref 2), further reducing independence. To establish the central claim, the authors should validate the pre-specified method on an external dataset, or at minimum use a split of the current data into development and validation subsets, and report the variability of the FP reduction.","section":"Discussion (first paragraph)"},{"comment":"Table 2 is internally inconsistent with respect to TP and FN counts. For a fixed set of 218 ground-truth aneurysms, TP + FN must equal 218 in every row. The table lists CPM-Net method 4 as TP=129, FN=79 (sum 208) and 3D-CNN-TR method 4 as TP=169, FN=39 (sum 208). Since method 4 includes method 2, which the text states removes 10 TPs per model, the FN entries should be 89 and 49, respectively. Please correct the table and ensure the text and abstract do not carry the inconsistent numbers.","section":"Table 2"},{"comment":"All results are reported at a single confidence threshold of 0.8 (Table 2 caption). The claim that method 5 reduces FPs without reducing TPs is therefore specific to that operating point; at other thresholds, the overlap between TP and vein masks may differ, so the method could remove TPs or fail to remove FPs. Please report sensitivity and FP/case across the full threshold range, or at least a few thresholds, and provide confidence intervals for the FP reduction proportions (e.g., Wilson intervals) to reflect sampling variability.","section":"Results and Table 2"},{"comment":"The artery-vein segmentation tool (ref 11) was developed on 4D dynamic CTA, and the paper does not quantify its accuracy on the standard CTA test images. The CVS subtraction is a manual atlas-based step with a 3.2 mm expansion, and the brain mask uses a 3.6 mm dilation; these parameters appear to have been selected on the same test set. If the vein mask bleeds into arterial lumen or the CVS subtraction is imperfect, true aneurysms in the skull base could be removed in other cohorts. Please report segmentation accuracy on a subset of the test data and perform a sensitivity analysis on the overlap ratio and dilation/expansion parameters.","section":"Methods (Artery and vein mask)"}],"minor_comments":[{"comment":"The title should read '...reduces false positives and improves interpretability...' for grammatical parallelism.","section":"Title"},{"comment":"The phrase 'confidence interval of 0.8' should be 'confidence threshold of 0.8.'","section":"Table 2 caption"},{"comment":"The statement '27/99 (27.3%) CPM-Net FP' is arithmetically incorrect for the extracranial fraction; Table 3 reports 27 extracranial of 126 total FPs, which is 21.4%. The 3D-CNN-TR figure (77/182, 42.3%) is correct.","section":"Results"},{"comment":"The affiliation line lists 'Washing University in St. Louis'—should be 'Washington University in St. Louis.'","section":"Author affiliations"},{"comment":"The typo 'dters radiologists' should be 'deters radiologists,' and 'cylindical' should be 'cylindrical.'","section":"Discussion"},{"comment":"The abbreviation CVS is used without spelling it out in the abstract; please define it at first use.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The authors should be encouraged to reframe the results as a pilot demonstration rather than a definitive claim, given the in-sample selection of post-processing parameters. The reuse of the 143-CT test set from ref [2] for one of the two models is a serious independence concern that the editor should weigh. The paper's title claims improved interpretability, but no interpretability assessment is performed; this should be tempered or supported with a user study. These issues are fixable with revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid, narrow contribution. The authors show that anatomy masks can clean up aneurysm-detector false positives and they provide a genuinely useful FP taxonomy. The catch is that the rules were chosen after looking at the same 143 CTAs used for evaluation, so the 'without reducing TP' headline is an in-sample result until shown otherwise.\n\nWhat's new: applying brain/artery/vein/CVS masks as an explicit post-detection filter, plus the vein-over-artery overlap criterion, and a detailed breakdown of where FPs live (venous, extracranial, branchpoints, etc.). The taxonomy alone is worth having. The numbers in Table 2 are internally consistent, and the method is cheap and interpretable, which matters for clinical adoption.\n\nWhere I'd push back: the stress-test note is on target. The Discussion says 'Based on this FP analysis, we devised post-processing methods,' and the five methods plus parameters (3.6 mm dilation, 3.2 mm CVS expansion, the overlap rule) were settled on the same test set. That makes the zero-TP-loss result an optimum, not a prediction. Also, the evaluation uses one confidence threshold (0.8), no confidence intervals, no external cohort, and the test set is the same one used for ref 2's detector. For 3D-CNN-TR specifically, arterial FPs dominate, and method 5 only removes 22/97 arterial FPs; the 51.6% FP reduction is driven mostly by extracranial and venous removals. That limits the clinical win for that model. The vein-mask bleed risk near the skull base is real, and the CVS subtraction is doing load-bearing work.\n\nThe paper is honest about the small test-set limitation, but it does not flag the in-sample selection issue as such, and that is the main thing to fix.\n\nWho this is for: researchers working on aneurysm detection and anyone building interpretable post-processing for medical object detection. The FP taxonomy has reuse value. It deserves a serious referee. I would accept it for review with the expectation that the revised version either validates on an independent cohort or reframes the claim as a hypothesis-generation/in-sample demonstration, and reports multi-threshold results with confidence intervals.","headline":"A useful, narrow post-hoc FP-filter paper with a genuinely informative FP taxonomy, but the headline zero-TP-loss claim is an in-sample result until independently validated.","tokens_in":9193,"tokens_out":1671,"would_cite":true,"duration_ms":18926,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An automated anatomy-based filter that compares vein and artery overlap removes most false positives from two deep-learning aneurysm detectors while preserving all true positives.","keywords":["intracranial aneurysm detection","false-positive reduction","post-processing","anatomy-based segmentation","CT angiography","deep learning","vein-artery overlap","interpretable AI"],"falsifier":"Run Method 5 on a larger independent CTA cohort and count how many ground-truth aneurysm bounding boxes are removed; the paper's claim predicts zero, so a single deleted true positive, especially near the cavernous sinus or skull base, would refute it.","tokens_in":8243,"feed_emoji":"🧠","tokens_out":9517,"duration_ms":98595,"temperature":0.7,"pith_summary":"Deep-learning models that flag intracranial aneurysms on CT angiography are held back from clinical use by false positives, because every extra alert costs radiologist time and trust. This paper tests whether a fully automated post-processing step, built from anatomical segmentation masks rather than model retraining, can strip those false alarms out. On two detectors, the best filter removes any output outside the brain or whose overlap with a vein mask exceeds its overlap with an artery mask, eliminating 70.6% of false positives for one model and 51.6% for the other while the number of true positives stays exactly the same. The paper argues that false-positive reduction can be decoupled from the black-box model and grounded in interpretable anatomy, which also makes it easier to see why a given detection was removed.","feed_headline":"Anatomy filter cuts aneurysm AI false positives by 70.6%","feed_subtitle":"A vein-vs-artery overlap mask removed most false alarms from two deep-learning detectors, losing no true positives.","key_machinery":"The load-bearing mechanism is the vein-more-than-artery overlap rule, combined with a brain mask. Each detector output is a 3D bounding box; the rule keeps a box only if its overlap with the arterial segmentation mask is at least as large as its overlap with the venous segmentation mask, and only if the box lies within a dilated brain mask with the cavernous venous sinus region added to include the skull base. The cavernous venous sinus is subtracted from the vein mask beforehand because it spatially overlaps common aneurysm locations. This one comparison encodes the anatomic fact that aneurysms arise from arteries: a true positive should look arterial, while the dominant false-positive classes—veins, extracranial vessels, and vein-adjacent tissue such as the choroid plexus—look venous or extracranial. It is the combination of the brain filter for extracranial false positives and the vein-versus-artery comparison for venous false positives that carries the argument.","core_discovery":"Using 143 held-out CTAs with 218 annotated aneurysms, the paper shows that the false positives produced by CPM-Net and a deformable 3D CNN-transformer hybrid are not random noise: most sit in identifiable anatomic compartments, including extracranial tissue, veins such as the vein of Galen and extracranial veins, arterial branchpoints and infundibula, and non-vascular structures such as the choroid plexus. The post-processor applies four segmentation masks—brain, artery, vein, and cavernous venous sinus—to detector output boxes. Removing everything outside the dilated brain mask eliminates all extracranial false positives on both models. Removing boxes whose vein overlap exceeds artery overlap, after subtracting the cavernous sinus from the vein mask, removes the bulk of venous false positives while preserving every true positive: CPM-Net goes from 126 to 37 false positives and 3D-CNN-TR from 182 to 88, with true positives unchanged at 139 and 179. The paper interprets this as evidence that anatomy-grounded, interpretable post-processing can replace or complement threshold tuning as a false-positive reduction strategy.","pith_inferences":["Beyond the paper, the same looks-more-venous-than-arterial overlap test could be transferred to other arterial targets such as arteriovenous malformations, dural fistulas, or venous sinus thrombosis, where the vessel of origin is known.","Beyond the paper, the unresolved false-positive class of normal arterial branchpoints and infundibula suggests that overlap-based masks have a natural limit, and that geometric vessel-shape criteria, such as saccular versus tubular contour analysis, would be the next logical filter to test.","Beyond the paper, because the method is applied at a fixed threshold, combining it with threshold tuning or an ensemble of detectors would likely push FP/case lower still, a combination the paper does not report.","Beyond the paper, a practical clinical check is whether the segmentation masks remain as reliable on routine CTA as on the dynamic CTA data used to develop the artery-vein segmentation algorithm; if not, the no-true-positive-loss result may shrink in broader deployment."],"forward_implications":["At the fixed confidence threshold of 0.8, the best filter cuts FP/case from 0.88 to 0.26 for CPM-Net and from 1.27 to 0.62 for 3D-CNN-TR, with true-positive counts unchanged at 139 and 179.","Brain-mask filtering alone removes all extracranial false positives, 27 of 27 for CPM-Net and 77 of 77 for 3D-CNN-TR, without removing a single true positive.","Venous false positives are the largest removable class, with 94.4% of CPM-Net's and 92.5% of 3D-CNN-TR's venous false positives removed by the vein mask.","Because the filter is applied after detection and uses only segmentation masks, it can in principle be attached to any existing detector without retraining.","The accompanying false-positive taxonomy, divided into extracranial, venous, arterial branchpoint, and non-vascular tissue categories, offers a map for designing further post-processing rules."],"supporting_citations":[{"why":"Supplies the 3D-CNN-TR detector and the shared held-out evaluation data used in this study.","marker":"[2]"},{"why":"Supplies CPM-Net, the other detector whose outputs are post-processed.","marker":"[8]"},{"why":"Supplies the 1,186 open-source CTAs with 1,373 annotated aneurysms used to train both detectors.","marker":"[9]"},{"why":"Supplies the artery-vein segmentation algorithm and the CTA atlas with annotated cavernous venous sinus used to construct the masks.","marker":"[11]"},{"why":"Supplies the brain segmentation used by Methods 1, 4, and 5.","marker":"[12]"}],"fun_headline_variants":["Anatomy filter removes 70% of aneurysm AI false alarms","Vein vs artery mask reduces most false aneurysm AI hits","Brain and vein masks cut aneurysm AI false positives, preserve true ones","Domain-informed post-processing lowers aneurysm AI false positives","Segmentation masks clear most false aneurysm AI detections"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The no-true-positive-loss claim rests on the assumption that the automatic vein segmentation, after the cavernous venous sinus is subtracted, never overlaps a real aneurysm more than the artery segmentation does; if the vein mask bleeds into arterial lumen near the skull base, true positives would be deleted in a broader cohort.","fun_headline_variants_meta":{"raw":{"variants":["Anatomy filter removes 70% of aneurysm AI false alarms","Vein vs artery mask reduces most false aneurysm AI hits","Brain and vein masks cut aneurysm AI false positives, preserve true ones","Domain-informed post-processing lowers aneurysm AI false positives","Segmentation masks clear most false aneurysm AI detections"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001421,"raw_usage":{"total_tokens":5888,"prompt_tokens":1248,"completion_tokens":4640,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":864,"completion_tokens_details":{"reasoning_tokens":4559}},"tokens_in":864,"tokens_out":4640,"duration_ms":41476,"temperature":1.0,"reasoning_tokens":4559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:05:18.373242+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Method 5 on a larger independent CTA cohort and count how many ground-truth aneurysm bounding boxes are removed; the paper's claim predicts zero, so a single deleted true positive, especially near the cavernous sinus or skull base, would refute it.","supporting_citations":[{"cited_title":"Vessel-aware aneurysm detection using multi-scale deformable 3D attention","cited_arxiv_id":null,"evidence_quote":"Supplies the 3D-CNN-TR detector and the shared held-out evaluation data used in this study."},{"cited_title":"CPM-Net: A 3D Center-Points Matching Network for Pulmonary Nodule Detection in CT Scans","cited_arxiv_id":null,"evidence_quote":"Supplies CPM-Net, the other detector whose outputs are post-processed."},{"cited_title":"Toward human intervention-free clinical diagnosis of intracranial aneurysm via deep neural network","cited_arxiv_id":null,"evidence_quote":"Supplies the 1,186 open-source CTAs with 1,373 annotated aneurysms used to train both detectors."},{"cited_title":"Dynamic-Computed Tomography Angiography for Cerebral Vessel Templates and Segmentation","cited_arxiv_id":"2502.09893","evidence_quote":"Supplies the artery-vein segmentation algorithm and the CTA atlas with annotated cavernous venous sinus used to construct the masks."},{"cited_title":"TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images","cited_arxiv_id":null,"evidence_quote":"Supplies the brain segmentation used by Methods 1, 4, and 5."}],"review_version":1}