{"id":"cec91b0d-1a5e-4d48-81bc-bc5372943255","arxiv_id":"2507.02349","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-step nnDetection plus U-Net pipeline detects Circle of Willis bifurcation landmarks in MRA with lower mean radial error and fewer false positives than two heatmap-regression baselines.","lead":"This paper describes a two-stage deep learning system that finds thirteen aneurysm-prone bifurcations in brain MR angiograms, first proposing candidate regions and then refining exact landmark positions with a U-Net. It reports better localization accuracy and far fewer false positives than two published heatmap-regression baselines, which makes automated screening of the Circle of Willis more plausible.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'highest performance' claim is not yet supported because the closest published method on the same public data ([40]) is never compared, and the only comparisons are unvalidated reimplementations without error bars or significance tests.","rationale":"The paper proposes a sensible two-stage pipeline and the In-House dataset with variable landmark counts is a useful evaluation setting. However, the abstract's central claim is about relative performance, and relative performance is only as strong as the comparator list and the statistical resolution of the reported numbers. Omitting [40] is especially damaging because the public data and its annotations come from that work; [40] is the natural state-of-the-art baseline for the public experiment. Self-reimplemented baselines may be weaker than the originals, so the observed advantage could be an artifact of implementation quality rather than of the proposed method. The absence of confidence intervals or significance tests is a further reason to treat the MRE/SDR gaps as unproven. The reader's weakest assumption about first-stage ROI misses is legitimate but secondary: it affects a small number of failure cases and is openly discussed, while the missing comparison and lack of statistical grounding bear directly on whether 'highest performance' has been demonstrated. I do not think the paper should be rejected: the method is plausible, the absolute numbers are strong, and the ROI-miss limitation is acknowledged. But the central claim is conditional on a proper comparison, which is exactly why the verdict remains conditional.","tokens_in":12828,"tokens_out":7243,"duration_ms":86638,"concrete_test":"On the public MRA dataset, evaluate the published method [40] (or its released predictions) on the same 27-image test split, using the same 11-landmark subset and the same MRE/SDR-3/4/5 metrics reported in Table II. If [40] matches or beats 1.47 mm MRE or exceeds 95.62% SDR-5, the 'highest performance' claim loses support. As a companion check, bootstrap the 48 In-House test-set MRE differences between OURS and SCN at the patient level and report the 95% confidence interval.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support 'highest performance', the comparison set must include the strongest relevant prior method and the metric gaps must be distinguishable from noise. The public dataset was annotated by [40], a multi-task UNet designed for exactly these vascular landmarks, yet [40] is never included in Tables I/II. The In-House comparison is restricted to self-reimplementations of SCN [8] and FARNet [26] trained with default settings; no evidence is given that these reimplementations reproduce the published methods' performance, and no error bars or significance tests accompany any metric. FARNet's SDR-5 on the public set (96.28%) exceeds the proposed method (95.62%), but this is not analyzed. The reported MRE advantages (0.24 mm to 0.76 mm) could be within run-to-run variability; with one test split and no repeated seeds this cannot be assessed. The reader's ROI-miss concern is real but secondary: it explains some E/F failures and the 3 false negatives, but it does not by itself undermine the relative superiority claim; the missing [40] comparison and lack of statistical grounding do.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a two-step deep learning pipeline for detecting thirteen Circle of Willis (CoW) bifurcation landmarks in MRA Time-of-Flight images. In the first step, a 3D nnDetection model proposes and classifies regions of interest (ROIs) around landmarks, with some ROIs grouping nearby landmark pairs (A/B, C/G, D/H). In the second step, a modified 3D U-Net with deep supervision regresses Gaussian heatmaps within each ROI to obtain final landmark coordinates. The method is evaluated on an in-house dataset of 145 scans and a public dataset of 104 scans, and is compared against reimplementations of SCN and FARNet. The paper reports a mean radial error of 1.97 mm on the in-house test set and 1.47 mm on the public test set, with fewer false positives than the baselines, and claims the highest level of performance on the bifurcation detection task. An additional experiment examines detection rates on 134 aneurysm-bearing scans.","tokens_in":13074,"tokens_out":3929,"duration_ms":45164,"significance":"If the reported results hold, the two-step design is practically valuable: the ROI stage provides a natural mechanism for handling scans with variable numbers of present landmarks, and the false-positive reduction relative to global heatmap regression is clinically relevant. The paper gives credit to a clear and reproducible pipeline description, a public-dataset evaluation, and explicit reporting of false positives and false negatives. However, the central 'highest performance' claim is not yet supported: the most directly relevant prior method on the public dataset, the multi-task U-Net of [40] that produced the landmark annotations, is never compared, and all metrics are point estimates without error bars or significance tests. The strengths are real but the evidence base is currently insufficient for the strength of the claim.","major_comments":[{"comment":"The abstract and Section III claim the 'highest level of performance,' but the most directly relevant prior method on the public dataset, the multi-task U-Net of [40] that produced the 19-landmark annotations, is never included in Tables I or II. Without [40] in the comparison, the state-of-the-art claim is unsupported. Moreover, on the public set FARNet attains SDR-5 = 96.28% versus 95.62% for the proposed method (Table II), so the proposed method is not the best on every reported metric; this discrepancy needs analysis (e.g., which landmarks and thresholds drive FARNet's advantage).","section":"§III-D2, Table II"},{"comment":"All comparison metrics are point estimates from a single train/test split, with no error bars, confidence intervals, or significance tests. The reported MRE differences (0.24–0.76 mm) may be within run-to-run variability, especially given that the SCN and FARNet baselines are reimplementations 'with their default settings' and no evidence is given that they reproduce the published methods' performance. Please add repeated-seed runs, confidence intervals, and at least a paired test (e.g., Wilcoxon signed-rank over landmarks or scans), or explicitly restrict the claim to 'comparable to' rather than 'highest.'","section":"§III-C, Tables I-II"},{"comment":"The inference procedure keeps only the highest-scoring ROI per label, so any missed or mislabeled first-stage proposal is unrecoverable; the paper acknowledges this for landmarks E and F in the aneurysm experiment. Because the method's large false-positive reduction is achieved by this hard selection, the trade-off between missed landmarks and false positives should be quantified (e.g., by sweeping the ROI retention threshold or reporting recall of the first stage). As written, the reader cannot tell how much of the improvement comes from the ROI mechanism versus the heatmap refinement.","section":"§II-B3, §III-E"}],"minor_comments":[{"comment":"There is a typo: 'we lake use of' should read 'we make use of.'","section":"§II-B"},{"comment":"The text says 'median radial error (MRE) of 1.47 mm,' but MRE is defined as the mean radial error in §III-B; please correct the terminology.","section":"§III-D2"},{"comment":"There is a typo: 'nnDetction' should be 'nnDetection.'","section":"§III-E"},{"comment":"The y-axis of Fig. 9 is unlabeled; please specify that the distance is in millimeters.","section":"Fig. 9"},{"comment":"The text introduces '10 classes' corresponding to 10 ROIs, but three of these classes group two landmarks each; it would be clearer to state explicitly that 10 ROI classes cover the 13 landmark labels.","section":"§II-A2"},{"comment":"The bounding-box size (32 voxels), the grouping of A/B, C/G, D/H, the Gaussian sigma, and the acceptance threshold th are validation-time design choices; a sensitivity analysis, even on the validation set, would strengthen the presentation.","section":"§II-A2, §II-B2"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the missing comparison to [40], which is the method that annotated the public dataset and is therefore the most natural state-of-the-art baseline. The authors should either add this comparison or temper the 'highest performance' claim. The paper is within scope for the journal, and I see no ethical concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful bit. The two-step cascade is a real, sensible design: nnDetection proposes ROIs, then a deep-supervised U-Net regresses heatmaps inside those boxes. Grouping close pairs (A/B, C/G, D/H) into single boxes and using a peak threshold to declare a landmark absent are thoughtful choices for an anatomy where the number of landmarks varies. The In-House result set is cleanly reported, including false positives/negatives, per-landmark MRE, and failure cases; the low FP count (9 vs 47) is the strongest practical argument. The paper also openly acknowledges its E/F failure mode and the dependence on the first-stage ROI, which is more honest than most.\n\nWhere it goes soft. The 'highest performance' claim is not supported as written. On the public dataset, FARNet's SDR-5 (96.28%) is higher than the proposed method (95.62%); the paper mentions this but does not analyze it. More importantly, the closest prior method for this exact task, [40] (multi-task UNet on the same public data with the same annotations), is never included in Tables I/II. Without that comparison, and with only two self-reimplementations of SCN and FARNet trained at 'default settings,' there is no evidence the baselines reproduce published behavior. There are no error bars, confidence intervals, or repeated-seed runs, so the MRE gaps of 0.24–0.76 mm could be run-to-run noise. The ROI-miss concern is real but secondary: it explains the E/F failures and the 3 false negatives, but it does not by itself break the relative claim; the missing [40] comparison and lack of statistical grounding do.\n\nAlso, no code is released, which makes the baseline-reimplementation question harder to check. The paper's self-citation [39] is fine; it cites its own prior work appropriately.\n\nWho should read it: people working on vascular landmark detection or coarse-to-fine cascades for anatomical points. It's a solid engineering contribution that needs firmer evidence before the performance claim can be accepted.\n\nRecommendation: send it to peer review. The method is worth referee time, the weaknesses are fixable, and the dataset (especially the varying-landmark setup) is valuable. The authors should be asked to add [40], validate their baseline reimplementations, and report variance across runs.","headline":"A sensible two-step cascade for Circle of Willis landmarks, but the 'highest performance' claim is under-supported without the [40] comparison and error bars.","tokens_in":13590,"tokens_out":2395,"would_cite":false,"duration_ms":26870,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a two-step pipeline—nnDetection for coarse regions of interest followed by a U-Net for heatmap refinement—achieves the highest Circle of Willis bifurcation landmark detection accuracy on its test sets, with mean…","keywords":["Circle of Willis","landmark detection","two-step neural network","nnDetection","heatmap regression","MRA time-of-flight","intracranial aneurysm","bifurcation detection"],"falsifier":"Take the In-House test set and inspect the stage-1 nnDetection proposals before the top-1 selection: if any true landmark is absent from the proposed ROIs or assigned the wrong label, the pipeline cannot recover it. Counting such cases across the 48 test volumes, especially for the MCA landmarks E and F, would directly show whether the 3 false negatives and the over-10 mm failures come from stage-1 misses rather than stage-2 localization.","tokens_in":12598,"feed_emoji":"🧠","tokens_out":5522,"duration_ms":57467,"temperature":0.7,"pith_summary":"Intracranial aneurysms tend to form at thirteen specific bifurcations of the Circle of Willis, the arterial loop at the base of the brain. This paper claims that a two-step neural network pipeline can find those bifurcation points automatically in MRA time-of-flight scans, without hand-crafted vessel centerlines or pre-segmentation. The first step uses an object detector to propose small regions of interest around candidate bifurcations; the second step uses a modified U-Net to regress heatmaps inside those regions and pick the exact coordinates. On the authors’ In-House test set the pipeline reports a mean radial error of 1.97 mm and only 9 false positives, compared with 2.38 mm and 47 false positives for SCN and 2.73 mm and 47 for FARNet. The method is also designed to handle scans where some bifurcations are absent because of anatomical variation or partial acquisition, which the compared whole-volume heatmap methods do not accommodate.","feed_headline":"Two-step pipeline finds brain-vessel landmarks to 1.97 mm","feed_subtitle":"Coarse detection plus heatmap refinement cuts false positives from 47 to 9 and lowers mean radial error.","key_machinery":"The mechanism has two stages. Stage one is nnDetection, a self-configuring 3D object detector built on a Retina U-Net topology; it regresses fixed-size bounding boxes centered on bifurcations, with ten classes that group landmarks that often sit close together and look alike (A/B, C/G, D/H are paired into single ROIs). Stage two is a residual 3D U-Net with deep supervision that takes $32^{3}$ patches cropped from the proposed ROIs and regresses Gaussian heatmaps for all thirteen labels; the argmax of the relevant heatmap channel gives the coordinate, and a peak-intensity threshold decides whether a landmark is present. The heatmaps from multiple decoder levels are merged by multiplication so that only locations agreed upon across scales survive. At inference only the highest-scoring ROI per label is kept, and the second stage can only refine inside those boxes.","core_discovery":"On its own terms, the central claim is that the correct architecture for vascular landmark detection is a coarse-to-fine pairing: an object-detection stage that localizes and labels candidate regions, followed by a heatmap-regression stage that refines coordinates inside those regions. The paper reports that this pairing outperforms both compared whole-volume heatmap-regression baselines on two datasets, with the largest advantage in false positives and in the MCA landmarks E and F that are visually similar and close together. It also claims the two-step structure makes the method robust to variable landmark counts, since the detector decides which landmarks exist in a given scan before the regressor tries to localize them. The aneurysm-bearing subset shows detection rates around 73 percent within 3 mm and 97 percent within 10 mm, with failure cases attributed to stage-1 ROI misclassification of MCA branches rather than to aneurysm presence.","pith_inferences":["The top-1 ROI selection rule makes recall the silent bottleneck: the paper’s own failure cases for E and F are stage-1 misclassifications, which implies that keeping a small set of candidate ROIs per label or adding a second-pass search could recover misses without re-introducing 47 false positives.","The paired-ROI trick suggests a general recipe: when two landmarks are close and visually confusable, merge them into one detection class and let the heatmap channels separate them.","On the complete-landmark public dataset the margin over SCN is smaller, so the two-step design may pay off most precisely in the variable-anatomy regime; this could be tested by removing the detector and running the same stage-2 U-Net on full-volume heatmaps, which the paper does not do."],"forward_implications":["The method can be applied directly to raw MRA-TOF volumes; no centerline extraction, graph matching, or pre-segmentation step is required.","Scans with missing bifurcations, whether from anatomical aplasia or hypoplasia or from acquisition protocols that cut off the posterior circulation, still get correct per-scan landmark counts instead of forced false positives.","The false-positive count drops from 47 to 9 on the In-House test set because the heatmap stage never searches the full volume, only the proposed ROIs.","The same two-stage recipe should transfer to other anatomical landmark sets where landmarks are locally similar and variable in number."],"supporting_citations":[{"why":"Supplies the first-stage object detector and its self-configuring rule/empirical parameter selection.","marker":"[46]"},{"why":"Provides the encoder-decoder architecture that stage two adapts with residual blocks and deep supervision.","marker":"[48]"},{"why":"Whole-volume heatmap baseline with spatial configuration constraints that the paper compares against on both datasets.","marker":"[8]"},{"why":"Baseline heatmap-regression method with feature aggregation and refinement; also compared on both datasets.","marker":"[26]"},{"why":"Topology underlying nnDetection, combining RetinaNet detection with U-Net segmentation supervision.","marker":"[47]"},{"why":"Coarse-to-fine two-step design that this paper adapts to vascular landmarks.","marker":"[30]"},{"why":"Source of the public dataset’s 19 landmark annotations and a prior multi-task method for the same task.","marker":"[40]"},{"why":"Prior method and source of the In-House ICAN database used for training and testing.","marker":"[39]"}],"fun_headline_variants":["Two-step AI finds brain vessel landmarks under 2 mm","Coarse-to-fine network pinpoints Circle of Willis bifurcations","Object detection plus heatmap: new accuracy in aneurysm screening","AI pipeline cuts false positives 5x in brain vessel landmarking","Two-stage net locates aneurysm-prone spots with 1.97 mm error"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes the first-stage detector always proposes a correct region of interest for every landmark that is actually present; only the highest-scoring box per label is kept, so if the right landmark is not the top box, the second stage never sees it and the landmark is lost.","fun_headline_variants_meta":{"raw":{"variants":["Two-step AI finds brain vessel landmarks under 2 mm","Coarse-to-fine network pinpoints Circle of Willis bifurcations","Object detection plus heatmap: new accuracy in aneurysm screening","AI pipeline cuts false positives 5x in brain vessel landmarking","Two-stage net locates aneurysm-prone spots with 1.97 mm error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3727,"prompt_tokens":930,"completion_tokens":2797,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":2707}},"tokens_in":546,"tokens_out":2797,"duration_ms":22129,"temperature":1.0,"reasoning_tokens":2707,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:31:27.924585+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the In-House test set and inspect the stage-1 nnDetection proposals before the top-1 selection: if any true landmark is absent from the proposed ROIs or assigned the wrong label, the pipeline cannot recover it. Counting such cases across the 48 test volumes, especially for the MCA landmarks E and F, would directly show whether the 3 false negatives and the over-10 mm failures come from stage-1 misses rather than stage-2 localization.","supporting_citations":[{"cited_title":"nnDetection: a self-configuring method for medical object detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the first-stage object detector and its self-configuring rule/empirical parameter selection."},{"cited_title":"U-Net: Convolutional networks for biomedical image segmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the encoder-decoder architecture that stage two adapts with residual blocks and deep supervision."},{"cited_title":"Integrating spatial configuration into heatmap regression based cnns for landmark localization,","cited_arxiv_id":null,"evidence_quote":"Whole-volume heatmap baseline with spatial configuration constraints that the paper compares against on both datasets."},{"cited_title":"Feature aggregation and refinement network for 2d anatomical landmark detection,","cited_arxiv_id":null,"evidence_quote":"Baseline heatmap-regression method with feature aggregation and refinement; also compared on both datasets."},{"cited_title":"Retina u-net: Embarrassingly simple exploitation of segmentation supervision for medical object detection,","cited_arxiv_id":null,"evidence_quote":"Topology underlying nnDetection, combining RetinaNet detection with U-Net segmentation supervision."},{"cited_title":"Fast and accurate craniomaxillofacial landmark detection via 3d faster r-cnn,","cited_arxiv_id":null,"evidence_quote":"Coarse-to-fine two-step design that this paper adapts to vascular landmarks."},{"cited_title":"Multi-task global optimization-based method for vascular landmark detection,","cited_arxiv_id":null,"evidence_quote":"Source of the public dataset’s 19 landmark annotations and a prior multi-task method for the same task."},{"cited_title":"Using deep learning for an automatic detection and classification of the vascular bifurcations along the circle of willis,","cited_arxiv_id":null,"evidence_quote":"Prior method and source of the In-House ICAN database used for training and testing."}],"review_version":1}