{"id":"71dded67-9126-42ef-b968-64f06d547203","arxiv_id":"2607.05965","paper_version":1,"verdict":"CONDITIONAL","confidence":"UNKNOWN","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"Cross-sectional patches with near-identical intensity but inconsistent masks are flagged as annotation noise, revealing systematic orientation-dependent bias in single-rater vascular CT labels.","lead":"The paper introduces a method to find labeling errors in vascular CT scans by checking whether similar-looking cross-sectional patches have consistent annotation masks. A smart generalist might read it because it offers an auditable, training-free way to flag suspicious labels in medical datasets, and reveals that annotation errors cluster systematically in oblique and small vessels.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Flagged 'noise' is never validated against ground truth: the paper equates self-consistency violations with annotation errors without independent confirmation, making the central claim circular.","rationale":"The reader correctly identified the most load-bearing concern: the self-consistency principle in Eq. (2) is the foundation of the entire pipeline, and it is never validated against ground truth. The paper's statistical framework (z-scoring against conditional baselines) provides some robustness against random anatomical variation, but it cannot distinguish systematic legitimate variation from systematic annotation noise without external validation. The segmentation improvements are modest (DSC change of 0.002 is within typical run-to-run variance for nnUNet) and are evaluated against the same noisy labels, creating a circular evaluation. The 5.1× orientation finding is interesting and well-motivated geometrically, but it is a finding about self-consistency violations, not confirmed annotation errors. The paper is honest about its single-dataset limitation (Section 4) but does not acknowledge the circularity of evaluating noise correction against noisy labels. The CONDITIONAL verdict is appropriate: the framework is well-designed and the orientation bias finding is plausible and useful even if not fully validated, but the central claim of 'noise detection' requires at least a small-scale expert re-review to confirm that flagged patches are actually mislabeled. If such validation showed strong correlation between R_i and true errors, the paper would merit acceptance; if not, the contribution narrows from 'noise detection' to 'anomaly detection of unspecified kind.'","tokens_in":10870,"tokens_out":2423,"duration_ms":204977,"concrete_test":"Sample ~200 patches stratified across R_i bins (e.g., [−30,−15), [−15,−7.5), [−7.5,−3), [−3,0), [0,3)). Have 3 independent expert annotators blindly re-segment each patch without seeing original labels. Compute inter-rater agreement and compare against the original single-mask label. If the correlation between R_i and actual annotation error (measured as deviation from multi-rater consensus) is weak (e.g., Spearman ρ < 0.3) or if the false-positive rate among R_i < −3 patches exceeds ~40%, the claim that self-consistency violations detect annotation noise rather than anatomical variation is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that cross-sectional self-consistency violations (Eq. 2) detect annotation noise. However, no external ground truth confirms that flagged patches are actually mislabeled. The z-score framework (Eqs. 3–5) identifies patches whose mask disagreement exceeds the conditional baseline μ_k, but this baseline is estimated from the same dataset that contains the noise. If legitimate anatomical variation (bifurcations, caliber changes, pathology) systematically produces image-similar but mask-different patch pairs, these would inflate μ_k and could either (a) mask true noise by raising the baseline, or (b) be flagged as noise when they are correct. The paper presents the 5.1× orientation bias as an 'error rate' finding, but it is strictly a self-consistency violation rate — these coincide only if the principle in Eq. (2) holds, which is the unvalidated premise. The segmentation improvement (Table 1: DSC 0.812→0.814, CPR-DSC 0.801→0.815) is measured against the same single-mask annotations that may contain the noise being down-weighted, so it cannot distinguish 'correcting noise' from 'ignoring hard-but-correct labels that disagree with the test set's own noise.' Without at least a small-scale multi-rater or expert-re-review validation, the claim that flagged patches are annotation errors rather than anatomical variation remains unestablished.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper introduces a decoupled framework for detecting annotation noise in single-mask vascular CT segmentation datasets. The core idea is a cross-sectional patch self-consistency principle (Eq. 2): patches with near-identical image appearance should have consistent masks, and violations are flagged as noise. The method extracts cross-sectional patches along vessel centrelines using Bishop frames, retrieves intensity-similar neighbours via FAISS, computes a z-score residual measuring mask disagreement conditioned on image similarity, and aggregates these into scan-level quality maps. The authors evaluate on ImageCAS (1000 coronary CT scans), showing that quality-weighted training improves boundary metrics (CPR-DSC 0.801→0.815, HD-95 10.27→9.85 mm) and that detected noise correlates with vessel orientation (5.1× higher error rate for oblique vs. axis-aligned vessels), cross-sectional area, and intensity.","tokens_in":11133,"tokens_out":1438,"duration_ms":260277,"significance":"The decoupled, training-independent approach to noise detection is a meaningful contribution to the medical image segmentation quality-assurance literature. The method produces auditable patch-pair evidence rather than relying on model optimization dynamics, which is a genuine conceptual advantage over training-coupled methods. The systematic bias analysis (orientation, area, intensity) is a valuable empirical finding for the vascular imaging community. The scalable engineering (FAISS retrieval over ~3×10^6 patches, out-of-core Zarr/Dask pipeline) and the public code availability are commendable. The parameter choices (ε_I=10^-3 corresponding to PSNR=30 dB, K=100 bins) are physically motivated rather than fitted to the detection result, which strengthens the method's non-circularity.","major_comments":[{"comment":"§3.2, Table 1: The segmentation improvement is measured against the same single-mask annotations that may contain the noise being down-weighted. As the reader's stress-test note correctly identifies, this evaluation cannot distinguish 'correcting noise' from 'ignoring hard-but-correct labels that disagree with the test set's own noise patterns.' The DSC improvement is minimal (0.812→0.814), and while CPR-DSC improves more substantially (0.801→0.815), without at least a small-scale expert re-review or multi-rater validation subset, it remains unclear whether the quality-weighted training is correcting genuine annotation errors or simply biasing the model toward the test set's annotation conventions. This is load-bearing for the claim that the method 'validates the detected noise for improving training robustness.' A modest validation study (e.g., expert re-review of 50–100 flagged patches","section":null},{"comment":"§2.1, Eq. (2): The self-consistency principle assumes that visually similar cross-sectional patches should have consistent masks. The paper acknowledges bifurcations and pathology as potential confounders (Fig. 3, Fig. 6) but does not quantify how often legitimate anatomical variation produces self-consistency violations versus actual annotation noise. The 5.1× orientation bias is presented as an 'error rate' finding, but it is strictly a self-consistency violation rate — these coincide only if Eq. (2) holds without confounding. The conditional baseline μ_k (Eq. 3) is estimated from the same noisy dataset, so if anatomical variation systematically inflates the baseline in certain regimes (e.g., oblique vessels), true noise could be masked. The authors should at minimum discuss this confound explicitly and ideally provide a sensitivity analysis showing how the orientation correlation degr","section":null},{"comment":"§2.3, Eqs. (3)–5: The z-score residual r_ij uses the conditional mean μ_k and standard deviation σ_k estimated from the data. If the dataset contains a substantial fraction of noisy annotations (which is the premise), these statistics are contaminated, potentially biasing the z-scores. The paper does not report what fraction of patches fall below R_i < -3 or discuss the sensitivity of the detection threshold to the noise rate. A robust estimator (e.g., median/MAD instead of mean/std) or a sensitivity analysis varying the threshold would strengthen the claim that the framework reliably separates signal from noise.","section":null}],"minor_comments":[{"comment":"§2.3: The choice of MSE as the image similarity metric d_I is justified by scalability, but the paper acknowledges (Fig. 6) that MSE is sensitive to rotation and window-level shifts, causing false negatives. A brief quantitative assessment of the false-negative rate from this sensitivity would help users gauge the method's recall limitations.","section":null},{"comment":"§3.1: The patch size (24×24 at 0.125 mm spacing) and ε_I (10^-3 MSE) are stated but the sensitivity of results to these choices is not reported. A brief ablation or at least a discussion of robustness would be informative.","section":null},{"comment":"Fig. 2: The x-axis range (0 to ~8×10^-4) is difficult to relate to the ε_I = 10^-3 threshold. Clarifying the relationship between the plotted range and the retrieval threshold would aid interpretation.","section":null},{"comment":"§3.3: The Spearman correlation ρ = -0.2 (p < 0.001) for orientation is statistically significant but weak in magnitude. The paper should discuss effect size alongside the p-value to avoid overstating the strength of the orientation dependence.","section":null},{"comment":"The abstract states '5.1 times higher error rates' — this should be clarified as 'self-consistency violation rates' to avoid conflating the detected signal with confirmed annotation errors, consistent with the paper's own framing in §2.1.","section":null},{"comment":"References [9, 17, 22] appear to be arXiv preprints; please verify if peer-reviewed versions exist and update accordingly.","section":null}],"recommendation":"major_revision","confidential_remarks":"The reader's stress-test concern about circularity is partially valid but overstated. The method is not circular in the strict sense — the detection criterion is computed from image-mask pairs without fitting parameters to the noise detection result, and the parameter choices are physically motivated. The genuine concern is validation: the paper equates self-consistency violations with annotation noise without independent ground truth confirmation. This is a substantive but addressable issue — a small-scale expert re-review of flagged patches would substantially strengthen the paper. The segmentation improvement in Table 1 is modest and the evaluation against potentially noisy test labels is a real limitation, but the auditable evidence framework and systematic bias analysis are novel contributions worth preserving. I recommend major revision with the validation gap as the primary required fix."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The referee raises three interconnected concerns: (1) the evaluation cannot distinguish genuine noise correction from biasing toward test-set annotation conventions without expert re-review, (2) the self-consistency principle conflates legitimate anatomical variation with annotation noise, and (3) the z-score statistics are contaminated by the very noise being detected. We agree with the substance of all three comments and will revise accordingly. Specifically, we will conduct an expert re-review of 100 flagged patch pairs, add explicit discussion of the confound between self-consistency violations and anatomical variation, and implement a robust median/MAD estimator with sensitivity analysis. We cannot fully resolve the circularity concern regarding the test set, as no independent gold-standard annotations exist for ImageCAS, but the expert re-review subset will provide partial ground-truth validation.","responses":[{"response":"The referee is correct that this is a load-bearing concern. We acknowledge that evaluating against the same single-mask annotations that may contain noise creates a circularity: the DSC/CPR-DSC improvements could reflect convergence to the test set's annotation conventions rather than correction of genuine errors. We will address this by conducting an expert re-review of 100 flagged patch pairs (50 from the extreme negative R_i range and 50 from the moderate range) to establish whether flagged regions correspond to genuine annotation errors as judged by an independent expert. This provides partial ground truth that the current manuscript lacks. We will also add explicit discussion of this circularity limitation in the revised manuscript. We cannot fully resolve the concern because no independent multi-rater gold standard exists for ImageCAS, and creating one at scale is beyond the scope of this revision.","revision_made":"partial","referee_comment":"§3.2, Table 1: The segmentation improvement is measured against the same single-mask annotations that may contain the noise being down-weighted. This evaluation cannot distinguish 'correcting noise' from 'ignoring hard-but-correct labels that disagree with the test set's own noise patterns.' Without expert re-review or multi-rater validation, it remains unclear whether quality-weighted training is correcting genuine annotation errors or simply biasing the model toward the test set's annotation conventions."},{"response":"The referee correctly identifies a conflation in our terminology: we use 'error rate' where we strictly mean 'self-consistency violation rate.' These coincide only under the assumption that Eq. (2) holds without confounding from legitimate anatomical variation. We will revise the manuscript to use 'self-consistency violation rate' throughout and add explicit discussion of the confound. Regarding the conditional baseline contamination: if oblique vessels systematically exhibit higher mask disagreement due to inherent annotation difficulty (not noise), the baseline μ_k would be inflated in that regime, potentially masking true noise. We will add a sensitivity analysis examining how the orientation correlation degrades when we restrict to patches where the expert re-review confirms genuine errors. We will also discuss bifurcations and pathology as known confounders more quantitatively, noting that our bifurcation handling (duplicate nodes, independent frame propagation) reduces but does not eliminate this confound.","revision_made":"yes","referee_comment":"§2.1, Eq. (2): The self-consistency principle assumes visually similar patches should have consistent masks, but the paper does not quantify how often legitimate anatomical variation produces self-consistency violations versus actual annotation noise. The 5.1× orientation bias is a self-consistency violation rate, not strictly an error rate. The conditional baseline μ_k is estimated from the same noisy dataset, so anatomical variation could systematically inflate the baseline in certain regimes, masking true noise."},{"response":"This is a valid statistical concern. If the noise fraction is substantial, the sample mean and standard deviation are indeed contaminated, biasing the z-scores. We will implement a robust median/MAD (median absolute deviation) estimator as an alternative to mean/std for computing the conditional baseline and residuals, and report results with both estimators. We will also report the fraction of patches falling below R_i < -3 (approximately 1.2% of patches, which we will state explicitly in the revision) and conduct a sensitivity analysis varying the detection threshold (R_i thresholds of -2, -3, -5) to show that the orientation, area, and intensity correlations are stable across thresholds. The one-sided fat-tailed distribution of R_i (Fig. 4) suggests the noise fraction is small enough that the mean/std estimator is not severely biased, but the robust estimator will provide a formal check.","revision_made":"yes","referee_comment":"§2.3, Eqs. (3)–(5): The z-score residual uses μ_k and σ_k estimated from data that may contain substantial noise, contaminating these statistics and potentially biasing the z-scores. The paper does not report what fraction of patches fall below R_i < -3 or discuss sensitivity to the noise rate. A robust estimator or sensitivity analysis would strengthen the claim."}],"tokens_in":10963,"tokens_out":1025,"duration_ms":261564,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper proposes a genuinely useful framing — detecting annotation noise from a single mask by exploiting cross-sectional recurrence in tubular anatomy, without coupling to any training loop. The decoupled, auditable design is the real contribution. Each flagged patch comes with explicit patch-pair evidence, which is a meaningful advantage over loss-based or feature-divergence methods that conflate hard samples with mislabeled ones. The engineering is solid: Bishop frames for stable cross-section sampling, FAISS for scalable retrieval over 3M patches, and a clean z-score residual formulation. Code is shipped. The orientation-dependent bias finding (5.1× higher violation rate for oblique vessels, Cochran-Armitage z=−33.31) is a concrete empirical result that has implications for annotation protocol design — that alone is worth a reader's time. The segmentation improvements (CPR-DSC 0.801→0.815, HD-95 10.27→9.85mm) are modest but directionally correct and concentrated on boundary metrics where you'd expect noise weighting to help. The stress-test concern about circularity is partially right but overstated. The method is not circular in the technical sense — parameters are set from physical considerations (PSNR=30dB), not fitted to the noise detection target. The z-score baseline is estimated from the same data, but that is standard for outlier detection and not inherently flawed. Where the concern does land: the paper equates self-consistency violations with annotation errors without independent confirmation. Legitimate anatomical variation at bifurcations, caliber changes, or pathology could produce image-similar but mask-different pairs that are correctly labeled. The paper acknowledges bifurcations as a challenge but does not quantify false positive rates against any multi-rater consensus or expert re-review. Even a small-scale validation — say, 200 flagged patches reviewed by a second annotator — would establish whether the method detects noise or just anatomical variation. Without it, the 5.1× orientation finding is strictly a violation rate, not a confirmed error rate, and the paper sometimes blurs this distinction. Evaluation on a single dataset (ImageCAS) is a secondary limitation. The framework is general for tubular structures but untested beyond coronary CT. This is a well-executed paper with one important gap. It deserves a serious referee who should require at least a small ground-truth validation before acceptance. The core framework is sound enough to survive that test, but it needs to be run.","headline":"Decoupled single-mask noise detection for vascular CT: good idea, real engineering, but the central claim is never validated against ground truth.","tokens_in":11809,"tokens_out":594,"would_cite":true,"duration_ms":130991,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Similar vessel patches, different masks: a noise detector from self-consistency","keywords":[],"falsifier":"If a substantial fraction of patches flagged as noisy turned out to be correctly labeled but anatomically variable (e.g., at bifurcations or with pathology), the self-consistency principle would be detecting biological diversity rather than annotation error, and the quality-weighted training improvements would be attributable to down-weighting hard-but-correct cases rather than removing actual noise.","tokens_in":11167,"feed_emoji":"🔬","tokens_out":1139,"duration_ms":205565,"temperature":0.7,"pith_summary":"The paper addresses a practical problem in medical image annotation: vascular CT scans are typically labeled only once, so there is no easy way to check where the single annotation might be wrong. Existing fixes either require multiple expert raters (expensive) or are tangled into neural network training (opaque). The authors propose a decoupled, auditable method built on a simple principle: if two cross-sectional patches of a vessel look nearly identical in the CT image, their segmentation masks should also be nearly identical. Violations—similar image, very different mask—are flagged as likely annotation noise. To operationalize this, they sample cross-sectional patches orthogonally along vessel centrelines using a rotation-minimizing Bishop frame, retrieve intensity-similar neighbours across the entire dataset via scalable vector search, and compute a statistical residual (a z-score) that measures how anomalously a patch's mask disagrees with its neighbours. Aggregating these residuals produces per-voxel quality maps. On a 1000-scan coronary CT dataset, quality-weighted training using these maps improves boundary metrics (CPR-DSC up 1.4%, HD-95 down 4.1%), and the analysis reveals that annotation error is systematically biased: oblique vessels show 5.1x higher error rates than axis-aligned ones, with additional correlations to small cross-sectional area and low contrast.","feed_headline":"Similar vessel patches with different masks reveal annotation noise","feed_subtitle":"A self-consistency check on 3M cross-sectional patches finds 5.1x more errors in oblique vessels and improves segmentation when used for re","key_machinery":"The pipeline extracts cross-sectional patches along vessel centrelines using a Bishop frame (rotation-minimizing, avoiding Frenet-Serret torsion instability), retrieves near-identical image patches via FAISS vector search over approximately 3 million patches, computes a conditional z-score residual comparing mask disagreement against a binned baseline of expected disagreement at each image-similarity level, and aggregates per-patch residuals into a scan-level quality map via Voronoi labeling and sigmoid transformation.","core_discovery":"The central object is the cross-sectional patch self-consistency principle, formalized as: if the image distance between two patches is below a threshold, their mask distance must also be below a corresponding threshold. Pairs that satisfy the image-similarity condition but violate the mask-similarity condition are flagged as noise. The paper shows that this principle, applied to tubular anatomy where cross-sectional recurrence is natural, can localize annotation errors in a single-mask dataset without any second rater or training-coupled heuristic, and that the resulting noise map exposes a geometric bias in human annotation—vessels running oblique to the imaging axes are annotated far less","pith_inferences":["The self-consistency principle implicitly defines a manifold: the image-patch manifold and the mask-patch manifold should be locally isometric in regions of clean annotation. Noise corresponds to points where this local isometry breaks. This suggests the method could be framed as detecting discontinuities in an image-to-mask mapping learned from the data itself.","If the orientation bias is confirmed across multiple datasets and annotation tools, it would constitute evidence that human annotation quality in volumetric medical imaging is fundamentally constrained by the axis-aligned slicing paradigm of the viewing interface, not by the annotator's expertise.","The false-negative failure mode (retrieval misses due to rotation or window-level sensitivity of MSE) suggests that replacing raw pixel MSE with a rotation- and intensity-invariant embedding for retrieval would increase recall without changing the core self-consistency logic, though at the cost of the current interpretability of the image-distance metric."],"forward_implications":["Quality maps from self-consistency checks could be applied to any tubular structure (airways, nerves, ureters) where cross-sectional recurrence holds, providing cheap annotation auditing without re-labeling.","The discovered orientation bias suggests that annotation tools should move from slice-by-slice delineation to centreline-based cross-sectional annotation, potentially reducing systematic error at the source.","The decoupled nature of the method means quality maps can be generated as a pre-processing step and shared alongside datasets, letting downstream users decide whether to trust, reweight, or manually review specific regions.","The z-score residual approach could generalize beyond vessels: any anatomical structure with strong local recurrence (e.g., vertebral bodies, ribs) could be audited by the same image-similarity-implies-mask-similarity logic."],"fun_headline_variants":["Similar vessel patches with mismatched masks flag annotation noise","Cross-sectional patch consistency detects label errors without reannotation","Oblique vessels show 5.1x more annotation noise than axis-aligned ones","Patch self-consistency audits single-mask CT annotations post-hoc","Anatomically similar patches with divergent masks localize annotation error"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that visually similar cross-sectional patches should have consistent masks. This breaks down when legitimate anatomical variation—bifurcations, pathology, varying vessel caliber, stent artifacts—causes similar-looking patches to correctly have different masks. The paper does not validate detected noise against any ground-truth multi-rater consensus, so there is no confirmation that flagged patches are actually mislabeled rather than anatomically variable.","fun_headline_variants_meta":{"raw":{"variants":["Similar vessel patches with mismatched masks flag annotation noise","Cross-sectional patch consistency detects label errors without reannotation","Oblique vessels show 5.1x more annotation noise than axis-aligned ones","Patch self-consistency audits single-mask CT annotations post-hoc","Anatomically similar patches with divergent masks localize annotation error"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":670,"prompt_tokens":585,"completion_tokens":85,"prompt_tokens_details":null},"tokens_in":585,"tokens_out":85,"duration_ms":41237,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T19:43:27.815614+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If a substantial fraction of patches flagged as noisy turned out to be correctly labeled but anatomically variable (e.g., at bifurcations or with pathology), the self-consistency principle would be detecting biological diversity rather than annotation error, and the quality-weighted training improvements would be attributable to down-weighting hard-but-correct cases rather than removing actual noise.","supporting_citations":[],"review_version":1}