pith. sign in

arxiv: 2606.12824 · v2 · pith:TEVAMGMUnew · submitted 2026-06-11 · 📡 eess.IV · cs.AI· cs.CV· physics.med-ph

Acquisition state behaves as a structured, measurable variable governing lung-nodule AI: kernel-driven measurement instability and noise-driven detection fragility, invisible to DICOM metadata

Daniel Soliman This is my paper

Pith reviewed 2026-06-27 05:54 UTC · model grok-4.3

classification 📡 eess.IV cs.AIcs.CVphysics.med-ph
keywords lung nodule detectionCT reconstruction kernelAI measurement instabilitydetection fragilityDICOM metadata limitsacquisition parametersAI governancepixel fingerprint
0
0 comments X

The pith

Acquisition state acts as an unmonitored variable that drives distinct failure modes in lung-nodule AI detectors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether CT acquisition parameters function as a structured input variable for lung-nodule AI. On real paired scans differing only in reconstruction kernel, kernel alone shifted AI diameter measurements enough to flip Fleischner size categories in 5.2 percent of nodules while detection confidence stayed the same. Controlled perturbations showed the noise axis degraded detection but not measurement, whereas the kernel axis corrupted measurement but not detection. A 4-feature pixel fingerprint recovered kernel identity where the DICOM ConvolutionKernel tag was uninformative, and the pattern held across four manufacturers. The work concludes that current metadata-based monitoring misses these input effects and that acquisition-aware validation is required.

Core claim

Acquisition state behaves as a structured, measurable variable governing lung-nodule AI: kernel-driven measurement instability and noise-driven detection fragility, invisible to DICOM metadata. On real paired CT differing only in reconstruction kernel, kernel alone shifted AI-measured diameter and flipped a Fleischner size category in 5.2% of nodules while detection confidence was unchanged; noise degraded detection but not measurement.

What carries the argument

The dissociation between the frequency/kernel axis (corrupting measurement) and the noise axis (degrading detection), recovered by a 4-feature pixel fingerprint that identifies reconstruction identity across vendors.

If this is right

  • Kernel changes alone can alter AI diameter measurements enough to change clinical size categories without affecting detection scores.
  • Noise increase primarily reduces detection confidence for nodules under 6 mm while leaving measurements stable.
  • A pixel fingerprint recovers acquisition identity where the ConvolutionKernel DICOM tag is identical across reconstructions.
  • The kernel effect transports across manufacturers with leave-one-vendor-out performance matching within-vendor levels.
  • Input-side validation beyond metadata monitoring is required to meet acceptance-testing and drift-monitoring standards for imaging AI.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same acquisition-state dissociation may appear in AI models trained for other detection or segmentation tasks in CT.
  • Pixel fingerprint checks could be added to clinical pipelines to flag studies outside a model's validated acquisition envelope.
  • Part of the observed drop in AI performance when moving from research datasets to hospital data may trace to untracked kernel and noise differences.
  • Repeating the paired-scan analysis on additional detector architectures would test whether the measurement-versus-detection split is model-specific.

Load-bearing premise

That the controlled perturbations applied to LIDC-IDRI data accurately isolate the frequency/kernel axis from the noise axis in a manner representative of real multi-vendor clinical acquisitions, and that the observed effects on this single MONAI RetinaNet model are not idiosyncratic to its training or architecture.

What would settle it

A controlled experiment on real paired CT scans from multiple vendors showing no change in AI diameter measurements or detection rates when reconstruction kernel is varied while holding all other factors fixed would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.12824 by Daniel Soliman.

Figure 1
Figure 1. Figure 1: Reconstruction kernel alone shifts AI-measured nodule diameter on real paired NLST [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Acquisition effects dissociate by physical axis. (A) Measurement effect (mean [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Reconstruction identity is recoverable from pixels but not from the DICOM header. A [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The kernel axis transports across manufacturers. (A) The mean soft [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
read the original abstract

AI governance for medical imaging is formalizing: the 2026 ACR-SIIM Practice Parameter recommends local acceptance testing and ongoing drift monitoring, and the ACR Assess-AI registry monitors AI outputs using DICOM metadata for context. We argue that a necessary, currently unmonitored layer sits beneath output metrics: whether incoming studies remain within the acquisition envelope a model was validated on. Using a LUNA16-trained MONAI RetinaNet lung-nodule detector, we test whether acquisition state behaves as a structured, measurable variable. On real paired CT differing only in reconstruction kernel (NLST B30f vs B80f), kernel alone shifted AI-measured diameter and flipped a Fleischner size category in 5.2% (8 of 155) of nodules at fixed patient and acquisition, while detection confidence was unchanged (Wilcoxon p=0.22). Under controlled LIDC-IDRI perturbations the effects dissociated by axis: the noise axis degraded detection confidence (p=5.9e-32, concentrated in nodules under 6 mm) but not measurement, while the frequency/kernel axis corrupted measurement (p=8.6e-13) but not detection. A 4-feature pixel fingerprint recovered reconstruction identity (patient-level AUC about 0.95 on real CT, 0.995 on a QIBA phantom) where the ConvolutionKernel DICOM tag was uninformative (identical labels across reconstructions). The kernel axis transported across four manufacturers (leave-one-vendor-out AUC 0.94-0.98, matching the within-vendor ceiling). Acquisition state thus maps to distinct AI failure modes, frequency content to measurement reliability and noise to detection sensitivity, and is not recoverable from metadata. Acquisition-aware, input-side validation is the missing layer for the acceptance-testing and drift-monitoring requirements now entering imaging-AI accreditation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript claims that acquisition state (reconstruction kernel and noise) functions as a structured, measurable variable that governs lung-nodule AI behavior in distinct ways: on real NLST paired CTs differing only in kernel (B30f vs B80f), kernel shifts AI-measured diameter and flips Fleischner size category in 5.2% (8/155) of nodules while leaving detection confidence unchanged (Wilcoxon p=0.22); controlled LIDC-IDRI perturbations dissociate the axes, with noise degrading detection (p=5.9e-32, especially <6 mm nodules) but not measurement and kernel corrupting measurement (p=8.6e-13) but not detection. A 4-feature pixel fingerprint recovers reconstruction identity (patient-level AUC ~0.95 on real CT, 0.995 on QIBA phantom) where the ConvolutionKernel DICOM tag is uninformative, and the kernel signal transports across four vendors (leave-one-vendor-out AUC 0.94-0.98). The work concludes that acquisition-aware input validation is needed beyond current DICOM-metadata-based governance.

Significance. If the reported dissociation and fingerprint results hold, the paper identifies a concrete, currently unmonitored input-side failure mode relevant to the 2026 ACR-SIIM Practice Parameter and ACR Assess-AI registry. The use of named public datasets (NLST, LIDC-IDRI, LUNA16, QIBA) and concrete statistics on a MONAI RetinaNet model provides a reproducible starting point for acquisition-envelope testing.

major comments (2)
  1. [Abstract / Results (perturbation experiments)] The dissociation claim—that the frequency/kernel axis selectively corrupts measurement (p=8.6e-13) while the noise axis selectively degrades detection (p=5.9e-32)—is shown exclusively on controlled synthetic perturbations of LIDC-IDRI; the real paired NLST data tests only the kernel effect. The manuscript provides no direct evidence that the perturbation method (synthetic kernel filtering plus noise injection) isolates the two axes as cleanly as real multi-vendor clinical acquisitions, where dose, collimation, and reconstruction interact. This assumption is load-bearing for the central claim of axis-specific failure modes.
  2. [Abstract / Methods (implied)] All quantitative results, including the 5.2% category-flip rate, p-values, and AUCs, are reported for a single MONAI RetinaNet model trained on LUNA16. No cross-architecture or cross-training-regime experiments are described, leaving open whether the observed kernel-driven measurement instability and noise-driven detection fragility are general properties of lung-nodule detectors or idiosyncratic to this model.
minor comments (1)
  1. [Abstract] The abstract reports approximate AUC values ('about 0.95', '0.94-0.98'); exact point estimates, confidence intervals, or sample sizes for the fingerprint experiments would improve interpretability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and for recognizing the potential relevance to ACR-SIIM and Assess-AI efforts. We respond point-by-point to the major comments below.

read point-by-point responses

  1. Referee: [Abstract / Results (perturbation experiments)] The dissociation claim—that the frequency/kernel axis selectively corrupts measurement (p=8.6e-13) while the noise axis selectively degrades detection (p=5.9e-32)—is shown exclusively on controlled synthetic perturbations of LIDC-IDRI; the real paired NLST data tests only the kernel effect. The manuscript provides no direct evidence that the perturbation method (synthetic kernel filtering plus noise injection) isolates the two axes as cleanly as real multi-vendor clinical acquisitions, where dose, collimation, and reconstruction interact. This assumption is load-bearing for the central claim of axis-specific failure modes.

    Authors: The paired NLST data supplies direct clinical evidence that kernel alone alters AI measurement and Fleischner category at fixed patient and scan parameters. The LIDC-IDRI perturbations were constructed to apply kernel filtering and noise injection independently, which is the only practical way to dissociate the axes when real acquisitions confound them. We agree that the perturbations do not capture every interaction present in heterogeneous multi-vendor clinical data. In revision we will explicitly qualify the dissociation results as obtained under controlled conditions and add a limitations paragraph calling for future validation on real matched multi-vendor acquisitions. revision: yes

  2. Referee: [Abstract / Methods (implied)] All quantitative results, including the 5.2% category-flip rate, p-values, and AUCs, are reported for a single MONAI RetinaNet model trained on LUNA16. No cross-architecture or cross-training-regime experiments are described, leaving open whether the observed kernel-driven measurement instability and noise-driven detection fragility are general properties of lung-nodule detectors or idiosyncratic to this model.

    Authors: We concur that every reported statistic derives from one detector architecture and training set. This choice was made to anchor the study in a fully reproducible public pipeline. We do not claim the axis-specific effects are universal. The revised manuscript will include an explicit limitations section stating that the observed failure modes require testing across additional detectors and training regimes before generality can be asserted. revision: yes

Circularity Check

0 steps flagged

No circularity; all claims rest on direct empirical measurements from external datasets

full rationale

The paper reports statistical tests (Wilcoxon p-values, AUCs) and category flips computed directly on real paired NLST scans and LIDC-IDRI perturbations, with the 4-feature fingerprint extracted from pixel intensities rather than defined in terms of the target labels. No equations appear that reduce a claimed result to a fitted parameter or self-referential definition; no self-citations are invoked as load-bearing uniqueness theorems; the dissociation between kernel and noise axes is presented as an observed pattern on the perturbed data, not derived by construction from the inputs. The derivation chain is therefore self-contained against the supplied external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on standard statistical assumptions for paired and unpaired tests plus the domain assumption that LIDC perturbations isolate the intended axes; no free parameters are fitted to produce the headline results and no new entities are postulated.

axioms (2)
  • standard math Wilcoxon signed-rank and rank-sum tests are appropriate for the paired and unpaired comparisons performed
    Invoked for all reported p-values on diameter, confidence, and detection outcomes
  • domain assumption LIDC-IDRI controlled perturbations accurately separate frequency content from additive noise in a manner representative of real scanner differences
    Used to dissociate the kernel axis from the noise axis while holding other factors fixed

pith-pipeline@v0.9.1-grok · 5887 in / 1518 out tokens · 26015 ms · 2026-06-27T05:54:12.586274+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

10 extracted references · 6 canonical work pages

  1. [1]

    ACR–SIIM Practice Parameter for Imaging Artificial Intelligence

    American College of Radiology, Society for Imaging Informatics in Medicine. ACR–SIIM Practice Parameter for Imaging Artificial Intelligence. Reston, VA: American College of Radiology; 2026 (Resolution 13, approved May 5, 2026)

    2026

  2. [2]

    ACR’s Assess-AI: a registry for real-world performance monitoring of clinical imaging artificial intelligence.J Am Coll Radiol

    Kim W, Cook T, Dreyer KJ, et al. ACR’s Assess-AI: a registry for real-world performance monitoring of clinical imaging artificial intelligence.J Am Coll Radiol. Published online April 29, 2026. doi:10.1016/j.jacr.2026.04.024

    work page doi:10.1016/j.jacr.2026.04.024 2026

  3. [3]

    Beyond Benchmarks: a framework for post-deployment validation of CT lung-nodule detection AI

    Soliman D. Beyond Benchmarks: a framework for post-deployment validation of CT lung-nodule detection AI. arXiv:2603.26785. 2026

    arXiv 2026

  4. [4]

    On instabilities of deep learning in image reconstruction and the potential costs of AI.Proc Natl Acad Sci USA

    Antun V, Renna F, Poon C, Adcock B, Hansen AC. On instabilities of deep learning in image reconstruction and the potential costs of AI.Proc Natl Acad Sci USA. 2020;117(48):30088–30095. doi:10.1073/pnas.1907377117

    work page doi:10.1073/pnas.1907377117 2020

  5. [5]

    The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI).Med Phys

    Armato SG 3rd, McLennan G, Bidaut L, et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI).Med Phys. 2011;38(2):915–931. doi:10.1118/1.3528204

    work page doi:10.1118/1.3528204 2011

  6. [6]

    Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules: the LUNA16 challenge.Med Image Anal

    Setio AAA, Traverso A, de Bel T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules: the LUNA16 challenge.Med Image Anal. 2017;42:1–

    2017

  7. [7]

    doi:10.1016/j.media.2017.06.015

    work page doi:10.1016/j.media.2017.06.015 2017

  8. [8]

    Re- duced lung-cancer mortality with low-dose computed tomographic screening.N Engl J Med

    National Lung Screening Trial Research Team; Aberle DR, Adams AM, Berg CD, et al. Re- duced lung-cancer mortality with low-dose computed tomographic screening.N Engl J Med. 2011;365(5):395–409. doi:10.1056/NEJMoa1102873

    work page doi:10.1056/nejmoa1102873 2011

  9. [9]

    MONAI: an open-source framework for deep learning in healthcare

    Cardoso MJ, Li W, Brown R, et al. MONAI: an open-source framework for deep learning in healthcare. arXiv:2211.02701. 2022. 8

    Pith/arXiv arXiv 2022

  10. [10]

    QIBA CT phantom dataset (FBP vs

    Quantitative Imaging Biomarkers Alliance (QIBA), RSNA. QIBA CT phantom dataset (FBP vs. iterative reconstruction). The Cancer Imaging Archive; 2026. doi:10.7937/TCIA.RMV0-9Y95. 9

    work page doi:10.7937/tcia.rmv0-9y95 2026