pith. machine review for the scientific record. sign in

arxiv: 2604.23047 · v1 · submitted 2026-04-24 · ⚛️ physics.med-ph

Recognition: unknown

Benchmarking Open-Source FDK Against Commercial and Iterative Reconstruction Methods for Preclinical Micro-CBCT

Falk L Wiegmann , Nancy L Ford
Authors on Pith no claims yet

Pith reviewed 2026-05-08 08:44 UTC · model grok-4.3

classification ⚛️ physics.med-ph
keywords micro-CTreconstructionFDKiterative reconstructiondetectability indexbenchmarkingpreclinical imagingspatial resolution
0
0 comments X

The pith

Task-based metrics show vendor micro-CT software underperforms on low-contrast detectability despite leading spatial resolution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper benchmarks an open-source FDK pipeline against vendor software and two iterative methods on identical preclinical micro-CT acquisitions using MTF, NPS, and NPW d'. Single-metric rankings prove misleading because the vendor reaches the highest resolution (MTF10 = 2.96 lp/mm) yet fails the Rose criterion (d' = 3) for 100 HU objects in half-scan mode. ASTRA SIRT delivers the best low-contrast detectability at five times the computation time of FDK, while TIGRE OS-SART at fifty times the cost adds no benefit and shows instability. The comparison matters because it clarifies when the high cost of iterative reconstruction is justified versus when fast analytic methods suffice, and it releases the FDK code as open-source.

Core claim

On identical half-scan acquisitions from a preclinical micro-CT scanner, the vendor reconstruction reaches the highest spatial resolution but fails the Rose criterion (d' = 3) for low-contrast 100 HU objects, whereas ASTRA SIRT iterative reconstruction at default settings delivers superior low-contrast detectability at five times the computation time of analytic FDK; TIGRE OS-SART at fifty times the cost provides no further improvement and is unstable across protocols. For high-contrast tasks all methods perform comparably.

What carries the argument

Task-based image quality assessment using modulation transfer function (MTF), noise power spectrum (NPS), and non-prewhitening detectability index (NPW d') applied to the same raw projection data across four reconstruction algorithms.

If this is right

  • For high-contrast imaging tasks, analytic FDK and all tested methods perform comparably.
  • ASTRA SIRT iterative reconstruction justifies its fivefold increase in computation time for low-contrast preclinical studies.
  • TIGRE OS-SART iterative reconstruction does not justify its fiftyfold cost increase and shows protocol instability.
  • Open-source FDK reconstruction offers a transparent, fast, and integrable baseline for preclinical micro-CT.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Task-specific metrics should replace single-metric comparisons when evaluating reconstruction algorithms for medical imaging.
  • Releasing the FDK code may accelerate development of hybrid analytic-iterative pipelines.
  • This benchmarking approach could be extended to other cone-beam CT modalities in clinical settings.

Load-bearing premise

Default parameter settings for ASTRA SIRT and TIGRE OS-SART constitute a fair and representative comparison without any task-specific tuning.

What would settle it

A re-analysis of the same half-scan projections showing that tuned parameters allow the vendor software or TIGRE OS-SART to reach d' greater than 3 for 100 HU objects while preserving or exceeding the reported MTF10 of 2.96 lp/mm would falsify the claim that ASTRA SIRT is superior for low-contrast tasks.

Figures

Figures reproduced from arXiv: 2604.23047 by Falk L Wiegmann, Nancy L Ford.

Figure 1
Figure 1. Figure 1: Image quality metrics for the half-scan ( view at source ↗
Figure 2
Figure 2. Figure 2: Mouse lung reconstructions displayed in a lung window (C view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of image quality metrics between the half-scan ( view at source ↗
Figure 4
Figure 4. Figure 4: Axial phantom slices from the half-scan (left column), full-scan (centre column), and their difference (right view at source ↗
read the original abstract

Preclinical micro-CT reconstruction involves large projection sizes and volumes that make iterative methods costly - 5x to 50x slower than analytic alternatives on modern GPUs. Whether this cost is justified depends on the imaging task, yet head-to-head comparisons using task-based metrics on identical preclinical data are lacking. We benchmark four reconstruction methods on identical acquisitions from an eXplore CT 120 micro-CT scanner (Trifoil Imaging, USA): an open-source Feldkamp-Davis-Kress (FDK) pipeline, the proprietary vendor software, and two iterative toolboxes at default settings - ASTRA SIRT and TIGRE OS-SART. Using the modulation transfer function (MTF), noise power spectrum (NPS), and non-prewhitening detectability index (NPW d'), we show that single-metric rankings are misleading: the vendor software achieves the highest spatial resolution ($\mathrm{MTF}_{10} = 2.96$ lp/mm) but fails to reach the Rose criterion ($d'=3$) for 100 HU contrast objects on a half-scan acquisition. ASTRA SIRT, at 5x the computation time of FDK, provides the best low-contrast detectability, while TIGRE OS-SART at 50x the cost offers no additional benefit and exhibits instability across scan protocols. For high-contrast tasks, all methods perform comparably. We release our FDK pipeline as open-source software, providing a fast, transparent, and integrable reconstruction tool for the preclinical micro-CT community.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The manuscript benchmarks an open-source Feldkamp-Davis-Kress (FDK) reconstruction pipeline against proprietary vendor software and two iterative methods (ASTRA SIRT and TIGRE OS-SART) on identical half-scan acquisitions from an eXplore CT 120 preclinical micro-CBCT scanner. Using standard metrics (MTF, NPS, NPW d'), it argues that single-metric rankings are misleading: the vendor achieves the highest spatial resolution (MTF10 = 2.96 lp/mm) but fails the Rose criterion (d' < 3) for 100 HU contrast objects, while ASTRA SIRT delivers the best low-contrast detectability at 5x the computation time of FDK and TIGRE OS-SART at 50x cost provides no benefit and shows instability. The work releases the FDK pipeline as open-source software.

Significance. If the empirical comparisons hold, the paper usefully demonstrates the value of task-based metrics (NPW d') over isolated resolution measures for preclinical micro-CT method selection, where iterative approaches carry high computational cost. The open-source FDK release and use of identical scanner acquisitions are concrete strengths that support reproducibility and community adoption for high-contrast tasks where all methods perform comparably.

major comments (2)
  1. [Methods] Methods: The reliance on 'default settings' for ASTRA SIRT and TIGRE OS-SART is central to the claim that ASTRA provides the best low-contrast detectability at moderate cost while TIGRE offers no benefit. Without listing the exact parameter values (e.g., number of iterations, relaxation factors) or a sensitivity check, it is unclear whether the reported rankings reflect representative performance or an artifact of untuned defaults.
  2. [Results] Results: The assertion that TIGRE OS-SART 'exhibits instability across scan protocols' is load-bearing for dismissing it at 50x cost, yet the text does not quantify instability (e.g., variance in d' or artifact metrics) or name the specific protocols (full-scan vs. half-scan angular ranges) used for this assessment.
minor comments (3)
  1. [Abstract] Abstract and Results: The Rose criterion threshold (d'=3) is invoked without a brief citation or reminder of its derivation for the specific object size and contrast levels tested.
  2. [Figures] Figure captions: Captions for MTF, NPS, and d' plots should explicitly state the contrast levels (e.g., 100 HU), object diameters, and whether half-scan or full-scan data are shown to allow immediate interpretation without cross-referencing the text.
  3. [Methods] The manuscript should clarify in Methods whether any projections were excluded from the identical-acquisition comparisons and what statistical test (if any) was applied to the d' differences.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive evaluation and the constructive comments, which help improve the clarity and reproducibility of our work. We address each major comment below.

read point-by-point responses

  1. Referee: [Methods] Methods: The reliance on 'default settings' for ASTRA SIRT and TIGRE OS-SART is central to the claim that ASTRA provides the best low-contrast detectability at moderate cost while TIGRE offers no benefit. Without listing the exact parameter values (e.g., number of iterations, relaxation factors) or a sensitivity check, it is unclear whether the reported rankings reflect representative performance or an artifact of untuned defaults.

    Authors: We agree that explicit documentation of the parameters is essential for reproducibility. The original manuscript states that defaults were used, but we will revise the Methods section to list the precise default values employed from each toolbox (e.g., iteration counts and relaxation factors as supplied by ASTRA and TIGRE). These defaults represent standard out-of-the-box usage, which aligns with the benchmarking goal of comparing typical user experience rather than optimized performance. A full sensitivity analysis lies outside the scope of the present study, but the reported rankings are consistent with the toolboxes' recommended settings in the literature. revision: yes

  2. Referee: [Results] Results: The assertion that TIGRE OS-SART 'exhibits instability across scan protocols' is load-bearing for dismissing it at 50x cost, yet the text does not quantify instability (e.g., variance in d' or artifact metrics) or name the specific protocols (full-scan vs. half-scan angular ranges) used for this assessment.

    Authors: We accept that additional quantification and protocol specification will strengthen the claim. In the revised manuscript we will explicitly name the protocols (half-scan acquisitions as the primary dataset, with full-scan data used for cross-checks) and provide more detail on the observed instability, including qualitative description of increased streak artifacts and variability in low-contrast regions for TIGRE relative to the other methods. While we did not compute formal variance statistics across repeats in the original analysis, the instability was consistently noted during visual and metric inspection of the reconstructions. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper performs an empirical benchmarking study comparing reconstruction algorithms on identical preclinical micro-CT acquisitions using standard, externally defined metrics (MTF, NPS, NPW d'). No derivation chain, fitted parameters, or equations are present; all results follow directly from applying these metrics to scanner data. No self-citations, ansatzes, or uniqueness claims are invoked to support any central result. The work is self-contained against external benchmarks and qualifies its assumptions explicitly.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an empirical benchmarking study with no mathematical derivations or postulated entities. The claims rest on experimental comparisons using standard metrics (MTF, NPS, d') applied to data from a specific scanner at default settings.

pith-pipeline@v0.9.0 · 5577 in / 1159 out tokens · 33007 ms · 2026-05-08T08:44:08.197212+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Resolution-Noise Characteristics of Common FDK Filter Kernels: A Practical Reference for Preclinical Cone-Beam Micro-CT

    physics.med-ph 2026-04 accept novelty 4.0

    Systematic evaluation of 16 FDK filter configurations on a GE eXplore CT 120 scanner produces MTF10 values from 0.93 to 2.35 lp/mm, integrated NPS from 75,670 to 13,259 HU², and Rose-criterion detectable diameters fro...

Reference graph

Works this paper leans on

15 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nancy Lee Ford, Xi Ren, Luca Egoriti, Nolan Esplen, Stephanie Radel, Brandon Humphries, Hui-Wen Koay, Thomas Planche, Cornelia Hoehr, Alexander Gottberg, and Magdalena Bazalova-Carter. Respiratory-gated micro-computed tomography imaging to measure radiation-induced lung injuries in mice following ultra-high dose-rate and conventional dose-rate radiation t...

    work page doi:10.1117/1.jmi.12.1.014002 2025

  2. [2]

    L. A. Feldkamp, L. C. Davis, and J. W. Kress. Practical cone-beam algorithm.Journal of the Optical Society of America A, 1(6):612–619, 1984. doi: 10.1364/JOSAA.1.000612

    work page doi:10.1364/josaa.1.000612 1984

  3. [3]

    Iterative methods for the three-dimensional reconstruction of an object from projections.Journal of Theoretical Biology, 36(1):105–117, 1972

    Peter Gilbert. Iterative methods for the three-dimensional reconstruction of an object from projections.Journal of Theoretical Biology, 36(1):105–117, 1972. doi: 10.1016/0022-5193(72)90180-4

    work page doi:10.1016/0022-5193(72)90180-4 1972

  4. [4]

    Joost Batenburg, and Jan Sijbers

    Wim van Aarle, Willem Jan Palenstijn, Jeroen Cant, Eline Janssens, Folkert Bleichrodt, Andrei Dabravolski, Jan De Beenhouwer, K. Joost Batenburg, and Jan Sijbers. Fast and flexible X-ray tomography using the ASTRA toolbox.Optics Express, 24(22):25129–25147, 2016. doi: 10.1364/OE.24.025129

    work page doi:10.1364/oe.24.025129 2016

  5. [5]

    A. H. Andersen and A. C. Kak. Simultaneous algebraic reconstruction technique (SART): A superior implemen- tation of the ART algorithm.Ultrasonic Imaging, 6(1):81–94, 1984. doi: 10.1177/016173468400600107. 8 FDK vs Iterative Reconstruction for Micro-CBCTA PREPRINT

    work page doi:10.1177/016173468400600107 1984

  6. [6]

    TIGRE: a MATLAB-GPU toolbox for CBCT image reconstruction.Biomedical Physics & Engineering Express, 2(5):055010, 2016

    Ander Biguri, Manjit Dosanjh, Steven Hancock, and Manuchehr Soleimani. TIGRE: a MATLAB-GPU toolbox for CBCT image reconstruction.Biomedical Physics & Engineering Express, 2(5):055010, 2016. doi: 10.1088/ 2057-1976/2/5/055010

    2016

  7. [7]

    Boedeker, Samuel Brady, Jia Fan, Shuai Leng, Kyle J

    Ehsan Samei, David Bakalyar, Kirsten L. Boedeker, Samuel Brady, Jia Fan, Shuai Leng, Kyle J. Myers, Lu- cretiu M. Popescu, Juan Carlos Ramirez Giraldo, Frank Ranallo, Justin Solomon, Jay Vaishnav, and Jia Wang. Performance evaluation of computed tomography systems: Summary of AAPM Task Group 233.Medical Physics, 46(11):e735–e756, 2019. doi: 10.1002/mp.13763

    work page doi:10.1002/mp.13763 2019

  8. [8]

    Sparse-view cone beam CT reconstruction using dual CNNs in projection domain and image domain.Neurocomputing, 493:536–547,

    Lianying Chao, Zhiwei Wang, Haobo Zhang, Wenting Xu, Peng Zhang, and Qiang Li. Sparse-view cone beam CT reconstruction using dual CNNs in projection domain and image domain.Neurocomputing, 493:536–547,

  9. [9]

    doi: 10.1016/j.neucom.2021.12.096

    work page doi:10.1016/j.neucom.2021.12.096 2021

  10. [10]

    Radiation dose and image-quality assessment in computed tomography.Journal of the ICRU, 12(1), 2012

    International Commission on Radiation Units and Measurements. Radiation dose and image-quality assessment in computed tomography.Journal of the ICRU, 12(1), 2012. doi: 10.1093/jicru/ndt007

    work page doi:10.1093/jicru/ndt007 2012

  11. [11]

    Dennis L. Parker. Optimal short scan convolution reconstruction for fanbeam CT.Medical Physics, 9(2):254– 257, 1982. doi: 10.1118/1.595078

    work page doi:10.1118/1.595078 1982

  12. [12]

    Photography — electronic still picture imaging — resolution and spatial frequency responses, 2017

    International Organization for Standardization. Photography — electronic still picture imaging — resolution and spatial frequency responses, 2017

    2017

  13. [13]

    Using in vivo respiratory-gated micro-computed tomography imaging to monitor pulmonary side effects in 10 MV FLASH and conventional radiotherapy

    Xi Ren, Luca Egoriti, Nolan Esplen, Stephanie Radel, Brandon Humphries, Hui-Wen Koay, Thomas Planche, Cornelia Hoehr, Alexander Gottberg, Magdalena Bazalova-Carter, and Nancy Lee Ford. Using in vivo respiratory-gated micro-computed tomography imaging to monitor pulmonary side effects in 10 MV FLASH and conventional radiotherapy. InMedical Imaging 2023: Bi...

    work page doi:10.1117/12.2654427 2023

  14. [14]

    Avey, Monya Baker, William J

    Nathalie Percie du Sert, Viki Hurst, Amrita Ahluwalia, Sabina Alam, Marc T. Avey, Monya Baker, William J. Browne, Alejandra Clark, Innes C. Cuthill, Ulrich Dirnagl, Michael Emerson, Paul Garner, Stephen T. Holgate, David W. Howells, Natasha A. Karp, Stanley E. Lazic, Katie Lidster, Catriona J. MacCallum, Malcolm Macleod, Esther J. Pearl, Ole H. Petersen, ...

    work page doi:10.1371/journal.pbio.3000410 2020

  15. [15]

    Plenum Press, New York, 1973

    Albert Rose.Vision: Human and Electronic. Plenum Press, New York, 1973. doi: 10.1007/978-1-4684-2037-1. Acknowledgments This work was supported by the BC Lung Foundation. Author Contributions Falk L Wiegmann and Nancy L Ford contributed to the research direction and conceptualisation. Falk L Wiegmann de- veloped the FDK pipeline, performed the benchmarkin...

    work page doi:10.1007/978-1-4684-2037-1 1973