arxiv: 2604.11775 · v1 · submitted 2026-04-13 · 💻 cs.CV · cs.AI

Recognition: unknown

Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation

Ricardo Coimbra Brioso , Giulio Sichili , Damiano Dei , Nicola Lambri , Pietro Mancosu , Marta Scorsetti , Daniele Loiacono

Authors on Pith no claims yet

Pith reviewed 2026-05-10 15:41 UTC · model grok-4.3

classification 💻 cs.CV cs.AI

keywords KernelSHAPexplainable AI3D medical image segmentationpatch-based modelsvolumetric CTsupervoxelsattribution methodsnnU-Net

0 comments

The pith

Restricting KernelSHAP to a region of interest plus receptive field and caching unchanged patch predictions makes explanations feasible for patch-based 3D CT segmentation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops an efficient KernelSHAP framework for explaining predictions from patch-based 3D medical image segmentation models. It limits all perturbation evaluations to a user-chosen region of interest and the image patches that can influence it, while reusing cached baseline logits for any patches left unchanged by a given perturbation. The method keeps the original nnU-Net fusion of overlapping patch outputs intact. Three different ways of grouping voxels into features are compared inside that crop, and several value functions are tested to emphasize either correct segmentation evidence or false-positive drivers. A reader would care because these models are widely used in radiology yet remain hard to inspect, and practical explanations could help users spot when a model is relying on the right or wrong image cues.

Core claim

The central claim is that KernelSHAP attributions for volumetric CT segmentation can be obtained efficiently by restricting all coalition evaluations to a user-defined region of interest and its receptive-field support, accelerating repeated inference through patch logit caching that reuses baseline predictions for unaffected patches, and preserving the nnU-Net fusion scheme. Within the receptive-field crop, three automatically generated feature abstractions—whole-organ units, regular FCC supervoxels, and hybrid organ-aware supervoxels—are compared, together with aggregation functions aimed at true-positive stabilization or false-positive behavior. Experiments on whole-body CT data show 15-

What carries the argument

Patch logit caching inside an ROI-plus-receptive-field crop that reuses baseline predictions for unaffected patches while preserving nnU-Net patch fusion.

If this is right

Computation drops by 15% to 30% because baseline predictions for unaffected patches are reused.
Regular supervoxels tend to score highest on perturbation-based faithfulness metrics.
Organ-aware supervoxels produce explanations that align better with anatomy and are stronger at surfacing false-positive drivers under normalized metrics.
The same caching and restriction steps work with any aggregation function that targets true-positive evidence or false-positive behavior.
The nnU-Net fusion scheme remains unchanged, so the explanations stay compatible with existing segmentation pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same caching pattern could be applied to other perturbation-based explainers that require many forward passes through patch-based models.
Clinicians might use organ-aware units when the goal is to communicate a model's reasoning to non-technical colleagues.
The observed faithfulness-interpretability trade-off suggests selecting the feature abstraction according to the downstream task rather than using a single default.
Scaling the method to larger volumes or real-time clinical workflows would depend on how much the receptive-field size grows with model depth.

Load-bearing premise

That attributions computed only inside the chosen region of interest and receptive field with the selected feature groupings still faithfully reflect the full model's behavior on the original image.

What would settle it

A side-by-side run on the same inputs where full-image KernelSHAP produces feature importance values or rankings that differ substantially from those produced by the restricted ROI-plus-caching version.

Figures

Figures reproduced from arXiv: 2604.11775 by Damiano Dei, Daniele Loiacono, Giulio Sichili, Marta Scorsetti, Nicola Lambri, Pietro Mancosu, Ricardo Coimbra Brioso.

**Figure 2.** Figure 2: Qualitative attribution maps for Full Organs (volume 7), comparing aggregation functions within the ROI. of destabilizing and stabilizing effects. Overall, explanations appear noisier, which is expected from an organ-agnostic tessellation and the coarser effective resolution. Hybrid (Organ-Aware FCC) supervoxels ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: Qualitative attribution maps for Regular (FCC) supervoxels (volume 7). Across TP, Dice, and Soft Dice aggregations, Regular supervoxels achieve the highest raw and normalized ABPC/AOPC values. These results are mainly due to two factors. First, the FCC tessellation is organ-agnostic and typically spans a larger overall spatial support than organconstrained partitions, so that successive perturbations rem… view at source ↗

**Figure 5.** Figure 5: MoRF and LeRF curves (median ± IQR) for Full Organs [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

**Figure 6.** Figure 6: MoRF and LeRF curves (median ± IQR) for Regular (FCC) supervoxels. C. Computational Performance with Caching Patch caching (Section III-E) substantially reduces redundant computation during coalition evaluation. Averaged over the eight validation cases, Full Organs achieves an average cache hit ratio of 32.4% ± 6.3% with an inference time of 3.58s ± 0.47s per sample, yielding a total runtime of 1h 01m 45s… view at source ↗

read the original abstract

Perturbation-based explainability methods such as KernelSHAP provide model-agnostic attributions but are typically impractical for patch-based 3D medical image segmentation due to the large number of coalition evaluations and the high cost of sliding-window inference. We present an efficient KernelSHAP framework for volumetric CT segmentation that restricts computation to a user-defined region of interest and its receptive-field support, and accelerates inference via patch logit caching, reusing baseline predictions for unaffected patches while preserving nnU-Net's fusion scheme. To enable clinically meaningful attributions, we compare three automatically generated feature abstractions within the receptive-field crop: whole-organ units, regular FCC supervoxels, and hybrid organ-aware supervoxels, and we study multiple aggregation/value functions targeting stabilizing evidence (TP/Dice/Soft Dice) or false-positive behavior. Experiments on whole-body CT segmentations show that caching substantially reduces redundant computation (with computational savings ranging from 15% to 30%) and that faithfulness and interpretability exhibit clear trade-offs: regular supervoxels often maximize perturbation-based metrics but lack anatomical alignment, whereas organ-aware units yield more clinically interpretable explanations and are particularly effective for highlighting false-positive drivers under normalized metrics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper makes KernelSHAP practical for 3D medical segmentation via ROI restriction and patch caching but does not verify that the results match standard KernelSHAP.

read the letter

The punchline here is that the authors make KernelSHAP feasible for patch-based 3D medical segmentation by restricting to an ROI and caching unaffected patch logits, claiming 15-30% compute savings, while also testing different feature abstractions for clinical relevance. What is new is the tailored application to nnU-Net style models on volumetric CT, with the hybrid organ-aware supervoxels as one option. They do a good job showing how regular supervoxels can maximize standard faithfulness metrics but organ-aware groupings produce attributions that better match anatomy and highlight false positive drivers. The caching approach that reuses baseline predictions without breaking the fusion scheme is a practical detail worth noting. The soft spots are around verification and reporting. There is no reported test that the restricted ROI plus caching actually yields the same attribution values as running KernelSHAP on the full volume, which matters because any difference from how overlaps are handled would mean these are not exact KernelSHAP explanations. The abstract mentions clear trade-offs but provides no error bars, dataset sizes, or statistical significance, so it is difficult to gauge how stable the findings are. Post-hoc choice of metrics could also play a role. This paper is for medical imaging researchers who want to add perturbation-based explanations to their segmentation pipelines. Readers dealing with high-stakes CT analysis will find the optimizations and the abstraction comparisons directly applicable. It deserves a serious referee because the problem it tackles is real and the proposed fixes are grounded in the model's architecture. The work is not transformative but it is honest engineering that could help others. I would recommend sending it to peer review, asking the authors to include a direct equivalence check against full KernelSHAP and to expand the experimental reporting with more rigorous statistics.

Referee Report

2 major / 2 minor

Summary. The paper claims to introduce an efficient KernelSHAP framework for patch-based 3D CT segmentation explanations. It restricts coalition sampling to a user-defined ROI plus receptive-field support, accelerates inference by caching patch logits and reusing baseline predictions for unaffected patches (while preserving nnU-Net fusion), and compares three feature abstractions (whole-organ units, regular FCC supervoxels, hybrid organ-aware supervoxels) under multiple aggregation/value functions focused on TP/Dice stabilization or false-positive behavior. Experiments on whole-body CT data report 15-30% computational savings and clear faithfulness-interpretability trade-offs, with regular supervoxels maximizing perturbation metrics and organ-aware units providing better clinical alignment.

Significance. If the optimizations preserve exact KernelSHAP attributions, the work would make perturbation-based explanations practical for large volumetric medical images, addressing a key barrier to clinical adoption of segmentation models. The empirical comparison of abstractions offers actionable guidance on trading off anatomical interpretability against metric faithfulness, which could inform explanation design in radiology.

major comments (2)

[Abstract / Proposed Method] The central efficiency claim (Abstract) rests on the assumption that ROI restriction plus patch logit caching produces numerically identical attributions to full-image KernelSHAP. No equivalence verification, error quantification, or ablation against naive full-image computation is reported, particularly for overlap handling in nnU-Net fusion or baseline definitions outside the crop. This is load-bearing because the paper positions the outputs as faithful KernelSHAP explanations rather than an unquantified approximation.
[Experiments] The reported 15-30% savings and faithfulness/interpretability trade-offs (Abstract) are presented without dataset sizes, number of volumes or cases, error bars on metrics, or statistical tests. Post-hoc choice of abstractions and value functions (TP/Dice vs. normalized FP) risks selection bias, undermining the strength of the empirical conclusions.

minor comments (2)

[Methods] Clarify how the receptive-field support is exactly computed and whether any boundary effects from the crop could alter coalition semantics.
[Methods] The abstract mentions 'automatically generated feature abstractions' but provides no algorithmic details or pseudocode for generating the hybrid organ-aware supervoxels.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the opportunity to respond to the referee's report. We address the major comments point by point below. We agree with several points raised and will make revisions to strengthen the manuscript accordingly.

read point-by-point responses

Referee: [Abstract / Proposed Method] The central efficiency claim (Abstract) rests on the assumption that ROI restriction plus patch logit caching produces numerically identical attributions to full-image KernelSHAP. No equivalence verification, error quantification, or ablation against naive full-image computation is reported, particularly for overlap handling in nnU-Net fusion or baseline definitions outside the crop. This is load-bearing because the paper positions the outputs as faithful KernelSHAP explanations rather than an unquantified approximation.

Authors: We appreciate this observation. The proposed optimizations are intended to yield numerically identical attributions to a full-image KernelSHAP computation by limiting the feature space to the ROI and its receptive-field support, where patches outside this support do not contribute to the predictions inside the crop. The patch logit caching reuses exact baseline predictions for coalitions that leave certain patches unaffected, without altering the nnU-Net fusion scheme. However, we did not provide an explicit equivalence check or error analysis in the submitted manuscript. In the revised version, we will include an ablation study on a subset of volumes comparing the attributions obtained with the optimized method to those from a naive full-image implementation, quantifying any discrepancies and addressing overlap handling and baseline definitions. revision: yes
Referee: [Experiments] The reported 15-30% savings and faithfulness/interpretability trade-offs (Abstract) are presented without dataset sizes, number of volumes or cases, error bars on metrics, or statistical tests. Post-hoc choice of abstractions and value functions (TP/Dice vs. normalized FP) risks selection bias, undermining the strength of the empirical conclusions.

Authors: We acknowledge the need for more complete experimental reporting. The full manuscript describes the dataset as whole-body CT volumes, but we will explicitly state the number of volumes/cases, dataset source, and split details in the abstract and experimental section. We will add error bars representing standard deviation across cases and include statistical significance tests (such as Wilcoxon signed-rank tests) for the reported savings and metric differences. Regarding the choice of abstractions and value functions, these were motivated by clinical relevance (e.g., organ units for interpretability) and standard practices in segmentation explainability literature; we will expand the methods section to justify the selection a priori and report results for all combinations to avoid any appearance of post-hoc bias. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper presents an engineering optimization for efficient KernelSHAP on patch-based 3D CT segmentation, using ROI restriction, receptive-field support, and patch logit caching while preserving nnU-Net fusion. No equations, first-principles derivations, or predictions are claimed that reduce to fitted inputs or self-referential definitions by construction. Claims of computational savings (15-30%) and faithfulness trade-offs are supported by direct experiments on feature abstractions (organ units, supervoxels) rather than tautological reductions. The work relies on external nnU-Net without load-bearing self-citations or ansatz smuggling, rendering the chain self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; the work relies on standard assumptions in machine learning explainability and medical image processing.

pith-pipeline@v0.9.0 · 5524 in / 1277 out tokens · 44952 ms · 2026-05-10T15:41:56.284074+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

27 extracted references · 12 canonical work pages

[1]

Explainable AI (XAI) in image segmentation in medicine, industry, and beyond: A survey,

R. Gipi ˇskis, C.-W. Tsai, and O. Kurasova, “Explainable AI (XAI) in image segmentation in medicine, industry, and beyond: A survey,”ICT Express, vol. 10, pp. 1331–1354, Dec. 2024

2024
[2]

Towards Interpretable Se- mantic Segmentation via Gradient-weighted Class Activation Mapping,

K. Vinogradova, A. Dibrov, and G. Myers, “Towards Interpretable Se- mantic Segmentation via Gradient-weighted Class Activation Mapping,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 13943–13944, Apr. 2020. arXiv:2002.11434 [cs]

work page arXiv 2020
[3]

A Guided Tour of Post- hoc XAI Techniques in Image Segmentation,

S. N. Hasany, F. M ´eriaudeau, and C. Petitjean, “A Guided Tour of Post- hoc XAI Techniques in Image Segmentation,” inExplainable Artificial Intelligence(L. Longo, S. Lapuschkin, and C. Seifert, eds.), (Cham), pp. 155–177, Springer Nature Switzerland, 2024

2024
[4]

FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision,

R. S. R. Silva and J. J. Bird, “FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision,” Apr. 2024. arXiv:2312.05975 [cs]

work page arXiv 2024
[5]

nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,

F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,”Nature Methods, vol. 18, pp. 203–211, Feb. 2021. Publisher: Nature Publishing Group

2021
[6]

A Unified Approach to Interpreting Model Predictions

S. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” Nov. 2017. arXiv:1705.07874 [cs]

work page Pith review arXiv 2017
[7]

Sensitivity analysis for interpretation of machine learning based segmentation models in cardiac MRI,

M. J. Ankenbrand, L. Shainberg, M. Hock, D. Lohr, and L. M. Schreiber, “Sensitivity analysis for interpretation of machine learning based segmentation models in cardiac MRI,”BMC Medical Imaging, vol. 21, p. 27, Feb. 2021

2021
[8]

Peering into the Heart: A Comprehensive Exploration of Semantic Segmentation and Explainable AI on the MnMs-2 Cardiac MRI Dataset,

M. Ayoob, O. Nettasinghe, V . Sylvester, H. Bowala, and H. Mohideen, “Peering into the Heart: A Comprehensive Exploration of Semantic Segmentation and Explainable AI on the MnMs-2 Cardiac MRI Dataset,” Applied Computer Systems, vol. 30, pp. 12–20, Jan. 2025

2025
[9]

``why should i trust you?": Explaining the predictions of any classifier

M. T. Ribeiro, S. Singh, and C. Guestrin, “”Why Should I Trust You?”: Explaining the Predictions of Any Classifier,” Aug. 2016. arXiv:1602.04938 [cs]

work page arXiv 2016
[10]

Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models,

P. Knab, S. Marton, and C. Bartelt, “Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models,” Feb
[11]

arXiv:2403.07733 [cs]

work page arXiv
[12]

SLICE: Stabilized LIME for Consistent Explanations for Image Clas- sification,

R. P. Bora, P. Terhorst, R. Veldhuis, R. Ramachandra, and K. Raja, “SLICE: Stabilized LIME for Consistent Explanations for Image Clas- sification,” in2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10988–10996, June 2024. ISSN: 2575- 7075

2024
[13]

Grid Saliency for Context Explanations of Semantic Segmentation,

L. Hoyer, M. Munoz, P. Katiyar, A. Khoreva, and V . Fischer, “Grid Saliency for Context Explanations of Semantic Segmentation,” Nov
[14]

arXiv:1907.13054 [cs]

work page arXiv 1907
[15]

U-Noise: Learnable Noise Masks for Interpretable Image Segmentation,

T. Koker, F. Mireshghallah, T. Titcombe, and G. Kaissis, “U-Noise: Learnable Noise Masks for Interpretable Image Segmentation,” in2021 IEEE International Conference on Image Processing (ICIP), pp. 394– 398, Sept. 2021. arXiv:2101.05791 [cs]

work page arXiv 2021
[16]

Generating Smooth Inter- pretability Map for Explainable Image Segmentation,

T. Okamoto, C. Gu, J. Yu, and C. Zhang, “Generating Smooth Inter- pretability Map for Explainable Image Segmentation,” in2023 IEEE 12th Global Conference on Consumer Electronics (GCCE), (Nara, Japan), pp. 1023–1025, IEEE, Oct. 2023

2023
[17]

MiSuRe is all you need to explain your image segmentation,

S. N. Hasany, F. M ´eriaudeau, and C. Petitjean, “MiSuRe is all you need to explain your image segmentation,” June 2024. arXiv:2406.12173 [cs]

work page arXiv 2024
[18]

RISE: Randomized Input Sampling for Explanation of Black-box Models

V . Petsiuk, A. Das, and K. Saenko, “RISE: Randomized Input Sampling for Explanation of Black-box Models,” Sept. 2018. arXiv:1806.07421 [cs]

work page Pith review arXiv 2018
[19]

Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models,

M. Chrabaszcz, H. Baniecki, P. Komorowski, S. Płotka, and P. Biecek, “Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models,” Nov. 2024. arXiv:2407.16653 [cs]

work page arXiv 2024
[20]

Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images,

J. Wasserthal, H.-C. Breit, M. T. Meyer, M. Pradella, D. Hinck, A. W. Sauter, T. Heye, D. T. Boll, J. Cyriac, S. Yang, M. Bach, and M. Segeroth, “Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images,”Radiology: Artificial Intelligence, vol. 5, no. 5, p. e230024, 2023

2023
[21]

Explainability of Image Semantic Segmentation Through SHAP Val- ues,

P. Dardouillet, A. Benoit, E. Amri, P. Bolon, D. Dubucq, and A. Cr ´edoz, “Explainability of Image Semantic Segmentation Through SHAP Val- ues,” inICPR-XAIE -26TH International Conference on Pattern Recog- nition 2-nd Workshop on Explainable and Ethical AI, (Montreal, Canada), Aug. 2022

2022
[22]

Investigating gender bias in lymph-node segmentation with anatomical priors,

R. C. Brioso, D. Dei, N. Lambri, P. Mancosu, M. Scorsetti, and D. Loiacono, “Investigating gender bias in lymph-node segmentation with anatomical priors,” inEthics and Fairness in Medical Imaging (E. Puyol-Ant ´on, G. Zamzmi, A. Feragen, A. P. King, V . Cheplygina, M. Ganz-Benjaminsen, E. Ferrante, B. Glocker, E. Petersen, J. S. H. Baxter, I. Rekik, and R...

2025
[23]

Evaluating the visualization of what a deep neural network has learned,

W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. M ¨uller, “Evaluating the visualization of what a deep neural network has learned,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 11, pp. 2660–2673, 2017

2017
[24]

Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability,

J. Edin, A. G. Motzfeldt, C. L. Christensen, T. Ruotsalo, L. Maaløe, and M. Maistro, “Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability,” May 2025. arXiv:2408.08137 [cs]

work page arXiv 2025
[25]

Slic superpixels compared to state-of-the-art superpixel methods,

R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S ¨usstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274–2282, 2012

2012
[26]

SEEDS: Superpixels Extracted via Energy-Driven Sampling,

M. V . d. Bergh, X. Boix, G. Roig, and L. V . Gool, “SEEDS: Superpixels Extracted via Energy-Driven Sampling,” Sept. 2013. arXiv:1309.3848 [cs]

work page arXiv 2013
[27]

Extending SEEDS to a Supervoxel Algorithm for Medical Image Analysis,

C. Zhao, Y . Jiang, and T. C. Hollon, “Extending SEEDS to a Supervoxel Algorithm for Medical Image Analysis,” Feb. 2025

2025