REVIEW 2 major objections 5 minor 33 references
LILI replaces LIME's flat-color occlusions with LaMa inpainting and mask expansion, yielding photorealistic perturbations and higher-saliency explanations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 23:13 UTC pith:MNX4RTRL
load-bearing objection Firm FID result, fragile explanation-quality claim: mask expansion is tuned on the same test set and no significance tests are reported. the 2 major comments →
Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
LILI modifies only the perturbation step of LIME. Instead of filling masked superpixels with a mean color, it feeds the original image and binary mask to LaMa, and it expands each superpixel mask by m pixels around its contour before inpainting. The expanded mask stops the inpainter from reconstructing the occluded object from border artifacts, so the object is actually removed rather than redrawn. The paper claims this yields photorealistic perturbations and, in its saliency-metric evaluation over 100 ImageNet images, the best mean score of -1.816 at m=3, outperforming LIME (-1.625) and LIME-G (best mean -1.744 at m=8; -1.699 at m=3). It also reports that LILI is less stable than LIME but m
What carries the argument
LaMa, a feed-forward inpainting network using fast Fourier convolutions, is the replacement engine: it produces plausible content conditioned on the whole image, so masked features are removed realistically rather than overwritten with a flat patch. The second mechanism is mask expansion: before inpainting, each superpixel mask is dilated by m pixels along its contour using border-following contour tracing, which hides quickshift's boundary artifacts and stops LaMa from resurrecting the occluded object.
Load-bearing premise
That the saliency metric, with min-over-alpha aggregation and the specific 100-image test set where m is tuned on the same images, faithfully reflects what users want from an explanation; if the metric is not a faithful proxy, the claimed superiority of LILI over LIME and LIME-G may vanish.
What would settle it
Re-run Table 2 on held-out images (or with cross-validation over m) and report saliency scores at each threshold alpha instead of only the minimum: if LILI no longer beats LIME-G and LIME, or the ordering reverses under average aggregation, the central claim fails. Also compute a faithfulness metric that removes the top salient superpixels and measures confidence drop, checking whether LILI's advantage persists.
If this is right
- If LILI is right, perturbation-based explanations can escape the out-of-distribution problem: masks remove features without creating artificial edges, so the surrogate model sees in-distribution inputs.
- Since the mask-expansion parameter m has an optimum (m=3 seems best for LILI), the paper implies that inpainting-based occlusion must be tuned jointly with the segmentation method; simply switching in a better inpainter is not enough.
- The FID numbers give a concrete, checkable ordering of perturbation generators: LaMa inpainting > superpixel mean color > DeepFill inpainting for this setup.
- The stability gap (LIME W=0.889, LILI 0.852, LIME-G 0.723) shows that realism and stability can move apart, so users should expect a trade-off rather than a strict improvement.
- Because LILI's runtime (~12.3s per explanation) is between classic LIME (~4.4s) and diffusion-based inpainting, the approach is usable where compute is limited.
Where Pith is reading between the lines
- The core claim suggests a general recipe: choose an inpainter that is fast enough for the many perturbations LIME needs, and add controlled mask dilation; the same recipe could transfer to other perturbation frameworks like RISE, not only LIME.
- If photorealism alone drives the saliency gain, then a cheaper realistic fill (for example, texture synthesis or a single fast generative pass) might capture much of the benefit; the paper does not isolate that factor.
- The min-over-alpha aggregation used for saliency scores is a notable choice; a reader could check whether the ordering survives per-threshold comparisons or average aggregation, since the minimum may reward explanations that are concentrated at one threshold.
- The band-aid example suggests LILI's practical value in diagnosing shortcut learning: when a spurious artifact is removed in a realistic way, the classifier's confidence drops, exposing features that flat-color occlusion masks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LILI, a modification of LIME for image explanations. Instead of replacing masked superpixels with a mean color (LIME) or with DeepFill inpainting (LIME-G), LILI uses the LaMa inpainting model and additionally expands the perturbation mask by m pixels. The authors report that LILI produces more photorealistic perturbed samples, with FID 6.727 versus 30.532 for LIME and 56.524 for LIME-G. They further report improved explanation quality on the saliency metric, with best mean score −1.816 at m=3 versus −1.625 for LIME and −1.699 for LIME-G at m=3, as well as intermediate explanation stability and runtime. The central claim is that generative inpainting with LaMa plus mask expansion makes LIME explanations more faithful.
Significance. If the explanation-quality claim holds, LILI is a simple and actionable recipe: retain LIME's sampling and surrogate fitting, but replace the occlusion step with a modern inpainter and expand masks to avoid residual-feature leakage. This is an incremental but useful contribution to perturbation-based XAI, and the FID realism result is quantitatively solid and clearly reported. The open-source code and reproducible experimental protocol are strengths. The main weakness is that the central explanation-quality claim rests on a single metric, a small 100-image evaluation, and a mask-expansion hyperparameter selected on the same test images, with no inferential statistics. The realism claim is well supported; the faithfulness claim is not yet established at the same standard.
major comments (2)
- [§4.2, Eq. (4)] The central comparison is vulnerable to selection bias. For LILI and LIME-G, Table 2 reports results for m ∈ {0,3,5,8}, and the text selects m=3 as the best setting for both methods. Because m is effectively chosen on the same 100 images used to report the scores, the best-of-four values overstate the methods' typical performance. The LILI advantage over LIME-G at m=3 is small: mean −1.816 vs −1.699, median −2.011 vs −1.864. Moreover, LIME-G at m=8 achieves mean −1.744 and median −1.944, close to LILI at m=3, so a different but equally defensible choice of m changes the ranking. The paper reports no confidence intervals, paired tests, or validation-split analysis. To support the claim that LILI improves explanation quality, the authors should either pre-specify m on a validation split, or report the comparison at a fixed m that was not selected on the test set, together with significance
- [§4.2, Eq. (4)] The saliency metric is aggregated over thresholds by taking the minimum over α: S*_{ν,m,i} = min_α S_{ν,m,i,α}. This aggregation is not the standard way to summarize threshold-dependent saliency metrics and should be justified. The min over α can reward an explanation that scores very well at a single threshold even if its overall concentration curve is poor, and it may inflate differences between methods. Since Table 2 and the explanation-quality conclusion depend entirely on this aggregation, the authors should report the full threshold curve or a threshold-averaged score and verify that the ranking of LILI versus LIME-G is preserved under a more standard aggregation.
minor comments (5)
- [§4.3, Table 3] The text says LILI achieves 'significantly higher explanation stability' than LIME-G, but no significance test is reported. With standard deviations of 0.109 for LILI and 0.184 for LIME-G and n=100, the difference in mean Kendall’s W (0.852 vs 0.723) may be significant, but the claim should be backed by a statistical test. Also, the stability experiment uses m=5, whereas the explanation-quality experiment selects m=3; the choice of m=5 for stability is not justified.
- [§2.4] The sentence 'no work has yet explored improving LIME-G with state-of-the-art inpainting models' is too strong given that related work [2] uses SAM plus Stable Diffusion for photorealistic inpainting-based perturbations. Please clarify the specific difference from prior image inpainting approaches in the context of LIME.
- [§2.2] The failure of the eight deterministic occlusion strategies is demonstrated only anecdotally on one 'band-aid' image. If this is meant as evidence that deterministic occlusion fails broadly, a small quantitative comparison would be more convincing; otherwise frame it as an illustrative motivating example.
- [§5] The Discussion correctly notes that LIME is sensitive to hyperparameters and that future work should test different combinations. This is an acknowledged limitation, but since the current evaluation uses only default LIME hyperparameters, the generalizability of the explanation-quality result remains untested. Consider adding a short Limitations paragraph rather than deferring this to future work.
- [Throughout] Minor language and typographical issues: 'it’s scalability' should be 'its scalability', 'explantions' in Fig. 4 should be 'explanations', 'inpaintedsample' missing space, and the abstract has an extra space before 'e.g.'. These do not affect the scientific content.
Circularity Check
No material circularity; the empirical comparison is self-contained and the central claims rest on external metrics and baselines.
full rationale
The paper's contribution is an algorithmic combination (LIME with LaMa inpainting and mask expansion) plus an empirical evaluation, not a derivation from first principles. The only self-citations are background: [10] supplies notation for LIME, [18] notes LIME's hyperparameter sensitivity, and [9] motivates segmentation artifacts; none of these establishes the central claim that LILI improves realism or explanation quality. The FID comparison (Eq. 2) uses a fixed real-image reference set and standard implementations, and the saliency metric (Eq. 3-4) is an external, pre-existing evaluation criterion. The reported saliency table lists all tested mask expansions (0, 3, 5, 8) rather than hiding a fitted quantity as a prediction; choosing m=3 as the best observed value is a statistical/benchmarking concern (no validation split or significance test), not circularity, because the scores are computed directly from the defined metric and are not constrained by construction to favor LILI. No equation reduces to its own input by definition, and no load-bearing step relies on a self-citation chain. Therefore the paper exhibits no significant circularity.
Axiom & Free-Parameter Ledger
free parameters (2)
- Mask expansion m =
3 (best saliency on test set; 5 used in stability; grid {0,3,5,8})
- Saliency threshold aggregation (min over alpha) =
S* = min_alpha S
axioms (6)
- domain assumption LaMa inpainting (pre-trained on Places/CelebA-HQ) generalizes to LIME superpixel masks at the ImageNet distribution.
- domain assumption Quickshift segmentation artifacts at object borders leak object identity and can be hidden by m-pixel mask expansion.
- domain assumption LIME's default hyperparameters (quickshift, n=1000 samples, ridge regression, kernel width) are a fair comparison basis.
- domain assumption The saliency metric of Dabkowski and Gal [7] is a valid proxy for explanation quality.
- domain assumption FID computed from InceptionV3 features is a valid measure of perturbation realism.
- domain assumption InceptionV3 on ImageNet-1k is a representative testbed for XAI methods.
read the original abstract
The increasing complexity of state-of-the-art machine learning models has made their behavior progressively harder to interpret, spurring rapid advancements in the field of eXplainable Artificial Intelligence (XAI). Among many methods proposed, perturbation-based approaches play a major role. By systematically altering (perturbing) input features, these approaches measure the impact on the model's predictions. For image data, traditional perturbation techniques, often involve replacing pixel values e.g., with a pre-defined color. However, such approaches, but also more refined deterministic techniques, generate unrealistic out-of-distribution samples and often leave visible artifacts, which can mislead the model and compromise explanation quality. In this work, we adjust LIME, a widely used perturbation-based method, to demonstrate how generative inpainting can improve perturbation-based explanations for images. We achieve photorealistic perturbed samples that align better with the original data distribution and enhance explanation quality.
Figures
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the ACCV (2020)
Agarwal, C., Nguyen, A.: Explaining image classifiers by removing input features using generative models. In: Proceedings of the ACCV (2020)
2020
-
[2]
In: NeurIPS 2025 Workshop for Imageomics: Discovering Biological Knowledge from Images Using AI (2025)
Aghakishiyeva, G., Zhou, J., Arya, S., Poling, J.D., Houliston, H.R., Womble, J.N., Johnston, D.W., Bent, B.: Photorealistic inpainting for perturbation-based explanations in ecological monitoring. In: NeurIPS 2025 Workshop for Imageomics: Discovering Biological Knowledge from Images Using AI (2025)
2025
-
[3]
Ang, D.: Daa233/generative-inpainting-pytorch: A pytorch reimplemen- tation for paper generative image inpainting with contextual atten- tion (February 2021), https://github.com/daa233/generative-inpainting- pytorch/commit/c6cdaea0427b37b5b38a3f48d4355abf9566c659, gitHub repository
2021
-
[4]
Sensors25(13) (2025)
Cheng, Z., Wu, Y., Li, Y., Cai, L., Ihnaini, B.: A comprehensive review of explain- able artificial intelligence (xai) in computer vision. Sensors25(13) (2025)
2025
-
[5]
Frontiers in Human Dynamics6 (2024)
Cheong, B.C.: Transparency and accountability in ai systems: safeguarding well- being in the age of algorithmic decision-making. Frontiers in Human Dynamics6 (2024)
2024
-
[6]
NeurIPS’20 (2020)
Chi, L., Jiang, B., Mu, Y.: Fast fourier convolution. NeurIPS’20 (2020)
2020
-
[7]
Dabkowski, P., Gal, Y.: Real time image saliency for black box classifiers. pp. 6970–6979. NeurIPS’17 (2017)
2017
-
[8]
Artifi- cial Intelligence Review58(4) (February 2025)
Elharrouss, O., Damseh, R., Belkacem, A.N., Badidi, E., Lakas, A.: Transformer- based image and video inpainting: current challenges and future directions. Artifi- cial Intelligence Review58(4) (February 2025)
2025
-
[9]
In: Camps-Valls, G., Ruiz, F.J.R., Valera, I
Garreau, D.: How to scale hyperparameters for quickshift image segmentation. In: Camps-Valls, G., Ruiz, F.J.R., Valera, I. (eds.) AISTATS 2022. Proceedings of Machine Learning Research, vol. 151, pp. 5243–5275 (28–30 Mar 2022)
2022
-
[10]
Garreau, D., Mardaoui, D.: What does LIME really see in images? In: ICML 2021 (2021)
2021
-
[11]
In: Advances in Neural Information Processing Systems
Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems. vol. 30 (2017)
2017
-
[12]
The Annals of Mathe- matical Statistics10(3), 275 – 287 (1939)
Kendall, M.G., Smith, B.B.: The Problem ofmRankings. The Annals of Mathe- matical Statistics10(3), 275 – 287 (1939)
1939
-
[13]
In: Proceedings of the CVPR
Kim, S., Suh, S., Lee, M.: RAD: Region-aware diffusion models for image inpaint- ing. In: Proceedings of the CVPR. pp. 2439–2448 (June 2025)
2025
-
[14]
In: Proceedings of the ICCV
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollar, P., Girshick, R.: Segment anything. In: Proceedings of the ICCV. pp. 4015–4026 (October 2023)
2023
-
[15]
ICML’23 (2023)
Kotelnikov, A., Baranchuk, D., Rubachev, I., Babenko, A.: TabDDPM: modelling tabular data with diffusion models. ICML’23 (2023)
2023
-
[16]
Nature Communications10(1), 1096 (Mar 2019) Inpainting Insights 15
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K.R.: Unmasking clever hans predictors and assessing what machines really learn. Nature Communications10(1), 1096 (Mar 2019) Inpainting Insights 15
2019
-
[17]
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. pp. 4768–4777. NIPS’17 (2017)
2017
-
[18]
In: Proceedings of The 24th AISTATS
Mardaoui, D., Garreau, D.: An analysis of LIME for text data. In: Proceedings of The 24th AISTATS. Proceedings of Machine Learning Research, vol. 130, pp. 3493–3501 (13–15 Apr 2021)
2021
-
[19]
(March 2024), https://github.com/mseitzer/pytorch- fid/commit/b9c18118d082cbd263c1b8963fc4221dc1cbb659
mseitzer: Mseitzer/pytorch-fid: Compute fid scores with py- torch. (March 2024), https://github.com/mseitzer/pytorch- fid/commit/b9c18118d082cbd263c1b8963fc4221dc1cbb659
2024
-
[20]
The Journal of Strategic Information Systems34(2), 101885 (2025)
Papagiannidis, E., Mikalef, P., Conboy, K.: Responsible artificial intelligence gov- ernance: A review and research framework. The Journal of Strategic Information Systems34(2), 101885 (2025)
2025
-
[21]
In: British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018
Petsiuk, V., Das, A., Saenko, K.: RISE: randomized input sampling for explanation of black-box models. In: British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018. p. 151 (2018)
2018
-
[22]
Applied Artificial Intelligence39(1), 2463722 (2025)
Radanliev, P.: AI ethics: Integrating transparency, fairness, and privacy in ai de- velopment. Applied Artificial Intelligence39(1), 2463722 (2025)
2025
-
[23]
In: Proceedings of the 22nd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016. pp. 1135–1144 (2016)
2016
-
[24]
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models (2021)
2021
-
[25]
In: 2022 WACV
Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., Lempitsky, V.: Resolution-robust large mask inpainting with fourier convolutions. In: 2022 WACV. pp. 3172–3182 (2022)
2022
-
[26]
Computer Vision, Graphics, and Image Processing30(1), 32–46 (1985)
Suzuki, S., Abe, K.: Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics, and Image Processing30(1), 32–46 (1985)
1985
-
[27]
In: 2016 CVPR
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the incep- tion architecture for computer vision. In: 2016 CVPR. pp. 2818–2826 (2016)
2016
-
[28]
In: Explainable Artificial Intelligence
Tritscher, J., Lissmann, P., Wolf, M., Krause, A., Hotho, A., Schlör, D.: Generative inpainting for shapley-value-based anomaly explanation. In: Explainable Artificial Intelligence. pp. 230–243 (2024)
2024
-
[29]
In: Com- puter Vision – ECCV 2008
Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Com- puter Vision – ECCV 2008. pp. 705–718 (2008)
2008
-
[30]
arXiv preprint arXiv:1806.03589 (2018)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution. arXiv preprint arXiv:1806.03589 (2018)
Pith/arXiv arXiv 2018
-
[31]
arXiv preprint arXiv:1801.07892 (2018)
Yu,J.,Lin,Z.,Yang,J.,Shen,X.,Lu,X.,Huang,T.S.:Generativeimageinpainting with contextual attention. arXiv preprint arXiv:1801.07892 (2018)
Pith/arXiv arXiv 2018
-
[32]
In: Computer Vision – ECCV 2014
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Computer Vision – ECCV 2014. pp. 818–833 (2014)
2014
-
[33]
In: 2018 CVPR
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: 2018 CVPR. pp. 586–595 (2018)
2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.