{"id":"561538e0-aebf-48c8-be4a-9e68eb09e116","arxiv_id":"2607.15482","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"LILI uses LaMa inpainting and mask expansion to make LIME's perturbations photorealistic, improving FID and saliency scores on ImageNet explanations.","lead":"LILI is a version of the LIME explanation method that occludes image regions with the LaMa generative inpainter and expands the occlusion mask to hide segmentation artifacts. On a 100-image ImageNet test, LILI produces much more realistic perturbations (FID 6.7 vs 30–56) and a small gain on a saliency-based quality score, but with lower stability than LIME.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mask-expansion hyperparameter m is selected on the same 100 test images used for the reported saliency comparison; the claimed LILI superiority could be post-hoc selection and should be re-evaluated on a held-out split.","rationale":"The reader's verdict is CONDITIONAL and flags the saliency metric and test-set selection. I agree with the core concern: the paper as written reports results on the same 100 test images used implicitly to choose m, and no validation split is described. This is the single most load-bearing issue because the paper's novelty over LIME-G is exactly the mask-expansion hyperparameter, and the reported gains are small. The FID result (LILI 6.727 vs 30.532 LIME, 56.524 LIME-G) is strong and expected — LaMa is a more modern inpainting model, and the relative FID ordering is plausible. The explanation-quality claim, however, is the main contribution and is fragile. The paper's own stability results (Table 3) show LIME is more stable (0.889 vs 0.852 vs 0.723), which the authors attribute to inpaint variability; this does not invalidate LILI but reinforces that the comparison should be statistically robust. My concrete check — validating m on a split and reporting held-out scores — would settle whether the superiority is real. I do not see an internal inconsistency in the method or a correctness flaw in the FID computation. The concern is about the evaluation protocol, not the method's soundness, so a CONDITIONAL verdict is appropriate rather than REJECT. An independent reproduction with the described validation protocol could move it to ACCEPT; failure would downgrade it. The agreement_with_reader is 'agree' because the reader's weakest_assumption explicitly names the same test-set issue and metric concern.","tokens_in":9944,"tokens_out":1903,"duration_ms":17053,"concrete_test":"Split the 100 images into a validation set (e.g., 50) and a test set (50), or use cross-validation. For each method (LIME, LIME-G, LILI), select the mask expansion m∈{0,3,5,8} that minimizes the average saliency score (Eq. 3 aggregated as in Eq. 4) on the validation fold, then report the score on the held-out fold using that fixed m. If LILI still beats LIME-G and LIME on the held-out images under this protocol (with bootstrapped CIs or a paired test), the central claim survives; if the ordering flips or becomes indistinguishable, the paper's explanation-quality claim is not supported. This test can be run from the released code and the 100-image evaluation setup.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that LILI improves explanation quality (Section 4.2, Table 2) rests on the saliency metric and on choosing m∈{0,3,5,8}. The paper reports LILI m=3 as best (mean -1.816) and LIME-G m=3 as best (-1.699), and it explicitly states (Section 4.2) that these are the values used in the comparison. If m is tuned on the same 100 test images that produce the reported scores, the comparison is vulnerable to selection bias. With only 100 images and four candidate values, the expected gain from picking the best m is substantial; the reported LILI advantage over LIME-G is small (|Δmean| ≈ 0.117, |Δmedian| ≈ 0.147) relative to per-image variability. The paper provides no significance test, no confidence interval, and no validation-split analysis. The strongest defense would be a pre-registered or validation-based selection of m, followed by an independent test evaluation. Because the best m differs across rows (LILI m=3, LIME-G m=3 vs LIME-G m=8 mean -1.744; LIME-G m=8 median -1.944 close to LILI m=3 -2.011), the observed ordering is fragile: a different but equally defensible choice of m can change the ranking. Thus the load-bearing condition — that the m-selection does not inflate the comparison — is not checked. Separately, the saliency metric itself (min-over-α in Eq. 4) is a secondary concern, but the m-selection issue is more concrete and directly addressable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LILI, a modification of LIME for image explanations. Instead of replacing masked superpixels with a mean color (LIME) or with DeepFill inpainting (LIME-G), LILI uses the LaMa inpainting model and additionally expands the perturbation mask by m pixels. The authors report that LILI produces more photorealistic perturbed samples, with FID 6.727 versus 30.532 for LIME and 56.524 for LIME-G. They further report improved explanation quality on the saliency metric, with best mean score −1.816 at m=3 versus −1.625 for LIME and −1.699 for LIME-G at m=3, as well as intermediate explanation stability and runtime. The central claim is that generative inpainting with LaMa plus mask expansion makes LIME explanations more faithful.","tokens_in":10346,"tokens_out":5523,"duration_ms":59465,"significance":"If the explanation-quality claim holds, LILI is a simple and actionable recipe: retain LIME's sampling and surrogate fitting, but replace the occlusion step with a modern inpainter and expand masks to avoid residual-feature leakage. This is an incremental but useful contribution to perturbation-based XAI, and the FID realism result is quantitatively solid and clearly reported. The open-source code and reproducible experimental protocol are strengths. The main weakness is that the central explanation-quality claim rests on a single metric, a small 100-image evaluation, and a mask-expansion hyperparameter selected on the same test images, with no inferential statistics. The realism claim is well supported; the faithfulness claim is not yet established at the same standard.","major_comments":[{"comment":"The central comparison is vulnerable to selection bias. For LILI and LIME-G, Table 2 reports results for m ∈ {0,3,5,8}, and the text selects m=3 as the best setting for both methods. Because m is effectively chosen on the same 100 images used to report the scores, the best-of-four values overstate the methods' typical performance. The LILI advantage over LIME-G at m=3 is small: mean −1.816 vs −1.699, median −2.011 vs −1.864. Moreover, LIME-G at m=8 achieves mean −1.744 and median −1.944, close to LILI at m=3, so a different but equally defensible choice of m changes the ranking. The paper reports no confidence intervals, paired tests, or validation-split analysis. To support the claim that LILI improves explanation quality, the authors should either pre-specify m on a validation split, or report the comparison at a fixed m that was not selected on the test set, together with significance","section":"§4.2, Eq. (4)"},{"comment":"The saliency metric is aggregated over thresholds by taking the minimum over α: S*_{ν,m,i} = min_α S_{ν,m,i,α}. This aggregation is not the standard way to summarize threshold-dependent saliency metrics and should be justified. The min over α can reward an explanation that scores very well at a single threshold even if its overall concentration curve is poor, and it may inflate differences between methods. Since Table 2 and the explanation-quality conclusion depend entirely on this aggregation, the authors should report the full threshold curve or a threshold-averaged score and verify that the ranking of LILI versus LIME-G is preserved under a more standard aggregation.","section":"§4.2, Eq. (4)"}],"minor_comments":[{"comment":"The text says LILI achieves 'significantly higher explanation stability' than LIME-G, but no significance test is reported. With standard deviations of 0.109 for LILI and 0.184 for LIME-G and n=100, the difference in mean Kendall’s W (0.852 vs 0.723) may be significant, but the claim should be backed by a statistical test. Also, the stability experiment uses m=5, whereas the explanation-quality experiment selects m=3; the choice of m=5 for stability is not justified.","section":"§4.3, Table 3"},{"comment":"The sentence 'no work has yet explored improving LIME-G with state-of-the-art inpainting models' is too strong given that related work [2] uses SAM plus Stable Diffusion for photorealistic inpainting-based perturbations. Please clarify the specific difference from prior image inpainting approaches in the context of LIME.","section":"§2.4"},{"comment":"The failure of the eight deterministic occlusion strategies is demonstrated only anecdotally on one 'band-aid' image. If this is meant as evidence that deterministic occlusion fails broadly, a small quantitative comparison would be more convincing; otherwise frame it as an illustrative motivating example.","section":"§2.2"},{"comment":"The Discussion correctly notes that LIME is sensitive to hyperparameters and that future work should test different combinations. This is an acknowledged limitation, but since the current evaluation uses only default LIME hyperparameters, the generalizability of the explanation-quality result remains untested. Consider adding a short Limitations paragraph rather than deferring this to future work.","section":"§5"},{"comment":"Minor language and typographical issues: 'it’s scalability' should be 'its scalability', 'explantions' in Fig. 4 should be 'explanations', 'inpaintedsample' missing space, and the abstract has an extra space before 'e.g.'. These do not affect the scientific content.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The FID result is solid but expected; the main risk is the saliency-based explanation-quality comparison due to test-set selection of m and lack of statistical testing. The manuscript is close to being a useful contribution, and the issues are addressable with a validation-split reanalysis, a paired significance test, and a more standard saliency aggregation. I would be comfortable with acceptance after these changes and appropriate softening of claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of arXiv:2607.15482. The paper is a competent incremental extension of LIME-G: swap DeepFill for LaMa and expand the perturbation mask by a few pixels to hide segmentation artifacts. The FID result is the strongest part — LILI gets 6.7 vs 30.5 for LIME and 56.5 for LIME-G on 10k reference images. That is a clear, reproducible win, and it makes sense given LaMa's known superiority. The mask expansion idea is simple but plausible, and the paper demonstrates it with examples.\n\nThe soft spot is the explanation-quality claim. Section 4.2 selects the mask expansion m on the same 100 test images used to report the saliency comparison. LILI's best m is 3; LIME-G's best mean is also at m=3, and the gap between the best LILI and best LIME-G rows is small (means -1.816 vs -1.699, medians -2.011 vs -1.944). With four candidate values and 100 images, picking the best m per method on the same test set can easily account for a gap that small. There are no significance tests or confidence intervals. The saliency metric itself is also aggregated with min-over-α (Eq. 4), which selects the most favorable threshold per image — another source of optimistic bias. The paper does not report how often LILI beats LIME-G per image, only summary statistics.\n\nThese issues are fixable. A validation split for choosing m, significance tests (paired or permutation-based), and at least one additional faithfulness metric would do a lot. The stability results in Section 4.3 already show LILI is less stable than LIME (Kendall's W 0.85 vs 0.89), which the paper honestly acknowledges in Section 5. So the phrasing in Section 4.2 ('faithful and robust explanations') oversells the evidence.\n\nThe paper is honest about its limitations, ships code, and the FID part is reproducible. The explanation-quality conclusion is plausible but not established. This is exactly the kind of paper that should go to peer review — it is a serious empirical study with a clear, addressable methodological weakness — but the authors need to redo the comparison properly before I'd believe the central claim.","headline":"Firm FID result, fragile explanation-quality claim: mask expansion is tuned on the same test set and no significance tests are reported.","tokens_in":10802,"tokens_out":2587,"would_cite":true,"duration_ms":25249,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LILI replaces LIME's flat-color occlusions with LaMa inpainting and mask expansion, yielding photorealistic perturbations and higher-saliency explanations.","keywords":["perturbation-based explanations","LIME","generative inpainting","LaMa","mask expansion","saliency metric","explanation faithfulness","image XAI"],"falsifier":"Re-run Table 2 on held-out images (or with cross-validation over m) and report saliency scores at each threshold alpha instead of only the minimum: if LILI no longer beats LIME-G and LIME, or the ordering reverses under average aggregation, the central claim fails. Also compute a faithfulness metric that removes the top salient superpixels and measures confidence drop, checking whether LILI's advantage persists.","tokens_in":9853,"feed_emoji":"🎨","tokens_out":3197,"duration_ms":30021,"temperature":0.7,"pith_summary":"The paper targets a known weakness of perturbation-based explanation methods for images: replacing superpixels with a flat color (or another deterministic fill) produces out-of-distribution samples that can mislead the classifier and blur the attribution. It proposes LILI, a variant of LIME in which occluded regions are inpainted by LaMa, a modern generative inpainting network, and in which the occlusion mask is dilated by m pixels to hide segmentation artifacts. On ImageNet-1k images explained with InceptionV3, LILI produces perturbations with substantially lower FID (6.727) than LIME (30.532) or LIME-G with DeepFill (56.524), and its saliency scores (best mean -1.816 with m=3) beat both baselines. The paper's point is that photorealism of the perturbation is not just cosmetic: it changes which features the explanation identifies.","feed_headline":"Photorealistic inpainting lifts LIME explanations","feed_subtitle":"Replacing flat-color occlusions with LaMa inpainting and dilated masks cuts FID to 6.7 and improves saliency scores.","key_machinery":"LaMa, a feed-forward inpainting network using fast Fourier convolutions, is the replacement engine: it produces plausible content conditioned on the whole image, so masked features are removed realistically rather than overwritten with a flat patch. The second mechanism is mask expansion: before inpainting, each superpixel mask is dilated by m pixels along its contour using border-following contour tracing, which hides quickshift's boundary artifacts and stops LaMa from resurrecting the occluded object.","core_discovery":"LILI modifies only the perturbation step of LIME. Instead of filling masked superpixels with a mean color, it feeds the original image and binary mask to LaMa, and it expands each superpixel mask by m pixels around its contour before inpainting. The expanded mask stops the inpainter from reconstructing the occluded object from border artifacts, so the object is actually removed rather than redrawn. The paper claims this yields photorealistic perturbations and, in its saliency-metric evaluation over 100 ImageNet images, the best mean score of -1.816 at m=3, outperforming LIME (-1.625) and LIME-G (best mean -1.744 at m=8; -1.699 at m=3). It also reports that LILI is less stable than LIME but m","pith_inferences":["The core claim suggests a general recipe: choose an inpainter that is fast enough for the many perturbations LIME needs, and add controlled mask dilation; the same recipe could transfer to other perturbation frameworks like RISE, not only LIME.","If photorealism alone drives the saliency gain, then a cheaper realistic fill (for example, texture synthesis or a single fast generative pass) might capture much of the benefit; the paper does not isolate that factor.","The min-over-alpha aggregation used for saliency scores is a notable choice; a reader could check whether the ordering survives per-threshold comparisons or average aggregation, since the minimum may reward explanations that are concentrated at one threshold.","The band-aid example suggests LILI's practical value in diagnosing shortcut learning: when a spurious artifact is removed in a realistic way, the classifier's confidence drops, exposing features that flat-color occlusion masks."],"forward_implications":["If LILI is right, perturbation-based explanations can escape the out-of-distribution problem: masks remove features without creating artificial edges, so the surrogate model sees in-distribution inputs.","Since the mask-expansion parameter m has an optimum (m=3 seems best for LILI), the paper implies that inpainting-based occlusion must be tuned jointly with the segmentation method; simply switching in a better inpainter is not enough.","The FID numbers give a concrete, checkable ordering of perturbation generators: LaMa inpainting > superpixel mean color > DeepFill inpainting for this setup.","The stability gap (LIME W=0.889, LILI 0.852, LIME-G 0.723) shows that realism and stability can move apart, so users should expect a trade-off rather than a strict improvement.","Because LILI's runtime (~12.3s per explanation) is between classic LIME (~4.4s) and diffusion-based inpainting, the approach is usable where compute is limited."],"fun_headline_variants":["Inpainting beats color-fill for LIME explanations","Photorealistic perturbations sharpen LIME saliency","LILI: Dilation and inpainting fix LIME artifacts","Realistic masks improve LIME's saliency scores","Inpaint, don't color-fill: better LIME explanations"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"That the saliency metric, with min-over-alpha aggregation and the specific 100-image test set where m is tuned on the same images, faithfully reflects what users want from an explanation; if the metric is not a faithful proxy, the claimed superiority of LILI over LIME and LIME-G may vanish.","fun_headline_variants_meta":{"raw":{"variants":["Inpainting beats color-fill for LIME explanations","Photorealistic perturbations sharpen LIME saliency","LILI: Dilation and inpainting fix LIME artifacts","Realistic masks improve LIME's saliency scores","Inpaint, don't color-fill: better LIME explanations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000414,"raw_usage":{"total_tokens":1966,"prompt_tokens":721,"completion_tokens":1245,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":1162}},"tokens_in":465,"tokens_out":1245,"duration_ms":10546,"temperature":1.0,"reasoning_tokens":1162,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T23:13:09.980687+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run Table 2 on held-out images (or with cross-validation over m) and report saliency scores at each threshold alpha instead of only the minimum: if LILI no longer beats LIME-G and LIME, or the ordering reverses under average aggregation, the central claim fails. Also compute a faithfulness metric that removes the top salient superpixels and measures confidence drop, checking whether LILI's advantage persists.","supporting_citations":[],"review_version":1}