Aligned explanations in neural networks

Corentin Lobet; Francesca Chiaromonte

arxiv: 2601.04378 · v3 · submitted 2026-01-07 · cs.LG · cs.CV· stat.ML

Aligned explanations in neural networks

Corentin Lobet , Francesca Chiaromonte This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-05-16 16:11 UTCgrok-4.3open to challenge →

classification cs.LG cs.CVstat.ML

keywords explanatory alignmentPiNetspointwise interpretable networksneural network interpretabilityfaithful explanationspseudo-linear architectureimage classificationtrustworthy AI

0 comments

The pith

PiNets embed instance-specific linear models inside neural networks so explanations directly construct each prediction.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper contends that typical post-hoc explanation techniques for neural networks rationalize decisions after they are made rather than revealing the actual reasoning steps. It defines explanatory alignment as the stricter requirement that explanations must participate in building the output. To meet this requirement on complex inputs, the authors develop Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that computes a distinct linear model for every individual data point. Experiments on image classification and segmentation show that the resulting explanations satisfy four faithfulness criteria: meaningfulness, alignment, robustness, and sufficiency.

Core claim

PiNets are a pseudo-linear architecture that forms linear models instance-wise, ensuring that explanations are aligned with the model's predictions by directly constructing them rather than post-hoc rationalizing.

What carries the argument

Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that forms linear models for each data instance.

If this is right

Explanations satisfy the MARS criteria of meaningfulness, alignment, robustness, and sufficiency on image tasks.
Deep learning retains its predictive strength while inheriting the direct interpretability of linear models.
Critical decisions driven by neural networks can rest on explanations that participate in the computation rather than describe it afterward.
Data-driven scientific discovery gains a route to inspect model reasoning without post-hoc approximation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The per-instance linear coefficients could be inspected directly to detect input-specific biases that global explanations would miss.
The same pseudo-linear design might be tested on tabular or time-series data to check whether alignment generalizes beyond images.
Comparing training time and memory of PiNets against standard networks plus separate explainers would quantify the practical overhead.

Load-bearing premise

That fitting a fresh linear model to each data point inside the network will expose the model's genuine reasoning process instead of imposing a separate rationalization.

What would settle it

Train PiNets on synthetic data whose ground-truth decision rule is a known nonlinear function; the extracted per-instance linear models would then have to reproduce the network outputs exactly, or the alignment claim would fail.

Figures

Figures reproduced from arXiv: 2601.04378 by Corentin Lobet, Francesca Chiaromonte.

**Figure 1.** Figure 1: Generic architecture of a PiNet (left) and examples of Grad-CAM and PiNet explanations (attribution maps) in the ToyShapes task (right); column headings specify the post-processing approach (naive or optimal) and the test detection score (defined in eq. (11)). 1 Growing out of white-painting White-painting The trust we place in a decision depends on the quality of its justification. When a decision is just… view at source ↗

**Figure 2.** Figure 2: MARS criteria. An explanation is meaningful if it explains the prediction with relevant signal, aligned if it directly underlies the prediction, robust if it does not heavily rely on context, and sufficient if the prediction can be recovered from it. Note that a sufficient explanation is not always meaningful, e.g., the cat’s head alone may suffice to correctly predict its presence so that the rest of the… view at source ↗

**Figure 3.** Figure 3: PiNet with recursive feedback. The explanation is used to construct the recursive input π(x) ◦ z. The discrepancy between the initial explanation π(x) and the recursive explanation π ′ (x) is penalized. Recursive stabilization As a direct response to the criteria of robustness to context and sufficiency defined in Section 2 we propose a training technique that targets the recursive stability of explanati… view at source ↗

**Figure 4.** Figure 4: Distribution of meaningfulness in ToyShapes depicted by violin plots. Marks inside each violin represent medians. Blue vertical lines extend the medians of the baseline (Grad-CAMs). “Naive” and “Optimal” refer to the postprocessing approach. March 3, 2026 8 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

**Figure 5.** Figure 5: Meaningfulness under the optimal post-processing approach (thresholding) in ToyShapes. Red contours show iso-levels of the detection score (eq. (11)). Points represent fitted models color-coded by group, and are shown together with 95% Gaussian confidence ellipses. Ellipses inside the gray insets (left and middle panels) are shown without points for visual clarity; middle and right panels magnify such inse… view at source ↗

**Figure 6.** Figure 6: Ease of fine-tuning meaningfulness in ToyShapes. Bars represent the ranges of thresholds satisfying a detection score of at least ξ (in the column title). Thresholds reported on the x-axis are log-transformed and range from 10−4 to 1. Within each model group, results are sorted by the breadth of the range, for clarity. The • PiNet Naive variant is flagged “do not use” as it yields spurious explanations fo… view at source ↗

**Figure 7.** Figure 7: Recursive accuracy shift in ToyShapes. The bars in each panel represent the model’s test predictive accuracy (note that the vertical axis starts at 0.3). The black sticks represent recursive accuracy shifts; they start at the model’s accuracy level on top, and extend downwards to reach the accuracy levels achieved after recursion. The horizontal black line shows the naive predictive accuracy level achieved… view at source ↗

**Figure 8.** Figure 8: Test attribution examples in ToyShapes for the top explainer (w.r.t. meaningfulness) of each model group. degree of meaningfulness can PiNets organize their explanations? The limited sharpness of the attribution maps we obtain in the ToyShapes experiment is in part due to the prediction task; a simple classification problem wherein only the presence of triangles matters. In this type of settings, maximizi… view at source ↗

**Figure 9.** Figure 9: Test segmentation maps in Sen1Floods11. From top to bottom: Sentinel-2 images (RGB bands), handannotated maps (proxy for ground-truth), SegNet segmentation maps, and PiNet attribution maps. Classes k = {−1, 0, 1} are assigned, respectively, the colors black (no data/not valid), gray (no water) and teal (water). March 3, 2026 13 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗

read the original abstract

As artificial intelligence increasingly drives critical decisions, the ability to genuinely explain how neural networks make predictions is essential for trust. Yet, most current explanation methods offer post-hoc rationalizations rather than guaranteeing a true reflection of the model's reasoning. We introduce the notion of explanatory alignment, a requirement that explanations directly construct predictions rather than rationalize them. To achieve this in complex data domains, we present Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that forms linear models instance-wise. Evaluated on image classification and segmentation tasks, PiNets demonstrate that their explanations are deeply faithful across four criteria: meaningfulness, alignment, robustness, and sufficiency (MARS). Our contributions pave the way for promising avenues: by reconciling the predictive power of deep learning with the interpretability of linear models, PiNets provide a principled foundation for trustworthy AI and data-driven scientific discovery.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

PiNets introduce explanatory alignment via instance-wise linear models, which is a concrete step toward built-in faithfulness, but the architecture may still leave non-linear feature steps unaligned.

read the letter

PiNets stand out as an attempt to create neural networks where explanations are not added after the fact but are part of how the prediction is made. The paper defines explanatory alignment as the requirement that explanations directly construct the predictions, and then proposes Pointwise-interpretable Networks that build linear models on a per-instance basis using a pseudo-linear design. This framing and the specific architecture look like genuine additions relative to the prior work referenced in the abstract. The work does well by laying out four specific criteria under MARS for checking if explanations are meaningful, aligned, robust, and sufficient. Applying this to classification and segmentation tasks gives a practical test bed for the idea. The approach tries to keep the predictive strength of deep learning while gaining the clarity of linear models, which is a useful goal for making AI more trustworthy in real applications such as vision. One area that needs more scrutiny is whether the instance-wise linear models cover the entire reasoning process. If non-linear layers handle feature extraction first, then the linear explanation might only account for how those features are combined, not how the features themselves were derived from the input. This could mean the explanations are still a form of rationalization for the upstream parts. The abstract does not include equations or specific results, so the strength of the faithfulness claim is not clear yet. The citation pattern appears standard and does not show obvious circularity in the definitions. The paper is for researchers and practitioners in explainable AI who want models that are interpretable by design rather than through separate explanation tools. Readers focused on vision tasks or data-driven science would see value in the architecture and the evaluation framework, provided the full details confirm the claims. I recommend putting this through peer review. The core idea has enough substance to warrant referee feedback on the architecture details and experimental validation.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces the notion of explanatory alignment, requiring that explanations directly construct predictions rather than rationalize them. It proposes Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that forms linear models instance-wise, and evaluates them on image classification and segmentation tasks, claiming that their explanations are deeply faithful according to the MARS criteria of meaningfulness, alignment, robustness, and sufficiency.

Significance. If the central claims hold, this would represent a significant contribution to interpretable machine learning by enabling deep networks to produce inherently aligned explanations that reconcile high predictive performance with linear-model interpretability, opening avenues for trustworthy AI in high-stakes applications and data-driven discovery.

major comments (1)

[§3] §3 (PiNets Architecture): The description of PiNets as a pseudo-linear architecture forming linear models instance-wise does not establish that the entire forward pass is equivalent to a single linear function of the raw input. If non-linear layers (convolutions, activations, pooling) extract features before the instance-wise linear component, the explanations would rationalize the feature map rather than construct predictions from the raw data, directly undermining the explanatory alignment claim and the MARS faithfulness results. A formal equivalence or architectural restriction ensuring end-to-end linearity per instance is required.

minor comments (2)

[Abstract] Abstract: Specific dataset names, quantitative performance numbers, and baseline comparisons (e.g., to LIME or SHAP) are omitted, making it difficult to gauge the practical gains.
[§4] §4 (Experiments): The MARS evaluation protocol should include ablation studies isolating the effect of the instance-wise linear head versus any preceding non-linear stages.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their valuable comments on our work. We have carefully considered the concern regarding the PiNets architecture and provide a detailed response below, including plans for revision.

read point-by-point responses

Referee: [§3] §3 (PiNets Architecture): The description of PiNets as a pseudo-linear architecture forming linear models instance-wise does not establish that the entire forward pass is equivalent to a single linear function of the raw input. If non-linear layers (convolutions, activations, pooling) extract features before the instance-wise linear component, the explanations would rationalize the feature map rather than construct predictions from the raw data, directly undermining the explanatory alignment claim and the MARS faithfulness results. A formal equivalence or architectural restriction ensuring end-to-end linearity per instance is required.

Authors: We appreciate the referee's careful reading of §3. In the PiNets architecture, non-linear layers are employed exclusively to generate the instance-specific linear coefficients and bias terms. The prediction is then computed as a linear function of the raw input using these coefficients. Formally, the model computes w = g_θ(x) and b = h_θ(x) where g and h are non-linear functions parameterized by θ, followed by the prediction ŷ = w · x + b. Thus, the linear model with parameters (w, b) directly constructs the prediction from the raw input x. This establishes the required equivalence: the forward pass, while overall non-linear, is exactly equivalent to applying the instance-wise linear model to the raw data. We will revise the manuscript to include this formal definition and a proof of the construction property in §3, ensuring the explanatory alignment claim is rigorously supported. Consequently, the MARS faithfulness results remain valid as they evaluate explanations that construct the predictions. revision: yes

Circularity Check

1 steps flagged

Explanatory alignment defined as constructing predictions, then achieved by architecture that forms linear models by construction

specific steps

self definitional [Abstract]
"We introduce the notion of explanatory alignment, a requirement that explanations directly construct predictions rather than rationalize them. To achieve this in complex data domains, we present Pointwise-interpretable Networks (PiNets), a pseudo-linear architecture that forms linear models instance-wise."

Alignment is defined as explanations that construct predictions. PiNets are introduced specifically to achieve this by forming linear models instance-wise, so the linear model serves as both the explanation and the direct constructor of the prediction. The property therefore holds by the paper's own definitional choice of architecture rather than by any non-circular derivation.

full rationale

The paper's central contribution rests on a new definition of explanatory alignment (explanations must construct predictions rather than rationalize them) and immediately presents PiNets as satisfying it because they form linear models instance-wise. This makes the claimed alignment hold tautologically from the architecture's stated design rather than from any independent derivation or external verification. The MARS evaluation criteria are applied after this definitional step, so they measure properties of the self-defined construction. No load-bearing equations or external uniqueness results are shown that would break the definitional loop.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that explanations must construct predictions to count as aligned, plus the invented PiNets architecture; no free parameters are stated in the abstract.

axioms (1)

domain assumption Explanations must directly construct predictions rather than rationalize them after the fact.
This is the definitional premise for explanatory alignment introduced in the abstract.

invented entities (1)

Pointwise-interpretable Networks (PiNets) no independent evidence
purpose: Pseudo-linear architecture that forms linear models instance-wise to enforce explanatory alignment.
New architecture presented as the technical contribution.

pith-pipeline@v0.9.0 · 5443 in / 1263 out tokens · 39078 ms · 2026-05-16T16:11:19.464779+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel matches

?

matches
MATCHES: this paper passage directly uses, restates, or depends on the cited Recognition theorem or module.

A pseudo-linear model takes the form: y = a + Σ* π(x) ◦ z ... the second look must be seen as a mechanistic operation enabling immediate precedence, that is, it must be explicitly performed in the last layer of the network.
IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean absolute_floor_iff_bare_distinguishability matches

?

matches
MATCHES: this paper passage directly uses, restates, or depends on the cited Recognition theorem or module.

Explanatory alignment at point (x, z, y=f(x)) is possible if z is fully interpretable and the model f embeds both a feature attribution π and a simple function g such that y = g(π, z).
IndisputableMonolith/Foundation/ArithmeticFromLogic.lean embed_injective echoes

?

echoes
ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

Recursive stabilization ... penalize the discrepancy between the initial explanation π(x) and the recursive explanation π'(x)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

72 extracted references · 72 canonical work pages

[1]

Festinger.A theory of cognitive disso- nance

L. Festinger.A theory of cognitive disso- nance. Stanford University Press, 1957

work page 1957
[2]

R. E. Nisbett and T. D. Wilson. Telling more than we can know: Verbal reports on mental processes.Psychological Re- view, 1977

work page 1977
[3]

Johansson, L

P. Johansson, L. Hall, S. Sikstrom, and A. Olsson. Failure to detect mismatches between intention and outcome in a simple decision task.Science, 2005

work page 2005
[4]

Z. C. Lipton. The mythos of model in- terpretability: In machine learning, the concept of interpretability is both impor- tant and slippery .Communications of the ACM, 2018

work page 2018
[5]

C. Rudin. Stop explaining black box ma- chine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 2019

work page 2019
[6]

Guidotti, A

R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi. A survey of methods for explaining black box models.ACM Computing Surveys, 2018

work page 2018
[7]

Barredo Arrieta, N

A. Barredo Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, et al. Explainable artificial intelligence (xAI): Concepts, taxonomies, opportuni- ties and challenges toward responsible AI.Information Fusion, 2020

work page 2020
[8]

Marcinkevi ˇcs and J

R. Marcinkevi ˇcs and J. E. Vogt. Inter- pretable and explainable machine learn- ing: A methods-centric overview with concrete examples.WIREs Data Mining and Knowledge Discovery, 2023

work page 2023
[9]

C. Molnar. Interpretable machine learn- ing, 2025

work page 2025
[10]

S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predic- tions.NeurIPS, 2017

work page 2017
[11]

Why should I trust you?

M. T. Ribeiro, S. Singh, and C. Guestrin. "Why should I trust you?" explaining the predictions of any classifier. InKDD. ACM, 2016

work page 2016
[12]

Petsiuk, A

V. Petsiuk, A. Das, and K. Saenko. RISE: Randomized input sampling for expla- nation of black-box models.BMCV, 2018

work page 2018
[13]

M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. InAAAI, 2018

work page 2018
[14]

J. Chen, L. Song, M. Wainwright, and M. Jordan. Learning to explain: An information-theoretic perspective on model interpretation. InICML. PMLR, 2018

work page 2018
[15]

Ivanovs, R

M. Ivanovs, R. Kadikis, and K. Ozols. Perturbation-based methods for explain- ing deep neural networks: A survey .Pat- tern Recognition Letters, 2021

work page 2021
[16]

Paissan, M

F. Paissan, M. Ravanelli, and C. Sub- akan. Listenable maps for audio classi- fiers. InICML. PMLR, 2024

work page 2024
[17]

J. Yoon, J. Jordon, and M. Van der Schaar. INVASE: Instance-wise variable selection using neural networks. In ICLR, 2018

work page 2018
[18]

Jethani, M

N. Jethani, M. Sudarshan, Y. Aphinyanaphongs, and R. Ran- ganath. Have we learned to explain?: How interpretability methods can learn to encode predictions in their interpretations. InICAIS. PMLR, 2021

work page 2021
[19]

Zhang, D

X. Zhang, D. Lee, and S. Wang. Compre- hensive attribution: Inherently explain- able vision model with feature detector. InECCV. Springer, 2024

work page 2024
[20]

Böhle, M

M. Böhle, M. Fritz, and B. Schiele. B-cos networks: Alignment is all we need for interpretability . InCVPR. IEEE, 2022

work page 2022
[21]

Vandenhirtz and J

M. Vandenhirtz and J. E. Vogt. From pix- els to perception: Interpretable predic- tions via instance-wise grouped feature selection. InICML. PMLR, 2025

work page 2025
[22]

B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). InICML. PMLR, 2018

work page 2018
[23]

P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, et al. Con- cept bottleneck models. InICML. PMLR, 2020. March 3, 202615

work page 2020
[24]

Espinosa Zarlenga, P

M. Espinosa Zarlenga, P. Barbiero, G. Ciravegna, G. Marra, F. Giannini, et al. Concept embedding models: Be- yond the accuracy-explainability trade- off.NeurIPS, 2022

work page 2022
[25]

V. V. Ramaswamy , S. S. Kim, R. Fong, and O. Russakovsky . Overlooked factors in concept-based explanations: Dataset choice, concept learnability , and human capability . InCVPR. IEEE, 2023

work page 2023
[26]

Sinha and A

S. Sinha and A. Zhang. A comprehen- sive survey on the risks and limitations of concept-based models.arXiv, 2025

work page 2025
[27]

Simonyan, A

K. Simonyan, A. Vedaldi, and A. Zisser- man. Deep inside convolutional net- works: Visualising image classification models and saliency maps. InICLR, 2014

work page 2014
[28]

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Ba- tra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. InICCV. IEEE, 2017

work page 2017
[29]

Sundararajan, A

M. Sundararajan, A. Taly , and Q. Yan. Axiomatic attribution for deep net- works. InInternational conference on machine learning, pages 3319–3328. PMLR, 2017

work page 2017
[30]

Adebayo, J

J. Adebayo, J. Gilmer, M. Muelly , I. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. NeurIPS, 2018

work page 2018
[31]

W. Nie, Y. Zhang, and A. Patel. A theo- retical explanation for perplexing behav- iors of backpropagation-based visualiza- tions. InICML. PMLR, 2018

work page 2018
[32]

Hooker, D

S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim. A benchmark for inter- pretability methods in deep neural net- works.NeurIPS, 2019

work page 2019
[33]

Y. Jia, E. Frank, B. Pfahringer, A. Bifet, and N. Lim. Studying and exploiting the relationship between model accu- racy and explanation quality . InECML PKDD. Springer, 2021

work page 2021
[34]

J. Chen, L. Q. R. Ooi, T. W. K. Tan, S. Zhang, J. Li, et al. Relationship be- tween prediction accuracy and feature importance reliability: An empirical and theoretical study .NeuroImage, 2023

work page 2023
[35]

Breiman, J

L. Breiman, J. Friedman, R. A. Olshen, and C. J. Stone.Classification and re- gression trees. Chapman and Hall/CRC, 1984

work page 1984
[36]

L. Breiman. Random forests.Machine learning, 2001

work page 2001
[37]

J. H. Friedman and B. E. Popescu. Pre- dictive learning via rule ensembles.The Annals of Applied Statistics, 2008

work page 2008
[38]

Hastie and R

T. Hastie and R. Tibshirani. Varying- coefficient models.Journal of the Royal Statistical Society Series B: Statistical Methodology, 1993

work page 1993
[39]

Fan and W

J. Fan and W. Zhang. Statistical methods with varying coefficient models.Statis- tics and its Interface, 2008

work page 2008
[40]

Alvarez Melis and T

D. Alvarez Melis and T. Jaakkola. To- wards robust interpretability with self- explaining neural networks.NeurIPS, 2018

work page 2018
[41]

Ignatiev, N

A. Ignatiev, N. Narodytska, and J. Marques-Silva. Abduction-based explanations for machine learning models. InAAAI, 2019

work page 2019
[42]

Dasgupta, N

S. Dasgupta, N. Frost, and M. Moshkovitz. Framework for evaluat- ing faithfulness of local explanations. In ICML. PMLR, 2022

work page 2022
[43]

Recht, R

B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do imagenet classifiers gen- eralize to imagenet? InICML. PMLR, 2019

work page 2019
[44]

Ilyas, S

A. Ilyas, S. Santurkar, D. Tsipras, L. En- gstrom, B. Tran, and A. Madry . Adversar- ial examples are not bugs, they are fea- tures.NeurIPS, 2019

work page 2019
[45]

Geirhos, J.-H

R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, et al. Shortcut learning in deep neural networks.Na- ture Machine Intelligence, 2020

work page 2020
[46]

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts.Neural Computation, 1991

work page 1991
[47]

D. H. Wolpert. Stacked generalization. Neural networks, 1992

work page 1992
[48]

Krogh and J

A. Krogh and J. Vedelsby . Neural net- work ensembles, cross validation, and active learning.NeurIPS, 1994

work page 1994
[49]

L. Breiman. Bagging predictors.Machine learning, 1996

work page 1996
[50]

Johansson, C

U. Johansson, C. Sönströd, U. Norinder, and H. Boström. Trade-off between ac- curacy and interpretability for predic- tive in silico modeling.Future Medicinal Chemistry, 2011

work page 2011
[51]

Audemard, S

G. Audemard, S. Bellart, L. Bounia, F. Koriche, J.-M. Lagniez, and P. Mar- quis. Trading complexity for sparsity in random forest explanations. InAAAI, 2022

work page 2022
[52]

Bassan, G

S. Bassan, G. Amir, M. Zehavi, and G. Katz. What makes an ensemble (un) interpretable? InICML. PMLR, 2025

work page 2025
[53]

Milletari, N

F. Milletari, N. Navab, and S.-A. Ahmadi. V-net: Fully convolutional neural net- works for volumetric medical image seg- mentation. In3DV. IEEE, 2016

work page 2016
[54]

C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. Jorge Cardoso. Gen- eralised dice overlap as a deep learning loss function for highly unbalanced seg- mentations. InDLMIA. Springer, 2017

work page 2017
[55]

Lecun, L

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning ap- plied to document recognition.Proceed- ings of the IEEE, 1998

work page 1998
[56]

J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for seman- tic segmentation. InCVPR. IEEE, 2015

work page 2015
[57]

Bonafilia, B

D. Bonafilia, B. Tellman, T. Anderson, and E. Issenberg. Sen1floods11: A geo- referenced dataset to train and test deep learning flood algorithms for sentinel-1. InCVPR. IEEE, 2020

work page 2020
[58]

Ronneberger, P

O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. Springer, 2015

work page 2015
[59]

Jakubik, S

J. Jakubik, S. Roy , C. E. Phillips, P. Frac- caro, D. Godwin, et al. Foundation mod- els for generalist geospatial artificial in- telligence.arXiv, 2023

work page 2023
[60]

Gomes, B

C. Gomes, B. Blumenstiel, J. L. d. S. Almeida, P. H. de Oliveira, P. Fraccaro, et al. Terratorch: The geospatial founda- tion models toolkit.arXiv, 2025

work page 2025
[61]

T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun. Unified perceptual parsing for scene understanding. InECCV. Springer, 2018

work page 2018
[62]

Tajbakhsh, L

N. Tajbakhsh, L. Jeyaseelan, Q. Li, J. N. Chiang, Z. Wu, and X. Ding. Embrac- ing imperfect datasets: A review of deep learning solutions for medical image segmentation.Medical Image Analysis, 2020

work page 2020
[63]

W. Shen, Z. Peng, X. Wang, H. Wang, J. Cen, et al. A survey on label-efficient deep image segmentation: Bridging the gap between weak supervision and dense prediction.IEEE Pattern Analysis and Machine Intelligence, 2023

work page 2023
[64]

Quinonero-Candela, M

J. Quinonero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence. Dataset Shift in Machine Learning. MIT Press, 2008

work page 2008
[65]

Ben-David, J

S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from dif- ferent domains.Machine learning, 2010

work page 2010
[66]

Szegedy , W

C. Szegedy , W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, et al. Intriguing properties of neural networks. InICLR, 2014

work page 2014
[67]

Madry , A

A. Madry , A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. InICLR, 2018

work page 2018
[68]

Zhang, M

H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. InICLR, 2018

work page 2018
[69]

S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo. Cutmix: Regular- ization strategy to train strong classifiers with localizable features. InICCV. IEEE, 2019

work page 2019
[70]

Paszke, S

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury , et al. Pytorch: An impera- tive style, high-performance deep learn- ing library .NeurIPS, 2019

work page 2019
[71]

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization.arXiv, 2014

work page 2014
[72]

Loshchilov and F

I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InICLR, 2019. March 3, 202616

work page 2019

[1] [1]

Festinger.A theory of cognitive disso- nance

L. Festinger.A theory of cognitive disso- nance. Stanford University Press, 1957

work page 1957

[2] [2]

R. E. Nisbett and T. D. Wilson. Telling more than we can know: Verbal reports on mental processes.Psychological Re- view, 1977

work page 1977

[3] [3]

Johansson, L

P. Johansson, L. Hall, S. Sikstrom, and A. Olsson. Failure to detect mismatches between intention and outcome in a simple decision task.Science, 2005

work page 2005

[4] [4]

Z. C. Lipton. The mythos of model in- terpretability: In machine learning, the concept of interpretability is both impor- tant and slippery .Communications of the ACM, 2018

work page 2018

[5] [5]

C. Rudin. Stop explaining black box ma- chine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 2019

work page 2019

[6] [6]

Guidotti, A

R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi. A survey of methods for explaining black box models.ACM Computing Surveys, 2018

work page 2018

[7] [7]

Barredo Arrieta, N

A. Barredo Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, et al. Explainable artificial intelligence (xAI): Concepts, taxonomies, opportuni- ties and challenges toward responsible AI.Information Fusion, 2020

work page 2020

[8] [8]

Marcinkevi ˇcs and J

R. Marcinkevi ˇcs and J. E. Vogt. Inter- pretable and explainable machine learn- ing: A methods-centric overview with concrete examples.WIREs Data Mining and Knowledge Discovery, 2023

work page 2023

[9] [9]

C. Molnar. Interpretable machine learn- ing, 2025

work page 2025

[10] [10]

S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predic- tions.NeurIPS, 2017

work page 2017

[11] [11]

Why should I trust you?

M. T. Ribeiro, S. Singh, and C. Guestrin. "Why should I trust you?" explaining the predictions of any classifier. InKDD. ACM, 2016

work page 2016

[12] [12]

Petsiuk, A

V. Petsiuk, A. Das, and K. Saenko. RISE: Randomized input sampling for expla- nation of black-box models.BMCV, 2018

work page 2018

[13] [13]

M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. InAAAI, 2018

work page 2018

[14] [14]

J. Chen, L. Song, M. Wainwright, and M. Jordan. Learning to explain: An information-theoretic perspective on model interpretation. InICML. PMLR, 2018

work page 2018

[15] [15]

Ivanovs, R

M. Ivanovs, R. Kadikis, and K. Ozols. Perturbation-based methods for explain- ing deep neural networks: A survey .Pat- tern Recognition Letters, 2021

work page 2021

[16] [16]

Paissan, M

F. Paissan, M. Ravanelli, and C. Sub- akan. Listenable maps for audio classi- fiers. InICML. PMLR, 2024

work page 2024

[17] [17]

J. Yoon, J. Jordon, and M. Van der Schaar. INVASE: Instance-wise variable selection using neural networks. In ICLR, 2018

work page 2018

[18] [18]

Jethani, M

N. Jethani, M. Sudarshan, Y. Aphinyanaphongs, and R. Ran- ganath. Have we learned to explain?: How interpretability methods can learn to encode predictions in their interpretations. InICAIS. PMLR, 2021

work page 2021

[19] [19]

Zhang, D

X. Zhang, D. Lee, and S. Wang. Compre- hensive attribution: Inherently explain- able vision model with feature detector. InECCV. Springer, 2024

work page 2024

[20] [20]

Böhle, M

M. Böhle, M. Fritz, and B. Schiele. B-cos networks: Alignment is all we need for interpretability . InCVPR. IEEE, 2022

work page 2022

[21] [21]

Vandenhirtz and J

M. Vandenhirtz and J. E. Vogt. From pix- els to perception: Interpretable predic- tions via instance-wise grouped feature selection. InICML. PMLR, 2025

work page 2025

[22] [22]

B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). InICML. PMLR, 2018

work page 2018

[23] [23]

P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, et al. Con- cept bottleneck models. InICML. PMLR, 2020. March 3, 202615

work page 2020

[24] [24]

Espinosa Zarlenga, P

M. Espinosa Zarlenga, P. Barbiero, G. Ciravegna, G. Marra, F. Giannini, et al. Concept embedding models: Be- yond the accuracy-explainability trade- off.NeurIPS, 2022

work page 2022

[25] [25]

V. V. Ramaswamy , S. S. Kim, R. Fong, and O. Russakovsky . Overlooked factors in concept-based explanations: Dataset choice, concept learnability , and human capability . InCVPR. IEEE, 2023

work page 2023

[26] [26]

Sinha and A

S. Sinha and A. Zhang. A comprehen- sive survey on the risks and limitations of concept-based models.arXiv, 2025

work page 2025

[27] [27]

Simonyan, A

K. Simonyan, A. Vedaldi, and A. Zisser- man. Deep inside convolutional net- works: Visualising image classification models and saliency maps. InICLR, 2014

work page 2014

[28] [28]

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Ba- tra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. InICCV. IEEE, 2017

work page 2017

[29] [29]

Sundararajan, A

M. Sundararajan, A. Taly , and Q. Yan. Axiomatic attribution for deep net- works. InInternational conference on machine learning, pages 3319–3328. PMLR, 2017

work page 2017

[30] [30]

Adebayo, J

J. Adebayo, J. Gilmer, M. Muelly , I. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. NeurIPS, 2018

work page 2018

[31] [31]

W. Nie, Y. Zhang, and A. Patel. A theo- retical explanation for perplexing behav- iors of backpropagation-based visualiza- tions. InICML. PMLR, 2018

work page 2018

[32] [32]

Hooker, D

S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim. A benchmark for inter- pretability methods in deep neural net- works.NeurIPS, 2019

work page 2019

[33] [33]

Y. Jia, E. Frank, B. Pfahringer, A. Bifet, and N. Lim. Studying and exploiting the relationship between model accu- racy and explanation quality . InECML PKDD. Springer, 2021

work page 2021

[34] [34]

J. Chen, L. Q. R. Ooi, T. W. K. Tan, S. Zhang, J. Li, et al. Relationship be- tween prediction accuracy and feature importance reliability: An empirical and theoretical study .NeuroImage, 2023

work page 2023

[35] [35]

Breiman, J

L. Breiman, J. Friedman, R. A. Olshen, and C. J. Stone.Classification and re- gression trees. Chapman and Hall/CRC, 1984

work page 1984

[36] [36]

L. Breiman. Random forests.Machine learning, 2001

work page 2001

[37] [37]

J. H. Friedman and B. E. Popescu. Pre- dictive learning via rule ensembles.The Annals of Applied Statistics, 2008

work page 2008

[38] [38]

Hastie and R

T. Hastie and R. Tibshirani. Varying- coefficient models.Journal of the Royal Statistical Society Series B: Statistical Methodology, 1993

work page 1993

[39] [39]

Fan and W

J. Fan and W. Zhang. Statistical methods with varying coefficient models.Statis- tics and its Interface, 2008

work page 2008

[40] [40]

Alvarez Melis and T

D. Alvarez Melis and T. Jaakkola. To- wards robust interpretability with self- explaining neural networks.NeurIPS, 2018

work page 2018

[41] [41]

Ignatiev, N

A. Ignatiev, N. Narodytska, and J. Marques-Silva. Abduction-based explanations for machine learning models. InAAAI, 2019

work page 2019

[42] [42]

Dasgupta, N

S. Dasgupta, N. Frost, and M. Moshkovitz. Framework for evaluat- ing faithfulness of local explanations. In ICML. PMLR, 2022

work page 2022

[43] [43]

Recht, R

B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do imagenet classifiers gen- eralize to imagenet? InICML. PMLR, 2019

work page 2019

[44] [44]

Ilyas, S

A. Ilyas, S. Santurkar, D. Tsipras, L. En- gstrom, B. Tran, and A. Madry . Adversar- ial examples are not bugs, they are fea- tures.NeurIPS, 2019

work page 2019

[45] [45]

Geirhos, J.-H

R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, et al. Shortcut learning in deep neural networks.Na- ture Machine Intelligence, 2020

work page 2020

[46] [46]

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts.Neural Computation, 1991

work page 1991

[47] [47]

D. H. Wolpert. Stacked generalization. Neural networks, 1992

work page 1992

[48] [48]

Krogh and J

A. Krogh and J. Vedelsby . Neural net- work ensembles, cross validation, and active learning.NeurIPS, 1994

work page 1994

[49] [49]

L. Breiman. Bagging predictors.Machine learning, 1996

work page 1996

[50] [50]

Johansson, C

U. Johansson, C. Sönströd, U. Norinder, and H. Boström. Trade-off between ac- curacy and interpretability for predic- tive in silico modeling.Future Medicinal Chemistry, 2011

work page 2011

[51] [51]

Audemard, S

G. Audemard, S. Bellart, L. Bounia, F. Koriche, J.-M. Lagniez, and P. Mar- quis. Trading complexity for sparsity in random forest explanations. InAAAI, 2022

work page 2022

[52] [52]

Bassan, G

S. Bassan, G. Amir, M. Zehavi, and G. Katz. What makes an ensemble (un) interpretable? InICML. PMLR, 2025

work page 2025

[53] [53]

Milletari, N

F. Milletari, N. Navab, and S.-A. Ahmadi. V-net: Fully convolutional neural net- works for volumetric medical image seg- mentation. In3DV. IEEE, 2016

work page 2016

[54] [54]

C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. Jorge Cardoso. Gen- eralised dice overlap as a deep learning loss function for highly unbalanced seg- mentations. InDLMIA. Springer, 2017

work page 2017

[55] [55]

Lecun, L

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning ap- plied to document recognition.Proceed- ings of the IEEE, 1998

work page 1998

[56] [56]

J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for seman- tic segmentation. InCVPR. IEEE, 2015

work page 2015

[57] [57]

Bonafilia, B

D. Bonafilia, B. Tellman, T. Anderson, and E. Issenberg. Sen1floods11: A geo- referenced dataset to train and test deep learning flood algorithms for sentinel-1. InCVPR. IEEE, 2020

work page 2020

[58] [58]

Ronneberger, P

O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. Springer, 2015

work page 2015

[59] [59]

Jakubik, S

J. Jakubik, S. Roy , C. E. Phillips, P. Frac- caro, D. Godwin, et al. Foundation mod- els for generalist geospatial artificial in- telligence.arXiv, 2023

work page 2023

[60] [60]

Gomes, B

C. Gomes, B. Blumenstiel, J. L. d. S. Almeida, P. H. de Oliveira, P. Fraccaro, et al. Terratorch: The geospatial founda- tion models toolkit.arXiv, 2025

work page 2025

[61] [61]

T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun. Unified perceptual parsing for scene understanding. InECCV. Springer, 2018

work page 2018

[62] [62]

Tajbakhsh, L

N. Tajbakhsh, L. Jeyaseelan, Q. Li, J. N. Chiang, Z. Wu, and X. Ding. Embrac- ing imperfect datasets: A review of deep learning solutions for medical image segmentation.Medical Image Analysis, 2020

work page 2020

[63] [63]

W. Shen, Z. Peng, X. Wang, H. Wang, J. Cen, et al. A survey on label-efficient deep image segmentation: Bridging the gap between weak supervision and dense prediction.IEEE Pattern Analysis and Machine Intelligence, 2023

work page 2023

[64] [64]

Quinonero-Candela, M

J. Quinonero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence. Dataset Shift in Machine Learning. MIT Press, 2008

work page 2008

[65] [65]

Ben-David, J

S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from dif- ferent domains.Machine learning, 2010

work page 2010

[66] [66]

Szegedy , W

C. Szegedy , W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, et al. Intriguing properties of neural networks. InICLR, 2014

work page 2014

[67] [67]

Madry , A

A. Madry , A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. InICLR, 2018

work page 2018

[68] [68]

Zhang, M

H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. InICLR, 2018

work page 2018

[69] [69]

S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo. Cutmix: Regular- ization strategy to train strong classifiers with localizable features. InICCV. IEEE, 2019

work page 2019

[70] [70]

Paszke, S

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury , et al. Pytorch: An impera- tive style, high-performance deep learn- ing library .NeurIPS, 2019

work page 2019

[71] [71]

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization.arXiv, 2014

work page 2014

[72] [72]

Loshchilov and F

I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InICLR, 2019. March 3, 202616

work page 2019